Raven-1/tests/test_agents_code_tools_plugin.py

1349 lines
60 KiB
Python
Raw Permalink Normal View History

feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
"""code-flow's tool face (code_flow/tools/): the fork's tools as same-name replacements.
Pins the manifest rows, the D6 admission from the slice, the fork's schema
spelling with the host's names accepted as aliases, the two pieces of conduct
the file tools carry (read-before-edit, the Python syntax verdict), the
``occurrence`` edit, ``glob`` and ``todo``, and that a plugin tool
registered after the built-in replaces it in the registry the loop reads.
"""
import asyncio
fix(agent): bound every tree walk and take it off the event loop (#566) ## Summary On 2026-09-20 one `find` call froze the whole gateway for 2h21m. The model asked for `find(pattern="**/gps_trilateration/index.html", path="/Users/admin")`, and `FindTool` ran `Path.glob` synchronously on the event loop, filtered the noise directories only after walking them, and had no deadline. The registry's 300s ceiling is an `asyncio.wait_for`, which cannot preempt synchronous code, so every session and every subagent on the gateway stopped until the walk came back (`gateway.log`: `Tool result: find duration=8476519ms`). `list_dir` with `recursive=true` had the same shape through `rglob`. The fix gives the three tree-walking tools one walk, `raven.agent.tools.tree_walk`, with these bounds: - noise directories (`node_modules`, `.git`, ...) are pruned before they are entered, not filtered afterwards -- the pruning tests spy on `os.scandir` and assert the noise directory is never opened; - the walk yields one entry at a time and checks a 20s wall-clock budget (the value grep's fallback already used) before every entry, so whatever the caller does with one entry -- the `stat` per `find` match, the read per grep file -- is covered by the check on the next. The budget cannot be overrun by one large or slow directory: the probe with eight matches, a 30ms stat each and a 50ms budget returns 2 matches and the notice. When the budget is hit, `find` and `list_dir` append a `PARTIAL result: ... traversal budget ...` clause to their trailer, and a deadline that fires before any match is not reported as a clean `No files found` / `Directory is empty`; - every caller runs the walk in `asyncio.to_thread`, so the loop keeps serving other sessions while the walk runs. Cancelling the thread does not stop the worker, which is why the deadline lives inside the walk. The pure-Python `grep` fallback already had the prune and the deadline; it moves onto the shared walk and off the loop with the others. `find` matches each path component with `fnmatch.translate`, with a component that is exactly `**` spanning any number of components; the matcher is a position-set simulation (`_advance`), linear in components times segments, so a pattern carrying many `**` cannot make one match call slow. A pattern's leading literal components decide where the walk starts: `src/**/*.py` begins inside `src` and never lists a sibling of it, a pattern that is all literal (`src/lib/c.py`, `src/lib`) names exactly one path and answers without a walk, and the prefix itself is a match when the rest of the pattern can match nothing (`src/**` still lists `src/`). It answers exactly what `Path.glob` answered for the same pattern: pinned by a parametrized oracle test over twenty-seven patterns on a fixture and nineteen on a fixture with links (a directory link out of the tree, a link cycle, a file link, a noise directory beneath a link), and checked by hand against `Path.glob` over `raven/` for 27 patterns including bracket edge cases and trailing slashes, 0 mismatches. Two deliberate differences: a trailing `**` also lists files beneath its prefix (3.12's `Path.glob` listed only directories there), and a pattern with `..` or a leading `/` is refused instead of followed, since a pattern must not reach outside the directory the fence resolved. A trailing slash keeps `Path.glob`'s directory-only meaning (`*/`, `src/**/`). A symbolic link to a directory is entered where a single component names or matches it and never under `**`, which is how `Path.glob` read a pattern: `*/util/helper.py` reaches through a linked `vendor`, `**/helper.py` does not, and a link cycle is entered once per component the pattern spends on it, so `*/*/*/b.py` enters `src/loop -> src` twice and stops. The walker asks its caller per link (`follow`), `find` answers from the pattern and hands the link positions of a path to the matcher so `**` cannot claim them, and `list_dir` and the grep fallback never ask, staying as `rglob` and `os.walk` left them. A third deliberate difference from the merge base: a noise directory inside a pattern's literal prefix (`node_modules/*.js`, `src/node_modules/*.js`) is walked, since the pattern asked for it, where `Path.glob`'s filter dropped it; one met below the prefix is pruned as before (`*/node_modules/*.js` answers nothing), `list_dir` on such a path lists it, and the `find`, `list_dir` and code-flow `glob` descriptions state that rule instead of an unconditional "skipped". Review found an earlier wording of the two `find` descriptions one component short ("except one the pattern starts with"), while the oracle helper already pruned below the prefix; the sentences now say what the helper checks, and the oracle has the `src/node_modules` shape on both sides. The raven-code `glob` tool (`agents/raven-code/.../tools/search.py`) parses trunk `find` output line by line and strips the trailer before merging brace alternatives. It learns the new clause, so a cut-short child is reported as cut short rather than as a tree that lacks the files; its `is_notice` grammar covers the new line so it is never pasted into a listing as a path. The code-flow tool guide (`prompts/TOOLS_CODE.md`), the wrapper's description and its `limit` parameter told the model that a PARTIAL `glob` result means "raise `limit` or narrow the pattern"; a traversal-budget PARTIAL is now a result `glob` can return and a higher limit does nothing for it, so all three now tell the two notices apart. The seeded copy of the guide refreshes through the receipt `seed_guide` keeps beside it, so no legacy digest is added. Fixed in passing, and now pinned: recursive `list_dir` filtered `rglob` results on the components of the absolute path, so a workspace beneath a directory named `build`, `dist`, `venv` or another noise name listed as "Directory ... is empty" (reproduced at the merge base under `<tmp>/build/proj`; this head lists it). The shared walk prunes below the listed path only; `find` and the grep fallback already did. Not changed: the 20s value. It is the budget grep's fallback already ran under; this PR does not re-measure it, it makes it hold for every entry of every walk. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification At head 8dade176, on origin/main at d50a13c1 (nine commits: the walk, the grep pin, the changelog, the per-entry budget with the prefix start, the literal-directory case, the symbolic-link reading with the noise-prefix wording, the noise-name-above-the-path pin, the guide wording, and the prefix-rule wording; `git range-diff` against bb836988: all nine replayed `=` onto the moved main, which conflicted only at the changelog anchor and touches no file this PR touches): ``` uv run pytest tests/test_file_search_traversal_guard.py tests/test_agents_code_tools_plugin.py \ tests/test_search_tools.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py tests/test_filesystem_tools.py -q # 203 passed COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q --ignore=tests/integration # in a throwaway worktree # 24652 passed, 110 skipped in 382.30s (0:06:22) at head 8dade176, in a throwaway worktree with COLORTERM=truecolor uv run pytest tests/test_agents_code_launcher.py tests/test_agents_code_tools_plugin.py -q # the guide's seeding and wording tests, the glob wrapper # 119 passed make lint-python lint-imports lint-types lint-deps # all pass git diff --check origin/main...HEAD # clean scripts/check_source_language.py origin/main...HEAD # exit 0 scripts/check_large_files.py origin/main...HEAD # exit 0 git merge-tree --write-tree HEAD origin/main # clean ``` Revert-to-red, with the tests kept (101 tests in `test_file_search_traversal_guard.py` + `test_agents_code_tools_plugin.py`): ``` the three production files reverted to the previously reviewed head (fd5663ee) 19 failed file_search.py + filesystem.py reverted to origin/main, tree_walk.py removed collection error ``` Mutation, each mutant applied alone, control restored between runs: ``` deadline checked once per directory (the reviewed placement) 10 failed walk starts at the root instead of the pattern's prefix 2 failed trailing slash matches files too 4 failed a pattern naming one file still walks 1 failed the prefix itself is never a match 7 failed a pattern naming one directory walks beneath it 1 failed control (nothing mutated) 101 passed ``` From the first review round (still holding at this head): find without `to_thread` 1 failed; list_dir without `to_thread` 1 failed; grep fallback without `to_thread` 1 failed; pruning removed 6 failed; fork drops the PARTIAL clause 1 failed. Fourth round, the symbolic-link reading (`test_file_search_traversal_guard.py` + `test_agents_code_tools_plugin.py`, 128 tests, `-n 0`; a mutant that lets `**` sweep a link, or forgets the link positions, re-enters the cycle until the OS refuses the path and answers paths `Path.glob` never did): ``` baseline 128 passed MUTANT ** may sweep a link 8 failed (7 oracle cases, and the link the pattern ends at is opened) MUTANT find never asks to enter a link 6 failed (the six oracle patterns that pass through a link) MUTANT a link is entered even when the pattern ends at it 1 failed (scandir spy on **/vendor) MUTANT ancestor links forgotten (links_of empty) 1 failed (a cycle case answers a path Path.glob never did) MUTANT the walker consults follow but never enters 7 failed MUTANT each followed root gets a fresh deadline 9 failed (every deadline test, and the walker's one-budget test) REVERT production files to reviewed head d67184c8 7 failed CONTROL restored 128 passed ``` Revert-to-red for the noise-name pin: with `raven/agent/tools/filesystem.py` at the merge base and the test kept, `test_a_noise_name_above_the_searched_path_hides_nothing` fails on the `Directory ... is empty` answer (1 failed); at this head it passes. The `agents/**` change is two prose strings and a guide line, outside the coverage gate's `raven/**` and `plugins-dist/**` pathspecs; the guide's name-list test and its seeding tests (`test_agents_code_launcher.py`) cover the file. The prefix rule is pinned by mutation, the other policy (prune a noise directory inside the prefix) against the two guard files (131 tests, `-n 0`): ``` baseline 131 passed MUTANT a noise directory inside the prefix is pruned 5 failed test_find_matches_what_path_glob_matched[node_modules/**/*.py], test_find_matches_what_path_glob_matched[node_modules/*/], test_find_matches_what_path_glob_matched[node_modules/*/d.py], test_find_matches_what_path_glob_matched[node_modules/x/d.py], test_find_matches_what_path_glob_matched[src/node_modules/*.py] CONTROL restored 131 passed ``` The symlink oracle is the reviewer's own test shape over a fixture with links: `vendor -> outside/real`, `src/loop -> src`, `alias.py -> a.py`, a `node_modules` beneath the link; 19 patterns, 0 mismatches against `Path.glob` on 3.12, plus a scandir spy pinning that `**/vendor` never opens the link, a walker test pinning that the default walk yields a link and does not enter it while a `follow`ed link is walked after the tree under one deadline, and a `list_dir` and grep test pinning `rglob`/`os.walk` parity through a link. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed (CHANGELOG `Unreleased / Fixed` entry) ## Risk User-visible behaviour changes, all in tool output the model reads: - `find` and `list_dir` (recursive) can now return a partial listing with a `PARTIAL result` clause in the trailer where they previously either returned everything or hung. The clause tells the model the absence of a match is not conclusive and to narrow the search. - A path-prefixed `find` pattern spends its budget only inside its prefix, so on a large tree it finds what a root-anchored walk used to miss. - `find` refuses patterns containing `..` or starting with `/` with an `Error running find:` line; `Path.glob` used to raise on the second and could climb on the first. - `find` with a trailing `**` lists files as well as directories beneath the prefix; a trailing `/` lists directories only, as before. - `find` reaches through a symbolic link to a directory where a literal or single-wildcard component names or matches it, as it did at the merge base; the earlier revisions of this PR had lost that and returned a clean miss with no notice. `**` never enters a link, as before. - A noise directory inside a `find` pattern's literal prefix, or named as the `list_dir` path, is walked: `find('node_modules/*.js')` and `find('src/node_modules/*.js')` answer its files and `list_dir('node_modules', recursive=true)` lists it, where the merge base answered nothing and "empty". One met below the prefix is pruned as before. Both descriptions state the rule. - Recursive `list_dir` on a workspace beneath a directory named like a noise directory (`/tmp/build/proj`) lists it instead of answering "is empty". - The code-flow `glob` description, its `limit` parameter and the tool guide tell a limit cut from a traversal-budget PARTIAL; the notices themselves are unchanged. - `list_dir` flat listings are unchanged; the empty-directory message is unchanged. Rollback: revert the squash commit. The new module has no other consumers, and no state or schema is involved. - [x] Security impact considered (a pattern can no longer climb out of the fenced directory via `..`; a directory link is entered exactly where `Path.glob` entered it at the merge base, one non-`**` component, and never under `**`, so the fence's reach is what it was and not wider; `list_dir` and grep never enter one) - [x] Backward compatibility considered (the fork's trailer grammar is extended, not changed; existing fork tests pass unchanged) - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 10:59:01 +08:00
import itertools
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
import os
import subprocess
import sys
feat(*): add bounded Python diagnostics after code edits (#967) ## Summary Raven-Code reports Python syntax errors after file edits but misses unresolved names and equivalent feedback after shell edits. Add advisory Pyflakes F821/F822/F823 checks to the existing tool result; optional F401 notices remain off by default. No additional model call or code execution is involved. ```text write_file / edit_file -> inspect final source, compare with a baseline that can be read and analyzed -> retain syntax warnings; suppress existing name findings exec -> require checkpoint visibility for the working directory -> read the changed files from the host's write record for the command -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently -> append ordered diagnostics, preserving exit status and file metadata ``` Existing syntax failures remain visible after unrelated edits and identical rewrites. A baseline that cannot be read, parsed or analyzed is marked unknown; overwriting non-UTF-8 source still checks the known replacement. Source nested too deeply for the parser or checker, and a changed file that cannot be read, report diagnostics unavailable for that file rather than passing as clean. Every check reads a file as the bytes Python runs: text a write or edit stores is checked as its UTF-8 bytes, the content it replaced included, and a file from disk as its bytes, each decoded as Python decodes source, by its coding declaration or as UTF-8. Bytes that do not decode are reported as a syntax error at the line where decoding stops, so a declaration the written text cannot satisfy is reported however the file was changed. Noqa comments are read as Ruff reads them, including after other comment directives and never inside string literals: a file-level `ruff: noqa` / `flake8: noqa` counts only on a line of its own, and a marker counts only when codes follow its colon or, with no colon, whitespace, another comment or the end of the comment follows it. Unlike Ruff, which takes the first directive in a comment, every directive in it counts. The seeded tool guide gives the syntax note's own advice. With no checkpoint or no visibility verdict, exec diagnostics do not inspect changed file contents; direct file checks remain available. Pyflakes is imported when the tool face loads, not by whichever check first needs it: imported lazily, its cold import fell inside that check's one-second wait, and on a slow Windows runner the one test that paid it reported diagnostics unavailable for a plain syntax error. Private workers bound reads and checks to four active jobs per process. Saturated capacity rejects new work without queuing, and timeout/cancellation retains each slot until the actual work finishes. Name checks are capped at 256 KiB of source and the syntax check at 2 MiB, and a check skipped for size says so in one line. Findings are capped at eight per file, and exec diagnostic text at 6,000 characters plus omission notices. Runtime changes stay inside the Raven-Code code-flow plugin, except for one fix to `CheckpointService.trackable`, the visibility verdict that exec diagnostics and the host's exec write record both rely on. On macOS, git precomposes a file name it reads off the disk before matching an ignore rule, but takes the paths `trackable` sends to `git check-ignore --stdin` as written. A file stored with a decomposed (NFD) name under a `.gitignore` rule written precomposed (NFC) is therefore left out of the checkpoint, yet main vouches for it, so the write record's diff carries its text, and exec diagnostics would quote its source lines to the model. Nor is git's precomposition NFC: it composes only neighbouring characters, by Unicode 3.2's table, and reorders nothing, so a name whose combining marks are out of canonical order comes out as neither the name as given nor its NFC. `trackable` now asks about each name as given, as NFC and in git's macOS spelling, and withholds it when any spelling is ignored. Other shared file/shell/snapshot implementations are unchanged. Raven-Code has no separate dependency manifest, so Pyflakes uses the existing root dependencies. It is the only added package; uv normalized existing lockfile markers without changing other package versions. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - **Before:** on main, Raven-Code's file tools report Python syntax errors only, and its exec reports nothing about the Python files a command changed. On macOS, main's `CheckpointService.trackable` vouches for a file stored as `secret_cafe` + U+0301 + `.py` under the rule `secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the checkpoint's staged tree leaves it out, yet its text reaches the exec write record's stored diff. - **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors over 22 test ids: 21 of them also fail on the exact base in the same environment, and the other, a remote process-tree cancel test that missed its deadline under the suite's load, passes when run alone. Tests cover actual worker overlap, retained capacity after timeout/cancellation, host-tool isolation, hidden-file quotas, visibility batching through the host's write-record wiring, baseline handling, parser-limit failures, the size bounds and their skip notes, undecodable bytes placed where decoding stops, coding declarations in written and edited text, the baseline read staying on a worker, the checker loaded with its module, the file each unavailable note names, comment directives and their marker boundary, the tool guide's agreement with the syntax note, portable commands in paths with spaces/non-ASCII characters, and the checkpoint's verdict on decomposed names and on names git spells as neither given nor NFC, held against its own staged tree. Removing any of the three spellings from the query fails the cases that pin it, as does removing the marker boundary, the parser-limit report, the unknown-baseline handling, the syntax check past the name-check bound and the decoding report, checking written text as its stored bytes, the worker around the baseline read, loading the checker with its module, or the file name in an unavailable note; restoring the guide's earlier advice fails its agreement test. git's macOS spelling is what libiconv's UTF-8-MAC conversion returns, which git precomposes with: composing neighbours by Unicode 3.2's table reproduced it for all 281,928 names built from every decomposable character's decomposition (reordered, partly composed, or with a mark added), and across 784,220 further mixed names and pairs no name was spelled outside the three asked; the same held under Python 3.13 and 3.14. The decomposed-name check asserts the macOS outcome only where git precomposes names, so it also passes with `core.precomposeunicode = false` set through `GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines match Ruff 0.15.12, except the three that hold several directives in one comment. Through the real `AgentLoop`, code-flow tool face, `CodeExecutor` and checkpoint, with the working directory under `/tmp`, under `$TMPDIR` and in a path with spaces and CJK characters, exec diagnostics report created, modified and decomposed-name files, follow `working_dir`, report an undecodable file at the line where decoding stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold `.gitignore`d files, including decomposed names and names git spells as neither given nor NFC; across repeated runs on both interpreters, a file near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600 ignored files plus one visible file take 73-90 ms with the event loop held at most 20 ms. At this head CI passes the unit suite on Linux/Python 3.12 (32619 tests over four shards), and its two Windows steps pass the diagnostic files on native Windows/Python 3.13: 122 file/worker tests, and 27 exec diagnostic tests with 17 pre-existing executor tests deselected; the symlink fixture skips only where the host cannot create links. ```sh env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3 uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py uv run --no-sync lint-imports uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py ``` Native Windows CI commands: ```sh uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short ``` Ruff, format and type checks passed; all 10 import contracts kept; the commit-message, large-file and source-language checks pass over the branch. The launcher tests unset inherited search API keys to keep the expected tool list deterministic. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes does not activate checks in other agents. The change is reversible and never fixes/removes code or rolls back edits. Static diagnostics can need judgment for dynamic names, target Python versions, and project lint configuration; only explicit noqa directives are honored. Passing diagnostics do not replace tests. The `trackable` fix reaches every exec write record, not only Raven-Code: a file whose name, as given, as NFC or in git's macOS spelling, matches an ignore rule keeps its counts and loses its diff and removal body. Where git matches another of those spellings, as off macOS for a decomposed name under a precomposed rule, the checkpoint stores the file but it is still withheld; its counts remain. Exec diagnostics now require an active checkpoint. Noninteractive runs using the default interactive checkpoint policy must explicitly enable checkpoints to receive this feedback. Visibility is checked in batches of up to 256 files and stops once eight visible files are found; each batch holds the event loop while the host resolves its paths, about 130 ms for a full batch on native Windows. Each file-check wait is one second, and post-command diagnostics have a three-second wait budget; past it, or when they fail as a whole, they are dropped and the command's result is kept. These are caller budgets, not hard I/O deadlines: running OS reads and inherited synchronous host callbacks cannot be forcibly interrupted. Private worker capacity remains bounded. Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took 0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on slower hardware such a check can outlast the one-second wait and report diagnostics unavailable. Python's GIL means concurrency does not promise a CPU speedup. Windows CI exercises the diagnostics layer with the real native DirectExecutor. The existing CodeExecutor environment probe uses POSIX quoting and already fails on Windows at the base; this PR neither changes that probe nor claims end-to-end compatibility for it. Symlink setup is skipped only if the host cannot create the fixture. Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics` for no post-command checks, or `pythonSyntaxNote` for no diagnostics, under `plugins.config["code-flow"].tools`. The PR can also be reverted as a unit. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn> Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
import threading
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
from pathlib import Path
fix(agent): bound every tree walk and take it off the event loop (#566) ## Summary On 2026-09-20 one `find` call froze the whole gateway for 2h21m. The model asked for `find(pattern="**/gps_trilateration/index.html", path="/Users/admin")`, and `FindTool` ran `Path.glob` synchronously on the event loop, filtered the noise directories only after walking them, and had no deadline. The registry's 300s ceiling is an `asyncio.wait_for`, which cannot preempt synchronous code, so every session and every subagent on the gateway stopped until the walk came back (`gateway.log`: `Tool result: find duration=8476519ms`). `list_dir` with `recursive=true` had the same shape through `rglob`. The fix gives the three tree-walking tools one walk, `raven.agent.tools.tree_walk`, with these bounds: - noise directories (`node_modules`, `.git`, ...) are pruned before they are entered, not filtered afterwards -- the pruning tests spy on `os.scandir` and assert the noise directory is never opened; - the walk yields one entry at a time and checks a 20s wall-clock budget (the value grep's fallback already used) before every entry, so whatever the caller does with one entry -- the `stat` per `find` match, the read per grep file -- is covered by the check on the next. The budget cannot be overrun by one large or slow directory: the probe with eight matches, a 30ms stat each and a 50ms budget returns 2 matches and the notice. When the budget is hit, `find` and `list_dir` append a `PARTIAL result: ... traversal budget ...` clause to their trailer, and a deadline that fires before any match is not reported as a clean `No files found` / `Directory is empty`; - every caller runs the walk in `asyncio.to_thread`, so the loop keeps serving other sessions while the walk runs. Cancelling the thread does not stop the worker, which is why the deadline lives inside the walk. The pure-Python `grep` fallback already had the prune and the deadline; it moves onto the shared walk and off the loop with the others. `find` matches each path component with `fnmatch.translate`, with a component that is exactly `**` spanning any number of components; the matcher is a position-set simulation (`_advance`), linear in components times segments, so a pattern carrying many `**` cannot make one match call slow. A pattern's leading literal components decide where the walk starts: `src/**/*.py` begins inside `src` and never lists a sibling of it, a pattern that is all literal (`src/lib/c.py`, `src/lib`) names exactly one path and answers without a walk, and the prefix itself is a match when the rest of the pattern can match nothing (`src/**` still lists `src/`). It answers exactly what `Path.glob` answered for the same pattern: pinned by a parametrized oracle test over twenty-seven patterns on a fixture and nineteen on a fixture with links (a directory link out of the tree, a link cycle, a file link, a noise directory beneath a link), and checked by hand against `Path.glob` over `raven/` for 27 patterns including bracket edge cases and trailing slashes, 0 mismatches. Two deliberate differences: a trailing `**` also lists files beneath its prefix (3.12's `Path.glob` listed only directories there), and a pattern with `..` or a leading `/` is refused instead of followed, since a pattern must not reach outside the directory the fence resolved. A trailing slash keeps `Path.glob`'s directory-only meaning (`*/`, `src/**/`). A symbolic link to a directory is entered where a single component names or matches it and never under `**`, which is how `Path.glob` read a pattern: `*/util/helper.py` reaches through a linked `vendor`, `**/helper.py` does not, and a link cycle is entered once per component the pattern spends on it, so `*/*/*/b.py` enters `src/loop -> src` twice and stops. The walker asks its caller per link (`follow`), `find` answers from the pattern and hands the link positions of a path to the matcher so `**` cannot claim them, and `list_dir` and the grep fallback never ask, staying as `rglob` and `os.walk` left them. A third deliberate difference from the merge base: a noise directory inside a pattern's literal prefix (`node_modules/*.js`, `src/node_modules/*.js`) is walked, since the pattern asked for it, where `Path.glob`'s filter dropped it; one met below the prefix is pruned as before (`*/node_modules/*.js` answers nothing), `list_dir` on such a path lists it, and the `find`, `list_dir` and code-flow `glob` descriptions state that rule instead of an unconditional "skipped". Review found an earlier wording of the two `find` descriptions one component short ("except one the pattern starts with"), while the oracle helper already pruned below the prefix; the sentences now say what the helper checks, and the oracle has the `src/node_modules` shape on both sides. The raven-code `glob` tool (`agents/raven-code/.../tools/search.py`) parses trunk `find` output line by line and strips the trailer before merging brace alternatives. It learns the new clause, so a cut-short child is reported as cut short rather than as a tree that lacks the files; its `is_notice` grammar covers the new line so it is never pasted into a listing as a path. The code-flow tool guide (`prompts/TOOLS_CODE.md`), the wrapper's description and its `limit` parameter told the model that a PARTIAL `glob` result means "raise `limit` or narrow the pattern"; a traversal-budget PARTIAL is now a result `glob` can return and a higher limit does nothing for it, so all three now tell the two notices apart. The seeded copy of the guide refreshes through the receipt `seed_guide` keeps beside it, so no legacy digest is added. Fixed in passing, and now pinned: recursive `list_dir` filtered `rglob` results on the components of the absolute path, so a workspace beneath a directory named `build`, `dist`, `venv` or another noise name listed as "Directory ... is empty" (reproduced at the merge base under `<tmp>/build/proj`; this head lists it). The shared walk prunes below the listed path only; `find` and the grep fallback already did. Not changed: the 20s value. It is the budget grep's fallback already ran under; this PR does not re-measure it, it makes it hold for every entry of every walk. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification At head 8dade176, on origin/main at d50a13c1 (nine commits: the walk, the grep pin, the changelog, the per-entry budget with the prefix start, the literal-directory case, the symbolic-link reading with the noise-prefix wording, the noise-name-above-the-path pin, the guide wording, and the prefix-rule wording; `git range-diff` against bb836988: all nine replayed `=` onto the moved main, which conflicted only at the changelog anchor and touches no file this PR touches): ``` uv run pytest tests/test_file_search_traversal_guard.py tests/test_agents_code_tools_plugin.py \ tests/test_search_tools.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py tests/test_filesystem_tools.py -q # 203 passed COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q --ignore=tests/integration # in a throwaway worktree # 24652 passed, 110 skipped in 382.30s (0:06:22) at head 8dade176, in a throwaway worktree with COLORTERM=truecolor uv run pytest tests/test_agents_code_launcher.py tests/test_agents_code_tools_plugin.py -q # the guide's seeding and wording tests, the glob wrapper # 119 passed make lint-python lint-imports lint-types lint-deps # all pass git diff --check origin/main...HEAD # clean scripts/check_source_language.py origin/main...HEAD # exit 0 scripts/check_large_files.py origin/main...HEAD # exit 0 git merge-tree --write-tree HEAD origin/main # clean ``` Revert-to-red, with the tests kept (101 tests in `test_file_search_traversal_guard.py` + `test_agents_code_tools_plugin.py`): ``` the three production files reverted to the previously reviewed head (fd5663ee) 19 failed file_search.py + filesystem.py reverted to origin/main, tree_walk.py removed collection error ``` Mutation, each mutant applied alone, control restored between runs: ``` deadline checked once per directory (the reviewed placement) 10 failed walk starts at the root instead of the pattern's prefix 2 failed trailing slash matches files too 4 failed a pattern naming one file still walks 1 failed the prefix itself is never a match 7 failed a pattern naming one directory walks beneath it 1 failed control (nothing mutated) 101 passed ``` From the first review round (still holding at this head): find without `to_thread` 1 failed; list_dir without `to_thread` 1 failed; grep fallback without `to_thread` 1 failed; pruning removed 6 failed; fork drops the PARTIAL clause 1 failed. Fourth round, the symbolic-link reading (`test_file_search_traversal_guard.py` + `test_agents_code_tools_plugin.py`, 128 tests, `-n 0`; a mutant that lets `**` sweep a link, or forgets the link positions, re-enters the cycle until the OS refuses the path and answers paths `Path.glob` never did): ``` baseline 128 passed MUTANT ** may sweep a link 8 failed (7 oracle cases, and the link the pattern ends at is opened) MUTANT find never asks to enter a link 6 failed (the six oracle patterns that pass through a link) MUTANT a link is entered even when the pattern ends at it 1 failed (scandir spy on **/vendor) MUTANT ancestor links forgotten (links_of empty) 1 failed (a cycle case answers a path Path.glob never did) MUTANT the walker consults follow but never enters 7 failed MUTANT each followed root gets a fresh deadline 9 failed (every deadline test, and the walker's one-budget test) REVERT production files to reviewed head d67184c8 7 failed CONTROL restored 128 passed ``` Revert-to-red for the noise-name pin: with `raven/agent/tools/filesystem.py` at the merge base and the test kept, `test_a_noise_name_above_the_searched_path_hides_nothing` fails on the `Directory ... is empty` answer (1 failed); at this head it passes. The `agents/**` change is two prose strings and a guide line, outside the coverage gate's `raven/**` and `plugins-dist/**` pathspecs; the guide's name-list test and its seeding tests (`test_agents_code_launcher.py`) cover the file. The prefix rule is pinned by mutation, the other policy (prune a noise directory inside the prefix) against the two guard files (131 tests, `-n 0`): ``` baseline 131 passed MUTANT a noise directory inside the prefix is pruned 5 failed test_find_matches_what_path_glob_matched[node_modules/**/*.py], test_find_matches_what_path_glob_matched[node_modules/*/], test_find_matches_what_path_glob_matched[node_modules/*/d.py], test_find_matches_what_path_glob_matched[node_modules/x/d.py], test_find_matches_what_path_glob_matched[src/node_modules/*.py] CONTROL restored 131 passed ``` The symlink oracle is the reviewer's own test shape over a fixture with links: `vendor -> outside/real`, `src/loop -> src`, `alias.py -> a.py`, a `node_modules` beneath the link; 19 patterns, 0 mismatches against `Path.glob` on 3.12, plus a scandir spy pinning that `**/vendor` never opens the link, a walker test pinning that the default walk yields a link and does not enter it while a `follow`ed link is walked after the tree under one deadline, and a `list_dir` and grep test pinning `rglob`/`os.walk` parity through a link. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed (CHANGELOG `Unreleased / Fixed` entry) ## Risk User-visible behaviour changes, all in tool output the model reads: - `find` and `list_dir` (recursive) can now return a partial listing with a `PARTIAL result` clause in the trailer where they previously either returned everything or hung. The clause tells the model the absence of a match is not conclusive and to narrow the search. - A path-prefixed `find` pattern spends its budget only inside its prefix, so on a large tree it finds what a root-anchored walk used to miss. - `find` refuses patterns containing `..` or starting with `/` with an `Error running find:` line; `Path.glob` used to raise on the second and could climb on the first. - `find` with a trailing `**` lists files as well as directories beneath the prefix; a trailing `/` lists directories only, as before. - `find` reaches through a symbolic link to a directory where a literal or single-wildcard component names or matches it, as it did at the merge base; the earlier revisions of this PR had lost that and returned a clean miss with no notice. `**` never enters a link, as before. - A noise directory inside a `find` pattern's literal prefix, or named as the `list_dir` path, is walked: `find('node_modules/*.js')` and `find('src/node_modules/*.js')` answer its files and `list_dir('node_modules', recursive=true)` lists it, where the merge base answered nothing and "empty". One met below the prefix is pruned as before. Both descriptions state the rule. - Recursive `list_dir` on a workspace beneath a directory named like a noise directory (`/tmp/build/proj`) lists it instead of answering "is empty". - The code-flow `glob` description, its `limit` parameter and the tool guide tell a limit cut from a traversal-budget PARTIAL; the notices themselves are unchanged. - `list_dir` flat listings are unchanged; the empty-directory message is unchanged. Rollback: revert the squash commit. The new module has no other consumers, and no state or schema is involved. - [x] Security impact considered (a pattern can no longer climb out of the fenced directory via `..`; a directory link is entered exactly where `Path.glob` entered it at the merge base, one non-`**` component, and never under `**`, so the fence's reach is what it was and not wider; `list_dir` and grep never enter one) - [x] Backward compatibility considered (the fork's trailer grammar is extended, not changed; existing fork tests pass unchanged) - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 10:59:01 +08:00
from types import SimpleNamespace
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
import pytest
from raven.agent import workdir
from raven.agent.tools.filesystem import ReadFileTool as TrunkReadFileTool
from raven.agent.tools.registry import ToolRegistry
from raven.contracts.tool import ToolResult
from raven.plugins import DiscoveredPlugin, ManifestOrigin, PluginManifest, PluginRegistry
from raven.plugins.context import PluginContext, ServiceLocator
REPO = Path(__file__).resolve().parent.parent
PLUGIN_DIR = REPO / "agents" / "raven-code" / "plugins" / "code-flow"
sys.path.insert(0, str(PLUGIN_DIR))
from code_flow.tools import plugin as factories # noqa: E402
from code_flow.tools.filesystem import EditFileTool, ReadFileTool, ReadLedger, WriteFileTool # noqa: E402
from code_flow.tools.read_state import ReadSessions, owner_for # noqa: E402
from code_flow.tools.search import GlobTool, expand_braces, is_notice # noqa: E402
MANIFEST_PATH = PLUGIN_DIR / "raven-plugin.toml"
FACE = ("read_file", "write_file", "edit_file", "list_dir", "glob", "todo", "exec")
def _ctx(tmp_path: Path, tools: dict) -> PluginContext:
"""A plugin context carrying the flow slice whose ``tools`` section is the
product's own tool face."""
return PluginContext(
config={"enabled": True, "tools": tools},
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
)
def trunk_read_file(workspace: Path):
from raven.agent.tools.filesystem import ReadFileTool as TrunkRead
return TrunkRead(workspace=workspace)
def registry_execute(reg: ToolRegistry, name: str, params: dict):
return reg.execute(name, params)
def _run(coro):
return asyncio.get_event_loop_policy().new_event_loop().run_until_complete(coro)
def _text(result) -> str:
return result.model_text if isinstance(result, ToolResult) else str(result)
@pytest.fixture(autouse=True)
def _fresh_ledger(tmp_path):
reads = ReadSessions(owner_for(_ctx(tmp_path, {"enabled": True})))
reads.bind("cli:unit")
yield
reads.unbind()
@pytest.fixture()
def registry(tmp_path):
ctx = _ctx(tmp_path, {"enabled": True})
reg = ToolRegistry()
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
):
reg.register(make(ctx))
return reg
# --- manifest and admission ----------------------------------------------------
def test_the_manifest_contributes_the_seven_tools_beside_the_flows_own_seats():
"""One plugin per product, the way every sibling ships: the tool face is a
subpackage of code-flow, not a plugin of its own."""
mf = PluginManifest.from_toml_path(MANIFEST_PATH)
assert mf.id == "code-flow"
assert [t.name for t in mf.contributes.tools] == list(FACE)
assert [h.name for h in mf.contributes.hooks] == ["code_flow"]
assert mf.contributes.tool_gates == []
assert [f.factory for f in mf.contributes.tools] == [f"code_flow.tools.plugin:make_{name}" for name in FACE]
def test_the_registry_builds_every_tool_from_the_real_factory_strings(tmp_path):
reg = PluginRegistry()
mf = PluginManifest.from_toml_path(MANIFEST_PATH)
reg.activate([DiscoveredPlugin(manifest=mf, source=ManifestOrigin.USER, location=MANIFEST_PATH)])
assert reg.tool_names() == sorted(FACE)
services = ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a")
slice_ = {"enabled": True, "tools": {"enabled": True}}
for name in FACE:
tool = reg.build_tool(name, config=slice_, services=services)
assert tool is not None and tool.name == name
feat(*): add bounded Python diagnostics after code edits (#967) ## Summary Raven-Code reports Python syntax errors after file edits but misses unresolved names and equivalent feedback after shell edits. Add advisory Pyflakes F821/F822/F823 checks to the existing tool result; optional F401 notices remain off by default. No additional model call or code execution is involved. ```text write_file / edit_file -> inspect final source, compare with a baseline that can be read and analyzed -> retain syntax warnings; suppress existing name findings exec -> require checkpoint visibility for the working directory -> read the changed files from the host's write record for the command -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently -> append ordered diagnostics, preserving exit status and file metadata ``` Existing syntax failures remain visible after unrelated edits and identical rewrites. A baseline that cannot be read, parsed or analyzed is marked unknown; overwriting non-UTF-8 source still checks the known replacement. Source nested too deeply for the parser or checker, and a changed file that cannot be read, report diagnostics unavailable for that file rather than passing as clean. Every check reads a file as the bytes Python runs: text a write or edit stores is checked as its UTF-8 bytes, the content it replaced included, and a file from disk as its bytes, each decoded as Python decodes source, by its coding declaration or as UTF-8. Bytes that do not decode are reported as a syntax error at the line where decoding stops, so a declaration the written text cannot satisfy is reported however the file was changed. Noqa comments are read as Ruff reads them, including after other comment directives and never inside string literals: a file-level `ruff: noqa` / `flake8: noqa` counts only on a line of its own, and a marker counts only when codes follow its colon or, with no colon, whitespace, another comment or the end of the comment follows it. Unlike Ruff, which takes the first directive in a comment, every directive in it counts. The seeded tool guide gives the syntax note's own advice. With no checkpoint or no visibility verdict, exec diagnostics do not inspect changed file contents; direct file checks remain available. Pyflakes is imported when the tool face loads, not by whichever check first needs it: imported lazily, its cold import fell inside that check's one-second wait, and on a slow Windows runner the one test that paid it reported diagnostics unavailable for a plain syntax error. Private workers bound reads and checks to four active jobs per process. Saturated capacity rejects new work without queuing, and timeout/cancellation retains each slot until the actual work finishes. Name checks are capped at 256 KiB of source and the syntax check at 2 MiB, and a check skipped for size says so in one line. Findings are capped at eight per file, and exec diagnostic text at 6,000 characters plus omission notices. Runtime changes stay inside the Raven-Code code-flow plugin, except for one fix to `CheckpointService.trackable`, the visibility verdict that exec diagnostics and the host's exec write record both rely on. On macOS, git precomposes a file name it reads off the disk before matching an ignore rule, but takes the paths `trackable` sends to `git check-ignore --stdin` as written. A file stored with a decomposed (NFD) name under a `.gitignore` rule written precomposed (NFC) is therefore left out of the checkpoint, yet main vouches for it, so the write record's diff carries its text, and exec diagnostics would quote its source lines to the model. Nor is git's precomposition NFC: it composes only neighbouring characters, by Unicode 3.2's table, and reorders nothing, so a name whose combining marks are out of canonical order comes out as neither the name as given nor its NFC. `trackable` now asks about each name as given, as NFC and in git's macOS spelling, and withholds it when any spelling is ignored. Other shared file/shell/snapshot implementations are unchanged. Raven-Code has no separate dependency manifest, so Pyflakes uses the existing root dependencies. It is the only added package; uv normalized existing lockfile markers without changing other package versions. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - **Before:** on main, Raven-Code's file tools report Python syntax errors only, and its exec reports nothing about the Python files a command changed. On macOS, main's `CheckpointService.trackable` vouches for a file stored as `secret_cafe` + U+0301 + `.py` under the rule `secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the checkpoint's staged tree leaves it out, yet its text reaches the exec write record's stored diff. - **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors over 22 test ids: 21 of them also fail on the exact base in the same environment, and the other, a remote process-tree cancel test that missed its deadline under the suite's load, passes when run alone. Tests cover actual worker overlap, retained capacity after timeout/cancellation, host-tool isolation, hidden-file quotas, visibility batching through the host's write-record wiring, baseline handling, parser-limit failures, the size bounds and their skip notes, undecodable bytes placed where decoding stops, coding declarations in written and edited text, the baseline read staying on a worker, the checker loaded with its module, the file each unavailable note names, comment directives and their marker boundary, the tool guide's agreement with the syntax note, portable commands in paths with spaces/non-ASCII characters, and the checkpoint's verdict on decomposed names and on names git spells as neither given nor NFC, held against its own staged tree. Removing any of the three spellings from the query fails the cases that pin it, as does removing the marker boundary, the parser-limit report, the unknown-baseline handling, the syntax check past the name-check bound and the decoding report, checking written text as its stored bytes, the worker around the baseline read, loading the checker with its module, or the file name in an unavailable note; restoring the guide's earlier advice fails its agreement test. git's macOS spelling is what libiconv's UTF-8-MAC conversion returns, which git precomposes with: composing neighbours by Unicode 3.2's table reproduced it for all 281,928 names built from every decomposable character's decomposition (reordered, partly composed, or with a mark added), and across 784,220 further mixed names and pairs no name was spelled outside the three asked; the same held under Python 3.13 and 3.14. The decomposed-name check asserts the macOS outcome only where git precomposes names, so it also passes with `core.precomposeunicode = false` set through `GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines match Ruff 0.15.12, except the three that hold several directives in one comment. Through the real `AgentLoop`, code-flow tool face, `CodeExecutor` and checkpoint, with the working directory under `/tmp`, under `$TMPDIR` and in a path with spaces and CJK characters, exec diagnostics report created, modified and decomposed-name files, follow `working_dir`, report an undecodable file at the line where decoding stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold `.gitignore`d files, including decomposed names and names git spells as neither given nor NFC; across repeated runs on both interpreters, a file near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600 ignored files plus one visible file take 73-90 ms with the event loop held at most 20 ms. At this head CI passes the unit suite on Linux/Python 3.12 (32619 tests over four shards), and its two Windows steps pass the diagnostic files on native Windows/Python 3.13: 122 file/worker tests, and 27 exec diagnostic tests with 17 pre-existing executor tests deselected; the symlink fixture skips only where the host cannot create links. ```sh env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3 uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py uv run --no-sync lint-imports uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py ``` Native Windows CI commands: ```sh uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short ``` Ruff, format and type checks passed; all 10 import contracts kept; the commit-message, large-file and source-language checks pass over the branch. The launcher tests unset inherited search API keys to keep the expected tool list deterministic. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes does not activate checks in other agents. The change is reversible and never fixes/removes code or rolls back edits. Static diagnostics can need judgment for dynamic names, target Python versions, and project lint configuration; only explicit noqa directives are honored. Passing diagnostics do not replace tests. The `trackable` fix reaches every exec write record, not only Raven-Code: a file whose name, as given, as NFC or in git's macOS spelling, matches an ignore rule keeps its counts and loses its diff and removal body. Where git matches another of those spellings, as off macOS for a decomposed name under a precomposed rule, the checkpoint stores the file but it is still withheld; its counts remain. Exec diagnostics now require an active checkpoint. Noninteractive runs using the default interactive checkpoint policy must explicitly enable checkpoints to receive this feedback. Visibility is checked in batches of up to 256 files and stops once eight visible files are found; each batch holds the event loop while the host resolves its paths, about 130 ms for a full batch on native Windows. Each file-check wait is one second, and post-command diagnostics have a three-second wait budget; past it, or when they fail as a whole, they are dropped and the command's result is kept. These are caller budgets, not hard I/O deadlines: running OS reads and inherited synchronous host callbacks cannot be forcibly interrupted. Private worker capacity remains bounded. Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took 0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on slower hardware such a check can outlast the one-second wait and report diagnostics unavailable. Python's GIL means concurrency does not promise a CPU speedup. Windows CI exercises the diagnostics layer with the real native DirectExecutor. The existing CodeExecutor environment probe uses POSIX quoting and already fails on Windows at the base; this PR neither changes that probe nor claims end-to-end compatibility for it. Symlink setup is skipped only if the host cannot create the fixture. Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics` for no post-command checks, or `pythonSyntaxNote` for no diagnostics, under `plugins.config["code-flow"].tools`. The PR can also be reverted as a unit. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn> Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
@pytest.mark.parametrize(
"config",
[
{},
{"enabled": True},
{"enabled": True, "tools": {"enabled": False}},
{"tools": {"pythonLint": True, "execPythonDiagnostics": True}},
{"tools": {"enabled": False, "pythonLint": True, "execPythonDiagnostics": True}},
],
)
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
def test_a_slice_that_does_not_ask_for_the_face_contributes_nothing(tmp_path, config):
"""The flow and the face have one switch each: a product can keep the flow
and hand the tools back to the host."""
ctx = PluginContext(config=config, services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"))
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
):
assert make(ctx) is None
feat(*): add bounded Python diagnostics after code edits (#967) ## Summary Raven-Code reports Python syntax errors after file edits but misses unresolved names and equivalent feedback after shell edits. Add advisory Pyflakes F821/F822/F823 checks to the existing tool result; optional F401 notices remain off by default. No additional model call or code execution is involved. ```text write_file / edit_file -> inspect final source, compare with a baseline that can be read and analyzed -> retain syntax warnings; suppress existing name findings exec -> require checkpoint visibility for the working directory -> read the changed files from the host's write record for the command -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently -> append ordered diagnostics, preserving exit status and file metadata ``` Existing syntax failures remain visible after unrelated edits and identical rewrites. A baseline that cannot be read, parsed or analyzed is marked unknown; overwriting non-UTF-8 source still checks the known replacement. Source nested too deeply for the parser or checker, and a changed file that cannot be read, report diagnostics unavailable for that file rather than passing as clean. Every check reads a file as the bytes Python runs: text a write or edit stores is checked as its UTF-8 bytes, the content it replaced included, and a file from disk as its bytes, each decoded as Python decodes source, by its coding declaration or as UTF-8. Bytes that do not decode are reported as a syntax error at the line where decoding stops, so a declaration the written text cannot satisfy is reported however the file was changed. Noqa comments are read as Ruff reads them, including after other comment directives and never inside string literals: a file-level `ruff: noqa` / `flake8: noqa` counts only on a line of its own, and a marker counts only when codes follow its colon or, with no colon, whitespace, another comment or the end of the comment follows it. Unlike Ruff, which takes the first directive in a comment, every directive in it counts. The seeded tool guide gives the syntax note's own advice. With no checkpoint or no visibility verdict, exec diagnostics do not inspect changed file contents; direct file checks remain available. Pyflakes is imported when the tool face loads, not by whichever check first needs it: imported lazily, its cold import fell inside that check's one-second wait, and on a slow Windows runner the one test that paid it reported diagnostics unavailable for a plain syntax error. Private workers bound reads and checks to four active jobs per process. Saturated capacity rejects new work without queuing, and timeout/cancellation retains each slot until the actual work finishes. Name checks are capped at 256 KiB of source and the syntax check at 2 MiB, and a check skipped for size says so in one line. Findings are capped at eight per file, and exec diagnostic text at 6,000 characters plus omission notices. Runtime changes stay inside the Raven-Code code-flow plugin, except for one fix to `CheckpointService.trackable`, the visibility verdict that exec diagnostics and the host's exec write record both rely on. On macOS, git precomposes a file name it reads off the disk before matching an ignore rule, but takes the paths `trackable` sends to `git check-ignore --stdin` as written. A file stored with a decomposed (NFD) name under a `.gitignore` rule written precomposed (NFC) is therefore left out of the checkpoint, yet main vouches for it, so the write record's diff carries its text, and exec diagnostics would quote its source lines to the model. Nor is git's precomposition NFC: it composes only neighbouring characters, by Unicode 3.2's table, and reorders nothing, so a name whose combining marks are out of canonical order comes out as neither the name as given nor its NFC. `trackable` now asks about each name as given, as NFC and in git's macOS spelling, and withholds it when any spelling is ignored. Other shared file/shell/snapshot implementations are unchanged. Raven-Code has no separate dependency manifest, so Pyflakes uses the existing root dependencies. It is the only added package; uv normalized existing lockfile markers without changing other package versions. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - **Before:** on main, Raven-Code's file tools report Python syntax errors only, and its exec reports nothing about the Python files a command changed. On macOS, main's `CheckpointService.trackable` vouches for a file stored as `secret_cafe` + U+0301 + `.py` under the rule `secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the checkpoint's staged tree leaves it out, yet its text reaches the exec write record's stored diff. - **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors over 22 test ids: 21 of them also fail on the exact base in the same environment, and the other, a remote process-tree cancel test that missed its deadline under the suite's load, passes when run alone. Tests cover actual worker overlap, retained capacity after timeout/cancellation, host-tool isolation, hidden-file quotas, visibility batching through the host's write-record wiring, baseline handling, parser-limit failures, the size bounds and their skip notes, undecodable bytes placed where decoding stops, coding declarations in written and edited text, the baseline read staying on a worker, the checker loaded with its module, the file each unavailable note names, comment directives and their marker boundary, the tool guide's agreement with the syntax note, portable commands in paths with spaces/non-ASCII characters, and the checkpoint's verdict on decomposed names and on names git spells as neither given nor NFC, held against its own staged tree. Removing any of the three spellings from the query fails the cases that pin it, as does removing the marker boundary, the parser-limit report, the unknown-baseline handling, the syntax check past the name-check bound and the decoding report, checking written text as its stored bytes, the worker around the baseline read, loading the checker with its module, or the file name in an unavailable note; restoring the guide's earlier advice fails its agreement test. git's macOS spelling is what libiconv's UTF-8-MAC conversion returns, which git precomposes with: composing neighbours by Unicode 3.2's table reproduced it for all 281,928 names built from every decomposable character's decomposition (reordered, partly composed, or with a mark added), and across 784,220 further mixed names and pairs no name was spelled outside the three asked; the same held under Python 3.13 and 3.14. The decomposed-name check asserts the macOS outcome only where git precomposes names, so it also passes with `core.precomposeunicode = false` set through `GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines match Ruff 0.15.12, except the three that hold several directives in one comment. Through the real `AgentLoop`, code-flow tool face, `CodeExecutor` and checkpoint, with the working directory under `/tmp`, under `$TMPDIR` and in a path with spaces and CJK characters, exec diagnostics report created, modified and decomposed-name files, follow `working_dir`, report an undecodable file at the line where decoding stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold `.gitignore`d files, including decomposed names and names git spells as neither given nor NFC; across repeated runs on both interpreters, a file near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600 ignored files plus one visible file take 73-90 ms with the event loop held at most 20 ms. At this head CI passes the unit suite on Linux/Python 3.12 (32619 tests over four shards), and its two Windows steps pass the diagnostic files on native Windows/Python 3.13: 122 file/worker tests, and 27 exec diagnostic tests with 17 pre-existing executor tests deselected; the symlink fixture skips only where the host cannot create links. ```sh env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3 uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py uv run --no-sync lint-imports uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py ``` Native Windows CI commands: ```sh uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short ``` Ruff, format and type checks passed; all 10 import contracts kept; the commit-message, large-file and source-language checks pass over the branch. The launcher tests unset inherited search API keys to keep the expected tool list deterministic. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes does not activate checks in other agents. The change is reversible and never fixes/removes code or rolls back edits. Static diagnostics can need judgment for dynamic names, target Python versions, and project lint configuration; only explicit noqa directives are honored. Passing diagnostics do not replace tests. The `trackable` fix reaches every exec write record, not only Raven-Code: a file whose name, as given, as NFC or in git's macOS spelling, matches an ignore rule keeps its counts and loses its diff and removal body. Where git matches another of those spellings, as off macOS for a decomposed name under a precomposed rule, the checkpoint stores the file but it is still withheld; its counts remain. Exec diagnostics now require an active checkpoint. Noninteractive runs using the default interactive checkpoint policy must explicitly enable checkpoints to receive this feedback. Visibility is checked in batches of up to 256 files and stops once eight visible files are found; each batch holds the event loop while the host resolves its paths, about 130 ms for a full batch on native Windows. Each file-check wait is one second, and post-command diagnostics have a three-second wait budget; past it, or when they fail as a whole, they are dropped and the command's result is kept. These are caller budgets, not hard I/O deadlines: running OS reads and inherited synchronous host callbacks cannot be forcibly interrupted. Private worker capacity remains bounded. Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took 0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on slower hardware such a check can outlast the one-second wait and report diagnostics unavailable. Python's GIL means concurrency does not promise a CPU speedup. Windows CI exercises the diagnostics layer with the real native DirectExecutor. The existing CodeExecutor environment probe uses POSIX quoting and already fails on Windows at the base; this PR neither changes that probe nor claims end-to-end compatibility for it. Symlink setup is skipped only if the host cannot create the fixture. Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics` for no post-command checks, or `pythonSyntaxNote` for no diagnostics, under `plugins.config["code-flow"].tools`. The PR can also be reverted as a unit. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn> Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
@pytest.mark.asyncio
async def test_loading_coding_tools_does_not_add_diagnostics_to_host_tools(tmp_path, monkeypatch):
from code_flow.tools.diagnostics import PythonDiagnostics
from raven.agent.tools.filesystem import WriteFileTool as TrunkWriteFileTool
from raven.agent.tools.shell import ExecTool as TrunkExecTool
calls = []
async def check(*args, **kwargs):
calls.append(args)
return "unexpected coding diagnostics"
monkeypatch.setattr(PythonDiagnostics, "check", check)
writer = TrunkWriteFileTool(workspace=tmp_path)
out = await writer.execute(path=str(tmp_path / "host.py"), content="value = missing_binding\n")
assert "Successfully wrote" in str(out)
shell = TrunkExecTool(working_dir=str(tmp_path))
out = await shell.execute(command="echo other = another_missing > shell.py")
assert "Exit code: 0" in str(out)
assert calls == []
assert "another_missing" in (tmp_path / "shell.py").read_text()
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
# --- the fork's schema, the host's names as aliases ----------------------------
def test_the_schemas_carry_the_forks_spelling(registry):
read = registry.get("read_file").parameters
assert read["required"] == ["file_path"] and "path" not in read["properties"]
edit = registry.get("edit_file").parameters
assert edit["required"] == ["file_path", "old_string", "new_string"]
assert "occurrence" in edit["properties"] and "old_text" not in edit["properties"]
write = registry.get("write_file").parameters
assert write["required"] == ["file_path", "content"]
assert registry.get("glob").name == "glob"
assert registry.get("find") is None
def test_the_hosts_spelling_still_runs_through_the_aliases(registry, tmp_path):
target = tmp_path / "a.txt"
target.write_text("alpha beta\n")
assert "1| alpha beta" in _text(_run(registry.execute("read_file", {"path": str(target)})))
result = _run(registry.execute("edit_file", {"path": str(target), "old_text": "beta", "new_text": "gamma"}))
assert "Successfully edited" in _text(result)
assert target.read_text() == "alpha gamma\n"
def test_the_forks_spelling_wins_when_a_call_carries_both(tmp_path):
tool = EditFileTool(workspace=tmp_path)
cast = tool.cast_params({"path": "legacy.py", "file_path": "fork.py", "old_text": "a", "old_string": "b"})
assert cast == {"file_path": "fork.py", "old_string": "b"}
# --- read before edit ---------------------------------------------------------------
def test_editing_a_file_nobody_read_is_refused_until_it_is_read(registry, tmp_path):
target = tmp_path / "b.txt"
target.write_text("one\n")
refused = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
)
assert refused.startswith("Error") and "have not read" in refused
assert target.read_text() == "one\n"
_run(registry.execute("read_file", {"file_path": str(target)}))
accepted = _run(registry.execute("edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
assert "Successfully edited" in _text(accepted)
assert target.read_text() == "two\n"
def test_a_write_counts_as_having_read_what_now_stands(registry, tmp_path):
target = tmp_path / "c.txt"
_run(registry.execute("write_file", {"file_path": str(target), "content": "x = 1\n"}))
result = _run(registry.execute("edit_file", {"file_path": str(target), "old_string": "1", "new_string": "2"}))
assert "Successfully edited" in _text(result)
def test_the_read_rule_is_a_switch(tmp_path):
ctx = _ctx(tmp_path, {"enabled": True, "requireReadBeforeEdit": False})
tool = factories.make_edit_file(ctx)
target = tmp_path / "d.txt"
target.write_text("k\n")
assert "Successfully edited" in _text(_run(tool.execute(file_path=str(target), old_string="k", new_string="v")))
# --- the syntax verdict ---------------------------------------------------------------
def test_a_write_that_breaks_python_says_so_in_the_same_result(registry, tmp_path):
target = tmp_path / "e.py"
result = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": "def f(:\n"})))
assert "Successfully wrote" in result
assert "syntax error" in result and "line 1" in result
def test_an_edit_that_breaks_python_says_so_and_a_clean_one_stays_quiet(registry, tmp_path):
target = tmp_path / "f.py"
target.write_text("x = 1\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
broken = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "1", "new_string": "("}))
)
assert "syntax error" in broken
fixed = _text(_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "(", "new_string": "2"})))
assert "Successfully edited" in fixed and "syntax error" not in fixed
def test_the_verdict_is_python_only_and_a_switch(tmp_path):
ctx = _ctx(tmp_path, {"enabled": True})
write = factories.make_write_file(ctx)
assert "syntax" not in _text(_run(write.execute(file_path=str(tmp_path / "g.txt"), content="def f(:\n")))
quiet = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonSyntaxNote": False}))
assert "syntax" not in _text(_run(quiet.execute(file_path=str(tmp_path / "h.py"), content="def f(:\n")))
feat(*): add bounded Python diagnostics after code edits (#967) ## Summary Raven-Code reports Python syntax errors after file edits but misses unresolved names and equivalent feedback after shell edits. Add advisory Pyflakes F821/F822/F823 checks to the existing tool result; optional F401 notices remain off by default. No additional model call or code execution is involved. ```text write_file / edit_file -> inspect final source, compare with a baseline that can be read and analyzed -> retain syntax warnings; suppress existing name findings exec -> require checkpoint visibility for the working directory -> read the changed files from the host's write record for the command -> ask visibility in batches of 256, select at most 8, check up to 4 concurrently -> append ordered diagnostics, preserving exit status and file metadata ``` Existing syntax failures remain visible after unrelated edits and identical rewrites. A baseline that cannot be read, parsed or analyzed is marked unknown; overwriting non-UTF-8 source still checks the known replacement. Source nested too deeply for the parser or checker, and a changed file that cannot be read, report diagnostics unavailable for that file rather than passing as clean. Every check reads a file as the bytes Python runs: text a write or edit stores is checked as its UTF-8 bytes, the content it replaced included, and a file from disk as its bytes, each decoded as Python decodes source, by its coding declaration or as UTF-8. Bytes that do not decode are reported as a syntax error at the line where decoding stops, so a declaration the written text cannot satisfy is reported however the file was changed. Noqa comments are read as Ruff reads them, including after other comment directives and never inside string literals: a file-level `ruff: noqa` / `flake8: noqa` counts only on a line of its own, and a marker counts only when codes follow its colon or, with no colon, whitespace, another comment or the end of the comment follows it. Unlike Ruff, which takes the first directive in a comment, every directive in it counts. The seeded tool guide gives the syntax note's own advice. With no checkpoint or no visibility verdict, exec diagnostics do not inspect changed file contents; direct file checks remain available. Pyflakes is imported when the tool face loads, not by whichever check first needs it: imported lazily, its cold import fell inside that check's one-second wait, and on a slow Windows runner the one test that paid it reported diagnostics unavailable for a plain syntax error. Private workers bound reads and checks to four active jobs per process. Saturated capacity rejects new work without queuing, and timeout/cancellation retains each slot until the actual work finishes. Name checks are capped at 256 KiB of source and the syntax check at 2 MiB, and a check skipped for size says so in one line. Findings are capped at eight per file, and exec diagnostic text at 6,000 characters plus omission notices. Runtime changes stay inside the Raven-Code code-flow plugin, except for one fix to `CheckpointService.trackable`, the visibility verdict that exec diagnostics and the host's exec write record both rely on. On macOS, git precomposes a file name it reads off the disk before matching an ignore rule, but takes the paths `trackable` sends to `git check-ignore --stdin` as written. A file stored with a decomposed (NFD) name under a `.gitignore` rule written precomposed (NFC) is therefore left out of the checkpoint, yet main vouches for it, so the write record's diff carries its text, and exec diagnostics would quote its source lines to the model. Nor is git's precomposition NFC: it composes only neighbouring characters, by Unicode 3.2's table, and reorders nothing, so a name whose combining marks are out of canonical order comes out as neither the name as given nor its NFC. `trackable` now asks about each name as given, as NFC and in git's macOS spelling, and withholds it when any spelling is ignored. Other shared file/shell/snapshot implementations are unchanged. Raven-Code has no separate dependency manifest, so Pyflakes uses the existing root dependencies. It is the only added package; uv normalized existing lockfile markers without changing other package versions. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - **Before:** on main, Raven-Code's file tools report Python syntax errors only, and its exec reports nothing about the Python files a command changed. On macOS, main's `CheckpointService.trackable` vouches for a file stored as `secret_cafe` + U+0301 + `.py` under the rule `secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the checkpoint's staged tree leaves it out, yet its text reaches the exec write record's stored diff. - **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors over 22 test ids: 21 of them also fail on the exact base in the same environment, and the other, a remote process-tree cancel test that missed its deadline under the suite's load, passes when run alone. Tests cover actual worker overlap, retained capacity after timeout/cancellation, host-tool isolation, hidden-file quotas, visibility batching through the host's write-record wiring, baseline handling, parser-limit failures, the size bounds and their skip notes, undecodable bytes placed where decoding stops, coding declarations in written and edited text, the baseline read staying on a worker, the checker loaded with its module, the file each unavailable note names, comment directives and their marker boundary, the tool guide's agreement with the syntax note, portable commands in paths with spaces/non-ASCII characters, and the checkpoint's verdict on decomposed names and on names git spells as neither given nor NFC, held against its own staged tree. Removing any of the three spellings from the query fails the cases that pin it, as does removing the marker boundary, the parser-limit report, the unknown-baseline handling, the syntax check past the name-check bound and the decoding report, checking written text as its stored bytes, the worker around the baseline read, loading the checker with its module, or the file name in an unavailable note; restoring the guide's earlier advice fails its agreement test. git's macOS spelling is what libiconv's UTF-8-MAC conversion returns, which git precomposes with: composing neighbours by Unicode 3.2's table reproduced it for all 281,928 names built from every decomposable character's decomposition (reordered, partly composed, or with a mark added), and across 784,220 further mixed names and pairs no name was spelled outside the three asked; the same held under Python 3.13 and 3.14. The decomposed-name check asserts the macOS outcome only where git precomposes names, so it also passes with `core.precomposeunicode = false` set through `GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines match Ruff 0.15.12, except the three that hold several directives in one comment. Through the real `AgentLoop`, code-flow tool face, `CodeExecutor` and checkpoint, with the working directory under `/tmp`, under `$TMPDIR` and in a path with spaces and CJK characters, exec diagnostics report created, modified and decomposed-name files, follow `working_dir`, report an undecodable file at the line where decoding stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold `.gitignore`d files, including decomposed names and names git spells as neither given nor NFC; across repeated runs on both interpreters, a file near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600 ignored files plus one visible file take 73-90 ms with the event loop held at most 20 ms. At this head CI passes the unit suite on Linux/Python 3.12 (32619 tests over four shards), and its two Windows steps pass the diagnostic files on native Windows/Python 3.13: 122 file/worker tests, and 27 exec diagnostic tests with 17 pre-existing executor tests deselected; the symlink fixture skips only where the host cannot create links. ```sh env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3 uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py uv run --no-sync lint-imports uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py ``` Native Windows CI commands: ```sh uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short ``` Ruff, format and type checks passed; all 10 import contracts kept; the commit-message, large-file and source-language checks pass over the branch. The launcher tests unset inherited search API keys to keep the expected tool list deterministic. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes does not activate checks in other agents. The change is reversible and never fixes/removes code or rolls back edits. Static diagnostics can need judgment for dynamic names, target Python versions, and project lint configuration; only explicit noqa directives are honored. Passing diagnostics do not replace tests. The `trackable` fix reaches every exec write record, not only Raven-Code: a file whose name, as given, as NFC or in git's macOS spelling, matches an ignore rule keeps its counts and loses its diff and removal body. Where git matches another of those spellings, as off macOS for a decomposed name under a precomposed rule, the checkpoint stores the file but it is still withheld; its counts remain. Exec diagnostics now require an active checkpoint. Noninteractive runs using the default interactive checkpoint policy must explicitly enable checkpoints to receive this feedback. Visibility is checked in batches of up to 256 files and stops once eight visible files are found; each batch holds the event loop while the host resolves its paths, about 130 ms for a full batch on native Windows. Each file-check wait is one second, and post-command diagnostics have a three-second wait budget; past it, or when they fail as a whole, they are dropped and the command's result is kept. These are caller budgets, not hard I/O deadlines: running OS reads and inherited synchronous host callbacks cannot be forcibly interrupted. Private worker capacity remains bounded. Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took 0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on slower hardware such a check can outlast the one-second wait and report diagnostics unavailable. Python's GIL means concurrency does not promise a CPU speedup. Windows CI exercises the diagnostics layer with the real native DirectExecutor. The existing CodeExecutor environment probe uses POSIX quoting and already fails on Windows at the base; this PR neither changes that probe nor claims end-to-end compatibility for it. Symlink setup is skipped only if the host cannot create the fixture. Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics` for no post-command checks, or `pythonSyntaxNote` for no diagnostics, under `plugins.config["code-flow"].tools`. The PR can also be reverted as a unit. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn> Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
def test_undefined_name_feedback_identifies_the_binding_and_keeps_the_edit(registry, tmp_path):
target = tmp_path / "undefined.py"
source = "def double():\n return missing_value * 2\n"
result = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
assert "F821" in result and "missing_value" in result and "line 2" in result
assert "spelling, scope, import, or assignment order" in result
assert target.read_text() == source
def test_existing_name_errors_do_not_reappear_when_unrelated_lines_move(registry, tmp_path):
target = tmp_path / "existing.py"
target.write_text("old_result = preexisting_missing\nvalue = 1\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _text(
_run(
registry.execute(
"edit_file",
{
"file_path": str(target),
"old_string": "old_result =",
"new_string": "\nold_result =",
},
)
)
)
assert "F821" not in result
result = _text(
_run(
registry.execute(
"edit_file",
{
"file_path": str(target),
"old_string": "value = 1",
"new_string": "value = newly_missing",
},
)
)
)
assert "F821" in result and "newly_missing" in result
assert "preexisting_missing" not in result
@pytest.mark.parametrize("tool_name", ["edit_file", "write_file"])
def test_existing_syntax_error_is_still_reported_after_an_unrelated_edit(registry, tmp_path, tool_name):
target = tmp_path / "still_broken.py"
source = "def f(:\n return 1\nvalue = 2\n"
_run(registry.execute("write_file", {"file_path": str(target), "content": source}))
params = {"file_path": str(target)}
if tool_name == "edit_file":
params.update(old_string="value = 2", new_string="value = 3")
else:
params["content"] = source.replace("value = 2", "value = 3")
result = _text(_run(registry.execute(tool_name, params)))
assert "Python syntax error" in result
assert "new findings compared" not in result
assert target.read_text() == source.replace("value = 2", "value = 3")
def test_fixing_syntax_does_not_label_old_name_errors_as_new(registry, tmp_path):
target = tmp_path / "syntax_fixed.py"
source = "value = old_missing\ndef f(:\n return 1\n"
_run(registry.execute("write_file", {"file_path": str(target), "content": source}))
result = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "f(:", "new_string": "f():"}))
)
assert "F821" in result and "old_missing" in result
assert "baseline is unknown" in result and "new findings compared" not in result
@pytest.mark.parametrize("source, expected", [("def f(:\n", "Python syntax error"), ("x = missing_name\n", "F821")])
def test_overwriting_non_utf8_source_still_checks_the_new_content(registry, tmp_path, source, expected):
target = tmp_path / "legacy.py"
target.write_bytes('# coding: gbk\nname = "\u4e2d"\n'.encode("gbk"))
result = _run(registry.execute("write_file", {"file_path": str(target), "content": source}))
assert result.file_change is None
assert expected in _text(result) and "baseline is unknown" in _text(result)
assert target.read_text(encoding="utf-8") == source
def test_import_notices_are_optional_and_explain_side_effects(tmp_path):
source = "import os\n"
quiet = factories.make_write_file(_ctx(tmp_path, {"enabled": True}))
result = _text(_run(quiet.execute(file_path=str(tmp_path / "quiet_import.py"), content=source)))
assert "F401" not in result
enabled = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonUnusedImports": True}))
target = tmp_path / "notice_import.py"
result = _text(_run(enabled.execute(file_path=str(target), content=source)))
assert "NOTICE" in result and "F401" in result and "side effects" in result
assert target.read_text() == source
def test_name_checks_handle_nested_scopes_and_public_exports(registry, tmp_path):
source = (
"from os import path\n__all__ = ['path']\n"
"def outer(value):\n def inner():\n return value + len([])\n return inner\n"
)
result = _text(
_run(
registry.execute(
"write_file",
{
"file_path": str(tmp_path / "scopes.py"),
"content": source,
},
)
)
)
assert "Python diagnostics" not in result
def test_name_checks_can_be_disabled_without_disabling_syntax_feedback(tmp_path):
tool = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonLint": False}))
target = tmp_path / "syntax_only.py"
assert "F821" not in _text(_run(tool.execute(file_path=str(target), content="result = missing\n")))
assert "syntax error" in _text(_run(tool.execute(file_path=str(target), content="def f(:\n")))
def test_append_checks_the_complete_file_and_suppresses_preexisting_errors(registry, tmp_path):
target = tmp_path / "append.py"
target.write_text("old = existing_missing\n")
result = _text(
_run(
registry.execute(
"write_file",
{
"file_path": str(target),
"content": "new = added_missing\n",
"mode": "append",
},
)
)
)
assert "added_missing" in result and "existing_missing" not in result
@pytest.mark.parametrize(
("source", "code"),
[
("__all__ = ['missing_export']\n", "F822"),
("value = 1\ndef f():\n print(value)\n value = 2\n", "F823"),
],
)
def test_name_feedback_covers_exports_and_use_before_local_assignment(registry, tmp_path, source, code):
target = tmp_path / "binding.py"
out = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
assert code in out
assert target.read_text() == source
@pytest.mark.parametrize(
("source", "warns"),
[
("value = injected_name # noqa\n", False),
("value = injected_name # noqa: F821, F401\n", False),
("value = injected_name # noqa: F401\n", True),
("value = injected_name # type: ignore[name-defined] # noqa: F821\n", False),
("value = injected_name # type: ignore[name-defined] # noqa: F401\n", True),
("value = injected_name # noqa: F401 # noqa: F821\n", False),
("value = injected_name # noqa: F401 # noqa\n", False),
("# ruff: noqa: F821\nvalue = injected_name\n", False),
("# flake8: noqa\nvalue = injected_name\n", False),
("value = injected_name # ruff: noqa\nother = second_name\n", True),
("value = injected_name # ruff: noqa\n", True),
("value = 1 # TODO # flake8: noqa\nother = injected_name\n", True),
("label = '# noqa'; value = injected_name\n", True),
("label = '# ruff: noqa'\nvalue = injected_name\n", True),
("value = injected_name # see https://example.com/page#noqa-section\n", True),
('# Use "# ruff: noqa" only for generated files\nvalue = injected_name\n', True),
("value = injected_name # noqa: needed for registration\n", True),
("value = injected_name # noqa#reason\n", False),
("# ruff: noqa: generated file\nvalue = injected_name\n", True),
("# ruff : noqa\nvalue = injected_name\n", False),
("# Ruff: noqa\nvalue = injected_name\n", True),
],
)
def test_explicit_noqa_comments_suppress_only_the_requested_checks(registry, tmp_path, source, warns):
target = tmp_path / "ignored.py"
out = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
assert ("WARNING" in out) is warns
def test_syntax_errors_are_not_hidden_by_noqa(registry, tmp_path):
source = "# ruff: noqa\ndef f(:\n"
out = _text(_run(registry.execute("write_file", {"file_path": str(tmp_path / "syntax.py"), "content": source})))
assert "Python syntax error" in out and "project's Python version" in out
def test_the_tool_guide_gives_the_syntax_note_s_advice():
"""The note and the seeded tool guide reach the model together, so the
guide must not tell it to fix outright what the note asks it to check
against the project's Python version first."""
from code_flow.tools.diagnostics import PythonDiagnostics
note = PythonDiagnostics().note(Path("broken.py"), "def f(:\n", before=None)
guide = " ".join((PLUGIN_DIR / "prompts" / "TOOLS_CODE.md").read_text(encoding="utf-8").split())
advice = "check the project's Python version before changing newer syntax"
assert advice in note and advice in guide
assert "fix it before moving on" not in guide
def test_the_name_checker_is_loaded_with_the_module_not_by_the_first_check():
"""A cold import of the checker inside a check's wait can outlast the wait
on a cold runner, which reported diagnostics unavailable for a file with a
plain syntax error. It is paid when the module loads instead. A fresh
interpreter, so no earlier test's check has already imported it."""
probe = (
"import sys; sys.path.insert(0, sys.argv[1]); import code_flow.tools.diagnostics; "
"print('pyflakes.checker' in sys.modules)"
)
done = subprocess.run([sys.executable, "-I", "-c", probe, str(PLUGIN_DIR)], capture_output=True, text=True)
assert done.returncode == 0, done.stderr
assert done.stdout.strip() == "True"
def test_checker_failure_does_not_turn_a_successful_write_into_a_failure(registry, tmp_path, monkeypatch):
from pyflakes import checker
def broken(*args, **kwargs):
raise RuntimeError("checker broke")
monkeypatch.setattr(checker, "Checker", broken)
target = tmp_path / "kept.py"
result = _run(registry.execute("write_file", {"file_path": str(target), "content": "value = 1\n"}))
assert "diagnostics unavailable (RuntimeError)" in _text(result)
assert result.file_change.after == target.read_text() == "value = 1\n"
assert "Successfully wrote" in _text(result)
@pytest.mark.parametrize("error", [RecursionError, MemoryError])
def test_source_the_parser_cannot_finish_reports_diagnostics_unavailable(registry, tmp_path, monkeypatch, error):
"""Python 3.12 raises RecursionError building the syntax tree of deeply
nested source (``compile`` too); that is no verdict, not a clean file."""
from code_flow.tools import diagnostics
def too_deep(*args, **kwargs):
raise error("nested too deeply")
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=too_deep))
target = tmp_path / "deep.py"
result = _run(registry.execute("write_file", {"file_path": str(target), "content": "value = 1\n"}))
assert f"diagnostics unavailable ({error.__name__})" in _text(result)
assert target.read_text() == "value = 1\n"
@pytest.mark.parametrize("error", [RecursionError, MemoryError])
def test_replaced_source_the_parser_cannot_finish_is_an_unknown_baseline(registry, tmp_path, monkeypatch, error):
from code_flow.tools import diagnostics
parse = diagnostics.ast.parse
def deep_before(source, *args, **kwargs):
if "nested_before" in source:
raise error("nested too deeply")
return parse(source, *args, **kwargs)
target = tmp_path / "replaced.py"
target.write_text("value = nested_before\n")
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=deep_before))
args = {"file_path": str(target), "content": "value = newly_missing\n"}
result = _text(_run(registry.execute("write_file", args)))
assert "newly_missing" in result and "baseline is unknown" in result
assert "new findings compared" not in result
def test_identical_rewrite_does_not_repeat_diagnostics(registry, tmp_path):
target = tmp_path / "same.py"
args = {"file_path": str(target), "content": "value = missing\n"}
assert "F821" in _text(_run(registry.execute("write_file", args)))
assert "Python diagnostics" not in _text(_run(registry.execute("write_file", args)))
def test_identical_rewrite_still_reports_a_syntax_error(registry, tmp_path):
target = tmp_path / "same_broken.py"
args = {"file_path": str(target), "content": "def f(:\n return 1\n"}
assert "Python syntax error" in _text(_run(registry.execute("write_file", args)))
result = _text(_run(registry.execute("write_file", args)))
assert result.startswith("File unchanged:") and "Python syntax error" in result
def test_diagnostics_bound_source_and_findings_without_executing_code(tmp_path):
from code_flow.tools.diagnostics import MAX_FINDINGS, MAX_SOURCE_BYTES, MAX_SYNTAX_BYTES, PythonDiagnostics
target = tmp_path / "source.py"
sentinel = tmp_path / "must_not_exist"
text = f"open({str(sentinel)!r}, 'w').close()\n"
diagnostics = PythonDiagnostics()
assert diagnostics.note(target, text, before=None) == ""
assert not sentinel.exists()
assert diagnostics.note(target, "#" * (MAX_SOURCE_BYTES + 1), before=None) == (
f"\nPython name checks skipped for {target}: the source is over 256 KiB, so only its syntax was checked."
)
assert diagnostics.note(target, "#" * (MAX_SYNTAX_BYTES + 1), before=None) == (
f"\nPython diagnostics skipped for {target}: the source is over 2 MiB. Run targeted project checks."
)
assert PythonDiagnostics(lint=False).note(target, "#" * (MAX_SOURCE_BYTES + 1), before=None) == ""
note = diagnostics.note(target, "\n".join(f"value_{i} = missing_{i}" for i in range(20)), before=None)
assert note.count("WARNING:") == MAX_FINDINGS
assert "12 more findings omitted" in note
@pytest.mark.parametrize("tool", ["write_file", "edit_file"])
def test_a_syntax_error_past_the_name_check_bound_is_still_reported(registry, tmp_path, tool):
"""Name checks stop at 256 KiB and the syntax check runs on past it, so an
edit that breaks one of a project's large modules still hears about it."""
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES
target = tmp_path / "large.py"
rows = MAX_SOURCE_BYTES // len("value = 1\n") + 1
body = "value = 1\n" * rows
if tool == "write_file":
args = {"file_path": str(target), "content": body + "def broken(:\n"}
else:
clean = {"file_path": str(target), "content": body + "def kept():\n pass\n"}
assert "name checks skipped" in _text(_run(registry.execute("write_file", clean)))
args = {"file_path": str(target), "old_string": "def kept():", "new_string": "def broken(:"}
result = _text(_run(registry.execute(tool, args)))
assert f"WARNING: line {rows + 1}, column 12: Python syntax error" in result
assert "name checks skipped" not in result
def test_an_append_python_cannot_decode_is_reported_where_decoding_stops(registry, tmp_path):
"""UTF-8 appended to a file that declares GBK: Python refuses the file, so
the append hears that, at the line Python cannot decode."""
target = tmp_path / "legacy.py"
target.write_bytes('# coding: gbk\nname = "\u4e2d"\n'.encode("gbk"))
args = {"file_path": str(target), "content": 'label = "\u4e2d"\ndef broken(:\n', "mode": "append"}
result = _text(_run(registry.execute("write_file", args)))
assert "line 3, column 12: Python syntax error: 'gbk' codec can't decode byte 0xad" in result
assert "new findings compared with the content before this call" in result
@pytest.mark.parametrize(
("raw", "place", "message"),
[
# A declared encoding, where Python's parser reports line 0.
(
"# -*- coding: gbk -*-\nx = 1\nname = '\u4e2d\u6587'\n".encode(),
"line 3, column 11",
"'gbk' codec can't decode",
),
# UTF-8 by default, a comment included, which Python 3.12's parser passes.
(b"a = 1\nb = 2\n# caf\xe9\n", "line 3, column 6", "'utf-8' codec can't decode byte 0xe9"),
# After a BOM, which its codec strips before decoding the rest.
(b"\xef\xbb\xbfa = 1\nb = '\xff'\n", "line 2, column 6", "'utf-8' codec can't decode byte 0xff"),
# The two lines a declaration may take are read before anything is decoded.
(b"name = 'caf\xe9'\n", "line 1, column 1", "invalid or missing encoding declaration"),
(b"#!/usr/bin/env python\n# coding: nope\nx = 1\n", "line 2, column 1", "unknown encoding: nope"),
],
ids=["declared-encoding", "utf-8-comment", "after-a-bom", "first-line", "unknown-declaration"],
)
def test_bytes_python_cannot_decode_are_a_syntax_error_where_decoding_stops(tmp_path, raw, place, message):
from code_flow.tools.diagnostics import PythonDiagnostics
target = tmp_path / "undecodable.py"
target.write_bytes(raw)
note = _run(PythonDiagnostics().check(target, before=None))
assert f"WARNING: {place}: Python syntax error: {message}" in note
def test_a_finding_already_there_stays_existing_under_a_coding_declaration(registry, tmp_path):
"""The content an edit replaced is decoded as the content it wrote is, so
a finding on a line the declaration decodes the same way both times is
an existing finding, not a new one."""
target = tmp_path / "declared.py"
body = "# coding: latin-1\nlabel = 'caf\u00e9' + missing_one\nvalue = 1\n"
assert "missing_one" in _text(_run(registry.execute("write_file", {"file_path": str(target), "content": body})))
args = {"file_path": str(target), "old_string": "value = 1", "new_string": "value = 2"}
edited = _text(_run(registry.execute("edit_file", args)))
assert "Successfully edited" in edited and "missing_one" not in edited
def test_replaced_text_over_the_name_check_bound_in_bytes_is_an_unknown_baseline(tmp_path):
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES, PythonDiagnostics
# Fewer characters than the bound, more bytes once stored.
before = "# " + "\u00e9" * (MAX_SOURCE_BYTES // 2)
assert len(before) <= MAX_SOURCE_BYTES < len(before.encode("utf-8"))
note = PythonDiagnostics().note(tmp_path / "wide.py", "value = missing_name\n", before=before)
assert "missing_name" in note and "baseline is unknown" in note
def test_text_is_checked_as_the_utf8_bytes_the_file_tool_stores(tmp_path):
"""Each of these compiles when the string alone is parsed and not as the
bytes on disk, which is what Python runs."""
from code_flow.tools.diagnostics import PythonDiagnostics
text = "# coding: ascii\nname = 'caf\u00e9'\n"
note = _run(PythonDiagnostics().check(tmp_path / "declared.py", text=text, before=None))
assert "line 2, column 12: Python syntax error: 'ascii' codec" in note
declared = "# coding: latin-1\nname = 'caf\u00e9'\nvalue = missing_name\n"
note = _run(PythonDiagnostics().check(tmp_path / "latin.py", text=declared, before=None))
assert "line 3, column 9: F821" in note
def test_a_checked_file_is_decoded_by_its_declaration_and_read_to_one_past_the_bound(tmp_path):
from code_flow.tools.diagnostics import MAX_SYNTAX_BYTES, PythonDiagnostics, read_source
target = tmp_path / "encoded.py"
target.write_bytes(b"# coding: latin-1\nname = 'caf\xe9'\nvalue = missing_name\n")
assert "WARNING: line 3, column 9: F821" in _run(PythonDiagnostics().check(target, before=None))
target.write_bytes(b"#" * (MAX_SYNTAX_BYTES + 5))
assert len(read_source(target)) == MAX_SYNTAX_BYTES + 1
assert "diagnostics skipped" in _run(PythonDiagnostics().check(target, before=None))
for absent in (tmp_path / "absent.py", tmp_path):
with pytest.raises(FileNotFoundError):
read_source(absent)
def test_a_baseline_is_read_as_bytes_and_bounded(tmp_path):
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES, UNKNOWN, read_python
target = tmp_path / "encoded.py"
assert read_python(target) is None
raw = b"# coding: latin-1\nname = 'caf\xe9'\n"
target.write_bytes(raw)
assert read_python(target) == raw
target.write_bytes(b"x" * (MAX_SOURCE_BYTES + 1))
assert read_python(target) is UNKNOWN
assert read_python(tmp_path) is UNKNOWN
@pytest.mark.asyncio
async def test_a_baseline_read_runs_on_a_worker_and_gives_up_at_the_wait(tmp_path, monkeypatch):
"""The content an append replaces is read as every diagnostic read is, on
a diagnostic worker: a stalled read leaves the event loop free and the
baseline unknown once the wait runs out."""
import time
from code_flow.tools import diagnostics
release = threading.Event()
readers = []
def stalled(path):
readers.append(threading.get_ident())
release.wait(5)
return b""
monkeypatch.setattr(diagnostics, "read_python", stalled)
monkeypatch.setattr(diagnostics, "CHECK_WAIT_SECONDS", 0.05)
try:
started = time.monotonic()
assert await diagnostics.read_before(tmp_path / "slow.py") is diagnostics.UNKNOWN
assert time.monotonic() - started < 1
assert readers and readers[0] != threading.get_ident()
finally:
release.set()
@pytest.mark.parametrize("tool", ["write_file", "edit_file"])
def test_a_coding_declaration_applies_to_the_text_a_write_or_edit_stores(registry, tmp_path, tool):
"""The file tool stores its text as UTF-8, so a declaration in that text
governs the file Python reads: content the declaration cannot decode is
reported, as it is for the same bytes read back after an append."""
target = tmp_path / "declared.py"
body = "# coding: ascii\nname = 'x'\n"
if tool == "write_file":
args = {"file_path": str(target), "content": body.replace("'x'", "'caf\u00e9'")}
else:
assert "Successfully" in _text(
_run(registry.execute("write_file", {"file_path": str(target), "content": body}))
)
args = {"file_path": str(target), "old_string": "'x'", "new_string": "'caf\u00e9'"}
result = _text(_run(registry.execute(tool, args)))
assert "line 2, column 12: Python syntax error: 'ascii' codec can't decode byte 0xc3" in result
with pytest.raises(SyntaxError):
compile(target.read_bytes(), str(target), "exec")
def test_an_unavailable_note_names_its_file(tmp_path, monkeypatch):
"""Post-command checks append one note per changed file, so each
unavailable note says which file it is about, from a failed read and a
failed parse alike."""
from code_flow.tools import diagnostics
unreadable = tmp_path / "locked.py"
unreadable.write_text("value = 1\n")
deep = tmp_path / "deep.py"
deep.write_text("value = 1\n")
read = diagnostics.read_source
def refused(path):
if path == unreadable:
raise PermissionError(f"permission denied: {path}")
return read(path)
def too_deep(*args, **kwargs):
raise RecursionError("nested too deeply")
monkeypatch.setattr(diagnostics, "read_source", refused)
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=too_deep))
read_note = _run(diagnostics.PythonDiagnostics().check(unreadable))
parse_note = _run(diagnostics.PythonDiagnostics().check(deep))
assert f"unavailable (PermissionError) for {unreadable};" in read_note
assert f"unavailable (RecursionError) for {deep};" in parse_note
@pytest.mark.asyncio
async def test_diagnostic_wait_is_bounded_and_cancellation_is_preserved(tmp_path, monkeypatch):
from code_flow.tools import diagnostics
from code_flow.tools.diagnostic_workers import DiagnosticWorkers
release = threading.Event()
def stalled(*args, **kwargs):
release.wait(2)
return ""
monkeypatch.setattr(diagnostics, "WORKERS", DiagnosticWorkers())
monkeypatch.setattr(diagnostics, "CHECK_WAIT_SECONDS", 0.01)
monkeypatch.setattr(diagnostics.PythonDiagnostics, "note", stalled)
try:
note = await diagnostics.PythonDiagnostics().check(tmp_path / "slow.py", text="value = 1\n")
assert "diagnostics unavailable (TimeoutError)" in note
task = asyncio.create_task(diagnostics.PythonDiagnostics().check(tmp_path / "cancelled.py", text="value = 1\n"))
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
finally:
release.set()
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
# --- occurrence -----------------------------------------------------------------------
def test_occurrence_replaces_exactly_the_nth_match(registry, tmp_path):
target = tmp_path / "i.txt"
target.write_text("a\na\na\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _run(
registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 2})
)
assert "occurrence 2 of 3" in _text(result) and "line 2" in _text(result)
assert target.read_text() == "a\nb\na\n"
direct = _run(
registry.get("edit_file").execute(file_path=str(target), old_string="a", new_string="c", occurrence=2)
)
assert isinstance(direct, ToolResult) and direct.file_change is not None and direct.diff
assert target.read_text() == "a\nb\nc\n"
@pytest.mark.parametrize(
("content", "old", "occurrence", "expected"),
[
("aaaa bb aa", "aa", 1, "Xaa bb aa"),
("aaaa bb aa", "aa", 2, "aaX bb aa"),
("aaaa bb aa", "aa", 3, "aaaa bb X"),
("ababa aba", "aba", 2, "ababa X"),
("aa\r\naa\r\n", "aa\r\n", 2, "aa\r\nX"),
],
)
def test_occurrence_counts_non_overlapping_matches(registry, tmp_path, content, old, occurrence, expected):
target = tmp_path / "matches.txt"
target.write_bytes(content.encode())
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _run(
registry.execute(
"edit_file",
{"file_path": str(target), "old_string": old, "new_string": "X", "occurrence": occurrence},
)
)
assert not _text(result).startswith("Error")
assert target.read_bytes() == expected.encode()
def test_occurrence_description_states_its_matching_rules(registry):
tool = registry.get("edit_file")
description = tool.parameters["properties"]["occurrence"]["description"]
assert "non-overlapping" in description and "1-based" in description
assert "exact" in description and "replace_all" in description
assert "non-overlapping" in tool.description
def test_occurrence_past_the_count_and_with_replace_all_are_refused(registry, tmp_path):
target = tmp_path / "j.txt"
target.write_text("a\na\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
too_far = _text(
_run(
registry.execute(
"edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 3}
)
)
)
assert too_far.startswith("Error") and "matches only 2" in too_far
both = _text(
_run(
registry.execute(
"edit_file",
{"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 1, "replace_all": True},
)
)
)
assert both.startswith("Error") and "mutually exclusive" in both
assert target.read_text() == "a\na\n"
def test_several_matches_without_a_selector_are_still_refused(registry, tmp_path):
target = tmp_path / "k.txt"
target.write_text("a\na\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b"}))
)
assert "appears 2 times" in result and "old_string" in result and "old_text" not in result
assert target.read_text() == "a\na\n"
# --- glob ------------------------------------------------------------------------------
def test_glob_finds_by_pattern_and_expands_a_brace_alternation(tmp_path):
(tmp_path / "one.py").write_text("")
(tmp_path / "two.txt").write_text("")
(tmp_path / "three.md").write_text("")
tool = GlobTool(workspace=tmp_path)
with workdir.bind(tmp_path):
py = _text(_run(tool.execute(pattern="*.py")))
assert "one.py" in py and "two.txt" not in py
both = _text(_run(tool.execute(pattern="*.{py,txt}")))
assert "one.py" in both and "two.txt" in both and "three.md" not in both
def test_brace_expansion_rules():
assert expand_braces("*.{a,b}") == ["*.a", "*.b"]
assert expand_braces("src/{a,b}/*.{ts,tsx}") == ["src/a/*.ts", "src/a/*.tsx", "src/b/*.ts", "src/b/*.tsx"]
assert expand_braces("*.py") == ["*.py"]
# --- the replacement in the registry -----------------------------------------------------
def test_a_plugin_tool_registered_after_the_builtin_replaces_it(tmp_path):
reg = ToolRegistry()
reg.register(TrunkReadFileTool(workspace=tmp_path))
assert reg.get("read_file").parameters["required"] == ["path"]
reg.register(ReadFileTool(workspace=tmp_path))
assert reg.get("read_file").parameters["required"] == ["file_path"]
assert isinstance(reg.get("read_file"), ReadFileTool)
def test_the_ledger_is_keyed_by_resolved_path(tmp_path):
ledger = ReadLedger()
target = tmp_path / "m.txt"
target.write_text("z\n")
read = ReadFileTool(workspace=tmp_path, ledger=ledger)
edit = EditFileTool(workspace=tmp_path, ledger=ledger)
with workdir.bind(tmp_path):
_run(read.execute(file_path="m.txt"))
assert ledger.has_read(target)
assert "Successfully edited" in _text(_run(edit.execute(file_path=str(target), old_string="z", new_string="w")))
def test_write_file_still_reports_the_diff_the_ui_renders(tmp_path):
tool = WriteFileTool(workspace=tmp_path)
target = tmp_path / "n.py"
target.write_text("a = 1\n")
result = _run(tool.execute(file_path=str(target), content="a = 2\n"))
assert isinstance(result, ToolResult) and result.file_change is not None
def _git(cwd: Path, *args: str) -> None:
subprocess.run(["git", "-C", str(cwd), *args], check=True, capture_output=True)
# --- what the adversarial audit of 2026-09-08 found, each pinned -------------
def test_an_occurrence_edit_keeps_the_files_own_line_endings(registry, tmp_path):
"""A CRLF file edited through ``occurrence`` used to come back with every
line changed: the path read and wrote through text mode, which normalises
to LF, so a one-line edit arrived as a whole-file diff (and a grader diffing
the working tree sees a rewrite). Trunk's own edit path reads and writes
bytes for exactly this reason; so does this one now."""
target = tmp_path / "crlf.txt"
target.write_bytes(b"a\r\nb\r\na\r\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
result = _run(
registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "Z", "occurrence": 2})
)
assert "occurrence 2 of 2" in _text(result)
assert target.read_bytes() == b"a\r\nb\r\nZ\r\n"
def test_an_error_report_quotes_the_file_verbatim(registry, tmp_path):
"""The respelling of trunk's parameter names must not reach quoted file
content. It used to be a blanket replacement over the whole result, so a
file that really contains ``old_text`` was shown to the model as containing
``old_string`` -- and the model then copied an excerpt that could never
match anything."""
target = tmp_path / "tool.py"
target.write_text("def execute(self, path, old_text, new_text):\n value = old_text\n return path\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
near_miss = "def execute(self, path, old_text, new_text):\n value = old_text\n return other\n"
report = _text(
_run(
registry.execute(
"edit_file",
{"file_path": str(target), "old_string": near_miss, "new_string": "x"},
)
)
)
sentence, _, excerpt = report.partition("\n")
assert sentence.startswith("Error") and "old_string" in sentence, "the sentence names the parameter shown"
assert "old_text" in excerpt, "the quoted file keeps its own spelling"
assert "old_string" not in excerpt, "and nothing rewrites the file's text"
def test_a_long_line_is_cut_the_way_the_description_promises(registry, tmp_path):
"""Trunk numbers a line whole, so one minified line can overrun the
whole-result budget by itself: the read came back with NO body and a note
telling the model to continue from the same offset, which it can do
forever. The description promised truncation; now it happens."""
target = tmp_path / "bundle.js"
target.write_text("x" * 200_000 + "\nlast\n")
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
assert "line truncated to 2000 chars" in body
assert "1| " in body and "2| last" in body
fix(*): show the head of a line longer than a read_file page instead of an empty page (#959) ## Summary A page of `read_file` is cut to whole lines within its 128k-character budget. A file whose first line is over the budget by itself (a minified script, a one-line JSON, a long CSV row) left nothing within it: the page came back empty, and its footer said to continue at the very line it had just not shown, so the reader went round for ever. Measured before the change: `(Showing lines 1-0 of 1. Use offset=1 to continue.)` for a single 200k-character line, and `(Showing lines 2-1 of 3. Use offset=2 to continue.)` once a shorter first line had filled page one. Such a line is now shown by its head, as much as the budget holds, followed by a note that says how long the line is, how much of it is shown, and that `offset` moves by lines and cannot reach the rest: `grep` tells whether a piece is in it, and a shell command that cuts by bytes (such as `cut -c`) shows another stretch. The paging footer then points at the next line, so paging moves on. A first line that just fits (the line break the budget counts put it over) is shown whole, without a note. Pages that fit are unchanged. The node runs the same tool for a file on a registered machine, so the remote read reads the same; one of the new tests goes through the real node behind the stand-in ssh. Raven-Code's `read_file` (the code-flow plugin) cuts every line to 2000 characters. It had its own way round the empty page: whenever trunk answered with no numbered line it re-read the whole file itself, and otherwise it cut the lines after trunk had counted the page, so a dozen 10k-character lines filled a page. Trunk's read now offers a hook, `_cut_line`, applied to each line before the page's budget counts it (trunk's own is a no-op); the plugin cuts on it, so minified lines share a page, each cut once, and its whole-file re-read is removed. Its cut after the read now runs only for a page a registered machine's node answered (the node runs trunk's tool, without the plugin's hook); a page read here was cut on the hook and is not looked at again. The first head told a cut line apart by its ending, so a line whose own text ends like the mark came back whole (both reviewers reproduced it); whether a line was cut is now decided by where the read ran, never by its text. Changed: `raven/agent/tools/filesystem.py` (the trim loop and the hook in `ReadFileTool`), `agents/raven-code/plugins/code-flow/code_flow/tools/filesystem.py` (the plugin's read), tests in `tests/test_filesystem_tools.py`, `tests/test_remote_file_tools.py` and `tests/test_agents_code_tools_plugin.py`. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - The six new tests fail on `origin/main` (the four about the page with the empty pages quoted above; the two about the hook with a dozen lines per page) and pass after the change: `pytest tests/test_filesystem_tools.py tests/test_remote_file_tools.py tests/test_agents_code_tools_plugin.py -k "longer_than_a_page or just_fills_the_page or going_round or cut_per_line or minified_lines"`. - Second commit: `test_a_line_whose_own_text_ends_like_the_mark_is_cut_like_any_other` fails on the first head (the line comes back whole) and passes now; `test_a_page_a_registered_machine_s_node_answered_is_cut_after_the_read` pins the pass after the read for a page a node answered, through a stand-in pool. - `pytest tests/test_agents_code_tools_plugin.py tests/test_filesystem_tools.py tests/test_remote_file_tools.py tests/test_node_server.py tests/test_read_file_image.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py -n 4`: 356 passed. - `scripts/coverage_gate.py diff --base-ref origin/main`: 100% of the 11 changed executable lines. - `ruff check`, `ruff format --check`, `ty check` on both changed modules: clean. - `scripts/check_commit_messages.py`, `scripts/check_source_language.py`, `scripts/check_large_files.py` over `origin/main..HEAD`: clean. - Full suite (`pytest -n 6`), run again at the second commit: 31623 passed, 233 skipped; the 3 failures are the ones main has on this machine (a LibreOffice-less host, two simulation cases), unrelated to this change. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed (checked CONTEXT.md, docs-site and the plugin's own text: nothing describes the page footer or the plugin's cut; both tools' descriptions still hold) ## Risk - [x] Security impact considered: nothing new is read, and the note names no path. - [x] Backward compatibility considered: a page that fits is unchanged, and the existing paging tests still hold; the hook is a no-op unless a subclass overrides it. In Raven-Code the page of a file with minified lines holds more lines than before (the cut lands before the budget) and the one mark reads as it did. - [x] Rollback path is clear for risky changes: revert the one commit. ## Related Issues N/A --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-15 21:05:17 +08:00
def test_minified_lines_are_cut_before_the_page_is_counted_so_they_share_one_page(registry, tmp_path):
"""Fifty lines of 10k characters: cut after trunk's page, a dozen made a
page; cut on trunk's hook, before its budget counts them, they all do, each
cut once (the cut after the read is only for a page a registered
machine's node answered)."""
target = tmp_path / "chunks.js"
target.write_text("\n".join("x" * 10_000 for _ in range(50)) + "\n")
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
lines = body.split("\n\n")[0].split("\n")
assert len(lines) == 50 and lines[0].startswith("1| ") and lines[-1].startswith("50| ")
assert lines[0] == "1| " + "x" * 2000 + "... (line truncated to 2000 chars)", "cut once, at the budget"
assert body.endswith("(End of file \u2014 50 lines total)")
assert "(Line 1 is" not in body
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
def test_a_read_that_failed_does_not_license_an_edit(registry, tmp_path):
"""A model probing for a file that does not exist yet used to have that
failed read recorded as having seen it; once something else created the
file, the edit went straight through on content nobody had read."""
target = tmp_path / "later.txt"
missing = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
assert missing.startswith("Error")
target.write_text("created by something else\n")
refused = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "created", "new_string": "x"}))
)
assert refused.startswith("Error") and "have not read" in refused
def test_a_file_that_changed_since_the_read_is_refused_as_stale(registry, tmp_path):
"""The fork's third state: a formatter, a build step or a checkout between
the read and the edit means the text being edited is no longer there."""
import os
target = tmp_path / "moving.txt"
target.write_text("one\n")
_run(registry.execute("read_file", {"file_path": str(target)}))
target.write_text("two\n")
os.utime(target, (1_000_000, 1_000_000))
stale = _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "two", "new_string": "three"}))
)
assert stale.startswith("Error") and "changed since you last read it" in stale
_run(registry.execute("read_file", {"file_path": str(target)}))
assert "Successfully edited" in _text(
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "two", "new_string": "three"}))
)
def test_the_write_ledger_does_not_depend_on_the_syntax_verdict(tmp_path):
"""Two separate knobs: turning the Python verdict off used to leave the
model unable to edit a file it had just written, because the ledger entry
sat after the verdict's early return."""
ctx = _ctx(tmp_path, {"enabled": True, "pythonSyntaxNote": False})
write = factories.make_write_file(ctx)
edit = factories.make_edit_file(ctx)
target = tmp_path / "quiet.py"
assert "Successfully wrote" in _text(_run(write.execute(file_path=str(target), content="x = 1\n")))
assert "Successfully edited" in _text(_run(edit.execute(file_path=str(target), old_string="1", new_string="2")))
def test_the_workspace_fence_follows_the_products_answer(tmp_path):
"""These tools replace the host's own by name, and the host fences its own
on ``tools.restrictToWorkspace``. Passing no ``allowed_dirs`` meant the
fence was OFF whatever the product asked for -- half the face fenced, half
not, with no warning. The launcher renders the answer into the slice."""
outside = tmp_path / "outside.txt"
outside.write_text("secret\n")
inside = tmp_path / "ws"
inside.mkdir()
unfenced = factories.make_read_file(_ctx(inside, {"enabled": True}))
assert "secret" in _text(_run(unfenced.execute(file_path=str(outside))))
fenced = factories.make_read_file(_ctx(inside, {"enabled": True, "restrictToWorkspace": True}))
refused = _text(_run(fenced.execute(file_path=str(outside))))
assert refused.startswith("Error") and "outside allowed" in refused
def test_a_zero_or_negative_occurrence_is_refused_in_code(tmp_path):
"""The schema's ``minimum`` stops the model, but a caller reaching the tool
directly used to get a write at a negative index reported as success."""
tool = factories.make_edit_file(_ctx(tmp_path, {"enabled": True, "requireReadBeforeEdit": False}))
target = tmp_path / "guard.txt"
target.write_text("aaa\n")
for bad in (0, -1):
refused = _text(_run(tool.execute(file_path=str(target), old_string="a", new_string="Z", occurrence=bad)))
assert refused.startswith("Error") and "1 or greater" in refused
assert target.read_text() == "aaa\n"
# --- glob: the merge that used to paste the engine's notices into the listing --
def test_glob_merges_only_paths_and_honours_the_callers_limit(tmp_path):
"""Every alternative used to spend the caller's ``limit`` again, and each
engine notice ("No files found…", "(showing first N of M results)") was
pasted into the listing as though it were a filename -- which the model
then tried to read."""
for name in ("a.py", "b.py", "c.ts", "d.ts"):
(tmp_path / name).write_text("")
tool = GlobTool(workspace=tmp_path)
with workdir.bind(tmp_path):
both = _text(_run(tool.execute(pattern="*.{py,ts}")))
assert sorted(both.splitlines()) == ["a.py", "b.py", "c.ts", "d.ts"]
missing = _text(_run(tool.execute(pattern="*.{py,zzz}")))
assert sorted(missing.splitlines()) == ["a.py", "b.py"], "a barren alternative adds no line"
capped = _text(_run(tool.execute(pattern="*.{py,ts}", limit=3)))
lines = capped.splitlines()
assert len(lines) == 4 and len([ln for ln in lines if is_notice(ln)]) == 1
assert len([ln for ln in lines if not is_notice(ln)]) == 3, "the caller's limit is the merged total"
# A single pattern is the engine's own answer, verbatim.
assert "No files found matching pattern" in _text(_run(tool.execute(pattern="*.nope")))
# A merged one is this tool's, and names the pattern it was asked for.
assert "No files found matching pattern: *.{nope,zzz}" == _text(_run(tool.execute(pattern="*.{nope,zzz}")))
def test_glob_preserves_a_child_truncation_when_other_patterns_are_empty(tmp_path):
for name in ("a.py", "b.py", "c.py"):
(tmp_path / name).write_text("")
tool = GlobTool(workspace=tmp_path)
out = _text(_run(tool.execute(pattern="*.{py,ts}", limit=1)))
assert "PARTIAL" in out and "raise limit" in out
assert len([line for line in out.splitlines() if not is_notice(line)]) == 1
fix(agent): bound every tree walk and take it off the event loop (#566) ## Summary On 2026-09-20 one `find` call froze the whole gateway for 2h21m. The model asked for `find(pattern="**/gps_trilateration/index.html", path="/Users/admin")`, and `FindTool` ran `Path.glob` synchronously on the event loop, filtered the noise directories only after walking them, and had no deadline. The registry's 300s ceiling is an `asyncio.wait_for`, which cannot preempt synchronous code, so every session and every subagent on the gateway stopped until the walk came back (`gateway.log`: `Tool result: find duration=8476519ms`). `list_dir` with `recursive=true` had the same shape through `rglob`. The fix gives the three tree-walking tools one walk, `raven.agent.tools.tree_walk`, with these bounds: - noise directories (`node_modules`, `.git`, ...) are pruned before they are entered, not filtered afterwards -- the pruning tests spy on `os.scandir` and assert the noise directory is never opened; - the walk yields one entry at a time and checks a 20s wall-clock budget (the value grep's fallback already used) before every entry, so whatever the caller does with one entry -- the `stat` per `find` match, the read per grep file -- is covered by the check on the next. The budget cannot be overrun by one large or slow directory: the probe with eight matches, a 30ms stat each and a 50ms budget returns 2 matches and the notice. When the budget is hit, `find` and `list_dir` append a `PARTIAL result: ... traversal budget ...` clause to their trailer, and a deadline that fires before any match is not reported as a clean `No files found` / `Directory is empty`; - every caller runs the walk in `asyncio.to_thread`, so the loop keeps serving other sessions while the walk runs. Cancelling the thread does not stop the worker, which is why the deadline lives inside the walk. The pure-Python `grep` fallback already had the prune and the deadline; it moves onto the shared walk and off the loop with the others. `find` matches each path component with `fnmatch.translate`, with a component that is exactly `**` spanning any number of components; the matcher is a position-set simulation (`_advance`), linear in components times segments, so a pattern carrying many `**` cannot make one match call slow. A pattern's leading literal components decide where the walk starts: `src/**/*.py` begins inside `src` and never lists a sibling of it, a pattern that is all literal (`src/lib/c.py`, `src/lib`) names exactly one path and answers without a walk, and the prefix itself is a match when the rest of the pattern can match nothing (`src/**` still lists `src/`). It answers exactly what `Path.glob` answered for the same pattern: pinned by a parametrized oracle test over twenty-seven patterns on a fixture and nineteen on a fixture with links (a directory link out of the tree, a link cycle, a file link, a noise directory beneath a link), and checked by hand against `Path.glob` over `raven/` for 27 patterns including bracket edge cases and trailing slashes, 0 mismatches. Two deliberate differences: a trailing `**` also lists files beneath its prefix (3.12's `Path.glob` listed only directories there), and a pattern with `..` or a leading `/` is refused instead of followed, since a pattern must not reach outside the directory the fence resolved. A trailing slash keeps `Path.glob`'s directory-only meaning (`*/`, `src/**/`). A symbolic link to a directory is entered where a single component names or matches it and never under `**`, which is how `Path.glob` read a pattern: `*/util/helper.py` reaches through a linked `vendor`, `**/helper.py` does not, and a link cycle is entered once per component the pattern spends on it, so `*/*/*/b.py` enters `src/loop -> src` twice and stops. The walker asks its caller per link (`follow`), `find` answers from the pattern and hands the link positions of a path to the matcher so `**` cannot claim them, and `list_dir` and the grep fallback never ask, staying as `rglob` and `os.walk` left them. A third deliberate difference from the merge base: a noise directory inside a pattern's literal prefix (`node_modules/*.js`, `src/node_modules/*.js`) is walked, since the pattern asked for it, where `Path.glob`'s filter dropped it; one met below the prefix is pruned as before (`*/node_modules/*.js` answers nothing), `list_dir` on such a path lists it, and the `find`, `list_dir` and code-flow `glob` descriptions state that rule instead of an unconditional "skipped". Review found an earlier wording of the two `find` descriptions one component short ("except one the pattern starts with"), while the oracle helper already pruned below the prefix; the sentences now say what the helper checks, and the oracle has the `src/node_modules` shape on both sides. The raven-code `glob` tool (`agents/raven-code/.../tools/search.py`) parses trunk `find` output line by line and strips the trailer before merging brace alternatives. It learns the new clause, so a cut-short child is reported as cut short rather than as a tree that lacks the files; its `is_notice` grammar covers the new line so it is never pasted into a listing as a path. The code-flow tool guide (`prompts/TOOLS_CODE.md`), the wrapper's description and its `limit` parameter told the model that a PARTIAL `glob` result means "raise `limit` or narrow the pattern"; a traversal-budget PARTIAL is now a result `glob` can return and a higher limit does nothing for it, so all three now tell the two notices apart. The seeded copy of the guide refreshes through the receipt `seed_guide` keeps beside it, so no legacy digest is added. Fixed in passing, and now pinned: recursive `list_dir` filtered `rglob` results on the components of the absolute path, so a workspace beneath a directory named `build`, `dist`, `venv` or another noise name listed as "Directory ... is empty" (reproduced at the merge base under `<tmp>/build/proj`; this head lists it). The shared walk prunes below the listed path only; `find` and the grep fallback already did. Not changed: the 20s value. It is the budget grep's fallback already ran under; this PR does not re-measure it, it makes it hold for every entry of every walk. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification At head 8dade176, on origin/main at d50a13c1 (nine commits: the walk, the grep pin, the changelog, the per-entry budget with the prefix start, the literal-directory case, the symbolic-link reading with the noise-prefix wording, the noise-name-above-the-path pin, the guide wording, and the prefix-rule wording; `git range-diff` against bb836988: all nine replayed `=` onto the moved main, which conflicted only at the changelog anchor and touches no file this PR touches): ``` uv run pytest tests/test_file_search_traversal_guard.py tests/test_agents_code_tools_plugin.py \ tests/test_search_tools.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py tests/test_filesystem_tools.py -q # 203 passed COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q --ignore=tests/integration # in a throwaway worktree # 24652 passed, 110 skipped in 382.30s (0:06:22) at head 8dade176, in a throwaway worktree with COLORTERM=truecolor uv run pytest tests/test_agents_code_launcher.py tests/test_agents_code_tools_plugin.py -q # the guide's seeding and wording tests, the glob wrapper # 119 passed make lint-python lint-imports lint-types lint-deps # all pass git diff --check origin/main...HEAD # clean scripts/check_source_language.py origin/main...HEAD # exit 0 scripts/check_large_files.py origin/main...HEAD # exit 0 git merge-tree --write-tree HEAD origin/main # clean ``` Revert-to-red, with the tests kept (101 tests in `test_file_search_traversal_guard.py` + `test_agents_code_tools_plugin.py`): ``` the three production files reverted to the previously reviewed head (fd5663ee) 19 failed file_search.py + filesystem.py reverted to origin/main, tree_walk.py removed collection error ``` Mutation, each mutant applied alone, control restored between runs: ``` deadline checked once per directory (the reviewed placement) 10 failed walk starts at the root instead of the pattern's prefix 2 failed trailing slash matches files too 4 failed a pattern naming one file still walks 1 failed the prefix itself is never a match 7 failed a pattern naming one directory walks beneath it 1 failed control (nothing mutated) 101 passed ``` From the first review round (still holding at this head): find without `to_thread` 1 failed; list_dir without `to_thread` 1 failed; grep fallback without `to_thread` 1 failed; pruning removed 6 failed; fork drops the PARTIAL clause 1 failed. Fourth round, the symbolic-link reading (`test_file_search_traversal_guard.py` + `test_agents_code_tools_plugin.py`, 128 tests, `-n 0`; a mutant that lets `**` sweep a link, or forgets the link positions, re-enters the cycle until the OS refuses the path and answers paths `Path.glob` never did): ``` baseline 128 passed MUTANT ** may sweep a link 8 failed (7 oracle cases, and the link the pattern ends at is opened) MUTANT find never asks to enter a link 6 failed (the six oracle patterns that pass through a link) MUTANT a link is entered even when the pattern ends at it 1 failed (scandir spy on **/vendor) MUTANT ancestor links forgotten (links_of empty) 1 failed (a cycle case answers a path Path.glob never did) MUTANT the walker consults follow but never enters 7 failed MUTANT each followed root gets a fresh deadline 9 failed (every deadline test, and the walker's one-budget test) REVERT production files to reviewed head d67184c8 7 failed CONTROL restored 128 passed ``` Revert-to-red for the noise-name pin: with `raven/agent/tools/filesystem.py` at the merge base and the test kept, `test_a_noise_name_above_the_searched_path_hides_nothing` fails on the `Directory ... is empty` answer (1 failed); at this head it passes. The `agents/**` change is two prose strings and a guide line, outside the coverage gate's `raven/**` and `plugins-dist/**` pathspecs; the guide's name-list test and its seeding tests (`test_agents_code_launcher.py`) cover the file. The prefix rule is pinned by mutation, the other policy (prune a noise directory inside the prefix) against the two guard files (131 tests, `-n 0`): ``` baseline 131 passed MUTANT a noise directory inside the prefix is pruned 5 failed test_find_matches_what_path_glob_matched[node_modules/**/*.py], test_find_matches_what_path_glob_matched[node_modules/*/], test_find_matches_what_path_glob_matched[node_modules/*/d.py], test_find_matches_what_path_glob_matched[node_modules/x/d.py], test_find_matches_what_path_glob_matched[src/node_modules/*.py] CONTROL restored 131 passed ``` The symlink oracle is the reviewer's own test shape over a fixture with links: `vendor -> outside/real`, `src/loop -> src`, `alias.py -> a.py`, a `node_modules` beneath the link; 19 patterns, 0 mismatches against `Path.glob` on 3.12, plus a scandir spy pinning that `**/vendor` never opens the link, a walker test pinning that the default walk yields a link and does not enter it while a `follow`ed link is walked after the tree under one deadline, and a `list_dir` and grep test pinning `rglob`/`os.walk` parity through a link. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed (CHANGELOG `Unreleased / Fixed` entry) ## Risk User-visible behaviour changes, all in tool output the model reads: - `find` and `list_dir` (recursive) can now return a partial listing with a `PARTIAL result` clause in the trailer where they previously either returned everything or hung. The clause tells the model the absence of a match is not conclusive and to narrow the search. - A path-prefixed `find` pattern spends its budget only inside its prefix, so on a large tree it finds what a root-anchored walk used to miss. - `find` refuses patterns containing `..` or starting with `/` with an `Error running find:` line; `Path.glob` used to raise on the second and could climb on the first. - `find` with a trailing `**` lists files as well as directories beneath the prefix; a trailing `/` lists directories only, as before. - `find` reaches through a symbolic link to a directory where a literal or single-wildcard component names or matches it, as it did at the merge base; the earlier revisions of this PR had lost that and returned a clean miss with no notice. `**` never enters a link, as before. - A noise directory inside a `find` pattern's literal prefix, or named as the `list_dir` path, is walked: `find('node_modules/*.js')` and `find('src/node_modules/*.js')` answer its files and `list_dir('node_modules', recursive=true)` lists it, where the merge base answered nothing and "empty". One met below the prefix is pruned as before. Both descriptions state the rule. - Recursive `list_dir` on a workspace beneath a directory named like a noise directory (`/tmp/build/proj`) lists it instead of answering "is empty". - The code-flow `glob` description, its `limit` parameter and the tool guide tell a limit cut from a traversal-budget PARTIAL; the notices themselves are unchanged. - `list_dir` flat listings are unchanged; the empty-directory message is unchanged. Rollback: revert the squash commit. The new module has no other consumers, and no state or schema is involved. - [x] Security impact considered (a pattern can no longer climb out of the fenced directory via `..`; a directory link is entered exactly where `Path.glob` entered it at the merge base, one non-`**` component, and never under `**`, so the fence's reach is what it was and not wider; `list_dir` and grep never enter one) - [x] Backward compatibility considered (the fork's trailer grammar is extended, not changed; existing fork tests pass unchanged) - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 10:59:01 +08:00
def test_glob_carries_a_childs_traversal_budget_into_the_merged_notice(tmp_path, monkeypatch):
"""When the engine's walk hits its deadline the child answer ends in a
PARTIAL trailer, sometimes above an otherwise clean miss. Neither line is a
path, and the merged listing has to say the search was cut short rather
than report a tree that lacks the files."""
from raven.agent.tools import tree_walk
(tmp_path / "a.py").write_text("")
tool = GlobTool(workspace=tmp_path)
monkeypatch.setattr(tree_walk, "WALK_DEADLINE_S", -1.0)
merged = _text(_run(tool.execute(pattern="*.{py,ts}")))
assert merged.startswith("No files found matching pattern: *.{py,ts}")
assert "traversal budget" in merged
assert all(is_notice(line) for line in merged.splitlines() if line)
# Each child walk reads the clock four times over this tree: its
# deadline, then before ``late/`` and ``a.py`` at the root (in budget),
# then before ``late/b.ts`` (past it).
monkeypatch.setattr(tree_walk, "WALK_DEADLINE_S", 20.0)
ticks = itertools.cycle([0.0, 0.0, 0.0, tree_walk.WALK_DEADLINE_S + 1])
monkeypatch.setattr(tree_walk, "time", SimpleNamespace(monotonic=lambda: next(ticks)))
(tmp_path / "late").mkdir()
(tmp_path / "late" / "b.ts").write_text("")
partial = _text(_run(tool.execute(pattern="*.{py,ts}")))
lines = partial.splitlines()
assert [line for line in lines if not is_notice(line)] == ["a.py"]
assert "PARTIAL result" in lines[-1] and "traversal budget" in lines[-1]
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer ## Summary An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight. - **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before. - **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client. - **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`. - **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together. What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy. ## Review round 1 (2026-09-09, on !600, superseded by this MR) - **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text. - Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it. ## Review round 2 (2026-09-10) - **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed. ## Review round 3 (2026-09-10) - **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed. - **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514). - Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row). - Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12. ## Review round 4 (2026-09-10) - **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \ tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \ tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \ tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean uv run lint-imports # 9 kept, 0 broken cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean ``` New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded). Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row) ## Risk - [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive. - [x] Rollback path is clear: revert the two commits. A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites. ## Related Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
@pytest.mark.parametrize("kind", ["missing", "not_directory", "outside"])
def test_glob_preserves_child_search_errors(tmp_path, kind):
root = tmp_path / "root"
root.mkdir()
target = tmp_path / "outside" if kind == "outside" else root / kind
if kind == "not_directory":
target.write_text("file")
elif kind == "outside":
target.mkdir()
tool = GlobTool(workspace=root, allowed_dirs=(root,))
single = _text(_run(tool.execute(pattern="*.py", path=str(target))))
merged = _text(_run(tool.execute(pattern="*.{py,ts}", path=str(target))))
assert single.startswith("Error:") and merged == single
def test_glob_merges_by_recency_before_applying_the_total_limit(tmp_path):
for timestamp, name in enumerate(("old.py", "middle.py", "new.ts"), start=100):
path = tmp_path / name
path.write_text("")
os.utime(path, (timestamp, timestamp))
tool = GlobTool(workspace=tmp_path)
out = _text(_run(tool.execute(pattern="*.{py,ts}", limit=2)))
assert out.splitlines()[:2] == ["new.ts", "middle.py"]
assert "PARTIAL" in out
def test_glob_keeps_paths_that_look_like_notice_prefixes_and_deduplicates(tmp_path):
names = ["ErrorHandler.py", "No files found matching pattern.py", "other.py"]
for name in names:
(tmp_path / name).write_text("")
tool = GlobTool(workspace=tmp_path)
out = _text(_run(tool.execute(pattern="{*.py,Error*}", limit=3)))
assert sorted(out.splitlines()) == sorted(names)
assert "PARTIAL" not in out
def test_nested_braces_expand_by_pairing_and_a_runaway_pattern_is_refused(tmp_path):
"""Scanning for the first ``}`` produced garbage patterns (``*.a}``) and
silently dropped whole alternatives -- a listing that omits half the
alternatives reads exactly like a tree that lacks them."""
assert expand_braces("*.{a,b}") == ["*.a", "*.b"]
assert expand_braces("*.{a,{b,c}}") == ["*.a", "*.b", "*.c"]
assert expand_braces("src/{a,b}/*.{ts,tsx}") == ["src/a/*.ts", "src/a/*.tsx", "src/b/*.ts", "src/b/*.tsx"]
assert expand_braces("*.py") == ["*.py"]
with pytest.raises(ValueError):
expand_braces("*.{a,b")
with pytest.raises(ValueError):
expand_braces("{a,b,c,d,e}{1,2,3,4,5}")
(tmp_path / "one.py").write_text("")
tool = GlobTool(workspace=tmp_path)
with workdir.bind(tmp_path):
refused = _text(_run(tool.execute(pattern="{a,b,c,d,e}{1,2,3,4,5}")))
assert refused.startswith("Error") and "separate calls" in refused
assert _text(_run(tool.execute(pattern="*.{py,{txt,md}}"))).splitlines() == ["one.py"]
# --- the boundary the same-name contribution stands on ------------------------
def test_the_registered_instance_is_this_products_under_every_shared_name(tmp_path):
"""The guarantee the whole face rests on: where a name is shared with the
host, the object the registry dispatches to is THIS product's.
Checked by identity, not by schema: a change that registers the plugin
tools before the built-ins instead of after would leave every name in
place and every schema silently reverted to the host's spelling."""
from raven.agent.tools import file_search as trunk_search
from raven.agent.tools import filesystem as trunk_fs
ctx = _ctx(tmp_path, {"enabled": True})
reg = ToolRegistry()
# The host's own order: built-ins first, plugin contributions last.
for cls in (
trunk_fs.ReadFileTool,
trunk_fs.WriteFileTool,
trunk_fs.EditFileTool,
trunk_fs.ListDirTool,
trunk_search.GrepTool,
trunk_search.FindTool,
):
reg.register(cls(workspace=tmp_path))
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
):
reg.register(make(ctx))
for name in ("read_file", "write_file", "edit_file", "list_dir"):
served = reg.get(name)
assert type(served).__module__.startswith("code_flow.tools"), name
assert not isinstance(served, (trunk_fs.ReadFileTool,)) or type(served) is not trunk_fs.ReadFileTool
# Names the host does not use come from here too, and the host's own
# spelling of the pathname tool is still registered until the launcher
# withholds it -- that withholding is the launcher's test, not this one.
assert type(reg.get("glob")).__module__.startswith("code_flow.tools")
assert type(reg.get("todo")).__module__.startswith("code_flow.tools")
assert type(reg.get("grep")).__module__.startswith("raven."), "a name we do not serve stays the host's"
assert reg.get("find") is not None, "trunk's spelling is withheld by config, not by this registry"
def test_the_replacement_answers_the_call_not_just_the_schema(tmp_path):
"""Dispatch, not advertising: the served instance is the one that runs, so
a call in the fork's spelling reaches this product's behaviour (the read
ledger) rather than the host's tool, which has none."""
ctx = _ctx(tmp_path, {"enabled": True})
reg = ToolRegistry()
reg.register(trunk_read_file(tmp_path))
reg.register(factories.make_read_file(ctx))
reg.register(factories.make_edit_file(ctx))
target = tmp_path / "b.txt"
target.write_text("one\n")
# The host's tool has no read-before-edit rule; ours does, and it is ours
# that answers.
refused = _text(
_run(registry_execute(reg, "edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
)
assert refused.startswith("Error") and "have not read" in refused
_run(registry_execute(reg, "read_file", {"file_path": str(target)}))
assert "Successfully edited" in _text(
_run(registry_execute(reg, "edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
)
def test_the_face_declines_as_a_whole_and_the_host_keeps_serving(tmp_path):
"""A product that hands the tools back gets the host's, unchanged: the
factories decline, nothing is registered over the built-ins, and the
schema the model sees is the host's own."""
from raven.agent.tools import filesystem as trunk_fs
ctx = PluginContext(
config={"enabled": True, "tools": {"enabled": False}},
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
)
reg = ToolRegistry()
reg.register(trunk_fs.ReadFileTool(workspace=tmp_path))
contributed = [
make(ctx)
for make in (
factories.make_read_file,
factories.make_write_file,
factories.make_edit_file,
factories.make_list_dir,
factories.make_glob,
factories.make_todo,
factories.make_exec,
)
]
assert contributed == [None] * 7
assert type(reg.get("read_file")) is trunk_fs.ReadFileTool
assert reg.get("read_file").parameters["required"] == ["path"]
def test_a_malformed_slice_declines_rather_than_serving_half_a_face(tmp_path):
"""Half a face is worse than none: the guide and the conduct teach one
spelling, so a slice that does not parse must take the whole face down."""
ctx = PluginContext(
config={"enabled": True, "tools": {"enabled": "yes please"}},
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
)
assert factories.make_read_file(ctx) is None
assert factories.make_glob(ctx) is None
fix(*): show the head of a line longer than a read_file page instead of an empty page (#959) ## Summary A page of `read_file` is cut to whole lines within its 128k-character budget. A file whose first line is over the budget by itself (a minified script, a one-line JSON, a long CSV row) left nothing within it: the page came back empty, and its footer said to continue at the very line it had just not shown, so the reader went round for ever. Measured before the change: `(Showing lines 1-0 of 1. Use offset=1 to continue.)` for a single 200k-character line, and `(Showing lines 2-1 of 3. Use offset=2 to continue.)` once a shorter first line had filled page one. Such a line is now shown by its head, as much as the budget holds, followed by a note that says how long the line is, how much of it is shown, and that `offset` moves by lines and cannot reach the rest: `grep` tells whether a piece is in it, and a shell command that cuts by bytes (such as `cut -c`) shows another stretch. The paging footer then points at the next line, so paging moves on. A first line that just fits (the line break the budget counts put it over) is shown whole, without a note. Pages that fit are unchanged. The node runs the same tool for a file on a registered machine, so the remote read reads the same; one of the new tests goes through the real node behind the stand-in ssh. Raven-Code's `read_file` (the code-flow plugin) cuts every line to 2000 characters. It had its own way round the empty page: whenever trunk answered with no numbered line it re-read the whole file itself, and otherwise it cut the lines after trunk had counted the page, so a dozen 10k-character lines filled a page. Trunk's read now offers a hook, `_cut_line`, applied to each line before the page's budget counts it (trunk's own is a no-op); the plugin cuts on it, so minified lines share a page, each cut once, and its whole-file re-read is removed. Its cut after the read now runs only for a page a registered machine's node answered (the node runs trunk's tool, without the plugin's hook); a page read here was cut on the hook and is not looked at again. The first head told a cut line apart by its ending, so a line whose own text ends like the mark came back whole (both reviewers reproduced it); whether a line was cut is now decided by where the read ran, never by its text. Changed: `raven/agent/tools/filesystem.py` (the trim loop and the hook in `ReadFileTool`), `agents/raven-code/plugins/code-flow/code_flow/tools/filesystem.py` (the plugin's read), tests in `tests/test_filesystem_tools.py`, `tests/test_remote_file_tools.py` and `tests/test_agents_code_tools_plugin.py`. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - The six new tests fail on `origin/main` (the four about the page with the empty pages quoted above; the two about the hook with a dozen lines per page) and pass after the change: `pytest tests/test_filesystem_tools.py tests/test_remote_file_tools.py tests/test_agents_code_tools_plugin.py -k "longer_than_a_page or just_fills_the_page or going_round or cut_per_line or minified_lines"`. - Second commit: `test_a_line_whose_own_text_ends_like_the_mark_is_cut_like_any_other` fails on the first head (the line comes back whole) and passes now; `test_a_page_a_registered_machine_s_node_answered_is_cut_after_the_read` pins the pass after the read for a page a node answered, through a stand-in pool. - `pytest tests/test_agents_code_tools_plugin.py tests/test_filesystem_tools.py tests/test_remote_file_tools.py tests/test_node_server.py tests/test_read_file_image.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py -n 4`: 356 passed. - `scripts/coverage_gate.py diff --base-ref origin/main`: 100% of the 11 changed executable lines. - `ruff check`, `ruff format --check`, `ty check` on both changed modules: clean. - `scripts/check_commit_messages.py`, `scripts/check_source_language.py`, `scripts/check_large_files.py` over `origin/main..HEAD`: clean. - Full suite (`pytest -n 6`), run again at the second commit: 31623 passed, 233 skipped; the 3 failures are the ones main has on this machine (a LibreOffice-less host, two simulation cases), unrelated to this change. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed (checked CONTEXT.md, docs-site and the plugin's own text: nothing describes the page footer or the plugin's cut; both tools' descriptions still hold) ## Risk - [x] Security impact considered: nothing new is read, and the note names no path. - [x] Backward compatibility considered: a page that fits is unchanged, and the existing paging tests still hold; the hook is a no-op unless a subclass overrides it. In Raven-Code the page of a file with minified lines holds more lines than before (the cut lands before the budget) and the one mark reads as it did. - [x] Rollback path is clear for risky changes: revert the one commit. ## Related Issues N/A --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-15 21:05:17 +08:00
_MARK = "... (line truncated to 2000 chars)"
def test_a_line_whose_own_text_ends_like_the_mark_is_cut_like_any_other(registry, tmp_path):
"""A 5000-character line ending the way a cut line does came back whole: the
pass after the read took the ending for its own mark (found by both
reviewers of the first head). Whether a line was cut is decided by where
the read ran, never by the line's text."""
target = tmp_path / "pasted.txt"
target.write_text("x" * (5_000 - len(_MARK)) + _MARK + "\nlast\n")
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
lines = body.split("\n")
assert lines[0] == "1| " + "x" * 2000 + _MARK and lines[1] == "2| last"
def test_a_page_a_registered_machine_s_node_answered_is_cut_after_the_read(tmp_path, monkeypatch):
"""The node runs trunk's tool, without this fork's hook, so its page is cut here, line by line, once."""
import json
from raven.ops import remote_path
rows = [
{"id": "box", "display_name": "Lab box", "host": "203.0.113.7", "port": 58717, "user": "worker", "key": "/k"}
]
(tmp_path / "connections.json").write_text(json.dumps({"connections": rows}), encoding="utf-8")
monkeypatch.setenv("RAVEN_CONNECTIONS", str(tmp_path / "connections.json"))
monkeypatch.setattr(remote_path, "_HOMES", {})
monkeypatch.setattr(remote_path, "_NODES", None)
monkeypatch.setattr(remote_path, "_READ", {})
monkeypatch.setattr(
"raven.config.loader.load_config", lambda *a, **k: SimpleNamespace(subagents=SimpleNamespace(agents=[]))
)
class Pool:
async def home(self, machine_id: str) -> str:
return "/home/worker"
async def call(self, machine_id: str, method: str, params: dict) -> dict:
assert method == "fs/read"
return {"text": "1| " + "y" * 5_000 + "\n2| last\n\n(End of file — 2 lines total)", "sha256": "ab" * 32}
remote_path.use_nodes(Pool)
read = ReadFileTool(workspace=tmp_path, allowed_dirs=(tmp_path,), follow_binding=False)
got = _run(read.execute(file_path="box:raven-work/t1/bundle.js"))
lines = got.split("\n")
assert lines[0] == ("1| " + "y" * 5_000)[:2000] + _MARK and lines[1] == "2| last"
assert got.endswith("(End of file — 2 lines total)")