feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
"""code-flow's tool face (code_flow/tools/): the fork's tools as same-name replacements.
|
|
|
|
|
|
|
|
|
|
Pins the manifest rows, the D6 admission from the slice, the fork's schema
|
|
|
|
|
spelling with the host's names accepted as aliases, the two pieces of conduct
|
|
|
|
|
the file tools carry (read-before-edit, the Python syntax verdict), the
|
|
|
|
|
``occurrence`` edit, ``glob`` and ``todo``, and that a plugin tool
|
|
|
|
|
registered after the built-in replaces it in the registry the loop reads.
|
|
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
import asyncio
|
fix(agent): bound every tree walk and take it off the event loop (#566)
## Summary
On 2026-09-20 one `find` call froze the whole gateway for 2h21m. The
model asked for `find(pattern="**/gps_trilateration/index.html",
path="/Users/admin")`, and `FindTool` ran `Path.glob` synchronously on
the event loop, filtered the noise directories only after walking them,
and had no deadline. The registry's 300s ceiling is an
`asyncio.wait_for`, which cannot preempt synchronous code, so every
session and every subagent on the gateway stopped until the walk came
back (`gateway.log`: `Tool result: find duration=8476519ms`). `list_dir`
with `recursive=true` had the same shape through `rglob`.
The fix gives the three tree-walking tools one walk,
`raven.agent.tools.tree_walk`, with these bounds:
- noise directories (`node_modules`, `.git`, ...) are pruned before they
are entered, not filtered afterwards -- the pruning tests spy on
`os.scandir` and assert the noise directory is never opened;
- the walk yields one entry at a time and checks a 20s wall-clock budget
(the value grep's fallback already used) before every entry, so whatever
the caller does with one entry -- the `stat` per `find` match, the read
per grep file -- is covered by the check on the next. The budget cannot
be overrun by one large or slow directory: the probe with eight matches,
a 30ms stat each and a 50ms budget returns 2 matches and the notice.
When the budget is hit, `find` and `list_dir` append a `PARTIAL result:
... traversal budget ...` clause to their trailer, and a deadline that
fires before any match is not reported as a clean `No files found` /
`Directory is empty`;
- every caller runs the walk in `asyncio.to_thread`, so the loop keeps
serving other sessions while the walk runs. Cancelling the thread does
not stop the worker, which is why the deadline lives inside the walk.
The pure-Python `grep` fallback already had the prune and the deadline;
it moves onto the shared walk and off the loop with the others.
`find` matches each path component with `fnmatch.translate`, with a
component that is exactly `**` spanning any number of components; the
matcher is a position-set simulation (`_advance`), linear in components
times segments, so a pattern carrying many `**` cannot make one match
call slow. A pattern's leading literal components decide where the walk
starts: `src/**/*.py` begins inside `src` and never lists a sibling of
it, a pattern that is all literal (`src/lib/c.py`, `src/lib`) names
exactly one path and answers without a walk, and the prefix itself is a
match when the rest of the pattern can match nothing (`src/**` still
lists `src/`). It answers exactly what `Path.glob` answered for the same
pattern: pinned by a parametrized oracle test over twenty-seven patterns
on a fixture and nineteen on a fixture with links (a directory link out
of the tree, a link cycle, a file link, a noise directory beneath a
link), and checked by hand against `Path.glob` over `raven/` for 27
patterns including bracket edge cases and trailing slashes, 0
mismatches. Two deliberate differences: a trailing `**` also lists files
beneath its prefix (3.12's `Path.glob` listed only directories there),
and a pattern with `..` or a leading `/` is refused instead of followed,
since a pattern must not reach outside the directory the fence resolved.
A trailing slash keeps `Path.glob`'s directory-only meaning (`*/`,
`src/**/`). A symbolic link to a directory is entered where a single
component names or matches it and never under `**`, which is how
`Path.glob` read a pattern: `*/util/helper.py` reaches through a linked
`vendor`, `**/helper.py` does not, and a link cycle is entered once per
component the pattern spends on it, so `*/*/*/b.py` enters `src/loop ->
src` twice and stops. The walker asks its caller per link (`follow`),
`find` answers from the pattern and hands the link positions of a path
to the matcher so `**` cannot claim them, and `list_dir` and the grep
fallback never ask, staying as `rglob` and `os.walk` left them. A third
deliberate difference from the merge base: a noise directory inside a
pattern's literal prefix (`node_modules/*.js`, `src/node_modules/*.js`)
is walked, since the pattern asked for it, where `Path.glob`'s filter
dropped it; one met below the prefix is pruned as before
(`*/node_modules/*.js` answers nothing), `list_dir` on such a path lists
it, and the `find`, `list_dir` and code-flow `glob` descriptions state
that rule instead of an unconditional "skipped". Review found an earlier
wording of the two `find` descriptions one component short ("except one
the pattern starts with"), while the oracle helper already pruned below
the prefix; the sentences now say what the helper checks, and the oracle
has the `src/node_modules` shape on both sides.
The raven-code `glob` tool (`agents/raven-code/.../tools/search.py`)
parses trunk `find` output line by line and strips the trailer before
merging brace alternatives. It learns the new clause, so a cut-short
child is reported as cut short rather than as a tree that lacks the
files; its `is_notice` grammar covers the new line so it is never pasted
into a listing as a path. The code-flow tool guide
(`prompts/TOOLS_CODE.md`), the wrapper's description and its `limit`
parameter told the model that a PARTIAL `glob` result means "raise
`limit` or narrow the pattern"; a traversal-budget PARTIAL is now a
result `glob` can return and a higher limit does nothing for it, so all
three now tell the two notices apart. The seeded copy of the guide
refreshes through the receipt `seed_guide` keeps beside it, so no legacy
digest is added.
Fixed in passing, and now pinned: recursive `list_dir` filtered `rglob`
results on the components of the absolute path, so a workspace beneath a
directory named `build`, `dist`, `venv` or another noise name listed as
"Directory ... is empty" (reproduced at the merge base under
`<tmp>/build/proj`; this head lists it). The shared walk prunes below
the listed path only; `find` and the grep fallback already did.
Not changed: the 20s value. It is the budget grep's fallback already ran
under; this PR does not re-measure it, it makes it hold for every entry
of every walk.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
At head 8dade176, on origin/main at d50a13c1 (nine commits: the walk,
the grep pin, the changelog, the per-entry budget with the prefix start,
the literal-directory case, the symbolic-link reading with the
noise-prefix wording, the noise-name-above-the-path pin, the guide
wording, and the prefix-rule wording; `git range-diff` against bb836988:
all nine replayed `=` onto the moved main, which conflicted only at the
changelog anchor and touches no file this PR touches):
```
uv run pytest tests/test_file_search_traversal_guard.py tests/test_agents_code_tools_plugin.py \
tests/test_search_tools.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py tests/test_filesystem_tools.py -q
# 203 passed
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q --ignore=tests/integration # in a throwaway worktree
# 24652 passed, 110 skipped in 382.30s (0:06:22) at head 8dade176, in a throwaway worktree with COLORTERM=truecolor
uv run pytest tests/test_agents_code_launcher.py tests/test_agents_code_tools_plugin.py -q # the guide's seeding and wording tests, the glob wrapper
# 119 passed
make lint-python lint-imports lint-types lint-deps # all pass
git diff --check origin/main...HEAD # clean
scripts/check_source_language.py origin/main...HEAD # exit 0
scripts/check_large_files.py origin/main...HEAD # exit 0
git merge-tree --write-tree HEAD origin/main # clean
```
Revert-to-red, with the tests kept (101 tests in
`test_file_search_traversal_guard.py` +
`test_agents_code_tools_plugin.py`):
```
the three production files reverted to the previously reviewed head (fd5663ee) 19 failed
file_search.py + filesystem.py reverted to origin/main, tree_walk.py removed collection error
```
Mutation, each mutant applied alone, control restored between runs:
```
deadline checked once per directory (the reviewed placement) 10 failed
walk starts at the root instead of the pattern's prefix 2 failed
trailing slash matches files too 4 failed
a pattern naming one file still walks 1 failed
the prefix itself is never a match 7 failed
a pattern naming one directory walks beneath it 1 failed
control (nothing mutated) 101 passed
```
From the first review round (still holding at this head): find without
`to_thread` 1 failed; list_dir without `to_thread` 1 failed; grep
fallback without `to_thread` 1 failed; pruning removed 6 failed; fork
drops the PARTIAL clause 1 failed.
Fourth round, the symbolic-link reading
(`test_file_search_traversal_guard.py` +
`test_agents_code_tools_plugin.py`, 128 tests, `-n 0`; a mutant that
lets `**` sweep a link, or forgets the link positions, re-enters the
cycle until the OS refuses the path and answers paths `Path.glob` never
did):
```
baseline 128 passed
MUTANT ** may sweep a link 8 failed (7 oracle cases, and the link the pattern ends at is opened)
MUTANT find never asks to enter a link 6 failed (the six oracle patterns that pass through a link)
MUTANT a link is entered even when the pattern ends at it 1 failed (scandir spy on **/vendor)
MUTANT ancestor links forgotten (links_of empty) 1 failed (a cycle case answers a path Path.glob never did)
MUTANT the walker consults follow but never enters 7 failed
MUTANT each followed root gets a fresh deadline 9 failed (every deadline test, and the walker's one-budget test)
REVERT production files to reviewed head d67184c8 7 failed
CONTROL restored 128 passed
```
Revert-to-red for the noise-name pin: with
`raven/agent/tools/filesystem.py` at the merge base and the test kept,
`test_a_noise_name_above_the_searched_path_hides_nothing` fails on the
`Directory ... is empty` answer (1 failed); at this head it passes. The
`agents/**` change is two prose strings and a guide line, outside the
coverage gate's `raven/**` and `plugins-dist/**` pathspecs; the guide's
name-list test and its seeding tests (`test_agents_code_launcher.py`)
cover the file.
The prefix rule is pinned by mutation, the other policy (prune a noise
directory inside the prefix) against the two guard files (131 tests, `-n
0`):
```
baseline 131 passed
MUTANT a noise directory inside the prefix is pruned 5 failed
test_find_matches_what_path_glob_matched[node_modules/**/*.py], test_find_matches_what_path_glob_matched[node_modules/*/], test_find_matches_what_path_glob_matched[node_modules/*/d.py], test_find_matches_what_path_glob_matched[node_modules/x/d.py], test_find_matches_what_path_glob_matched[src/node_modules/*.py]
CONTROL restored 131 passed
```
The symlink oracle is the reviewer's own test shape over a fixture with
links: `vendor -> outside/real`, `src/loop -> src`, `alias.py -> a.py`,
a `node_modules` beneath the link; 19 patterns, 0 mismatches against
`Path.glob` on 3.12, plus a scandir spy pinning that `**/vendor` never
opens the link, a walker test pinning that the default walk yields a
link and does not enter it while a `follow`ed link is walked after the
tree under one deadline, and a `list_dir` and grep test pinning
`rglob`/`os.walk` parity through a link.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed (CHANGELOG
`Unreleased / Fixed` entry)
## Risk
User-visible behaviour changes, all in tool output the model reads:
- `find` and `list_dir` (recursive) can now return a partial listing
with a `PARTIAL result` clause in the trailer where they previously
either returned everything or hung. The clause tells the model the
absence of a match is not conclusive and to narrow the search.
- A path-prefixed `find` pattern spends its budget only inside its
prefix, so on a large tree it finds what a root-anchored walk used to
miss.
- `find` refuses patterns containing `..` or starting with `/` with an
`Error running find:` line; `Path.glob` used to raise on the second and
could climb on the first.
- `find` with a trailing `**` lists files as well as directories beneath
the prefix; a trailing `/` lists directories only, as before.
- `find` reaches through a symbolic link to a directory where a literal
or single-wildcard component names or matches it, as it did at the merge
base; the earlier revisions of this PR had lost that and returned a
clean miss with no notice. `**` never enters a link, as before.
- A noise directory inside a `find` pattern's literal prefix, or named
as the `list_dir` path, is walked: `find('node_modules/*.js')` and
`find('src/node_modules/*.js')` answer its files and
`list_dir('node_modules', recursive=true)` lists it, where the merge
base answered nothing and "empty". One met below the prefix is pruned as
before. Both descriptions state the rule.
- Recursive `list_dir` on a workspace beneath a directory named like a
noise directory (`/tmp/build/proj`) lists it instead of answering "is
empty".
- The code-flow `glob` description, its `limit` parameter and the tool
guide tell a limit cut from a traversal-budget PARTIAL; the notices
themselves are unchanged.
- `list_dir` flat listings are unchanged; the empty-directory message is
unchanged.
Rollback: revert the squash commit. The new module has no other
consumers, and no state or schema is involved.
- [x] Security impact considered (a pattern can no longer climb out of
the fenced directory via `..`; a directory link is entered exactly where
`Path.glob` entered it at the merge base, one non-`**` component, and
never under `**`, so the fence's reach is what it was and not wider;
`list_dir` and grep never enter one)
- [x] Backward compatibility considered (the fork's trailer grammar is
extended, not changed; existing fork tests pass unchanged)
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 10:59:01 +08:00
|
|
|
import itertools
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
import os
|
|
|
|
|
import subprocess
|
|
|
|
|
import sys
|
feat(*): add bounded Python diagnostics after code edits (#967)
## Summary
Raven-Code reports Python syntax errors after file edits but misses
unresolved names and equivalent feedback after shell edits. Add advisory
Pyflakes F821/F822/F823 checks to the existing tool result; optional
F401 notices remain off by default. No additional model call or code
execution is involved.
```text
write_file / edit_file
-> inspect final source, compare with a baseline that can be read and analyzed
-> retain syntax warnings; suppress existing name findings
exec
-> require checkpoint visibility for the working directory
-> read the changed files from the host's write record for the command
-> ask visibility in batches of 256, select at most 8, check up to 4 concurrently
-> append ordered diagnostics, preserving exit status and file metadata
```
Existing syntax failures remain visible after unrelated edits and
identical rewrites. A baseline that cannot be read, parsed or analyzed
is marked unknown; overwriting non-UTF-8 source still checks the known
replacement. Source nested too deeply for the parser or checker, and a
changed file that cannot be read, report diagnostics unavailable for
that file rather than passing as clean. Every check reads a file as the
bytes Python runs: text a write or edit stores is checked as its UTF-8
bytes, the content it replaced included, and a file from disk as its
bytes, each decoded as Python decodes source, by its coding declaration
or as UTF-8. Bytes that do not decode are reported as a syntax error at
the line where decoding stops, so a declaration the written text cannot
satisfy is reported however the file was changed. Noqa comments are read
as Ruff reads them, including after other comment directives and never
inside string literals: a file-level `ruff: noqa` / `flake8: noqa`
counts only on a line of its own, and a marker counts only when codes
follow its colon or, with no colon, whitespace, another comment or the
end of the comment follows it. Unlike Ruff, which takes the first
directive in a comment, every directive in it counts. The seeded tool
guide gives the syntax note's own advice. With no checkpoint or no
visibility verdict, exec diagnostics do not inspect changed file
contents; direct file checks remain available.
Pyflakes is imported when the tool face loads, not by whichever check
first needs it: imported lazily, its cold import fell inside that
check's one-second wait, and on a slow Windows runner the one test that
paid it reported diagnostics unavailable for a plain syntax error.
Private workers bound reads and checks to four active jobs per process.
Saturated capacity rejects new work without queuing, and
timeout/cancellation retains each slot until the actual work finishes.
Name checks are capped at 256 KiB of source and the syntax check at 2
MiB, and a check skipped for size says so in one line. Findings are
capped at eight per file, and exec diagnostic text at 6,000 characters
plus omission notices.
Runtime changes stay inside the Raven-Code code-flow plugin, except for
one fix to `CheckpointService.trackable`, the visibility verdict that
exec diagnostics and the host's exec write record both rely on. On
macOS, git precomposes a file name it reads off the disk before matching
an ignore rule, but takes the paths `trackable` sends to `git
check-ignore --stdin` as written. A file stored with a decomposed (NFD)
name under a `.gitignore` rule written precomposed (NFC) is therefore
left out of the checkpoint, yet main vouches for it, so the write
record's diff carries its text, and exec diagnostics would quote its
source lines to the model. Nor is git's precomposition NFC: it composes
only neighbouring characters, by Unicode 3.2's table, and reorders
nothing, so a name whose combining marks are out of canonical order
comes out as neither the name as given nor its NFC. `trackable` now asks
about each name as given, as NFC and in git's macOS spelling, and
withholds it when any spelling is ignored. Other shared
file/shell/snapshot implementations are unchanged.
Raven-Code has no separate dependency manifest, so Pyflakes uses the
existing root dependencies. It is the only added package; uv normalized
existing lockfile markers without changing other package versions.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- **Before:** on main, Raven-Code's file tools report Python syntax
errors only, and its exec reports nothing about the Python files a
command changed. On macOS, main's `CheckpointService.trackable` vouches
for a file stored as `secret_cafe` + U+0301 + `.py` under the rule
`secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the
checkpoint's staged tree leaves it out, yet its text reaches the exec
write record's stored diff.
- **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command
below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default
suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors
over 22 test ids: 21 of them also fail on the exact base in the same
environment, and the other, a remote process-tree cancel test that
missed its deadline under the suite's load, passes when run alone. Tests
cover actual worker overlap, retained capacity after
timeout/cancellation, host-tool isolation, hidden-file quotas,
visibility batching through the host's write-record wiring, baseline
handling, parser-limit failures, the size bounds and their skip notes,
undecodable bytes placed where decoding stops, coding declarations in
written and edited text, the baseline read staying on a worker, the
checker loaded with its module, the file each unavailable note names,
comment directives and their marker boundary, the tool guide's agreement
with the syntax note, portable commands in paths with spaces/non-ASCII
characters, and the checkpoint's verdict on decomposed names and on
names git spells as neither given nor NFC, held against its own staged
tree. Removing any of the three spellings from the query fails the cases
that pin it, as does removing the marker boundary, the parser-limit
report, the unknown-baseline handling, the syntax check past the
name-check bound and the decoding report, checking written text as its
stored bytes, the worker around the baseline read, loading the checker
with its module, or the file name in an unavailable note; restoring the
guide's earlier advice fails its agreement test. git's macOS spelling is
what libiconv's UTF-8-MAC conversion returns, which git precomposes
with: composing neighbours by Unicode 3.2's table reproduced it for all
281,928 names built from every decomposable character's decomposition
(reordered, partly composed, or with a mark added), and across 784,220
further mixed names and pairs no name was spelled outside the three
asked; the same held under Python 3.13 and 3.14. The decomposed-name
check asserts the macOS outcome only where git precomposes names, so it
also passes with `core.precomposeunicode = false` set through
`GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines
match Ruff 0.15.12, except the three that hold several directives in one
comment. Through the real `AgentLoop`, code-flow tool face,
`CodeExecutor` and checkpoint, with the working directory under `/tmp`,
under `$TMPDIR` and in a path with spaces and CJK characters, exec
diagnostics report created, modified and decomposed-name files, follow
`working_dir`, report an undecodable file at the line where decoding
stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold
`.gitignore`d files, including decomposed names and names git spells as
neither given nor NFC; across repeated runs on both interpreters, a file
near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a
baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600
ignored files plus one visible file take 73-90 ms with the event loop
held at most 20 ms. At this head CI passes the unit suite on
Linux/Python 3.12 (32619 tests over four shards), and its two Windows
steps pass the diagnostic files on native Windows/Python 3.13: 122
file/worker tests, and 27 exec diagnostic tests with 17 pre-existing
executor tests deselected; the symlink fixture skips only where the host
cannot create links.
```sh
env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3
uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py
uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py
uv run --no-sync lint-imports
uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
```
Native Windows CI commands:
```sh
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short
```
Ruff, format and type checks passed; all 10 import contracts kept; the
commit-message, large-file and source-language checks pass over the
branch. The launcher tests unset inherited search API keys to keep the
expected tool list deterministic.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes
does not activate checks in other agents. The change is reversible and
never fixes/removes code or rolls back edits. Static diagnostics can
need judgment for dynamic names, target Python versions, and project
lint configuration; only explicit noqa directives are honored. Passing
diagnostics do not replace tests.
The `trackable` fix reaches every exec write record, not only
Raven-Code: a file whose name, as given, as NFC or in git's macOS
spelling, matches an ignore rule keeps its counts and loses its diff and
removal body. Where git matches another of those spellings, as off macOS
for a decomposed name under a precomposed rule, the checkpoint stores
the file but it is still withheld; its counts remain.
Exec diagnostics now require an active checkpoint. Noninteractive runs
using the default interactive checkpoint policy must explicitly enable
checkpoints to receive this feedback. Visibility is checked in batches
of up to 256 files and stops once eight visible files are found; each
batch holds the event loop while the host resolves its paths, about 130
ms for a full batch on native Windows. Each file-check wait is one
second, and post-command diagnostics have a three-second wait budget;
past it, or when they fail as a whole, they are dropped and the
command's result is kept. These are caller budgets, not hard I/O
deadlines: running OS reads and inherited synchronous host callbacks
cannot be forcibly interrupted. Private worker capacity remains bounded.
Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took
0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on
slower hardware such a check can outlast the one-second wait and report
diagnostics unavailable. Python's GIL means concurrency does not promise
a CPU speedup.
Windows CI exercises the diagnostics layer with the real native
DirectExecutor. The existing CodeExecutor environment probe uses POSIX
quoting and already fails on Windows at the base; this PR neither
changes that probe nor claims end-to-end compatibility for it. Symlink
setup is skipped only if the host cannot create the fixture.
Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics`
for no post-command checks, or `pythonSyntaxNote` for no diagnostics,
under `plugins.config["code-flow"].tools`. The PR can also be reverted
as a unit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
|
|
|
import threading
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
from pathlib import Path
|
fix(agent): bound every tree walk and take it off the event loop (#566)
## Summary
On 2026-09-20 one `find` call froze the whole gateway for 2h21m. The
model asked for `find(pattern="**/gps_trilateration/index.html",
path="/Users/admin")`, and `FindTool` ran `Path.glob` synchronously on
the event loop, filtered the noise directories only after walking them,
and had no deadline. The registry's 300s ceiling is an
`asyncio.wait_for`, which cannot preempt synchronous code, so every
session and every subagent on the gateway stopped until the walk came
back (`gateway.log`: `Tool result: find duration=8476519ms`). `list_dir`
with `recursive=true` had the same shape through `rglob`.
The fix gives the three tree-walking tools one walk,
`raven.agent.tools.tree_walk`, with these bounds:
- noise directories (`node_modules`, `.git`, ...) are pruned before they
are entered, not filtered afterwards -- the pruning tests spy on
`os.scandir` and assert the noise directory is never opened;
- the walk yields one entry at a time and checks a 20s wall-clock budget
(the value grep's fallback already used) before every entry, so whatever
the caller does with one entry -- the `stat` per `find` match, the read
per grep file -- is covered by the check on the next. The budget cannot
be overrun by one large or slow directory: the probe with eight matches,
a 30ms stat each and a 50ms budget returns 2 matches and the notice.
When the budget is hit, `find` and `list_dir` append a `PARTIAL result:
... traversal budget ...` clause to their trailer, and a deadline that
fires before any match is not reported as a clean `No files found` /
`Directory is empty`;
- every caller runs the walk in `asyncio.to_thread`, so the loop keeps
serving other sessions while the walk runs. Cancelling the thread does
not stop the worker, which is why the deadline lives inside the walk.
The pure-Python `grep` fallback already had the prune and the deadline;
it moves onto the shared walk and off the loop with the others.
`find` matches each path component with `fnmatch.translate`, with a
component that is exactly `**` spanning any number of components; the
matcher is a position-set simulation (`_advance`), linear in components
times segments, so a pattern carrying many `**` cannot make one match
call slow. A pattern's leading literal components decide where the walk
starts: `src/**/*.py` begins inside `src` and never lists a sibling of
it, a pattern that is all literal (`src/lib/c.py`, `src/lib`) names
exactly one path and answers without a walk, and the prefix itself is a
match when the rest of the pattern can match nothing (`src/**` still
lists `src/`). It answers exactly what `Path.glob` answered for the same
pattern: pinned by a parametrized oracle test over twenty-seven patterns
on a fixture and nineteen on a fixture with links (a directory link out
of the tree, a link cycle, a file link, a noise directory beneath a
link), and checked by hand against `Path.glob` over `raven/` for 27
patterns including bracket edge cases and trailing slashes, 0
mismatches. Two deliberate differences: a trailing `**` also lists files
beneath its prefix (3.12's `Path.glob` listed only directories there),
and a pattern with `..` or a leading `/` is refused instead of followed,
since a pattern must not reach outside the directory the fence resolved.
A trailing slash keeps `Path.glob`'s directory-only meaning (`*/`,
`src/**/`). A symbolic link to a directory is entered where a single
component names or matches it and never under `**`, which is how
`Path.glob` read a pattern: `*/util/helper.py` reaches through a linked
`vendor`, `**/helper.py` does not, and a link cycle is entered once per
component the pattern spends on it, so `*/*/*/b.py` enters `src/loop ->
src` twice and stops. The walker asks its caller per link (`follow`),
`find` answers from the pattern and hands the link positions of a path
to the matcher so `**` cannot claim them, and `list_dir` and the grep
fallback never ask, staying as `rglob` and `os.walk` left them. A third
deliberate difference from the merge base: a noise directory inside a
pattern's literal prefix (`node_modules/*.js`, `src/node_modules/*.js`)
is walked, since the pattern asked for it, where `Path.glob`'s filter
dropped it; one met below the prefix is pruned as before
(`*/node_modules/*.js` answers nothing), `list_dir` on such a path lists
it, and the `find`, `list_dir` and code-flow `glob` descriptions state
that rule instead of an unconditional "skipped". Review found an earlier
wording of the two `find` descriptions one component short ("except one
the pattern starts with"), while the oracle helper already pruned below
the prefix; the sentences now say what the helper checks, and the oracle
has the `src/node_modules` shape on both sides.
The raven-code `glob` tool (`agents/raven-code/.../tools/search.py`)
parses trunk `find` output line by line and strips the trailer before
merging brace alternatives. It learns the new clause, so a cut-short
child is reported as cut short rather than as a tree that lacks the
files; its `is_notice` grammar covers the new line so it is never pasted
into a listing as a path. The code-flow tool guide
(`prompts/TOOLS_CODE.md`), the wrapper's description and its `limit`
parameter told the model that a PARTIAL `glob` result means "raise
`limit` or narrow the pattern"; a traversal-budget PARTIAL is now a
result `glob` can return and a higher limit does nothing for it, so all
three now tell the two notices apart. The seeded copy of the guide
refreshes through the receipt `seed_guide` keeps beside it, so no legacy
digest is added.
Fixed in passing, and now pinned: recursive `list_dir` filtered `rglob`
results on the components of the absolute path, so a workspace beneath a
directory named `build`, `dist`, `venv` or another noise name listed as
"Directory ... is empty" (reproduced at the merge base under
`<tmp>/build/proj`; this head lists it). The shared walk prunes below
the listed path only; `find` and the grep fallback already did.
Not changed: the 20s value. It is the budget grep's fallback already ran
under; this PR does not re-measure it, it makes it hold for every entry
of every walk.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
At head 8dade176, on origin/main at d50a13c1 (nine commits: the walk,
the grep pin, the changelog, the per-entry budget with the prefix start,
the literal-directory case, the symbolic-link reading with the
noise-prefix wording, the noise-name-above-the-path pin, the guide
wording, and the prefix-rule wording; `git range-diff` against bb836988:
all nine replayed `=` onto the moved main, which conflicted only at the
changelog anchor and touches no file this PR touches):
```
uv run pytest tests/test_file_search_traversal_guard.py tests/test_agents_code_tools_plugin.py \
tests/test_search_tools.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py tests/test_filesystem_tools.py -q
# 203 passed
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q --ignore=tests/integration # in a throwaway worktree
# 24652 passed, 110 skipped in 382.30s (0:06:22) at head 8dade176, in a throwaway worktree with COLORTERM=truecolor
uv run pytest tests/test_agents_code_launcher.py tests/test_agents_code_tools_plugin.py -q # the guide's seeding and wording tests, the glob wrapper
# 119 passed
make lint-python lint-imports lint-types lint-deps # all pass
git diff --check origin/main...HEAD # clean
scripts/check_source_language.py origin/main...HEAD # exit 0
scripts/check_large_files.py origin/main...HEAD # exit 0
git merge-tree --write-tree HEAD origin/main # clean
```
Revert-to-red, with the tests kept (101 tests in
`test_file_search_traversal_guard.py` +
`test_agents_code_tools_plugin.py`):
```
the three production files reverted to the previously reviewed head (fd5663ee) 19 failed
file_search.py + filesystem.py reverted to origin/main, tree_walk.py removed collection error
```
Mutation, each mutant applied alone, control restored between runs:
```
deadline checked once per directory (the reviewed placement) 10 failed
walk starts at the root instead of the pattern's prefix 2 failed
trailing slash matches files too 4 failed
a pattern naming one file still walks 1 failed
the prefix itself is never a match 7 failed
a pattern naming one directory walks beneath it 1 failed
control (nothing mutated) 101 passed
```
From the first review round (still holding at this head): find without
`to_thread` 1 failed; list_dir without `to_thread` 1 failed; grep
fallback without `to_thread` 1 failed; pruning removed 6 failed; fork
drops the PARTIAL clause 1 failed.
Fourth round, the symbolic-link reading
(`test_file_search_traversal_guard.py` +
`test_agents_code_tools_plugin.py`, 128 tests, `-n 0`; a mutant that
lets `**` sweep a link, or forgets the link positions, re-enters the
cycle until the OS refuses the path and answers paths `Path.glob` never
did):
```
baseline 128 passed
MUTANT ** may sweep a link 8 failed (7 oracle cases, and the link the pattern ends at is opened)
MUTANT find never asks to enter a link 6 failed (the six oracle patterns that pass through a link)
MUTANT a link is entered even when the pattern ends at it 1 failed (scandir spy on **/vendor)
MUTANT ancestor links forgotten (links_of empty) 1 failed (a cycle case answers a path Path.glob never did)
MUTANT the walker consults follow but never enters 7 failed
MUTANT each followed root gets a fresh deadline 9 failed (every deadline test, and the walker's one-budget test)
REVERT production files to reviewed head d67184c8 7 failed
CONTROL restored 128 passed
```
Revert-to-red for the noise-name pin: with
`raven/agent/tools/filesystem.py` at the merge base and the test kept,
`test_a_noise_name_above_the_searched_path_hides_nothing` fails on the
`Directory ... is empty` answer (1 failed); at this head it passes. The
`agents/**` change is two prose strings and a guide line, outside the
coverage gate's `raven/**` and `plugins-dist/**` pathspecs; the guide's
name-list test and its seeding tests (`test_agents_code_launcher.py`)
cover the file.
The prefix rule is pinned by mutation, the other policy (prune a noise
directory inside the prefix) against the two guard files (131 tests, `-n
0`):
```
baseline 131 passed
MUTANT a noise directory inside the prefix is pruned 5 failed
test_find_matches_what_path_glob_matched[node_modules/**/*.py], test_find_matches_what_path_glob_matched[node_modules/*/], test_find_matches_what_path_glob_matched[node_modules/*/d.py], test_find_matches_what_path_glob_matched[node_modules/x/d.py], test_find_matches_what_path_glob_matched[src/node_modules/*.py]
CONTROL restored 131 passed
```
The symlink oracle is the reviewer's own test shape over a fixture with
links: `vendor -> outside/real`, `src/loop -> src`, `alias.py -> a.py`,
a `node_modules` beneath the link; 19 patterns, 0 mismatches against
`Path.glob` on 3.12, plus a scandir spy pinning that `**/vendor` never
opens the link, a walker test pinning that the default walk yields a
link and does not enter it while a `follow`ed link is walked after the
tree under one deadline, and a `list_dir` and grep test pinning
`rglob`/`os.walk` parity through a link.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed (CHANGELOG
`Unreleased / Fixed` entry)
## Risk
User-visible behaviour changes, all in tool output the model reads:
- `find` and `list_dir` (recursive) can now return a partial listing
with a `PARTIAL result` clause in the trailer where they previously
either returned everything or hung. The clause tells the model the
absence of a match is not conclusive and to narrow the search.
- A path-prefixed `find` pattern spends its budget only inside its
prefix, so on a large tree it finds what a root-anchored walk used to
miss.
- `find` refuses patterns containing `..` or starting with `/` with an
`Error running find:` line; `Path.glob` used to raise on the second and
could climb on the first.
- `find` with a trailing `**` lists files as well as directories beneath
the prefix; a trailing `/` lists directories only, as before.
- `find` reaches through a symbolic link to a directory where a literal
or single-wildcard component names or matches it, as it did at the merge
base; the earlier revisions of this PR had lost that and returned a
clean miss with no notice. `**` never enters a link, as before.
- A noise directory inside a `find` pattern's literal prefix, or named
as the `list_dir` path, is walked: `find('node_modules/*.js')` and
`find('src/node_modules/*.js')` answer its files and
`list_dir('node_modules', recursive=true)` lists it, where the merge
base answered nothing and "empty". One met below the prefix is pruned as
before. Both descriptions state the rule.
- Recursive `list_dir` on a workspace beneath a directory named like a
noise directory (`/tmp/build/proj`) lists it instead of answering "is
empty".
- The code-flow `glob` description, its `limit` parameter and the tool
guide tell a limit cut from a traversal-budget PARTIAL; the notices
themselves are unchanged.
- `list_dir` flat listings are unchanged; the empty-directory message is
unchanged.
Rollback: revert the squash commit. The new module has no other
consumers, and no state or schema is involved.
- [x] Security impact considered (a pattern can no longer climb out of
the fenced directory via `..`; a directory link is entered exactly where
`Path.glob` entered it at the merge base, one non-`**` component, and
never under `**`, so the fence's reach is what it was and not wider;
`list_dir` and grep never enter one)
- [x] Backward compatibility considered (the fork's trailer grammar is
extended, not changed; existing fork tests pass unchanged)
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 10:59:01 +08:00
|
|
|
from types import SimpleNamespace
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
|
|
|
|
|
import pytest
|
|
|
|
|
|
|
|
|
|
from raven.agent import workdir
|
|
|
|
|
from raven.agent.tools.filesystem import ReadFileTool as TrunkReadFileTool
|
|
|
|
|
from raven.agent.tools.registry import ToolRegistry
|
|
|
|
|
from raven.contracts.tool import ToolResult
|
|
|
|
|
from raven.plugins import DiscoveredPlugin, ManifestOrigin, PluginManifest, PluginRegistry
|
|
|
|
|
from raven.plugins.context import PluginContext, ServiceLocator
|
|
|
|
|
|
|
|
|
|
REPO = Path(__file__).resolve().parent.parent
|
|
|
|
|
PLUGIN_DIR = REPO / "agents" / "raven-code" / "plugins" / "code-flow"
|
|
|
|
|
sys.path.insert(0, str(PLUGIN_DIR))
|
|
|
|
|
from code_flow.tools import plugin as factories # noqa: E402
|
|
|
|
|
from code_flow.tools.filesystem import EditFileTool, ReadFileTool, ReadLedger, WriteFileTool # noqa: E402
|
|
|
|
|
from code_flow.tools.read_state import ReadSessions, owner_for # noqa: E402
|
|
|
|
|
from code_flow.tools.search import GlobTool, expand_braces, is_notice # noqa: E402
|
|
|
|
|
|
|
|
|
|
MANIFEST_PATH = PLUGIN_DIR / "raven-plugin.toml"
|
|
|
|
|
FACE = ("read_file", "write_file", "edit_file", "list_dir", "glob", "todo", "exec")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _ctx(tmp_path: Path, tools: dict) -> PluginContext:
|
|
|
|
|
"""A plugin context carrying the flow slice whose ``tools`` section is the
|
|
|
|
|
product's own tool face."""
|
|
|
|
|
return PluginContext(
|
|
|
|
|
config={"enabled": True, "tools": tools},
|
|
|
|
|
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def trunk_read_file(workspace: Path):
|
|
|
|
|
from raven.agent.tools.filesystem import ReadFileTool as TrunkRead
|
|
|
|
|
|
|
|
|
|
return TrunkRead(workspace=workspace)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def registry_execute(reg: ToolRegistry, name: str, params: dict):
|
|
|
|
|
return reg.execute(name, params)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _run(coro):
|
|
|
|
|
return asyncio.get_event_loop_policy().new_event_loop().run_until_complete(coro)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _text(result) -> str:
|
|
|
|
|
return result.model_text if isinstance(result, ToolResult) else str(result)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
|
|
|
def _fresh_ledger(tmp_path):
|
|
|
|
|
reads = ReadSessions(owner_for(_ctx(tmp_path, {"enabled": True})))
|
|
|
|
|
reads.bind("cli:unit")
|
|
|
|
|
yield
|
|
|
|
|
reads.unbind()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture()
|
|
|
|
|
def registry(tmp_path):
|
|
|
|
|
ctx = _ctx(tmp_path, {"enabled": True})
|
|
|
|
|
reg = ToolRegistry()
|
|
|
|
|
for make in (
|
|
|
|
|
factories.make_read_file,
|
|
|
|
|
factories.make_write_file,
|
|
|
|
|
factories.make_edit_file,
|
|
|
|
|
factories.make_list_dir,
|
|
|
|
|
factories.make_glob,
|
|
|
|
|
factories.make_todo,
|
|
|
|
|
factories.make_exec,
|
|
|
|
|
):
|
|
|
|
|
reg.register(make(ctx))
|
|
|
|
|
return reg
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- manifest and admission ----------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_manifest_contributes_the_seven_tools_beside_the_flows_own_seats():
|
|
|
|
|
"""One plugin per product, the way every sibling ships: the tool face is a
|
|
|
|
|
subpackage of code-flow, not a plugin of its own."""
|
|
|
|
|
mf = PluginManifest.from_toml_path(MANIFEST_PATH)
|
|
|
|
|
assert mf.id == "code-flow"
|
|
|
|
|
assert [t.name for t in mf.contributes.tools] == list(FACE)
|
|
|
|
|
assert [h.name for h in mf.contributes.hooks] == ["code_flow"]
|
|
|
|
|
assert mf.contributes.tool_gates == []
|
|
|
|
|
assert [f.factory for f in mf.contributes.tools] == [f"code_flow.tools.plugin:make_{name}" for name in FACE]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_registry_builds_every_tool_from_the_real_factory_strings(tmp_path):
|
|
|
|
|
reg = PluginRegistry()
|
|
|
|
|
mf = PluginManifest.from_toml_path(MANIFEST_PATH)
|
|
|
|
|
reg.activate([DiscoveredPlugin(manifest=mf, source=ManifestOrigin.USER, location=MANIFEST_PATH)])
|
|
|
|
|
assert reg.tool_names() == sorted(FACE)
|
|
|
|
|
services = ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a")
|
|
|
|
|
slice_ = {"enabled": True, "tools": {"enabled": True}}
|
|
|
|
|
for name in FACE:
|
|
|
|
|
tool = reg.build_tool(name, config=slice_, services=services)
|
|
|
|
|
assert tool is not None and tool.name == name
|
|
|
|
|
|
|
|
|
|
|
feat(*): add bounded Python diagnostics after code edits (#967)
## Summary
Raven-Code reports Python syntax errors after file edits but misses
unresolved names and equivalent feedback after shell edits. Add advisory
Pyflakes F821/F822/F823 checks to the existing tool result; optional
F401 notices remain off by default. No additional model call or code
execution is involved.
```text
write_file / edit_file
-> inspect final source, compare with a baseline that can be read and analyzed
-> retain syntax warnings; suppress existing name findings
exec
-> require checkpoint visibility for the working directory
-> read the changed files from the host's write record for the command
-> ask visibility in batches of 256, select at most 8, check up to 4 concurrently
-> append ordered diagnostics, preserving exit status and file metadata
```
Existing syntax failures remain visible after unrelated edits and
identical rewrites. A baseline that cannot be read, parsed or analyzed
is marked unknown; overwriting non-UTF-8 source still checks the known
replacement. Source nested too deeply for the parser or checker, and a
changed file that cannot be read, report diagnostics unavailable for
that file rather than passing as clean. Every check reads a file as the
bytes Python runs: text a write or edit stores is checked as its UTF-8
bytes, the content it replaced included, and a file from disk as its
bytes, each decoded as Python decodes source, by its coding declaration
or as UTF-8. Bytes that do not decode are reported as a syntax error at
the line where decoding stops, so a declaration the written text cannot
satisfy is reported however the file was changed. Noqa comments are read
as Ruff reads them, including after other comment directives and never
inside string literals: a file-level `ruff: noqa` / `flake8: noqa`
counts only on a line of its own, and a marker counts only when codes
follow its colon or, with no colon, whitespace, another comment or the
end of the comment follows it. Unlike Ruff, which takes the first
directive in a comment, every directive in it counts. The seeded tool
guide gives the syntax note's own advice. With no checkpoint or no
visibility verdict, exec diagnostics do not inspect changed file
contents; direct file checks remain available.
Pyflakes is imported when the tool face loads, not by whichever check
first needs it: imported lazily, its cold import fell inside that
check's one-second wait, and on a slow Windows runner the one test that
paid it reported diagnostics unavailable for a plain syntax error.
Private workers bound reads and checks to four active jobs per process.
Saturated capacity rejects new work without queuing, and
timeout/cancellation retains each slot until the actual work finishes.
Name checks are capped at 256 KiB of source and the syntax check at 2
MiB, and a check skipped for size says so in one line. Findings are
capped at eight per file, and exec diagnostic text at 6,000 characters
plus omission notices.
Runtime changes stay inside the Raven-Code code-flow plugin, except for
one fix to `CheckpointService.trackable`, the visibility verdict that
exec diagnostics and the host's exec write record both rely on. On
macOS, git precomposes a file name it reads off the disk before matching
an ignore rule, but takes the paths `trackable` sends to `git
check-ignore --stdin` as written. A file stored with a decomposed (NFD)
name under a `.gitignore` rule written precomposed (NFC) is therefore
left out of the checkpoint, yet main vouches for it, so the write
record's diff carries its text, and exec diagnostics would quote its
source lines to the model. Nor is git's precomposition NFC: it composes
only neighbouring characters, by Unicode 3.2's table, and reorders
nothing, so a name whose combining marks are out of canonical order
comes out as neither the name as given nor its NFC. `trackable` now asks
about each name as given, as NFC and in git's macOS spelling, and
withholds it when any spelling is ignored. Other shared
file/shell/snapshot implementations are unchanged.
Raven-Code has no separate dependency manifest, so Pyflakes uses the
existing root dependencies. It is the only added package; uv normalized
existing lockfile markers without changing other package versions.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- **Before:** on main, Raven-Code's file tools report Python syntax
errors only, and its exec reports nothing about the Python files a
command changed. On macOS, main's `CheckpointService.trackable` vouches
for a file stored as `secret_cafe` + U+0301 + `.py` under the rule
`secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the
checkpoint's staged tree leaves it out, yet its text reaches the exec
write record's stored diff.
- **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command
below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default
suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors
over 22 test ids: 21 of them also fail on the exact base in the same
environment, and the other, a remote process-tree cancel test that
missed its deadline under the suite's load, passes when run alone. Tests
cover actual worker overlap, retained capacity after
timeout/cancellation, host-tool isolation, hidden-file quotas,
visibility batching through the host's write-record wiring, baseline
handling, parser-limit failures, the size bounds and their skip notes,
undecodable bytes placed where decoding stops, coding declarations in
written and edited text, the baseline read staying on a worker, the
checker loaded with its module, the file each unavailable note names,
comment directives and their marker boundary, the tool guide's agreement
with the syntax note, portable commands in paths with spaces/non-ASCII
characters, and the checkpoint's verdict on decomposed names and on
names git spells as neither given nor NFC, held against its own staged
tree. Removing any of the three spellings from the query fails the cases
that pin it, as does removing the marker boundary, the parser-limit
report, the unknown-baseline handling, the syntax check past the
name-check bound and the decoding report, checking written text as its
stored bytes, the worker around the baseline read, loading the checker
with its module, or the file name in an unavailable note; restoring the
guide's earlier advice fails its agreement test. git's macOS spelling is
what libiconv's UTF-8-MAC conversion returns, which git precomposes
with: composing neighbours by Unicode 3.2's table reproduced it for all
281,928 names built from every decomposable character's decomposition
(reordered, partly composed, or with a mark added), and across 784,220
further mixed names and pairs no name was spelled outside the three
asked; the same held under Python 3.13 and 3.14. The decomposed-name
check asserts the macOS outcome only where git precomposes names, so it
also passes with `core.precomposeunicode = false` set through
`GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines
match Ruff 0.15.12, except the three that hold several directives in one
comment. Through the real `AgentLoop`, code-flow tool face,
`CodeExecutor` and checkpoint, with the working directory under `/tmp`,
under `$TMPDIR` and in a path with spaces and CJK characters, exec
diagnostics report created, modified and decomposed-name files, follow
`working_dir`, report an undecodable file at the line where decoding
stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold
`.gitignore`d files, including decomposed names and names git spells as
neither given nor NFC; across repeated runs on both interpreters, a file
near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a
baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600
ignored files plus one visible file take 73-90 ms with the event loop
held at most 20 ms. At this head CI passes the unit suite on
Linux/Python 3.12 (32619 tests over four shards), and its two Windows
steps pass the diagnostic files on native Windows/Python 3.13: 122
file/worker tests, and 27 exec diagnostic tests with 17 pre-existing
executor tests deselected; the symlink fixture skips only where the host
cannot create links.
```sh
env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3
uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py
uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py
uv run --no-sync lint-imports
uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
```
Native Windows CI commands:
```sh
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short
```
Ruff, format and type checks passed; all 10 import contracts kept; the
commit-message, large-file and source-language checks pass over the
branch. The launcher tests unset inherited search API keys to keep the
expected tool list deterministic.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes
does not activate checks in other agents. The change is reversible and
never fixes/removes code or rolls back edits. Static diagnostics can
need judgment for dynamic names, target Python versions, and project
lint configuration; only explicit noqa directives are honored. Passing
diagnostics do not replace tests.
The `trackable` fix reaches every exec write record, not only
Raven-Code: a file whose name, as given, as NFC or in git's macOS
spelling, matches an ignore rule keeps its counts and loses its diff and
removal body. Where git matches another of those spellings, as off macOS
for a decomposed name under a precomposed rule, the checkpoint stores
the file but it is still withheld; its counts remain.
Exec diagnostics now require an active checkpoint. Noninteractive runs
using the default interactive checkpoint policy must explicitly enable
checkpoints to receive this feedback. Visibility is checked in batches
of up to 256 files and stops once eight visible files are found; each
batch holds the event loop while the host resolves its paths, about 130
ms for a full batch on native Windows. Each file-check wait is one
second, and post-command diagnostics have a three-second wait budget;
past it, or when they fail as a whole, they are dropped and the
command's result is kept. These are caller budgets, not hard I/O
deadlines: running OS reads and inherited synchronous host callbacks
cannot be forcibly interrupted. Private worker capacity remains bounded.
Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took
0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on
slower hardware such a check can outlast the one-second wait and report
diagnostics unavailable. Python's GIL means concurrency does not promise
a CPU speedup.
Windows CI exercises the diagnostics layer with the real native
DirectExecutor. The existing CodeExecutor environment probe uses POSIX
quoting and already fails on Windows at the base; this PR neither
changes that probe nor claims end-to-end compatibility for it. Symlink
setup is skipped only if the host cannot create the fixture.
Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics`
for no post-command checks, or `pythonSyntaxNote` for no diagnostics,
under `plugins.config["code-flow"].tools`. The PR can also be reverted
as a unit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
"config",
|
|
|
|
|
[
|
|
|
|
|
{},
|
|
|
|
|
{"enabled": True},
|
|
|
|
|
{"enabled": True, "tools": {"enabled": False}},
|
|
|
|
|
{"tools": {"pythonLint": True, "execPythonDiagnostics": True}},
|
|
|
|
|
{"tools": {"enabled": False, "pythonLint": True, "execPythonDiagnostics": True}},
|
|
|
|
|
],
|
|
|
|
|
)
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
def test_a_slice_that_does_not_ask_for_the_face_contributes_nothing(tmp_path, config):
|
|
|
|
|
"""The flow and the face have one switch each: a product can keep the flow
|
|
|
|
|
and hand the tools back to the host."""
|
|
|
|
|
ctx = PluginContext(config=config, services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"))
|
|
|
|
|
for make in (
|
|
|
|
|
factories.make_read_file,
|
|
|
|
|
factories.make_write_file,
|
|
|
|
|
factories.make_edit_file,
|
|
|
|
|
factories.make_list_dir,
|
|
|
|
|
factories.make_glob,
|
|
|
|
|
factories.make_todo,
|
|
|
|
|
factories.make_exec,
|
|
|
|
|
):
|
|
|
|
|
assert make(ctx) is None
|
|
|
|
|
|
|
|
|
|
|
feat(*): add bounded Python diagnostics after code edits (#967)
## Summary
Raven-Code reports Python syntax errors after file edits but misses
unresolved names and equivalent feedback after shell edits. Add advisory
Pyflakes F821/F822/F823 checks to the existing tool result; optional
F401 notices remain off by default. No additional model call or code
execution is involved.
```text
write_file / edit_file
-> inspect final source, compare with a baseline that can be read and analyzed
-> retain syntax warnings; suppress existing name findings
exec
-> require checkpoint visibility for the working directory
-> read the changed files from the host's write record for the command
-> ask visibility in batches of 256, select at most 8, check up to 4 concurrently
-> append ordered diagnostics, preserving exit status and file metadata
```
Existing syntax failures remain visible after unrelated edits and
identical rewrites. A baseline that cannot be read, parsed or analyzed
is marked unknown; overwriting non-UTF-8 source still checks the known
replacement. Source nested too deeply for the parser or checker, and a
changed file that cannot be read, report diagnostics unavailable for
that file rather than passing as clean. Every check reads a file as the
bytes Python runs: text a write or edit stores is checked as its UTF-8
bytes, the content it replaced included, and a file from disk as its
bytes, each decoded as Python decodes source, by its coding declaration
or as UTF-8. Bytes that do not decode are reported as a syntax error at
the line where decoding stops, so a declaration the written text cannot
satisfy is reported however the file was changed. Noqa comments are read
as Ruff reads them, including after other comment directives and never
inside string literals: a file-level `ruff: noqa` / `flake8: noqa`
counts only on a line of its own, and a marker counts only when codes
follow its colon or, with no colon, whitespace, another comment or the
end of the comment follows it. Unlike Ruff, which takes the first
directive in a comment, every directive in it counts. The seeded tool
guide gives the syntax note's own advice. With no checkpoint or no
visibility verdict, exec diagnostics do not inspect changed file
contents; direct file checks remain available.
Pyflakes is imported when the tool face loads, not by whichever check
first needs it: imported lazily, its cold import fell inside that
check's one-second wait, and on a slow Windows runner the one test that
paid it reported diagnostics unavailable for a plain syntax error.
Private workers bound reads and checks to four active jobs per process.
Saturated capacity rejects new work without queuing, and
timeout/cancellation retains each slot until the actual work finishes.
Name checks are capped at 256 KiB of source and the syntax check at 2
MiB, and a check skipped for size says so in one line. Findings are
capped at eight per file, and exec diagnostic text at 6,000 characters
plus omission notices.
Runtime changes stay inside the Raven-Code code-flow plugin, except for
one fix to `CheckpointService.trackable`, the visibility verdict that
exec diagnostics and the host's exec write record both rely on. On
macOS, git precomposes a file name it reads off the disk before matching
an ignore rule, but takes the paths `trackable` sends to `git
check-ignore --stdin` as written. A file stored with a decomposed (NFD)
name under a `.gitignore` rule written precomposed (NFC) is therefore
left out of the checkpoint, yet main vouches for it, so the write
record's diff carries its text, and exec diagnostics would quote its
source lines to the model. Nor is git's precomposition NFC: it composes
only neighbouring characters, by Unicode 3.2's table, and reorders
nothing, so a name whose combining marks are out of canonical order
comes out as neither the name as given nor its NFC. `trackable` now asks
about each name as given, as NFC and in git's macOS spelling, and
withholds it when any spelling is ignored. Other shared
file/shell/snapshot implementations are unchanged.
Raven-Code has no separate dependency manifest, so Pyflakes uses the
existing root dependencies. It is the only added package; uv normalized
existing lockfile markers without changing other package versions.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- **Before:** on main, Raven-Code's file tools report Python syntax
errors only, and its exec reports nothing about the Python files a
command changed. On macOS, main's `CheckpointService.trackable` vouches
for a file stored as `secret_cafe` + U+0301 + `.py` under the rule
`secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the
checkpoint's staged tree leaves it out, yet its text reaches the exec
write record's stored diff.
- **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command
below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default
suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors
over 22 test ids: 21 of them also fail on the exact base in the same
environment, and the other, a remote process-tree cancel test that
missed its deadline under the suite's load, passes when run alone. Tests
cover actual worker overlap, retained capacity after
timeout/cancellation, host-tool isolation, hidden-file quotas,
visibility batching through the host's write-record wiring, baseline
handling, parser-limit failures, the size bounds and their skip notes,
undecodable bytes placed where decoding stops, coding declarations in
written and edited text, the baseline read staying on a worker, the
checker loaded with its module, the file each unavailable note names,
comment directives and their marker boundary, the tool guide's agreement
with the syntax note, portable commands in paths with spaces/non-ASCII
characters, and the checkpoint's verdict on decomposed names and on
names git spells as neither given nor NFC, held against its own staged
tree. Removing any of the three spellings from the query fails the cases
that pin it, as does removing the marker boundary, the parser-limit
report, the unknown-baseline handling, the syntax check past the
name-check bound and the decoding report, checking written text as its
stored bytes, the worker around the baseline read, loading the checker
with its module, or the file name in an unavailable note; restoring the
guide's earlier advice fails its agreement test. git's macOS spelling is
what libiconv's UTF-8-MAC conversion returns, which git precomposes
with: composing neighbours by Unicode 3.2's table reproduced it for all
281,928 names built from every decomposable character's decomposition
(reordered, partly composed, or with a mark added), and across 784,220
further mixed names and pairs no name was spelled outside the three
asked; the same held under Python 3.13 and 3.14. The decomposed-name
check asserts the macOS outcome only where git precomposes names, so it
also passes with `core.precomposeunicode = false` set through
`GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines
match Ruff 0.15.12, except the three that hold several directives in one
comment. Through the real `AgentLoop`, code-flow tool face,
`CodeExecutor` and checkpoint, with the working directory under `/tmp`,
under `$TMPDIR` and in a path with spaces and CJK characters, exec
diagnostics report created, modified and decomposed-name files, follow
`working_dir`, report an undecodable file at the line where decoding
stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold
`.gitignore`d files, including decomposed names and names git spells as
neither given nor NFC; across repeated runs on both interpreters, a file
near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a
baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600
ignored files plus one visible file take 73-90 ms with the event loop
held at most 20 ms. At this head CI passes the unit suite on
Linux/Python 3.12 (32619 tests over four shards), and its two Windows
steps pass the diagnostic files on native Windows/Python 3.13: 122
file/worker tests, and 27 exec diagnostic tests with 17 pre-existing
executor tests deselected; the symlink fixture skips only where the host
cannot create links.
```sh
env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3
uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py
uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py
uv run --no-sync lint-imports
uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
```
Native Windows CI commands:
```sh
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short
```
Ruff, format and type checks passed; all 10 import contracts kept; the
commit-message, large-file and source-language checks pass over the
branch. The launcher tests unset inherited search API keys to keep the
expected tool list deterministic.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes
does not activate checks in other agents. The change is reversible and
never fixes/removes code or rolls back edits. Static diagnostics can
need judgment for dynamic names, target Python versions, and project
lint configuration; only explicit noqa directives are honored. Passing
diagnostics do not replace tests.
The `trackable` fix reaches every exec write record, not only
Raven-Code: a file whose name, as given, as NFC or in git's macOS
spelling, matches an ignore rule keeps its counts and loses its diff and
removal body. Where git matches another of those spellings, as off macOS
for a decomposed name under a precomposed rule, the checkpoint stores
the file but it is still withheld; its counts remain.
Exec diagnostics now require an active checkpoint. Noninteractive runs
using the default interactive checkpoint policy must explicitly enable
checkpoints to receive this feedback. Visibility is checked in batches
of up to 256 files and stops once eight visible files are found; each
batch holds the event loop while the host resolves its paths, about 130
ms for a full batch on native Windows. Each file-check wait is one
second, and post-command diagnostics have a three-second wait budget;
past it, or when they fail as a whole, they are dropped and the
command's result is kept. These are caller budgets, not hard I/O
deadlines: running OS reads and inherited synchronous host callbacks
cannot be forcibly interrupted. Private worker capacity remains bounded.
Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took
0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on
slower hardware such a check can outlast the one-second wait and report
diagnostics unavailable. Python's GIL means concurrency does not promise
a CPU speedup.
Windows CI exercises the diagnostics layer with the real native
DirectExecutor. The existing CodeExecutor environment probe uses POSIX
quoting and already fails on Windows at the base; this PR neither
changes that probe nor claims end-to-end compatibility for it. Symlink
setup is skipped only if the host cannot create the fixture.
Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics`
for no post-command checks, or `pythonSyntaxNote` for no diagnostics,
under `plugins.config["code-flow"].tools`. The PR can also be reverted
as a unit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_loading_coding_tools_does_not_add_diagnostics_to_host_tools(tmp_path, monkeypatch):
|
|
|
|
|
from code_flow.tools.diagnostics import PythonDiagnostics
|
|
|
|
|
|
|
|
|
|
from raven.agent.tools.filesystem import WriteFileTool as TrunkWriteFileTool
|
|
|
|
|
from raven.agent.tools.shell import ExecTool as TrunkExecTool
|
|
|
|
|
|
|
|
|
|
calls = []
|
|
|
|
|
|
|
|
|
|
async def check(*args, **kwargs):
|
|
|
|
|
calls.append(args)
|
|
|
|
|
return "unexpected coding diagnostics"
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(PythonDiagnostics, "check", check)
|
|
|
|
|
writer = TrunkWriteFileTool(workspace=tmp_path)
|
|
|
|
|
out = await writer.execute(path=str(tmp_path / "host.py"), content="value = missing_binding\n")
|
|
|
|
|
assert "Successfully wrote" in str(out)
|
|
|
|
|
shell = TrunkExecTool(working_dir=str(tmp_path))
|
|
|
|
|
out = await shell.execute(command="echo other = another_missing > shell.py")
|
|
|
|
|
assert "Exit code: 0" in str(out)
|
|
|
|
|
assert calls == []
|
|
|
|
|
assert "another_missing" in (tmp_path / "shell.py").read_text()
|
|
|
|
|
|
|
|
|
|
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
# --- the fork's schema, the host's names as aliases ----------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_schemas_carry_the_forks_spelling(registry):
|
|
|
|
|
read = registry.get("read_file").parameters
|
|
|
|
|
assert read["required"] == ["file_path"] and "path" not in read["properties"]
|
|
|
|
|
edit = registry.get("edit_file").parameters
|
|
|
|
|
assert edit["required"] == ["file_path", "old_string", "new_string"]
|
|
|
|
|
assert "occurrence" in edit["properties"] and "old_text" not in edit["properties"]
|
|
|
|
|
write = registry.get("write_file").parameters
|
|
|
|
|
assert write["required"] == ["file_path", "content"]
|
|
|
|
|
assert registry.get("glob").name == "glob"
|
|
|
|
|
assert registry.get("find") is None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_hosts_spelling_still_runs_through_the_aliases(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "a.txt"
|
|
|
|
|
target.write_text("alpha beta\n")
|
|
|
|
|
assert "1| alpha beta" in _text(_run(registry.execute("read_file", {"path": str(target)})))
|
|
|
|
|
result = _run(registry.execute("edit_file", {"path": str(target), "old_text": "beta", "new_text": "gamma"}))
|
|
|
|
|
assert "Successfully edited" in _text(result)
|
|
|
|
|
assert target.read_text() == "alpha gamma\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_forks_spelling_wins_when_a_call_carries_both(tmp_path):
|
|
|
|
|
tool = EditFileTool(workspace=tmp_path)
|
|
|
|
|
cast = tool.cast_params({"path": "legacy.py", "file_path": "fork.py", "old_text": "a", "old_string": "b"})
|
|
|
|
|
assert cast == {"file_path": "fork.py", "old_string": "b"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- read before edit ---------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_editing_a_file_nobody_read_is_refused_until_it_is_read(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "b.txt"
|
|
|
|
|
target.write_text("one\n")
|
|
|
|
|
refused = _text(
|
|
|
|
|
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
|
|
|
|
|
)
|
|
|
|
|
assert refused.startswith("Error") and "have not read" in refused
|
|
|
|
|
assert target.read_text() == "one\n"
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
accepted = _run(registry.execute("edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
|
|
|
|
|
assert "Successfully edited" in _text(accepted)
|
|
|
|
|
assert target.read_text() == "two\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_write_counts_as_having_read_what_now_stands(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "c.txt"
|
|
|
|
|
_run(registry.execute("write_file", {"file_path": str(target), "content": "x = 1\n"}))
|
|
|
|
|
result = _run(registry.execute("edit_file", {"file_path": str(target), "old_string": "1", "new_string": "2"}))
|
|
|
|
|
assert "Successfully edited" in _text(result)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_read_rule_is_a_switch(tmp_path):
|
|
|
|
|
ctx = _ctx(tmp_path, {"enabled": True, "requireReadBeforeEdit": False})
|
|
|
|
|
tool = factories.make_edit_file(ctx)
|
|
|
|
|
target = tmp_path / "d.txt"
|
|
|
|
|
target.write_text("k\n")
|
|
|
|
|
assert "Successfully edited" in _text(_run(tool.execute(file_path=str(target), old_string="k", new_string="v")))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the syntax verdict ---------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_write_that_breaks_python_says_so_in_the_same_result(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "e.py"
|
|
|
|
|
result = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": "def f(:\n"})))
|
|
|
|
|
assert "Successfully wrote" in result
|
|
|
|
|
assert "syntax error" in result and "line 1" in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_an_edit_that_breaks_python_says_so_and_a_clean_one_stays_quiet(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "f.py"
|
|
|
|
|
target.write_text("x = 1\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
broken = _text(
|
|
|
|
|
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "1", "new_string": "("}))
|
|
|
|
|
)
|
|
|
|
|
assert "syntax error" in broken
|
|
|
|
|
fixed = _text(_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "(", "new_string": "2"})))
|
|
|
|
|
assert "Successfully edited" in fixed and "syntax error" not in fixed
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_verdict_is_python_only_and_a_switch(tmp_path):
|
|
|
|
|
ctx = _ctx(tmp_path, {"enabled": True})
|
|
|
|
|
write = factories.make_write_file(ctx)
|
|
|
|
|
assert "syntax" not in _text(_run(write.execute(file_path=str(tmp_path / "g.txt"), content="def f(:\n")))
|
|
|
|
|
quiet = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonSyntaxNote": False}))
|
|
|
|
|
assert "syntax" not in _text(_run(quiet.execute(file_path=str(tmp_path / "h.py"), content="def f(:\n")))
|
|
|
|
|
|
|
|
|
|
|
feat(*): add bounded Python diagnostics after code edits (#967)
## Summary
Raven-Code reports Python syntax errors after file edits but misses
unresolved names and equivalent feedback after shell edits. Add advisory
Pyflakes F821/F822/F823 checks to the existing tool result; optional
F401 notices remain off by default. No additional model call or code
execution is involved.
```text
write_file / edit_file
-> inspect final source, compare with a baseline that can be read and analyzed
-> retain syntax warnings; suppress existing name findings
exec
-> require checkpoint visibility for the working directory
-> read the changed files from the host's write record for the command
-> ask visibility in batches of 256, select at most 8, check up to 4 concurrently
-> append ordered diagnostics, preserving exit status and file metadata
```
Existing syntax failures remain visible after unrelated edits and
identical rewrites. A baseline that cannot be read, parsed or analyzed
is marked unknown; overwriting non-UTF-8 source still checks the known
replacement. Source nested too deeply for the parser or checker, and a
changed file that cannot be read, report diagnostics unavailable for
that file rather than passing as clean. Every check reads a file as the
bytes Python runs: text a write or edit stores is checked as its UTF-8
bytes, the content it replaced included, and a file from disk as its
bytes, each decoded as Python decodes source, by its coding declaration
or as UTF-8. Bytes that do not decode are reported as a syntax error at
the line where decoding stops, so a declaration the written text cannot
satisfy is reported however the file was changed. Noqa comments are read
as Ruff reads them, including after other comment directives and never
inside string literals: a file-level `ruff: noqa` / `flake8: noqa`
counts only on a line of its own, and a marker counts only when codes
follow its colon or, with no colon, whitespace, another comment or the
end of the comment follows it. Unlike Ruff, which takes the first
directive in a comment, every directive in it counts. The seeded tool
guide gives the syntax note's own advice. With no checkpoint or no
visibility verdict, exec diagnostics do not inspect changed file
contents; direct file checks remain available.
Pyflakes is imported when the tool face loads, not by whichever check
first needs it: imported lazily, its cold import fell inside that
check's one-second wait, and on a slow Windows runner the one test that
paid it reported diagnostics unavailable for a plain syntax error.
Private workers bound reads and checks to four active jobs per process.
Saturated capacity rejects new work without queuing, and
timeout/cancellation retains each slot until the actual work finishes.
Name checks are capped at 256 KiB of source and the syntax check at 2
MiB, and a check skipped for size says so in one line. Findings are
capped at eight per file, and exec diagnostic text at 6,000 characters
plus omission notices.
Runtime changes stay inside the Raven-Code code-flow plugin, except for
one fix to `CheckpointService.trackable`, the visibility verdict that
exec diagnostics and the host's exec write record both rely on. On
macOS, git precomposes a file name it reads off the disk before matching
an ignore rule, but takes the paths `trackable` sends to `git
check-ignore --stdin` as written. A file stored with a decomposed (NFD)
name under a `.gitignore` rule written precomposed (NFC) is therefore
left out of the checkpoint, yet main vouches for it, so the write
record's diff carries its text, and exec diagnostics would quote its
source lines to the model. Nor is git's precomposition NFC: it composes
only neighbouring characters, by Unicode 3.2's table, and reorders
nothing, so a name whose combining marks are out of canonical order
comes out as neither the name as given nor its NFC. `trackable` now asks
about each name as given, as NFC and in git's macOS spelling, and
withholds it when any spelling is ignored. Other shared
file/shell/snapshot implementations are unchanged.
Raven-Code has no separate dependency manifest, so Pyflakes uses the
existing root dependencies. It is the only added package; uv normalized
existing lockfile markers without changing other package versions.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- **Before:** on main, Raven-Code's file tools report Python syntax
errors only, and its exec reports nothing about the Python files a
command changed. On macOS, main's `CheckpointService.trackable` vouches
for a file stored as `secret_cafe` + U+0301 + `.py` under the rule
`secret_caf` + U+00E9 + `.py`: the user's own git ignores it and the
checkpoint's staged tree leaves it out, yet its text reaches the exec
write record's stored diff.
- **After:** on macOS 26.6.2 / arm64 with Apple Git 2.54.0, the command
below passes 364 tests on Python 3.12.13 and on 3.14.6. The full default
suite on 3.12.13 has 32283 passed, 169 skipped, 20 failed and 3 errors
over 22 test ids: 21 of them also fail on the exact base in the same
environment, and the other, a remote process-tree cancel test that
missed its deadline under the suite's load, passes when run alone. Tests
cover actual worker overlap, retained capacity after
timeout/cancellation, host-tool isolation, hidden-file quotas,
visibility batching through the host's write-record wiring, baseline
handling, parser-limit failures, the size bounds and their skip notes,
undecodable bytes placed where decoding stops, coding declarations in
written and edited text, the baseline read staying on a worker, the
checker loaded with its module, the file each unavailable note names,
comment directives and their marker boundary, the tool guide's agreement
with the syntax note, portable commands in paths with spaces/non-ASCII
characters, and the checkpoint's verdict on decomposed names and on
names git spells as neither given nor NFC, held against its own staged
tree. Removing any of the three spellings from the query fails the cases
that pin it, as does removing the marker boundary, the parser-limit
report, the unknown-baseline handling, the syntax check past the
name-check bound and the decoding report, checking written text as its
stored bytes, the worker around the baseline read, loading the checker
with its module, or the file name in an unavailable note; restoring the
guide's earlier advice fails its agreement test. git's macOS spelling is
what libiconv's UTF-8-MAC conversion returns, which git precomposes
with: composing neighbours by Unicode 3.2's table reproduced it for all
281,928 names built from every decomposable character's decomposition
(reordered, partly composed, or with a mark added), and across 784,220
further mixed names and pairs no name was spelled outside the three
asked; the same held under Python 3.13 and 3.14. The decomposed-name
check asserts the macOS outcome only where git precomposes names, so it
also passes with `core.precomposeunicode = false` set through
`GIT_CONFIG_GLOBAL`. Across 102 noqa comment variants the reported lines
match Ruff 0.15.12, except the three that hold several directives in one
comment. Through the real `AgentLoop`, code-flow tool face,
`CodeExecutor` and checkpoint, with the working directory under `/tmp`,
under `$TMPDIR` and in a path with spaces and CJK characters, exec
diagnostics report created, modified and decomposed-name files, follow
`working_dir`, report an undecodable file at the line where decoding
stops, a syntax error past 256 KiB and a skip past 2 MiB, and withhold
`.gitignore`d files, including decomposed names and names git spells as
neither given nor NFC; across repeated runs on both interpreters, a file
near the 256 KiB cap is checked in 52-135 ms (117-128 ms against a
baseline), the syntax of a file near 2 MiB in 318-371 ms, and 600
ignored files plus one visible file take 73-90 ms with the event loop
held at most 20 ms. At this head CI passes the unit suite on
Linux/Python 3.12 (32619 tests over four shards), and its two Windows
steps pass the diagnostic files on native Windows/Python 3.13: 122
file/worker tests, and 27 exec diagnostic tests with 17 pre-existing
executor tests deselected; the symlink fixture skips only where the host
cannot create links.
```sh
env -u SERPER_API_KEY -u RAVEN_MIRRORED_SERPER_API_KEY uv run --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_flow_exec.py tests/test_agents_code_flow_plugin.py tests/test_agents_code_flow_read_state.py tests/test_agents_code_launcher.py tests/test_workdir_snapshot.py tests/test_shell_command_writes.py tests/test_runtime_checkpoint.py tests/test_trajectory_upload.py::test_blocked_upload_hits_the_deadline -o addopts= -q --tb=short --maxfail=3
uv run --no-sync ruff check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py tests/test_agents_code_diagnostic_workers.py
uv run --no-sync ty check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py
uv run --no-sync lint-imports
uv run --no-sync ruff format --check raven/agent/loop/checkpoint.py agents/raven-code/plugins/code-flow/code_flow/tools agents/raven-code/plugins/code-flow/code_flow/config.py tests/test_runtime_checkpoint.py tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
```
Native Windows CI commands:
```sh
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_diagnostic_workers.py tests/test_agents_code_tools_plugin.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/test_agents_code_flow_exec.py -k test_exec_ -q -n 0 --tb=short
```
Ruff, format and type checks passed; all 10 import contracts kept; the
commit-message, large-file and source-language checks pass over the
branch. The launcher tests unset inherited search API keys to keep the
expected tool list deterministic.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Scope: Raven-Code's enabled code-flow tool face; installing Pyflakes
does not activate checks in other agents. The change is reversible and
never fixes/removes code or rolls back edits. Static diagnostics can
need judgment for dynamic names, target Python versions, and project
lint configuration; only explicit noqa directives are honored. Passing
diagnostics do not replace tests.
The `trackable` fix reaches every exec write record, not only
Raven-Code: a file whose name, as given, as NFC or in git's macOS
spelling, matches an ignore rule keeps its counts and loses its diff and
removal body. Where git matches another of those spellings, as off macOS
for a decomposed name under a precomposed rule, the checkpoint stores
the file but it is still withheld; its counts remain.
Exec diagnostics now require an active checkpoint. Noninteractive runs
using the default interactive checkpoint policy must explicitly enable
checkpoints to receive this feedback. Visibility is checked in batches
of up to 256 files and stops once eight visible files are found; each
batch holds the event loop while the host resolves its paths, about 130
ms for a full batch on native Windows. Each file-check wait is one
second, and post-command diagnostics have a three-second wait budget;
past it, or when they fail as a whole, they are dropped and the
command's result is kept. These are caller budgets, not hard I/O
deadlines: running OS reads and inherited synchronous host callbacks
cannot be forcibly interrupted. Private worker capacity remains bounded.
Between 256 KiB and 2 MiB only syntax is checked: near 2 MiB that took
0.3-0.4 s and about 190 MiB here, up to four checks run at once, and on
slower hardware such a check can outlast the one-second wait and report
diagnostics unavailable. Python's GIL means concurrency does not promise
a CPU speedup.
Windows CI exercises the diagnostics layer with the real native
DirectExecutor. The existing CodeExecutor environment probe uses POSIX
quoting and already fails on Windows at the base; this PR neither
changes that probe nor claims end-to-end compatibility for it. Symlink
setup is skipped only if the host cannot create the fixture.
Disable `pythonLint` for syntax-only feedback, `execPythonDiagnostics`
for no post-command checks, or `pythonSyntaxNote` for no diagnostics,
under `plugins.config["code-flow"].tools`. The PR can also be reverted
as a unit.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-18 20:21:04 +08:00
|
|
|
def test_undefined_name_feedback_identifies_the_binding_and_keeps_the_edit(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "undefined.py"
|
|
|
|
|
source = "def double():\n return missing_value * 2\n"
|
|
|
|
|
result = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
|
|
|
|
|
assert "F821" in result and "missing_value" in result and "line 2" in result
|
|
|
|
|
assert "spelling, scope, import, or assignment order" in result
|
|
|
|
|
assert target.read_text() == source
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_existing_name_errors_do_not_reappear_when_unrelated_lines_move(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "existing.py"
|
|
|
|
|
target.write_text("old_result = preexisting_missing\nvalue = 1\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
result = _text(
|
|
|
|
|
_run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"edit_file",
|
|
|
|
|
{
|
|
|
|
|
"file_path": str(target),
|
|
|
|
|
"old_string": "old_result =",
|
|
|
|
|
"new_string": "\nold_result =",
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
assert "F821" not in result
|
|
|
|
|
result = _text(
|
|
|
|
|
_run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"edit_file",
|
|
|
|
|
{
|
|
|
|
|
"file_path": str(target),
|
|
|
|
|
"old_string": "value = 1",
|
|
|
|
|
"new_string": "value = newly_missing",
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
assert "F821" in result and "newly_missing" in result
|
|
|
|
|
assert "preexisting_missing" not in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("tool_name", ["edit_file", "write_file"])
|
|
|
|
|
def test_existing_syntax_error_is_still_reported_after_an_unrelated_edit(registry, tmp_path, tool_name):
|
|
|
|
|
target = tmp_path / "still_broken.py"
|
|
|
|
|
source = "def f(:\n return 1\nvalue = 2\n"
|
|
|
|
|
_run(registry.execute("write_file", {"file_path": str(target), "content": source}))
|
|
|
|
|
params = {"file_path": str(target)}
|
|
|
|
|
if tool_name == "edit_file":
|
|
|
|
|
params.update(old_string="value = 2", new_string="value = 3")
|
|
|
|
|
else:
|
|
|
|
|
params["content"] = source.replace("value = 2", "value = 3")
|
|
|
|
|
result = _text(_run(registry.execute(tool_name, params)))
|
|
|
|
|
assert "Python syntax error" in result
|
|
|
|
|
assert "new findings compared" not in result
|
|
|
|
|
assert target.read_text() == source.replace("value = 2", "value = 3")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_fixing_syntax_does_not_label_old_name_errors_as_new(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "syntax_fixed.py"
|
|
|
|
|
source = "value = old_missing\ndef f(:\n return 1\n"
|
|
|
|
|
_run(registry.execute("write_file", {"file_path": str(target), "content": source}))
|
|
|
|
|
result = _text(
|
|
|
|
|
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "f(:", "new_string": "f():"}))
|
|
|
|
|
)
|
|
|
|
|
assert "F821" in result and "old_missing" in result
|
|
|
|
|
assert "baseline is unknown" in result and "new findings compared" not in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("source, expected", [("def f(:\n", "Python syntax error"), ("x = missing_name\n", "F821")])
|
|
|
|
|
def test_overwriting_non_utf8_source_still_checks_the_new_content(registry, tmp_path, source, expected):
|
|
|
|
|
target = tmp_path / "legacy.py"
|
|
|
|
|
target.write_bytes('# coding: gbk\nname = "\u4e2d"\n'.encode("gbk"))
|
|
|
|
|
result = _run(registry.execute("write_file", {"file_path": str(target), "content": source}))
|
|
|
|
|
assert result.file_change is None
|
|
|
|
|
assert expected in _text(result) and "baseline is unknown" in _text(result)
|
|
|
|
|
assert target.read_text(encoding="utf-8") == source
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_import_notices_are_optional_and_explain_side_effects(tmp_path):
|
|
|
|
|
source = "import os\n"
|
|
|
|
|
quiet = factories.make_write_file(_ctx(tmp_path, {"enabled": True}))
|
|
|
|
|
result = _text(_run(quiet.execute(file_path=str(tmp_path / "quiet_import.py"), content=source)))
|
|
|
|
|
assert "F401" not in result
|
|
|
|
|
enabled = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonUnusedImports": True}))
|
|
|
|
|
target = tmp_path / "notice_import.py"
|
|
|
|
|
result = _text(_run(enabled.execute(file_path=str(target), content=source)))
|
|
|
|
|
assert "NOTICE" in result and "F401" in result and "side effects" in result
|
|
|
|
|
assert target.read_text() == source
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_name_checks_handle_nested_scopes_and_public_exports(registry, tmp_path):
|
|
|
|
|
source = (
|
|
|
|
|
"from os import path\n__all__ = ['path']\n"
|
|
|
|
|
"def outer(value):\n def inner():\n return value + len([])\n return inner\n"
|
|
|
|
|
)
|
|
|
|
|
result = _text(
|
|
|
|
|
_run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"write_file",
|
|
|
|
|
{
|
|
|
|
|
"file_path": str(tmp_path / "scopes.py"),
|
|
|
|
|
"content": source,
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
assert "Python diagnostics" not in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_name_checks_can_be_disabled_without_disabling_syntax_feedback(tmp_path):
|
|
|
|
|
tool = factories.make_write_file(_ctx(tmp_path, {"enabled": True, "pythonLint": False}))
|
|
|
|
|
target = tmp_path / "syntax_only.py"
|
|
|
|
|
assert "F821" not in _text(_run(tool.execute(file_path=str(target), content="result = missing\n")))
|
|
|
|
|
assert "syntax error" in _text(_run(tool.execute(file_path=str(target), content="def f(:\n")))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_append_checks_the_complete_file_and_suppresses_preexisting_errors(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "append.py"
|
|
|
|
|
target.write_text("old = existing_missing\n")
|
|
|
|
|
result = _text(
|
|
|
|
|
_run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"write_file",
|
|
|
|
|
{
|
|
|
|
|
"file_path": str(target),
|
|
|
|
|
"content": "new = added_missing\n",
|
|
|
|
|
"mode": "append",
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
assert "added_missing" in result and "existing_missing" not in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("source", "code"),
|
|
|
|
|
[
|
|
|
|
|
("__all__ = ['missing_export']\n", "F822"),
|
|
|
|
|
("value = 1\ndef f():\n print(value)\n value = 2\n", "F823"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
def test_name_feedback_covers_exports_and_use_before_local_assignment(registry, tmp_path, source, code):
|
|
|
|
|
target = tmp_path / "binding.py"
|
|
|
|
|
out = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
|
|
|
|
|
assert code in out
|
|
|
|
|
assert target.read_text() == source
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("source", "warns"),
|
|
|
|
|
[
|
|
|
|
|
("value = injected_name # noqa\n", False),
|
|
|
|
|
("value = injected_name # noqa: F821, F401\n", False),
|
|
|
|
|
("value = injected_name # noqa: F401\n", True),
|
|
|
|
|
("value = injected_name # type: ignore[name-defined] # noqa: F821\n", False),
|
|
|
|
|
("value = injected_name # type: ignore[name-defined] # noqa: F401\n", True),
|
|
|
|
|
("value = injected_name # noqa: F401 # noqa: F821\n", False),
|
|
|
|
|
("value = injected_name # noqa: F401 # noqa\n", False),
|
|
|
|
|
("# ruff: noqa: F821\nvalue = injected_name\n", False),
|
|
|
|
|
("# flake8: noqa\nvalue = injected_name\n", False),
|
|
|
|
|
("value = injected_name # ruff: noqa\nother = second_name\n", True),
|
|
|
|
|
("value = injected_name # ruff: noqa\n", True),
|
|
|
|
|
("value = 1 # TODO # flake8: noqa\nother = injected_name\n", True),
|
|
|
|
|
("label = '# noqa'; value = injected_name\n", True),
|
|
|
|
|
("label = '# ruff: noqa'\nvalue = injected_name\n", True),
|
|
|
|
|
("value = injected_name # see https://example.com/page#noqa-section\n", True),
|
|
|
|
|
('# Use "# ruff: noqa" only for generated files\nvalue = injected_name\n', True),
|
|
|
|
|
("value = injected_name # noqa: needed for registration\n", True),
|
|
|
|
|
("value = injected_name # noqa#reason\n", False),
|
|
|
|
|
("# ruff: noqa: generated file\nvalue = injected_name\n", True),
|
|
|
|
|
("# ruff : noqa\nvalue = injected_name\n", False),
|
|
|
|
|
("# Ruff: noqa\nvalue = injected_name\n", True),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
def test_explicit_noqa_comments_suppress_only_the_requested_checks(registry, tmp_path, source, warns):
|
|
|
|
|
target = tmp_path / "ignored.py"
|
|
|
|
|
out = _text(_run(registry.execute("write_file", {"file_path": str(target), "content": source})))
|
|
|
|
|
assert ("WARNING" in out) is warns
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_syntax_errors_are_not_hidden_by_noqa(registry, tmp_path):
|
|
|
|
|
source = "# ruff: noqa\ndef f(:\n"
|
|
|
|
|
out = _text(_run(registry.execute("write_file", {"file_path": str(tmp_path / "syntax.py"), "content": source})))
|
|
|
|
|
assert "Python syntax error" in out and "project's Python version" in out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_tool_guide_gives_the_syntax_note_s_advice():
|
|
|
|
|
"""The note and the seeded tool guide reach the model together, so the
|
|
|
|
|
guide must not tell it to fix outright what the note asks it to check
|
|
|
|
|
against the project's Python version first."""
|
|
|
|
|
from code_flow.tools.diagnostics import PythonDiagnostics
|
|
|
|
|
|
|
|
|
|
note = PythonDiagnostics().note(Path("broken.py"), "def f(:\n", before=None)
|
|
|
|
|
guide = " ".join((PLUGIN_DIR / "prompts" / "TOOLS_CODE.md").read_text(encoding="utf-8").split())
|
|
|
|
|
advice = "check the project's Python version before changing newer syntax"
|
|
|
|
|
assert advice in note and advice in guide
|
|
|
|
|
assert "fix it before moving on" not in guide
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_name_checker_is_loaded_with_the_module_not_by_the_first_check():
|
|
|
|
|
"""A cold import of the checker inside a check's wait can outlast the wait
|
|
|
|
|
on a cold runner, which reported diagnostics unavailable for a file with a
|
|
|
|
|
plain syntax error. It is paid when the module loads instead. A fresh
|
|
|
|
|
interpreter, so no earlier test's check has already imported it."""
|
|
|
|
|
probe = (
|
|
|
|
|
"import sys; sys.path.insert(0, sys.argv[1]); import code_flow.tools.diagnostics; "
|
|
|
|
|
"print('pyflakes.checker' in sys.modules)"
|
|
|
|
|
)
|
|
|
|
|
done = subprocess.run([sys.executable, "-I", "-c", probe, str(PLUGIN_DIR)], capture_output=True, text=True)
|
|
|
|
|
assert done.returncode == 0, done.stderr
|
|
|
|
|
assert done.stdout.strip() == "True"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_checker_failure_does_not_turn_a_successful_write_into_a_failure(registry, tmp_path, monkeypatch):
|
|
|
|
|
from pyflakes import checker
|
|
|
|
|
|
|
|
|
|
def broken(*args, **kwargs):
|
|
|
|
|
raise RuntimeError("checker broke")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(checker, "Checker", broken)
|
|
|
|
|
target = tmp_path / "kept.py"
|
|
|
|
|
result = _run(registry.execute("write_file", {"file_path": str(target), "content": "value = 1\n"}))
|
|
|
|
|
assert "diagnostics unavailable (RuntimeError)" in _text(result)
|
|
|
|
|
assert result.file_change.after == target.read_text() == "value = 1\n"
|
|
|
|
|
assert "Successfully wrote" in _text(result)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("error", [RecursionError, MemoryError])
|
|
|
|
|
def test_source_the_parser_cannot_finish_reports_diagnostics_unavailable(registry, tmp_path, monkeypatch, error):
|
|
|
|
|
"""Python 3.12 raises RecursionError building the syntax tree of deeply
|
|
|
|
|
nested source (``compile`` too); that is no verdict, not a clean file."""
|
|
|
|
|
from code_flow.tools import diagnostics
|
|
|
|
|
|
|
|
|
|
def too_deep(*args, **kwargs):
|
|
|
|
|
raise error("nested too deeply")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=too_deep))
|
|
|
|
|
target = tmp_path / "deep.py"
|
|
|
|
|
result = _run(registry.execute("write_file", {"file_path": str(target), "content": "value = 1\n"}))
|
|
|
|
|
assert f"diagnostics unavailable ({error.__name__})" in _text(result)
|
|
|
|
|
assert target.read_text() == "value = 1\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("error", [RecursionError, MemoryError])
|
|
|
|
|
def test_replaced_source_the_parser_cannot_finish_is_an_unknown_baseline(registry, tmp_path, monkeypatch, error):
|
|
|
|
|
from code_flow.tools import diagnostics
|
|
|
|
|
|
|
|
|
|
parse = diagnostics.ast.parse
|
|
|
|
|
|
|
|
|
|
def deep_before(source, *args, **kwargs):
|
|
|
|
|
if "nested_before" in source:
|
|
|
|
|
raise error("nested too deeply")
|
|
|
|
|
return parse(source, *args, **kwargs)
|
|
|
|
|
|
|
|
|
|
target = tmp_path / "replaced.py"
|
|
|
|
|
target.write_text("value = nested_before\n")
|
|
|
|
|
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=deep_before))
|
|
|
|
|
args = {"file_path": str(target), "content": "value = newly_missing\n"}
|
|
|
|
|
result = _text(_run(registry.execute("write_file", args)))
|
|
|
|
|
assert "newly_missing" in result and "baseline is unknown" in result
|
|
|
|
|
assert "new findings compared" not in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_identical_rewrite_does_not_repeat_diagnostics(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "same.py"
|
|
|
|
|
args = {"file_path": str(target), "content": "value = missing\n"}
|
|
|
|
|
assert "F821" in _text(_run(registry.execute("write_file", args)))
|
|
|
|
|
assert "Python diagnostics" not in _text(_run(registry.execute("write_file", args)))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_identical_rewrite_still_reports_a_syntax_error(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "same_broken.py"
|
|
|
|
|
args = {"file_path": str(target), "content": "def f(:\n return 1\n"}
|
|
|
|
|
assert "Python syntax error" in _text(_run(registry.execute("write_file", args)))
|
|
|
|
|
result = _text(_run(registry.execute("write_file", args)))
|
|
|
|
|
assert result.startswith("File unchanged:") and "Python syntax error" in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_diagnostics_bound_source_and_findings_without_executing_code(tmp_path):
|
|
|
|
|
from code_flow.tools.diagnostics import MAX_FINDINGS, MAX_SOURCE_BYTES, MAX_SYNTAX_BYTES, PythonDiagnostics
|
|
|
|
|
|
|
|
|
|
target = tmp_path / "source.py"
|
|
|
|
|
sentinel = tmp_path / "must_not_exist"
|
|
|
|
|
text = f"open({str(sentinel)!r}, 'w').close()\n"
|
|
|
|
|
diagnostics = PythonDiagnostics()
|
|
|
|
|
assert diagnostics.note(target, text, before=None) == ""
|
|
|
|
|
assert not sentinel.exists()
|
|
|
|
|
assert diagnostics.note(target, "#" * (MAX_SOURCE_BYTES + 1), before=None) == (
|
|
|
|
|
f"\nPython name checks skipped for {target}: the source is over 256 KiB, so only its syntax was checked."
|
|
|
|
|
)
|
|
|
|
|
assert diagnostics.note(target, "#" * (MAX_SYNTAX_BYTES + 1), before=None) == (
|
|
|
|
|
f"\nPython diagnostics skipped for {target}: the source is over 2 MiB. Run targeted project checks."
|
|
|
|
|
)
|
|
|
|
|
assert PythonDiagnostics(lint=False).note(target, "#" * (MAX_SOURCE_BYTES + 1), before=None) == ""
|
|
|
|
|
note = diagnostics.note(target, "\n".join(f"value_{i} = missing_{i}" for i in range(20)), before=None)
|
|
|
|
|
assert note.count("WARNING:") == MAX_FINDINGS
|
|
|
|
|
assert "12 more findings omitted" in note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("tool", ["write_file", "edit_file"])
|
|
|
|
|
def test_a_syntax_error_past_the_name_check_bound_is_still_reported(registry, tmp_path, tool):
|
|
|
|
|
"""Name checks stop at 256 KiB and the syntax check runs on past it, so an
|
|
|
|
|
edit that breaks one of a project's large modules still hears about it."""
|
|
|
|
|
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES
|
|
|
|
|
|
|
|
|
|
target = tmp_path / "large.py"
|
|
|
|
|
rows = MAX_SOURCE_BYTES // len("value = 1\n") + 1
|
|
|
|
|
body = "value = 1\n" * rows
|
|
|
|
|
if tool == "write_file":
|
|
|
|
|
args = {"file_path": str(target), "content": body + "def broken(:\n"}
|
|
|
|
|
else:
|
|
|
|
|
clean = {"file_path": str(target), "content": body + "def kept():\n pass\n"}
|
|
|
|
|
assert "name checks skipped" in _text(_run(registry.execute("write_file", clean)))
|
|
|
|
|
args = {"file_path": str(target), "old_string": "def kept():", "new_string": "def broken(:"}
|
|
|
|
|
result = _text(_run(registry.execute(tool, args)))
|
|
|
|
|
assert f"WARNING: line {rows + 1}, column 12: Python syntax error" in result
|
|
|
|
|
assert "name checks skipped" not in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_an_append_python_cannot_decode_is_reported_where_decoding_stops(registry, tmp_path):
|
|
|
|
|
"""UTF-8 appended to a file that declares GBK: Python refuses the file, so
|
|
|
|
|
the append hears that, at the line Python cannot decode."""
|
|
|
|
|
target = tmp_path / "legacy.py"
|
|
|
|
|
target.write_bytes('# coding: gbk\nname = "\u4e2d"\n'.encode("gbk"))
|
|
|
|
|
args = {"file_path": str(target), "content": 'label = "\u4e2d"\ndef broken(:\n', "mode": "append"}
|
|
|
|
|
result = _text(_run(registry.execute("write_file", args)))
|
|
|
|
|
assert "line 3, column 12: Python syntax error: 'gbk' codec can't decode byte 0xad" in result
|
|
|
|
|
assert "new findings compared with the content before this call" in result
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("raw", "place", "message"),
|
|
|
|
|
[
|
|
|
|
|
# A declared encoding, where Python's parser reports line 0.
|
|
|
|
|
(
|
|
|
|
|
"# -*- coding: gbk -*-\nx = 1\nname = '\u4e2d\u6587'\n".encode(),
|
|
|
|
|
"line 3, column 11",
|
|
|
|
|
"'gbk' codec can't decode",
|
|
|
|
|
),
|
|
|
|
|
# UTF-8 by default, a comment included, which Python 3.12's parser passes.
|
|
|
|
|
(b"a = 1\nb = 2\n# caf\xe9\n", "line 3, column 6", "'utf-8' codec can't decode byte 0xe9"),
|
|
|
|
|
# After a BOM, which its codec strips before decoding the rest.
|
|
|
|
|
(b"\xef\xbb\xbfa = 1\nb = '\xff'\n", "line 2, column 6", "'utf-8' codec can't decode byte 0xff"),
|
|
|
|
|
# The two lines a declaration may take are read before anything is decoded.
|
|
|
|
|
(b"name = 'caf\xe9'\n", "line 1, column 1", "invalid or missing encoding declaration"),
|
|
|
|
|
(b"#!/usr/bin/env python\n# coding: nope\nx = 1\n", "line 2, column 1", "unknown encoding: nope"),
|
|
|
|
|
],
|
|
|
|
|
ids=["declared-encoding", "utf-8-comment", "after-a-bom", "first-line", "unknown-declaration"],
|
|
|
|
|
)
|
|
|
|
|
def test_bytes_python_cannot_decode_are_a_syntax_error_where_decoding_stops(tmp_path, raw, place, message):
|
|
|
|
|
from code_flow.tools.diagnostics import PythonDiagnostics
|
|
|
|
|
|
|
|
|
|
target = tmp_path / "undecodable.py"
|
|
|
|
|
target.write_bytes(raw)
|
|
|
|
|
note = _run(PythonDiagnostics().check(target, before=None))
|
|
|
|
|
assert f"WARNING: {place}: Python syntax error: {message}" in note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_finding_already_there_stays_existing_under_a_coding_declaration(registry, tmp_path):
|
|
|
|
|
"""The content an edit replaced is decoded as the content it wrote is, so
|
|
|
|
|
a finding on a line the declaration decodes the same way both times is
|
|
|
|
|
an existing finding, not a new one."""
|
|
|
|
|
target = tmp_path / "declared.py"
|
|
|
|
|
body = "# coding: latin-1\nlabel = 'caf\u00e9' + missing_one\nvalue = 1\n"
|
|
|
|
|
assert "missing_one" in _text(_run(registry.execute("write_file", {"file_path": str(target), "content": body})))
|
|
|
|
|
args = {"file_path": str(target), "old_string": "value = 1", "new_string": "value = 2"}
|
|
|
|
|
edited = _text(_run(registry.execute("edit_file", args)))
|
|
|
|
|
assert "Successfully edited" in edited and "missing_one" not in edited
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_replaced_text_over_the_name_check_bound_in_bytes_is_an_unknown_baseline(tmp_path):
|
|
|
|
|
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES, PythonDiagnostics
|
|
|
|
|
|
|
|
|
|
# Fewer characters than the bound, more bytes once stored.
|
|
|
|
|
before = "# " + "\u00e9" * (MAX_SOURCE_BYTES // 2)
|
|
|
|
|
assert len(before) <= MAX_SOURCE_BYTES < len(before.encode("utf-8"))
|
|
|
|
|
note = PythonDiagnostics().note(tmp_path / "wide.py", "value = missing_name\n", before=before)
|
|
|
|
|
assert "missing_name" in note and "baseline is unknown" in note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_text_is_checked_as_the_utf8_bytes_the_file_tool_stores(tmp_path):
|
|
|
|
|
"""Each of these compiles when the string alone is parsed and not as the
|
|
|
|
|
bytes on disk, which is what Python runs."""
|
|
|
|
|
from code_flow.tools.diagnostics import PythonDiagnostics
|
|
|
|
|
|
|
|
|
|
text = "# coding: ascii\nname = 'caf\u00e9'\n"
|
|
|
|
|
note = _run(PythonDiagnostics().check(tmp_path / "declared.py", text=text, before=None))
|
|
|
|
|
assert "line 2, column 12: Python syntax error: 'ascii' codec" in note
|
|
|
|
|
declared = "# coding: latin-1\nname = 'caf\u00e9'\nvalue = missing_name\n"
|
|
|
|
|
note = _run(PythonDiagnostics().check(tmp_path / "latin.py", text=declared, before=None))
|
|
|
|
|
assert "line 3, column 9: F821" in note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_checked_file_is_decoded_by_its_declaration_and_read_to_one_past_the_bound(tmp_path):
|
|
|
|
|
from code_flow.tools.diagnostics import MAX_SYNTAX_BYTES, PythonDiagnostics, read_source
|
|
|
|
|
|
|
|
|
|
target = tmp_path / "encoded.py"
|
|
|
|
|
target.write_bytes(b"# coding: latin-1\nname = 'caf\xe9'\nvalue = missing_name\n")
|
|
|
|
|
assert "WARNING: line 3, column 9: F821" in _run(PythonDiagnostics().check(target, before=None))
|
|
|
|
|
target.write_bytes(b"#" * (MAX_SYNTAX_BYTES + 5))
|
|
|
|
|
assert len(read_source(target)) == MAX_SYNTAX_BYTES + 1
|
|
|
|
|
assert "diagnostics skipped" in _run(PythonDiagnostics().check(target, before=None))
|
|
|
|
|
for absent in (tmp_path / "absent.py", tmp_path):
|
|
|
|
|
with pytest.raises(FileNotFoundError):
|
|
|
|
|
read_source(absent)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_baseline_is_read_as_bytes_and_bounded(tmp_path):
|
|
|
|
|
from code_flow.tools.diagnostics import MAX_SOURCE_BYTES, UNKNOWN, read_python
|
|
|
|
|
|
|
|
|
|
target = tmp_path / "encoded.py"
|
|
|
|
|
assert read_python(target) is None
|
|
|
|
|
raw = b"# coding: latin-1\nname = 'caf\xe9'\n"
|
|
|
|
|
target.write_bytes(raw)
|
|
|
|
|
assert read_python(target) == raw
|
|
|
|
|
target.write_bytes(b"x" * (MAX_SOURCE_BYTES + 1))
|
|
|
|
|
assert read_python(target) is UNKNOWN
|
|
|
|
|
assert read_python(tmp_path) is UNKNOWN
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_a_baseline_read_runs_on_a_worker_and_gives_up_at_the_wait(tmp_path, monkeypatch):
|
|
|
|
|
"""The content an append replaces is read as every diagnostic read is, on
|
|
|
|
|
a diagnostic worker: a stalled read leaves the event loop free and the
|
|
|
|
|
baseline unknown once the wait runs out."""
|
|
|
|
|
import time
|
|
|
|
|
|
|
|
|
|
from code_flow.tools import diagnostics
|
|
|
|
|
|
|
|
|
|
release = threading.Event()
|
|
|
|
|
readers = []
|
|
|
|
|
|
|
|
|
|
def stalled(path):
|
|
|
|
|
readers.append(threading.get_ident())
|
|
|
|
|
release.wait(5)
|
|
|
|
|
return b""
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(diagnostics, "read_python", stalled)
|
|
|
|
|
monkeypatch.setattr(diagnostics, "CHECK_WAIT_SECONDS", 0.05)
|
|
|
|
|
try:
|
|
|
|
|
started = time.monotonic()
|
|
|
|
|
assert await diagnostics.read_before(tmp_path / "slow.py") is diagnostics.UNKNOWN
|
|
|
|
|
assert time.monotonic() - started < 1
|
|
|
|
|
assert readers and readers[0] != threading.get_ident()
|
|
|
|
|
finally:
|
|
|
|
|
release.set()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize("tool", ["write_file", "edit_file"])
|
|
|
|
|
def test_a_coding_declaration_applies_to_the_text_a_write_or_edit_stores(registry, tmp_path, tool):
|
|
|
|
|
"""The file tool stores its text as UTF-8, so a declaration in that text
|
|
|
|
|
governs the file Python reads: content the declaration cannot decode is
|
|
|
|
|
reported, as it is for the same bytes read back after an append."""
|
|
|
|
|
target = tmp_path / "declared.py"
|
|
|
|
|
body = "# coding: ascii\nname = 'x'\n"
|
|
|
|
|
if tool == "write_file":
|
|
|
|
|
args = {"file_path": str(target), "content": body.replace("'x'", "'caf\u00e9'")}
|
|
|
|
|
else:
|
|
|
|
|
assert "Successfully" in _text(
|
|
|
|
|
_run(registry.execute("write_file", {"file_path": str(target), "content": body}))
|
|
|
|
|
)
|
|
|
|
|
args = {"file_path": str(target), "old_string": "'x'", "new_string": "'caf\u00e9'"}
|
|
|
|
|
result = _text(_run(registry.execute(tool, args)))
|
|
|
|
|
assert "line 2, column 12: Python syntax error: 'ascii' codec can't decode byte 0xc3" in result
|
|
|
|
|
with pytest.raises(SyntaxError):
|
|
|
|
|
compile(target.read_bytes(), str(target), "exec")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_an_unavailable_note_names_its_file(tmp_path, monkeypatch):
|
|
|
|
|
"""Post-command checks append one note per changed file, so each
|
|
|
|
|
unavailable note says which file it is about, from a failed read and a
|
|
|
|
|
failed parse alike."""
|
|
|
|
|
from code_flow.tools import diagnostics
|
|
|
|
|
|
|
|
|
|
unreadable = tmp_path / "locked.py"
|
|
|
|
|
unreadable.write_text("value = 1\n")
|
|
|
|
|
deep = tmp_path / "deep.py"
|
|
|
|
|
deep.write_text("value = 1\n")
|
|
|
|
|
read = diagnostics.read_source
|
|
|
|
|
|
|
|
|
|
def refused(path):
|
|
|
|
|
if path == unreadable:
|
|
|
|
|
raise PermissionError(f"permission denied: {path}")
|
|
|
|
|
return read(path)
|
|
|
|
|
|
|
|
|
|
def too_deep(*args, **kwargs):
|
|
|
|
|
raise RecursionError("nested too deeply")
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(diagnostics, "read_source", refused)
|
|
|
|
|
monkeypatch.setattr(diagnostics, "ast", SimpleNamespace(parse=too_deep))
|
|
|
|
|
read_note = _run(diagnostics.PythonDiagnostics().check(unreadable))
|
|
|
|
|
parse_note = _run(diagnostics.PythonDiagnostics().check(deep))
|
|
|
|
|
assert f"unavailable (PermissionError) for {unreadable};" in read_note
|
|
|
|
|
assert f"unavailable (RecursionError) for {deep};" in parse_note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
|
|
|
async def test_diagnostic_wait_is_bounded_and_cancellation_is_preserved(tmp_path, monkeypatch):
|
|
|
|
|
from code_flow.tools import diagnostics
|
|
|
|
|
from code_flow.tools.diagnostic_workers import DiagnosticWorkers
|
|
|
|
|
|
|
|
|
|
release = threading.Event()
|
|
|
|
|
|
|
|
|
|
def stalled(*args, **kwargs):
|
|
|
|
|
release.wait(2)
|
|
|
|
|
return ""
|
|
|
|
|
|
|
|
|
|
monkeypatch.setattr(diagnostics, "WORKERS", DiagnosticWorkers())
|
|
|
|
|
monkeypatch.setattr(diagnostics, "CHECK_WAIT_SECONDS", 0.01)
|
|
|
|
|
monkeypatch.setattr(diagnostics.PythonDiagnostics, "note", stalled)
|
|
|
|
|
try:
|
|
|
|
|
note = await diagnostics.PythonDiagnostics().check(tmp_path / "slow.py", text="value = 1\n")
|
|
|
|
|
assert "diagnostics unavailable (TimeoutError)" in note
|
|
|
|
|
task = asyncio.create_task(diagnostics.PythonDiagnostics().check(tmp_path / "cancelled.py", text="value = 1\n"))
|
|
|
|
|
task.cancel()
|
|
|
|
|
with pytest.raises(asyncio.CancelledError):
|
|
|
|
|
await task
|
|
|
|
|
finally:
|
|
|
|
|
release.set()
|
|
|
|
|
|
|
|
|
|
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
# --- occurrence -----------------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_occurrence_replaces_exactly_the_nth_match(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "i.txt"
|
|
|
|
|
target.write_text("a\na\na\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
result = _run(
|
|
|
|
|
registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 2})
|
|
|
|
|
)
|
|
|
|
|
assert "occurrence 2 of 3" in _text(result) and "line 2" in _text(result)
|
|
|
|
|
assert target.read_text() == "a\nb\na\n"
|
|
|
|
|
direct = _run(
|
|
|
|
|
registry.get("edit_file").execute(file_path=str(target), old_string="a", new_string="c", occurrence=2)
|
|
|
|
|
)
|
|
|
|
|
assert isinstance(direct, ToolResult) and direct.file_change is not None and direct.diff
|
|
|
|
|
assert target.read_text() == "a\nb\nc\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
|
|
|
("content", "old", "occurrence", "expected"),
|
|
|
|
|
[
|
|
|
|
|
("aaaa bb aa", "aa", 1, "Xaa bb aa"),
|
|
|
|
|
("aaaa bb aa", "aa", 2, "aaX bb aa"),
|
|
|
|
|
("aaaa bb aa", "aa", 3, "aaaa bb X"),
|
|
|
|
|
("ababa aba", "aba", 2, "ababa X"),
|
|
|
|
|
("aa\r\naa\r\n", "aa\r\n", 2, "aa\r\nX"),
|
|
|
|
|
],
|
|
|
|
|
)
|
|
|
|
|
def test_occurrence_counts_non_overlapping_matches(registry, tmp_path, content, old, occurrence, expected):
|
|
|
|
|
target = tmp_path / "matches.txt"
|
|
|
|
|
target.write_bytes(content.encode())
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
result = _run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"edit_file",
|
|
|
|
|
{"file_path": str(target), "old_string": old, "new_string": "X", "occurrence": occurrence},
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
assert not _text(result).startswith("Error")
|
|
|
|
|
assert target.read_bytes() == expected.encode()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_occurrence_description_states_its_matching_rules(registry):
|
|
|
|
|
tool = registry.get("edit_file")
|
|
|
|
|
description = tool.parameters["properties"]["occurrence"]["description"]
|
|
|
|
|
assert "non-overlapping" in description and "1-based" in description
|
|
|
|
|
assert "exact" in description and "replace_all" in description
|
|
|
|
|
assert "non-overlapping" in tool.description
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_occurrence_past_the_count_and_with_replace_all_are_refused(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "j.txt"
|
|
|
|
|
target.write_text("a\na\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
too_far = _text(
|
|
|
|
|
_run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 3}
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
assert too_far.startswith("Error") and "matches only 2" in too_far
|
|
|
|
|
both = _text(
|
|
|
|
|
_run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"edit_file",
|
|
|
|
|
{"file_path": str(target), "old_string": "a", "new_string": "b", "occurrence": 1, "replace_all": True},
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
assert both.startswith("Error") and "mutually exclusive" in both
|
|
|
|
|
assert target.read_text() == "a\na\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_several_matches_without_a_selector_are_still_refused(registry, tmp_path):
|
|
|
|
|
target = tmp_path / "k.txt"
|
|
|
|
|
target.write_text("a\na\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
result = _text(
|
|
|
|
|
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "b"}))
|
|
|
|
|
)
|
|
|
|
|
assert "appears 2 times" in result and "old_string" in result and "old_text" not in result
|
|
|
|
|
assert target.read_text() == "a\na\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- glob ------------------------------------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_glob_finds_by_pattern_and_expands_a_brace_alternation(tmp_path):
|
|
|
|
|
(tmp_path / "one.py").write_text("")
|
|
|
|
|
(tmp_path / "two.txt").write_text("")
|
|
|
|
|
(tmp_path / "three.md").write_text("")
|
|
|
|
|
tool = GlobTool(workspace=tmp_path)
|
|
|
|
|
with workdir.bind(tmp_path):
|
|
|
|
|
py = _text(_run(tool.execute(pattern="*.py")))
|
|
|
|
|
assert "one.py" in py and "two.txt" not in py
|
|
|
|
|
both = _text(_run(tool.execute(pattern="*.{py,txt}")))
|
|
|
|
|
assert "one.py" in both and "two.txt" in both and "three.md" not in both
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_brace_expansion_rules():
|
|
|
|
|
assert expand_braces("*.{a,b}") == ["*.a", "*.b"]
|
|
|
|
|
assert expand_braces("src/{a,b}/*.{ts,tsx}") == ["src/a/*.ts", "src/a/*.tsx", "src/b/*.ts", "src/b/*.tsx"]
|
|
|
|
|
assert expand_braces("*.py") == ["*.py"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the replacement in the registry -----------------------------------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_plugin_tool_registered_after_the_builtin_replaces_it(tmp_path):
|
|
|
|
|
reg = ToolRegistry()
|
|
|
|
|
reg.register(TrunkReadFileTool(workspace=tmp_path))
|
|
|
|
|
assert reg.get("read_file").parameters["required"] == ["path"]
|
|
|
|
|
reg.register(ReadFileTool(workspace=tmp_path))
|
|
|
|
|
assert reg.get("read_file").parameters["required"] == ["file_path"]
|
|
|
|
|
assert isinstance(reg.get("read_file"), ReadFileTool)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_ledger_is_keyed_by_resolved_path(tmp_path):
|
|
|
|
|
ledger = ReadLedger()
|
|
|
|
|
target = tmp_path / "m.txt"
|
|
|
|
|
target.write_text("z\n")
|
|
|
|
|
read = ReadFileTool(workspace=tmp_path, ledger=ledger)
|
|
|
|
|
edit = EditFileTool(workspace=tmp_path, ledger=ledger)
|
|
|
|
|
with workdir.bind(tmp_path):
|
|
|
|
|
_run(read.execute(file_path="m.txt"))
|
|
|
|
|
assert ledger.has_read(target)
|
|
|
|
|
assert "Successfully edited" in _text(_run(edit.execute(file_path=str(target), old_string="z", new_string="w")))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_write_file_still_reports_the_diff_the_ui_renders(tmp_path):
|
|
|
|
|
tool = WriteFileTool(workspace=tmp_path)
|
|
|
|
|
target = tmp_path / "n.py"
|
|
|
|
|
target.write_text("a = 1\n")
|
|
|
|
|
result = _run(tool.execute(file_path=str(target), content="a = 2\n"))
|
|
|
|
|
assert isinstance(result, ToolResult) and result.file_change is not None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _git(cwd: Path, *args: str) -> None:
|
|
|
|
|
subprocess.run(["git", "-C", str(cwd), *args], check=True, capture_output=True)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- what the adversarial audit of 2026-09-08 found, each pinned -------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_an_occurrence_edit_keeps_the_files_own_line_endings(registry, tmp_path):
|
|
|
|
|
"""A CRLF file edited through ``occurrence`` used to come back with every
|
|
|
|
|
line changed: the path read and wrote through text mode, which normalises
|
|
|
|
|
to LF, so a one-line edit arrived as a whole-file diff (and a grader diffing
|
|
|
|
|
the working tree sees a rewrite). Trunk's own edit path reads and writes
|
|
|
|
|
bytes for exactly this reason; so does this one now."""
|
|
|
|
|
target = tmp_path / "crlf.txt"
|
|
|
|
|
target.write_bytes(b"a\r\nb\r\na\r\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
result = _run(
|
|
|
|
|
registry.execute("edit_file", {"file_path": str(target), "old_string": "a", "new_string": "Z", "occurrence": 2})
|
|
|
|
|
)
|
|
|
|
|
assert "occurrence 2 of 2" in _text(result)
|
|
|
|
|
assert target.read_bytes() == b"a\r\nb\r\nZ\r\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_an_error_report_quotes_the_file_verbatim(registry, tmp_path):
|
|
|
|
|
"""The respelling of trunk's parameter names must not reach quoted file
|
|
|
|
|
content. It used to be a blanket replacement over the whole result, so a
|
|
|
|
|
file that really contains ``old_text`` was shown to the model as containing
|
|
|
|
|
``old_string`` -- and the model then copied an excerpt that could never
|
|
|
|
|
match anything."""
|
|
|
|
|
target = tmp_path / "tool.py"
|
|
|
|
|
target.write_text("def execute(self, path, old_text, new_text):\n value = old_text\n return path\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
near_miss = "def execute(self, path, old_text, new_text):\n value = old_text\n return other\n"
|
|
|
|
|
report = _text(
|
|
|
|
|
_run(
|
|
|
|
|
registry.execute(
|
|
|
|
|
"edit_file",
|
|
|
|
|
{"file_path": str(target), "old_string": near_miss, "new_string": "x"},
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
)
|
|
|
|
|
sentence, _, excerpt = report.partition("\n")
|
|
|
|
|
assert sentence.startswith("Error") and "old_string" in sentence, "the sentence names the parameter shown"
|
|
|
|
|
assert "old_text" in excerpt, "the quoted file keeps its own spelling"
|
|
|
|
|
assert "old_string" not in excerpt, "and nothing rewrites the file's text"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_long_line_is_cut_the_way_the_description_promises(registry, tmp_path):
|
|
|
|
|
"""Trunk numbers a line whole, so one minified line can overrun the
|
|
|
|
|
whole-result budget by itself: the read came back with NO body and a note
|
|
|
|
|
telling the model to continue from the same offset, which it can do
|
|
|
|
|
forever. The description promised truncation; now it happens."""
|
|
|
|
|
target = tmp_path / "bundle.js"
|
|
|
|
|
target.write_text("x" * 200_000 + "\nlast\n")
|
|
|
|
|
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
|
|
|
|
|
assert "line truncated to 2000 chars" in body
|
|
|
|
|
assert "1| " in body and "2| last" in body
|
|
|
|
|
|
|
|
|
|
|
fix(*): show the head of a line longer than a read_file page instead of an empty page (#959)
## Summary
A page of `read_file` is cut to whole lines within its 128k-character
budget. A file whose first line is over the budget by itself (a minified
script, a one-line JSON, a long CSV row) left nothing within it: the
page came back empty, and its footer said to continue at the very line
it had just not shown, so the reader went round for ever. Measured
before the change: `(Showing lines 1-0 of 1. Use offset=1 to continue.)`
for a single 200k-character line, and `(Showing lines 2-1 of 3. Use
offset=2 to continue.)` once a shorter first line had filled page one.
Such a line is now shown by its head, as much as the budget holds,
followed by a note that says how long the line is, how much of it is
shown, and that `offset` moves by lines and cannot reach the rest:
`grep` tells whether a piece is in it, and a shell command that cuts by
bytes (such as `cut -c`) shows another stretch. The paging footer then
points at the next line, so paging moves on. A first line that just fits
(the line break the budget counts put it over) is shown whole, without a
note. Pages that fit are unchanged.
The node runs the same tool for a file on a registered machine, so the
remote read reads the same; one of the new tests goes through the real
node behind the stand-in ssh.
Raven-Code's `read_file` (the code-flow plugin) cuts every line to 2000
characters. It had its own way round the empty page: whenever trunk
answered with no numbered line it re-read the whole file itself, and
otherwise it cut the lines after trunk had counted the page, so a dozen
10k-character lines filled a page. Trunk's read now offers a hook,
`_cut_line`, applied to each line before the page's budget counts it
(trunk's own is a no-op); the plugin cuts on it, so minified lines share
a page, each cut once, and its whole-file re-read is removed. Its cut
after the read now runs only for a page a registered machine's node
answered (the node runs trunk's tool, without the plugin's hook); a page
read here was cut on the hook and is not looked at again. The first head
told a cut line apart by its ending, so a line whose own text ends like
the mark came back whole (both reviewers reproduced it); whether a line
was cut is now decided by where the read ran, never by its text.
Changed: `raven/agent/tools/filesystem.py` (the trim loop and the hook
in `ReadFileTool`),
`agents/raven-code/plugins/code-flow/code_flow/tools/filesystem.py` (the
plugin's read), tests in `tests/test_filesystem_tools.py`,
`tests/test_remote_file_tools.py` and
`tests/test_agents_code_tools_plugin.py`.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- The six new tests fail on `origin/main` (the four about the page with
the empty pages quoted above; the two about the hook with a dozen lines
per page) and pass after the change: `pytest
tests/test_filesystem_tools.py tests/test_remote_file_tools.py
tests/test_agents_code_tools_plugin.py -k "longer_than_a_page or
just_fills_the_page or going_round or cut_per_line or minified_lines"`.
- Second commit:
`test_a_line_whose_own_text_ends_like_the_mark_is_cut_like_any_other`
fails on the first head (the line comes back whole) and passes now;
`test_a_page_a_registered_machine_s_node_answered_is_cut_after_the_read`
pins the pass after the read for a page a node answered, through a
stand-in pool.
- `pytest tests/test_agents_code_tools_plugin.py
tests/test_filesystem_tools.py tests/test_remote_file_tools.py
tests/test_node_server.py tests/test_read_file_image.py
tests/test_tools_workdir.py tests/test_tool_registry_execute.py -n 4`:
356 passed.
- `scripts/coverage_gate.py diff --base-ref origin/main`: 100% of the 11
changed executable lines.
- `ruff check`, `ruff format --check`, `ty check` on both changed
modules: clean.
- `scripts/check_commit_messages.py`,
`scripts/check_source_language.py`, `scripts/check_large_files.py` over
`origin/main..HEAD`: clean.
- Full suite (`pytest -n 6`), run again at the second commit: 31623
passed, 233 skipped; the 3 failures are the ones main has on this
machine (a LibreOffice-less host, two simulation cases), unrelated to
this change.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed (checked
CONTEXT.md, docs-site and the plugin's own text: nothing describes the
page footer or the plugin's cut; both tools' descriptions still hold)
## Risk
- [x] Security impact considered: nothing new is read, and the note
names no path.
- [x] Backward compatibility considered: a page that fits is unchanged,
and the existing paging tests still hold; the hook is a no-op unless a
subclass overrides it. In Raven-Code the page of a file with minified
lines holds more lines than before (the cut lands before the budget) and
the one mark reads as it did.
- [x] Rollback path is clear for risky changes: revert the one commit.
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-15 21:05:17 +08:00
|
|
|
def test_minified_lines_are_cut_before_the_page_is_counted_so_they_share_one_page(registry, tmp_path):
|
|
|
|
|
"""Fifty lines of 10k characters: cut after trunk's page, a dozen made a
|
|
|
|
|
page; cut on trunk's hook, before its budget counts them, they all do, each
|
|
|
|
|
cut once (the cut after the read is only for a page a registered
|
|
|
|
|
machine's node answered)."""
|
|
|
|
|
target = tmp_path / "chunks.js"
|
|
|
|
|
target.write_text("\n".join("x" * 10_000 for _ in range(50)) + "\n")
|
|
|
|
|
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
|
|
|
|
|
lines = body.split("\n\n")[0].split("\n")
|
|
|
|
|
assert len(lines) == 50 and lines[0].startswith("1| ") and lines[-1].startswith("50| ")
|
|
|
|
|
assert lines[0] == "1| " + "x" * 2000 + "... (line truncated to 2000 chars)", "cut once, at the budget"
|
|
|
|
|
assert body.endswith("(End of file \u2014 50 lines total)")
|
|
|
|
|
assert "(Line 1 is" not in body
|
|
|
|
|
|
|
|
|
|
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
def test_a_read_that_failed_does_not_license_an_edit(registry, tmp_path):
|
|
|
|
|
"""A model probing for a file that does not exist yet used to have that
|
|
|
|
|
failed read recorded as having seen it; once something else created the
|
|
|
|
|
file, the edit went straight through on content nobody had read."""
|
|
|
|
|
target = tmp_path / "later.txt"
|
|
|
|
|
missing = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
|
|
|
|
|
assert missing.startswith("Error")
|
|
|
|
|
target.write_text("created by something else\n")
|
|
|
|
|
refused = _text(
|
|
|
|
|
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "created", "new_string": "x"}))
|
|
|
|
|
)
|
|
|
|
|
assert refused.startswith("Error") and "have not read" in refused
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_file_that_changed_since_the_read_is_refused_as_stale(registry, tmp_path):
|
|
|
|
|
"""The fork's third state: a formatter, a build step or a checkout between
|
|
|
|
|
the read and the edit means the text being edited is no longer there."""
|
|
|
|
|
import os
|
|
|
|
|
|
|
|
|
|
target = tmp_path / "moving.txt"
|
|
|
|
|
target.write_text("one\n")
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
target.write_text("two\n")
|
|
|
|
|
os.utime(target, (1_000_000, 1_000_000))
|
|
|
|
|
stale = _text(
|
|
|
|
|
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "two", "new_string": "three"}))
|
|
|
|
|
)
|
|
|
|
|
assert stale.startswith("Error") and "changed since you last read it" in stale
|
|
|
|
|
_run(registry.execute("read_file", {"file_path": str(target)}))
|
|
|
|
|
assert "Successfully edited" in _text(
|
|
|
|
|
_run(registry.execute("edit_file", {"file_path": str(target), "old_string": "two", "new_string": "three"}))
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_write_ledger_does_not_depend_on_the_syntax_verdict(tmp_path):
|
|
|
|
|
"""Two separate knobs: turning the Python verdict off used to leave the
|
|
|
|
|
model unable to edit a file it had just written, because the ledger entry
|
|
|
|
|
sat after the verdict's early return."""
|
|
|
|
|
ctx = _ctx(tmp_path, {"enabled": True, "pythonSyntaxNote": False})
|
|
|
|
|
write = factories.make_write_file(ctx)
|
|
|
|
|
edit = factories.make_edit_file(ctx)
|
|
|
|
|
target = tmp_path / "quiet.py"
|
|
|
|
|
assert "Successfully wrote" in _text(_run(write.execute(file_path=str(target), content="x = 1\n")))
|
|
|
|
|
assert "Successfully edited" in _text(_run(edit.execute(file_path=str(target), old_string="1", new_string="2")))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_workspace_fence_follows_the_products_answer(tmp_path):
|
|
|
|
|
"""These tools replace the host's own by name, and the host fences its own
|
|
|
|
|
on ``tools.restrictToWorkspace``. Passing no ``allowed_dirs`` meant the
|
|
|
|
|
fence was OFF whatever the product asked for -- half the face fenced, half
|
|
|
|
|
not, with no warning. The launcher renders the answer into the slice."""
|
|
|
|
|
outside = tmp_path / "outside.txt"
|
|
|
|
|
outside.write_text("secret\n")
|
|
|
|
|
inside = tmp_path / "ws"
|
|
|
|
|
inside.mkdir()
|
|
|
|
|
unfenced = factories.make_read_file(_ctx(inside, {"enabled": True}))
|
|
|
|
|
assert "secret" in _text(_run(unfenced.execute(file_path=str(outside))))
|
|
|
|
|
fenced = factories.make_read_file(_ctx(inside, {"enabled": True, "restrictToWorkspace": True}))
|
|
|
|
|
refused = _text(_run(fenced.execute(file_path=str(outside))))
|
|
|
|
|
assert refused.startswith("Error") and "outside allowed" in refused
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_zero_or_negative_occurrence_is_refused_in_code(tmp_path):
|
|
|
|
|
"""The schema's ``minimum`` stops the model, but a caller reaching the tool
|
|
|
|
|
directly used to get a write at a negative index reported as success."""
|
|
|
|
|
tool = factories.make_edit_file(_ctx(tmp_path, {"enabled": True, "requireReadBeforeEdit": False}))
|
|
|
|
|
target = tmp_path / "guard.txt"
|
|
|
|
|
target.write_text("aaa\n")
|
|
|
|
|
for bad in (0, -1):
|
|
|
|
|
refused = _text(_run(tool.execute(file_path=str(target), old_string="a", new_string="Z", occurrence=bad)))
|
|
|
|
|
assert refused.startswith("Error") and "1 or greater" in refused
|
|
|
|
|
assert target.read_text() == "aaa\n"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- glob: the merge that used to paste the engine's notices into the listing --
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_glob_merges_only_paths_and_honours_the_callers_limit(tmp_path):
|
|
|
|
|
"""Every alternative used to spend the caller's ``limit`` again, and each
|
|
|
|
|
engine notice ("No files found…", "(showing first N of M results)") was
|
|
|
|
|
pasted into the listing as though it were a filename -- which the model
|
|
|
|
|
then tried to read."""
|
|
|
|
|
for name in ("a.py", "b.py", "c.ts", "d.ts"):
|
|
|
|
|
(tmp_path / name).write_text("")
|
|
|
|
|
tool = GlobTool(workspace=tmp_path)
|
|
|
|
|
with workdir.bind(tmp_path):
|
|
|
|
|
both = _text(_run(tool.execute(pattern="*.{py,ts}")))
|
|
|
|
|
assert sorted(both.splitlines()) == ["a.py", "b.py", "c.ts", "d.ts"]
|
|
|
|
|
missing = _text(_run(tool.execute(pattern="*.{py,zzz}")))
|
|
|
|
|
assert sorted(missing.splitlines()) == ["a.py", "b.py"], "a barren alternative adds no line"
|
|
|
|
|
capped = _text(_run(tool.execute(pattern="*.{py,ts}", limit=3)))
|
|
|
|
|
lines = capped.splitlines()
|
|
|
|
|
assert len(lines) == 4 and len([ln for ln in lines if is_notice(ln)]) == 1
|
|
|
|
|
assert len([ln for ln in lines if not is_notice(ln)]) == 3, "the caller's limit is the merged total"
|
|
|
|
|
# A single pattern is the engine's own answer, verbatim.
|
|
|
|
|
assert "No files found matching pattern" in _text(_run(tool.execute(pattern="*.nope")))
|
|
|
|
|
# A merged one is this tool's, and names the pattern it was asked for.
|
|
|
|
|
assert "No files found matching pattern: *.{nope,zzz}" == _text(_run(tool.execute(pattern="*.{nope,zzz}")))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_glob_preserves_a_child_truncation_when_other_patterns_are_empty(tmp_path):
|
|
|
|
|
for name in ("a.py", "b.py", "c.py"):
|
|
|
|
|
(tmp_path / name).write_text("")
|
|
|
|
|
tool = GlobTool(workspace=tmp_path)
|
|
|
|
|
out = _text(_run(tool.execute(pattern="*.{py,ts}", limit=1)))
|
|
|
|
|
assert "PARTIAL" in out and "raise limit" in out
|
|
|
|
|
assert len([line for line in out.splitlines() if not is_notice(line)]) == 1
|
|
|
|
|
|
|
|
|
|
|
fix(agent): bound every tree walk and take it off the event loop (#566)
## Summary
On 2026-09-20 one `find` call froze the whole gateway for 2h21m. The
model asked for `find(pattern="**/gps_trilateration/index.html",
path="/Users/admin")`, and `FindTool` ran `Path.glob` synchronously on
the event loop, filtered the noise directories only after walking them,
and had no deadline. The registry's 300s ceiling is an
`asyncio.wait_for`, which cannot preempt synchronous code, so every
session and every subagent on the gateway stopped until the walk came
back (`gateway.log`: `Tool result: find duration=8476519ms`). `list_dir`
with `recursive=true` had the same shape through `rglob`.
The fix gives the three tree-walking tools one walk,
`raven.agent.tools.tree_walk`, with these bounds:
- noise directories (`node_modules`, `.git`, ...) are pruned before they
are entered, not filtered afterwards -- the pruning tests spy on
`os.scandir` and assert the noise directory is never opened;
- the walk yields one entry at a time and checks a 20s wall-clock budget
(the value grep's fallback already used) before every entry, so whatever
the caller does with one entry -- the `stat` per `find` match, the read
per grep file -- is covered by the check on the next. The budget cannot
be overrun by one large or slow directory: the probe with eight matches,
a 30ms stat each and a 50ms budget returns 2 matches and the notice.
When the budget is hit, `find` and `list_dir` append a `PARTIAL result:
... traversal budget ...` clause to their trailer, and a deadline that
fires before any match is not reported as a clean `No files found` /
`Directory is empty`;
- every caller runs the walk in `asyncio.to_thread`, so the loop keeps
serving other sessions while the walk runs. Cancelling the thread does
not stop the worker, which is why the deadline lives inside the walk.
The pure-Python `grep` fallback already had the prune and the deadline;
it moves onto the shared walk and off the loop with the others.
`find` matches each path component with `fnmatch.translate`, with a
component that is exactly `**` spanning any number of components; the
matcher is a position-set simulation (`_advance`), linear in components
times segments, so a pattern carrying many `**` cannot make one match
call slow. A pattern's leading literal components decide where the walk
starts: `src/**/*.py` begins inside `src` and never lists a sibling of
it, a pattern that is all literal (`src/lib/c.py`, `src/lib`) names
exactly one path and answers without a walk, and the prefix itself is a
match when the rest of the pattern can match nothing (`src/**` still
lists `src/`). It answers exactly what `Path.glob` answered for the same
pattern: pinned by a parametrized oracle test over twenty-seven patterns
on a fixture and nineteen on a fixture with links (a directory link out
of the tree, a link cycle, a file link, a noise directory beneath a
link), and checked by hand against `Path.glob` over `raven/` for 27
patterns including bracket edge cases and trailing slashes, 0
mismatches. Two deliberate differences: a trailing `**` also lists files
beneath its prefix (3.12's `Path.glob` listed only directories there),
and a pattern with `..` or a leading `/` is refused instead of followed,
since a pattern must not reach outside the directory the fence resolved.
A trailing slash keeps `Path.glob`'s directory-only meaning (`*/`,
`src/**/`). A symbolic link to a directory is entered where a single
component names or matches it and never under `**`, which is how
`Path.glob` read a pattern: `*/util/helper.py` reaches through a linked
`vendor`, `**/helper.py` does not, and a link cycle is entered once per
component the pattern spends on it, so `*/*/*/b.py` enters `src/loop ->
src` twice and stops. The walker asks its caller per link (`follow`),
`find` answers from the pattern and hands the link positions of a path
to the matcher so `**` cannot claim them, and `list_dir` and the grep
fallback never ask, staying as `rglob` and `os.walk` left them. A third
deliberate difference from the merge base: a noise directory inside a
pattern's literal prefix (`node_modules/*.js`, `src/node_modules/*.js`)
is walked, since the pattern asked for it, where `Path.glob`'s filter
dropped it; one met below the prefix is pruned as before
(`*/node_modules/*.js` answers nothing), `list_dir` on such a path lists
it, and the `find`, `list_dir` and code-flow `glob` descriptions state
that rule instead of an unconditional "skipped". Review found an earlier
wording of the two `find` descriptions one component short ("except one
the pattern starts with"), while the oracle helper already pruned below
the prefix; the sentences now say what the helper checks, and the oracle
has the `src/node_modules` shape on both sides.
The raven-code `glob` tool (`agents/raven-code/.../tools/search.py`)
parses trunk `find` output line by line and strips the trailer before
merging brace alternatives. It learns the new clause, so a cut-short
child is reported as cut short rather than as a tree that lacks the
files; its `is_notice` grammar covers the new line so it is never pasted
into a listing as a path. The code-flow tool guide
(`prompts/TOOLS_CODE.md`), the wrapper's description and its `limit`
parameter told the model that a PARTIAL `glob` result means "raise
`limit` or narrow the pattern"; a traversal-budget PARTIAL is now a
result `glob` can return and a higher limit does nothing for it, so all
three now tell the two notices apart. The seeded copy of the guide
refreshes through the receipt `seed_guide` keeps beside it, so no legacy
digest is added.
Fixed in passing, and now pinned: recursive `list_dir` filtered `rglob`
results on the components of the absolute path, so a workspace beneath a
directory named `build`, `dist`, `venv` or another noise name listed as
"Directory ... is empty" (reproduced at the merge base under
`<tmp>/build/proj`; this head lists it). The shared walk prunes below
the listed path only; `find` and the grep fallback already did.
Not changed: the 20s value. It is the budget grep's fallback already ran
under; this PR does not re-measure it, it makes it hold for every entry
of every walk.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
At head 8dade176, on origin/main at d50a13c1 (nine commits: the walk,
the grep pin, the changelog, the per-entry budget with the prefix start,
the literal-directory case, the symbolic-link reading with the
noise-prefix wording, the noise-name-above-the-path pin, the guide
wording, and the prefix-rule wording; `git range-diff` against bb836988:
all nine replayed `=` onto the moved main, which conflicted only at the
changelog anchor and touches no file this PR touches):
```
uv run pytest tests/test_file_search_traversal_guard.py tests/test_agents_code_tools_plugin.py \
tests/test_search_tools.py tests/test_tools_workdir.py tests/test_tool_registry_execute.py tests/test_filesystem_tools.py -q
# 203 passed
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q --ignore=tests/integration # in a throwaway worktree
# 24652 passed, 110 skipped in 382.30s (0:06:22) at head 8dade176, in a throwaway worktree with COLORTERM=truecolor
uv run pytest tests/test_agents_code_launcher.py tests/test_agents_code_tools_plugin.py -q # the guide's seeding and wording tests, the glob wrapper
# 119 passed
make lint-python lint-imports lint-types lint-deps # all pass
git diff --check origin/main...HEAD # clean
scripts/check_source_language.py origin/main...HEAD # exit 0
scripts/check_large_files.py origin/main...HEAD # exit 0
git merge-tree --write-tree HEAD origin/main # clean
```
Revert-to-red, with the tests kept (101 tests in
`test_file_search_traversal_guard.py` +
`test_agents_code_tools_plugin.py`):
```
the three production files reverted to the previously reviewed head (fd5663ee) 19 failed
file_search.py + filesystem.py reverted to origin/main, tree_walk.py removed collection error
```
Mutation, each mutant applied alone, control restored between runs:
```
deadline checked once per directory (the reviewed placement) 10 failed
walk starts at the root instead of the pattern's prefix 2 failed
trailing slash matches files too 4 failed
a pattern naming one file still walks 1 failed
the prefix itself is never a match 7 failed
a pattern naming one directory walks beneath it 1 failed
control (nothing mutated) 101 passed
```
From the first review round (still holding at this head): find without
`to_thread` 1 failed; list_dir without `to_thread` 1 failed; grep
fallback without `to_thread` 1 failed; pruning removed 6 failed; fork
drops the PARTIAL clause 1 failed.
Fourth round, the symbolic-link reading
(`test_file_search_traversal_guard.py` +
`test_agents_code_tools_plugin.py`, 128 tests, `-n 0`; a mutant that
lets `**` sweep a link, or forgets the link positions, re-enters the
cycle until the OS refuses the path and answers paths `Path.glob` never
did):
```
baseline 128 passed
MUTANT ** may sweep a link 8 failed (7 oracle cases, and the link the pattern ends at is opened)
MUTANT find never asks to enter a link 6 failed (the six oracle patterns that pass through a link)
MUTANT a link is entered even when the pattern ends at it 1 failed (scandir spy on **/vendor)
MUTANT ancestor links forgotten (links_of empty) 1 failed (a cycle case answers a path Path.glob never did)
MUTANT the walker consults follow but never enters 7 failed
MUTANT each followed root gets a fresh deadline 9 failed (every deadline test, and the walker's one-budget test)
REVERT production files to reviewed head d67184c8 7 failed
CONTROL restored 128 passed
```
Revert-to-red for the noise-name pin: with
`raven/agent/tools/filesystem.py` at the merge base and the test kept,
`test_a_noise_name_above_the_searched_path_hides_nothing` fails on the
`Directory ... is empty` answer (1 failed); at this head it passes. The
`agents/**` change is two prose strings and a guide line, outside the
coverage gate's `raven/**` and `plugins-dist/**` pathspecs; the guide's
name-list test and its seeding tests (`test_agents_code_launcher.py`)
cover the file.
The prefix rule is pinned by mutation, the other policy (prune a noise
directory inside the prefix) against the two guard files (131 tests, `-n
0`):
```
baseline 131 passed
MUTANT a noise directory inside the prefix is pruned 5 failed
test_find_matches_what_path_glob_matched[node_modules/**/*.py], test_find_matches_what_path_glob_matched[node_modules/*/], test_find_matches_what_path_glob_matched[node_modules/*/d.py], test_find_matches_what_path_glob_matched[node_modules/x/d.py], test_find_matches_what_path_glob_matched[src/node_modules/*.py]
CONTROL restored 131 passed
```
The symlink oracle is the reviewer's own test shape over a fixture with
links: `vendor -> outside/real`, `src/loop -> src`, `alias.py -> a.py`,
a `node_modules` beneath the link; 19 patterns, 0 mismatches against
`Path.glob` on 3.12, plus a scandir spy pinning that `**/vendor` never
opens the link, a walker test pinning that the default walk yields a
link and does not enter it while a `follow`ed link is walked after the
tree under one deadline, and a `list_dir` and grep test pinning
`rglob`/`os.walk` parity through a link.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed (CHANGELOG
`Unreleased / Fixed` entry)
## Risk
User-visible behaviour changes, all in tool output the model reads:
- `find` and `list_dir` (recursive) can now return a partial listing
with a `PARTIAL result` clause in the trailer where they previously
either returned everything or hung. The clause tells the model the
absence of a match is not conclusive and to narrow the search.
- A path-prefixed `find` pattern spends its budget only inside its
prefix, so on a large tree it finds what a root-anchored walk used to
miss.
- `find` refuses patterns containing `..` or starting with `/` with an
`Error running find:` line; `Path.glob` used to raise on the second and
could climb on the first.
- `find` with a trailing `**` lists files as well as directories beneath
the prefix; a trailing `/` lists directories only, as before.
- `find` reaches through a symbolic link to a directory where a literal
or single-wildcard component names or matches it, as it did at the merge
base; the earlier revisions of this PR had lost that and returned a
clean miss with no notice. `**` never enters a link, as before.
- A noise directory inside a `find` pattern's literal prefix, or named
as the `list_dir` path, is walked: `find('node_modules/*.js')` and
`find('src/node_modules/*.js')` answer its files and
`list_dir('node_modules', recursive=true)` lists it, where the merge
base answered nothing and "empty". One met below the prefix is pruned as
before. Both descriptions state the rule.
- Recursive `list_dir` on a workspace beneath a directory named like a
noise directory (`/tmp/build/proj`) lists it instead of answering "is
empty".
- The code-flow `glob` description, its `limit` parameter and the tool
guide tell a limit cut from a traversal-budget PARTIAL; the notices
themselves are unchanged.
- `list_dir` flat listings are unchanged; the empty-directory message is
unchanged.
Rollback: revert the squash commit. The new module has no other
consumers, and no state or schema is involved.
- [x] Security impact considered (a pattern can no longer climb out of
the fenced directory via `..`; a directory link is entered exactly where
`Path.glob` entered it at the merge base, one non-`**` component, and
never under `**`, so the fence's reach is what it was and not wider;
`list_dir` and grep never enter one)
- [x] Backward compatibility considered (the fork's trailer grammar is
extended, not changed; existing fork tests pass unchanged)
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 10:59:01 +08:00
|
|
|
def test_glob_carries_a_childs_traversal_budget_into_the_merged_notice(tmp_path, monkeypatch):
|
|
|
|
|
"""When the engine's walk hits its deadline the child answer ends in a
|
|
|
|
|
PARTIAL trailer, sometimes above an otherwise clean miss. Neither line is a
|
|
|
|
|
path, and the merged listing has to say the search was cut short rather
|
|
|
|
|
than report a tree that lacks the files."""
|
|
|
|
|
from raven.agent.tools import tree_walk
|
|
|
|
|
|
|
|
|
|
(tmp_path / "a.py").write_text("")
|
|
|
|
|
tool = GlobTool(workspace=tmp_path)
|
|
|
|
|
monkeypatch.setattr(tree_walk, "WALK_DEADLINE_S", -1.0)
|
|
|
|
|
merged = _text(_run(tool.execute(pattern="*.{py,ts}")))
|
|
|
|
|
assert merged.startswith("No files found matching pattern: *.{py,ts}")
|
|
|
|
|
assert "traversal budget" in merged
|
|
|
|
|
assert all(is_notice(line) for line in merged.splitlines() if line)
|
|
|
|
|
|
|
|
|
|
# Each child walk reads the clock four times over this tree: its
|
|
|
|
|
# deadline, then before ``late/`` and ``a.py`` at the root (in budget),
|
|
|
|
|
# then before ``late/b.ts`` (past it).
|
|
|
|
|
monkeypatch.setattr(tree_walk, "WALK_DEADLINE_S", 20.0)
|
|
|
|
|
ticks = itertools.cycle([0.0, 0.0, 0.0, tree_walk.WALK_DEADLINE_S + 1])
|
|
|
|
|
monkeypatch.setattr(tree_walk, "time", SimpleNamespace(monotonic=lambda: next(ticks)))
|
|
|
|
|
(tmp_path / "late").mkdir()
|
|
|
|
|
(tmp_path / "late" / "b.ts").write_text("")
|
|
|
|
|
partial = _text(_run(tool.execute(pattern="*.{py,ts}")))
|
|
|
|
|
lines = partial.splitlines()
|
|
|
|
|
assert [line for line in lines if not is_notice(line)] == ["a.py"]
|
|
|
|
|
assert "PARTIAL result" in lines[-1] and "traversal budget" in lines[-1]
|
|
|
|
|
|
|
|
|
|
|
feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary
An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.
- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.
What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.
## Review round 1 (2026-09-09, on !600, superseded by this MR)
- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.
## Review round 2 (2026-09-10)
- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.
## Review round 3 (2026-09-10)
- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.
## Review round 4 (2026-09-10)
- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
tests/test_agents_oncall_flow_escalation_tools.py -q # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents # clean
uv run lint-imports # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit # in sync, clean
```
New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).
Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)
## Risk
- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.
A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.
## Related
Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
|
|
|
@pytest.mark.parametrize("kind", ["missing", "not_directory", "outside"])
|
|
|
|
|
def test_glob_preserves_child_search_errors(tmp_path, kind):
|
|
|
|
|
root = tmp_path / "root"
|
|
|
|
|
root.mkdir()
|
|
|
|
|
target = tmp_path / "outside" if kind == "outside" else root / kind
|
|
|
|
|
if kind == "not_directory":
|
|
|
|
|
target.write_text("file")
|
|
|
|
|
elif kind == "outside":
|
|
|
|
|
target.mkdir()
|
|
|
|
|
tool = GlobTool(workspace=root, allowed_dirs=(root,))
|
|
|
|
|
single = _text(_run(tool.execute(pattern="*.py", path=str(target))))
|
|
|
|
|
merged = _text(_run(tool.execute(pattern="*.{py,ts}", path=str(target))))
|
|
|
|
|
assert single.startswith("Error:") and merged == single
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_glob_merges_by_recency_before_applying_the_total_limit(tmp_path):
|
|
|
|
|
for timestamp, name in enumerate(("old.py", "middle.py", "new.ts"), start=100):
|
|
|
|
|
path = tmp_path / name
|
|
|
|
|
path.write_text("")
|
|
|
|
|
os.utime(path, (timestamp, timestamp))
|
|
|
|
|
tool = GlobTool(workspace=tmp_path)
|
|
|
|
|
out = _text(_run(tool.execute(pattern="*.{py,ts}", limit=2)))
|
|
|
|
|
assert out.splitlines()[:2] == ["new.ts", "middle.py"]
|
|
|
|
|
assert "PARTIAL" in out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_glob_keeps_paths_that_look_like_notice_prefixes_and_deduplicates(tmp_path):
|
|
|
|
|
names = ["ErrorHandler.py", "No files found matching pattern.py", "other.py"]
|
|
|
|
|
for name in names:
|
|
|
|
|
(tmp_path / name).write_text("")
|
|
|
|
|
tool = GlobTool(workspace=tmp_path)
|
|
|
|
|
out = _text(_run(tool.execute(pattern="{*.py,Error*}", limit=3)))
|
|
|
|
|
assert sorted(out.splitlines()) == sorted(names)
|
|
|
|
|
assert "PARTIAL" not in out
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_nested_braces_expand_by_pairing_and_a_runaway_pattern_is_refused(tmp_path):
|
|
|
|
|
"""Scanning for the first ``}`` produced garbage patterns (``*.a}``) and
|
|
|
|
|
silently dropped whole alternatives -- a listing that omits half the
|
|
|
|
|
alternatives reads exactly like a tree that lacks them."""
|
|
|
|
|
assert expand_braces("*.{a,b}") == ["*.a", "*.b"]
|
|
|
|
|
assert expand_braces("*.{a,{b,c}}") == ["*.a", "*.b", "*.c"]
|
|
|
|
|
assert expand_braces("src/{a,b}/*.{ts,tsx}") == ["src/a/*.ts", "src/a/*.tsx", "src/b/*.ts", "src/b/*.tsx"]
|
|
|
|
|
assert expand_braces("*.py") == ["*.py"]
|
|
|
|
|
with pytest.raises(ValueError):
|
|
|
|
|
expand_braces("*.{a,b")
|
|
|
|
|
with pytest.raises(ValueError):
|
|
|
|
|
expand_braces("{a,b,c,d,e}{1,2,3,4,5}")
|
|
|
|
|
(tmp_path / "one.py").write_text("")
|
|
|
|
|
tool = GlobTool(workspace=tmp_path)
|
|
|
|
|
with workdir.bind(tmp_path):
|
|
|
|
|
refused = _text(_run(tool.execute(pattern="{a,b,c,d,e}{1,2,3,4,5}")))
|
|
|
|
|
assert refused.startswith("Error") and "separate calls" in refused
|
|
|
|
|
assert _text(_run(tool.execute(pattern="*.{py,{txt,md}}"))).splitlines() == ["one.py"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# --- the boundary the same-name contribution stands on ------------------------
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_registered_instance_is_this_products_under_every_shared_name(tmp_path):
|
|
|
|
|
"""The guarantee the whole face rests on: where a name is shared with the
|
|
|
|
|
host, the object the registry dispatches to is THIS product's.
|
|
|
|
|
|
|
|
|
|
Checked by identity, not by schema: a change that registers the plugin
|
|
|
|
|
tools before the built-ins instead of after would leave every name in
|
|
|
|
|
place and every schema silently reverted to the host's spelling."""
|
|
|
|
|
from raven.agent.tools import file_search as trunk_search
|
|
|
|
|
from raven.agent.tools import filesystem as trunk_fs
|
|
|
|
|
|
|
|
|
|
ctx = _ctx(tmp_path, {"enabled": True})
|
|
|
|
|
reg = ToolRegistry()
|
|
|
|
|
# The host's own order: built-ins first, plugin contributions last.
|
|
|
|
|
for cls in (
|
|
|
|
|
trunk_fs.ReadFileTool,
|
|
|
|
|
trunk_fs.WriteFileTool,
|
|
|
|
|
trunk_fs.EditFileTool,
|
|
|
|
|
trunk_fs.ListDirTool,
|
|
|
|
|
trunk_search.GrepTool,
|
|
|
|
|
trunk_search.FindTool,
|
|
|
|
|
):
|
|
|
|
|
reg.register(cls(workspace=tmp_path))
|
|
|
|
|
for make in (
|
|
|
|
|
factories.make_read_file,
|
|
|
|
|
factories.make_write_file,
|
|
|
|
|
factories.make_edit_file,
|
|
|
|
|
factories.make_list_dir,
|
|
|
|
|
factories.make_glob,
|
|
|
|
|
factories.make_todo,
|
|
|
|
|
factories.make_exec,
|
|
|
|
|
):
|
|
|
|
|
reg.register(make(ctx))
|
|
|
|
|
for name in ("read_file", "write_file", "edit_file", "list_dir"):
|
|
|
|
|
served = reg.get(name)
|
|
|
|
|
assert type(served).__module__.startswith("code_flow.tools"), name
|
|
|
|
|
assert not isinstance(served, (trunk_fs.ReadFileTool,)) or type(served) is not trunk_fs.ReadFileTool
|
|
|
|
|
# Names the host does not use come from here too, and the host's own
|
|
|
|
|
# spelling of the pathname tool is still registered until the launcher
|
|
|
|
|
# withholds it -- that withholding is the launcher's test, not this one.
|
|
|
|
|
assert type(reg.get("glob")).__module__.startswith("code_flow.tools")
|
|
|
|
|
assert type(reg.get("todo")).__module__.startswith("code_flow.tools")
|
|
|
|
|
assert type(reg.get("grep")).__module__.startswith("raven."), "a name we do not serve stays the host's"
|
|
|
|
|
assert reg.get("find") is not None, "trunk's spelling is withheld by config, not by this registry"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_replacement_answers_the_call_not_just_the_schema(tmp_path):
|
|
|
|
|
"""Dispatch, not advertising: the served instance is the one that runs, so
|
|
|
|
|
a call in the fork's spelling reaches this product's behaviour (the read
|
|
|
|
|
ledger) rather than the host's tool, which has none."""
|
|
|
|
|
ctx = _ctx(tmp_path, {"enabled": True})
|
|
|
|
|
reg = ToolRegistry()
|
|
|
|
|
reg.register(trunk_read_file(tmp_path))
|
|
|
|
|
reg.register(factories.make_read_file(ctx))
|
|
|
|
|
reg.register(factories.make_edit_file(ctx))
|
|
|
|
|
target = tmp_path / "b.txt"
|
|
|
|
|
target.write_text("one\n")
|
|
|
|
|
# The host's tool has no read-before-edit rule; ours does, and it is ours
|
|
|
|
|
# that answers.
|
|
|
|
|
refused = _text(
|
|
|
|
|
_run(registry_execute(reg, "edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
|
|
|
|
|
)
|
|
|
|
|
assert refused.startswith("Error") and "have not read" in refused
|
|
|
|
|
_run(registry_execute(reg, "read_file", {"file_path": str(target)}))
|
|
|
|
|
assert "Successfully edited" in _text(
|
|
|
|
|
_run(registry_execute(reg, "edit_file", {"file_path": str(target), "old_string": "one", "new_string": "two"}))
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_the_face_declines_as_a_whole_and_the_host_keeps_serving(tmp_path):
|
|
|
|
|
"""A product that hands the tools back gets the host's, unchanged: the
|
|
|
|
|
factories decline, nothing is registered over the built-ins, and the
|
|
|
|
|
schema the model sees is the host's own."""
|
|
|
|
|
from raven.agent.tools import filesystem as trunk_fs
|
|
|
|
|
|
|
|
|
|
ctx = PluginContext(
|
|
|
|
|
config={"enabled": True, "tools": {"enabled": False}},
|
|
|
|
|
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
|
|
|
|
|
)
|
|
|
|
|
reg = ToolRegistry()
|
|
|
|
|
reg.register(trunk_fs.ReadFileTool(workspace=tmp_path))
|
|
|
|
|
contributed = [
|
|
|
|
|
make(ctx)
|
|
|
|
|
for make in (
|
|
|
|
|
factories.make_read_file,
|
|
|
|
|
factories.make_write_file,
|
|
|
|
|
factories.make_edit_file,
|
|
|
|
|
factories.make_list_dir,
|
|
|
|
|
factories.make_glob,
|
|
|
|
|
factories.make_todo,
|
|
|
|
|
factories.make_exec,
|
|
|
|
|
)
|
|
|
|
|
]
|
|
|
|
|
assert contributed == [None] * 7
|
|
|
|
|
assert type(reg.get("read_file")) is trunk_fs.ReadFileTool
|
|
|
|
|
assert reg.get("read_file").parameters["required"] == ["path"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_malformed_slice_declines_rather_than_serving_half_a_face(tmp_path):
|
|
|
|
|
"""Half a face is worse than none: the guide and the conduct teach one
|
|
|
|
|
spelling, so a slice that does not parse must take the whole face down."""
|
|
|
|
|
ctx = PluginContext(
|
|
|
|
|
config={"enabled": True, "tools": {"enabled": "yes please"}},
|
|
|
|
|
services=ServiceLocator(workspace=tmp_path, user_id="u", agent_id="a"),
|
|
|
|
|
)
|
|
|
|
|
assert factories.make_read_file(ctx) is None
|
|
|
|
|
assert factories.make_glob(ctx) is None
|
fix(*): show the head of a line longer than a read_file page instead of an empty page (#959)
## Summary
A page of `read_file` is cut to whole lines within its 128k-character
budget. A file whose first line is over the budget by itself (a minified
script, a one-line JSON, a long CSV row) left nothing within it: the
page came back empty, and its footer said to continue at the very line
it had just not shown, so the reader went round for ever. Measured
before the change: `(Showing lines 1-0 of 1. Use offset=1 to continue.)`
for a single 200k-character line, and `(Showing lines 2-1 of 3. Use
offset=2 to continue.)` once a shorter first line had filled page one.
Such a line is now shown by its head, as much as the budget holds,
followed by a note that says how long the line is, how much of it is
shown, and that `offset` moves by lines and cannot reach the rest:
`grep` tells whether a piece is in it, and a shell command that cuts by
bytes (such as `cut -c`) shows another stretch. The paging footer then
points at the next line, so paging moves on. A first line that just fits
(the line break the budget counts put it over) is shown whole, without a
note. Pages that fit are unchanged.
The node runs the same tool for a file on a registered machine, so the
remote read reads the same; one of the new tests goes through the real
node behind the stand-in ssh.
Raven-Code's `read_file` (the code-flow plugin) cuts every line to 2000
characters. It had its own way round the empty page: whenever trunk
answered with no numbered line it re-read the whole file itself, and
otherwise it cut the lines after trunk had counted the page, so a dozen
10k-character lines filled a page. Trunk's read now offers a hook,
`_cut_line`, applied to each line before the page's budget counts it
(trunk's own is a no-op); the plugin cuts on it, so minified lines share
a page, each cut once, and its whole-file re-read is removed. Its cut
after the read now runs only for a page a registered machine's node
answered (the node runs trunk's tool, without the plugin's hook); a page
read here was cut on the hook and is not looked at again. The first head
told a cut line apart by its ending, so a line whose own text ends like
the mark came back whole (both reviewers reproduced it); whether a line
was cut is now decided by where the read ran, never by its text.
Changed: `raven/agent/tools/filesystem.py` (the trim loop and the hook
in `ReadFileTool`),
`agents/raven-code/plugins/code-flow/code_flow/tools/filesystem.py` (the
plugin's read), tests in `tests/test_filesystem_tools.py`,
`tests/test_remote_file_tools.py` and
`tests/test_agents_code_tools_plugin.py`.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- The six new tests fail on `origin/main` (the four about the page with
the empty pages quoted above; the two about the hook with a dozen lines
per page) and pass after the change: `pytest
tests/test_filesystem_tools.py tests/test_remote_file_tools.py
tests/test_agents_code_tools_plugin.py -k "longer_than_a_page or
just_fills_the_page or going_round or cut_per_line or minified_lines"`.
- Second commit:
`test_a_line_whose_own_text_ends_like_the_mark_is_cut_like_any_other`
fails on the first head (the line comes back whole) and passes now;
`test_a_page_a_registered_machine_s_node_answered_is_cut_after_the_read`
pins the pass after the read for a page a node answered, through a
stand-in pool.
- `pytest tests/test_agents_code_tools_plugin.py
tests/test_filesystem_tools.py tests/test_remote_file_tools.py
tests/test_node_server.py tests/test_read_file_image.py
tests/test_tools_workdir.py tests/test_tool_registry_execute.py -n 4`:
356 passed.
- `scripts/coverage_gate.py diff --base-ref origin/main`: 100% of the 11
changed executable lines.
- `ruff check`, `ruff format --check`, `ty check` on both changed
modules: clean.
- `scripts/check_commit_messages.py`,
`scripts/check_source_language.py`, `scripts/check_large_files.py` over
`origin/main..HEAD`: clean.
- Full suite (`pytest -n 6`), run again at the second commit: 31623
passed, 233 skipped; the 3 failures are the ones main has on this
machine (a LibreOffice-less host, two simulation cases), unrelated to
this change.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed (checked
CONTEXT.md, docs-site and the plugin's own text: nothing describes the
page footer or the plugin's cut; both tools' descriptions still hold)
## Risk
- [x] Security impact considered: nothing new is read, and the note
names no path.
- [x] Backward compatibility considered: a page that fits is unchanged,
and the existing paging tests still hold; the hook is a no-op unless a
subclass overrides it. In Raven-Code the page of a file with minified
lines holds more lines than before (the cut lands before the budget) and
the one mark reads as it did.
- [x] Rollback path is clear for risky changes: revert the one commit.
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-15 21:05:17 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
_MARK = "... (line truncated to 2000 chars)"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_line_whose_own_text_ends_like_the_mark_is_cut_like_any_other(registry, tmp_path):
|
|
|
|
|
"""A 5000-character line ending the way a cut line does came back whole: the
|
|
|
|
|
pass after the read took the ending for its own mark (found by both
|
|
|
|
|
reviewers of the first head). Whether a line was cut is decided by where
|
|
|
|
|
the read ran, never by the line's text."""
|
|
|
|
|
target = tmp_path / "pasted.txt"
|
|
|
|
|
target.write_text("x" * (5_000 - len(_MARK)) + _MARK + "\nlast\n")
|
|
|
|
|
body = _text(_run(registry.execute("read_file", {"file_path": str(target)})))
|
|
|
|
|
lines = body.split("\n")
|
|
|
|
|
assert lines[0] == "1| " + "x" * 2000 + _MARK and lines[1] == "2| last"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def test_a_page_a_registered_machine_s_node_answered_is_cut_after_the_read(tmp_path, monkeypatch):
|
|
|
|
|
"""The node runs trunk's tool, without this fork's hook, so its page is cut here, line by line, once."""
|
|
|
|
|
import json
|
|
|
|
|
|
|
|
|
|
from raven.ops import remote_path
|
|
|
|
|
|
|
|
|
|
rows = [
|
|
|
|
|
{"id": "box", "display_name": "Lab box", "host": "203.0.113.7", "port": 58717, "user": "worker", "key": "/k"}
|
|
|
|
|
]
|
|
|
|
|
(tmp_path / "connections.json").write_text(json.dumps({"connections": rows}), encoding="utf-8")
|
|
|
|
|
monkeypatch.setenv("RAVEN_CONNECTIONS", str(tmp_path / "connections.json"))
|
|
|
|
|
monkeypatch.setattr(remote_path, "_HOMES", {})
|
|
|
|
|
monkeypatch.setattr(remote_path, "_NODES", None)
|
|
|
|
|
monkeypatch.setattr(remote_path, "_READ", {})
|
|
|
|
|
monkeypatch.setattr(
|
|
|
|
|
"raven.config.loader.load_config", lambda *a, **k: SimpleNamespace(subagents=SimpleNamespace(agents=[]))
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
class Pool:
|
|
|
|
|
async def home(self, machine_id: str) -> str:
|
|
|
|
|
return "/home/worker"
|
|
|
|
|
|
|
|
|
|
async def call(self, machine_id: str, method: str, params: dict) -> dict:
|
|
|
|
|
assert method == "fs/read"
|
|
|
|
|
return {"text": "1| " + "y" * 5_000 + "\n2| last\n\n(End of file — 2 lines total)", "sha256": "ab" * 32}
|
|
|
|
|
|
|
|
|
|
remote_path.use_nodes(Pool)
|
|
|
|
|
read = ReadFileTool(workspace=tmp_path, allowed_dirs=(tmp_path,), follow_binding=False)
|
|
|
|
|
|
|
|
|
|
got = _run(read.execute(file_path="box:raven-work/t1/bundle.js"))
|
|
|
|
|
|
|
|
|
|
lines = got.split("\n")
|
|
|
|
|
assert lines[0] == ("1| " + "y" * 5_000)[:2000] + _MARK and lines[1] == "2| last"
|
|
|
|
|
assert got.endswith("(End of file — 2 lines total)")
|