37 Commits

Author SHA1 Message Date
Zuyi Zhou
63ab3f4189
fix(agent): keep one cause one streak class in the media envelopes (#502)
## Summary

`failure_class` keys the loop's tool-failure streak on an envelope's
`error`
string alone. That works only while the string comes from the small
fixed
vocabulary its producers are meant to keep, and it is a convention the
module
cannot enforce - the docstring added in #410 names exactly this failure
mode:
one cause counting as many.

Every media tool broke it. `_format_http_error` interpolated up to 400
characters of vendor body (which carries a request id) into `error`; the
unreadable-reference handlers interpolated the path; the four catch-alls
put
`str(e)` there, which spells the host. So a dead endpoint produced a
class per
call, `(tool, failure_class)` never repeated, the streak never reached
`_LOOP_BREAK_THRESHOLD`, and the stop-repeating nudge was unreachable
for
`image_generate`, `text_to_speech` and `video_generate` alike.

The URL gate in front of `web_fetch` had it too, in both copies of the
tool:
most reasons `validate_url_target` composes name the address they
refused, so
a model walking an internal range got one class per address. #410 fixed
the
handlers behind that gate and left the gate itself.

`error` now carries only what a reader could enumerate. The variable
part moves
to `detail`, which keeps it in front of the model and inside
`is_hard_tool_failure`'s transient-marker scan. Where the split gave the
single
call a further key, the batch reducer forwards that too: `hint`, which
names the
proxy a 403 can be routed through, and `url`, which names the reference
a
guarded fetch refused. A batched picture is told what the single call is
told,
which is the property the split has to preserve. Distinct causes stay
distinct,
which is the failure mode this trades against: two HTTP statuses, two
exception
types and the four vendor refusals each keep their own class, and the
tests
assert that direction too.

Found by sweeping for the shape rather than by reading the code: every
`"error":` whose value is not a literal is a candidate, and that grep is
the
cheapest guard against this recurring.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/ -q -p no:randomly` -> 23443 passed, 109 skipped
- `make lint-python` -> ruff check and format clean
- `make lint-types` -> all checks passed
- `make lint-imports` -> 10 contracts kept, 0 broken
- `make check-source-language`, `make check-large-files` -> clean
- Revert-to-red, per fix, restoring the source from a backup afterwards:
reverting `media_gen.py` alone turns the four new class tests red and
leaves
`test_two_different_statuses_stay_two_classes` green; reverting either
copy
of `web.py` turns only that copy's gate test red and leaves the two #410
  streak tests green. For the batch fold, each key is pinned on its own:
reverting it to `detail` alone turns both new batch tests red, restoring
`hint` alone leaves the reference test red, restoring `url` alone leaves
the
  403 test red.
- After rebase onto 45203a4a6: `uv run pytest
tests/test_media_gen_tool.py
  tests/test_security_web_ssrf.py -q -p no:randomly` -> 100 passed;
  `make lint-python`, `make lint-types`, `make check-source-language`,
  `make check-large-files` -> clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Tool output shape changes: a caller reading the reason out of `error`
now
finds it in `detail`. The readers in this repo are the model itself and
the
tests updated here; `tests/test_security_web_ssrf.py` is the one
behavioural
assertion that moved, and it still pins that a private address is
refused for
being private. Rollback is a plain revert of this commit - no state, no
migration, no config.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A. Finishes the envelope work started in #410, which fixed the
reader's
handlers and left the gate in front of them and every media tool.

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-19 05:23:01 -07:00
Zuyi Zhou
210e2971c2
feat(*): dead-end predicates, verbatim tool bodies and envelope failures (#410)
## Summary

Absorbs the first of two remaining commits from the research fork before
it stopped
iterating. This one lands instruments only: nothing here changes what a
shipped run does,
and the research flow label does not move because no model-visible text
changed.

`raven/agent/loop/dead_end.py` (new) asks whether a finished turn ended
on the model's own
prose. The test is structural, with a textual sub-label for the tail
shape. Three decisions
are worth review:

- It lives under `raven/agent/loop/` rather than in the research plugin,
because the loop
is what would re-run a dead turn, and a host module cannot import a
plugin without
  breaking `make lint-imports`.
- The ask labeller is injected (`ask_kind`) rather than imported,
because only the product
that writes the harness asks knows how they are worded. With no labeller
the failure mode
is still recognised structurally and reported as `harness_ask_unknown`,
so the boolean
  never depends on wording.
- The fork marked its injected turns with a key of its own. This loop
already had an
equivalent in `_HOOK_INJECTED_KEY`, so the predicate reads that rather
than adding a
  second marker for the same fact.

The conditional rerun that reads the predicate arrived on this branch
afterwards, in
#418, along with the turn's wall clock. Both are supplied as data a hook
writes onto the
turn's metadata rather than as config the loop reads, so an agent whose
hooks leave no
budget runs exactly as it did before either existed: the rerun is
unreachable for it and
the clock is never consulted. The rerun starts from the original
question rather than
salvaging the failed attempt, because salvage was measured on 25 real
duds to produce
content 23 times and the right answer none of them, turning a detectable
zero into a
confident wrong answer.

Three defects in that work were found by review and fixed on this
branch, each with a
reproduction that is red on the commit before it. The loop numbered
episodes from an
iteration count that restarts, so a rerun emitted indices [0, 1, 2, 0]
and the TUI, which
keys episode rows and their fold state by that index, had the rerun's
first step collide
with the first attempt's. Each attempt took its own clock reading, so a
first attempt
ending just under the limit handed the rerun a fresh full budget and a
ten-second turn ran
to eighteen. And the research hook opens its per-turn ledger at
iteration 1, which a rerun
reaches again, so the second open replaced the path the close reads and
left the first
attempt's file unreachable on disk.

Reading that back afterwards turned up three more, none of them raised
in review. The
turn's end record carried the attempt's iteration count beside the
turn's elapsed time
with nothing saying which was which, so it gained ``attempt``. The hook
contract described
that record as "status and iteration count" while it carried five
fields, which hid
``stopped_by`` from anyone writing a terminal gate against the contract.
And the rerun's
progress line named research in a loop that serves every agent, and
called itself the
first attempt when the budget allows more than one.

A verbatim body sink (`RAVEN_VERBATIM_SINK`) records search and fetch
results at three
chokepoints. It is a second file rather than more columns on the ledger
because a 250 KB
row would leave no complete line in the 64 KB tail window
`_resolve_session_seq` reads,
which would silently reset the re-run split. Off unless the environment
names a file, and a
write failure is logged rather than raised.

`DigestOutput` lets a digest return a write-only ledger annotation
beside its text. The
annotation is stripped from the tool result and merged into the fetch
row, so a digest can
never label its own output where the model can read the label. Nothing
emits one yet; this
is the pipe a later digest sidecar uses.

`is_hard_tool_failure` was blind to the whole reader path. `web_fetch`
never returns a bare
string, it returns a JSON envelope, so a spent quota, a rejected URL and
a failed
validation all read as success and reset the streak `web_search` had
accumulated.
Transient markers are still tested first, so a 429 inside an envelope
stays retryable.

Forced finalize now books its own salvage spend. The phase fires on
every run that produced
no answer and burns up to `max_tokens` doing it, and it was counted
nowhere.

The Chinese refusal opener lives in `raven/i18n/zh_lexicon.py`, which is
the exemption zone
this repo keeps Chinese forms in, and the predicate imports it from
there rather than
carrying a literal outside the zone.

Two defects found in the first review round are fixed on this head, both
reproduced before
being changed.

The verbatim sink borrowed the ledger's `_resolve_session_seq`, which
recovers the re-run
split by reading the last 64 KB of the file and parsing the last
complete line. That works
for the ledger because its rows are small; this sink stores the large
ones, so the first
record wider than the window leaves nothing parseable and the sequence
resets to 1, making
two runs indistinguishable. Runs are now told apart by a tag minted in
memory once per
process and never recovered from disk, which removes the read a large
row defeats. The
argument for keeping this file separate from the ledger was already in
its docstring; it
applies to the sequence too, and now says so.

`is_hard_tool_failure` started counting JSON envelopes while
`failure_class` still read
them as undifferentiated text. The streak key is `(tool, failure_class)`
and the break
threshold is two, so a blocked URL followed by a reader's HTTP refusal
fired the
stop-repeating nudge at a model that had changed both its cause and its
approach. Both
predicates now share one envelope parse, and an envelope is classified
by its own `error`
string rather than by a textual ladder that cannot see into one.
Coarseness is for model
prose, which varies without meaning anything; a tool writes its error
from a small fixed
vocabulary, so equal strings are the same failure and different ones are
different. The
parts that vary with the page travel in the envelope's other keys, so
they cannot split a
streak.

The last commit removes an `isinstance` guard in the envelope reader
that the diff-coverage
report flagged as the only uncovered changed line. JSON has one shape
that opens with a
brace, so the branch was unreachable; removed rather than covered.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run at this head (95c975b3), on `origin/main` with nothing to rebase
onto.

```
uv run --frozen --python 3.12 --all-extras pytest -q tests/
  22132 passed, 43 skipped, 109 warnings in 388.67s
```

Integration and e2e are deselected by default (`-m "not integration and
not e2e"`), so the
run above does not include them. This change adds no integration
surface.

```
make coverage
make coverage-ratchet
  line   current=88.66% baseline=87.30% delta=+1.36pp
  branch current=81.59% baseline=79.43% delta=+2.15pp
  Coverage ratchet passed with 0.05pp tolerance.
make coverage-baseline-check
  Coverage baseline update is monotonic.
COVERAGE_BASE_REF=origin/main make coverage-diff
  Diff coverage: 100.00% (126/126 executable changed lines)
```

Diff coverage reported 99.21% on the commit before last, with one
uncovered changed line:
the sentence the rerun says to the reader, which no test drove because
none passed an
`on_progress` into a retrying turn. The last commit pins it rather than
waiving it.

```
make lint-python
  All checks passed! / 1946 files already formatted
make lint-imports
  Contracts: 10 kept, 0 broken.
make lint-deps
  Success! No dependency issues found.
make check-source-language
  clean
make check-large-files
  clean
make check-commits
  clean
```

The kernel budget gate caught one thing worth naming: documenting the
turn-end record in
the hook contract put `raven/contracts` at 2808 lines against a ceiling
of 2791. The
per-field detail moved to the write site in `turn_path`, which is not
budgeted, and the
paper now names the fields and says where the rest is. `raven/contracts`
is at 2788.

Each of the three review fixes was proved against the commit it fixes
rather than
reasoned about: a detached worktree at that commit, the new tests copied
in, red there and
green here. They reproduce the reviewer's own numbers -- the episode
list `[0, 1, 2, 0]`,
the eighteen seconds, and the stranded ledger file. The strengthened
comparison in
`test_the_retry_starts_from_the_question_not_from_the_wreckage` is a
tightened guarantee
rather than a regression test and passes either way, which is why it is
called out here.

`agents/raven-research/.env` was moved aside for every suite run: it is
gitignored and
absent in CI, and two launcher tests that delete a key from the
environment otherwise find
one in the file.

This machine has no search-provider key, so no research brief can
actually be run here.
The retrieval-path changes are verified by unit tests only, which is why
the three
chokepoints are pinned by tests that assert both the presence and the
absence of a row
rather than by an observed run. The same limit applies to the rerun: the
clock and the
dead-end trigger are driven by a deterministic clock in tests, never by
a real turn.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing surface changed, so there is nothing to document. The one
operator-visible
knob, `RAVEN_VERBATIM_SINK`, is an instrument that is off unless set and
is documented in
the module that reads it.

## Risk

The `is_hard_tool_failure` change is the only unconditional one, and it
reaches every agent
rather than just research: any tool returning a JSON envelope with a
non-empty top-level
`error` now counts toward the tool-failure streak that triggers the
change-approach nudge.
That is the intended correction, and the direction is conservative
because transient
markers are still tested first. Reverting it is a single branch.

Everything else is inert by default. The body sink writes nothing unless
the environment
names a file. `DigestOutput` has no emitter in this change, and a digest
returning a plain
string produces the same envelope it always did, which
`test_a_bare_string_digest_still_works` pins.

The dead-end predicate now has a caller, and it reaches only an agent
whose hooks write a
budget onto the turn's metadata. No agent on this branch does:
`agents/raven-research`
ships `wallClockSeconds` and inherits the rerun from its plugin's class
default, and every
other agent leaves the key unwritten and is bounded exactly as before.
The turn's overrun
is one budget plus at most one iteration, because the clock is read
between iterations and
never mid-generation: cancelling a call in flight would discard a
finished generation and
leave no answer at all. Rolling the rerun back is the budget key;
rolling the clock back is
the same.

Rollback is the whole commit; nothing here is depended on by anything
already on main.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
Co-authored-by: KT <74288668+0xKT@users.noreply.github.com>
2026-09-18 15:45:02 +08:00
0xKT
8716ec525c feat(docker): run raven in a docker container and make a first run configurable from the page
## Summary

Two things arrive together because the second is what made the first usable.

**Run raven from a container.** A Dockerfile builds the page, the Python
environment and the engine into one image; `cd docker && docker compose up`
serves it. nginx serves the built page and proxies the RPC socket to the engine
over the loopback they share -- the engine binds 127.0.0.1 only, so a proxy in a
second container would have no address to dial. Compose publishes on 127.0.0.1,
and the entrypoint prints a one-time sign-in link on boot.

**A first run is no longer a dead end.** The gateway used to refuse to start
until a provider was configured, while being the thing that serves the page
where a provider gets configured. It now starts, says what is missing, and picks
up a key saved in Settings on the next turn without a restart. The onboarding
overlay no longer takes the page either: New chat, the model chip and Send send
the reader to Settings > Models instead, and an empty model list is marked on
the control that fills it.

Also carried: the provider catalogue registry (capabilities, modalities and
context windows per model), the reworked Settings > Models pane, and four fixes
found in review of that work -- a sibling catalogue answering 200 with a
non-catalogue body no longer crashes the whole fetch; adding a tag no longer
erases the label and description already stored for that model; `authenticated`
no longer demands a bearer token from a local provider reached by address; and
registry rows are keyed by the spelling every lookup produces, which recovers
593 rows that no reader could reach.

## Type

- [ ] Fix
- [x] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Commands run on the branch head, with their results:

- `uv run pytest tests -q -k "provider or registry or catalog or model or rpc or gateway or docker or i18n"` -- 4389 passed, 46 skipped
- `npx vitest run` in `ui-web` -- 1573 passed (101 files)
- `npx vitest run` in `ui-tui` -- 2037 passed (142 files)
- `uv run ruff check raven scripts tests` and `ruff format --check` -- clean
- `npx tsc --noEmit` in `ui-web` and `npm run type-check` in `ui-tui` -- clean
- `npm run lint:rpc` in `ui-tui` -- generated.ts in sync
- `make check-source-language`, `make check-large-files` -- clean

Each of the four review fixes was checked by reverting it and watching a named
test go red, then restoring it.

The container image itself has not been built by CI or by me -- no Docker daemon
was available on the machine this was written on. It has been built and run by
the author of the branch.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Two deliberate security trade-offs in the container deployment, both documented
in `docker/README.md`:

- nginx rewrites the `Origin` header, because the engine accepts only an Origin
  naming its own loopback port and no browser reaching a published port can send
  one. This drops a defence-in-depth layer. The session cookie is
  SameSite=Strict, so an unauthenticated socket is still refused -- but SameSite
  ignores ports, so another page on localhost counts as same-site.
- `RAVEN_AUTO_LOGIN=1` is the default: nginx attaches the session cookie to
  every page load, so anyone who can reach the published port is signed in. It
  is contained by Compose publishing on 127.0.0.1 only. Set it to 0 for any
  deployment reachable from elsewhere. These are two settings that must agree,
  and nothing warns when they do not.

Behaviour changes outside the container: the gateway now starts without a
configured provider rather than exiting 1, and the first-run onboarding overlay
no longer opens by default (`?onboard=1` still forces it).

Rollback is by revert; nothing here migrates state or changes an on-disk format.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 03:27:52 +00:00
KevinHuSh
4a4a3fe15b feat(docker): run raven in a docker container and make a first run configurable from the page
## Summary

Two things arrive together because the second is what made the first usable.

**Run raven from a container.** A Dockerfile builds the page, the Python
environment and the engine into one image; `cd docker && docker compose up`
serves it. nginx serves the built page and proxies the RPC socket to the engine
over the loopback they share -- the engine binds 127.0.0.1 only, so a proxy in a
second container would have no address to dial. Compose publishes on 127.0.0.1,
and the entrypoint prints a one-time sign-in link on boot.

**A first run is no longer a dead end.** The gateway used to refuse to start
until a provider was configured, while being the thing that serves the page
where a provider gets configured. It now starts, says what is missing, and picks
up a key saved in Settings on the next turn without a restart. The onboarding
overlay no longer takes the page either: New chat, the model chip and Send send
the reader to Settings > Models instead, and an empty model list is marked on
the control that fills it.

Also carried: the provider catalogue registry (capabilities, modalities and
context windows per model), the reworked Settings > Models pane, and four fixes
found in review of that work -- a sibling catalogue answering 200 with a
non-catalogue body no longer crashes the whole fetch; adding a tag no longer
erases the label and description already stored for that model; `authenticated`
no longer demands a bearer token from a local provider reached by address; and
registry rows are keyed by the spelling every lookup produces, which recovers
593 rows that no reader could reach.

## Type

- [ ] Fix
- [x] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Commands run on the branch head, with their results:

- `uv run pytest tests -q -k "provider or registry or catalog or model or rpc or gateway or docker or i18n"` -- 4389 passed, 46 skipped
- `npx vitest run` in `ui-web` -- 1573 passed (101 files)
- `npx vitest run` in `ui-tui` -- 2037 passed (142 files)
- `uv run ruff check raven scripts tests` and `ruff format --check` -- clean
- `npx tsc --noEmit` in `ui-web` and `npm run type-check` in `ui-tui` -- clean
- `npm run lint:rpc` in `ui-tui` -- generated.ts in sync
- `make check-source-language`, `make check-large-files` -- clean

Each of the four review fixes was checked by reverting it and watching a named
test go red, then restoring it.

The container image itself has not been built by CI or by me -- no Docker daemon
was available on the machine this was written on. It has been built and run by
the author of the branch.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Two deliberate security trade-offs in the container deployment, both documented
in `docker/README.md`:

- nginx rewrites the `Origin` header, because the engine accepts only an Origin
  naming its own loopback port and no browser reaching a published port can send
  one. This drops a defence-in-depth layer. The session cookie is
  SameSite=Strict, so an unauthenticated socket is still refused -- but SameSite
  ignores ports, so another page on localhost counts as same-site.
- `RAVEN_AUTO_LOGIN=1` is the default: nginx attaches the session cookie to
  every page load, so anyone who can reach the published port is signed in. It
  is contained by Compose publishing on 127.0.0.1 only. Set it to 0 for any
  deployment reachable from elsewhere. These are two settings that must agree,
  and nothing warns when they do not.

Behaviour changes outside the container: the gateway now starts without a
configured provider rather than exiting 1, and the first-run onboarding overlay
no longer opens by default (`?onboard=1` still forces it).

Rollback is by revert; nothing here migrates state or changes an on-disk format.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 03:27:52 +00:00
0xKT
9c965e2470 fix(rpc): anchor the file viewer's state-dir fence on agent home, not the session workdir
## Summary

The web file viewer refused every file a sub-agent's DAG node writes, and said the
wrong thing about why. Clicking such a report answered `403`, which the page renders
as "Raven is confined to the working directory" -- while `restrict_to_workspace` was
off and the real refusal came from the state-directory fence.

`/file` resolves a request against the session's working directory when the page names
a session. It also handed that directory to `in_state_dir` in the role that function
reserves for agent home. The fence derives its two exemptions from that argument -- the
workspace itself, and the per-channel working-directory root beside it -- so with the
session's directory in home's place the exemptions covered `~/.raven/tmp/<channel>` and
a sibling that does not exist. Everything the DAG runner writes under
`~/.raven/workspace/sessions/` then read as raven's own state and was refused, which is
the same defect the workspace exemption exists to prevent, one directory over. A
node's report was drawn on the shelf as a delivery and would not open.

`resolve_readable` now keeps the two roles apart. The caller's workspace still anchors
relative paths and the `restrict_to_workspace` fence; the state-directory fence is
anchored on `cfg.workspace_path`, and the session's directory is passed alongside it
through a new variadic parameter that adds to what the fence exempts rather than
replacing what it derives. A session pinned to a directory under the state root
therefore keeps its files, and every exemption -- derived or added -- is held to the
same direction check, so the state root itself and its ancestors still earn none.
`serve.json` and `config.json` stay refused on every path tested.

`_resolve_inside`, which backs `fs.list` and `fs.read`, carried the same substitution.
Its containment check hid the symptom, so nothing had surfaced there yet; it is
anchored the same way now, because the fence's contract is that the viewer, the `fs.*`
panel and `fs.reveal` answer alike for a given file.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Commands and results, all at the branch head after rebasing onto `main`:

```console
$ uv run pytest tests/test_rpc_transport.py tests/test_rpc_console.py tests/test_rpc_knowledge.py -q
147 passed in 2.84s

$ uv run pytest tests/test_rpc_files.py tests/test_utils_office.py -q
33 passed in 2.70s

$ make lint-python          # ruff check + ruff format --check
exit 0

$ make lint-imports
Contracts: 9 kept, 0 broken.

$ make lint-ui
exit 0

$ make check-source-language
exit 0

$ uv run python scripts/check_commit_messages.py origin/main..HEAD
exit 0
```

Two regression tests were added next to the existing state-directory fence cases:

- the session-workdir call reads a node's report under the sessions tree and still
  refuses `serve.json`;
- a session pinned to a directory under the state root keeps serving its own files,
  while one pinned at the state root itself does not.

The first was proved load-bearing by reverting the fix and re-running it, which failed
with `PermissionError: ... is inside raven's state directory` on the report path. The
second is unchanged by the fix and is there to pin the behaviour the change must not
lose.

Behaviour was also checked against the running gateway's own files, on the real paths:
with the session's working directory supplied, a node's report resolves and both
`serve.json` and `config.json` raise `PermissionError`.

Known gaps, disclosed rather than worked around:

- `make lint-tui` and `make lint-bridge` do not pass in this environment. The first
  reports `src/i18n/messages.generated.ts` stale, the second `tsc: not found`. Both fail
  identically on a clean `main` worktree, and this change touches neither `ui-tui/` nor
  `bridge/`.
- No browser check. The refusal and the fix were exercised through the resolver and the
  gateway's own file paths, not through a rebuilt page bundle.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

The fence's decisions are unchanged except for the one it was getting wrong. Nothing
that was refused before this change becomes readable, apart from the session-directory
subtree the fence was never meant to refuse: the added exemption is bounded by the same
direction check as the derived ones, so no ancestor of raven's state root and not the
root itself can ride it. The credential files were re-checked as refused on every path
the tests and the live probe cover.

`in_state_dir` gains a variadic parameter with no required argument, so existing calls
keep compiling and keep their previous meaning. Both in-tree callers were updated in
this change; there are no others.

Rollback is reverting the commit. The change is three files of resolver logic plus its
tests, with no migration, no stored state and no wire-format change.

## Related Issues

N/A
2026-09-11 03:26:58 +00:00
userName20260323
236f66af24 fix(rpc): anchor the file viewer's state-dir fence on agent home, not the session workdir
## Summary

The web file viewer refused every file a sub-agent's DAG node writes, and said the
wrong thing about why. Clicking such a report answered `403`, which the page renders
as "Raven is confined to the working directory" -- while `restrict_to_workspace` was
off and the real refusal came from the state-directory fence.

`/file` resolves a request against the session's working directory when the page names
a session. It also handed that directory to `in_state_dir` in the role that function
reserves for agent home. The fence derives its two exemptions from that argument -- the
workspace itself, and the per-channel working-directory root beside it -- so with the
session's directory in home's place the exemptions covered `~/.raven/tmp/<channel>` and
a sibling that does not exist. Everything the DAG runner writes under
`~/.raven/workspace/sessions/` then read as raven's own state and was refused, which is
the same defect the workspace exemption exists to prevent, one directory over. A
node's report was drawn on the shelf as a delivery and would not open.

`resolve_readable` now keeps the two roles apart. The caller's workspace still anchors
relative paths and the `restrict_to_workspace` fence; the state-directory fence is
anchored on `cfg.workspace_path`, and the session's directory is passed alongside it
through a new variadic parameter that adds to what the fence exempts rather than
replacing what it derives. A session pinned to a directory under the state root
therefore keeps its files, and every exemption -- derived or added -- is held to the
same direction check, so the state root itself and its ancestors still earn none.
`serve.json` and `config.json` stay refused on every path tested.

`_resolve_inside`, which backs `fs.list` and `fs.read`, carried the same substitution.
Its containment check hid the symptom, so nothing had surfaced there yet; it is
anchored the same way now, because the fence's contract is that the viewer, the `fs.*`
panel and `fs.reveal` answer alike for a given file.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Commands and results, all at the branch head after rebasing onto `main`:

```console
$ uv run pytest tests/test_rpc_transport.py tests/test_rpc_console.py tests/test_rpc_knowledge.py -q
147 passed in 2.84s

$ uv run pytest tests/test_rpc_files.py tests/test_utils_office.py -q
33 passed in 2.70s

$ make lint-python          # ruff check + ruff format --check
exit 0

$ make lint-imports
Contracts: 9 kept, 0 broken.

$ make lint-ui
exit 0

$ make check-source-language
exit 0

$ uv run python scripts/check_commit_messages.py origin/main..HEAD
exit 0
```

Two regression tests were added next to the existing state-directory fence cases:

- the session-workdir call reads a node's report under the sessions tree and still
  refuses `serve.json`;
- a session pinned to a directory under the state root keeps serving its own files,
  while one pinned at the state root itself does not.

The first was proved load-bearing by reverting the fix and re-running it, which failed
with `PermissionError: ... is inside raven's state directory` on the report path. The
second is unchanged by the fix and is there to pin the behaviour the change must not
lose.

Behaviour was also checked against the running gateway's own files, on the real paths:
with the session's working directory supplied, a node's report resolves and both
`serve.json` and `config.json` raise `PermissionError`.

Known gaps, disclosed rather than worked around:

- `make lint-tui` and `make lint-bridge` do not pass in this environment. The first
  reports `src/i18n/messages.generated.ts` stale, the second `tsc: not found`. Both fail
  identically on a clean `main` worktree, and this change touches neither `ui-tui/` nor
  `bridge/`.
- No browser check. The refusal and the fix were exercised through the resolver and the
  gateway's own file paths, not through a rebuilt page bundle.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

The fence's decisions are unchanged except for the one it was getting wrong. Nothing
that was refused before this change becomes readable, apart from the session-directory
subtree the fence was never meant to refuse: the added exemption is bounded by the same
direction check as the derived ones, so no ancestor of raven's state root and not the
root itself can ride it. The credential files were re-checked as refused on every path
the tests and the live probe cover.

`in_state_dir` gains a variadic parameter with no required argument, so existing calls
keep compiling and keep their previous meaning. Both in-tree callers were updated in
this change; there are no others.

Rollback is reverting the commit. The change is three files of resolver logic plus its
tests, with no migration, no stored state and no wire-format change.

## Related Issues

N/A
2026-09-11 03:26:57 +00:00
0xKT
324a750b07 fix(*): retire the trunk output-drop pin and the claims it held
## Summary

`tests/test_agents_code_flow_exec.py` carried a pin on trunk's `DirectExecutor`
dropping a timed-out command's output. That was one of the reasons the code flow
replaced the executor, and the pin's own docstring said what should happen to
it: "pinned so a trunk fix can retire it". The trunk fix landed, the pin
asserted the opposite of the truth, and `main` went red.

**`main` is no longer red.** While this sat in review, a separate change
replaced the pin with `test_trunks_executor_preserves_output_on_timeout`. That
resolved the breakage and is the version kept here; this branch was rebased onto
it rather than carrying a competing test. What remains is what that replacement
did not cover:

- **Why the fork exists on that axis.** The replacement asserts the new fact in
  one line and says nothing about what still separates the two executors. Trunk
  signals a kill through the stderr line and `exit_code` and nothing more, while
  `CodeExecResult` also flags `timed_out` and renders `TIMED_OUT_NOTE`, which
  tells the model its output is a fragment rather than a whole run. That
  difference is now asserted and explained where the pin used to sit.
- **Three prose claims that are still false on `main`.** Both module docstrings
  advertise "the fork's three fixes" and describe trunk waiting on
  `communicate()` and losing the buffered output, in the present tense; the exec
  factory's docstring lists partial output as a difference. Each is corrected to
  say what is true now and what is left.

The fork's drain now duplicates trunk's. That is noted in the docstring as the
fork owners' call, because nothing depends on which of the two performs it and
this change should not spend itself on that. No code is removed and no behaviour
changes.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
  tests/test_sandbox_unit.py tests/test_agents_oncall_flow_hooks.py` -- 183 passed
- `ruff check` and `ruff format --check` over the changed files -- clean
- `lint-imports` -- 9 contracts kept, 0 broken
- `python scripts/check_source_language.py origin/main...HEAD` -- exit 0
- `python scripts/check_commit_messages.py origin/main..HEAD` -- exit 0

Mutation, to show the added assertions are load-bearing rather than decorative:

```
trunk discards partial stdout again   -> 1 failed   (the inherited assertions)
fork stops rendering TIMED_OUT_NOTE   -> 2 failed   (the added pair)
```

One mutation deliberately reported because it did **not** bite. An earlier
revision asserted only that trunk's rendering lacks `TIMED_OUT_NOTE`. Making
trunk emit a partial-output note in its own wording left the file green, because
the constant is matched verbatim -- and that form of assertion is equally
satisfied by a fork that stopped marking anything at all. Both sides are now
measured in one run, so the absence reads as a difference rather than as a
feature nobody has; removing the fork's append is what the second mutation
above does.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Left unchecked: no user-facing document describes either executor's timeout
behaviour. The three corrected texts are module and factory docstrings. The
model-facing half was checked separately and needed nothing -- the `timeout`
parameter description says a command that hits its limit "is killed and the
output it produced so far comes back marked partial", which is still true of
this tool.

## Disclosed

Four failures in `tests/test_ppt_engine_build_tool.py` appear in a wider
selection and are **not** from this change. They reproduce at the `main` tip in
a worktree holding none of it, and the cause is local layout rather than code:
`raven_ppt` resolves to the primary checkout's `plugins-dist/ppt-engine`, which
sits at an older revision than a fresh worktree's tests. Not investigated
further here.

How the red reached `main` is worth stating, since the cause was process rather
than code. The executor fix was pushed and then rebased onto a `main` that had
gained the pin in the meantime; the rebase surfaced the collision, and the merge
landed on the pre-rebase head before the rebase was pushed. Rebasing before the
push, as the contribution rules require, would have surfaced it while that
merge request was still open.

## Risk

No behaviour change: assertions added to an existing test, three docstrings
corrected, no production statement altered and no code removed. The duplicated
drain in the fork's executor is left exactly as it is.

Rollback is a revert of the two commits. `main` stays green either way, since
the breakage itself was resolved separately; what a revert costs is the stated
difference between the two executors and the accuracy of the three docstrings.

## Related Issues

N/A
2026-09-11 02:45:09 +00:00
LivXue
f2d76abce8 fix(*): retire the trunk output-drop pin and the claims it held
## Summary

`tests/test_agents_code_flow_exec.py` carried a pin on trunk's `DirectExecutor`
dropping a timed-out command's output. That was one of the reasons the code flow
replaced the executor, and the pin's own docstring said what should happen to
it: "pinned so a trunk fix can retire it". The trunk fix landed, the pin
asserted the opposite of the truth, and `main` went red.

**`main` is no longer red.** While this sat in review, a separate change
replaced the pin with `test_trunks_executor_preserves_output_on_timeout`. That
resolved the breakage and is the version kept here; this branch was rebased onto
it rather than carrying a competing test. What remains is what that replacement
did not cover:

- **Why the fork exists on that axis.** The replacement asserts the new fact in
  one line and says nothing about what still separates the two executors. Trunk
  signals a kill through the stderr line and `exit_code` and nothing more, while
  `CodeExecResult` also flags `timed_out` and renders `TIMED_OUT_NOTE`, which
  tells the model its output is a fragment rather than a whole run. That
  difference is now asserted and explained where the pin used to sit.
- **Three prose claims that are still false on `main`.** Both module docstrings
  advertise "the fork's three fixes" and describe trunk waiting on
  `communicate()` and losing the buffered output, in the present tense; the exec
  factory's docstring lists partial output as a difference. Each is corrected to
  say what is true now and what is left.

The fork's drain now duplicates trunk's. That is noted in the docstring as the
fork owners' call, because nothing depends on which of the two performs it and
this change should not spend itself on that. No code is removed and no behaviour
changes.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest tests/test_agents_code_flow_exec.py tests/test_agents_code_tools_plugin.py
  tests/test_sandbox_unit.py tests/test_agents_oncall_flow_hooks.py` -- 183 passed
- `ruff check` and `ruff format --check` over the changed files -- clean
- `lint-imports` -- 9 contracts kept, 0 broken
- `python scripts/check_source_language.py origin/main...HEAD` -- exit 0
- `python scripts/check_commit_messages.py origin/main..HEAD` -- exit 0

Mutation, to show the added assertions are load-bearing rather than decorative:

```
trunk discards partial stdout again   -> 1 failed   (the inherited assertions)
fork stops rendering TIMED_OUT_NOTE   -> 2 failed   (the added pair)
```

One mutation deliberately reported because it did **not** bite. An earlier
revision asserted only that trunk's rendering lacks `TIMED_OUT_NOTE`. Making
trunk emit a partial-output note in its own wording left the file green, because
the constant is matched verbatim -- and that form of assertion is equally
satisfied by a fork that stopped marking anything at all. Both sides are now
measured in one run, so the absence reads as a difference rather than as a
feature nobody has; removing the fork's append is what the second mutation
above does.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Left unchecked: no user-facing document describes either executor's timeout
behaviour. The three corrected texts are module and factory docstrings. The
model-facing half was checked separately and needed nothing -- the `timeout`
parameter description says a command that hits its limit "is killed and the
output it produced so far comes back marked partial", which is still true of
this tool.

## Disclosed

Four failures in `tests/test_ppt_engine_build_tool.py` appear in a wider
selection and are **not** from this change. They reproduce at the `main` tip in
a worktree holding none of it, and the cause is local layout rather than code:
`raven_ppt` resolves to the primary checkout's `plugins-dist/ppt-engine`, which
sits at an older revision than a fresh worktree's tests. Not investigated
further here.

How the red reached `main` is worth stating, since the cause was process rather
than code. The executor fix was pushed and then rebased onto a `main` that had
gained the pin in the meantime; the rebase surfaced the collision, and the merge
landed on the pre-rebase head before the rebase was pushed. Rebasing before the
push, as the contribution rules require, would have surfaced it while that
merge request was still open.

## Risk

No behaviour change: assertions added to an existing test, three docstrings
corrected, no production statement altered and no code removed. The duplicated
drain in the fork's executor is left exactly as it is.

Rollback is a revert of the two commits. `main` stays green either way, since
the breakage itself was resolved separately; what a revert costs is the stated
difference between the two executors and the accuracy of the three docstrings.

## Related Issues

N/A
2026-09-11 02:45:09 +00:00
0xKT
7a0b5dd311 feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary

An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.

- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.

What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.

## Review round 1 (2026-09-09, on !600, superseded by this MR)

- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.

## Review round 2 (2026-09-10)

- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.

## Review round 3 (2026-09-10)

- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.

## Review round 4 (2026-09-10)

- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
  tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q      # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
  tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
  tests/test_agents_oncall_flow_escalation_tools.py -q                                       # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents        # clean
uv run lint-imports                                                                          # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit   # in sync, clean
```

New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).

Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)

## Risk

- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.

A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.

## Related

Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
silverLXT
83aac42909 feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary

An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.

- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.

What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.

## Review round 1 (2026-09-09, on !600, superseded by this MR)

- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.

## Review round 2 (2026-09-10)

- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.

## Review round 3 (2026-09-10)

- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.

## Review round 4 (2026-09-10)

- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
  tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q      # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
  tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
  tests/test_agents_oncall_flow_escalation_tools.py -q                                       # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents        # clean
uv run lint-imports                                                                          # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit   # in sync, clean
```

New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).

Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)

## Risk

- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.

A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.

## Related

Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
Tchen
3f1453d5fe fix(providers): honour the declared output ceiling, retry differently, record a failed call
## Summary

A model call that returned nothing used to end a turn, and nothing in the record said why.
This branch closes that in five parts, all on the host runtime.

- **The output ceiling is the model's, not ours.** `resolve_max_output_tokens` takes the
  figure the model declares instead of a repo default, reserves it against the id the
  request actually goes out under, and counts cached prompt tokens toward the window so a
  cached prefix stops looking free. One call is given 64k of room and says so on the wire;
  the transcript summary budget goes to 32k. Two small output pins keep deliberate
  headroom, and the commit says why they do not move.
- **A retry has to ask a different question.** An empty completion previously repeated the
  same request and got the same answer. The ladder now descends by **what a rung sends**
  rather than by its label, the transcript summary is no longer asked to think, and the
  empty-completion retry differs from the call it replaces.
- **Time to first byte is bounded.** `llmFirstByteTimeout` applies on the direct adapters
  as well as through the gateway, a bound of zero means off rather than a bound at the
  total, and the guard's deadline is distinguished from a strategy's own `TimeoutError` so
  a fallback answers for the right one.
- **An effort travels as an effort.** `glm` is out of `inferred_protocol`'s Anthropic prefix
  tuple and the hand-written `_THINKING_BUDGETS` table is deleted, so a label reaches the
  vendor as a label rather than as a token count this repo invented. Measured on the wire
  before the change: 42 calls produced 261,980 characters of reasoning against a budget of
  1024 that the route discarded, while our own `max_tokens` went out on every call because
  that transport requires the field.
- **The record says what was asked and what came back.** A failed call is visible in the
  turn and readable in the record; the reasoning count is kept; a failure's headers are
  recorded as the failure's, not as a stream's server; `llm.input` gains a `generation`
  block and the span gains `llm.request.*`, with a test pinning that no credential appears
  in either. `llm.request_bytes` is documented as an estimate instead of claiming to be
  the wire. A frozen turn is stopped rather than nudged once.

## Type

- [x] Fix
- [x] Feature

## Verification

- `uv run --frozen pytest` over the full suite: **20354 passed**, with 6 failures and 20
  errors that are this box lacking `/usr/share/zoneinfo` -- all in `test_cron_tool.py` and
  `test_cli_cron_commands.py`, which this branch does not touch. Reproduced with the
  identical signature on a clean detached `origin/main` worktree, and CI ships tzdata, so
  its `tests` job is the authoritative check. Gated on `set -o pipefail` and
  `${PIPESTATUS[0]}`.
- `make lint-python` exit 0; `lint-imports` 9 kept, 0 broken.
- Contract pins re-derived against current `main` rather than carried over:
  `CONTRACTS_VERSION` **17** (main is 16), the digest from one run of the pinning test, and
  `CONTRACTS_LINE_CEILING` **2_791** against 2771 actual. 34 pinning tests pass.
- The request shape verified on real captured bodies rather than inferred: after the change
  a deck call is `POST /v1/chat/completions` carrying `reasoning: {"effort": ...}` at the
  top level, the configured temperature, and **no** `max_tokens`; before it, the same call
  was `POST /v1/messages` carrying `thinking: {"type": "enabled", "budget_tokens": 1024}`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

The routing change moves every `glm` model from the native Anthropic Messages transport to
LiteLLM chat. That changes which client serves the deck agent and the host default model,
so it is the one item here worth a real run before it is relied on. Measured consequences
on a live deck: no truncations, no length-stops, no empty completions and no retries across
40 calls, and prompt-cache read rates in the same band as the Messages route -- `glm` takes
no `cache_control` breakpoints on either transport because OpenRouter caches it
automatically, so caching is unaffected. A provider that needs an Anthropic-compatible
address can still name `protocol: "anthropic"` explicitly, and a test pins that route.

Rollback is per part: each of the five is its own commit range, and the routing change is a
single line in `raven/providers/protocol.py`.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 02:37:37 +00:00
Handsome-wzw
7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00
forrestjgq
f01231fef6 fix(cli): overhaul trajectory browser interaction ux (#379)
## Summary

Overhaul the interactive `raven trajectory` browser end to end, fixing
every
issue from real-terminal testing:

- Navigation: every screen opens with a help area under its title (page
  purpose plus "[Key] function" hotkey hints, rendered in a new readable
help color). Esc walks one level up (action -> attempt -> session); on
the
top-level session screen Esc stays put and quitting is Ctrl+C only,
which
prints a single exit notice. Esc inside a running action cancels just
that
action through a dedicated exception at the action boundary, so the back
  sentinel can never reach a conversion or the data layer; a cancelled
  report keeps its pre-confirm bundle-and-pin side effects (the existing
  declined-report contract). Back/Exit menu entries are gone; answered
  prompts erase themselves and navigation is echoed as breadcrumb lines
  with markup-escaped untrusted text.
- Tables: the session list is TITLE | ATTEMPTS | LAST ACTIVITY and the
  attempt list is # | STARTED | TURNS | SPANS | VERDICT | PIN | MERGED |
PREVIEW with green check cells. Time cells keep the year. The layout is
  budgeted per screen build from the live terminal width (row indent, a
safety margin, and inter-column gaps are all part of the math); PREVIEW
  drops as a whole column when it no longer fits, and below the dynamic
  minimum width the tightest layout is kept while the renderer clips
  overlong lines at the terminal edge. Every dynamic cell passes a
  single-line sanitization gate before width math (verdict sidecars only
  guarantee a non-empty string), cells are measured and truncated by
terminal display width (CJK is 2 cells), and fixed column widths always
  fit their headers.
- Space preview: Space on an attempt row prints that attempt's per-turn
input/output previews, collected in the same snapshot scan and sorted by
a fully-stringified key so ordering never follows log source order (the
  table PREVIEW cell derives from the same sorted collection). Any key,
arrow keys included, returns with the cursor kept on the row; the waiter
  passes the terminal's CPR reply through to the renderer instead of
  treating it as a keypress, and Ctrl+C keeps the browser-wide cancel.
- Merge hotkey: with two or more attempts, m/M opens the multi-select
merge
  (advertised in the help line; the old "Merge attempts" menu entry is
gone). The checkbox screen carries its own help line for toggle,
confirm,
  and cancel. A single attempt neither binds nor advertises the key.
- Theme and feel: new success (green check) and help (readable hint
text)
colors in both palettes, all WCAG AA on their backgrounds; a lone ESC is
flushed after 50ms instead of prompt_toolkit's 0.5s default, removing
the
  laggy Esc feel.

The data layer is untouched: changes are confined to
raven/cli/trajectory_browse.py, raven/cli/_theme.py, and their tests.
The
id-taking subcommands (raven trajectory save/list/...) behave as before.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `COLORTERM=truecolor uv run pytest tests/test_cli_theme.py
tests/test_cli_trajectory_browse.py
tests/test_cli_trajectory_commands.py -q` -> 220 passed
- `COLORTERM=truecolor uv run pytest tests/ --ignore=tests/integration
-q` -> 6917 passed; the 10 failed + 20 errors are the pre-existing
environment-dependent set, identical file-by-file on a clean main
checkout (cron, everos server, config loader, tracing viewer)
- `uv run ruff check` and `uv run ruff format --check` on the four
touched files -> clean
- PTY smoke on a CPR-answering pseudo terminal at 100 and 58 columns:
full flow (preview, merge cancel, action screen, staged Esc backout,
Ctrl+C quit) and the narrow-width degradation contract
- New coverage includes real pipe-input key-binding tests (Esc on all
four prompt kinds, Space/m/M hits, waiter key paths with CPR
pass-through), table budget/clipping boundaries at the computed minimum
width, and corrupt verdict/timestamp corpora

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The interactive browser's look and key protocol change wholesale (that
is
  the point of this PR): Esc no longer quits from the top level (Ctrl+C
  does), Back/Exit entries are gone, and lists render as tables. The
machine-facing subcommands and the trajectory data layer are untouched.
- Rollback: revert the squash commit; the browser returns to the
previous
  interaction wholesale, no data migration involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-05 23:58:20 +08:00
0xKT
66cc5b5964 fix(design-engine): settle the task-state ledger before the reply ships
## Summary

A turn ends on the first reply without a tool call, and the design engine's `completion_notice` rides that reply out. In a direct chat that is a nudge with a next turn to act on it. Dispatched as a DAG node there is no next turn, so the same text reaches the caller's `dag_verdict` judge as a verdict.

The run that prompted this wrote a 147 KB page, took ten desktop screenshots and three narrow ones, reported the absolute path, and was failed. Its ledger's last revision was written 37 minutes before the reply, with two items still `in_progress` and `pending` -- the work in them had been done and never ticked. The notice was accurate; where it landed made it a verdict on work that existed.

`DesignEngineHook.after_iteration` now sends such a turn back **once**, carrying the notice plus an instruction to settle the record, using the `rollback` + `rollback_inject` seam the hook contract documents for exactly this ("bouncing a draft back with feedback").

Two decisions worth stating:

- **Decided from the ledger, not from the prose.** Sniffing whether a reply "reads as finished" is a language judgement and would need a marker list per language. Settling the record is an action the agent can always take, and both honest outcomes -- ticking what is done, or naming in the reply what is not -- are reachable from one nudge.
- **`after_send` is untouched.** If the second answer still leaves items open, the notice is appended exactly as before and the judge fails the node. That case is a fact the judge should see, so the fallback is the unchanged path rather than a new branch.

This is the guard `plugins-dist/ppt-engine` has had at the same seat (`UNFINISHED_NUDGE`, `hook.py`). The design engine was the one of the two without it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks
                            or hook_contract or hook_adapters"
  217 passed in 33.68s         (re-run at the rebased head)

python -m ruff check   plugins-dist/design-engine/raven_design/plugin/hook.py \
                       tests/test_design_engine_plugin_face.py
  All checks passed!
python -m ruff format --check <same two files>
  2 files already formatted

python scripts/check_commit_messages.py origin/main..HEAD
  exit 0, 1 commit
```

The new test was watched red before the fix and again after it was strengthened, both times against this repo's own unmodified copy of the engine:

```
FAILED tests/test_design_engine_plugin_face.py::test_stale_ledger_sends_the_turn_back_once
  assert False is True     (the base class returns a pass-through decision)
```

It drives the real `AgentHookContext`, the real `LLMResponse` and a real `TaskStateManager`, matching `tests/test_agents_oncall_flow_hooks.py` rather than the `SimpleNamespace` its neighbours in the same file use for `after_send`.

The instruction's means of compliance was checked rather than assumed: `make_update_task_state` returns `None` unless `shared.manager` is set, and `make_hook` is handed that same `shared.manager`, so the nudge cannot arrive on an install where the `update_task_state` tool that satisfies it is absent.

`lint-imports` is the gate GitLab CI does not run, and it is the one that has produced a blocker on this project before:

```
lint-imports
  Contracts: 9 kept, 0 broken.
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**User-visible change.** A design-engine turn that would have ended on an unfinished ledger now costs one extra model call. It fires on both lanes: in a direct chat the reader waits one call longer and gets a reply whose ledger agrees with it; on a dispatch the caller's judge stops failing finished work.

Bounded three ways: once per turn (a flag in `ctx.metadata`), then the loop's own `_MAX_HOOK_ROLLBACKS = 8`, and a rollback consumes no iteration.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security surface: the change adds no capability, reads no new input, and the injected text is engine-authored.

Rollback is deleting the `after_iteration` method -- the base class's pass-through restores today's behaviour exactly, and `after_send` is unmodified.

## Related Issues

N/A

## Review round 1

One blocking finding from @chandler.zhang, reproduced and fixed.

`after_iteration` fires after `stream_llm_call` has delivered every content delta, so the rollback popped history the reader had already seen, and the ACP collector concatenated the rejected draft with its replacement. Worse than reported in one respect: the retry did not tick the ledger either, so `after_send` appended the notice again -- the caller received draft, reply and notice, strictly worse than the stale notice this change was written to prevent.

```
before: ['FIRST DRAFT', 'FINAL REPLY', '<the task-state notice>']
after:  ['FINAL REPLY', '<the task-state notice>']
```

`DraftGate` (in `recovery.py`, beside `ContinuationGate`) holds a response's deltas until something keeps it. Released as soon as a response carries tool calls, so a preamble is not parked for a whole tool run; where both gates would apply the draft gate owns the continuation head cut, because holding the whole text cuts it exactly rather than on a prefix.

**Two deviations from the remedy as asked, both deliberate.**

Asked for it buffered "on this lane". There is no lane to scope to: `raven/rpc/spine.py` runs the human TUI turn and the ACP turn through the same `run_turn(..., stream=True)`, and only a CRON turn takes `stream=False`.

Scoped by capability instead -- the gate installs only where an installed hook overrides `after_iteration`. That is narrower than a lane check, and a regression is what found it. Buffering coalesces the reply into one delta, and with the predicate on `hook_ctx is not None` two existing tests went red on the incremental sequence (`assert ['Hello'] == ['Hel', 'lo']`). `_Announce` there overrides only `after_send`, so under the narrowed predicate it installs no gate and both tests stand unchanged. They are the guard on that half; nothing was weakened to make room.

Also corrected: the comment above that hook site claimed a rollback "discards-and-re-samples it", true of history and false of the stream.

**Scope this closes beyond the finding.** `plugins-dist/ppt-engine`'s `after_iteration` rolls back a text response the same way and had the identical exposure. This closes it too, the defect being a property of the seam rather than of either hook.

**Disclosed, not fixed.** A hook's `short_circuit_result` still streams the draft it replaces. That path is untouched here and predates this branch.

Re-verified at the rebased head (`origin/main` moved twice during review; both replays pure, `range-diff` all `=`). The base's own commits touched the ACP question surface, so those suites are in the re-run rather than only mine:

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks or hook_contract
                            or hook_adapters or run_emit or agent_loop_stream or acp_questions
                            or acp_elicitation or acp_dialects or ask_user"
  665 passed in 34.09s

python -m pytest tests/            19954 passed, 138 skipped in 519.84s
python -m ruff check / format --check   clean
lint-imports                            9 kept, 0 broken
scripts/check_commit_messages.py        exit 0
```

Covered at the loop level rather than by calling the hook. Disabling the gate turns it red:

```
FAILED test_a_rolled_back_draft_never_reaches_the_stream
  AssertionError: the caller saw the rejected draft: ['FIRST DRAFT', 'FINAL REPLY']
```

## Review round 3

@chandler.zhang's round-two blocker: the draft gate was scoped by whether a hook *overrides* `after_iteration`, which only says the hook watches the phase. `eval_engine`'s `AfterIterationHook` overrides it, awaits an LLM judge inside it, and its own docstring says an evaluator should never interrupt the user's reply chain -- so enabling the Eval Engine took incremental output away from a supported configuration and held the answer until the judge returned, for a draft that was never going to be rejected. Reproduced: `['Hel', 'lo']` arrived as `['Hello']`.

`AgentHook` now declares `rolls_back_iterations` (default `False`) and the loop reads the declaration. The design engine's hook and the deck engine's hook declare it -- the latter because this description claims the deck engine's exposure is closed, and the declaration is what keeps that true.

The observer path is pinned, and the pin asserts the await completed as well as the chunk sequence: `CompositeHook` swallows a hook exception, so a probe that raised past its await would otherwise pass while proving half of what it claimed. The first draft of that test did exactly that.

**Disclosed here, closed in round four.** Six research-flow gates under `agents/raven-research` (`evidence_floor`, `finalize`, `plain_first`, `report_shape`, `spin_breaker`, `verify`) answer `rollback` while their contributed hook does not declare it, so at this revision their rejected drafts still stream as they do on `main` today.

```
pytest tests/                                     19960 passed, 139 skipped
ruff check + format --check over the job's targets clean, 1849 files
lint-imports                                       9 kept, 0 broken
```

## Review round 4

@chandler.zhang's nonblocking note from round three, closed. `ResearchFlowHook` forwards every phase to a composite it builds per session, and six of that chain's gates answer `rollback`. The loop reads `rolls_back_iterations` off the contributed hook object once per turn, before any session chain exists, so nothing could derive the flag from the members that carry the behaviour -- and nothing in `flow.py` hinted the declaration was needed. The hook now declares it, with the reason recorded at the attribute.

**Enumerated rather than taken on the note.** Every `HookDecision` built with `rollback=True` across `agents`, `plugins-dist`, `raven` and `evolver` was located by AST; the research flow was the only tree whose contributed hook did not declare. The deck and design engines declare it in the same class that rolls back. `raven-code` and `raven-oncall` contribute hooks that never roll back and correctly leave the flag `False`, so this is not a case of one lane opting out of a rule. The research manifest declares exactly one hook contribution and its other three are tools, so no gate is registered directly and the contributed hook is the only level the declaration can sit at.

**Not a regression.** Draft suppression did not exist before this branch, so the research flow never held its deltas either. It gains the guarantee rather than losing one, which is why the note was graded nonblocking and why it is closed here rather than earlier.

The new test derives that producing set instead of listing it, so the omission cannot recur silently: a tree that grows a rollback and is not mapped fails as an unmapped root, and two further cases guard the derivation itself against passing vacuously. It reads the source by AST rather than importing, because `raven_design` and `raven_ppt` are editable installs rooted at the main checkout, and importing them from a worktree reads the main checkout's code instead of the revision under test.

Watched red before the fix and green after, with the derivation guards passing both times:

```
FAILED test_a_tree_that_rolls_back_declares_it_on_its_contributed_hook[agents/raven-research/...]
  AssertionError: ResearchFlowHook can roll back an iteration but does not declare
  rolls_back_iterations = True, so the loop streams a draft it may then retract
  1 failed, 4 passed
```

Read back through the loop's own predicate rather than off the class, in both directions: a real `ResearchFlowHook` instance now makes the loop hold drafts, and `eval_engine`'s judge -- which overrides the phase and only annotates -- still does not. The declaration is also not a `dataclass` field, so the hook's constructor signature is unchanged; an annotated spelling would have added a parameter and could not have sat above the fields without a default-ordering error.

Re-verified at the rebased head. `origin/main` moved six commits during review; the replay is pure (`range-diff` reports every commit `=`) and the base's three changes touch no file this branch touches, with the repo-wide rollback-producer set identical on both sides.

```
pytest tests/ -k research                            917 passed
pytest (loop, hook contract, iteration hooks, design face)   139 passed
ruff check + format --check over the job's targets   clean, 1850 files
lint-imports                                         9 kept, 0 broken
check_commit_messages.py / check_source_language.py  exit 0
make check-large-files                               exit 0
```
2026-09-09 12:31:40 +00:00
LivXue
42796ed28e fix(design-engine): settle the task-state ledger before the reply ships
## Summary

A turn ends on the first reply without a tool call, and the design engine's `completion_notice` rides that reply out. In a direct chat that is a nudge with a next turn to act on it. Dispatched as a DAG node there is no next turn, so the same text reaches the caller's `dag_verdict` judge as a verdict.

The run that prompted this wrote a 147 KB page, took ten desktop screenshots and three narrow ones, reported the absolute path, and was failed. Its ledger's last revision was written 37 minutes before the reply, with two items still `in_progress` and `pending` -- the work in them had been done and never ticked. The notice was accurate; where it landed made it a verdict on work that existed.

`DesignEngineHook.after_iteration` now sends such a turn back **once**, carrying the notice plus an instruction to settle the record, using the `rollback` + `rollback_inject` seam the hook contract documents for exactly this ("bouncing a draft back with feedback").

Two decisions worth stating:

- **Decided from the ledger, not from the prose.** Sniffing whether a reply "reads as finished" is a language judgement and would need a marker list per language. Settling the record is an action the agent can always take, and both honest outcomes -- ticking what is done, or naming in the reply what is not -- are reachable from one nudge.
- **`after_send` is untouched.** If the second answer still leaves items open, the notice is appended exactly as before and the judge fails the node. That case is a fact the judge should see, so the fallback is the unchanged path rather than a new branch.

This is the guard `plugins-dist/ppt-engine` has had at the same seat (`UNFINISHED_NUDGE`, `hook.py`). The design engine was the one of the two without it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks
                            or hook_contract or hook_adapters"
  217 passed in 33.68s         (re-run at the rebased head)

python -m ruff check   plugins-dist/design-engine/raven_design/plugin/hook.py \
                       tests/test_design_engine_plugin_face.py
  All checks passed!
python -m ruff format --check <same two files>
  2 files already formatted

python scripts/check_commit_messages.py origin/main..HEAD
  exit 0, 1 commit
```

The new test was watched red before the fix and again after it was strengthened, both times against this repo's own unmodified copy of the engine:

```
FAILED tests/test_design_engine_plugin_face.py::test_stale_ledger_sends_the_turn_back_once
  assert False is True     (the base class returns a pass-through decision)
```

It drives the real `AgentHookContext`, the real `LLMResponse` and a real `TaskStateManager`, matching `tests/test_agents_oncall_flow_hooks.py` rather than the `SimpleNamespace` its neighbours in the same file use for `after_send`.

The instruction's means of compliance was checked rather than assumed: `make_update_task_state` returns `None` unless `shared.manager` is set, and `make_hook` is handed that same `shared.manager`, so the nudge cannot arrive on an install where the `update_task_state` tool that satisfies it is absent.

`lint-imports` is the gate GitLab CI does not run, and it is the one that has produced a blocker on this project before:

```
lint-imports
  Contracts: 9 kept, 0 broken.
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**User-visible change.** A design-engine turn that would have ended on an unfinished ledger now costs one extra model call. It fires on both lanes: in a direct chat the reader waits one call longer and gets a reply whose ledger agrees with it; on a dispatch the caller's judge stops failing finished work.

Bounded three ways: once per turn (a flag in `ctx.metadata`), then the loop's own `_MAX_HOOK_ROLLBACKS = 8`, and a rollback consumes no iteration.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security surface: the change adds no capability, reads no new input, and the injected text is engine-authored.

Rollback is deleting the `after_iteration` method -- the base class's pass-through restores today's behaviour exactly, and `after_send` is unmodified.

## Related Issues

N/A

## Review round 1

One blocking finding from @chandler.zhang, reproduced and fixed.

`after_iteration` fires after `stream_llm_call` has delivered every content delta, so the rollback popped history the reader had already seen, and the ACP collector concatenated the rejected draft with its replacement. Worse than reported in one respect: the retry did not tick the ledger either, so `after_send` appended the notice again -- the caller received draft, reply and notice, strictly worse than the stale notice this change was written to prevent.

```
before: ['FIRST DRAFT', 'FINAL REPLY', '<the task-state notice>']
after:  ['FINAL REPLY', '<the task-state notice>']
```

`DraftGate` (in `recovery.py`, beside `ContinuationGate`) holds a response's deltas until something keeps it. Released as soon as a response carries tool calls, so a preamble is not parked for a whole tool run; where both gates would apply the draft gate owns the continuation head cut, because holding the whole text cuts it exactly rather than on a prefix.

**Two deviations from the remedy as asked, both deliberate.**

Asked for it buffered "on this lane". There is no lane to scope to: `raven/rpc/spine.py` runs the human TUI turn and the ACP turn through the same `run_turn(..., stream=True)`, and only a CRON turn takes `stream=False`.

Scoped by capability instead -- the gate installs only where an installed hook overrides `after_iteration`. That is narrower than a lane check, and a regression is what found it. Buffering coalesces the reply into one delta, and with the predicate on `hook_ctx is not None` two existing tests went red on the incremental sequence (`assert ['Hello'] == ['Hel', 'lo']`). `_Announce` there overrides only `after_send`, so under the narrowed predicate it installs no gate and both tests stand unchanged. They are the guard on that half; nothing was weakened to make room.

Also corrected: the comment above that hook site claimed a rollback "discards-and-re-samples it", true of history and false of the stream.

**Scope this closes beyond the finding.** `plugins-dist/ppt-engine`'s `after_iteration` rolls back a text response the same way and had the identical exposure. This closes it too, the defect being a property of the seam rather than of either hook.

**Disclosed, not fixed.** A hook's `short_circuit_result` still streams the draft it replaces. That path is untouched here and predates this branch.

Re-verified at the rebased head (`origin/main` moved twice during review; both replays pure, `range-diff` all `=`). The base's own commits touched the ACP question surface, so those suites are in the re-run rather than only mine:

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks or hook_contract
                            or hook_adapters or run_emit or agent_loop_stream or acp_questions
                            or acp_elicitation or acp_dialects or ask_user"
  665 passed in 34.09s

python -m pytest tests/            19954 passed, 138 skipped in 519.84s
python -m ruff check / format --check   clean
lint-imports                            9 kept, 0 broken
scripts/check_commit_messages.py        exit 0
```

Covered at the loop level rather than by calling the hook. Disabling the gate turns it red:

```
FAILED test_a_rolled_back_draft_never_reaches_the_stream
  AssertionError: the caller saw the rejected draft: ['FIRST DRAFT', 'FINAL REPLY']
```

## Review round 3

@chandler.zhang's round-two blocker: the draft gate was scoped by whether a hook *overrides* `after_iteration`, which only says the hook watches the phase. `eval_engine`'s `AfterIterationHook` overrides it, awaits an LLM judge inside it, and its own docstring says an evaluator should never interrupt the user's reply chain -- so enabling the Eval Engine took incremental output away from a supported configuration and held the answer until the judge returned, for a draft that was never going to be rejected. Reproduced: `['Hel', 'lo']` arrived as `['Hello']`.

`AgentHook` now declares `rolls_back_iterations` (default `False`) and the loop reads the declaration. The design engine's hook and the deck engine's hook declare it -- the latter because this description claims the deck engine's exposure is closed, and the declaration is what keeps that true.

The observer path is pinned, and the pin asserts the await completed as well as the chunk sequence: `CompositeHook` swallows a hook exception, so a probe that raised past its await would otherwise pass while proving half of what it claimed. The first draft of that test did exactly that.

**Disclosed here, closed in round four.** Six research-flow gates under `agents/raven-research` (`evidence_floor`, `finalize`, `plain_first`, `report_shape`, `spin_breaker`, `verify`) answer `rollback` while their contributed hook does not declare it, so at this revision their rejected drafts still stream as they do on `main` today.

```
pytest tests/                                     19960 passed, 139 skipped
ruff check + format --check over the job's targets clean, 1849 files
lint-imports                                       9 kept, 0 broken
```

## Review round 4

@chandler.zhang's nonblocking note from round three, closed. `ResearchFlowHook` forwards every phase to a composite it builds per session, and six of that chain's gates answer `rollback`. The loop reads `rolls_back_iterations` off the contributed hook object once per turn, before any session chain exists, so nothing could derive the flag from the members that carry the behaviour -- and nothing in `flow.py` hinted the declaration was needed. The hook now declares it, with the reason recorded at the attribute.

**Enumerated rather than taken on the note.** Every `HookDecision` built with `rollback=True` across `agents`, `plugins-dist`, `raven` and `evolver` was located by AST; the research flow was the only tree whose contributed hook did not declare. The deck and design engines declare it in the same class that rolls back. `raven-code` and `raven-oncall` contribute hooks that never roll back and correctly leave the flag `False`, so this is not a case of one lane opting out of a rule. The research manifest declares exactly one hook contribution and its other three are tools, so no gate is registered directly and the contributed hook is the only level the declaration can sit at.

**Not a regression.** Draft suppression did not exist before this branch, so the research flow never held its deltas either. It gains the guarantee rather than losing one, which is why the note was graded nonblocking and why it is closed here rather than earlier.

The new test derives that producing set instead of listing it, so the omission cannot recur silently: a tree that grows a rollback and is not mapped fails as an unmapped root, and two further cases guard the derivation itself against passing vacuously. It reads the source by AST rather than importing, because `raven_design` and `raven_ppt` are editable installs rooted at the main checkout, and importing them from a worktree reads the main checkout's code instead of the revision under test.

Watched red before the fix and green after, with the derivation guards passing both times:

```
FAILED test_a_tree_that_rolls_back_declares_it_on_its_contributed_hook[agents/raven-research/...]
  AssertionError: ResearchFlowHook can roll back an iteration but does not declare
  rolls_back_iterations = True, so the loop streams a draft it may then retract
  1 failed, 4 passed
```

Read back through the loop's own predicate rather than off the class, in both directions: a real `ResearchFlowHook` instance now makes the loop hold drafts, and `eval_engine`'s judge -- which overrides the phase and only annotates -- still does not. The declaration is also not a `dataclass` field, so the hook's constructor signature is unchanged; an annotated spelling would have added a parameter and could not have sat above the fields without a default-ordering error.

Re-verified at the rebased head. `origin/main` moved six commits during review; the replay is pure (`range-diff` reports every commit `=`) and the base's three changes touch no file this branch touches, with the repo-wide rollback-producer set identical on both sides.

```
pytest tests/ -k research                            917 passed
pytest (loop, hook contract, iteration hooks, design face)   139 passed
ruff check + format --check over the job's targets   clean, 1850 files
lint-imports                                         9 kept, 0 broken
check_commit_messages.py / check_source_language.py  exit 0
make check-large-files                               exit 0
```
2026-09-09 12:31:39 +00:00
LivXue
a54627a566 fix(*): break a loop that keeps succeeding, and name a clobbered helper
## Summary

One livelock, three causes, all three closed here. A run read one file 127 times over an hour, exit 0 on every call, and ended its turn exactly where it started -- at iteration 343 of 600 when it was killed, with the file's md5 unchanged throughout.

**The guard did not arm.** `failure_streak` counts only hard failures and `turn_path` resets the streak on any success, so a call that keeps working and keeps answering the same thing is structurally invisible to it. `raven/agent/loop/no_progress.py` counts the other shape.

It counts **per turn rather than consecutively**, and that is the load-bearing decision: the run alternated two spellings of one read -- a plain read, and the same read behind an `md5sum` -- so no two neighbouring calls were identical and a consecutive streak would have stayed at one forever. The cost is that two spellings each reach the threshold on their own, so an alternating pair fires at about fifteen calls rather than eight. Against 127 that is the trade worth taking. A test pins the arithmetic so the design is not "simplified" back to consecutive.

Threshold 8 (higher than the failure threshold's 2, because repeating a working call is ordinary -- a poll, a re-read after an edit), capped at 2 per turn like the failure nudge, and it appends a nudge and nothing else.

**The cause was silent.** `provision` rewrites every helper beside the build script on every build. That is deliberate and its docstring says why (a quietly edited helper fails in a way that reads as a deck bug), but it said nothing -- so a run whose own `themes.json` went back to the engine's copy could not tell a reverted write from one that never happened. `ppt_build` now names what it put back, and says nothing when it put nothing back.

**The documentation contradicted the data.** The generated `ppt_theme` docstring promised a single entry and called a named theme a `KeyError`. That holds on the template route, where `with_template_helpers` narrows `themes.json` to the template's own palette. Without a template the same file carries the ten reviewed themes and naming one is how an author picks -- which `tests/test_ppt_engine_assets_script_helpers.py` itself does (`THEMES["indigo-scholar"]`). The text now follows the payload.

`ppt_charts` and `ppt_layout` opened with the same `T = THEMES[...]` line and carried the template route's claim beside it, so fixing `ppt_theme` alone left two generated modules telling a catalog-route author the opposite. They now point at the module that owns the rule, and a test pins that they do not restate it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
python -m pytest tests/ -k "ppt_engine or agent_loop or turn_path
                            or failure_streak or no_progress"
  3956 passed, 94 skipped in 93.34s      (re-run at the rebased head)

python -m ruff check raven/agent/loop/ plugins-dist/ppt-engine/raven_ppt/backends/script/ \
                     plugins-dist/ppt-engine/raven_ppt/services/assets/script_helpers.py \
                     tests/test_agent_loop_tool_loop_break.py \
                     tests/test_ppt_engine_assets_script_helpers.py \
                     tests/test_ppt_engine_script_backend.py
  All checks passed!
python -m ruff format --check <the eight changed files>
  8 files already formatted

python scripts/check_commit_messages.py origin/main..HEAD
  exit 0, 1 commit
```

The 94 skips are the pre-existing `templates.manifest.json` payload skips; they are identical on `origin/main`.

Each of the three was watched red first, and the loop guard was mutation-tested rather than trusted:

```
# the nudge computed and then withheld (one line replaced by `pass`)
FAILED test_an_unchanging_successful_call_is_nudged_and_bounded
  AssertionError: an unchanging successful call was never nudged

# the one-home test against this repo's unmodified charts/layout
FAILED test_only_the_theme_module_states_the_theme_rule
  AssertionError: ppt_charts restates the template route's rule

# the restore note, before ppt_build carried it
FAILED test_a_restored_helper_is_named_in_the_build_note
  AssertionError: the build names the helper it put back
```

The nudge was then driven through a real `AgentLoop` to see it arrive rather than inferred from the counter:

```
THRESHOLD=8  MAX=2
[loop] `list_dir` has returned the same result for the same arguments 8 times in
this turn. Calling it again will not answer differently. ...
```

Both directions of the new guard were probed:

```
identical call and result -> one key          (fires: correct)
a poll whose answer moves -> 8 distinct keys  (never fires: correct)
```

`lint-imports` is the gate GitLab CI does not run, and it is the one that has produced a blocker on this project before:

```
lint-imports
  Contracts: 9 kept, 0 broken.
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**User-visible change.** Three, all additive:

1. A turn that repeats one exact call with one exact answer eight times gets one extra line appended to that tool result, at most twice per turn. Nothing is blocked, refused or rerouted.
2. A `ppt_build` that restored a helper the author had edited says so in its `note`. A build that restored nothing is unchanged.
3. A build directory with no template now ships a `ppt_theme` docstring that says to pick a theme by name. The code in that module is byte-identical; only the text differs.

**Disclosed, and deliberately not fixed.** The no-progress key includes the result, which is what separates a poll making progress from one that is not. A result carrying a clock or a per-call nonce therefore keys differently every time and never accumulates, so the guard is too tight in that direction rather than too loose. Widening it (hashing a prefix, stripping digits) would start firing on genuine progress, which is the worse failure for a nudge that tells the model to stop looking.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security surface: no new input is read, no capability is added, and all three texts are engine-authored.

Rollback is per cause and independent -- delete the `elif` in `turn_path` for the guard, drop the `if put_back:` block in `runner` for the note, or pass no flag to `theme_module_source` for the docstring.

## Related Issues

N/A

## Review round 1

Two blocking findings from @chandler.zhang, both reproduced and fixed, one commit each.

**The restore note was dropped on a failed build** (`02ef9aa4`). `outcome.note` was read only after the tool's success return, and two post-provision exits in `run_script` dropped it as well -- the missing-script return and the timeout. The livelock this note exists for was **nine failed `ppt_build` calls**, so as first submitted it would not have fired once in the incident that motivated it. The early exits above `stage.run` are pre-provision and correctly carry nothing. Now tested through `ppt_build`, on the tool's own JSON.

**The no-progress key hashed `model_text` alone** (`5a318c2e`). An image read answers with path, dimensions and a token estimate -- stable across a repaint at the same size -- while the pixels ride in the routed blocks, so a render-edit-inspect pass read "the same result" eight times in exactly the workflow the guard was added to help. Reproduced through the loop's own path:

```
what the loop counts on   : '[image: /tmp/.../slide.png] | 400x300px | ~165 tokens'
model_text identical      : True      # two 400x300 images, different pixels
image blocks identical    : False
no_progress key identical : True
```

Of the two remedies offered I took the first, a stable identity for the routed blocks, rather than excluding multimodal results -- so the guard stays live for an image loop that genuinely is stuck. Covered at loop level with a real repainted image and vision forced on, paired against the same loop over an unchanging one.

Re-verified at the rebased head (`origin/main` moved twice during review; both replays pure, `range-diff` all `=`):

```
python -m pytest tests/ -k "ppt_engine or agent_loop or turn_path or failure_streak or no_progress"
  3958 passed, 94 skipped in 87.74s

python -m ruff check / format --check   clean
lint-imports                            9 kept, 0 broken
scripts/check_commit_messages.py        exit 0
```

Each fix was watched red first:

```
FAILED test_a_restored_helper_is_named_on_a_failed_build
  AssertionError: a failed build does not name the helper it put back
FAILED test_changing_pixels_behind_one_metadata_line_are_progress
  AssertionError: a repainted image counted as no progress
```

## Review round 2

`1a601700`. @chandler.zhang's clearing note flagged a formatter-only failure and it was real: `ruff format --check` wanted the three-line `BuildOutcome(...)` added by `02ef9aa4` on one line, and the CI lint job reported it. Now clean over every changed `.py` file, with `ruff check` and the three ppt suites (208 passed) re-run after.

Cause on my side: `ruff check` was re-run over that edit and `ruff format --check` was not, and this repo's `core.hooksPath` points at a directory that does not exist, so nothing catches it at commit time.
2026-09-09 12:26:51 +00:00
0xKT
2f862d8ca2 feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
hylarucoder
c703db6ffd feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
0xKT
a671fdddcf refactor(agent): the AgentLoop constructor takes five wiring bundles and nothing else
The constructor kept a **legacy shim that folded forty-odd flat keyword names
into the wiring bundles field by field. Production stopped using it when the
entrances moved to bundles; fifty-five test files had not. Every one of those
call sites (and the helpers that forward **kw to the loop) now constructs the
bundle it means, ToolWiring / SubagentWiring / EngineWiring / TurnPolicy /
HostWiring, so a wiring field has one address in the tests as in the code. The
shim, its field-to-bundle map's private name and the docstring that promised
the old names go; resolve_wiring only fills in the default bundles.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-30 03:52:45 +08:00
0xKT
0727b2aa6a refactor(*): dissolve the five transit papers; every importer names the paper
agent/tools/base, context_engine/base, memory_engine/base, memory_engine/backend
and token_wise/base had become pure re-exports of raven.contracts, kept "so
existing import paths keep resolving". A module with no behaviour that exists
only to forward an address is debt: two names for one shape, a second place
for the ledger to miss, and a docstring that narrates a move instead of
describing anything. 154 import statements across 116 files now name the
paper directly (raven.contracts.tool / .context / .assembled / .memory /
.token_strategy / .subagent_backend), the memory_engine package face stops
re-exporting shapes it does not own, and the five modules are gone.

Docstrings that pointed at the old addresses follow (the plugin manifest and
registry, the evolver path guard, the skeleton test's relocation map); the
assembled paper's docstring describes its two carriers instead of the history
of the package they once lived in, and the loop's lazy cron-tool import says
the reason that still holds.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-29 22:05:53 +08:00
Handsome-wzw
9a87470fdf fix(rpc): turn.started is a turn event, and says so on all three sides
## Summary

Found while investigating the first row of Dizhan's `子智能体功能与显示问题` doc (*回注内容过程中未正常显示,刷新后才会正常显示*). This is not that bug's fix, but it is a real defect on the same wire, and it is the one that explains why "前后端接线没有约定好" was a fair thing to say.

**`turn.started` is not in the `TurnEvent` union — on either side.** It has a schema component, it has a Pydantic model, and neither is reachable from the union that `tests/test_rpc_schema_match.py` walks. The 290-test contract guard has never looked at this event.

So it drifted the first time anyone touched it. `05957452` (judge a dag node) taught `announce_dag_exception` to emit:

```python
mark = {"kind": "dag", "label": run_id, "status": "exception", "run_id": run_id, "node_id": node_id}
```

while `TurnStartedEvent.payload.delegated` still declared `status: enum ["ok","error"]` with `additionalProperties: false`, and `TurnStartedPayload` had **no `delegated` field at all** under `extra="forbid"`. Verified with `jsonschema` against the published contract before changing anything:

```
Additional properties are not allowed ('node_id' was unexpected)
```

That commit did carry `exception` into `TranscriptDelegated` and `SubagentDeliveredEvent` — two of the three places this status lives. The third is the one no test covers.

**It does not crash today**, and I want to be exact about that: `SubscriptionEmitter.emit` pushes a raw dict with no validation, and the page reads only the fields it knows. But the contract test's own docstring says what the cost is — *"both sides declare `additionalProperties: false` / `extra="forbid"`, so a consumer validating against the schema rejects the whole event, not just the unknown field"*. Losing this event loses the boundary that advances the workspace turn counter and draws the delegated row.

## What this does

- `TurnStartedEvent` joins the `TurnEvent` union, in `openrpc.json` and in `raven/rpc/models.py`, so the existing sweep covers it from now on.
- The schema's `delegated.status` gains `exception`; `node_id` is declared.
- `TurnStartedPayload` gains `delegated` as a typed model (`TurnStartedDelegated`), three-valued status included.
- Both generated TypeScript clients regenerated.
- A new test drives **the exact marker `manager.py` builds** through both validators, so the next status or field fails here instead of in a client nobody is watching.

## One thing the union surfaced

`ui-tui` never handled `turn.started`. Its exhaustiveness check said so the moment the variant existed. It draws the delegated row from `subagent.delivered`, which the page deliberately ignores in favour of this event — the two mark different moments: **submitted** versus **started**.

Left as it is, with an explicit `case 'turn.started'` saying why. Moving that surface onto the truer moment changes when the TUI draws the row, which is a product change, not a contract fix.

## Verification

- New test red before the fix, by construction: adding the variant to the union alone reds `test_schema_match_turn_event_discriminated_union` and `..._payload_fields` (schema/pydantic drift), which is how the gap was proved.
- Mutation checks, each reverting one thing:

| reverted | reds |
|---|---|
| `exception` out of the schema enum | 1 |
| `node_id` out of the schema | 1 |
| `delegated` off the Pydantic payload | 2 |

- `uv run pytest -q` — **12900 passed**, 50 skipped, 5 failed; the same five pre-existing on `origin/main` (four `test_security_network.py` NAT64 cases, one `test_doctor_answers_where_the_memories_are`), reproduced on an untouched merge-base worktree.
- `uv run ruff check` / `ruff format --check` on `raven tests scripts` — both clean (both halves of `make lint-python`).
- `ui`: `gen:check` 147 methods, `type-check` clean, `npx vitest run` 89 files / **1225 passed**.
- `ui-tui`: `type-check` clean, `lint:rpc` in sync, `lint:i18n` up to date, `npm test` 138 files / **1990 passed**.
  - Worth recording: with `ui-tui/node_modules` symlinked from another checkout the suite reports 82 failed / 59 failed — and reports **the same numbers on an untouched `origin/main`**. It is the symlink, not the branch. Ran `npm ci` in the worktree to get a trustworthy number.
- `scripts/commit_lint` — no errors.
- `git ls-files | grep -c node_modules` — 0.

## Type

- [x] Fix

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear

Widening a contract to match what the runtime already emits: every payload valid before is valid now. No emit site changes, no behaviour changes. The one code change outside the schema and the models is an explicit no-op branch in `ui-tui`. Rollback is a revert.
2026-08-28 12:51:10 +00:00
forrestjgq
8ecfae3e0c test(tracing): isolate span emission into per-test tmp dirs (#371)
## Summary

Unit tests were leaking tracing spans into the real `~/.raven/traces`
store. Tracing is on by default and `trace.span()` calls are embedded in
library code (agent loop, subagents, TUI RPC), so any test exercising
those paths emitted spans with fabricated session keys (`session-a`,
`weixin:c`, `tui:default`, ...). Those spans then surfaced as phantom
sessions in `raven trajectory`, while the TUI `/sessions` list (which
reads the real session store) stayed clean.

This PR adds an autouse fixture in `tests/conftest.py` that:

- redirects `RAVEN_TRACING_DIR` to a per-test tmp directory, and
- resets the lazy module-level `TraceStore` cache
(`raven.tracing.spans._store`) around each test, since it pins the
directory resolved at first emit.

Key decisions:

- Only the directory is redirected, not the enabled switch, so the suite
keeps exercising the real emission path with zero behavior change; leaks
land in tmp instead.
- This centralizes the setenv-plus-cache-reset pattern that 8
tracing/trajectory test files already perform manually. Those tests keep
working: their own `monkeypatch.setenv` runs after the fixture and wins,
and they already reset the cache themselves.
- Existing phantom sessions come from archived span logs written before
this fix; the fixture only prevents new leaks. One archive file mixes a
real session with test keys, so cleanup needs line-level filtering and
is left out of scope.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Test-suite hygiene (`test` commit type).

## Verification

- `uv run pytest tests/test_tracing_api.py
tests/test_trajectory_store.py tests/test_trajectory_bundle.py
tests/test_trajectory_replay.py tests/test_trajectory_redact.py
tests/test_cli_tracing_commands.py tests/test_cli_trajectory_commands.py
tests/test_cli_trajectory_browse.py -q`: 308 passed, 1 failed
(`test_viewer_health_true_for_our_viewer`, fails identically on
unmodified `main`, environment-specific)
- `uv run pytest tests/test_agent_loop_run_emit.py
tests/test_subagent_manager.py tests/test_tui_rpc_turn_send.py
tests/test_deep_research_tool.py tests/test_agent_loop_approval.py -q`:
109 passed; the real `~/.raven/traces` log did not grow during the run
- Positive check: running `tests/test_agent_loop_run_emit.py` with
`--basetemp` shows the previously leaked session keys (`cli:c`, `tg:c`,
`weixin:c`) now land in per-test tmp dirs
- `uv run pytest tests/ --ignore=tests/integration -q`: 6837 passed, 11
failed, 20 errors; the same failure set reproduces on unmodified `main`
(environment-specific: cron/tz, everos server, theme, config loader,
viewer health), zero regressions from this change
- `uv run ruff check tests/conftest.py` and `uv run ruff format --check
tests/conftest.py`: both pass

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Docs box left blank: test-only change, no user-facing docs affected.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Test-only change; no production code touched. One known interaction:
`RAVEN_TRACING_DIR` takes precedence over `RAVEN_HOME`, so a future test
that fakes `RAVEN_HOME` and expects traces under it must also set
`RAVEN_TRACING_DIR` itself. Rollback: revert the single commit.

## Related Issues

N/A

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-28 19:21:16 +08:00
arelchan
7c33c4133a fix(rpc): let the viewer open the files a turn actually wrote
## Summary

A delivered artifact could not be opened from the page: the file viewer answered
403 and the transcript drew "Raven is confined to the workspace, so this file
cannot be opened here" over a delivery card it had just rendered.

The viewer's state-directory fence exempts the workspace subtree, on the stated
grounds that the default workspace lives inside raven's home and denying it
would refuse every file the agent itself wrote. Since agent home and the working
directory were split apart, a turn no longer writes there: it writes in
`~/.raven/tmp/<channel>`, a sibling of the workspace, inside the same state root
and outside the carve-out. So the fence denied exactly what the carve-out exists
to allow, one directory over, for every artifact produced since that split.

This exempts the working-directory root alongside the workspace, using the same
containment test in the same direction, so a working-directory root that happens
to sit above raven's home earns no exemption. `config.json`, `oauth/` and
`serve.json` are siblings of both and stay refused, which is what the fence is
for.

Worth noting for a reviewer: the bytes were already reachable. `/files/download`
serves a delivered file by token with no path fence, so this was never a
security boundary for deliveries -- only a viewer that refused to render what
the same page could already download.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_rpc_transport.py tests/test_rpc_files.py tests/test_rpc_knowledge.py tests/test_rpc_console.py -q` -> 123 passed
- `uv run ruff check` and `uv run ruff format --check` on both changed files -> clean
- Both new assertions were mutation-checked, each against its own mutation:
  - narrowing the carve-out back to the workspace alone reds
    `test_the_turn_working_directory_is_exempt_from_the_state_dir_fence` only (1 failed, 3 passed)
  - dropping the containment-direction guard reds
    `test_a_working_directory_root_that_contains_the_state_root_earns_no_exemption`
    and the pre-existing state-dir test (2 failed, 2 passed)
- Against the real reporting case: the two files that were answering 403
  (`~/.raven/tmp/tui/*.md`, `*.pptx`) now resolve, while `~/.raven/config.json`
  and `~/.raven/serve.json` still raise PermissionError.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- Widens what the file viewer will render by exactly one subtree: the directory
  turns work in. Everything the fence was written to protect (provider keys in
  `config.json`, the OAuth token store, `serve.json`) sits beside that subtree,
  not inside it, and stays refused -- pinned by the tests above.
- `fs.list` / `fs.reveal` share the function but pass the session's working
  directory as the root, which their own containment check already bounds, so
  their behaviour is unchanged.
- Rollback is a one-line revert of the exemption tuple.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-28 04:57:29 +00:00
LivXue
4e35a5d627 fix(tui): show the copy-on-select confirmation as a transient notice (#366)
## Summary

PR #360 reports every copy-on-select write as a permanent transcript
line, so a
reading session that drags through the transcript piles up "copied N
characters" rows under the conversation. The confirmation splits by what
the
line is for: the unbounded terse repeats become a transient notice
rendered
above the input box -- one line that replaces itself on every new copy
and
clears after 3 seconds -- while the first copy of a session keeps its
path
caveat in the transcript. The caveat is the answer a user comes looking
for
after a paste comes up empty minutes later, and it cannot pile up
because it
is once per session by construction.

The reporter still owns the per-session caveat tally and now returns
`{ text, firstOfSession }` so the caller can route the caveat to the
transcript and the repeats to the notice. A new `copyNoticeStore` holds
the
single notice; `showCopyNotice` keeps one dismiss timer, clearing any
pending
one before arming the next, so an earlier deadline can never clear a
newer
notice. `/copy` is deliberately untouched: an explicit command's answer
still
belongs in the transcript, and it cannot stack on its own.

## Type

- [x] Fix

## Verification

Run from `ui-tui/` at branch head, in a fresh worktree of `github/main`.
Dependencies came from `npm ci` in `ui-tui/` and in the hermes package;
the
hermes package's own `node_modules` holds only esbuild, so the ink
render
tests run against one react copy (two copies break every hook call).

```
npx vitest run --no-file-parallelism
  Test Files  93 passed (93)
  Tests  1084 passed | 13 skipped (1097)

npm run type-check
  clean, exit 0

npm run lint
  22 problems (0 errors, 22 warnings)

npx prettier --check src/app/copyNoticeStore.ts src/__tests__/copyNotice.test.tsx
  src/__tests__/clipboard.test.ts src/app/useMainApp.ts
  src/components/appLayout.tsx src/lib/clipboard.ts
  All matched files use Prettier code style!

python3 -m scripts.check_commit_messages github/main..HEAD
  exit 0
```

The 22 lint warnings are the baseline, not new: 22 before the branch and
22
after, none in a file this branch touches. The suite is run with
`--no-file-parallelism` deliberately: the ink render tests fail under
default
worker parallelism at this suite size, and no CI job runs them.

Suites that exercise the changed code directly:
`src/__tests__/copyNotice.test.tsx`
(7 cases), `src/__tests__/clipboard.test.ts` (30 cases, the 3 reporter
cases
updated to the new return shape).

New tests were proved load-bearing rather than assumed:

- Removing the notice render line in `appLayout.tsx` makes the render
test
  fail (content and placement above the input box).
- Removing the pending-timer clear in `showCopyNotice` makes the
re-shown-notice test fail: the earlier 3s deadline then clears the newer
  notice.
- The dismissed-notice case now shows a notice first, and removing
  `dismissCopyNotice`'s atom clear turns it red (review fix).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No doc change. `ui-tui/CONTEXT.md` has no clipboard or selection entry,
and
"copy notice" continues the existing "notice" vocabulary of
`copyResultNotice`
/ `copyOnSelectNotice`, so nothing new is coined.

## Risk

Behaviour change: the per-drag copy confirmation is now transient and
above
the input box instead of a permanent transcript line, and the first copy
of a
session lands one transcript line, exactly as on `main`. Reverting the
branch
restores the #360 behaviour; no data or on-disk format is involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (deepseek-v4-pro) <noreply@anthropic.com>
2026-08-25 14:27:24 +08:00
gloryfromca
df9a7d7f37 chore(*): merge upstream v0.1.13 into the gitlab trunk
Absorbs the GitHub release into the downstream trunk. A merge rather than a
rebase so v0.1.13 becomes a real ancestor: the next sync starts from v0.1.13
instead of v0.1.12, and the eventual contribution back to GitHub can still tell
our commits from the upstream ones it already has.

Cost this round: 51 conflict hunks over 27 files, against 54 over 31 for
v0.1.12. The hunks were the cheap half. Upstream's #284 moves the model and
provider from process-wide state to a per-session binding, and the trunk had 230
commits standing on the old shape.

What git reported as a clean merge and was not
---------------------------------------------

Fourteen upstream-changed paths do not exist here, because we renamed or moved
them since v0.1.12. Rename detection found none of them, so their changes were
merged into paths nothing reads, silently:

* raven/tui_rpc/* -> raven/rpc/* (renamed in ce526ad5). Nine files, including
  question_broker.py, whose whole #354 rewrite was lost, and methods/config.py,
  methods/model.py, methods/session.py, models.py and errors.py. Each was
  replayed by three-way merge against dd82f52c. The three test modules were
  rebased function by function onto upstream's version instead, after a
  three-way merge spliced assertions into the wrong tests.
* raven/agent/tools/mcp.py -> raven/mcp/client.py. Carried the release-blocking
  fix, and is handled in the commit that follows this one.
* ui-tui/rpc-schema/openrpc.json -> rpc-schema/openrpc.json.

Decisions
---------

Follow upstream wherever the two disagree about provider and model:

* providers/pin.py is gone, so a bare model id is refused instead of derived.
  A model id does not name whose credential serves it, and the derivation
  walked a list -- with two vendors configured, which one pays came down to
  their order in it. rpc/methods/config.py now requires the provider.
* ModelSwitchInTurnError (-32009) is gone with it. A session switch lands on
  that session's next turn, so there is nothing to refuse mid-turn.
* AgentLoop.refresh_context_window and the LazyProvider on_built wiring are
  gone: the window rides on the binding now.
* The skill gate no longer forwards an unpaired pin.

Keep ours where the trunk is ahead, and graft upstream's behaviour into it:

* raven/agent/subagent/backends/raven_loop.py is the live sub-agent path, so
  manager.py's inline loop stays retired and the web_search gate from #312
  applies there.
* skill_hub/client.py keeps the public zip limits the skillhub RPC path imports,
  and takes upstream's query-string cap.
* ask_user keeps our JSON-string normalizers and feeds them into upstream's
  batch contract, so a batch validated by _prepare is one _normalize_questions
  has already coerced.
* The chat view composes both filters: hideIntroAfterFirstTurn drops the cover,
  visibleRows still hands the view to a direct chat.

Tests removed, not repaired
---------------------------

Eight tests covered mechanisms this merge deletes, and one of them hung rather
than failed:

* seven in test_agent_loop_model_switch.py over AgentLoop._pending_provider,
  the switch-parking mechanism #284 replaced with per-session bindings. Its
  replacement arrives with upstream as test_agent_loop_session_model.py.
* test_a_refusal_during_a_turn_keeps_its_own_code in test_acp_methods.py, whose
  error class no longer exists.

Baseline: the pipeline for 01d39e58, this merge's base, is green, so nothing
here can be charged to a pre-existing failure.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 16:12:43 +08:00
Handsome-wzw
a1d5b24de3 feat(agent): withhold web_search without a key, and make tool capabilities visible (#312)
## Summary

A WeChat user asked for a recommendation and the bot answered "the web
search
tool is not configured here". That sentence was not the model's own
phrasing --
it was the tool's error text, relayed outward. `web_search` needs a
Serper key
and that deployment has never had one, but the tool was registered
unconditionally, so on every search-shaped question the model saw it,
reached
for it, and passed the setup error on to whoever was in the chat.

Withholding the tool fixes that and creates a second problem: an
unregistered
tool is invisible. Nothing in a running Raven then says the capability
exists at
all -- the model is never offered it, no document lists it, and `raven
doctor`
reports on providers and memory but has never mentioned tools. The only
way to
learn that web search is one account and one edit away was to read the
source.

This does both halves, because either alone leaves a real gap: the
deployer is
told what this install can and cannot do, and the model is still not
offered a
tool it cannot run.

### Withholding the unusable tool

`web_search` is gated on a resolved key, in the main loop and in the
sub-agent
surface (`subagent/manager.py`) -- a sub-agent that reaches for a search
it
cannot run reports the failure to its caller, and that text lands in the
parent
turn, the same leak one level down.

The gate asks the tool, not the config. `WebSearchTool.api_key` resolves
at call
time from the constructor value *or* `SERPER_API_KEY`, so reading
`tools.web.search.apiKey` alone would withdraw a working tool from any
deploy
that exports the variable and configures nothing.

The tool's error message also hard-coded `~/.raven/config.json` while
the
gateway runs with `--config` elsewhere, so following it meant editing a
file the
process never reads. It names the path actually in force now. That text
is
reachable only if the key disappears after registration, which is
exactly why it
should be right: it is the message for the case the gate cannot cover.

### Three rules, and nowhere to read them

Registration is decided per family, each a different shape:

| family | rule |
|---|---|
| `web_search` | a resolved key, asked of the built tool |
| `web_fetch` | nothing -- always registered, a key only improves
extraction |
| media x3 | an `api_key` *or* a `model`, either counting as configured
|

For media those are two questions, not one. A section naming only a
model is
registered, because a model alone counts as asking for the tool, and
then every
call returns a missing-key error. Whether a capability is *offered* and
whether
it *works* come apart there, and a report treating them as one fact
ticks a
capability that cannot run.

Each rule is defensible where it sits. What is missing is anywhere to
read them.
Providers had the same sprawl once and answered it with
`providers.auth`: a
declarative table plus `credential_status` as the single authority, with
an AST
invariant enforcing that authority, because six surfaces had answered
the same
question six ways and each looked reasonable alone. Tools never got the
equivalent.

### What this adds

**Each tool answers for itself,** twice where the questions differ.
`WebSearchTool.is_configured` reads the config value *or*
`SERPER_API_KEY`,
because those are two sources and only the tool consults both. The media
base
answers `is_configured` on a model *or* a key -- which is what stops an
OpenRouter credential set for chat from silently switching on three
tools that
bill per call -- and answers `has_key` separately on the chain it
actually
resolves at call time: its own section, the borrowed provider key, then
`OPENROUTER_API_KEY`. Both rules live with the credential they read.

**`capabilities.py` describes the five** for a human deciding what to
set up:
what each does in one line, how much work it is (nothing / reuse a
credential
you already have / obtain an account), where the key goes, where to get
one, and
what it costs. It rules on nothing -- `is_configured` and
`has_credential` both
ask the tools -- so it cannot become a second opinion.

**`raven doctor` grows a section**, listing every capability configured
or not,
ordered by how much the deployer has to do:

```
(markers render as a green check and a yellow bang; spelled [ok] and [!] here to
keep this description ASCII, since it becomes the squash commit body)

Tool capabilities
  web_fetch:       [ok] Read a web page the agent already has the URL for
  image_generate:  [ok] Generate an image  (borrowed: providers.openrouter.apiKey)
  text_to_speech:  [!] Generate speech from text
                   no key resolves; calls will fail
                   set: tools.media.speech.apiKey
                   or env: OPENROUTER_API_KEY
  video_generate:  -  Generate a video
                   switch on: tools.media.video.model
                   key: reusing providers.openrouter.apiKey
                   Billed per call; needs prepaid OpenRouter credit.
  web_search:      -  Search the web
                   set: tools.web.search.apiKey
                   or env: SERPER_API_KEY
                   key from: https://serper.dev
  2 capability(s) available but not set up; the agent is not offered them.
```

Three deliberate details.

Naming the credential is load-bearing in both directions. A row that
cannot
distinguish a reused credential from a missing one sends someone to
create an
account they already have -- and a row that claims a reuse with nothing
to reuse
is worse, because acting on it means setting a model, getting a
registered tool,
and watching every call fail on a credential they were told they had. So
the
reuse line prints only when a key is genuinely there to pick up, and a
capability already in that broken state says so outright instead of
showing a
satisfied tick.

The credential is named at the path that holds it. For the media family
`config_path` names the *model*, so reusing it as the key source pointed
the
deployer at a line with no credential in it.

And each fact is on its own line rather than in a sentence, because the
terminal
wraps a long line mid-path and a config key broken across two rows
cannot be
copied, which is the only thing that row is for.

Nothing here moves the exit code, including the warned row. An install
without
image generation is a choice, not a fault, and a doctor that fails on it
teaches
people to ignore doctor. The half-finished one is arguable -- the memory
section
does exit non-zero for a role the user configured that the server could
not
build -- but that failure is silent where it happens, recall just
returning
nothing, whereas this one returns an error string to the model on every
call.
Say the word and it becomes an exit code instead.

### Not in this change

Registration still lives in `AgentLoop`. Having it read the table is the
point
of this shape and removes the last duplicate reader, but it edits
`agent/loop/main.py`, which is under active change, and it is worth
doing on its
own once that settles. Until then the table is a description, and the
tests
below are what keep it honest.

`deep_research` is deliberately absent: it is moving to the sub-agent
surface
and its tool is going away.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

Mixed on purpose: the first commit is a fix and the rest are the feature
it
made necessary. Splitting them would land a change that hides a
capability
without landing the one that makes it discoverable.

## Verification

```
make lint                                            exit 0
make build                                           exit 0
npm run test --prefix ui-tui                         969 passed, 13 skipped
uv run --all-extras pytest -q                        6186 passed, 35 skipped, 1 failed
uv run pytest tests/test_tool_capabilities.py \
              tests/test_cli_doctor_commands.py \
              tests/test_agent_loop_web_tools.py     64 passed
uv run --extra dev ruff check raven tests scripts    All checks passed
uv run --extra dev ruff format --check               828 files already formatted
npx commitlint --from origin/main --to HEAD          exit 0
scripts/check_commit_messages.py origin/main..HEAD   exit 0
```

The one failure is
`test_read_file_image.py::test_an_attachment_that_cannot_be_
read_costs_a_note_not_the_turn`, and it is not from this branch: it
fails the
same way on `main` at `1cb604a` with these commits absent. The case
makes a file
unreadable with `chmod 000`, which does not block a root user, so it
fails for
anyone running the suite as root and passes in CI. This branch does not
touch
that file or the code under it.

The capability tests drive a real `AgentLoop` and compare what it
registered
against what the table predicts, rather than asserting the table against
itself.
The gated set is derived -- tools present once credentials are supplied,
absent
without -- so a sixth gated tool whose author forgets the table fails
here rather
than going unnoticed.

That last claim was false when first written, and the last commit is
what makes
it true. The fixture listing the media tools by hand never switched a
fourth one
on, so a new one never joined the gated set and the assertion held over
an
already-incomplete table. #305, which adds a MiniMax voice-clone tool,
is that
case: merged against this branch the assertion passed. Read from
`MediaGenConfig` instead, it fails and names the tool -- `gated but
undeclared:
['voice_clone']`.

Which means whichever of the two lands second turns this red,
deliberately. The
fix is one table entry, and it needs the rule that PR settles:
`voice_clone`
counts as configured on `api_key or api_base or model` in
`effective_media_config` while registration still gates on `api_key or
model`,
so an `apiBase`-only install with no MiniMax key is configured by one
rule and
withheld by the other. That is the divergence this table exists to make
visible,
and it is worth resolving there rather than papering over here.

Fifteen mutations, each caught:

```
the web_search gate removed from the main loop    3 failed
the same gate removed from the sub-agent surface  1 failed
media rule reads only the key                     7 failed
web_search rule reads only the config             1 failed
a tool removed from the table                     6 failed
the doctor section not rendered                   6 failed
doctor lists only configured tools                4 failed
unconfigured treated as a failure                17 failed
the reuse line is printed unconditionally         1 failed
the key source falls back to the model path       3 failed
a keyless registered capability is not flagged    1 failed
has_key always answers yes                        1 failed
has_credential collapses into is_configured       1 failed
the credential chain drops OPENROUTER_API_KEY     3 failed
the reuse check ignores the environment           3 failed
```

Four of those tests exist only because a mutation pass found the earlier
versions insufficient, and the last rounds are why the shape changed. A
media
case that sets both a model and an OpenRouter key proves nothing about
the "or
model" half: the borrow fills the key in, so a rule reading only the key
still
answers correctly. The case that pins it has nothing to borrow.

The sub-agent case is the same argument applied to the second call site:
with
the gate present only in the main loop, every test above still passes,
because
nothing was watching what the sub-agent surface registers.

More usefully, mutating `has_key` to always answer yes changed no test
at all in
the first version, which said the ruling was not load-bearing: the
doctor was
inferring "no credential" from an *empty source string* rather than from
the
tool's answer, so a fourth credential source would have been reported as
a
missing one. `has_credential` is now its own fact, asked of the tools,
and that
mutation fails.

Separately, autouse fixtures clear `SERPER_API_KEY` and
`OPENROUTER_API_KEY`,
without which several of these pass for the wrong reason on any machine
where a
developer exported one.

Rendering was checked against a real config in six states -- nothing
set, a
provider key present, a model with nothing to borrow, a tool with its
own key, a
model plus a borrowable key, and an exported variable as the only source
--
rather than only asserted on.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

One behaviour change: `web_search` is no longer offered to the model
when no key
resolves. That is the fix. A deployment that has a key, in config or in
the
environment, is unaffected; one that has none was getting an error
string in
place of an answer.

The rest is additive. No other registration logic changed, so which
tools an
agent is offered is otherwise exactly what it was; `is_configured` moved
the
existing predicates onto the tools without altering them, and the
mutation
results above are what pins that. `has_key` and `_resolve_key` are new
names for
the chain `api_key` already resolved -- the property calls the extracted
one, so
callers see the same answers.

`raven doctor` gains a section and no new exit code. Its zero-network
guarantee
holds: the table reads config and environment only.

Two allowlist entries were added to
`test_only_the_auth_module_decides_configuredness_from_a_key`, argued in
place:
`subagent/manager.py`, which asks the built tool whether a key resolved
so an
unusable search is withheld, and `capabilities.py`, which reads keys to
*report*
-- which source supplied one, and whether one is there to reuse. Both
ruling
halves are delegated to the tools, whose files were already listed.

Rollback is a revert. Nothing is written and no configuration is read
differently.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 10:05:58 +08:00
arelchan
82eea1dd20 fix(ui): show Raven logo as page favicon
## Summary

- Declare the existing Raven application logo as the served page favicon.
- Copy the logo into the packaged page assets so installed builds render the same branding.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

- `npm test --prefix ui` - 32 test files and 528 tests passed.
- `npm run type-check --prefix ui` - passed.
- `npm run build --prefix ui` - passed.
- `python3 ui/build.py` - built the page and copied the Raven logo asset.
- `node ui/scripts/check-page.mjs` - passed.
- `make check-large-files` - passed.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Low risk. The change only adds favicon metadata and packages an existing application asset. Reverting the two source changes restores the prior browser-default icon.

## Related Issues

N/A
2026-08-23 11:11:06 +00:00
gloryfromca
56858acbce fix(*): report an invalid memory identity as the config error it is (#350)
## Summary

An identity EverOS cannot accept becomes a directory segment on its
write path, so `EverosBackend.start()` rejects it. The rejection carried
the one message a user can act on -- it names the key and the accepted
pattern -- and no surface delivered it as one. Three of the four call
sites fold the raise into `logger.exception`; under the TUI loguru is
file-only, so there an invalid identity turned memory off with nothing
on screen. The fourth, in `import_commands._build_and_run`, is not
wrapped at all, so the `ValueError` escaped as a bare traceback. Where
it was visible it was a traceback, which is not a thing a user reads as
"edit this key".

The state machine then told the user the opposite of the truth.
`start()` died before assigning a state, so `_state` stayed `UNKNOWN`,
which is not in `_NEVER_HAD_MEMORY`; every `store()` after it counted a
dropped write, and the session ended with:

```
1 turn(s) were not written to long-term memory because the memory service was
unavailable.
```

The service was fine. That sends the user to debug a server that never
broke.

**The change.** Catch the `ValueError` in `start()`, give it a
`BAD_IDENTITY` state, and file that state with the other two the user
must fix themselves:

- terminal, so a stray probe cannot promote it to `READY`;
- never-had-memory, so no dropped-write count and no outage wording;
- the message goes to stderr, matching what the unconfigured-LLM and
missing-binary paths beside it already do.

Two things came out of running it rather than reading it:

1. **The message needs escaping.** It quotes the accepted-character
class, and rich reads `[a-zA-Z0-9_.@+-]` as a markup tag and eats it,
leaving the user the pattern `^+$` to match their id against. The test
now pins the full class.
2. **`raven import` had the same defect on its own gate.** It kept the
outage wording and pointed at the server log, which holds nothing about
a config error. It now names the config keys instead.

Before / after, same config (`memory.userId: "bad/id"`), same command:

```
- 1 turn(s) were not written to long-term memory because the memory service was
- unavailable.

+ Long-term memory is off: memory.userId='bad/id' is not accepted by EverOS: it
+ becomes a directory segment on the write path, so it must match
+ ^[a-zA-Z0-9_.@+-]+$ and must not be '.' or '..'.
+ Fix memory.userId / memory.agentId in your config.json, then start a new session.
```

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Re-run after rebasing onto `d35e398c`, the current `main`.

```
uv run pytest tests/test_everos_backend.py tests/test_cli_import_commands.py \
              tests/test_everos_plugin_discovery.py -q
162 passed

uv run pytest -q
1 failed, 6478 passed, 43 skipped, 13 deselected

uv run ruff check <the four touched files>            All checks passed
uv run ruff format --check <the four touched files>   4 files already formatted
```

The one failure is
`tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare`. It
is pre-existing on this base: checked out `d35e398c` in a clean worktree
and ran `uv run pytest tests/test_cli_theme.py` there, same failure, 1
failed / 44 passed. It passes in isolation and fails when the file runs
as a whole, so it is order-dependent, and it is unrelated to anything
here. That file is untouched by this change.

Manual check, sandbox `RAVEN_HOME`, `memory.userId: "bad/id"`, `raven
agent -m "hi"`: prints the block quoted above, and the session no longer
ends with the dropped-write line.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Terminal-only wording plus one new enum member. No config, schema, or
protocol change; an identity that was accepted before is accepted now,
on the same code path. The only behaviour change for a valid setup is
none.

For an invalid identity the turn behaves as it did -- memory off, turn
completes -- and only what the user is told changes. Rollback is
reverting the single commit.

`ServiceState.BAD_IDENTITY` is new. Every consumer was checked:
`raven/cli/import_commands.py` (handled here) and
`tests/test_everos_plugin_discovery.py` (reads `READY` only).
`_may_spawn` allowlists `UNKNOWN` via `_SPAWNABLE_STATES`, so a new
member is non-spawnable without a change.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #271

Co-authored-by: zhanghui <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-08-20 14:41:45 +08:00
arelchan
e0be472abf fix(tools): ask whether a tool is offered, not whether it is registered
## Summary

Three follow-ups the hot-switch MR (!170) left behind, each raised in review there and held back rather than clear an approval that had taken a full round to earn.

**`get(name) is not None` no longer means what its callers meant.** It read as "may the model use this", and that was true while the off switch unregistered the tool. A withheld tool now stays in the registry -- that is what makes the switch reversible -- so `get` answers "this build has such a tool" instead. The registry gains `offers_by_name` for the question those callers are actually asking.

The one site whose answer becomes prompt text is the tool-failure nudge (`main.py`, `suggest_find_skill`). With `find_skill` in `tools.disabledTools` it still told the model to " call find_skill('<task keywords>') once before the next attempt", and `execute` then refused it as absent. The cost is one wasted call on a turn that is already failing, but it is model-visible text advertising a tool the operator switched off, which is the one thing the switch exists to make impossible. Every other registration-as-existence site was checked and none has the problem: `message`, `ask_user`, `deep_research`, `load_playbook`, `cron` and `run_subagent_dag` are fetched to wire a callback or a context into, which is inert on a tool that can no longer be invoked, and the `has()` pair in `_sync_mcp_meta_tools` is the exempt set.

**The test suite read the config file of whoever ran it.** A pathless `LiveConfig` resolves to the real home, and the tool array is assembled through one of those reads, so the developer who has actually used the off switch is the one whose suite fails. `HOME` was already redirected in `conftest.py`; `RAVEN_HOME` outranks it and was not. The fix is to **unset** it rather than set it -- setting the variable that beats every other isolation mechanism overrides the tests that isolate their own home (measured: 17 failures), while unsetting it overrides nothing, because a test that wants the variable sets it itself and lands after the fixture. It also clears two failures that predate this line.

**Tool search asked the withheld source once per candidate** while walking the whole ranked catalog. Beyond paying a file read N times, it let one list disagree with itself: a tool offered and its neighbour withheld because the file moved between them. `offers` already accepts the set; the search now passes it, the way schema assembly does.

**Two tests were passing on an empty index rather than on the thing they name.** `ToolSearchController` indexes nothing until `refresh()`, so `search()` answered `[]` for every query and an assertion that a withheld tool is absent from the results could not fail. Both now refresh, and each gained the pairing case that makes the original assertion mean something.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest -q -p no:randomly` -> **2 failed, 9899 passed, 33 skipped**. Both failures are `test_start_webapp.py` (needs bash 4; this host has 3.2), verified pre-existing by stashing the branch, and green in CI.
- `uv run ruff check raven/ tests/` and `ruff format --check` clean. Re-run after rebasing onto the current `main` tip.
- Five mutation checks, all caught, each asserting its own patch applied before the result was trusted:
  - nudge gate back to `get(name) is not None` -> 1 red (`test_the_nudge_does_not_name_a_switched_off_find_skill`)
  - `offers_by_name` back to a registration check -> 2 red
  - the `delenv` line removed, run under a populated `RAVEN_HOME` -> 3 red, including the two that predate this branch
  - the withheld read put back per candidate -> 2 red
  - `offers` ignoring the withheld set -> 8 red
- The nudge is asserted end to end rather than at the seam: the existing loop-break harness drives a real failing streak, `find_skill` is switched off **through the config file** so the path under test is the one the settings page writes, and the assertion reads the nudge text the provider actually received. Both directions are covered, so neither can pass by never mentioning the tool.
- Worth writing down about the method: the first version of the tool-search tests passed under mutation, and the reason was that they never exercised anything -- an unrefreshed `ToolSearchController` returns `[]`, so the assertions were vacuous. That is what turned up the two pre-existing tests with the same defect.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- `execute` and the nudge now agree with the schema and tool-search surfaces. A model that asks for a switched-off tool is refused everywhere and advertised nowhere; nothing that was reachable becomes unreachable.
- One fewer file read per tool-search scan. No behaviour change when nothing is switched off.
- The `conftest.py` fixture changes what the suite reads, not what the product does. A test that wants `RAVEN_HOME` set still sets it and still wins.

Rollback: revert. Nothing is persisted.

## Related Issues

N/A
2026-08-21 17:28:19 +00:00
gloryfromca
c20556161d feat(mcp): load and unload servers at runtime, from one origin record
## Summary

MCP servers can be attached, detached and reconnected while raven runs; the
model is told when one goes away mid-turn; and everything raven knows about MCP
now lives in one package with one record of where a tool came from.

**Reconcile, not restart.** `MCPConnectionManager.sync` becomes `apply_config`
and returns an `ApplyReport`. A new `config_changed` answers "would applying do
anything?" for the price of a dict comparison, so a caller that fires on a timer
never reaches a transport; a server whose config did not change is not touched at
all. That gate matters more than it looks: Hermes has a source-level record of a
retry loop on one failed server destabilising the healthy servers beside it.

**Unload takes effect at once, and the miss says so.** Closing the transport
immediately is what makes credential deletion and subprocess teardown real. A
turn already in flight had those tools in its prompt -- the model was *told* they
exist -- so its next call lands on the registry's miss. That answer used to list
every other registered tool, which reads as "you got the name wrong, pick from
these" and costs 1470 tokens on a 210-tool deploy, in the tool result, kept in
history for the rest of the run. It names both readings instead: the tool may
have been unloaded, or the name may be wrong. The registry genuinely cannot tell
them apart, and asserting either one is wrong half the time.

An earlier revision of this branch kept a tombstone per withdrawn name so the
answer could name the server. It was removed. Its whole justification was the
contrast with the catalog listing, so once the listing was gone the tombstone
bought a wording difference in exchange for three questions it had to answer:
who owns a name while a tombstone stands, when a tombstone expires, and in what
order the sweep runs relative to the callers that create one. Each of the three
had a defect. Twelve tokens of wording replaced all of it, and the sentence now
has one home that both the direct and the folded surface read.

**One home, one record.** Two data sources for one fact was the defect underneath
a third of the findings on this MR. The manager recorded which names a server
contributed; a second record of the same thing sat on the wrapper for config
matching; neither was complete -- the manager's was a set of names with no tool
name in it. Three consumers then each grew their own way of recovering the parts
from a registered name, and each was wrong differently:

- `ext.list` matched the longest configured server name that prefixes the tool
  name. A server called `context7.io` registers `mcp_context7_io_search`, which
  no configured key prefixes, so it reported connected with zero tools while the
  agent was calling them.
- the web panel stripped an `mcp_<server>_` prefix to show a bare tool name, so
  those same servers showed their tools under the full registered name, collision
  hash included.
- `resolve_configured` regenerated spellings per origin and returned every match.
  Two different pairs can produce one historical spelling, so an entry naming a
  live tool also disabled a tool from a server it never mentioned.

None of that is recoverable from the name: sanitising rewrites characters, the cap
truncates, a collision appends a hash. The name is a one-way function of the pair,
so the record has to carry the parts. `MCPToolRef` does, the registry holds one
index of them filed at registration, and every consumer asks it. The manager keeps
no copy -- it reads its own teardown list back out of the index.

`resolve_configured` now lets an exact match win outright rather than fanning out
from it. The registry is keyed by name, so an entry naming a tool that exists
names that tool; the fallback is for a spelling no tool carries, which is the case
the compatibility exists for.

**Tool names.** The `mcp__` double-underscore namespace is dropped -- the shape
stays `mcp_<server>_<tool>`, which is what ships today and what two consumers
parse. The sanitiser keeps the dash, verified against three independent sources
rather than assumed: Anthropic documents `^[a-zA-Z0-9_-]{1,64}$`, OpenAI's SDK
says "underscores and dashes, maximum length 64", and litellm replaces
`[^a-zA-Z0-9_-]`. Two consequences drove the design. litellm rewrites only names
that are already illegal, so a dashed name reaches the provider untouched today
and must not be renamed. And litellm truncates at 128 while Anthropic documents
64, so a name between the two passes litellm and is rejected upstream -- the cap
has to be enforced here. Its sanitiser also exists only on the Anthropic path, so
the OpenAI-compatible route gets nothing from it.

An unsanitary server name now warns rather than being rejected. Refusing to start
over a character in a config file is far steeper than the problem, whose whole
effect is that some names get rewritten -- but the rewriting was silent, and it is
what makes a `disabled_tools` entry stop matching. All 40 servers in the shipped
catalogue are already clean and dashes survive, so the warning should be rare.

**Resources and prompts, reached by server name.** MCP has three primitives and
raven spoke one of them. Five global tools cover the other two:
`list_mcp_resources`, `list_mcp_resource_templates`, `read_mcp_resource`,
`list_mcp_prompts`, `get_mcp_prompt`. Global with a `server` argument rather than
a wrapper pair per server, which is what Claude Code, Codex and opencode all do,
and the reason is schema cost: per-server wrappers grow the tool array with the
server count and push a deploy over the tool-folding threshold, while five cost
the same whether one server serves resources or ten.

Taking a `server` name and no tool wrapper means there is no session to borrow,
so the manager grew one: `session_of(name)` hands out a session only for a record
in `connected`. It needs no generation of its own, because of where the session
is written rather than what is checked on the way out -- only a committing
attempt stores one, under the same epoch check that accepts its tool
registrations, and `_disconnect_locked` clears it in the same breath as closing
the stack. A losing attempt therefore never publishes a session, so there is no
way to be handed one whose stack is gone.

Two shapes are worth naming because they are departures. `get_mcp_prompt` makes
prompts model-selected where the spec positions them as user-selected with a
slash command as the carrier; the slash route would need a command registry that
accepts outside contributions, and `commands.catalog` is a pure reflection of the
Typer app while `slash.exec` returns display text that cannot become this turn's
user input. The product landing in a tool result is what makes that affordable:
the fencing every tool result gets applies unchanged, and an assistant-role
message in a template can be kept and labelled rather than replayed as something
the model itself said. And `read_mcp_resource` returns text inline but writes
binary to a file under the workspace, named by a digest of `(server, uri)` --
base64 in a tool result costs tokens for something the model cannot read, and a
resource URI is free-form enough that it must never reach the filesystem.

**The five exist only while a server serves them.** A server states which
primitives it offers once, at `initialize`, and cannot be asked again, so the
handshake result is now captured instead of dropped. `servers_offering(primitive)`
reads it, and the five are registered when some connected server offers the
primitive and withdrawn when none does. Most MCP servers offer only tools, so a
gate per primitive rather than "is MCP configured at all" is what keeps a
tools-only deploy from paying for five schemas whose calls can only fail. Gating
moves the tool array on connect and disconnect, which costs the prompt-cache
prefix -- but that array was already moving at exactly those moments, because the
server's own tools appear and disappear with it. What it buys is that the array
does not carry these five the rest of the time.

Registration is re-gated after every connect and after every config apply,
because a detach fires no connect and can take the last server that served the
primitive with it. Every call also re-checks the server it names, which covers
the window between two applies.

### Existing defects this uncovered and fixed

- **`reload.mcp` did nothing and said nothing.** It returned a hardcoded
  `{"reloaded": 0, "tools_changed": false}` and ignored its params, while the TUI
  branched on an `r.status` the handler never sent -- so every `/reload-mcp`
  printed the same fall-through line and the confirmation machinery was dead code
  on both sides.
- **Its published contract said it was a no-op.** `rpc-schema/openrpc.json` and
  `ReloadMcpResult` both still declared that, and both forbid extra keys, so all
  three response branches were rejected by the model that claims to describe them.
  The suite stayed green because `test_rpc_schema_match` compares the schema to
  the models and nothing compared either to a real response -- the two sides went
  stale in agreement. A contract test now feeds each branch's actual response to
  the model.
- **`raven.mcp.list` reported the wrong thing twice** -- see the ownership story
  above, plus `_mcp_connected`, which only says the one-shot startup connect ran.
- **`raven.mcp.set` told the user to restart.** Untrue since the connection
  manager landed. It reconciles the running loop now and reports `applied`,
  keeping the write if the reconcile fails.
- **A guard had stopped guarding.** The test pinning "`raven.mcp.list` must not
  emit a key `raven.mcp.set` would persist" used a fake that no longer reached the
  live half. It uses a manager-backed fake now and asserts the live half
  contributed keys at all.
- **`ApplyReport` misreported twice.** A config change that emptied a server's
  tools reported `tools_changed` false unless the reconnect put something back.
  And "reconnected N MCP server(s)" counted records touched, which includes a
  detach and includes an attempt that failed or parked at the browser -- so a
  config where every server is down reported a successful reload.
- **`always` claimed to be remembered.** It never was: the setting the TUI names
  has no section in this config, and the server logged that it was ignoring the
  flag while the TUI announced future reloads would skip the prompt. Removed end
  to end rather than accepted-and-ignored.
- **`always_visible` matched by exact set membership, decided in the
  constructor.** Both halves were wrong for a namespaced tool: an MCP server can
  attach after the controller is built, and the registered spelling can differ
  from the configured one -- so the tool quietly stopped being held resident.
- **The web UI's MCP call counter never counted.** `MessageBubble` matched an
  `mcp__` prefix no raven build has ever registered, so the "N MCP tools" summary
  has read zero since it was written. Pre-existing; fixed here because it is the
  same mistake about the same name.

### Defects found by review of this branch, and fixed here

Five, from an adversarial pass at `be39ed03`. Two were in the resource/prompt
work itself; three had been on the branch for several revisions.

- **`disabled_tools` could not switch off an MCP meta-tool, and churned the set
  on every connect.** `_after_mcp_connect` applied the blacklist and then
  re-gated, so an entry naming one of the five was unregistered and immediately
  registered back. It could never have worked -- the gate's predicate is
  `all(...)` over each set, so one name missing reads as "the set is not
  installed" -- and what it did instead was move the tool array on every
  connect, which is the cached-prompt-prefix cost the gate exists to avoid. The
  gate now owns those names and the blacklist skips them, reporting such an
  entry once rather than appearing to work.
- **Saving MCP servers from the panel could hang for 17 minutes.**
  `raven.mcp.set` awaited the reconcile, and a handshake parked at the browser
  authorization step is exempt from the 90s progress bound for up to
  `OAUTH_FLOW_TIMEOUT + _AUTH_PARK_GRACE`. Adding an OAuth server therefore hung
  the panel on the consent page it had just opened. The reconcile is now a task
  behind a bounded, shielded wait -- the same call `plughub.kick_sync` already
  makes fire-and-forget -- so `applied` narrows to "a live loop was handed it",
  which is what its own documentation said it meant.
- **The panel told the reader "connects on first use" about servers that had
  been reached and failed.** `raven.mcp.list`'s top-level `connected` changed
  meaning in this branch, from "the startup connect has run" to "at least one
  server is up now", and `McpPanel.tsx` still branched on the old one. With
  every server in `error` nothing is connected, so every row fell through to the
  lazy-connect line. The old semantics rendered that state as "not connected",
  so this was a regression. Rows now read their own `state`, which this branch
  had already plumbed to the API type without giving it a reader.
- **Three docstrings claimed things the code does not do.** The unsanitary-name
  warning said a `disabled_tools` entry would stop matching -- this branch is
  what makes it match; `_suffix` claimed a disambiguated name is stable across
  restarts, which `spellings` correctly denies ninety lines down; and
  `reload.mcp`'s published description still named the removed `always` flag and
  a poller that does not exist. Fixed as documentation, plus `tools_by_server`
  so `_mcp_live`'s "walks the index once" is true rather than reworded.

- **`reload.mcp` threw, and its own published description says MUST NOT.**
  `apply_config` re-raises a `SandboxInitError` from any attempt, and any other
  exception from one, so a single misconfigured stdio server came back as a
  JSON-RPC internal error with a traceback in the log -- for the method whose
  contract exists precisely so a caller that fires on a timer never has to
  explain a transport fault. Now reported as `noop` with the reason in
  `message`, which is the branch `ops.ts` already prints from, so no client
  changes. `raven.mcp.set` already guarded the identical call for the identical
  reason; this was the asymmetry. The fake in the handler's tests could only
  succeed, which is why fourteen tests over it -- two enumerating every branch
  -- never reached the one that does not return.

Three findings from the same pass were retracted after checking them, and one
was downgraded, all on the MR threads: the meta-tools *are* withdrawn on plugin
uninstall (`remove()` ends in `kick_sync`), `/reload-mcp` is hidden from the
palette by an owner-reviewed curation with a test pinning it, and the image-size
gap in `read_mcp_resource` is inherited from `blocks_from_mcp_content` rather
than introduced -- the new path is the stricter of the two.

### Bugs introduced during the work, and how they were caught

Worth listing because two of them were found by reading rather than by a test, and
each was then covered by one -- verified by reintroducing the bug and confirming
the new test went red.

- `_take_back` read `epoch is None` -- a record that wants no attempt, which
  `_disconnect_locked` sets as its first act -- as "a newer attempt owns these
  names", so a disable racing an in-flight handshake left that attempt's tools
  registered while its own stack was closed underneath them. A `disconnected`
  server advertised a tool whose session was dead, and `config_changed` read the
  orphan through `names_from` and answered "still work to do" on every poll.
  Raised in review, reproduced independently before fixing, and now covered: it
  was the one `drop=False` path no test reached.
- Rewording the miss broke `failure_class`, which matched on "not found", so every
  missing tool fell to the catch-all class and shared a streak bucket with
  unrelated failures on the same tool -- reaching the loop-break nudge early.
- `spellings` offered a truncated no-suffix form that no build ever registers,
  because `tool_name` makes the suffix mandatory once a pair overflows the cap. It
  answered a config entry with a name that never existed. The overflow rule now
  has one home that both functions read.
- Deleting the tombstone by file took three classes of manager test with it, only
  one of which was about retirement. `config_changed` lost all seven of its unit
  tests, including the two pinning its subtle branches. Restored in their own
  file, and verified to still bite by breaking the probe on purpose.
- `_unregister_unclaimed` computed ownership from the registry index, which
  includes *uncommitted* registrations, so a cancelled connect no longer took back
  what it had registered. Replaced with epoch ownership, which is both simpler and
  covers a case the old set missed: a server dropped from config while its
  handshake was parked at the browser used to leave its late registrations behind
  forever.
- `config_changed` returned from inside its loop on the first settled removal,
  masking a later server that really had changed. The first regression test for
  this passed against the bug -- a set-difference check short-circuited ahead of
  the loop -- and had to be rewritten to reach it.

### Decisions reversed from the original plan, with reasons

- The `mcp__` namespace: dropped, above.
- **A tombstone for a withdrawn tool: added, then removed.** The largest reversal
  in the branch, and it deleted far more than it changed -- `retire`, the sweep,
  the name reservation a tombstone implied, and two test files that existed only
  for them. Reasoning above.
- A `Reclaimable` protocol (`generation` / `sweep`): dropped. Once nothing
  outlives its config, `sweep` degenerates to `dict.clear()`.
- A per-connection generation counter: dropped. `epoch` already arbitrates the
  in-flight connect race, and now the teardown race too.
- `Scheduler.busy_conversations()`: dropped. Its only consumer was an idle check
  that no longer exists.
- The `ConfigDriven` / `ChangeProbe` protocols: dropped. Nothing referenced them
  and this repo type-checks no Python, so they asserted a contract nothing could
  check -- and one docstring described a caller that does not exist. What they
  meant is stated on the methods that implement it.
- An `approvals.mcp_reload_confirm` setting: not added; `always` removed instead.
- `resolve_configured` returning every match: reversed to exact-match-wins. This
  contradicts an earlier review position that asked for breadth, and the reviewer
  has since agreed the two cases are different -- two spellings of one pair, where
  breadth is right, versus two pairs that regenerate one string, where it reaches
  across servers.

### Deliberately not done

- **A `disabled_tools` entry for one of the five meta-tools.** The gate's
  predicate is `all(...)` over each set, so one name missing reads as "the set is
  not installed" and the next connect puts the whole set back; an entry could
  therefore never switch one off, only turn every connect into an
  unregister-and-re-register of all of them. Their lifetime is a consequence, not
  a preference, so `_apply_disabled_tools` skips them by name and says so once
  per entry rather than appearing to work.
- **`eval_engine`'s `tool_denylist`.** It matches by exact equality and fails
  open, but not for the reason the review gave: the phase it runs on is never
  invoked. The agent loop drives two of the five hook phases `CompositeHook`
  forwards, and `before_execute_tools` is not one of them -- correspondingly
  `AgentHookContext`'s `iteration` / `tools` / `response` are never populated.
  Adding name resolution there would fix code that cannot execute and would make
  the gate look wired. Reported as its own finding on this MR instead; driving
  three hook phases means changing the loop's iteration body and has nothing to do
  with MCP naming.
- **`skill_local` `requires.tools`.** Exactly one skill in the tree declares it
  (`subagent-dag-orchestration`, requiring the built-in `run_subagent_dag`), so no
  MCP tool reaches that gate -- and `filter_by_required_tools` is the single shared
  gate for the main-agent and sub-agent surfaces, so teaching it about MCP
  spellings would push scenario knowledge into a general component for a case that
  does not occur.
- **A backend config-file watch.** `config_changed` is the gate such a caller
  needs and it is in place; what is missing is only something to knock on a timer,
  which is a new background loop and its own decision.
- Wrapper late binding, the runtime-context digest of unavailable servers, and the
  GUI consumers for `oauth.pending` / `oauth.done` / `mcp.status`.

## Type

- [x] Feature

## Verification

```
uv run pytest tests/ -q
uv run ruff check raven/ tests/
cd ui-webui/frontend && npx tsc -b
cd ui-tui && npx tsc --noEmit -p tsconfig.json && npm run lint
make check-large-files
```

Full suite: `9501 passed, 33 skipped, 0 failed` (215s). The resource and prompt work
adds four files: `test_mcp_resources.py`, `test_mcp_prompts.py`,
`test_mcp_session_addressing.py`, `test_agent_loop_mcp_meta_tools.py`. Lint, both front-end type-checks
and the large-file check are clean.

Test accounting against `main` is an ID diff rather than a count, because upstream
churn hides regressions in a total: 9389 IDs on `main`, 9468 here, and every ID
that exists only on `main` is a rename (`sync_*` to `apply_config_*`) or a
contract this MR changed (the `reload.mcp` stub cases, `restart_required`, the
longest-prefix attribution test). Nothing was dropped silently.

One flake, disclosed rather than smoothed over: a run immediately after a rebase
reported one failure and the identical tree re-ran clean, on a run that took 328s
against a usual 213s. Only the tail of the output was captured, so the test cannot
be named; it is a real defect in the suite and not yet closed.

The package move was verified behaviour-neutral before anything else changed, and
the name collision it invites was checked explicitly: `raven.mcp` and the MCP
SDK's top-level `mcp` resolve independently, and `from mcp import ClientSession`
still reaches the SDK from inside the new package.

Also verified by hand, because a set-equality guard cannot prove it: the panel's
own read-modify-write round trip (`raven.mcp.list` -> hand the payload straight
back to `raven.mcp.set`) leaves `connected`, `state` and `tools` out of the file
on disk.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally

## Risk

**Backward compatibility.** `disabled_tools` entries keep matching. An entry may
name an MCP tool by a spelling this build no longer produces, so each origin
generates its own spellings forward and the entry is tested against them --
except that an exact match wins first, which is what stops the resolution
reaching across servers.

**Behaviour changes a user can see.** `/reload-mcp` with no argument asks for
confirmation instead of printing "reload complete" and doing nothing;
`/reload-mcp now` reloads; `/reload-mcp always` is gone. Saving MCP servers from
the web UI applies to the running agent instead of asking for a restart, and says
so when it could not. A tool call that does not resolve -- a name the
model invented, or one whose MCP server was unloaded mid-turn -- returns an error
naming both possibilities, and no longer returns the list of every other
registered tool. A server name carrying
a character no provider accepts logs a warning at first apply.

**New tools, and when they appear.** Five tool schemas appear the first time a
connected MCP server declares the `resources` or `prompts` capability, and go
away when the last such server does. Nothing appears on a deploy with no MCP
servers, or one whose servers offer only tools -- which is most of them. A
`tools.disabled_tools` entry naming one of the five has no effect and is logged
as such; their lifetime follows the servers, not the config.

**Import paths.** Every `raven.agent.tools.mcp*` module moved under `raven.mcp`.
Internal only -- no entry point, CLI surface or RPC method name changes -- but any
out-of-tree code importing those paths breaks and the rename is mechanical.

**Rollback.** Revert the commits. Nothing changes on-disk state except that
`raven.mcp.set` no longer returns `restart_required: true`, which no caller
branches on for MCP.

## Related Issues

N/A
2026-08-20 14:16:38 +00:00
gloryfromca
0809794b66 chore(*): merge the trunk again before the v0.1.12 sync lands
Three more commits landed on the trunk during the review round, so the branch
stopped fast-forwarding again. Merged rather than rebased for the same reason as
before: a rebase flattens the v0.1.12 merge commit and costs the ancestry this
whole branch exists to establish.

One conflict, additive: tests/test_cli_gateway_commands.py had a _risk_banner
group added on this branch and a gateway-shutdown ordering test added on the
trunk. Both kept.

ui-tui/src/rpc/generated.ts merged automatically. Since it is a generated file,
its result was checked rather than assumed: `npm run lint:rpc` reports it in sync
with rpc-schema/openrpc.json, which merged alongside it.

Verification:

    uv run pytest                9018 passed, 33 skipped, 13 deselected
    uv run ruff check            all checks passed
    uv run ruff format --check   1022 files already formatted
    npx vitest run               99 files, 1265 tests passed
    npx tsc --noEmit             clean
    npm run lint:rpc             schema and codegen in sync

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-18 17:30:49 +08:00
LivXue
ede8e7ed3c refactor(*): install the shipped sub-agents without a running raven
## Summary

Installing one of the sub-agents that ship under `subagents/` required a running
raven: each folder's `install.py` registered its entry through
`PUT /raven/subagents`, so both the web service and the gateway had to be up. On a
machine that has just cloned the repository, neither is.

The installer now writes `subagents.thirdParty` in the host raven's config
directly, through `raven.config.update_subagents` - the supported write path,
which validates the entry against the schema, replaces the entry sharing its name,
refuses a list that would hold duplicates, and replaces the file atomically. Hand-editing the JSON gets none of that, and a
half-written config is one raven will not start on. That module is importable only
from the host raven's environment, which is rarely the interpreter running the
installer, so it is reached in a subprocess resolved from the shebang of `raven`
on `PATH` (or `RAVEN_PYTHON`, or `--raven-python`). The installer therefore stays
standard-library only and runs under a bare `python3`.

What the file write costs is hot-apply: a running raven holds the roster it read
at startup, so the installer prints a restart instruction and the guide says the
same. `--endpoint` is gone; `--config` and `--raven-python` are new.

Two additions on top:

- `subagents/install.sh` does the whole install for every folder, or the ones
  named: builds each checkout's venv, scaffolds `.env` from its template, then
  registers. It refuses to register a folder whose api key is still blank, so a
  half-configured agent never reaches the roster, and exits non-zero while
  anything still needs attention. It discovers the checkout by its lone
  `pyproject.toml` and derives the key prefix from the folder name, so a new
  folder needs no edit in the script.
- `subagents/README.md` is the directory-level guide: what a folder is made of,
  the install steps, what the installer does and does not do, how to remove an
  agent, what a new folder has to satisfy, and what never gets committed.

`subagents/*` stays excluded by default. Only the folders that were already
tracked are whitelisted, plus those two new files by name; a folder holding a
docker image tar or a private checkout is unaffected. The change also drops a
sentence that pointed a published README at an unpublished sibling folder.

The pre-install backup is written before the config is touched, holds the list as
it sits on disk rather than a schema round-trip of it (loading coerces an `openai`
entry's `readsLocalFiles` away, so a validated copy would restore something the
user never wrote), and is at mode 600 because another entry among the others can
carry its own api key. A backup that cannot be written stops the install with the
config untouched, rather than going missing after a config that already changed.

`install.sh` reads the api key the way the launcher does, first non-empty match
wins, so the two cannot disagree about a `.env` that sets it twice. It expands its
argument array through `${args[@]+...}`, since an empty array under `set -u` is an
unbound-variable error before bash 4.4 - the stock `/bin/bash` on macOS. A relative
`--config` is made absolute at parse time, because `install.py` runs with each
folder as its cwd and would otherwise resolve the path once per folder.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

The branch touches 12 files: `.gitignore` and 11 under `subagents/`. Nothing under
`raven/` or `tests/`, and `testpaths = ["tests"]`, so no unit test collects any of
it.

```
uv run pytest -q          8237 passed, 46 skipped, 13 deselected in 170s
uv run ruff check  --no-force-exclude subagents/raven-code/install.py   All checks passed
uv run ruff format --check --no-force-exclude subagents/raven-code/install.py   1 file already formatted
bash -n subagents/install.sh                              clean
python3 scripts/check_commit_messages.py origin/main..HEAD exit 0
python3 scripts/check_large_files.py origin/main..HEAD     exit 0
```

`subagents/` is excluded in `[tool.ruff] extend-exclude`, so the two ruff commands
above need `--no-force-exclude` to reach the file at all.

End-to-end on a freshly provisioned host with no `raven`, no `uv`, and Python 3.10:
cloned the repository, installed `uv`, built the venvs, ran the installer twice and
spawned one agent for real.

- Installer on a clean clone with no keys: three checkout venvs built, three `.env`
  files scaffolded at mode 600, stopped at the key gate for all three, exit 1. The
  host config's mtime was unchanged and no backup file was written, so nothing
  half-configured reached the roster.
- Same host with keys filled in: all three entries `replaced` in a config that
  already held 8 third-party sub-agents, and all 8 survived. Each backup was mode
  600 and held the full 8-entry list. `load_config()` then listed all 8, so the
  written entry is valid to the runtime and not only to the writer.
- The resolved `command` of each entry pointed at the clone's own path, confirming
  the placeholders resolve against the installer's location rather than the machine
  that built the folder.
- One real spawn answered correctly in 22s.
- Before that, on a machine with no `raven` on `PATH`, the installer reported
  `no raven on PATH; pass --raven-python ...` and recorded the folder as failed.

Probes for the fixed defects: a config seeded with `readsLocalFiles: true`
on disk produced a backup recording `false` before the fix and `true` after; a
`.env` setting the key twice with an empty second line made the shell installer
refuse an agent the launcher would run, and both now agree; an interpreter stub
printing non-JSON now yields `cannot read the writer's output` instead of a
traceback; a `--config` whose parent is a regular file makes the config write fail
with the backup already on disk, which is what shows the backup precedes it; and a
run with no flags at all registers against a scratch `HOME`, which is the path the
empty-array expansion used to break on bash before 4.4.

Proving the backup ordering took two attempts. Making the folder unwritable with
`chmod 555` proved nothing, because this runs as root and root ignores mode bits -
a trap the repository's own skipped test records. The failing-config probe above is
the one that holds.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The first box is deliberately unchecked: there is no unit test covering this tree,
because pytest does not collect it. The end-to-end run above is the evidence
offered in its place. The unit suite was run anyway, on the rebased tree, and
reported no failures at all - so there are no failing ids to diff against the
target branch.

## Risk

**Behaviour change.** An install no longer takes effect immediately. Anyone
following the old flow will see the agent appear only after raven or the gateway
restarts; the installer's last line and the guide both say so. `--endpoint` no
longer exists, so a caller passing it gets an argparse error rather than a silent
fallback.

**Not reproduced, environment-dependent.** On the test host, the provider's model
listing answered 200 while inference answered
`403 This model is not available in your region.`, which the launcher reports as a
run that exits 0 having committed no answer - it reads like a launcher bug and is
not one. Exporting the site proxy fixed it, and the guide now records both that and
the fact that a missing EverOS degrades rather than fails.

**Security.** The pre-write backup can contain another entry's api key, so it is
written at mode 600 and is covered by the existing ignore rules. Keys stay in each
folder's `.env` at mode 600; the tracked `config.json` files hold none. The
installer no longer sends the entry list over a loopback HTTP service.

**Rollback.** Revert the commit: the previous installer talked to the RPC and needs
nothing else restored. Entries already written stay valid either way, since the
schema did not change.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-18 09:07:22 +00:00
gloryfromca
aaf993896e chore(*): merge upstream v0.1.12 into the gitlab trunk
Absorbs the GitHub release into the downstream trunk. A merge rather than a
rebase so v0.1.12 becomes a real ancestor: the next sync starts from v0.1.12
instead of v0.1.11, and the eventual contribution back to GitHub can still tell
our commits from the upstream ones it already has.

Cost this round: 54 conflict hunks over 31 files, against 19 over 14 for
v0.1.11. The growth is ours, not upstream's -- 33 commits landed on the trunk in
the six days since the last sync, and they concentrate in the three files the
sync notes already name as hot (agent/loop/main.py, cli/tui_commands.py,
config/schema.py).

Notable conflict resolutions:

* agent/loop/streaming.py keeps our extracted stream_llm_call and grafts
  upstream's one real addition into it: the truncation verdict
  (upstream_finish_reason, flag_truncation, and the two LLMResponse fields).
  Comparing function bodies against the merge base was what showed aclosing()
  and the structured error classification were already there, not new.
* agent/loop/main.py splits an 87-line block that upstream left empty: two
  helpers moved into upstream's new loop/failure_streak.py and had to go, four
  with live call sites outside the block had to stay. Taking the block whole in
  either direction was wrong.
* _dispatch_backend_store takes upstream's detached-task architecture and keeps
  our memory-failure tracking inside the task body. The shared tail after the
  hunk reads `e`, which upstream's `except Exception:` does not bind, and
  drain_backend_stores already had three merged-in callers.
* context_engine/segments/render.py merges two dimensions into identity_text:
  our work_dir split and upstream's routed model id. The shared body already
  reads names from both sides.
* agent/tools/registry.py keeps both unparseable-argument mechanisms. They sit
  at different layers on purpose -- RunMeta comes from a provider on the
  non-streaming path, RAW_ARGUMENTS_KEY from the loop on the streaming one.

Nine breakages git merged cleanly and never flagged. This is the shape the
change fails in, and it is worth recording in full:

* The three vendored builds under subagents/ carry a copy of the pre-rename
  raven/tui_rpc/ tree, so rename detection sent upstream's 50 lines of
  tui_rpc changes into a dead copy instead of raven/rpc/. Only one of the four
  files conflicted; the other three were silent.
* subagent/backends/raven_loop.py holds the tool loop we extracted from the
  subagent manager, so upstream's run_meta argument never reached it.
* AgentDefaults.max_tokens: upstream dropped it for a dynamic
  send_max_tokens(). Both make_provider sites merged onto the new scheme;
  providers/resolving_provider.py did not, because upstream has no such file.
* ResolvingProvider never forwarded wire_model_id. Only upstream's own
  invariant test catches that class, and the cost is a silently mis-sized
  request (16384 against 4096 for one gpt-4o).
* CronService.add_job kept our body but took upstream's signature, losing
  `deliver` from both the parameter list and CronPayload while two RPC methods
  still pass it, and leaving a call to the _arm_timer scheme upstream replaced
  with a wake Event.
* rpc/bootstrap.py still called make_on_cron_job positionally after the
  signature narrowed.
* update_everos lost _EVEROS_CONFIG for a root-lookup family, and gained an
  ownership gate over its write primitives.
* SessionInitInfo never declared config_notices, in the model or the schema,
  while methods/session.py returns it and our own TUI already reads it through
  a hand-written type intersection. Both are strict, so a config migration
  would have made session.create fail validation in production.
* tests/test_skill_hub_tools.py keyed a fixture under the "local" router
  namespace rather than a real registry layer, which upstream's new assertion
  exists to refuse.

Also fixed while here: the rich Style cache poisoning that the ci branch
documents as a known crutch (one pinned color_system), and the coverage
baseline's 29 stale raven/tui_rpc/ paths. The baseline totals are still
upstream's own numbers and should be regenerated separately.

Verification, on macOS with a non-root runner:

    uv run pytest                8888 passed, 33 skipped, 13 deselected
    uv run ruff check            all checks passed
    uv run ruff format --check   1017 files already formatted
    npx vitest run               99 files, 1265 tests passed
    npx tsc --noEmit             clean, TurnEvent exhaustiveness included
    npm run lint:rpc             schema and codegen in sync
    make check-large-files       exit 0

rpc-schema/openrpc.json is the source of truth here, so ui-tui/src/rpc/
generated.ts was regenerated rather than hand-resolved. Nothing under
subagents/ changed.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-18 15:50:04 +08:00
LivXue
32f8044e86 feat(*): direct chat with a sub-agent instance, in the TUI and the web UI
## Summary

Adds direct chat with a sub-agent instance: talking to one of the instances a
session has spawned, without that exchange entering the main agent's context.
The TUI gets the feature end to end; the web UI then gets the same thing on the
same runtime.

**The runtime.** A direct turn runs on its own lane (`session#agent/handle`)
rather than the session's, so several instances answer at once while the main
agent keeps its own serial lane, and direct turns draw on a separate
concurrency pool so a user is never queued behind them. Direct turns are not
cancellable: `turn.cancel` keeps meaning the main agent's turn on both
surfaces. Each turn writes a record (`prompt.md` / `out.md` / `meta.json`), and
the main agent learns only that the exchange happened, through a handoff block.

**Sub-agent replies over ACP.** Three separate causes were losing most of a
reply. Raven answered every agent-initiated request with `method not found`,
including `session/request_permission`, and the codex adapter turns any error
from that request into a cancel -- so a turn died as soon as the agent asked to
run something. Raven now approves, choosing among the options the agent offered
by their protocol kind. A stop reason other than `end_turn` was returned as a
finished answer, so a truncated reply read as a complete one; it now carries a
notice. And several messages in one turn were joined with no boundary, so a
preamble, its tool calls and the answer ran together as one paragraph.

**Instance visibility.** A failed resume deleted the whole registry row, which
took the instance off the strip for the length of the turn recovering it, and
permanently if that turn failed; it now drops only the stale binding. A
capability snapshot whose fingerprint no longer matched was discarded, which
defaulted the agent to *stateless* -- editing an agent's `env` silently cost it
resume, its chip, and the `instance` parameter in the spawn schema. Its
capabilities are now kept and flagged, while its verdict is not, so the
`/subagents` row asks for a re-test.

**Working directory.** A direct chat was the one sub-agent dispatch that
supplied none, so it fell back to agent home and ran its commands against
raven's own memory instead of the user's checkout.

**The web UI.** Two wiring gaps came first, both silent: the browser's
dispatcher registered neither instance method, and the web channel never built
the map that tags an event with its addressee, so nothing a client could
demultiplex on was ever sent. The service then routes on that tag at the push
rather than at the read, because the main reply and a direct turn share one
subscription and two consumers on one queue would each swallow the other's
events -- including a `message.complete`, which would end the wrong turn. A
direct turn's events are published out of band; feeding them into the reply
stream is not an option, since that stream is one reply per turn.

Decisions and their measurements are recorded in `docs/specs/`. Notably, the
vendored AgentScope tree is *not* edited, which is a decision rather than an
omission: making a direct chat a real reply would mean re-keying a blocking
per-session lock that guards message storage and tool-call state as well as the
reply loop, and rendering reuse was obtained without it.

## Type

- [x] Feature

## Verification

- `uv run pytest -q` -- 8021 passed, 36 skipped. The one deselected failure,
  `tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare`, fails the
  same way on `origin/main` in a clean worktree, so this introduces none.
- `uv run ruff check raven/ tests/ ui-webui/service/` -- clean;
  `uv run ruff format --check` -- 957 files already formatted.
- Web service: `python -m pytest ui-webui/service/tests -q` -- 13 passed.
- Frontend: `pnpm lint` -- 0 errors, 18 warnings, each one also present before
  this change; `pnpm build` -- succeeds; `tsc --noEmit` -- clean.
- Every fix here was checked by reverting it and watching its test go red. Two
  tests that passed either way were rewritten after that check exposed them.
- End to end against a running gateway: a direct chat sent through the new web
  route completed twice, each leaving a record whose `out.md` held the exact
  reply asked for, and the history route returned those turns afterwards.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The last box is unchecked: the terms are defined in `CONTEXT.md` and the designs
in `docs/specs/`, but no end-user documentation or screenshots were produced.

## Risk

**Approving sub-agent permission requests is a widened trust boundary and the
main thing to review here.** An ACP sub-agent that asks to run something now
gets a yes, and the codex preset is configured for its most permissive mode --
wider than the `-a never` the cli transport used. It is unattended by
construction: a dispatch has no operator and no surface that could render a
prompt, so the alternatives were approving or refusing, and refusing leaves the
agent unable to do the work. Rolling back is per-agent config, or reverting
`raven/agent/acp/permissions.py` to restore the refusal.

Two known gaps, neither reachable today, both stated rather than fixed:

- The web UI's event consumer has no `target` guard of its own. The two
  surfaces use disjoint session keys and the browser never sends a target, so a
  direct turn cannot reach it; a caller that sent one on a `web:` session would
  see a sub-agent's reply rendered as the main agent's. Fixing it would widen
  this into unrelated files.
- Direct-chat record directories have no TTL and no count cap, like the spawn
  and DAG trees beside them. Reclaiming them would also start deleting audit
  trails that intentionally outlive session deletion, which is its own change.

Behaviour changes worth naming: a direct chat under a sandbox whose volumes do
not cover the working directory now fails where it used to run quietly in agent
home, matching what the main path already does; and a partial reply now carries
a `[raven]` notice, so anything parsing a sub-agent's reply verbatim sees it.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-17 02:52:57 +00:00
gloryfromca
0340760d94 fix(*): stop silent output truncation from becoming a retry loop (#308)
## Summary

A session that asked the agent to write a terminal game produced 15 tool
failures, 11 of them from one root cause, and burned 51% of its output
budget re-sending the same payload. Output was capped far below what the
model could produce, the resulting truncation was reported as a missing
required field, and the model's most reasonable answer to that message
was to send the same oversized call again.

### The ceiling

`chat_stream` declared literal defaults for `max_tokens`, `temperature`
and `reasoning_effort`, and the agent loop calls it with
messages/tools/model only, so those literals shadowed the configured
values on every streaming turn. Lifting the shadow exposed what it had
been hiding: `agents.defaults.maxTokens` defaulted to 8192 for every
model and onboarding wrote it into the user's config, where it was
indistinguishable from a number the user chose. No single number works
-- claude-sonnet-4 emits 64000 and gpt-4o 16384, while models below 8192
reject the larger value. The setting is retired; an old config carrying
the key is ignored rather than rejected, so no migration is needed.

What replaced it went through two shapes. It first resolved a ceiling
per model and sent it on every request. It now sends nothing unless a
caller pins one: the field is optional in the OpenAI-compatible shape
and required only by Anthropic's Messages API, whose LiteLLM
transformation supplies it. Measured by intercepting the outgoing body,
an Anthropic request built without one goes out at 64000 on both
response paths, and LiteLLM's number matches what this repo would have
computed on all 26 rows it files under that provider, since both read
the same table. Volunteering a number meant carrying a copy of LiteLLM's
routing knowledge -- which vendor speaks which API -- and keeping the
copy aligned; the surveyed agents (LiteLLM, OpenClaw, opencode,
hermes-agent) do not have that problem because each protocol adapter
owns its own required fields.

The catalogue is still read, for the budget and to bound an explicit
pin, and it is not always right: 984 of the pinned LiteLLM's 3040 rows
report `max_output_tokens` at or above `max_input_tokens` -- the window
filed as the ceiling. `openrouter/anthropic/claude-sonnet-4.5`, the id
this repo documents as an example, reports 1000000 where Anthropic
allows 64000. Such a row is treated as no answer, and only when it is at
least as large as the fallback that replaces it, which is what keeps the
rule safe: the replacement is then never larger than what the row
claimed, where an unconditional version raises 252 small rows past their
real ceiling.

### Detecting a cut

Two pieces of evidence, and neither is arithmetic.

**The upstream says so** -- `finish_reason == "length"`. Reliable when
it speaks and never waited on: measured through openrouter, gpt-4o
answers a ceiling hit with `"tool_calls"`, a positive claim of success,
in 4 of 4 probes. It under-reports rather than over-reports.

**The last call's arguments did not parse.** Generation is sequential,
so a cut leaves nothing after it: a repair on the last call means the
turn ended while writing it, and a repair on any earlier call cannot be
that. Measured over four realistic argument blobs, every one of 323 cut
points that drops a field leaves JSON that will not parse, because JSON
is only complete once the object closes and closing it means nothing was
lost.

An earlier shape compared output tokens against the ceiling the request
carried. That comparison is why the request and the check had to agree
on both a number and a model id, and it broke at one call site or
another six times on this branch. No surveyed agent carries such a
comparison. Removing it is what let the decision stop reading the
catalogue at all.

Facts and readings are kept apart. `run_meta` records that the arguments
were repaired and that nothing followed the call; that the turn ran out
of room is a reading of those two and lives in the sentence the model
reads, which offers both causes under their own conditions and says
outright that nothing distinguishes them. It is marked `[incomplete
arguments]`, not `[truncated]`, which is a claim only the upstream can
earn. `truncation_hint` keeps its meaning -- advice for a confirmed cut
-- and the ambiguous case reads `incomplete_hint`, phrased as the
consequent of a condition. A guard fails any tool answering one and not
the other.

Both signals reach every transport: azure was already calling
`json_repair` and dropping the fact, and codex wrapped unparsed text as
`{"raw": ...}`, which stayed dispatchable and made the model read a
schema complaint about a field it had sent.

### A truncated call is not dispatched at all

Originally out of scope as "a call whose fields all validate still
executes, writing a half-file". Running the branch showed the boundary
is worse: `mode` is optional on `write_file`, so a cut landing before it
falls back to `overwrite` and silently replaces what an earlier append
wrote, then reports success. Validation cannot see this -- the call is
well-formed. The refusal moves ahead of validation, and it is wired at
every loop that dispatches, not only the main one: `SubagentManager` and
the curator computed the verdict and dropped it.

`write_file` gained `mode=append` so a file too long for one call has a
second move; `overwrite` stays the default. An append with empty content
is refused rather than treated as a no-op -- that is what a call cut off
before its content field looks like.

### The budget

`reserved_output` was a configured 8192 and is now the model's own
output ceiling, bounded only by the window. Requests do not name a
ceiling, so the one that applies is whatever the vendor or LiteLLM fills
in, and the prompt has to fit beside it: reserving a share of the window
instead lets a prompt reach 150000 of a 200000 window against a reply
allowed 64000, and the sum is refused at request time. `classify_error`
reads that refusal as `should_compress`, but `_emergency_shrink` only
elides `role="tool"` bodies and gates its retry on having elided
something, so a session grown on conversation gets no retry.

### Downstream effects of a cut-off turn

- **Think-tag debris.** A turn cut inside an inlined reasoning block
arrives as a lone closing tag; three patterns matched only bare
spellings, missing both the namespaced form vendors emit (`<mm:think>`)
and the orphan tag, which let a complete think block reach the user as
if the reasoning were the answer.
- **Failure-streak misfires.** The loop-break nudge fired after N
consecutive failures of the same *tool* regardless of whether they were
the same *failure*. It keys on (tool, failure class) now, and the
truncated and ambiguous classes get their own wording -- "change
approach: a different tool" is advice `write_file` cannot take.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests/
  6373 passed, 43 skipped, 1 failed

uv run ruff check raven/ tests/
  All checks passed!

uv run ruff format --check raven/ tests/
  829 files already formatted
```

The one failure is
`test_cli_theme::test_bold_accent_renders_styled_not_bare`, pre-existing
and unrelated: it passes alone and fails when its file runs as a whole,
on this branch and on a clean checkout of the base alike.

**Live reproduction.** Run against a real backend
(openrouter/anthropic/claude-sonnet-4-5) with the ceiling pinned low
enough to force truncation, on the task the original incident used:

| | before | after |
| --- | --- | --- |
| iterations | 40 (hit the cap) | **4** |
| bytes written | 0 | **6213** |
| file parses | n/a | **yes**, 218 lines |
| model's closing line | "sorry, I hit a technical limit" | "done!" |

The before run is the incident in miniature: forty `write_file({"path":
"snake.py"})` calls with the content withheld, each carrying exactly
twenty characters of arguments while the first eight were billed 1200
output tokens each. That run also exercised a design decision argued
only on paper: turn 2 emitted two calls, the first landed 2934 bytes and
the second was cut -- marking only the last call of a turn is what let
the first execute.

**Measurements the design rests on**, each reproducible from the
checkout:

| Claim | How it was checked |
|---|---|
| A cut that drops a field always leaves unparseable JSON | 323 cut
positions across four argument blobs, 0 missed |
| gpt-4o reports a ceiling hit as `tool_calls` | 4 of 4 probes through
openrouter |
| The catalogue files windows as ceilings | 984 of 3040 rows; 252 would
be raised by an unconditional rule |
| LiteLLM supplies Anthropic's `max_tokens` when we omit it | outgoing
body intercepted at the HTTP handler; 64000 on both paths |
| LiteLLM's number matches ours where it knows the model | 26 of 26 rows
under `litellm_provider == "anthropic"`, 0 disagreements |
| MiniMax, marked as speaking Anthropic's API here, is routed
OpenAI-shaped | `/v1/chat/completions`, field absent |

**Regression detection.** This class of defect does not turn a suite
red; it makes the suite unable to see the behaviour. For each change the
implementation was reverted while keeping its tests, and the new cases
confirmed to fail first. Assertions land on outgoing request bodies,
returned messages, the serialized assistant message, and file contents
on disk -- not on whether a call happened. Several cases exist
specifically because a first attempt drove a stub rather than the
assembly: a stub provider builds no request body, and stubbing the
id-resolution hook on one manufactures the agreement a real provider has
to earn.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**`agents.defaults.maxTokens` is gone.** A config still carrying it is
ignored, not rejected, so nothing breaks and no file needs editing --
but a user who set a low value to cap cost will find it no longer has an
effect. There is no replacement setting.

**Requests no longer carry an output ceiling.** Replies are bounded by
whatever the vendor applies rather than by a number this repo chose,
which is longer than the old 4096 on every model and reaches installs
that never configured anything. For a model LiteLLM's table does not
know, its Anthropic fallback is 4096 where this branch would have sent
16384; that caps a reply shorter than the model could produce, which the
truncation machinery absorbs -- the call is refused, the model is told
to continue in pieces, and the work completes over more turns.

**Less history fits in a turn.** The budget reserves the model's full
output ceiling, where it reserved a configured 8192 before. On
`anthropic/claude-opus-4-5` that is 64000 of a 200000 window; on
`z-ai/glm-4.6`, 131000 of 202800. Long sessions compact sooner. The
alternative is a prompt the reply cannot coexist with.

**`write_file` gained a parameter.** Additive: `mode` is optional and
defaults to the existing behaviour.

**A truncated tool call is no longer executed.** Previously it ran
whenever its required fields happened to have arrived. The cost when the
refusal is wrong is one retry.

**Error text the model reads changed.** Anything parsing tool errors by
string match will see different wording, and a new `[incomplete
arguments]` marker alongside `[truncated]`.

`registry.execute` gained one keyword-only parameter with a default.
Truncation metadata rides beside the arguments rather than inside them,
so it reaches neither the tool nor the assistant message. Rollback is a
straight revert of the branch; nothing persists state and no data
migration is involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: zhanghui <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-17 20:56:15 +08:00
JasonJarvan
256b78b23b build: whole-repo lint + pre-commit convention (config only) (#59)
## Change description

> Whole-repo lint + pre-commit convention, CONFIG ONLY (no source
reformat in this PR). Python (ruff) and TypeScript (eslint + prettier)
share a 120-col margin via the committed pre-commit gate. This upgrades
main's existing basic ruff setup and adds the TS lint stack (none
existed on main).
>
> What this PR adds:
> - ruff: line-length 120; security rules
S101/S105/S106/S107/S506/S605/S607; policy per-file-ignores (tests +
accepted CLI deferred-imports + product asserts + N818, each commented);
ruff pinned <0.16.
> - eslint/prettier: shared base (`eslint.base.mjs` +
`prettier.config.mjs`); ui-tui extends it with React rules, bridge with
a Node-only config; hermes-ink vendor and generated RPC types excluded.
> - pre-commit: keeps main's hooks (conventional-commit,
detect-private-key, etc.), adds a top-level exclude and 4 local
eslint/prettier hooks; bumps the pre-commit dev dep.
> - 2 lint-surfaced fixes: PlannerContext import (F821) in the sentinel
runner; lower-cased file-local cron-heartbeat constants (N806).
(raven/config/raven.py's F821 was already fixed on main.)
>
> Sequencing: config only. `ruff check` is intentionally red on ~307
auto-fixables (I001/F401/F541/E701/E702); a SEPARATE reformat branch
runs `ruff --fix` + `ruff format` + eslint/prettier --fix to clear them
(100% auto-resolvable, zero manual). Enforcement (pre-commit install /
CI) turns on only after the reformat lands, so do not expect a green
tree from this PR alone.
>
> CLI-owned files (the reformat and several per-file-ignores touch
`raven/cli/*`) are under review by the CLI team. Judgment items (TS
no-explicit-any, no-unused-expressions, S101 convert-vs-keep, N818
rename, C90, mypy) are deferred to v2.

## Type of change
- [ ] Bug fix
- [x] New feature
- [ ] Document
- [x] Others

## Related issues (if there is)

> none

## Checklists

### Development
- [ ] Lint rules pass locally
- [ ] Application changes have been tested thoroughly
- [ ] Automated tests covering modified code pass

> config-only: `ruff check` is red on auto-fixables by design (the
reformat branch clears them); eslint/prettier configs load in both
ui-tui and bridge.

### Security
- [x] Security impact of change has been considered
- [ ] Code follows company security practices and guidelines

> Only dependency change is the dev-only pre-commit tool plus the ruff
pin; the detect-private-key hook from main is retained.

### Code review
- [x] Pull request has a descriptive title and context useful to a
reviewer
- [ ] Pull request linked to JIRA ticket when applicable

---------

Co-authored-by: Sheng Zhao <sheng.zhao@evermind.ai>
Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
2026-07-01 12:09:42 +08:00
0xKT
dc0fb2822c raven init 2026-06-29 00:23:07 +08:00