139 Commits

Author SHA1 Message Date
Kevin Hu
82c76a82a9
feat(knowledge): read every format, show what the index holds, and let an agent search it (#489)
<img width="2318" height="1448" alt="image"
src="https://github.com/user-attachments/assets/bca71d69-4972-420c-b14e-c8cf74a17de9"
/>

## Summary

Everything a knowledge base could not read, and the pieces it cut them
into
saying what they are.

**Reading a PDF.** A PDF records glyphs at coordinates; whether they are
a
heading, a table cell or a running footer is inferred from how they
look. There
are now two readers. The geometric one infers structure from type size
and
position and needs nothing but the file. The other asks deepdoc's
models, ported
into `raven/knowledge/parser/deepdoc`: a layout model that says what
each region
is, a detector and recognizer that read a page carrying no text at all,
and a
table-structure model that rebuilds a grid from a picture of one. 
 Detecting boxes around text the file already spells
out exactly costs an inference pass per page to learn less, and
recognising it
costs accuracy.

The layout graph answers with a shape its own postprocessing
raises on, and its classes are DocLayout-YOLO's order rather than the
list
Both were established by running the model, and the label order is
pinned by a test.

**Tables go in as tables.** `<table>` is the only shape that keeps what
a grid
states -- which column a cell stood under, which cells are merged, which
row is
the header -- and it is the shape a model reads a table in, which is
what these
chunks are for. PDF, Word and PowerPoint all state their merges exactly,
and
they now survive as `colspan` and `rowspan`. The flat form this replaces
had to
repeat a header standing over two columns across both of them, because
there was
nowhere to record a span; it had no callers left afterwards and is gone.
Two
deviations from upstream, both about what the string is for: every
cell's text
is escaped, where upstream interpolates it raw and a cell holding a `<`
produces
markup that is no longer a table, and the span attributes are written
only where
there is a span, where upstream emits `<td  >` for every ordinary cell.

**A picture of where each chunk came from.** A chunk is a few hundred
words with
a page number on it, which is enough to cite and not enough to answer
whether
the parser read the page correctly. So every PDF chunk that knows where
it sits
carries one image: the regions it covers, cropped out of the rendered
pages and
stacked. here they are already structured in the parser's element spans,
so
the markers would only put text nobody wrote into the thing being
embedded.
The panel draws a chunk's tables as tables, parsed and
rebuilt as elements rather than injected: a person can rewrite any piece
from
that same panel, so what comes back from the store is user input however
it got
there.

**The page that reads them.** Upstream retired the knowledge page while
this
branch was out; it is rebuilt against the new architecture, and the half
that
never existed is the one a reader needs -- what a document actually
became once
it was cut and embedded. Documents sit in one-level folders and arrive
by four
routes: a file, a folder, a URL, a written note. Adding and indexing are
two
calls on the contract and nothing drains that queue on its own, so a
page that
stopped after the first left every upload saying Queued for good.

Opening a file takes the whole page: the document on the left, the
pieces the
index holds of it on the right. The left half is the file drawn as its
own
pages rather than framed by the browser's PDF viewer, which reads where
to open
once and answers to nothing afterwards -- and which nothing can be drawn
on.
Pages are rendered once and kept, and the region a piece was cut from is
washed
over the page it came from, placed in fractions of the page so the
picture's
own scale never enters the arithmetic. Clicking a piece takes the file
there;
double-clicking opens it for rewriting.

The pieces can be searched, filtered by state, turned on and off,
deleted,
rewritten and added to, one at a time or in a batch. Every one of those
was
already on the contract with no reader for it. A base's settings are
back and
fuller: how many pieces a search returns, how documents are cut, and how
much
prose comes with a table or a figure. The embedding model is a picker
while the
base holds no pieces and a stated fact once it does -- moving it then is
the
collection made again at the new width and every document re-cut, which
is not
a thing a panel does by being saved.

**An agent can read them.** The retrieval engine was there and nothing
could
reach it: no agent had a tool that opened a library. A reader now points
a
conversation at bases from the composer, the way they pin a working
directory,
and the selection rides in the same place -- the session's metadata,
bound
around the turn.

The tool takes a query and never a base. The scope is the reader's
choice, so a
model cannot reach material nobody offered it, and it is withheld where
nothing
was picked rather than offered and refused. A hit says which document
and page
it came from, because a hit a reader cannot place is a hit they cannot
check --
and the passages are pressable in the transcript, walking into the base,
the
document, the folder it is filed under and then the piece. A citation a
reader
cannot follow is a claim they have to take on trust, and the whole point
of
searching their own material is that they can go and look.

**The word segmenter and the resources.** `raven.core.Tokenizer` is
Infinity's
`rag_tokenizer`, ported: a forward and backward maximum match over a
dictionary
trie, with stemming and lemmatisation on the latin path. XGBoost is not
carried
over. None of the data lives in git -- the dictionary, the NLTK corpora
and the
ONNX graphs run to about a hundred megabytes, so `make fetch-resources`
on an
install and an optional Docker layer behind `RAVEN_RESOURCES` both point
at one
script. A checkout that could not reach the network still works, and
every
feature that reads this data says so when asked.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on macOS against Python 3.12, with the deepdoc weights fetched (the
model
tests skip without them):

```
uv run pytest                        27450 passed, 111 skipped, 3 failed
npm run test --prefix ui-web          3276 passed
npm run test --prefix ui-tui          2106 passed
npm run type-check --prefix ui-web    clean (tsc)
npm run lint --prefix ui-web          clean (4 pre-existing hook warnings)
make lint-python                      clean (ruff check + format)
make lint-types                       clean (ty)
make lint-imports                     10 contracts kept, 0 broken
make check-source-language            clean
make check-large-files                clean
make build-ui                         built, both boot snapshots match
node ui-web/scripts/check-css.mjs     clean
node ui-web/scripts/check-class-namespace.mjs  clean
```

The three Python failures are not this branch's and were each confirmed
on the
base alone: one asks a host without LibreOffice to refuse a render and
this host
has LibreOffice installed, and two simulation cases fail on an
unmodified
checkout of the base commit.

The RPC contract was regenerated for both consumers and both match it
(`npm run gen --prefix ui-web`, `npm run gen:rpc --prefix ui-tui`, then
`gen:check`). The page's boot snapshots and region goldens were
refreshed where
the composer bar gained a button, and each records that button and
nothing else.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

A PDF's tables, and a Word or PowerPoint file's, are now indexed as HTML
rather
than as header-prefixed lines. Chunk text changes shape for those
documents, so
a base indexed before this and searched after it holds both forms until
its
documents are reindexed; retrieval keeps working either way, because
both forms
are text. The same applies to how much prose a table or a figure carries
with
it: the setting now reaches as far as it says, where before it stopped
at the
nearest paragraph, so a chunk cut before this keeps the narrower context
until
its document is rebuilt. Rolling back is reverting the branch and
reindexing.

The deepdoc models are optional. An install that never ran `make
fetch-resources` reads PDFs geometrically, exactly as before, and says
so when
the models are asked for by name. Nothing here reaches the network at
request
time.

Chunk crops and rendered pages are written at index time and on first
view, both
under the state directory rather than in the repository, and both are
deleted
with their document. A page is immutable for the life of its document
id, so
the route serving them caches; the document route does not. All three
routes
take ids and no path, and the two image routes answer under the
strictest
sandbox in the tree.

What an agent may search is the reader's choice and nothing else. The
tool takes
no base argument, so a model cannot name one, and it is absent from the
schema
entirely for a conversation that named none -- a lane nobody pointed at
a base
is unchanged by this branch. The bases a session carries are stored
beside its
working directory and travel no further.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-12 19:05:16 +08:00
Dizhan Xue
9c893bb65e
fix(*): trust the system certificate store and stop unverified Codex retries (#878)
## Summary

Behind a TLS-inspecting corporate proxy, or against a company-hosted
gateway signed by a private CA, `raven onboard` failed with
`CERTIFICATE_VERIFY_FAILED` even with the corporate root installed in
the operating system (#637). httpx, LiteLLM and the OpenAI SDK verify
against certifi's bundled roots unless told otherwise, so the
operating-system store was never consulted on the paths that reach a
model. aiohttp, urllib and websockets read OpenSSL's defaults instead,
so one process trusted two different sets of roots depending on which
client made the call.

What changes:

- `raven/security/tls.py` (new): `use_system_ca()` injects truststore,
so every TLS context built afterwards is verified by the operating
system (CryptoAPI on Windows, the Security framework on macOS, OpenSSL's
default paths on Linux). A bundle a caller loads into a context is still
honoured: on Linux it joins the store, on macOS its roots become extra
anchors, on Windows it is tried when the system roots fail. So every
root trusted before stays trusted. `RAVEN_NO_SYSTEM_CA=1` opts out,
parsed like `RAVEN_NO_UPDATE_CHECK`.
- `raven/cli/entry.py` (new) is now the entry of both the console script
and `python -m raven`, the form the web supervisor starts its gateway
with. It switches trust before importing the CLI, because the CLI's
imports reach aiohttp (through the sub-agent backends), and aiohttp
builds its default context when it is imported. `raven/cli/commands.py`
no longer starts the CLI when run as a script, which skipped the switch;
pointing it at the entry would not have helped, since by then its own
imports have built aiohttp's context.
- The provider probe names a certificate that fails verification
`certificate_untrusted` instead of `network_error`. It checks for that
ahead of the proxy branch: through a proxy that answered the tunnel, a
refused certificate used to be reported as `proxy_unreachable`. The
Codex check (its backend's list of the account's models), MiniMax's
token renewal (which reported `oauth_token_missing`) and the sub-agent
key prompt's probe name it too. LiteLLM's Copilot token exchange and its
ChatGPT renewal drop the handshake error before Raven sees it, so those
still read as the credential.
- The three surfaces that read the probe say so. `raven onboard` and
`raven provider test` no longer send the reader to check the network,
proxy or VPN; their hints name the three causes: a CA to add to the
system certificate store; macOS and Windows refusing a certificate by
their own rules even when `SSL_CERT_FILE` trusts its CA, which
`RAVEN_NO_SYSTEM_CA=1` turns off; and an expired certificate, or one for
another host, which is the server's to fix. `raven onboard` offers the
menu a network error gets (re-entering the key cannot fix a
certificate); the web Settings page has its own line, in English and
Chinese, in `i18n/messages.json` and the TUI's catalogue generated from
it.
- The RPC contract's description of the model catalogue's `status`
(`rpc-schema/openrpc.json` and both generated clients) now lists the
probe's real words. It named `unauthorized`, which the probe never
returns (a 401 or 403 is `invalid_key`), and missed `proxy_unreachable`
as well as the new status.
- The Codex provider no longer retries a request whose certificate
failed verification with verification turned off (`verify=False`). The
second attempt carried the ChatGPT OAuth bearer token and the whole
conversation to whoever presented the certificate (a captive portal, a
hostile network, a proxy that re-signs TLS), and on a network like the
one in #637 it is what would keep Codex working unverified while every
other provider fails. The retry dates from the initial import;
`_request_codex` loses the `verify` parameter that existed only to carry
it.
- `raven upgrade` and the page's Update verify against the same store as
the release lookup. The upgrade helper runs on the base interpreter,
outside the tool environment that holds truststore, and fetched the
release lists with urllib's default context; uv fetched the rest with
its bundled roots. When the spawning process itself verifies against the
system store, it now copies truststore into a fresh temporary directory
and hands the helper its path, as it hands over the marker path; one
that opted out, or never went through `cli.entry`, hands over nothing
and needs no truststore to say so. The helper loads the copy before its
first download and deletes it, and gives `UV_SYSTEM_CERTS` (and
`UV_NATIVE_TLS`, which older uv reads) to uv's own calls only, so the
Raven it relaunches inherits neither. A copy, because
`inject_into_ssl()` imports requests and urllib3 wherever it finds them,
and the tool environment's belong to what uv is about to rewrite.
- Quick Start, in both languages, gains a section on corporate proxies
and private CAs: WSL2 does not see the Windows store, and the EverOS
memory server and Node-based agents run as separate programs that need
`SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` in the shell profile.
`SSL_CERT_FILE` must name a complete bundle that exists, since httpx
refuses to start with a missing one:
`/etc/ssl/certs/ca-certificates.crt` on Debian, Ubuntu and WSL, and one
built from `/etc/ssl/cert.pem` on macOS. A note says macOS and Windows
apply their own certificate policy and names `RAVEN_NO_SYSTEM_CA=1` as
the way back.

The LiteLLM warning in the issue ("Failed to fetch remote model cost map
... CERTIFICATE_VERIFY_FAILED", with `SSL_CERT_FILE` exported) is not
LiteLLM building its own store. That fetch is a plain `httpx.get`, which
reads `SSL_CERT_FILE`, and the variable replaces the whole trusted set:
with a corporate-only PEM the gateway verifies while
raw.githubusercontent.com, signed by a public root, no longer does.
Measured with a private CA: corporate-only PEM, warning; certifi plus
the corporate root, remote fetch succeeds.

Not covered, by construction: the EverOS server is a separate `everos`
executable and Node-based agents are not Python, so an injection in this
process cannot reach them. The new docs section gives them their
environment variables.

`uv.lock`: adding truststore re-resolved the lock (uv 0.11.25, revision
3 kept). Besides the truststore entries, five packages' dependency edges
lose a marker: matrix-nio, h2, aiohttp-socks, aiologic and culsans. Each
dropped marker is implied by the only path that reaches its package
(matrix-nio, and through it h2 and aiohttp-socks, only where
`sys_platform != 'win32'`; culsans and aiologic only below Python 3.13),
so the resolved set is unchanged on every platform. uv 0.12.23 produces
the same edges and also rewrites the lock to revision 5, so it was not
used; both versions accept the committed lock under `uv lock --locked`.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Red first, on the base: the end-to-end `raven provider test custom`
against a gateway whose root only the stand-in system store holds exits
1 with `network_error ... unable to get local issuer certificate` under
both launch forms, and its opted-out twin prints "Check network /
firewall / VPN settings"; the probe test returns `network_error`, and
through a CONNECT proxy `proxy_unreachable`; the onboarding menu offers
`rekey` for a certificate failure; the web note falls to
`probe_unknown`; and the Codex test's endpoint receives `Bearer
tok-secret`, with the captured log line `SSL certificate verification
failed for Codex API; retrying with verify=False`.

Mutations, each applied and then restored byte for byte:

- the entry no longer switching trust: both end-to-end trust cases fail;
- the switch moved after the CLI import: only
`test_trust_is_settled_before_the_cli_builds_a_context` fails;
- the classifier accepting any `ssl.SSLError`: the plain-HTTP case
fails;
- the certificate check placed after the proxy branch: the proxy case
fails.

Review round two, red first at the round-one head: against an endpoint
signed by a root no store holds, the Codex check returned
`network_error`, MiniMax's token renewal `oauth_token_missing` and the
sub-agent key probe `network_error`; both hint tests failed on the
missing opt-out; the guard scan found `raven/cli/commands.py`; and the
upgrade tests found nothing handed over. Round-two mutations, each
applied and then restored byte for byte:

- the classifier matching `CERTIFICATE_VERIFY_FAILED` in the message:
the new macOS and Windows cases fail (282 of 282 stayed green before
them);
- 17 single-construct mutants of the upgrade hand-over (not switching,
not putting the copy on the path, keeping it, hiding a failed load,
leaving the path in the environment, not telling uv, overwriting a uv
setting the user made, dropping the older uv name, telling uv without a
copy, uv not given its environment, passing on an inherited path,
importing truststore instead of reading whether this process switched,
handing over without that check, handing over always, handing over no
copy, and keeping a copy no helper started, at each spawn site): each
fails at least one test;
- the copy failure's warning and its cleanup: each fails the
copy-failure test.

Commands and results, at the head after rebasing onto `main`. The
commits `main` gained since round one touch none of this branch's lines:
`git range-diff` shows every commit unchanged, and the aggregate
patch-id is the same before and after.

- The touched Python test files (`tests/test_security_tls.py`,
`tests/test_cli_system_ca.py`, `tests/test_config_update_providers.py`,
`tests/test_cli_onboard_commands.py`,
`tests/test_cli_provider_commands.py`,
`tests/test_cli_subagent_setup.py`, `tests/test_updates_upgrade.py`,
`tests/test_openai_codex_provider.py`), the Codex provider's neighbours
(`tests/test_providers_first_byte_budget.py`,
`tests/test_provider_protocol.py`) and the i18n tests, in full: 1010
passed.
- Full suite with branch coverage, `pytest -q -p no:randomly --cov=raven
--cov=raven_everos --cov-branch` on the all-extras environment with the
box proxy variables unset: 2 failed, 27750 passed, 119 skipped. Both
failures fail the same way on a clean tree of the new merge base under
the same interpreter, so neither is introduced; they are local to this
box (an `npm` on its PATH, and running as root):
-
`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`
-
`tests/test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing`
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over that run: 100.00% (57/57 executable changed lines).
- `tests/integration/test_cli_upgrade_real_uv.py`, which CI runs only in
the Windows self-upgrade job (`--noconftest`): 3 passed locally. Its
fixture calls the handoff without `cli.entry`; the `system store`
variant switches first, as `cli.entry` does, so that job runs the whole
hand-over on Windows, and the fixture prints what it handed over so
neither variant passes without its branch running.
- `ruff check`, `ruff format --check`, `ty check` over the changed
modules, `lint-imports` (10 contracts kept), `deptry raven`: clean.
- Repository gates over `origin/main...HEAD`:
`check_source_language.py`, `check_large_files.py` and
`check_commit_messages.py` exit 0.
- Generated files on the rebased tree, where `main` had also changed the
RPC schema, the message catalogue and their generated copies:
`gen-i18n.mjs --check` and `gen-rpc-types.mjs --check` (ui-tui) and
`gen-rpc-client.mjs --check` (ui-web) exit 0.
- Round two changes no file under `ui-web`, `ui-tui` or `rpc-schema`;
these were measured at round one:
- ui-web, `node node_modules/vitest/vitest.mjs run
src/features/settings/SetupBodies.test.tsx
scripts/gates/i18n-keys.test.mjs`: 33 passed; `scripts/gates`: 46 files,
205 passed; `tsc --noEmit` clean, its `--listFiles` resolving the edited
files inside this tree; ESLint clean on the three files.
- ui-tui, the steps of TUI checks: ESLint 0 errors; `lint:rpc` and
`lint:i18n` pass; `tsc --noEmit` clean; the build passes; `vitest run`
has 9 failures in 4 files (renderer and typing-burst tests), the same 9
that fail on a clean worktree of the merge base on this box, so none is
introduced.
- RPC contract: `node scripts/gen-rpc-client.mjs --check` (ui-web) and
`node scripts/gen-rpc-types.mjs --check` (ui-tui) exit 0, each generated
file differing from the base by the one description line; the 9 Python
test files that read the schema, 798 passed.
- Docs: `mkdocs build --strict` clean, with the new paragraphs on the
English and Chinese pages; `tests/test_docs_site.py` and
`tests/test_living_docs.py` pass in the full suite.

The end-to-end cases run on Linux only. They stand the system store in
with OpenSSL's directory store through `SSL_CERT_DIR`, and name a bundle
of public roots in `SSL_CERT_FILE` so that httpx cannot pass by reading
`SSL_CERT_DIR` itself. Verification on macOS and Windows was read from
truststore 0.10.4's source, not run. The upgrade helper's tests share
the limit: on Linux truststore and the standard library read the same
store, so they show the hand-over (the copy loads on the base
interpreter under `-I` before the first download, and uv alone is told),
not the Keychain difference it exists for.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Every Raven process started as `raven` or `python -m raven` now trusts
the operating system's roots in addition to what each library bundles.
For those processes `SSL_CERT_FILE` adds to the trusted set instead of
replacing it; someone using it to narrow trust to one private root keeps
that root but also trusts the system store.
- On macOS and Windows the system verifier applies its own certificate
policy as well. A private certificate that OpenSSL accepted from
`SSL_CERT_FILE` can be refused there if it breaks that policy;
`RAVEN_NO_SYSTEM_CA=1` restores OpenSSL's verification.
- On macOS and Windows truststore also verifies the client's chain on
server-side contexts, refusing a client that sends none. Raven builds no
server-side TLS context: it serves plain HTTP on localhost.
- Upgrades verify their downloads the way the release lookup does: on
macOS that includes Apple's certificate policy, which the helper's
OpenSSL did not apply. uv is told to use the system store only for the
upgrade's own calls, and a `UV_SYSTEM_CERTS` or `UV_NATIVE_TLS` the user
set is kept.
- Behind a proxy that re-signs TLS, Codex requests now fail with the
certificate error instead of continuing unverified. Trusting the proxy's
root, in the system store or in a complete bundle named by
`SSL_CERT_FILE`, is the remedy.
- Startup: about 100 ms warm and 270 ms cold, measured on Linux, mostly
truststore importing urllib3 and requests to patch their references.
- Rollback: `RAVEN_NO_SYSTEM_CA=1` restores the previous trust per
process, and an upgrade started under it hands its helper and uv
nothing; the Codex change has no switch, by design. Reverting the squash
commit removes both.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #637

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-10 10:40:12 +08:00
KT
3632e6040c
chore: bump version to 0.2.4 (#856)
## Summary

Bump the package version from 0.2.3 to 0.2.4 for the next release. Only
pyproject.toml and uv.lock change.

Since v0.2.3, main merged 29 PRs: four features (#817 previews files by
what renders and answers approvals with cmd+enter, #830 moves channels
out of settings beside agents, #833 lets raven read and change its own
configuration, #853 adds reply-style rules to the host prompt), 19
fixes, four docs PRs, one test PR and one chore. None carries a BREAKING
CHANGE footer, so this is a patch bump, the way v0.2.1 and v0.2.3 were
for their features. A v0.2.4-rc1 tag goes first to dry-run the release
pipeline; the stable tag and release notes follow once this lands.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

## Verification

- `uv lock` on a clean worktree cut from origin/main: Resolved 251
packages, Updated raven v0.2.3 -> v0.2.4
- `git diff --stat origin/main..HEAD`: pyproject.toml and uv.lock, 2
insertions, 2 deletions
- `PYTHONPATH=. python scripts/check_commit_messages.py
origin/main..HEAD`: exit 0
- `npx commitlint --from origin/main --to HEAD --config
commitlint.config.cjs`: exit 0
- No test run: the change is version metadata only.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No code change. Rollback is reverting this commit before any v0.2.4 tag
is pushed.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-04 00:07:09 +08:00
KT
a9765e55fd
chore: bump version to 0.2.3 (#811)
## Summary

Bump the package version from 0.2.2 to 0.2.3 for the next release. Only
pyproject.toml and uv.lock change.

Since v0.2.2, main merged 13 PRs: three features (#802 an experimental
Curator that cultivates harness-of-harnesses workers, #805 one header
per desk pane, #806 upgrade from the page with progress), four fixes
(#792 and #799 name grok and copilot connection failures, #807 records
the questions an unattended turn could not ask, #808 keeps the report
shape on interrupted research turns) and six docs PRs. None carries a
BREAKING CHANGE footer, so this is a patch bump, the way v0.2.1 and
v0.2.2 were for their features. The tag and release notes follow once
this lands.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

## Verification

- `uv lock` on a clean worktree cut from origin/main: Resolved 251
packages, Updated raven v0.2.2 -> v0.2.3
- `git diff --stat origin/main..HEAD`: pyproject.toml and uv.lock, 2
insertions, 2 deletions
- `PYTHONPATH=. python scripts/check_commit_messages.py
origin/main..HEAD`: exit 0
- `npx commitlint --from origin/main --to HEAD --config
commitlint.config.cjs`: exit 0
- `uv version` in the worktree: raven 0.2.3
- No test run: the change is version metadata only.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No code change. Rollback is reverting this commit before any v0.2.3 tag
is pushed.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-27 11:46:49 +08:00
zhanghui
f49359eea3
chore: bump version to 0.2.2 (#793)
## Summary

Bump the package version from 0.2.1 to 0.2.2 for the next release. Only
pyproject.toml and uv.lock change.

Since v0.2.1, main merged one PR (#791). It moves the memory plugin's
everos pin to 1.4.1, removes the Windows memory gate, and adds two
guards: an embedding narrower than the memory index is refused where it
is pinned and withheld from the spawn, and a running server whose
version no longer matches the installed everos is replaced rather than
reused. It carries no BREAKING CHANGE footer, so this is a patch bump,
the way v0.2.1 was for its three features. The tag and release notes
follow once this lands.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

## Verification

- `uv lock` on a clean worktree cut from main: Resolved 251 packages,
Updated raven v0.2.1 -> v0.2.2
- `git diff --stat github/main..HEAD`: pyproject.toml and uv.lock, 2
insertions, 2 deletions
- `PYTHONPATH=. python scripts/check_commit_messages.py
github/main..HEAD`: exit 0
- `npx commitlint --from github/main --to HEAD --config
commitlint.config.cjs`: exit 0
- `python -c "import importlib.metadata as m;
print(m.version('raven'))"` after the sync: 0.2.2
- No test run: the change is version metadata only.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No code change. Rollback is reverting this commit before any v0.2.2 tag
is pushed.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-25 19:48:52 +08:00
zhanghui
2b3cce70fa
feat(*): upgrade everos to 1.4.1, lift the windows gate, guard the embedding width (#791)
## Summary

EverOS 1.4.0 runs natively on Windows and 1.4.1 pins `openai` below 3,
so the memory plugin's exact pin moves from `everos[multimodal]==1.2.3`
to `==1.4.1`. The lock follows: `pyarrow` 24.0.0 -> 25.0.1 (EverOS's new
floor), `everalgo-boundary` 0.2.0 -> 0.2.1, `everalgo-core` 0.4.0 ->
0.3.0 (EverOS now pins its everalgo layer with `==`), and `msvc-runtime`
on `win32` only. `lancedb` stays at 0.34.0. Every internal everos symbol
the adapter reaches is present in 1.4.1 with the same signature, so the
adapter itself is unchanged. The plugin manifest and package version
move to 1.4.0.

**The Windows gate is gone.** `everos_platform_note` (#737) and its five
call sites, `ServiceState.UNSUPPORTED`, the import.scan refusal (#750),
the wizard's WSL notice and its zh catalogue line are removed;
`_everos_executable` looks for `everos.exe` on `win32`. Spawning,
probing and reusing the server were already portable. Identifying and
stopping a stale server was not: `_cmdline_of` asked `ps`, and the
marker `everos server start` never matched a Windows command line
(`...\Scripts\everos.exe server start`). The command line now comes from
WMI through PowerShell on `win32` and the marker accepts both shapes;
`_listening_port` asks the TCP table through PowerShell there, walking
the launcher's descendants: the socket belongs to the base interpreter
two launchers below `everos.exe`. The server is spawned in a process
group of its own (the Windows counterpart of `start_new_session`, which
also keeps a Ctrl-C at the gateway off it) and stopped with Ctrl-Break,
which uvicorn takes as a shutdown; one that ignores it for ten seconds,
or one no group can be addressed to, gets `TerminateProcess` as before.
And because Windows cannot replace a running executable, the upgrade
helper stops whatever still runs from under the tool environment before
it installs (the server outlives the gateway by design); without that,
`uv` failed on `Scripts\everos.exe` with os error 32. Recorded in
`docs/memory-plugin-architecture.md` section 7.4.

**Two guards, found while verifying against a copy of a real 1.2.x
store:**

- `set_embedding_endpoint` measures the model's vector width before
writing the pin and refuses anything narrower than the 1024-wide memory
index (`REQUIRED_EMBEDDING_DIMENSIONS`), but only where an EverOS
backend will read the pin: the `everos-memory` distribution is installed
and the config names `everos` as the memory backend (an absent key is
the schema default; an explicit `null` or another backend is not gated,
and a knowledge base sizes itself to whatever width the model returns).
The wizard's own check imports the same constant and probe. A
768-dimension model pinned through the settings page had left every
memory store and search answering 500 about a mismatched width, with
nothing on the page saying why. A probe that cannot reach the provider
is not a verdict: the pin is written and a warning logged. The two RPC
writers run the (now network-touching) writer in a thread.
- `ensure_everos_server` reads the running server's `/health` version
and, for a root raven owns, sends a mismatch with the installed everos
through the same precheck / stop / spawn chain a rotated credential
takes (`stale_reason`). Raven used to reuse whatever answered on the
port, so an upgrade left the old server serving until something
unrelated restarted it. A root the user manages is never touched.

**The wire contract is pinned.** Every request body the plugin and the
memory page send (`/add` with a tool exchange, `/flush`, the four
`/search` shapes, the four `/get` kinds) is validated against everos
1.4.1's own request models in `TestRequestBodiesMatchEverosModels`, so a
schema move on the next upgrade fails in the suite rather than as a 422
in a gateway log.

**A six-angle review then hardened the degraded paths** (turn survival
under every EverOS failure, Windows lifecycle, upgrade paths, the
embedding guard, the wire contract, concurrency). The rule: memory
degrades with a notice and never blocks a turn, a start or an upgrade.

- A gateway starts the server again when it finds nothing listening (or
a probe that timed out) and nothing holding the OME lock: only for a
root raven owns, never over a child of its own, at most once per thirty
seconds. The only spawn used to be in `start()`, so a server that went
away left a running gateway without memory until restarted by hand. A
stop that was sent and did not finish is not adopted (uvicorn closes the
port at once and finishes what it had), so the version-mismatch restart
after an upgrade cannot leave the gateway on a dying server.
- A command-line lookup that failed (no `ps`, PowerShell blocked, WMI
wedged, timeout) is distinct from a process that is gone: the stop keeps
waiting, the lock names nothing, and a settings save over a server that
answers but cannot be identified is reported as not applied. A losing
spawn no longer overwrites the live server's pidfile. `stop_pid` counts
wall time and takes a grace; on Windows a backend drains the server it
started at `stop()`.
- The upgrade helper asks again after `Stop-Process` and refuses to run
`uv` while anything from the environment survives (an elevated or
another user's process): `--force` removes the environment before
writing and would have left every file but the one that could not go.
- An embedding pin narrower than the index is measured at backend start
(once per pin per process) and withheld from the spawn: EverOS runs
keyword recall and keeps storing, and the notice names the model and
width. This covers a pin written around the write-time check (before
memory was on, unreachable at save time, edited by hand). The probe no
longer raises on an odd response, times out at ten seconds, honours
`plugins.disabled`; `migrate_roles` runs off the event loop it had put a
provider round-trip onto.
- The memory page uses the `/api/v2` routes (v1 is EverOS's legacy
alias), asks `/health` before a search the way the chat adapter does,
and shows the server's own sentence on a refusal. The recalled profile
is rendered as lines rather than a `namespace(...)` repr with its
evidence fields; a tool-call-only row stores empty content, not
`"None"`; `top_k` stays within 1..100; an empty query asks nothing; a
failed capability probe is not cached. The wizard accepts a model wider
than 1024 (EverOS keeps the first 1024, as the settings page already
allowed) and re-prompts on the host's refusal instead of a traceback.

`docs/memory-plugin-architecture.md` section 7 records the upgrade (7.4)
the way 7.3 recorded 1.2.1 -> 1.2.3.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Unit and RPC tests over every touched area:

```
env -u SERPER_API_KEY uv run --frozen --python 3.12 --extra dev pytest -q tests/test_everos_server.py tests/test_everos_backend.py tests/test_cli_onboard_commands.py tests/test_everos_discover.py tests/test_rpc_settings.py tests/test_everos_config.py tests/test_everos_plugin_discovery.py tests/test_cli_plugin_commands.py tests/test_core_plugin_stack.py tests/test_rpc_memory.py tests/test_rpc_import_sync.py tests/test_everos_http_adapter.py tests/test_memory_backend_protocol.py tests/test_memory_backend_contract.py tests/test_plugin_boundary.py tests/test_i18n_boundary.py tests/test_config_update*.py
# 1312 passed, 1 skipped (inotify is Linux-only) on the first commit; 960 passed after the review round; full suite 26214 passed, 2 failed, 120 skipped, the 2 failing on an untouched github/main checkout too (LibreOffice path, background child env)
uv run --frozen --python 3.12 --extra dev ruff check <changed files>   # All checks passed!
uv run --frozen --python 3.12 --extra dev ruff format --check <changed files>   # 20 files already formatted
```

Against a 46 MB copy of a store written by everos 1.2.x (568 episodes,
5528 atomic facts, seven LanceDB tables), never touching the live one:

- `everos server start` on 1.4.1 opens it with no schema complaint;
`/health` reports `version 1.4.1`, `cascade.healthy true`.
- The same four `/api/v2/memory/search` requests on 1.2.3 and 1.4.1
return the same ids in the same order; BM25 scores drift in the third
decimal place.
- `raven agent -m` on this branch spawns everos 1.4.1, a "remember this"
turn lands in `episodes/episode-2026-09-24.md` and `.atomic_facts/`, a
fresh one-shot session recalls it, and a third session recalls episodes
written under 1.2.x months earlier.
- With a 1.2.3 server left on the port, the next `raven agent -m` logs
`runs everos 1.2.3 while this raven installs 1.4.1; restarting`, the old
pid exits and `/health` reports 1.4.1. The 1.2.3 server had opened the
index 1.4.1 wrote with no complaint.
- Windows 11 box, this branch checked out and synced: `raven serve`
spawns `everos.exe` natively, the Memory page shows its four tabs
instead of the platform sentence, a chat turn lands as an episode
(`/add` 200, `/flush` 200, cascade `upserted=1`), and after a memory
role change the next start logs `holds credentials raven has since
changed; restarting`, the old everos pid exits and the new one answers
`/health` 1.4.1.
- The settings page refuses `BAAI/bge-base-en-v1.5` on deepinfra with
`returns 768-dimension vectors and the memory index is 1024 wide` (real
probe), and the pin stays unchanged.
- Adapter import smoke against 1.4.1: all 13 internal symbols present,
signatures unchanged.
- Windows 11 box, second round: with the server up, `uv pip install
--reinstall --no-deps everos==1.4.1` fails on `Scripts\everos.exe` (os
error 32, exit 2); after the helper's sweep (`Stopped what was still
running from the old install (pid ...)`) the same command succeeds and
the relaunched gateway spawns a fresh 1.4.1. A stop from the spawning
console runs the full uvicorn shutdown (`Shutting down` ... `Finished
server process`) in 3 s; a stop from another console falls back and the
process is gone in 2 s. `lock_holder(root).port` reads 18997 (launcher
-> venv python -> base python).
- `tests/test_everos_server.py tests/test_updates_upgrade.py`: 261
passed, 1 skipped; `tests/test_everos_backend.py -k RequestBodies`: 7
passed.
- Review round (`env -u SERPER_API_KEY uv run --frozen --python 3.12
--extra dev pytest` over the everos, config, settings, memory page,
import, onboard, plugin-stack, boundary, cycle-budget and contract
files): 1225 passed, 1 skipped; `test_everos_server.py` 124,
`test_everos_backend.py` 188.
- Windows 11 box, review round: with the gateway serving, the everos
launcher was terminated by hand (`Stop-Process`, nothing on 18997); the
next turn answered without memory (`recall failed ...
state=unresponsive`), and six seconds after it began the gateway logged
`started everos server` / `everos server ready`, the pidfile named the
new pid and `/health` answered 1.4.1. The first attempt did not fire
because a dead port there takes 2-4 s to be refused, longer than the 1 s
probe; the respawn now fires on a timed-out probe as well, guarded by
the lock holder. The memory page's four counts and a search on the Cases
tab went through the v2 routes.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Windows installs that used to read "long-term memory is not available
on Windows yet" now spawn EverOS, and a stale server is identified and
replaced there too. The stop there is Ctrl-Break with a ten-second
fallback to `TerminateProcess`. An upgrade on Windows first stops every
process whose executable lives under the tool environment: raven's own
gateway, sub-agents and server, nothing outside that directory.
- An embedding pin narrower than 1024 dimensions is refused at the
settings page and the wizard, with the width in the message, only for an
install whose memory runs on EverOS (plugin installed, backend `everos`
or the default). Knowledge-only installs, memory off, and other backends
keep any width. Existing pins are not touched.
- An owned EverOS server whose version differs from the installed
package is restarted on the next session start. A root the user manages
is never restarted.
- A gateway may now start an EverOS server on its own when its probe
finds nothing listening and nothing holding the lock (owned roots only,
at most every thirty seconds). On Windows a backend stops the server it
started when it shuts down (Ctrl-Break, sixty seconds to drain), so a
settings change or an upgrade there restarts the server rather than
leaving it. An upgrade on Windows refuses to proceed while a process
from the tool environment cannot be stopped, and says which pid.
- An embedding pin narrower than 1024 that reached the config around the
settings check is no longer handed to EverOS: recall runs on keywords
with a notice naming the model, until a 1024-dimension model is pinned.
- Rollback: revert the commit and `uv sync`. `lancedb` did not move, so
an index written by 1.4.1 opens under 1.2.3 (exercised on the copy).

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-25 19:33:18 +08:00
KT
e17694113b
chore: bump version to 0.2.1 (#790)
## Summary

Bump the package version from 0.2.0 to 0.2.1 for the next release. Only
pyproject.toml and uv.lock change.

Since v0.2.0, main merged 31 PRs (3 feat, 22 fix, 5 docs, 1 refactor),
none carrying a BREAKING CHANGE footer. The three features are small UI
additions (wizard steps, file-bar download button, task-palette stop
buttons), so this is a patch bump, as v0.1.12 and v0.1.13 were. The tag
and release notes follow once this lands.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

## Verification

- `uv lock` in a fresh worktree: Resolved 250 packages, Updated raven
v0.2.0 -> v0.2.1
- `git diff --stat origin/main..HEAD`: pyproject.toml and uv.lock, 2
insertions, 2 deletions
- `PYTHONPATH=. python scripts/check_commit_messages.py
origin/main..HEAD`: exit 0
- `npx commitlint --from origin/main --to HEAD --config
commitlint.config.cjs`: exit 0
- No test run: the change is version metadata only.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No code change. Rollback is reverting this commit before any v0.2.1 tag
is pushed.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-24 22:06:16 +08:00
KT
92281ec3a4
chore: bump version to 0.2.0 (#759)
## Summary

Bump the package version from 0.1.13 to 0.2.0 for the next release. Only
pyproject.toml and uv.lock change.

Since v0.1.13, main merged 313 PRs (69 feat, 180 fix), two of them
carrying a BREAKING CHANGE footer, so this is a minor bump rather than a
patch. The tag and release notes follow once this lands.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

## Verification

- `uv lock` in a fresh worktree: Resolved 250 packages, Updated raven
v0.1.13 -> v0.2.0
- `git diff --stat origin/main..HEAD`: pyproject.toml and uv.lock, 2
insertions, 2 deletions
- `PYTHONPATH=. python scripts/check_commit_messages.py
origin/main..HEAD`: exit 0
- `npx commitlint --from origin/main --to HEAD --config
commitlint.config.cjs`: exit 0
- No test run: the change is version metadata only.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No code change. Rollback is reverting this commit before any v0.2.0 tag
is pushed.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-24 01:06:58 +08:00
KT
18203c4b82
fix(channels): run the whatsapp bridge from the gateway and pair only on its status frame (#746)
## Summary

Enabling WhatsApp from Settings > Channels never produced a QR: the
adapter only opened a websocket client to `ws://localhost:3001`, and the
only code that started the Node bridge was the TTY-only CLI login. The
page sat on "waiting for a sign-in code" forever while the gateway
logged a connection refusal every 5 s.

- The adapter now owns the bridge: when `bridge_url` names this machine
and nothing listens there, `start()` builds the bridge once (`npm
install` + `tsc` into `RAVEN_HOME/bridge`), spawns `node dist/index.js`
as its child with the shared token, and `stop()` terminates it. A remote
`bridge_url` keeps the client-only behaviour; a missing runtime or a
child that never listens logs one error and stops the adapter instead of
retrying.
- Pairing is only the bridge's `status: connected` frame. The websocket
handshake used to set `connected`, so a bridge that was merely reachable
drew the row as "receiving messages" before anyone scanned; the
handshake now sets a separate `_bridge_up` flag that guards `send()`.
- The bridge replays its last QR and last status to a client that
authenticates after they were emitted, so an adapter attaching between
two Baileys emissions (about 60 s apart) is not blank until the next
one.
- `send()` raises a transient `ConnectionError` while the socket is down
so the delivery hub retries and counts the drop, instead of recording an
undelivered reply as delivered.
- `channel-whatsapp` now declares `websockets`: the extra was empty
although the adapter imports it, so a wheel installed with only this
extra failed inside `start()` with `No module named 'websockets'`.
- `raven channels login whatsapp` and the onboarding scan step drive the
same spawn path and return once the bridge reports a paired session,
instead of blocking on `npm start` until Ctrl+C.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `python -m pytest tests/test_channels_whatsapp.py
tests/test_rpc_control.py tests/test_channels_manager.py
tests/test_cli_channel_commands.py tests/test_cli_onboard_commands.py
-q` -> 482 passed. New lifecycle tests use a fake bridge and a fake
spawn; reverting each of the three adapter changes turned 5 of them red
(mutation run, then restored).
- `python -m ruff check` / `ruff format --check` on the touched Python
files -> clean. `python -m ty check raven/channels/adapters/whatsapp/`
-> clean. `lint-imports` -> 10 contracts kept.
- `uv lock` after the extra change; `uv lock --check` -> up to date.
Relocking the base pyproject produced no diff, so the marker rewrites in
uv.lock come from this dependency only.
- `cd bridge && npm ci && npm run build` -> tsc exit 0; `npx prettier@3
--check "src/**/*.ts"` -> clean.
- Real machine: an isolated gateway started from this branch
(`RAVEN_HOME=<fresh> python -m raven gateway --page-port 18811`),
WhatsApp enabled from the settings page with a fresh home: the gateway
built the bridge in 5 s, spawned it (`Started the WhatsApp bridge on
port 3001 (pid ...)`), the card showed a QR 10 s after the press, the
row read "running, waiting to be signed in" and stayed so 20 s later
with nobody scanning, and pressing disconnect left no `node
dist/index.js` process behind.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- The gateway now runs `npm install` and a Node child on the first
WhatsApp enable. Without node the adapter stops with one logged error
(the CLI still shows its install menu). Rollback: revert the branch; the
config keys are unchanged.
- `channels login whatsapp` returns on pairing instead of blocking;
scripts that relied on the blocking run must background the gateway
instead.
- A send during a bridge outage now raises for the hub to retry (bounded
by the hub's retry ladder) where it used to return silently.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-23 22:08:13 +08:00
Tchen-data
1b83ffdfee
feat(rpc): show the template picker in the reader's language (#661)
## Summary

The bundled templates ship written in Chinese, so a reader on an English
page saw ten Chinese covers and picked one blind. Translating each
template into a file of its own would double what the wheel carries for
every language, so the templates stay as they are and a phrasebook
beside them (`assets/templates/i18n/en.json`, 843 entries, 59 KB) says
each of their stand-in strings in the other language. The reader's copy
is built from that once and kept in the cache; a language with no
phrasebook, or a copy that cannot be built, falls back to the shipped
file.

The new text is then fitted to the box it landed in, which is the part
worth reading. A Chinese label breaks between any two glyphs, so a
narrow box holds a long one by stacking it and the label's own length
says nothing about how wide the box is. English cannot break inside a
word, so the box is measured instead -- from its own size and the number
of glyphs it held -- and a shape whose text now wants more room is told
to scale that text down. Shapes drawn to one size are given one scale,
so a row of cards does not come back at five sizes. Without this, card
titles wrapped onto a second line that landed on the body text beneath
them, section titles overran their card and covered the subtitle, and
one label came out cut mid-word.

The same pass takes the vendor's marks off the templates themselves.
Every one was cut from a commercial kit that signs the file in places a
slide never shows: core and application properties, the theme's name and
its font and colour schemes, an add-in's tag records, a credit run on a
layout, and a bare web address on the cover and closing pages. The
engine already stripped all of that from a built deck before measuring
it, so a delivered deck was clean; the templates were not, and the
picker is what a reader sees. The cover's second field now says a date
rather than losing the box the address held.

Three things the templates get wrong on their own go with it: a pyramid
whose third tier is numbered 02 like the one above it, a five-item list
that counts 1 3 3 4 5, and two type-size notes ("-12px", "-48px") left
in placeholder copy.

This is stacked on #624 -- `gateway_commands.py` passes the reader's
language into `warm_covers_in_background`, which that PR adds. Merge it
first and this retargets to main.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

`uv run --frozen pytest tests/test_rpc_deck_templates.py
tests/test_rpc_console.py tests/test_cli_gateway_commands.py
tests/test_cli_serve_commands.py tests/test_rpc_files.py
tests/test_utils_office.py -q` -> 370 passed.

`uv run --frozen ruff check` and `ruff format --check` on the touched
files -> clean. `make check-large-files` -> passed.
`plugins-dist/ppt-engine/fetch_templates.py --verify` -> all ten pins
hold. A wheel built from the engine carries the phrasebook and all ten
templates.

Rendered all ten templates in both languages through LibreOffice, 210
pages each, and read every English page. Three checks over the same
renders: no word broken across a line (16 such labels at the start, 0
now), no text overlapping another shape's text that the Chinese does not
also overlap, and no box whose ink runs past it in English alone. The
two circles a reader would have seen cut ("Teamwor/k Matters",
"Communi/cation") and the chart axis that came back rotated and
truncated are in that count.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

A reader whose language is the source language sees exactly what they
saw before: the phrasebook is consulted only for another language, and
`source_for` returns the shipped file. The template bytes changed for
everyone, so a cached cover is redrawn (the cache key carries the file's
size and mtime) and a deck built on one of these templates no longer
inherits the vendor's name in its PDF title. Rolling back is reverting
the three commits; the pins in `templates.manifest.json` move with the
bytes, so a revert leaves the payload and the manifest agreeing.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-22 23:09:10 +08:00
Dizhan Xue
7faa2060e2
feat(*): raven speaks Agent2Agent 1.0, inbound and outbound (#446)
## Summary

Raven speaks the Agent2Agent protocol, version 1.0, in both directions.
The host
agent is the peer: nothing here makes a sub-agent addressable from
outside, and a
raven launched as a sub-agent refuses to serve the face at all, because
it is
already reached over ACP by the host that started it.

Inbound is an aiohttp face carrying the two cards and a JSON-RPC
endpoint. It
mounts on the gateway when `a2a.server.enabled` is set, and `raven a2a
serve` runs
it standalone for a headless deployment, where running the command is
itself the
opt-in. Outbound is a single tool, `a2a_send`, that takes a card URL and
a message;
it enters the tool list only when `a2a.peers` is non-empty, so a host
with nowhere
to send a message does not carry the schema. The two switches are
deliberately
independent: calling out and being called are separate deployments.

Onboarding mints `a2a.server.token` and stops there. The face ships
closed:
finishing an unrelated install is not consent to open a second network
surface, so
`raven a2a enable` is what opens it, and it is the only writer of that
flag.
Enabling mints a missing token in the same command, because a face
switched on with
an empty token answers every caller with a 401 and reads as a broken
deployment
rather than the closed door it is. `raven a2a disable` closes it again
and keeps the
token, so reopening does not re-issue a credential callers already hold.
The wizard
never writes the flag at all, not even as false, so a repeat onboard
leaves a face
the operator opened by hand alone. A config nobody onboarded, and any
`A2aConfig()`
assembled in process, stays inert.

What that opens is narrower than a network face. The app carrying it
binds loopback
at both call sites, hardcoded rather than read from config, and every
request is
checked against a freshly minted 256-bit bearer token. The minting write
is also
what fixes `config.json` to owner-only, from the temporary file's first
byte rather
than with a chmod afterwards; a later enable or disable adds no secret
material, so
it leaves whatever mode the operator chose. Nothing else narrows that
file and it
already held provider API keys, so the obligation predates this work and
is merely
discharged here.

There are two cards because the protocol splits one description across
two
authentication states. The public card answers an unauthenticated GET,
and has to:
a caller reads it to learn which scheme to authenticate with, so
requiring the
credential first would be circular. It therefore names no installed
agent. The
extended card is a `GetExtendedAgentCard` RPC riding the same bearer
check as every
other method, and is the only one of the two that may say what this host
can
actually do.

On that extended card the sub-agent roster rides
`capabilities.extensions`, not
`skills`. A skill is the protocol's word for something an agent can be
asked to do,
and a peer cannot ask for one named sub-agent: it can only send a
message to this
host, which then decides what to dispatch. One skill per sub-agent would
advertise
call targets that do not exist. What a peer can genuinely request is the
orchestration, so that is a single skill, and the roster moves to the
protocol's
own extension point under a versioned URI. That is the only place a
conformant card
may carry a payload the spec does not define, because `AgentCard` is a
closed set
of fourteen protobuf fields: a custom top-level key is rejected by a
strict parser
and silently dropped by a lenient one. The entry declares `required`
false, so a
reader that has never heard of the URI ignores it and still talks to
this host.

Both cards derive `supportedInterfaces[].url` from the origin the caller
arrived
on. One face answers under every name that routes to it, and a URL fixed
when the
face is mounted is right for at most one of them. This matters more than
it looks:
a caller that finds an interface off the card's own origin must refuse
it, since a
card fetched from a trusted origin would otherwise redirect the
credential
elsewhere, and a relative URL reads as off-origin to that check.

## Type

- [x] Feature

## Verification

- `make test-python` at this head: 23729 passed, 112 skipped, 1 failed,
in 288 s.
  The one failure is

`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`,
which fails with the same assertion on `origin/main` by itself; this
branch changes
  no install or node file.
- `uv run --frozen --all-extras pytest
tests/integration/test_a2a_protocol_e2e.py -q`:
6 passed. That directory sits outside the default pytest scope, so it is
run by
  hand rather than reached by the line above.
- `make lint-imports`: 10 contracts kept, 0 broken. `make lint-types`:
all checks
  passed.
- `uv run ruff check raven/ tests/`: clean. `ruff format --check`:
clean.
- `make check-commits` over all 68 commits, `make
check-source-language`,
  `make check-large-files`: all exit 0.
- Live, over a real network boundary rather than in process: a raven in
a Docker
container and a raven on the host, each running this branch, called each
other in
both directions. Nineteen assertions per direction covering both cards,
the four
refusal paths, and a completed turn. The container publishes its face on
a
different port than it binds, so the card has to name the port the
caller used and
not the socket the server opened; both cards did, in both directions.
That run
predates the default-off change described above, which touches no file
under
`raven/a2a/` or `raven/a2a_client/`: both packages are byte-identical to
the head
  it ran against.
- Reverting each fix turns its own tests red, checked one fix at a time
rather than
all together, including the interface-URL fix mutated out inside the
running
  container.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Behaviour changes for an operator who runs `raven onboard` after this
lands: a
token is minted into `a2a.server.token`, and `config.json` is narrowed
to
owner-only. No listening face is opened. Opening one takes `raven a2a
enable` and a
gateway restart, so the second network surface only ever exists because
somebody
typed for it.

Rolling an opened face back is `raven a2a disable` and a restart. The
token stays in
place and is never rotated, so callers already holding it keep working
when the face
is reopened, and a later onboard will not reopen it by itself. Nothing
else in the
install changes.

Costs and limits, stated rather than left to be discovered:

- A gateway that serves A2A imports `a2a-sdk`, measured at roughly 570
ms and 391
modules on top of the gateway's own stack, sqlalchemy and the
OpenTelemetry api
among them. It is paid once at mount, not per request, and an operator
who
switches the face off pays none of it. Moving it to the first request is
recorded
  as deferred in the design note rather than done here.
- `a2a_send` adds 157 tokens to the tool array of every turn, measured
off a
  captured provider call, and only on a host with a peer configured.
- The task store is in memory and unbounded, and inbound concurrency is
not capped.
A local face behind a bearer token is the deployment this is written
for; a
  host exposed more widely should bound both first.
- Pulling in `a2a-sdk` brings the OpenTelemetry api into the gateway's
import
graph. This is a trade accepted for the SDK's protocol types and
handler, not an
  adoption of OpenTelemetry by this project.

Checked during review and deliberately not fixed here: config validation
accepts an
agent row with an empty or whitespace name, which would reach the
extended card's
roster as an empty entry. It is not introduced by this work, the
previous shape
carried the same value, and closing it belongs with the agent schema
rather than
with the card.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-20 14:58:15 +08:00
Dizhan Xue
15cadbd80b
fix(docs): make the Chinese pages searchable by word (#518)
## Summary

Chinese runs no spaces, so the indexed text carried no word boundary at
all.
The search separator broke on whitespace and hyphens, which left each
Chinese
run as a single token, and a reader who typed a word rather than a whole
heading found nothing.

Two things had to line up, and neither works without the other:

- the build segments the text with jieba, which marks every boundary
with a
  zero-width space. jieba was reaching the build only as a transitive
dependency of another group, so it was absent wherever the docs group
was
installed on its own, which is every build that ships. It is declared
now.
- the separator breaks on that mark. The whitespace class the theme
defaults
  to does not cover it, so the marks stayed buried inside the token.

The guard for this passed while the site was broken, which is worth
saying
plainly. It took its query from the page's own heading, boundary marks
and
all, and a query carrying those marks matches the stored token exactly.
No
reader can type that string. The query strips the marks now, and a
second
guard searches a word taken from the middle of a title, which can only
be
found if the run was indexed as words rather than kept whole.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Measured on a site built inside an environment holding only the docs
group,
served locally and driven in a real browser, searching the Chinese site.
Repeated three times with identical counts:

| query | before | after |
| --- | --- | --- |
| a word from the middle of a heading | 0 | 36 |
| a single-word heading | 5 | 12 |
| another single-word heading | 5 | 10 |
| a compound the segmenter splits | 5 | 0 |

The same measurement against the published site reproduced the "before"
column exactly, which is what says the local environment matches the one
that builds it.

- `uv run pytest tests/integration/test_docs_site_search_e2e.py
tests/integration/test_docs_site_controls_e2e.py
tests/test_docs_site.py` -> 34 passed
- both new guards were mutation-tested: with the separator reverted, the
build guard reports that the separator does not break on the boundary
and
  the browser guard finds nothing for its word
- `make lint-python`, `make lint-imports`, `make lint-deps`, `make
lint-types`
-> exit 0, run against an environment synced the way CI installs, not
the
  working one
- `make check-commits`, `make check-source-language`, `make
check-large-files`,
  and `pre-commit run --from-ref origin/main --to-ref HEAD` -> exit 0
- `mkdocs build --strict` -> built, 146 of 146 Chinese entries carry
word
  boundaries

## Risk

User-visible: Chinese search returns results for ordinary words where it
returned none. One case gets worse: a compound the segmenter splits into
parts is no longer found, because the theme does not segment the query,
so a
run that is not itself a word no longer matches any token. A custom
segmenter dictionary would recover those; it is not in this change.

Rollback: revert the commit. The two halves are one line of mkdocs.yml
and
one dependency entry, and dropping either one restores the previous
behaviour on its own.

## Related Issues

N/A

Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-19 23:40:58 +08:00
Dizhan Xue
3dd06f6ea0
feat(docs): add a bilingual documentation site and its deploy pipeline (#507)
## Summary

Stands up a bilingual MkDocs site under `docs-site/`, published to
GitHub Pages
by a workflow that builds on every pull request and deploys only from
`main`.
Fourteen topics ship in English and Chinese: the quick start,
self-hosting,
Docker, the WebUI, the sandbox manual, the runtime architecture, the
command
reference, the repository layout, the developer workflow, the
proactivity design
and reference pair, the self-evolution map and the tracing API.

This pull request is additive. Both READMEs keep their prose and gain
one link
to the site in the header row; trimming them is deliberately a second
pull
request, so the site can be confirmed live before anything depends on
it.

Key decisions:

- **`docs-site/`, not `docs/`.** `docs/README.md` defines `docs/` as
design
notes and dated records, and `docs/plans/` is an archive explicitly not
held
  to today's layout. A user manual has the opposite lifecycle.
- **Full bilingual parity**, via the `mkdocs-static-i18n` suffix
structure
(`page.md` beside `page.zh.md`). Parity decays immediately without a
guard, so
  a test fails when a page has no twin.
- **Anchors are pinned on the Chinese side.** A CJK heading slugifies to
  nothing, so Markdown numbers it `_1`, `_2`, and inserting one heading
renumbers every anchor below it, silently breaking links readers already
hold.
Each Chinese heading therefore states the anchor its English twin gets
for
  free: 133 heading anchors, identical across the two languages, none
  positional.
- **A left rail, not a top bar.** The header carries the brand, the
search box
and the navigation tree down the left; the table of contents on the
right is
  one continuous polyline clipped to the headings currently on screen.
- **No template overrides.** Material customises through
`theme.custom_dir` with
Jinja partials, those partials are `.html`, and
`scripts/check_large_files.py`
  holds `.html` in `BLOCKED_ASSET_EXTENSIONS`. The skin is therefore one
stylesheet remapping Material's own custom properties, plus one script
for the
table-of-contents rail. The brand mark is a `data:` URI for the same
reason.
- **An owner-signed `docs-site/` exemption zone** in AGENTS.md section
1.3. The
i18n config carries the Chinese navigation labels, and `mkdocs.yml` is
not
`*.md`, so the blanket suffix exemption does not reach it. A contract
test
holds the AGENTS.md list and `EXEMPT_PREFIXES` equal, so both moved
together.
- **The site never links out to a file it does not host.** Documents
that stay
in the repository are named as plain paths rather than linked, and a
guard
fails a page that links to an external webpage or a repository-relative
path.

Two defects found and closed here rather than after publication. The
Chinese
site rendered an English sidebar: navigation labels live in
`mkdocs.yml`, not in
any page, so comparing the two languages' Markdown finds nothing and
only the
rendered output shows it. And a control could be present, sized,
animated and
answer `querySelector` while being unclickable, because a popup Material
sized
for a top bar landed outside the viewport once the header became a rail.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

All commands run at the pushed head, rebased onto `origin/main`.

```
make check-large-files                                         exit 0
make check-source-language                                     exit 0
make docs-build                                                exit 0, 0 errors
  docs-site/site/index.html and docs-site/site/zh/index.html   both present
  mkdocs_static_i18n: Translated 17 navigation elements to zh
uv run pytest tests/test_docs_site.py tests/test_source_language_check.py \
  tests/test_readme_scope_canon.py tests/test_docker_runtime.py \
  tests/test_living_docs.py                                    44 passed
uv run pytest tests/integration/test_docs_site_controls_e2e.py 13 passed
pre-commit run --files <changed files>                         all hooks pass
git diff --check                                               clean
```

Anchor parity was measured on the built HTML rather than assumed: 14
page pairs,
133 heading anchors compared at h2 through h6, every one identical
between the
two languages and none positional.

The controls are hit-tested rather than queried.
`document.elementFromPoint` at
the exact point a reader clicks must return the control itself, which is
the
only check that catches an off-viewport popup; the language switcher,
the search
panel, the repository link and the table-of-contents rail each pass it.

The brand lockup is measured, not eyeballed. A failing test was written
first
and reported the wordmark sitting 4.00px below the mark's centre line;
it is
0.00px after the fix, and a 4x screenshot puts the residual optical
offset at
0.25px.

Every guard this branch adds was confirmed to fail first. The
brand-class guard
was mutated from both sides in turn: renaming `.em-eyebrow` in the
stylesheet
turned it red, renaming `class="em-standfirst"` in `index.md` turned it
red, and
it is green again with both restored. The anchor guard was shown to
catch both
failure modes, a missing anchor and a wrong one.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

No behaviour change to any shipped code path. Nothing imports
`docs-site/`, the
new Makefile targets are additive, and the only change outside
`docs-site/` and
its tests is one link added to each README's header row.

Rollback is deleting the branch: the site does not exist until Pages is
enabled
with `Source: GitHub Actions`, and every other change is inert without
it.

Disclosed residue, checked and deliberately not changed:

- The README header now links to `https://evermind-ai.github.io/Raven/`,
which
404s until Pages is enabled. It is the same URL the Documentation
section
  already carries.
- The `deploy` job is skipped on pull requests by design, so a green
check here
does not prove deployment works. The first real evidence is the `deploy`
job's
  own conclusion on the `main` run after merge.
- GitHub Pages is not enabled on this repository. Measured: `GET
/repos/.../pages`
returns 404. Under `Deploy from a branch` the uploaded artifact is
silently
ignored while the workflow still reports success, so the source must be
set to
  `GitHub Actions`.
- `tests/integration/` is excluded by `norecursedirs`, so the browser
suite is a
  local guard and never runs in CI.
- The self-hosting page's Makefile target, docker path and port claims
are
unguarded. They were equally unguarded in the README they came from, and
the
  owner ruled against adding a guard for them.
- The self-evolution SOP and the benchmark contract stay in the
repository and
are named as plain paths from the pages that reference them. The SOP is
a
translation of an upstream project's internal document and carries
redaction
placeholders, which is a publication decision rather than a formatting
one.
- `docs/plans/` still shows the pre-restyle `mkdocs.yml`. CONTEXT.md
defines that
directory as an archive of the tree as it stood, with an explicit rule
against
  updating an old plan to match a later change.
- The Chinese pages drop the brand's italic serif accent word. Cormorant
Garamond
carries no CJK glyphs, so it would fall back to another face without any
sign.
- The code font is Material's default Roboto Mono. The brand names four
  typefaces and no monospace, so none was invented for it.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-19 07:06:24 +00:00
zhanghui
a018351f95
chore(*): gate call correctness with a pinned type checker (#416)
## Summary

Nothing in this repo could see a signature change reach its callers.
Ruff reads one file at a time, the import contracts read edges rather
than calls, and the suite only covers lines it executes. The consequence
was concrete: a required `sandbox_home` argument added to
`BoxliteExecutor` reached `plugins-dist/design-engine`'s render path
with CI green, and a reviewer -- not a gate -- found it.

`ty` answers that question. This adds it as a pinned dev dependency, a
`make lint-types` target and a CI step, with **110 of its 133 rules
live**: nine switched off with their measured backlog recorded beside
each, fourteen shipped disabled upstream and untouched.

Getting there meant clearing 81 diagnostics. This PR folds what were
three stacked PRs (#422 and #423, now closed); each rule's story is a
commit of its own.

### 1. The gate, at 108 rules (`chore(*): gate call correctness...`)

21 diagnostics to clear, five of them real:

| Where | What |
|---|---|
| `evolver/orchestrator/production.py` | `design_of` annotated as taking
two parameters while its docstring, its only implementation and its one
call site all use three |
| `raven/cli/provider_commands.py` | `dict[str, callable]` -- the
builtin function, not a type, so the annotation asserted nothing |
| `raven/config/schema.py` | `Config` is a `BaseSettings` but declared
`env_prefix` / `env_nested_delimiter` through pydantic's `ConfigDict`,
which does not carry those keys |
| `raven/spine/delivery.py` | `StreamDelta.conversation_id` is optional
and was used as a `dict[str, str]` key; a delta without one stored a
`None` entry the close path then read as "no open stream", losing that
stream's done marker |
| `research_flow/support/harness_text.py` | a predicate table typed its
matchers as `object`, so nothing checked they are callable |

The rest were unproven invariants and annotations wider than the code:
three late-bound callables asserted where a comment used to carry the
claim, one closure binding the callable it guarded instead of re-reading
it, four `hasattr` probes that now check what the call actually needs,
and a contextmanager yielding the value it just set.

### 2. `unresolved-import` (`fix(*): resolve the imports the type gate
could not`)

21 diagnostics; three were stale module paths nothing had run into:

- `HarnessNode.current_core_version()` read `raven.__core_version__`, a
module that exists neither here nor in the wheel
`scripts/build_core_wheel.py` produces. The read sat inside `try/except
ImportError`, so the except branch was the only branch and **every node
`store.py` saved recorded `core_version="unknown"`**. It reads
`raven.__version__` now and returns `0.1.13`.
- `raven/agent/loop/wiring.py` imported `SkillForgeRouterConfig` from
`raven.config.schema`; it lives in `raven.config.raven`. Under
`TYPE_CHECKING`, so the annotation simply meant nothing.
- three scripts imported `raven.context_engine.base` and
`raven.memory_engine.base`, neither of which exists. **Two failed at
module import** -- running them was not possible.

The other eighteen were the environment: the three agent products keep
their plugin package beside them, so their roots are declared in
`[tool.ty.environment]`; `lxml` ships a compiled `etree` with no type
information, so `lxml-stubs` is a dev dependency (real types rather than
five silenced lines); `uno` and `com.sun.star.*` are suppressed per
line, since that worker runs inside LibreOffice's own interpreter.

### 3. `invalid-return-type` (`fix(*): make the declared return types
true`)

39 diagnostics, and **one stale annotation held eight of them**:
`run_turn` and the `_run_turn` behind it were declared `-> LoopOutcome`
while all three of their returns build a `TurnOutcome` -- a different
dataclass with no field in common. Nothing broke, because the spine
worker reads what it actually gets and `LoopOutcome` never leaves
`_run_agent_loop`. The annotation was the only thing wrong.

The other 31: nine annotations narrower than the code (five `return
tuple(<genexpr>)` typed as fixed-arity tuples, an inner `one()` still
declaring the two-tuple it returned before a third element was added,
`image_block`'s TypedDict), five declarations refusing a value the
function returns (four tool `execute` overrides saying `-> str` while
returning `ToolResult`), seven invariants that lived in a `# type:
ignore` comment, and ten honest widenings.

### 4. What the gates then caught in this PR's own work

- The qq factory guard passed locally only because an earlier test had
left an event loop behind. Reproduced the CI state
(`asyncio.set_event_loop(None)`), then made the test async so the loop
is its own.
- `invalid-return-type`, while still disabled, hid a regression
introduced here: the qq factory returning the class instead of an
instance. Found by diffing the rule histogram, which is why that rule is
on now.
- The diff-coverage gate named four lines in `raven/routing/cache.py` --
the asserts this work had just added, sitting inside a `try` whose
`except Exception` would have swallowed the `AssertionError` and served
the stale cache. `_do_refresh` returns its data now; the asserts, their
noqa and the swallowing all go with it.

`child_scope`, `QQChannel.start()` and `BenchmarkCache` had no tests at
all and have them now. Every one was mutation-checked.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [x] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `make lint-types` -> All checks passed; 81 diagnostics cleared across
the three rule sets
- Gate proven live three times, once per rule: a missing argument, an
import of a module that does not exist, and a function returning a str
under `-> int` each turn it red, and removing the probe turns it green
- `uv run ruff check` + `ruff format --check` over all six trees ->
clean
- `uv run lint-imports` -> 10 kept, 0 broken; `deptry raven` (all
extras) -> no issues; `uv lock --check` -> resolved
- `make check-source-language` -> exit 0; `commitlint` -> exit 0
- `make coverage` locally: **22050 passed, 45 skipped, 4 failed**; diff
coverage 92.59%
- All four failures are **pre-existing on `main`**, checked by a real
checkout in the same environment where they fail identically: two
`test_the_products_tool_face_*` launcher tests,
`test_a_ceiling_as_large_as_the_window_still_leaves_room_for_history`
(`reserved_tools = 6044 > 6_000`), and
`test_a_missing_key_is_said_once_rather_than_per_query`
(environment-dependent: it asserts `SERPER_API_KEY` is absent)
- Rule counts measured with both `--extra dev` and `--all-extras` --
identical, so this gate's verdict does not depend on which optional
packages are installed

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Behaviour changes are confined to the fixes listed above, and each
replaces an error path with a correct one: nodes are stamped with the
real raven version instead of `"unknown"`, two scripts that could not
start now start, the stream bookkeeping no longer stores a `None` key,
and a cache refresh returns what it fetched. Everything else is a
declaration moved to match what the function already returned.

`tests/` is out of the gate's scope (113 diagnostics under the same
rules). Tests are executed, so a broken call there is already loud; this
gate is for code CI does not run. Bringing them in is sized, separate
work.

Rollback: revert the whole PR to drop the gate and its fixes together,
or delete the `[tool.ty]` block, the `lint-types` target and its CI step
to keep the fixes and drop only the gate.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-16 07:33:14 -07:00
zhanghui
e70a868e9c
refactor(*): seal the memory backend seam so a memory plugin needs no host edits (#414)
## Summary

Make `MemoryBackend` the only seam between the host and the memory
subsystem, so a second memory plugin can be installed without editing
host code.

The abstraction is the goal; no second backend is being added.

What the branch does:

- `health()` joins the Protocol. `raven doctor` and `raven import` read
a `BackendHealth(ready, checks)` the backend fills in with its own
vocabulary, instead of reaching into EverOS internals. Doctor prints the
checks verbatim and never interprets a label.
- A new `[[plugin.contributes.onboard]]` kind, with
`raven/contracts/onboard.py` as its paper. The host lends the wizard
shell as one `OnboardUI` and records the result; the screen returns a
`StepOutcome`. The plugin never writes `memory.backend` -- that key
stays the host's. EverOS's step-4 screen and its `everos.toml`
management move into the plugin.
- The skill source and the after-turn feedback prefix are named from
`memory.backend` rather than the literal `everos`.
- `enabled_by_default` is gone from the manifest: an installed plugin
the user has not disabled is active.
- `tests/test_plugin_boundary.py` pins the dependency direction in both
directions, with an equality assertion so the remaining exceptions
cannot grow.

That guard's allowlist started at five host files still speaking
EverOS's wire protocol. The follow-ups below close three of them.

Two things this branch itself introduced were found in manual acceptance
on an install without the plugin and fixed here: a turn printed which
backends were installed -- empty on exactly the install that needs the
sentence -- and `settings.everos` / `settings.everosSet` let
`ModuleNotFoundError` reach the dispatcher as an internal error carrying
a traceback. Both now carry the note doctor and the wizard already use,
naming the distribution to install.

Review on this PR found four blockers, all fixed and covered. A fifth
defect came out of manual acceptance rather than review, and four more
out of the rounds that followed it -- every one of those in the
embedding endpoint's own plumbing, each found by the round after the fix
that introduced it. They are listed under Risk; the blockers are:

1. `onboard` is an optional contribution, but the wizard equated "no
matching screen" with "not configured", so one `raven onboard
--skip-memory` wrote `memory.backend` back to null for a third-party
backend configured by hand. A backend with no screen now answers on the
strength of having been selected; only one whose own screen says it is
unconfigured is cleared, which is the case that rule exists for.
2. The record a sub-agent call leaves behind ran through a backend
nobody had started. The shipped adapter begins in an unknown state, so
the first flushing write only scheduled a readiness probe and answered
false -- the record said "unavailable" while the service was up the
whole time, and each per-record HTTP client was left open. One context
manager now owns start and stop around a record, in both the manager and
the DAG runner.
3. That record's write and the recall after it addressed different
owners: a real config spells its keys in camelCase and the block reaches
the backend as written, while the adapter read only the contract's
snake_case, so a sub-agent's memories were filed under the host
identity. The adapter reads both spellings.
4. Legacy embedding discovery had narrowed to the recorded plugin root,
so an install predating root recording lost its knowledge-base endpoint
with a valid file still on disk.

**The five follow-ups the seam left behind are in this PR too**, rather
than in three stacked ones, because they are one goal and the branch
protection dismisses an approval on every push:

- The embedding endpoint is raven's. A knowledge base indexes inside the
gateway process and never speaks to the memory service, yet it read its
endpoint out of everos.toml, so selecting another backend left it with
no vectors and no message naming the cause. Raven holds the block now
and hands it to the backend at start -- as a shared default, not an
override: a plugin that was given an endpoint of its own keeps it, and
only one that has none takes raven's. Discovery of an existing
everos.toml still checks all three places the plugin itself looks, read
with plain tomllib so it works where the plugin is not installed. No
data migrates: a base is keyed on its model and measured width, not on
the base url.
- `delete()` joins the Protocol. The browser used to delete a row out of
EverOS's index, in the host process, through the plugin's repository
layer -- while the markdown EverOS treats as the source of truth kept
the text, so the memory came back on the next rebuild, after the user
was told it was gone. The adapter now uses only operations EverOS
already performs, for the kinds it performs them for.
- Sub-agent memory goes through the contract. The read half is a new
optional `recall_session`; the write half needed no new surface, only
two metadata conventions written down. `agents.<name>.config.everos`
becomes `config.memory`, opaque to the host except the two keys the host
reads; the old spelling still loads.
- The memory and settings pages say why they are empty -- no plugin, or
a backend this page does not read -- instead of four zeros and a retry
button that cannot help.
- The `everos` package leaves the host's dependencies for the plugin's
own, at the same exact pin; the host reaches the tree through the
plugin. This is the package layer only: the installers already deliver
`everos-memory` as a sibling wheel from the same release, so an ordinary
install still has long-term memory.

The boundary guard's allowlist falls from five entries to two, both GUI
surfaces still shaped around EverOS's own store. Generalising their wire
is a frontend change and is deliberately not in scope.

This branch was reviewed on GitLab as merge request 639 over three
rounds, including a post-rebase range-diff check. It is being moved here
because GitHub is now the source of truth.

## Type

- [x] Refactor

## Verification

- `uv run pytest` on this branch, rebased onto `main`: 22146 passed, 4
failed, 45 skipped. Three of the four (`test_agents_oncall_launcher`,
`test_agent_loop_token_budget`, `test_agents_code_launcher`) were re-run
on a detached checkout of that same `main` commit in this environment
and fail there identically. The fourth (`test_ppt_engine_image_search`)
could not run in that checkout, whose environment lacks the deck
engine's extras; this branch's only change anywhere under
`plugins-dist/ppt-engine` is the removal of the `enabled_by_default`
manifest key, which that test does not reach. So this branch adds no
regression.
- `npx vitest run` in ui-web: 1680 passed across 103 files. `npm run
type-check` and `npm run gen:check` clean, so the generated client
matches the contract.
- Every fix in the follow-up commits was mutated away and its test
confirmed red, then restored and confirmed green. For the record path
that is three separate mutations: dropping `start()`, dropping `stop()`,
and narrowing the owner lookup back to one spelling.
- `uv run ruff check .` and `uv run ruff format --check` on the changed
files: clean.
- `uv lock --check`: consistent.
- `scripts/check_commit_messages.py`, `scripts/check_large_files.py`,
`scripts/check_source_language.py` over the branch range: clean.
- Manual acceptance against a real isolated install (temporary
RAVEN_HOME, isolated EverOS root and port), 34 cases: `raven plugins`,
`raven doctor` across four backend states, `raven onboard` interactive
and non-interactive including all three step-4 lanes, `raven import`
refusing and importing, an end-to-end write and recall, the web UI's
memory and settings pages driven through a real browser, a sub-agent
trace record, and an install with no memory plugin at all. 31 passed; 1
failed and was fixed; 1 partial; 1 carried over from the earlier round.
- The failure is the one worth reading. On a fresh managed install the
wizard reported embedding configured and the service it started the next
moment ran keyword-only: the spawned process carried no
`EVEROS_EMBEDDING__` values, and EverOS logged that the embedding key
was not configured. The rule deciding whether everos.toml already held
an operator's own endpoint asked whether its model name was a `"<...>"`
placeholder; the shipped template seeds every section with a real model
name and an empty key, so every fresh install read as a deliberate
choice and the host's endpoint was never bound. It now asks the
criterion the module already had -- model and key. The unit tests missed
it because each built its "operator's own" section with a key in it; the
regression copies the shipped template itself.
- The partial: the memory page's search returned relevant results
without error, but every memory in the test corpus was on one subject,
so narrowing could not be observed. The carried-over case is the
skill-source prefix, whose path this branch does not touch.
- Findings recorded and not fixed here, none introduced by this branch:
a progress spinner is still drawn after `import` reports it imported
nothing; an unsupported delete surfaces as a generic internal error; the
TUI log and `doctor` word "no memory plugin" differently; a turn that
did reach disk still prints that it was not written.
- The partial case: an induced agent skill is discovered through
`find_skill` with the backend's prefix, but `read_skill` / `use_skill`
on that id fail. Nothing writes the on-disk layer those tools look in,
on this branch or on `main`, and no test covers it. Pre-existing; filed
as a follow-up.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes:

1. User-track recall is now strictly bounded by `top_k`. A profile row
used to ride outside that bound.
2. `raven doctor` builds the plugin factory, so a managed EverOS setup
writes its env var and config templates during a doctor run. It used to
be read-only. The `--json` memory section is reshaped around
`health.checks`.
3. A third-party drop-in plugin without `enabled_by_default = true` used
to be skipped and now loads.
4. The loader no longer back-fills the EverOS config slice's `root` /
`owned`, nor strips a legacy `mode` key; a leftover `mode` now draws an
unknown-key warning.
5. The wizard's recap shows the contribution name `everos` rather than
`EverOS`.
6. `use_skill` with an unknown prefix reports "not found" after a disk
lookup instead of "unknown source".

From the follow-ups:

7. Config gains an optional top-level `embedding` block. Absent, the
knowledge base reads the EverOS file as before and says so once. An
operator who moves the values across changes nothing else; one who names
a *different* model there gets the existing `StaleBaseError` telling
them to rebuild that base. The block is a default for the memory plugin
rather than an override, so an install that had configured the plugin
separately keeps what it had. Note that a config carrying this block
does not load on an older Raven, which rejects unknown top-level keys.
8. `agents.<name>.everos` is written back as `agents.<name>.memory`. The
old spelling still loads; a per-agent `baseUrl` is ignored with a
warning, having never had a user.
9. Deleting a memory now changes what EverOS treats as the source of
truth, so it stays deleted. Profiles and cases report that they cannot
be deleted rather than appearing to succeed.
10. `pip install raven` no longer pulls the memory engine in.
`install.sh` and `install.ps1` are unaffected: both already resolve
`everos_memory-*.whl` from the release they install and pass it as
`--with everos-memory`, and the source path installs it editable, so
those installs keep long-term memory. An install that reaches neither --
a release predating the sibling wheel, or a bare `pip install` -- now
boots without memory and says which distribution to install rather than
failing on an import.

Found after the acceptance run, in review, all fixed here. They share
one cause worth stating once: the endpoint acquired a second home, and
each fix taught one more reader about it while leaving the next one
behind.

11. The endpoint the wizard saved never reached the EverOS process it
then started. The rule for "does everos.toml already hold an operator's
own endpoint" asked whether its model name was a `"<...>"` placeholder;
the shipped template seeds every section with a real model name and an
empty key, so every fresh managed install read as a deliberate choice
and the host's endpoint was never bound. The service came out of the
wizard running keyword-only while the wizard said embedding was
configured. The rule now asks the criterion the module already had:
model and key.
12. The exported `EVEROS_EMBEDDING__*` variables are a third home,
documented by EverOS and ranked above its file. They were being
overwritten. A complete triple is now an operator's choice; an
incomplete one is not an endpoint and is replaced whole rather than
topped up from the host.
13. Raven writes those same variables itself at startup, so a reader
that asked only whether all three were set called raven's own binding an
operator's export -- a blank settings card and a save refused by naming
variables nobody had set. The module now records which of them it set.
14. That record lived in a module-level set, and `raven gateway
--restart` re-launches through `os.execv`: the values survive, module
state does not. The record now travels in a variable beside them, and is
stripped from the environment handed to EverOS.
15. A settings save the environment would outrank is refused with the
variable names, rather than written and silently ignored.

Four of the five arrived one per review round, each in the fix for the
one before it. The tests that missed them all shared a shape: each built
its "an operator configured this" fixture with a key in it, or cleared
the environment before the case, so the states the product actually
produces -- a template with no key, a live process that has just bound
its own endpoint -- were never the input. The regressions now copy the
shipped template, and run the startup sequence and a second interpreter
rather than a prepared state.

Rollback is a revert of the branch: the seam is additive, and
`memory.backend` keeps its shipped default.

- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-16 22:19:50 +08:00
KT
10451a705d
build(deps): raise the python dependencies whose advisories have a path here (#448)
## Summary

Thirty-three Dependabot alerts sit on `uv.lock`, the only manifest in
this repo
whose packages are installed on a user's machine, and no pull request
had been
raised for any of them. The python-security group PR (#227) was
auto-closed by a
history rewrite on 2026-09-12 and has not returned. This closes all
thirty-three
of them.

| Package | From | To | Alerts | Why |
|---|---|---|---|---|
| pillow | 12.2.0 | 12.3.0 | 13 (10 high) | `prepare_image` in
`raven/agent/tools/media.py` calls `Image.open` on bytes the agent is
handed, and the design engine calls `paste()` and `crop()` -- the APIs
the heap out-of-bounds advisories name |
| mistune | 3.2.1 | 3.3.4 | 10 (4 high) | One call site, rendering
outbound markdown for the Matrix adapter |
| aiohttp | 3.14.1 | 3.14.3 | 3 (1 high) | The C response parser,
reading answers from provider endpoints a user configures |
| json-repair | 0.59.5 | 0.63.4 | 1 (1 high) | Unbounded CPU on circular
schema refs |
| h2 | 4.3.0 | 4.4.1 | 1 | Ordinary minor bump |
| hpack | 4.1.0 | 4.2.0 | - | Pulled by h2 |
| pip | 26.1.2 | 26.2.1 | 1 | Ordinary minor bump |
| mcp | 1.27.0 | 1.30.0 | 3 (3 high) | Advisories are server-side only
and this repo is a client; taken to leave no alert standing |
| cryptography | 48.0.1 | 50.0.1 | 1 (1 high) | Advisory is PKCS#7
`EnvelopedData`, which nothing here decrypts; taken for the same reason
|

The last two were held back in the first commit on a cost argument, and
the
argument was overruled: every alert closes rather than carrying a
documented
exception. It is worth recording what the exception would have said,
because it
is still true of the code. `mcp`'s three high advisories are all
server-side --
HTTP transports serving session requests without auth, the WebSocket
server
transport, experimental task handlers -- and every import in this repo
is
`mcp.client.*` or `mcp.shared.*`, with no `mcp.server` import anywhere.
`cryptography`'s advisory is PKCS#7 `EnvelopedData` decryption, while
the only
import here is `cryptography.hazmat.primitives.ciphers`, the AES
fallback in the
weixin adapter. Neither was reachable; both are now moot.

A major version is where an entry point disappears quietly, so that was
checked
before the suite ran: all sixteen symbols this repo imports from the two
still
resolve on the new versions.

Four land above the version their advisory requires -- `json-repair` at
0.63.4
for a 0.60.1 fix, `mistune` at 3.3.4 for 3.3.0, `mcp` at 1.30.0 for
1.28.1,
`cryptography` at 50.0.1 for 50.0.0 -- because `uv lock
--upgrade-package`
resolves to the newest version the constraints allow. Say the word and
any of
them can be pinned to the minimum fix instead.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [x] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Full suite against the upgraded lock, twice: once on the first cut and
again
after rebasing onto current `main`.

```
$ uv sync --locked --all-extras --dev && uv run pytest -q
1 failed, 22930 passed, 121 skipped, 110 warnings in 229.20s
```

The single failure is

`tests/test_read_file_image.py::test_image_block_is_the_only_place_the_shape_is_written`,
and it is **not** from this change. It shells out to `grep` and filters
the
result on the string `raven/utils/images.py`; the `grep` on this
developer's
PATH emits `raven//utils/images.py` with a doubled separator, so the
filter
misses its own file and the test reports itself as an offender. Checked
against
an unmodified `main` worktree, where it fails identically. CI uses GNU
grep and
is green on it.

So: 22930 passed, zero regressions from the nine bumps.

- [x] Relevant tests pass locally
- [ ] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

This is the batch that reaches users: `uv.lock` is what `pip` and
`uv tool install` resolve, so unlike the npm trees these packages run on
the
machine of whoever installs raven.

All nine stay inside the constraints already declared in
`pyproject.toml`; no
constraint was widened and `pyproject.toml` is untouched. `pillow` and
`aiohttp`
are patch-level. `mistune` and `json-repair` move a minor, and both are
used in
one narrow place each, which the suite covers.

`cryptography` is the one to watch: two major versions, and it is a
build
dependency of several other packages in the tree as well as a direct
import
here. `mcp` moves three minors, and seven files use its client.

`pillow` ships platform wheels, so the failure mode to watch is an
import error
on a platform the resolver picked differently, which shows at install
rather
than at build.

Rollback is to revert the commit and `uv sync`.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A. Closes all 33 Dependabot alerts on `uv.lock`.

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-16 22:07:41 +08:00
0xKT
d08a416de4 build(*): the three playwright pins read as one specifier
The same library was pinned three ways: the host browser extra said
>=1.49,<2.0 while the design render extra and the ppt-engine carrier
both said >=1.58,<2. One environment resolves one version, so the host
floor rises to match through uv (lockfile re-resolved in the same
change; the resolved version does not move). A contract test now holds
the three declarations byte-identical and each table to exactly one
playwright entry, so a lone bump -- or a cleanup that reads the
unimported carrier as stale -- turns red here before it breaks a
rendering install. dev.md gains the one line naming how a source
checkout gets the chromium binary itself.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-11 23:48:28 +08:00
gloryfromca
58dffbf13c chore(*): merge the github trunk into the gitlab main
## Summary

Merges the GitHub trunk into this fork's main, as a merge rather than a rebase:
the resolutions below live in the merge commit, and rebasing would drop them.

Eleven upstream commits since 377a1da14 (2026-08-25), fifteen files
conflicting. The conflicts worth a reviewer's attention:

- **tracing**: upstream retired the runtime attempt id. `attempt.id` is now a
  reserved span attribute and `attempts.json` owns the grouping, so
  `begin_attempt` / `end_attempt` / `current_attempt` and every `attempt_id`
  parameter go with it. Nothing on this side called them outside tests. This
  fork's `surface` field and its `use()` helper share those hunks and stay.
- **appLayout.tsx**: took upstream's copy-notice line only. Its
  `FloatingOverlays` hunk would render that component twice, because this side
  had already moved it into its own anchor box.
- **onboard_everos.py**: upstream only refreshed three model ids, which this
  side already carries. Kept the English-only role table (AGENTS.md 1.3).
- **test_everos_server.py**: took upstream's inotify-gate tests and pointed
  them at `raven_everos.server`, where this side moved the module.
- **README.zh-CN.md**: the benchmark table had to be rebuilt by hand. Its
  separator row auto-merged with the prerequisites table's and silently
  dropped the three data rows.

Three follow-up commits fit upstream's new tests to this fork's environment.
Each is a fork/upstream difference, not a defect upstream:

- The listing tests widen the console with `env={"COLUMNS": "200"}`. That
  cannot work here: this suite runs under xdist (`addopts` carries `-n 4`) and
  its workers export `COLUMNS=80`, which rich freezes into `Console._width` at
  import time, before any invoke sets the variable. An autouse fixture gives
  the module console a real width, the way upstream's own browse test does.
- One test patched `raven.config.loader._current_config_path`; this fork keeps
  that name in `raven.home`.
- Three CJK fixtures are written as `\uXXXX` escapes so the language gate
  passes. `test_table_cjk_cells_stay_within_budget` measures what an
  east-asian glyph costs in a table cell, so an ASCII replacement would void
  the test; the escape keeps the runtime value upstream's.

**Reviewer attention, one deliberate rule break**: `MODULES_IN_CYCLES_CEILING`
in `tests/test_import_cycle_budget.py` goes 42 -> 46, and that file's comment
says the ceilings may only go down. Upstream's four new modules import each
other in pairs (`cli.trajectory_browse` with `cli.trajectory_commands`,
`trajectory.bugreport` with `trajectory.review`). Neither pair can be unpicked
by deferring an import, because that graph counts a function-local import like
a top-level one, so unpicking either means moving code that has just arrived
from upstream. The package and pair ceilings did not move. Raising the number
here is the call a reviewer may want to reverse; doing so means a refactor of
upstream's new modules, roughly 110 lines moved.

Project settings changed alongside this: `squash_option` `always` ->
`default_off`. A squash would flatten the merge commit and lose the manual
resolutions above, after which the next sync would have to redo all of them.
`merge_method` was already `merge` and is untouched.


A manual smoke run on the real hosts found one bug the suites did not, now
fixed on this branch: the sync took upstream's copy-on-select wiring into
`useMainApp` but not the two import lines it needs. Both sides had edited that
import block, so git resolved it to this fork's version and the added lines
went with it -- a semantic conflict, no marker. `raven tui --dev` died on entry
with `createCopyOnSelectReporter is not defined`. The 2072 vitest tests did not
catch it (none imports and renders the whole hook) and neither did eslint
(undefined names are left to the type checker here); `npm run type-check` does,
and is clean after the fix.


**Known red gate, recorded rather than bypassed.** Five of the eleven imported
commits fail `scripts/check_commit_messages.py` over this MR's range
(`1aef9f50`, `a83a3565`, `5ab90aa5`, `9c55d545`, `dcedd1c4`). The only
non-ASCII in them is three characters: the display name in a colleague's
`Co-authored-by` trailer (U+6C5F U+56FD U+5E86, address `guoqingjiang@`
unchanged and ASCII). Subjects and bodies are English. The job that runs this
checker fires on GitHub `pull_request` events, so a merge performed on GitLab
never runs it -- which is why this pipeline is green while the gate is red over
the same range.

The disposition, settled between the maintainer and the reviewer: a single
deliberate pass over the imported history will rewrite committer and author
information wholesale, and these five trailers belong to that pass. Rewriting
them here would be redone by it. So this is assigned to the change that can do
it once across all imported history, rather than five commits at a time inside
a sync -- not waved through as unimportant. This paragraph exists so the next
reader sees the gate was weighed, not missed.

One cleaner route remains open and is deliberately not taken here: teaching
`commit_lint.py` that a trailer's display name may be non-ASCII while its
address stays ASCII, which is arguably what AGENTS.md 3.1.1 already means when
its table calls the trailer "ASCII by format".

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other (upstream sync)

## Verification

- `uv run --all-extras pytest -q` -> 21814 passed, 4 failed, 45 skipped.
  The four are pre-existing on main, verified by running them on a clean
  `origin/main` worktree in the same environment:
  `test_agents_oncall_launcher.py::test_the_products_tool_face_equals_the_forks_config_intent`,
  `test_agents_code_launcher.py::test_the_products_tool_face_is_the_forks_config_intent_minus_the_ledger`,
  `test_agent_loop_token_budget.py::test_a_ceiling_as_large_as_the_window_still_leaves_room_for_history`,
  `test_ppt_engine_image_search.py::test_a_missing_key_is_said_once_rather_than_per_query`.
- `npm test` in `ui-tui/` -> 143 files, 2072 tests, all passing.
- `make lint-python` -> 1930 files already formatted.
- `make lint-imports` -> 10 contracts kept, 0 broken.
- `make lint-deps` -> no dependency issues.
- `make check-source-language` -> clean.
- `make check-large-files` -> clean.

Manual smoke on the real hosts (isolated `RAVEN_HOME`, own tmux session, the
running instance on this machine left untouched):

- `raven trajectory list` -> three attempts, matching the span log on disk.
- Interactive browser (bare `raven trajectory`, driven under tmux): session
  list, attempt list, and the `turn.input_preview` values all render from the
  real log. `[M]` merged two attempts; `attempts.json` gained the definition
  with exactly those two member trace ids, and the span log stayed append-only.
- `raven trajectory split` put it back: `attempts.json` empty, listing back to
  three rows.
- `raven trajectory report-bug` produced `br-20260911-b19368.tar.gz` with the
  envelope and the nested trajectory inside; it correctly reported the fixture
  as unreplayable and pinned the attempt.
- `raven tui --dev` reaches the main layout, and the completion overlay anchors
  correctly (the `FloatingOverlays` hunk this merge resolved by hand).
- Copy-on-select driven end to end by sending SGR mouse press/drag/release
  through tmux, against a sentinel value placed on the clipboard first. The
  first drag put the selected span on the clipboard (`pbpaste` returned
  exactly the 76 characters under the drag) and reported `copied 76
  characters` into the transcript; a second drag reported `copied 10
  characters` as the transient notice above the composer, which is the line
  this merge resolved by hand. One drag, one clipboard write, one notice:
  the duplicate subscription is gone from the observable behaviour too, not
  only from the source. The host clipboard was restored byte-for-byte after.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes all come from upstream: the interactive trajectory
browser, the tester-facing bug report flow, the feishu websocket backoff, the
EverOS inotify spawn gate, and the TUI copy-on-select notice. The fork's own
behaviour is unchanged; the only fork-side edits are in tests.

The one runtime removal is the tracing attempt API. No caller outside tests
used it on this side, and upstream replaced what it did with `attempts.json`.

Rollback: revert the merge commit. Because this lands as a merge rather than a
squash, the branch history stays intact and a revert leaves main exactly where
it was.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 15:43:23 +00:00
gloryfromca
d0af1b89cc chore(*): merge the trunk into the sync branch
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-11 18:01:21 +08:00
gloryfromca
46029007ce chore(*): merge upstream main into the gitlab trunk
Eleven commits from the GitHub trunk since 377a1da14, fifteen files
conflicting. How each was resolved:

- raven/tracing/context.py, trace.py, spans.py and tests/test_tracing_api.py:
  upstream retired the runtime attempt id. attempt.id is now a reserved span
  attribute and attempts.json owns the grouping, so begin_attempt,
  end_attempt, current_attempt and every attempt_id parameter go with it.
  Nothing on this side called them outside tests. The fork's surface field
  and its use() helper share those hunks and stay.
- raven/trajectory/store.py: upstream's module docstring documents
  attempts.json and is a superset of ours.
- raven/cli/onboard_everos.py and tests/test_cli_onboard_commands.py:
  upstream only refreshed three model ids, which this side already carries.
  Kept our English-only role table (AGENTS.md 1.3).
- tests/test_everos_server.py: took upstream's inotify-gate tests and pointed
  them at raven_everos.server, where this side moved the module.
- ui-tui/src/app/useMainApp.ts: copy-on-select was never wired up here;
  upstream wires it.
- ui-tui/src/components/appLayout.tsx: took upstream's copy notice line only.
  Its FloatingOverlays hunk would render that component twice, because this
  side had already moved it into its own anchor box.
- README.md and README.zh-CN.md: upstream's wording under this side's emoji
  headings. The zh benchmark table had to be rebuilt: its separator row
  auto-merged with the prerequisites table's and dropped the data rows.
- tests/test_provider_rates.py and tests/test_trajectory_bundle.py: both
  sides appended independent tests at the end of the file; kept both.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-11 17:52:06 +08:00
Tchen
e5bf6f8aed build(ppt-engine): carry playwright so an install that renders has it
The design lane imports playwright and declares it in an optional extra, so a
release install arrived without it and the browser half of rendering was off --
visible only when a render came back unavailable.

An extra is one more thing every installer has to know to ask for. The deck
engine's distribution is already fetched unconditionally by the same install, so
the pin rides along with it and every packaging route gets the library for free.
2026-09-11 12:53:39 +00:00
0xKT
a15e49eac5 fix(*): stop the page hiding a refused delete and a refused upload
## Summary

Two bugs in the served page, both of the same shape: a refusal the surface did
not report as one. They are one MR because they were found in one pass over the
page's write paths and each is a few lines; the commits are separate and can be
read independently.

**1. Bulk delete dropped rows the server had kept.** `DS.sessions.deleteAll`
awaited `session.delete` and counted every call that did not throw as a
removal. A refusal does not throw: the method answers
`{deleted: null, still_on_disk: true}` when the filesystem refused to remove
the file, and that is a resolved promise. The rail dropped the row while the
transcript stayed on disk, so the session came back on the next reload -- the
one thing the note above that function says it must never do. The bulk path now
reads the answer with the predicate the single delete already uses. The
single-delete half was fixed in 3d4075c8, which added `still_on_disk`; the bulk
path was not part of that change.

**2. Attachments over about 3 MB killed the socket.** `fs.upload` accepts
25 MB. The socket carrying it accepted 4 MiB, because
`web.WebSocketResponse(heartbeat=30)` left `max_msg_size` at aiohttp's default.
An upload rides as base64 inside one JSON-RPC frame and base64 costs four bytes
for every three, so the real ceiling was about 2.99 MB of file -- enforced by
closing the connection, before `fs.upload` was ever called. The reader saw a
reconnect rather than a reason, and every other in-flight call on that socket
died with it. The advertised 25 MB limit was unreachable, and so was the
message that states it. The knowledge base feeds the same method, and manuals
and reports clear 3 MB far more often than pasted screenshots do.

The upload limit moves to `raven/rpc/files.py` beside `MAX_VIEW_BYTES`, and
`frame_ceiling_for_upload` derives the transport ceiling from it, so raising one
can no longer leave the other behind. The page also refuses an oversized file
itself: past the ceiling the socket still closes, so the server's own refusal
cannot reach a client that sent one.

**Checked and deliberately not changed.**

- The page carries its own copy of the 25 MB number. No limit rides the wire
  today, and publishing one through `system.hello` is a schema change with
  generated-client churn on both front ends, which this fix does not need. The
  Python constant is named as the source in the comment beside the copy.
- A client that is not the page -- a script, a future surface -- can still cross
  the frame ceiling and lose the socket, because the guard is in the page. The
  transport cannot answer a frame it refused to read; the fix for that is an
  upload route that does not ride the socket at all, which is a larger change
  than this bug warrants.
- A file one byte over the limit is refused by a message that rounds both
  numbers to `25.0 MB`. Showing exact byte counts reads worse for every other
  size, and the server refuses that file too.
- The composer still encodes before it refuses, unlike the knowledge path. Its
  caller in the React island holds the `File`, so moving the check there is an
  island change; the files it stages are screenshots, where the encode is cheap.
- `MAX_VIEW_BYTES` is unchanged. It bounds what the viewer reads back out and
  travels over HTTP, not over the socket.
- `<agent home>/uploads` has no TTL, no count cap and no reclaim path, and this
  change raises what a single upload can put there from about 3 MB to 25 MB.
  Deferred rather than solved here: a reclaim policy is a decision about that
  directory's whole lifetime, and the ceiling it would be sized against is the
  one this MR is what makes reachable. Named so that it reads as a deferral
  rather than as an oversight.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Gates, each run as its own target and its exit code read directly rather than
through a pipe:

- `make lint-python` -- 0. First run reported `raven/rpc/files.py` and
  `tests/test_rpc_transport.py` would be reformatted; `ruff format` applied and
  the rerun is clean.
- `make lint-imports` -- 0
- `make lint-ui` -- 0 (`gen:check` in sync, `type-check` clean)
- `make test-ui` -- 0. 103 files, 1671 tests, 0 failed.
- `make build-ui` -- 0. `dist/index.html` rebuilt.
- `make test-python` -- 0. 21341 passed, 55 skipped, 0 failed, in 389s. No
  failures to reconcile against a clean `origin/main`, because there were none.
- `make check-commits`, `make check-source-language`, `make check-large-files`
  -- all 0.

Mutation tests, each reverted afterwards with `git status` confirmed clean:

- Removing `max_msg_size` from `WebSocketResponse` fails
  `test_the_socket_carries_a_frame_larger_than_aiohttp_would_allow_by_default`
  and nothing else.
- Making `frame_ceiling_for_upload` return the limit without the base64
  expansion fails
  `test_the_frame_ceiling_can_carry_the_largest_upload_the_method_accepts` and
  nothing else.
- Restoring the bare `await` in `deleteAll` fails two of the four new bulk
  cases: `keeps the rows whose files survived their removal` and `keeps a row
  when the server does not carry the field`.

Verified against a running gateway, not only in tests. Uploading through
`/rpc` with the session token, before the change: 1 MB and 2.5 MB stored, 3 MB
and 3.2 MB and 5 MB lost the connection. After: all five stored, and 24 MB
stored. The page's own guard was then exercised out of the built
`dist/index.html` rather than the source: a file of exactly the limit passes,
one byte more is refused, and the refusal names the file and both sizes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

An authenticated client can now make the gateway buffer a frame of about
33 MiB rather than 4 MiB. The socket is reachable only with the session cookie
or the token, and the method it feeds already accepted 25 MB, so this raises
what an authorised caller can spend, not who can spend it.

No contract changes: no method gains or loses a field, and no response shape
moves. A page built before this change works against a gateway carrying it, and
the reverse holds with the old ceiling. Rollback is a revert of either commit
independently.

## Related Issues

N/A
2026-09-11 10:37:15 +00:00
Handsome-wzw
e3c876339a fix(*): stop the page hiding a refused delete and a refused upload
## Summary

Two bugs in the served page, both of the same shape: a refusal the surface did
not report as one. They are one MR because they were found in one pass over the
page's write paths and each is a few lines; the commits are separate and can be
read independently.

**1. Bulk delete dropped rows the server had kept.** `DS.sessions.deleteAll`
awaited `session.delete` and counted every call that did not throw as a
removal. A refusal does not throw: the method answers
`{deleted: null, still_on_disk: true}` when the filesystem refused to remove
the file, and that is a resolved promise. The rail dropped the row while the
transcript stayed on disk, so the session came back on the next reload -- the
one thing the note above that function says it must never do. The bulk path now
reads the answer with the predicate the single delete already uses. The
single-delete half was fixed in 3d4075c8, which added `still_on_disk`; the bulk
path was not part of that change.

**2. Attachments over about 3 MB killed the socket.** `fs.upload` accepts
25 MB. The socket carrying it accepted 4 MiB, because
`web.WebSocketResponse(heartbeat=30)` left `max_msg_size` at aiohttp's default.
An upload rides as base64 inside one JSON-RPC frame and base64 costs four bytes
for every three, so the real ceiling was about 2.99 MB of file -- enforced by
closing the connection, before `fs.upload` was ever called. The reader saw a
reconnect rather than a reason, and every other in-flight call on that socket
died with it. The advertised 25 MB limit was unreachable, and so was the
message that states it. The knowledge base feeds the same method, and manuals
and reports clear 3 MB far more often than pasted screenshots do.

The upload limit moves to `raven/rpc/files.py` beside `MAX_VIEW_BYTES`, and
`frame_ceiling_for_upload` derives the transport ceiling from it, so raising one
can no longer leave the other behind. The page also refuses an oversized file
itself: past the ceiling the socket still closes, so the server's own refusal
cannot reach a client that sent one.

**Checked and deliberately not changed.**

- The page carries its own copy of the 25 MB number. No limit rides the wire
  today, and publishing one through `system.hello` is a schema change with
  generated-client churn on both front ends, which this fix does not need. The
  Python constant is named as the source in the comment beside the copy.
- A client that is not the page -- a script, a future surface -- can still cross
  the frame ceiling and lose the socket, because the guard is in the page. The
  transport cannot answer a frame it refused to read; the fix for that is an
  upload route that does not ride the socket at all, which is a larger change
  than this bug warrants.
- A file one byte over the limit is refused by a message that rounds both
  numbers to `25.0 MB`. Showing exact byte counts reads worse for every other
  size, and the server refuses that file too.
- The composer still encodes before it refuses, unlike the knowledge path. Its
  caller in the React island holds the `File`, so moving the check there is an
  island change; the files it stages are screenshots, where the encode is cheap.
- `MAX_VIEW_BYTES` is unchanged. It bounds what the viewer reads back out and
  travels over HTTP, not over the socket.
- `<agent home>/uploads` has no TTL, no count cap and no reclaim path, and this
  change raises what a single upload can put there from about 3 MB to 25 MB.
  Deferred rather than solved here: a reclaim policy is a decision about that
  directory's whole lifetime, and the ceiling it would be sized against is the
  one this MR is what makes reachable. Named so that it reads as a deferral
  rather than as an oversight.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Gates, each run as its own target and its exit code read directly rather than
through a pipe:

- `make lint-python` -- 0. First run reported `raven/rpc/files.py` and
  `tests/test_rpc_transport.py` would be reformatted; `ruff format` applied and
  the rerun is clean.
- `make lint-imports` -- 0
- `make lint-ui` -- 0 (`gen:check` in sync, `type-check` clean)
- `make test-ui` -- 0. 103 files, 1671 tests, 0 failed.
- `make build-ui` -- 0. `dist/index.html` rebuilt.
- `make test-python` -- 0. 21341 passed, 55 skipped, 0 failed, in 389s. No
  failures to reconcile against a clean `origin/main`, because there were none.
- `make check-commits`, `make check-source-language`, `make check-large-files`
  -- all 0.

Mutation tests, each reverted afterwards with `git status` confirmed clean:

- Removing `max_msg_size` from `WebSocketResponse` fails
  `test_the_socket_carries_a_frame_larger_than_aiohttp_would_allow_by_default`
  and nothing else.
- Making `frame_ceiling_for_upload` return the limit without the base64
  expansion fails
  `test_the_frame_ceiling_can_carry_the_largest_upload_the_method_accepts` and
  nothing else.
- Restoring the bare `await` in `deleteAll` fails two of the four new bulk
  cases: `keeps the rows whose files survived their removal` and `keeps a row
  when the server does not carry the field`.

Verified against a running gateway, not only in tests. Uploading through
`/rpc` with the session token, before the change: 1 MB and 2.5 MB stored, 3 MB
and 3.2 MB and 5 MB lost the connection. After: all five stored, and 24 MB
stored. The page's own guard was then exercised out of the built
`dist/index.html` rather than the source: a file of exactly the limit passes,
one byte more is refused, and the refusal names the file and both sizes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

An authenticated client can now make the gateway buffer a frame of about
33 MiB rather than 4 MiB. The socket is reachable only with the session cookie
or the token, and the method it feeds already accepted 25 MB, so this raises
what an authorised caller can spend, not who can spend it.

No contract changes: no method gains or loses a field, and no response shape
moves. A page built before this change works against a gateway carrying it, and
the reverse holds with the old ceiling. Rollback is a revert of either commit
independently.

## Related Issues

N/A
2026-09-11 10:37:14 +00:00
0xKT
42e6fbdacb feat(agent): let image_generate take a list of prompts
## Summary

`image_generate` could only be handed one prompt, so a caller that needs several
pictures pays for them one call at a time. In a real deck run, eight backdrops
were asked for one after another and the nine calls took 38.6, 35.2, 43.3, 54.2,
81.6, 85.9, 95.4, 103.6 and 108.0 seconds: 645.8 seconds of pure serial waiting
for work that has no order between its pieces.

The deck engine had already answered exactly this for its own generator
(`ppt_generate_image` takes `prompts` and gathers them). This puts that shape on
the host tool every caller shares, rather than leaving it behind a plugin.

Key decisions:

- `prompts[]` carries a full spec per picture: its own `prompt`, and optional
  `filename`, `images`, `aspect_ratio` and `quality`. Whatever an item leaves
  out falls back to the call's top-level value, so a batch that varies only the
  prompt stays short. `model` and `output_dir` stay shared by the whole call --
  one call speaks to one deployment and writes into one directory.
- A top-level `prompt` given alongside `prompts` is generated as the first
  picture, so the two forms compose instead of conflicting.
- One prompt, in whichever form it arrives, is answered in exactly the shape it
  always was, key for key. Existing callers see no change.
- The batch holds at four in flight, the ceiling the deck engine settled on: a
  batch is waited on as a whole, so the cap only has to keep the slowest picture
  from arriving last, and a wider fan-out against one key buys 429s.
- Failures are per picture. The batch is gathered with `return_exceptions=True`,
  so a directory refused for one of eight does not throw the other seven away;
  the answer keeps every path under `paths` for the `message` tool's `media`
  field and says on each result what was made, or why it was not.
- An optional `filename` is sanitised to a bare stem before it names a file. The
  name comes from the model, so a directory inside it is dropped rather than
  honoured, and `output_dir` stays the only thing that decides where a picture
  lands.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

All run from the rebased branch tip, after `uv sync --frozen --all-extras`
(exit 0), with `set -o pipefail` and the exit code checked each time.

- `uv run --frozen pytest tests/test_media_gen_tool.py -q` -- 84 passed, exit 0.
  The file gained 211 lines of tests covering: the single-prompt reply being
  unchanged key for key, top-level values as batch defaults, per-item overrides,
  the top-level prompt landing first, a missing or blank prompt being refused,
  one picture failing while the rest come back, an exception inside one picture
  not taking the batch, concurrency capped at four, and `filename` refusing a
  path traversal.
- `uv run --frozen pytest` over the 15 test files that touch `image_generate` /
  `media_gen` / `ImageGenerateTool` (`test_media_gen_tool`,
  `test_ppt_engine_generate_image_tool`, `test_ppt_engine_plugin`,
  `test_ppt_engine_assembly`, `test_agent_loop_web_tools`, `test_tool_registry_timeout`,
  `test_tool_registry_turn_scope`, `test_tool_capabilities`, `test_tool_search`,
  `test_core_plugin_stack`, `test_plugin_contribution_ledger`, `test_rpc_console`,
  `test_cli_doctor_commands`, `test_provider_auth_method`,
  `test_agents_design_launcher`) -- 592 passed, exit 0.
- `make lint-python` -- exit 0. Ruff check: all checks passed. Ruff format:
  1921 files already formatted.
- `COMMIT_RANGE=origin/main..HEAD make check-source-language` -- exit 0.
- `COMMIT_RANGE=origin/main..HEAD make check-commits` -- exit 0.
- `COMMIT_RANGE=origin/main..HEAD make check-large-files` -- exit 0.

Not verified here: a live batch against the OpenRouter endpoint. The tests drive
the HTTP layer through a stub.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

User-visible behaviour changes:

- The tool schema no longer marks `prompt` as required, because either `prompt`
  or `prompts` will do. A call carrying neither is refused at runtime with
  `nothing to generate: give prompt, or prompts=[{"prompt": ...}, ...]` instead
  of being rejected by the schema, so the model reads a sentence rather than a
  validation error.
- `filename` is new at the top level and per item. A name that carries a
  directory is reduced to its stem; it is not honoured and not an error.
- The tool description now tells the caller to plan every picture and ask for
  them together, so an agent that used to emit N calls will tend to emit one.
  That is the point of the change, but it does shift how the tool gets called.
- Every reply for a single prompt keeps its old shape, so nothing that reads
  `paths` or `success` today needs to change.

Rollback: the change is one commit touching `raven/agent/tools/media_gen.py` and
`tests/test_media_gen_tool.py`. Reverting it restores the single-prompt tool
exactly; no state, config, schema or stored data is carried forward.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 10:08:38 +00:00
Tchen
1f9505e839 feat(agent): let image_generate take a list of prompts
## Summary

`image_generate` could only be handed one prompt, so a caller that needs several
pictures pays for them one call at a time. In a real deck run, eight backdrops
were asked for one after another and the nine calls took 38.6, 35.2, 43.3, 54.2,
81.6, 85.9, 95.4, 103.6 and 108.0 seconds: 645.8 seconds of pure serial waiting
for work that has no order between its pieces.

The deck engine had already answered exactly this for its own generator
(`ppt_generate_image` takes `prompts` and gathers them). This puts that shape on
the host tool every caller shares, rather than leaving it behind a plugin.

Key decisions:

- `prompts[]` carries a full spec per picture: its own `prompt`, and optional
  `filename`, `images`, `aspect_ratio` and `quality`. Whatever an item leaves
  out falls back to the call's top-level value, so a batch that varies only the
  prompt stays short. `model` and `output_dir` stay shared by the whole call --
  one call speaks to one deployment and writes into one directory.
- A top-level `prompt` given alongside `prompts` is generated as the first
  picture, so the two forms compose instead of conflicting.
- One prompt, in whichever form it arrives, is answered in exactly the shape it
  always was, key for key. Existing callers see no change.
- The batch holds at four in flight, the ceiling the deck engine settled on: a
  batch is waited on as a whole, so the cap only has to keep the slowest picture
  from arriving last, and a wider fan-out against one key buys 429s.
- Failures are per picture. The batch is gathered with `return_exceptions=True`,
  so a directory refused for one of eight does not throw the other seven away;
  the answer keeps every path under `paths` for the `message` tool's `media`
  field and says on each result what was made, or why it was not.
- An optional `filename` is sanitised to a bare stem before it names a file. The
  name comes from the model, so a directory inside it is dropped rather than
  honoured, and `output_dir` stays the only thing that decides where a picture
  lands.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

All run from the rebased branch tip, after `uv sync --frozen --all-extras`
(exit 0), with `set -o pipefail` and the exit code checked each time.

- `uv run --frozen pytest tests/test_media_gen_tool.py -q` -- 84 passed, exit 0.
  The file gained 211 lines of tests covering: the single-prompt reply being
  unchanged key for key, top-level values as batch defaults, per-item overrides,
  the top-level prompt landing first, a missing or blank prompt being refused,
  one picture failing while the rest come back, an exception inside one picture
  not taking the batch, concurrency capped at four, and `filename` refusing a
  path traversal.
- `uv run --frozen pytest` over the 15 test files that touch `image_generate` /
  `media_gen` / `ImageGenerateTool` (`test_media_gen_tool`,
  `test_ppt_engine_generate_image_tool`, `test_ppt_engine_plugin`,
  `test_ppt_engine_assembly`, `test_agent_loop_web_tools`, `test_tool_registry_timeout`,
  `test_tool_registry_turn_scope`, `test_tool_capabilities`, `test_tool_search`,
  `test_core_plugin_stack`, `test_plugin_contribution_ledger`, `test_rpc_console`,
  `test_cli_doctor_commands`, `test_provider_auth_method`,
  `test_agents_design_launcher`) -- 592 passed, exit 0.
- `make lint-python` -- exit 0. Ruff check: all checks passed. Ruff format:
  1921 files already formatted.
- `COMMIT_RANGE=origin/main..HEAD make check-source-language` -- exit 0.
- `COMMIT_RANGE=origin/main..HEAD make check-commits` -- exit 0.
- `COMMIT_RANGE=origin/main..HEAD make check-large-files` -- exit 0.

Not verified here: a live batch against the OpenRouter endpoint. The tests drive
the HTTP layer through a stub.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

User-visible behaviour changes:

- The tool schema no longer marks `prompt` as required, because either `prompt`
  or `prompts` will do. A call carrying neither is refused at runtime with
  `nothing to generate: give prompt, or prompts=[{"prompt": ...}, ...]` instead
  of being rejected by the schema, so the model reads a sentence rather than a
  validation error.
- `filename` is new at the top level and per item. A name that carries a
  directory is reduced to its stem; it is not honoured and not an error.
- The tool description now tells the caller to plan every picture and ask for
  them together, so an agent that used to emit N calls will tend to emit one.
  That is the point of the change, but it does shift how the tool gets called.
- Every reply for a single prompt keeps its old shape, so nothing that reads
  `paths` or `success` today needs to change.

Rollback: the change is one commit touching `raven/agent/tools/media_gen.py` and
`tests/test_media_gen_tool.py`. Reverting it restores the single-prompt tool
exactly; no state, config, schema or stored data is carried forward.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 10:08:37 +00:00
gloryfromca
ba82cee1d5 chore(*): a gate against importing what nobody declared
## Summary

`pyproject.toml` already carried three hand-written pins against a single
failure mode:

```
# transitive dep of litellm -- pinning it directly so the everos
# extractor keeps working if upstream ever drops it.

# transitive dep (via lancedb) -- pinning it directly so routing keeps
# working if upstream ever drops it.

# Reaches us anyway via everos[multimodal] -> everalgo-parser, but that is a
# three-hop transitive path -- pinning it directly so reading an image keeps
# working if any link in that chain drops it.
```

The mode is real and the team was already defending against it one package at a
time. This wires up `deptry`, which is that habit automated, and it found seven
more the manual pass had missed.

**What it catches.** A direct import of a package that only reaches the install
through someone else's dependency table. The day that package drops it, the
install stops carrying it and raven fails at import time, far from the cause.

**This is insurance, not iteration speed.** It adds a gate; it makes nothing
faster. The argument for it is that the failure it prevents is one this
repository has already decided is worth defending against by hand.

### What the 45 reports actually were

Four real, three correct-as-written:

| Package | Sites | Disposition |
|---|---|---|
| `aiohttp` | 15 (3 module-level) | **Declared.** Imported at module level in `rpc/control.py` and both rpc transports, yet it appeared only under `[tool.uv] constraint-dependencies` -- which constrains a version without declaring a dependency |
| `click` | 2 | **Declared.** Used directly, arrives through typer |
| `pyarrow` | 1 | **Declared.** Used directly, arrives through lancedb |
| `pydantic_core` | 1 | **No dependency added.** `pydantic.fields` re-exports the same `PydanticUndefined` object, verified `is`-identical, so the import moves instead |
| `raven_everos` | 24 | **Ignored, reason recorded.** A separate distribution discovered through the `raven.plugins` entry-point group; every import is inside a function for that reason, and declaring it would turn an optional plugin into a hard runtime requirement |
| `requests` | 1 | **Ignored, reason recorded.** Probed under `try: import ... except ImportError` in `channels/errors.py` so its connection errors join the transient set when the host happens to carry it |
| `cryptography` | 1 | **Ignored, reason recorded.** Fallback AES path in the weixin adapter under `with suppress(ImportError)` |

The `aiohttp` case is the one worth a reviewer's attention: a
`constraint-dependencies` entry looks like a declaration in a diff and is not
one. Three module-level imports were relying on something else to bring it.

### Only DEP003 is a gate

`DEP001` and `DEP002` are off. Both are dominated here by the gap between a
distribution name and its import name (`matrix-nio` -> `nio`, `qq-botpy` ->
`botpy`, `python-socketio` -> `socketio`), so they report the same fact twice as
noise. `DEP002` would additionally report the three deliberate pins quoted above
as unused dependencies, which is the opposite of their intent.

Wired as `make lint-deps`, run from `lint` and from CI beside the import
contracts. `lint-imports` was missing from `.PHONY`; both are added.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Other: a dependency gate, three dependency declarations, and one import moved.

## Verification

```
$ uv run --frozen --python 3.12 deptry raven
Success! No dependency issues found.

$ make lint-deps
Success! No dependency issues found.

$ uv run --frozen --python 3.12 --extra dev ruff check --no-cache \
    raven evolver agents plugins-dist tests scripts
All checks passed!

$ uv run --frozen --python 3.12 lint-imports
Contracts: 9 kept, 0 broken.

$ uv run --frozen --python 3.12 pytest tests/ -q -k "update_providers or provider_commands or knowledge or weixin or channels_errors or confirm_injection or cli_dispatch"
496 passed

$ make check-commits COMMIT_RANGE=origin/main..HEAD
exit 0
```

Checked in the failing direction, since a gate never seen red is not known to
work. The first probe was a mistake worth recording: importing `litellm` changed
nothing, because litellm is a declared dependency and the gate was right to stay
quiet. A genuinely undeclared transitive package does trip it:

```
$ echo 'import certifi' >> raven/utils/text.py   # arrives via httpx, declared nowhere
raven/utils/text.py:81:8: DEP003 'certifi' imported but it is a transitive dependency
Found 1 dependency issue.
```

The probe was reverted.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing surface changes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Three packages move into `[project.dependencies]`. All three are already present
in every install today -- they arrive through typer, lancedb and the aiohttp
constraint -- so this records what is already true rather than adding weight.
The one code change is an import moved to a re-export of the identical object,
verified with `is`.

The risk the gate itself carries is a false negative on a package that is both
optional and genuinely needed; the three ignores are the place that could go
wrong, which is why each carries its reason in the config rather than a bare
name.

Rollback is reverting the one commit. Loosening instead is removing `lint-deps`
from the `lint` target, which leaves the declarations in place.

## Related Issues

N/A
2026-09-11 09:41:53 +00:00
0xKT
7a0b5dd311 feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary

An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.

- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.

What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.

## Review round 1 (2026-09-09, on !600, superseded by this MR)

- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.

## Review round 2 (2026-09-10)

- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.

## Review round 3 (2026-09-10)

- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.

## Review round 4 (2026-09-10)

- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
  tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q      # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
  tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
  tests/test_agents_oncall_flow_escalation_tools.py -q                                       # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents        # clean
uv run lint-imports                                                                          # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit   # in sync, clean
```

New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).

Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)

## Risk

- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.

A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.

## Related

Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
silverLXT
83aac42909 feat(acp): a prompt is held open while the agent sleeps on an armed wake, and takes a steer
## Summary

An on-call agent submits a job, arms a wake and ends its turn saying so; the wake runs a later turn on the session on its own clock. Over ACP that turn's `end_turn` read as "finished" to the caller: a DAG judging its node spent every continuation on "still watching, wake at 13:13" while the job ran fine (ar-r4, 2026-09-08), and the main agent re-dispatched the work by hand. ACP has no stop reason for "waiting". Replaces the closed !573: **no DAG code changes**, no new node status. The node is `running` because a turn is genuinely in flight.

- **The signal.** The loop-hooks paper gains one key the ACP layer reads for itself: `acp_meta["raven.holdTurn"]` (`{"untilMs", "why"}`). oncall-flow files it at the send fire when the window's campaign is claimed, not concluded, and has one pending wake -- read the same way the owner's answer pulls that wake forward. No campaign, a wake that stood down, or a concluded campaign files nothing and the turn ends as before.
- **The hold** (`raven/acp/methods.py`). A turn that ended `end_turn` with the key filed is not answered. One `tool_call` stands for the wait (its title names the campaign and the next look), with a `tool_call_update` heartbeat every `HOLD_HEARTBEAT_S` (60 s) so a client that clocks liveness by frames sees the agent alive while its model sleeps. The turn slot is reopened with no id to correlate on, so whichever turn ends next on the session -- the wake's, a steer's -- settles it, and the handler decides again from that turn's stash. A wake that arms again keeps the hold; a turn ending with nothing pending answers `end_turn`; a cancel answers `cancelled` as before. The key itself is not passed to the client.
- **Steer.** `_raven/session/steer` is served and advertised under `agentCapabilities._meta` as `raven.steer` (the names raven's own client already uses). Accepted while a prompt is open on the session, held or running: the text is shown on the wire as the person's own line, then sent with `turn.send`'s new `busy: "inject"`. Answers `injected` / `no_turn`.
- **Wire.** `turn.send` gains the optional `busy` param (`const "inject"`): an ordinary send on an idle lane, `BusyPolicy.INJECT` into the running turn otherwise -- the merge a channel's mid-turn message already gets (`gateway_commands.py`). No slot is bound for an injected message; the running turn keeps the lane's. `openrpc.json` and both generated clients moved together.

What raven's client side already does, unchanged: the DAG stall watcher resets on any frame (the heartbeat keeps it quiet, and a stopped heartbeat raises the existing stall notice after ten minutes); a sub-agent process that dies fails the request and the node takes its ordinary path; the pool's `_responsive` treats a prompt in flight as busy.

## Review round 1 (2026-09-09, on !600, superseded by this MR)

- **`busy: inject` reads the scheduler's lane state too.** `is_turn_active` knows only the turns `turn.send` started; an armed wake is submitted to the scheduler directly (`make_on_session_wake`) and never binds that map, so a steer during a running wake queued a second turn and answered `injected`. `turn.send` now asks `Scheduler.has_inflight(lane)` as well, and merges (`BusyPolicy.INJECT`) when either record says the lane is running. Pinned: `test_busy_inject_merges_into_a_wake_the_real_scheduler_is_running` -- the reviewer's reproduction on the real `Scheduler`, a turn submitted the way `make_on_session_wake` submits one; with the lane check bypassed it reads `pending=1, inject_mailbox=0`, with it `(1, 0)` in the mailbox and the running turn drains the text.
- Branch renamed to snake_case and both commits carry the `Co-authored-by: Claude (<model-id>)` trailer, per AGENTS.md §2–3; that is why this MR replaces !600 rather than continuing it.

## Review round 2 (2026-09-10)

- **A wake re-armed during its own run survives the writeback.** `CronService._writeback_locked` reloaded the store after the run and, for a one-shot with `delete_after_run`, overwrote the id with the finished run's state and removed it -- but the turn the wake ran had just called `schedule_wake` under the same key, so the id now named the *next* look. It vanished, and the held prompt had nothing left to settle it. A stored job under the fired id whose `at` schedule the run never had is treated as the new wake it is: claim cleared, run recorded, schedule kept, not deleted. Pinned on the real service: `test_a_wake_re_armed_during_its_own_run_survives_the_writeback` (reads `[]` without the fix, the reviewer's state) and the unchanged one-shot deletion beside it. Cron, proactive, sentinel and on-call suites: 968 passed.

## Review round 3 (2026-09-10)

- **An undrained inject stays visible and cancellable.** `busy: inject` dropped the scheduler's handle; when the host turn ended past its last drain, the spine's fallback ran the text as a turn nothing owned -- `turn.cancel` a no-op, no `message.start`. The handle is now kept per lane under an id minted for the text; `turn.cancel` / `session.interrupt` reach it when the active slot is empty (mailbox, queued fallback, or running); and when the fallback turn starts, the RPC sink (`on_turn_start`, wired from bootstrap) binds the lane to it and opens it on the wire with the person's words. `turn.send` answers the inject's own id. Pinned through `build_rpc_spine` with the reviewer's shape (a loop that drains once, an inject after it): `test_an_undrained_inject_runs_as_a_turn_the_surface_can_see_and_cancel` and `test_a_pending_inject_can_be_cancelled_before_it_merges_or_falls_back`; both read the reviewer's state (`turn_ids` empty, cancel finding nothing) with the fix bypassed.
- **Line ceiling.** The contract comment is five lines; `raven/contracts` is 2,519 against 2,520, one line of headroom on the current `main` (2,514).
- Non-blocking items taken: `SchedulerDrainingError` on the inject path exits as `turn_failed`; the title is announced before the hold; the steer constants sit below the cancel constant's docstring; three edges pinned (a held prompt settled by a wake turn's uncorrelated ending through `send_frame`, the hold key never in the client's `_meta`, `raven.toolName` on the wait row).
- Focused suite (reviewer's list plus the touched files): 948 passed on Python 3.12.

## Review round 4 (2026-09-10)

- **Every pending inject is kept, per lane and per id.** One replaceable record per lane kept only the newest steer; two steers after the host's last drain fall back in turn, and the first started unowned while a cancel took out the second. The record is now `lane -> {inject id: (handle, text)}` in arrival order: each fallback promotes its own handle, and a cancel with no active turn reaches the oldest pending one -- the one the spine runs next. Pinned: `test_two_undrained_injects_each_get_their_own_turn_promoted_and_cancelled` (both promoted with their own ids and `message.start`, cancelled one after the other; runner saw all three turns). Focused suite: 949 passed.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests/test_acp_methods.py tests/test_acp_protocol.py tests/test_rpc_turn_send.py \
  tests/test_rpc_schema_match.py tests/test_acp_updates.py tests/test_acp_server.py -q      # 766 passed
uv run pytest tests/test_agents_oncall_flow_hooks.py tests/test_agents_oncall_flow_actions.py \
  tests/test_agents_oncall_flow_budget.py tests/test_agents_oncall_flow_readings.py \
  tests/test_agents_oncall_flow_escalation_tools.py -q                                       # 99 passed
uv run ruff check raven tests agents && uv run ruff format --check raven tests agents        # clean
uv run lint-imports                                                                          # 9 kept, 0 broken
cd ui-tui && npm run lint:rpc && npx tsc --noEmit; cd ui-web && npm run gen:check && npx tsc --noEmit   # in sync, clean
```

New tests: `TestHeldTurn` (held across an armed wake, held again on a re-arm, heartbeat on the wait call, cancel while held, a non-`end_turn` stop is not held; every frame validated against the ACP schema), `TestSteer` (advertised, `no_turn` when idle, a held prompt takes the words as a turn with `busy: inject`, refusals), `busy: inject` on a busy and an idle lane in `test_rpc_turn_send.py`, and the plugin's hold stamping through the real `CronService` grant (filed with a pending wake; not filed without one, after the wake stands down, or once the campaign concluded).

Not done: no end-to-end run of a real on-call wake through a held prompt yet -- unit tests only.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the hold is invisible to a person except as a "waiting for ..." tool row)

## Risk

- [x] Backward compatibility considered: a turn that files no `raven.holdTurn` is answered exactly as before; `busy` is optional and absent from every existing caller; the steer method and capability are additive.
- [x] Rollback path is clear: revert the two commits.

A held prompt holds one `max_concurrent_subagents` slot (default 8) for the wait's duration on the caller's side; raised in config if it bites.

## Related

Replaces !573 (closed). Follows !518 (stall notice) and !520 (unprompted-turn route; a wake that runs under a held prompt no longer takes it for DAG nodes).
2026-09-11 02:37:55 +00:00
0xKT
8eb69f59f2 fix(providers): honour the declared output ceiling, retry differently, record a failed call
## Summary

A model call that returned nothing used to end a turn, and nothing in the record said why.
This branch closes that in five parts, all on the host runtime.

- **The output ceiling is the model's, not ours.** `resolve_max_output_tokens` takes the
  figure the model declares instead of a repo default, reserves it against the id the
  request actually goes out under, and counts cached prompt tokens toward the window so a
  cached prefix stops looking free. One call is given 64k of room and says so on the wire;
  the transcript summary budget goes to 32k. Two small output pins keep deliberate
  headroom, and the commit says why they do not move.
- **A retry has to ask a different question.** An empty completion previously repeated the
  same request and got the same answer. The ladder now descends by **what a rung sends**
  rather than by its label, the transcript summary is no longer asked to think, and the
  empty-completion retry differs from the call it replaces.
- **Time to first byte is bounded.** `llmFirstByteTimeout` applies on the direct adapters
  as well as through the gateway, a bound of zero means off rather than a bound at the
  total, and the guard's deadline is distinguished from a strategy's own `TimeoutError` so
  a fallback answers for the right one.
- **An effort travels as an effort.** `glm` is out of `inferred_protocol`'s Anthropic prefix
  tuple and the hand-written `_THINKING_BUDGETS` table is deleted, so a label reaches the
  vendor as a label rather than as a token count this repo invented. Measured on the wire
  before the change: 42 calls produced 261,980 characters of reasoning against a budget of
  1024 that the route discarded, while our own `max_tokens` went out on every call because
  that transport requires the field.
- **The record says what was asked and what came back.** A failed call is visible in the
  turn and readable in the record; the reasoning count is kept; a failure's headers are
  recorded as the failure's, not as a stream's server; `llm.input` gains a `generation`
  block and the span gains `llm.request.*`, with a test pinning that no credential appears
  in either. `llm.request_bytes` is documented as an estimate instead of claiming to be
  the wire. A frozen turn is stopped rather than nudged once.

## Type

- [x] Fix
- [x] Feature

## Verification

- `uv run --frozen pytest` over the full suite: **20354 passed**, with 6 failures and 20
  errors that are this box lacking `/usr/share/zoneinfo` -- all in `test_cron_tool.py` and
  `test_cli_cron_commands.py`, which this branch does not touch. Reproduced with the
  identical signature on a clean detached `origin/main` worktree, and CI ships tzdata, so
  its `tests` job is the authoritative check. Gated on `set -o pipefail` and
  `${PIPESTATUS[0]}`.
- `make lint-python` exit 0; `lint-imports` 9 kept, 0 broken.
- Contract pins re-derived against current `main` rather than carried over:
  `CONTRACTS_VERSION` **17** (main is 16), the digest from one run of the pinning test, and
  `CONTRACTS_LINE_CEILING` **2_791** against 2771 actual. 34 pinning tests pass.
- The request shape verified on real captured bodies rather than inferred: after the change
  a deck call is `POST /v1/chat/completions` carrying `reasoning: {"effort": ...}` at the
  top level, the configured temperature, and **no** `max_tokens`; before it, the same call
  was `POST /v1/messages` carrying `thinking: {"type": "enabled", "budget_tokens": 1024}`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

The routing change moves every `glm` model from the native Anthropic Messages transport to
LiteLLM chat. That changes which client serves the deck agent and the host default model,
so it is the one item here worth a real run before it is relied on. Measured consequences
on a live deck: no truncations, no length-stops, no empty completions and no retries across
40 calls, and prompt-cache read rates in the same band as the Messages route -- `glm` takes
no `cache_control` breakpoints on either transport because OpenRouter caches it
automatically, so caching is unaffected. A provider that needs an Anthropic-compatible
address can still name `protocol: "anthropic"` explicitly, and a test pins that route.

Rollback is per part: each of the five is its own commit range, and the routing change is a
single line in `raven/providers/protocol.py`.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 02:37:37 +00:00
Tchen
3f1453d5fe fix(providers): honour the declared output ceiling, retry differently, record a failed call
## Summary

A model call that returned nothing used to end a turn, and nothing in the record said why.
This branch closes that in five parts, all on the host runtime.

- **The output ceiling is the model's, not ours.** `resolve_max_output_tokens` takes the
  figure the model declares instead of a repo default, reserves it against the id the
  request actually goes out under, and counts cached prompt tokens toward the window so a
  cached prefix stops looking free. One call is given 64k of room and says so on the wire;
  the transcript summary budget goes to 32k. Two small output pins keep deliberate
  headroom, and the commit says why they do not move.
- **A retry has to ask a different question.** An empty completion previously repeated the
  same request and got the same answer. The ladder now descends by **what a rung sends**
  rather than by its label, the transcript summary is no longer asked to think, and the
  empty-completion retry differs from the call it replaces.
- **Time to first byte is bounded.** `llmFirstByteTimeout` applies on the direct adapters
  as well as through the gateway, a bound of zero means off rather than a bound at the
  total, and the guard's deadline is distinguished from a strategy's own `TimeoutError` so
  a fallback answers for the right one.
- **An effort travels as an effort.** `glm` is out of `inferred_protocol`'s Anthropic prefix
  tuple and the hand-written `_THINKING_BUDGETS` table is deleted, so a label reaches the
  vendor as a label rather than as a token count this repo invented. Measured on the wire
  before the change: 42 calls produced 261,980 characters of reasoning against a budget of
  1024 that the route discarded, while our own `max_tokens` went out on every call because
  that transport requires the field.
- **The record says what was asked and what came back.** A failed call is visible in the
  turn and readable in the record; the reasoning count is kept; a failure's headers are
  recorded as the failure's, not as a stream's server; `llm.input` gains a `generation`
  block and the span gains `llm.request.*`, with a test pinning that no credential appears
  in either. `llm.request_bytes` is documented as an estimate instead of claiming to be
  the wire. A frozen turn is stopped rather than nudged once.

## Type

- [x] Fix
- [x] Feature

## Verification

- `uv run --frozen pytest` over the full suite: **20354 passed**, with 6 failures and 20
  errors that are this box lacking `/usr/share/zoneinfo` -- all in `test_cron_tool.py` and
  `test_cli_cron_commands.py`, which this branch does not touch. Reproduced with the
  identical signature on a clean detached `origin/main` worktree, and CI ships tzdata, so
  its `tests` job is the authoritative check. Gated on `set -o pipefail` and
  `${PIPESTATUS[0]}`.
- `make lint-python` exit 0; `lint-imports` 9 kept, 0 broken.
- Contract pins re-derived against current `main` rather than carried over:
  `CONTRACTS_VERSION` **17** (main is 16), the digest from one run of the pinning test, and
  `CONTRACTS_LINE_CEILING` **2_791** against 2771 actual. 34 pinning tests pass.
- The request shape verified on real captured bodies rather than inferred: after the change
  a deck call is `POST /v1/chat/completions` carrying `reasoning: {"effort": ...}` at the
  top level, the configured temperature, and **no** `max_tokens`; before it, the same call
  was `POST /v1/messages` carrying `thinking: {"type": "enabled", "budget_tokens": 1024}`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

The routing change moves every `glm` model from the native Anthropic Messages transport to
LiteLLM chat. That changes which client serves the deck agent and the host default model,
so it is the one item here worth a real run before it is relied on. Measured consequences
on a live deck: no truncations, no length-stops, no empty completions and no retries across
40 calls, and prompt-cache read rates in the same band as the Messages route -- `glm` takes
no `cache_control` breakpoints on either transport because OpenRouter caches it
automatically, so caching is unaffected. A provider that needs an Anthropic-compatible
address can still name `protocol: "anthropic"` explicitly, and a test pins that route.

Rollback is per part: each of the five is its own commit range, and the routing change is a
single line in `raven/providers/protocol.py`.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 02:37:37 +00:00
0xKT
cfe806a6e0 feat(cli): onboard spends one real search on the web_search key
## Summary

The onboard wizard stored the web_search vendor's key without checking it, so a mistyped or revoked key surfaced as the tool's first failure inside a conversation. Step 5 now spends one real query on the key right after the prompt and before anything is written.

- `WebSearchTool.probe()` runs the same provider request `execute()` uses, asking for one result, and returns `(ok, detail)`. The detail carries vendor and HTTP status only. Exception text is never echoed: SerpApi puts the key in the request URL, and an unusable proxy URL can carry its own credentials, so that branch reports the exception class alone.
- Interactive step 5 loops the key prompt through the probe. On failure the user picks re-enter, retry, or continue anyway. A stored key kept with Enter is probed too, since it may have been revoked. An empty slot is not probed, since leaving web_search off is a choice, not a failure.
- The flag path (`--search-api-key`, `--serper-api-key`) writes first and then probes, warning on failure without dropping the write, because a scripted run has nobody to re-enter the key. Only the active search vendor's key is probed there.
- The page reader is not probed. Jina works with no key, and a keyed reader whose key does not resolve is replaced by Jina at registration. A keyed reader with a wrong key still fails at call time; that is unchanged and out of scope here.
- All wizard strings reuse existing zh catalog entries; README step 5 and the `--search-api-key` help mention the check.

Deliberately not done, for the reviewer: the web settings page writes the same key slot and does not probe; a key supplied only through the vendor's env var is not probed; step 5 does not feed the wizard's end-of-run warnings list. A vendor that answers 200 with zero results passes, since only HTTP status and a Firecrawl refusal count as failure.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests/test_agent_tools_web_providers.py tests/test_cli_onboard_commands.py \
  tests/test_i18n_boundary.py tests/test_agent_loop_web_tools.py    410 passed
make lint-python                                                     clean
make check-source-language                                           exit 0
uv run python scripts/check_commit_messages.py origin/main..HEAD     exit 0
```

Revert-the-fix check: with the onboard change reverted, the five new step 5 tests error out on the missing probe seam. No test reaches the network: an autouse fixture in the onboard tests replaces the probe, and the provider tests patch the HTTP client. `make lint-tui` fails on a stale generated i18n file that is identical on `main`; `make lint-bridge` needs `tsc`, which this box lacks.

Not verified: a live run against a real vendor key.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: onboarding spends one search query per key check against the chosen vendor, and a rejected key adds one prompt. The key still goes only to its own vendor's endpoint, through the configured `tools.web.proxy`. Nothing in config changes shape. Rollback is reverting the commit; no data migration.

## Related Issues

N/A
2026-09-11 02:35:44 +00:00
userName20260323
62aefbd1e5 feat(cli): onboard spends one real search on the web_search key
## Summary

The onboard wizard stored the web_search vendor's key without checking it, so a mistyped or revoked key surfaced as the tool's first failure inside a conversation. Step 5 now spends one real query on the key right after the prompt and before anything is written.

- `WebSearchTool.probe()` runs the same provider request `execute()` uses, asking for one result, and returns `(ok, detail)`. The detail carries vendor and HTTP status only. Exception text is never echoed: SerpApi puts the key in the request URL, and an unusable proxy URL can carry its own credentials, so that branch reports the exception class alone.
- Interactive step 5 loops the key prompt through the probe. On failure the user picks re-enter, retry, or continue anyway. A stored key kept with Enter is probed too, since it may have been revoked. An empty slot is not probed, since leaving web_search off is a choice, not a failure.
- The flag path (`--search-api-key`, `--serper-api-key`) writes first and then probes, warning on failure without dropping the write, because a scripted run has nobody to re-enter the key. Only the active search vendor's key is probed there.
- The page reader is not probed. Jina works with no key, and a keyed reader whose key does not resolve is replaced by Jina at registration. A keyed reader with a wrong key still fails at call time; that is unchanged and out of scope here.
- All wizard strings reuse existing zh catalog entries; README step 5 and the `--search-api-key` help mention the check.

Deliberately not done, for the reviewer: the web settings page writes the same key slot and does not probe; a key supplied only through the vendor's env var is not probed; step 5 does not feed the wizard's end-of-run warnings list. A vendor that answers 200 with zero results passes, since only HTTP status and a Firecrawl refusal count as failure.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests/test_agent_tools_web_providers.py tests/test_cli_onboard_commands.py \
  tests/test_i18n_boundary.py tests/test_agent_loop_web_tools.py    410 passed
make lint-python                                                     clean
make check-source-language                                           exit 0
uv run python scripts/check_commit_messages.py origin/main..HEAD     exit 0
```

Revert-the-fix check: with the onboard change reverted, the five new step 5 tests error out on the missing probe seam. No test reaches the network: an autouse fixture in the onboard tests replaces the probe, and the provider tests patch the HTTP client. `make lint-tui` fails on a stale generated i18n file that is identical on `main`; `make lint-bridge` needs `tsc`, which this box lacks.

Not verified: a live run against a real vendor key.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: onboarding spends one search query per key check against the chosen vendor, and a rejected key adds one prompt. The key still goes only to its own vendor's endpoint, through the configured `tools.web.proxy`. Nothing in config changes shape. Rollback is reverting the commit; no data migration.

## Related Issues

N/A
2026-09-11 02:35:44 +00:00
0xKT
cf9a0a8b5a chore(*): a cyclomatic complexity ceiling that only stops the untestable
## Summary

Ruff carried no complexity rule, so nothing bounded the number of paths through
a function. This adds `C901` with a ceiling chosen to stop the untestable and
nothing else.

**What the number means.** Cyclomatic complexity is the count of linearly
independent paths through a function, which makes it the floor on how many cases
a suite needs to walk them all. A body with a hundred of them has paths nobody
will ever write a case for, and those paths are first executed in production.
That is the failure this rule prevents; it is not a style preference.

Two properties of the metric matter for reading the numbers below. A nested
function's score is counted into its enclosing function as well as reported on
its own, so an outer function with no branches of its own scores one more than
the inner function it holds. And Ruff reports `C901` only when the score is
*greater* than `max-complexity`, so a function at exactly 40 passes a ceiling of
40 and needs no suppression.

**Where 40 comes from.** The measured distribution, not McCabe's recommendation
of 10:

| max-complexity | functions flagged |
|---|---|
| 10 | 253 |
| 25 | 26 |
| 30 | 17 |
| **40** | **9** |
| 50 | 5 |

Ten would flag 253 functions and be switched off within the week. Forty is not a
claim that 39 is fine -- it is where a function stops being complex and starts
being untestable. Tightening later means lowering the number, which is a
reviewed edit to `pyproject.toml`.

**The nine over the ceiling.** Each carries a `noqa: C901` naming its actual
score. Seven are in `raven/`, two in the oncall-flow plugin under `agents/`:

```
100  raven/agent/loop/turn_path.py                             _run_agent_loop
 87  raven/cli/gateway_commands.py                             register
 86  raven/cli/gateway_commands.py                             gateway
 69  agents/raven-oncall/.../oncall_flow/tools/ops.py           execute
 68  raven/cli/gateway_commands.py                             run
 54  agents/raven-oncall/.../oncall_flow/tools/ops.py           execute
 47  raven/agent/loop/turn_path.py                             _process_message
 46  raven/config/loader.py                                    _migrate_config
 41  raven/agent/loop/turn_path.py                             _run_turn
```

A tenth suppression sits on `benchmarks/proactivity_eval/runners/_common/longrun_adapters.py`
(`tick_to`, 41). `benchmarks/` is outside `PYTHON_LINT_TARGETS` and so outside
CI, but inside what the pre-commit Ruff hook sees on a changed file -- marking it
means the gate does not ambush whoever next edits that adapter.

**Nine entries, fewer distinct problems.** In `gateway_commands.py` the three
are nested inside each other at indent 0, 4 and 8: `run` at 68 is the body,
`gateway` adds its own branches on top, and `register` contributes about one. In
`turn_path.py` the three are sibling methods, but `_run_turn` holds ten nested
callbacks (`on_token`, `on_tool`, `on_progress` and so on) which is most of its
41. The one that is a large body by itself is `_run_agent_loop`: 1134 lines with
only two nested functions, so its 100 is its own.

Per-function rather than per-file, on purpose: a whole-file ignore on
`gateway_commands.py` would let a new 80-path function in the same file through,
while these entries do not. The score is in the comment so the debt is visible
where the code is, not parked in a config file.

Nothing has to be refactored to land this. Normal work is undisturbed unless
someone writes a forty-one-path function.

**Why the PLR09xx rules are left out.** They count arguments (230 occurrences
here), branches (125), statements (78) and returns (81). All four are proxies
for what `C901` measures directly, so adding them would report one fact four
times and bury the signal under 230 argument-count warnings.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Other: a lint gate plus the `noqa` entries that let it land green. No runtime
code changes; every production edit is an end-of-line comment.

## Verification

Verified against the lint roster read out of the Makefile
(`PYTHON_LINT_TARGETS = raven evolver agents plugins-dist tests scripts`) rather
than a chosen subset, and then against the whole tree. The first revision of
this MR checked only `raven/` and went red in CI on the two `agents/` functions;
that is what the roster-wide check exists to prevent.

```
$ uv run --frozen --python 3.12 --extra dev ruff check --no-cache \
    raven evolver agents plugins-dist tests scripts
All checks passed!

$ uv run --frozen --python 3.12 --extra dev ruff check --no-cache .
All checks passed!

$ uv run --frozen --python 3.12 --extra dev ruff check --no-cache \
    --extend-select RUF100 raven evolver agents plugins-dist tests scripts
(no unused C901 directive)

$ uv run --frozen --python 3.12 --extra dev ruff format --check \
    raven evolver agents plugins-dist tests scripts
1886 files already formatted

$ uv run --frozen --python 3.12 pytest tests/ -q -k "turn_path or gateway or loader or console or config_precedence"
511 passed, 19 skipped

$ uv run --frozen --python 3.12 lint-imports
Contracts: 9 kept, 0 broken.

$ make check-commits COMMIT_RANGE=origin/main..HEAD
exit 0
```

The 19 skips are pre-existing and environmental: `test_provider_resolution_invariants.py`
skips its gateway-only cases when no gateway is configured.

Checked in the failing direction, since a gate never seen red is not known to
work. A fresh forty-branch function appended to `config/loader.py` is reported,
and the `noqa` on `_migrate_config` in the same file keeps holding rather than
being swallowed by it:

```
raven/config/loader.py:1088:5: C901 `_probe_over_ceiling` is too complex (41 > 40)
Found 1 error.
```

The probe was reverted. Note for the reviewer: reverting that file also reverted
its `noqa`, which the roster-wide check caught immediately.

One entry was removed after review rather than kept: `settings_usage` scores
exactly 40, which a ceiling of 40 does not flag, so its suppression was dead.
`--extend-select RUF100` reports such a directive as unused, and keeping it would
have defeated the documented tightening path -- lowering the number later would
have left that function exempt with nothing to show it.

The production diff is comment-only and can be read as such:

```
-    async def _run_agent_loop(
+    async def _run_agent_loop(  # noqa: C901 (cc 100: pre-existing, above the ceiling)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing surface changes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No behaviour change: every production edit is an end-of-line comment, and the
diff can be verified as such. The risk this carries is a false sense of
coverage, which is why the rationale in `pyproject.toml` states what the ceiling
does not mean. Two limits of the metric are worth a reviewer knowing: it does
not see nesting depth (five sequential `if`s and five nested ones both score 6),
and in Ruff's implementation boolean operators, ternaries and comprehension
conditions do not count at all. It is a filter for the untestable, not a
readability score.

Rollback is reverting the commits. Loosening instead of reverting is a one-line
change to `max-complexity`, which is the intended escape if the gate turns out
to obstruct real work.

## Related Issues

N/A
2026-09-11 02:35:31 +00:00
gloryfromca
4f94ad43cc chore(*): a cyclomatic complexity ceiling that only stops the untestable
## Summary

Ruff carried no complexity rule, so nothing bounded the number of paths through
a function. This adds `C901` with a ceiling chosen to stop the untestable and
nothing else.

**What the number means.** Cyclomatic complexity is the count of linearly
independent paths through a function, which makes it the floor on how many cases
a suite needs to walk them all. A body with a hundred of them has paths nobody
will ever write a case for, and those paths are first executed in production.
That is the failure this rule prevents; it is not a style preference.

Two properties of the metric matter for reading the numbers below. A nested
function's score is counted into its enclosing function as well as reported on
its own, so an outer function with no branches of its own scores one more than
the inner function it holds. And Ruff reports `C901` only when the score is
*greater* than `max-complexity`, so a function at exactly 40 passes a ceiling of
40 and needs no suppression.

**Where 40 comes from.** The measured distribution, not McCabe's recommendation
of 10:

| max-complexity | functions flagged |
|---|---|
| 10 | 253 |
| 25 | 26 |
| 30 | 17 |
| **40** | **9** |
| 50 | 5 |

Ten would flag 253 functions and be switched off within the week. Forty is not a
claim that 39 is fine -- it is where a function stops being complex and starts
being untestable. Tightening later means lowering the number, which is a
reviewed edit to `pyproject.toml`.

**The nine over the ceiling.** Each carries a `noqa: C901` naming its actual
score. Seven are in `raven/`, two in the oncall-flow plugin under `agents/`:

```
100  raven/agent/loop/turn_path.py                             _run_agent_loop
 87  raven/cli/gateway_commands.py                             register
 86  raven/cli/gateway_commands.py                             gateway
 69  agents/raven-oncall/.../oncall_flow/tools/ops.py           execute
 68  raven/cli/gateway_commands.py                             run
 54  agents/raven-oncall/.../oncall_flow/tools/ops.py           execute
 47  raven/agent/loop/turn_path.py                             _process_message
 46  raven/config/loader.py                                    _migrate_config
 41  raven/agent/loop/turn_path.py                             _run_turn
```

A tenth suppression sits on `benchmarks/proactivity_eval/runners/_common/longrun_adapters.py`
(`tick_to`, 41). `benchmarks/` is outside `PYTHON_LINT_TARGETS` and so outside
CI, but inside what the pre-commit Ruff hook sees on a changed file -- marking it
means the gate does not ambush whoever next edits that adapter.

**Nine entries, fewer distinct problems.** In `gateway_commands.py` the three
are nested inside each other at indent 0, 4 and 8: `run` at 68 is the body,
`gateway` adds its own branches on top, and `register` contributes about one. In
`turn_path.py` the three are sibling methods, but `_run_turn` holds ten nested
callbacks (`on_token`, `on_tool`, `on_progress` and so on) which is most of its
41. The one that is a large body by itself is `_run_agent_loop`: 1134 lines with
only two nested functions, so its 100 is its own.

Per-function rather than per-file, on purpose: a whole-file ignore on
`gateway_commands.py` would let a new 80-path function in the same file through,
while these entries do not. The score is in the comment so the debt is visible
where the code is, not parked in a config file.

Nothing has to be refactored to land this. Normal work is undisturbed unless
someone writes a forty-one-path function.

**Why the PLR09xx rules are left out.** They count arguments (230 occurrences
here), branches (125), statements (78) and returns (81). All four are proxies
for what `C901` measures directly, so adding them would report one fact four
times and bury the signal under 230 argument-count warnings.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Other: a lint gate plus the `noqa` entries that let it land green. No runtime
code changes; every production edit is an end-of-line comment.

## Verification

Verified against the lint roster read out of the Makefile
(`PYTHON_LINT_TARGETS = raven evolver agents plugins-dist tests scripts`) rather
than a chosen subset, and then against the whole tree. The first revision of
this MR checked only `raven/` and went red in CI on the two `agents/` functions;
that is what the roster-wide check exists to prevent.

```
$ uv run --frozen --python 3.12 --extra dev ruff check --no-cache \
    raven evolver agents plugins-dist tests scripts
All checks passed!

$ uv run --frozen --python 3.12 --extra dev ruff check --no-cache .
All checks passed!

$ uv run --frozen --python 3.12 --extra dev ruff check --no-cache \
    --extend-select RUF100 raven evolver agents plugins-dist tests scripts
(no unused C901 directive)

$ uv run --frozen --python 3.12 --extra dev ruff format --check \
    raven evolver agents plugins-dist tests scripts
1886 files already formatted

$ uv run --frozen --python 3.12 pytest tests/ -q -k "turn_path or gateway or loader or console or config_precedence"
511 passed, 19 skipped

$ uv run --frozen --python 3.12 lint-imports
Contracts: 9 kept, 0 broken.

$ make check-commits COMMIT_RANGE=origin/main..HEAD
exit 0
```

The 19 skips are pre-existing and environmental: `test_provider_resolution_invariants.py`
skips its gateway-only cases when no gateway is configured.

Checked in the failing direction, since a gate never seen red is not known to
work. A fresh forty-branch function appended to `config/loader.py` is reported,
and the `noqa` on `_migrate_config` in the same file keeps holding rather than
being swallowed by it:

```
raven/config/loader.py:1088:5: C901 `_probe_over_ceiling` is too complex (41 > 40)
Found 1 error.
```

The probe was reverted. Note for the reviewer: reverting that file also reverted
its `noqa`, which the roster-wide check caught immediately.

One entry was removed after review rather than kept: `settings_usage` scores
exactly 40, which a ceiling of 40 does not flag, so its suppression was dead.
`--extend-select RUF100` reports such a directive as unused, and keeping it would
have defeated the documented tightening path -- lowering the number later would
have left that function exempt with nothing to show it.

The production diff is comment-only and can be read as such:

```
-    async def _run_agent_loop(
+    async def _run_agent_loop(  # noqa: C901 (cc 100: pre-existing, above the ceiling)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing surface changes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No behaviour change: every production edit is an end-of-line comment, and the
diff can be verified as such. The risk this carries is a false sense of
coverage, which is why the rationale in `pyproject.toml` states what the ceiling
does not mean. Two limits of the metric are worth a reviewer knowing: it does
not see nesting depth (five sequential `if`s and five nested ones both score 6),
and in Ruff's implementation boolean operators, ternaries and comprehension
conditions do not count at all. It is a filter for the untestable, not a
readability score.

Rollback is reverting the commits. Loosening instead of reverting is a one-line
change to `max-complexity`, which is the intended escape if the gate turns out
to obstruct real work.

## Related Issues

N/A
2026-09-11 02:35:31 +00:00
0xKT
02ea26aa6f refactor(permissions): the shell command policy seats where it is enforced
## Summary

`raven.permissions` imported `raven.agent.tools.shell_policy` for
`CommandDecision` and `ShellCommandPolicy`, while the agent side imports
`raven.permissions` from the loop, the turn path and the `raven_loop` backend.
Four edges down, one edge back up, and a cycle between two top-level packages.

The one edge back was the misplaced one. `shell_policy` answers whether a
command may run and whether it needs approval -- which is what the permissions
package is for. It sat under `agent/tools` because that is where the shell tool
that consumes it lives, not because the policy belongs to the tool.

Moving it is unusually cheap: its 1262 lines import `re`, `shlex`,
`contextvars`, `dataclasses`, `enum` and `pathlib` and nothing from raven, so
the module arrives in `permissions` carrying no dependency of its own. Nothing
new enters that package's dependency set.

Its three consumers are repointed rather than reading through a shim at the old
path: the shell tool, the ACP server's approval-family wiring, and the
permissions rules that started this. `agent -> permissions` and
`acp -> permissions` are both directions that already existed.

This is the third instance of one shape in this tree: a capability with no
dependencies of its own, seated inside one of its consumers, which a lower layer
then has to reach up for. The vocabulary split in MR 610 was the second.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

```
$ uv run --frozen --python 3.12 lint-imports
Contracts: 9 kept, 0 broken.

$ uv run --frozen --python 3.12 pytest tests/test_shell_approval.py \
    tests/test_shell_comments.py tests/test_shell_policy_reasons.py \
    tests/test_permissions_gate.py tests/test_acp_server.py \
    tests/test_agents_ppt_launcher.py -q
407 passed in 4.65s

$ uv run --frozen --python 3.12 pytest tests/ -q -k "shell or permission or policy"
746 passed, 8 skipped, 20 warnings in 23.73s

$ uv run --frozen --python 3.12 --extra dev ruff check raven/permissions/ \
    raven/acp/server.py raven/agent/tools/shell.py
All checks passed!

$ make check-commits COMMIT_RANGE=origin/main..HEAD
exit 0
```

The eight skips in the wider sweep are pre-existing and environmental: optional
extras that are not installed in this checkout (`wecom_aibot_sdk`, the render
extra). None of them exercise shell policy.

Every consumer of the module was enumerated before the move -- three in `raven/`
and six test files -- and all nine are repointed in this commit, so no import of
the old path remains anywhere in the tree.

Effect on the dependency graph:

```
permissions -> agent edges     1  ->  0
mutually importing pairs      14  ->  13
packages in an import cycle   22  ->  22   (unchanged)
```

The package count does not move. Unlike the config vocabulary split, this pair
was not the only thing holding any other package in the component -- worth
recording, because it is the case where a mutual pair disappears and the
component does not shrink.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing surface changes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

This module decides whether a shell command runs, so the security-relevant
question is whether the decision changed: it did not. The file moves unedited --
git records it as a rename -- and the matchers, the approval families and the
policy class are byte-identical. `tests/test_shell_approval.py`,
`tests/test_shell_comments.py` and `tests/test_permissions_gate.py` cover the
deny and approval paths and pass unchanged apart from the import line.

No public surface is affected: `shell_policy` is internal to raven and its old
path had no consumers outside this repository. Rollback is reverting the one
commit, which git will apply as the reverse rename.

## Related Issues

N/A
2026-09-11 02:35:05 +00:00
gloryfromca
c01af5faa3 refactor(permissions): the shell command policy seats where it is enforced
## Summary

`raven.permissions` imported `raven.agent.tools.shell_policy` for
`CommandDecision` and `ShellCommandPolicy`, while the agent side imports
`raven.permissions` from the loop, the turn path and the `raven_loop` backend.
Four edges down, one edge back up, and a cycle between two top-level packages.

The one edge back was the misplaced one. `shell_policy` answers whether a
command may run and whether it needs approval -- which is what the permissions
package is for. It sat under `agent/tools` because that is where the shell tool
that consumes it lives, not because the policy belongs to the tool.

Moving it is unusually cheap: its 1262 lines import `re`, `shlex`,
`contextvars`, `dataclasses`, `enum` and `pathlib` and nothing from raven, so
the module arrives in `permissions` carrying no dependency of its own. Nothing
new enters that package's dependency set.

Its three consumers are repointed rather than reading through a shim at the old
path: the shell tool, the ACP server's approval-family wiring, and the
permissions rules that started this. `agent -> permissions` and
`acp -> permissions` are both directions that already existed.

This is the third instance of one shape in this tree: a capability with no
dependencies of its own, seated inside one of its consumers, which a lower layer
then has to reach up for. The vocabulary split in MR 610 was the second.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

```
$ uv run --frozen --python 3.12 lint-imports
Contracts: 9 kept, 0 broken.

$ uv run --frozen --python 3.12 pytest tests/test_shell_approval.py \
    tests/test_shell_comments.py tests/test_shell_policy_reasons.py \
    tests/test_permissions_gate.py tests/test_acp_server.py \
    tests/test_agents_ppt_launcher.py -q
407 passed in 4.65s

$ uv run --frozen --python 3.12 pytest tests/ -q -k "shell or permission or policy"
746 passed, 8 skipped, 20 warnings in 23.73s

$ uv run --frozen --python 3.12 --extra dev ruff check raven/permissions/ \
    raven/acp/server.py raven/agent/tools/shell.py
All checks passed!

$ make check-commits COMMIT_RANGE=origin/main..HEAD
exit 0
```

The eight skips in the wider sweep are pre-existing and environmental: optional
extras that are not installed in this checkout (`wecom_aibot_sdk`, the render
extra). None of them exercise shell policy.

Every consumer of the module was enumerated before the move -- three in `raven/`
and six test files -- and all nine are repointed in this commit, so no import of
the old path remains anywhere in the tree.

Effect on the dependency graph:

```
permissions -> agent edges     1  ->  0
mutually importing pairs      14  ->  13
packages in an import cycle   22  ->  22   (unchanged)
```

The package count does not move. Unlike the config vocabulary split, this pair
was not the only thing holding any other package in the component -- worth
recording, because it is the case where a mutual pair disappears and the
component does not shrink.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing surface changes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

This module decides whether a shell command runs, so the security-relevant
question is whether the decision changed: it did not. The file moves unedited --
git records it as a rename -- and the matchers, the approval families and the
policy class are byte-identical. `tests/test_shell_approval.py`,
`tests/test_shell_comments.py` and `tests/test_permissions_gate.py` cover the
deny and approval paths and pass unchanged apart from the import line.

No public surface is affected: `shell_policy` is internal to raven and its old
path had no consumers outside this repository. Rollback is reverting the one
commit, which git will apply as the reverse rename.

## Related Issues

N/A
2026-09-11 02:35:05 +00:00
0xKT
3262294989 fix(rpc): a watched sub-agent relay turn can approve an ask-tier tool
## Summary

When a DAG node fails, the sub-agent's report opens a turn in the user's own conversation (origin `SUBAGENT`). The asker already bound in that turn whenever a surface was watching the conversation, so the person could answer `ask_user` there and pick "continue". The approval responder, however, bound only for `USER` turns, so the `resolve_dag_node` call that acts on that answer was refused in the same turn as "this turn is not interactive". The run ended on the person's own decision: the node could be neither continued nor abandoned from the TUI or the WebUI.

The fix binds the approval responder on the gate the asker already used: a `USER` turn always, a `SUBAGENT` relay when its conversation has a live watcher, every other origin never. One expression, shared by both bindings, in `RpcTurnRunner.run`.

The gateway's channel runner (`GatewayTurnRunner`) is deliberately left as it is: a channel session has no watched-signal to gate on, and widening there would turn an abandoned-channel question from an instant decline into a stall, as its own comment says.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_rpc_spine.py tests/test_agent_loop_approval.py tests/test_shell_approval.py tests/test_permissions*.py -q`: 361 passed. Two new cases: a `SUBAGENT` turn in a watched conversation reaches the responder and the gated tool runs; the same turn with nobody watching is still refused as not interactive.
- Reverse check: with the spine change reverted, the watched-relay case fails on the gated tool not running.
- `make lint-python`: clean.
- `uv run --all-extras pytest -q` (before rebase): 20150 passed, 4 failed, 165 skipped. The 4 failures (`test_agent_loop_token_budget`, `test_agents_code_launcher`, `test_agents_oncall_launcher`, `test_ppt_engine_image_search`) fail identically with this change reverted and again on the rebased head; none of them imports `raven.rpc.spine`. Pre-existing on `main`.

End to end, no permission override in the config, host started from this branch, port owner start time checked against the last code change:

| Frontend | Flow | Result |
|---|---|---|
| TUI (`raven tui --dev`) | run a playbook whose carried server is down -> node fails -> host calls `resolve_dag_node continue` -> approval overlay "Approve this action: resolve_dag_node {...}" -> key 1 -> tool runs -> attempt-2 dispatched | pass (before the fix the same step ended in "This call requires user approval (ask tier), but this turn is not interactive") |
| WebUI (`raven serve`) | same flow -> approval sheet "Approve this action: resolve_dag_node {...}" -> "Allow once" -> tool runs (the model chose abandon this time, so no second attempt; the point under test is that the approval reaches the person and the gated tool executes) | pass |

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (no user-facing docs exist for the approval gate)

## Risk

- Permission semantics: an ask-tier tool called in a sub-agent relay turn now reaches the person watching the conversation instead of being refused. It still fails closed with nobody watching (`has_subscribers` false), on timeout, and on every other origin. This is the same reachability rule the asker has had since the relay widening; approval was the one binding that had not followed it.
- Backward compatibility: `USER` and `CRON` turns are unchanged. The channel gateway runner is unchanged.
- Rollback: revert the squash commit; no data or config involved.

## Related Issues

N/A
2026-09-10 18:07:37 +00:00
gloryfromca
86651393ca fix(rpc): a watched sub-agent relay turn can approve an ask-tier tool
## Summary

When a DAG node fails, the sub-agent's report opens a turn in the user's own conversation (origin `SUBAGENT`). The asker already bound in that turn whenever a surface was watching the conversation, so the person could answer `ask_user` there and pick "continue". The approval responder, however, bound only for `USER` turns, so the `resolve_dag_node` call that acts on that answer was refused in the same turn as "this turn is not interactive". The run ended on the person's own decision: the node could be neither continued nor abandoned from the TUI or the WebUI.

The fix binds the approval responder on the gate the asker already used: a `USER` turn always, a `SUBAGENT` relay when its conversation has a live watcher, every other origin never. One expression, shared by both bindings, in `RpcTurnRunner.run`.

The gateway's channel runner (`GatewayTurnRunner`) is deliberately left as it is: a channel session has no watched-signal to gate on, and widening there would turn an abandoned-channel question from an instant decline into a stall, as its own comment says.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_rpc_spine.py tests/test_agent_loop_approval.py tests/test_shell_approval.py tests/test_permissions*.py -q`: 361 passed. Two new cases: a `SUBAGENT` turn in a watched conversation reaches the responder and the gated tool runs; the same turn with nobody watching is still refused as not interactive.
- Reverse check: with the spine change reverted, the watched-relay case fails on the gated tool not running.
- `make lint-python`: clean.
- `uv run --all-extras pytest -q` (before rebase): 20150 passed, 4 failed, 165 skipped. The 4 failures (`test_agent_loop_token_budget`, `test_agents_code_launcher`, `test_agents_oncall_launcher`, `test_ppt_engine_image_search`) fail identically with this change reverted and again on the rebased head; none of them imports `raven.rpc.spine`. Pre-existing on `main`.

End to end, no permission override in the config, host started from this branch, port owner start time checked against the last code change:

| Frontend | Flow | Result |
|---|---|---|
| TUI (`raven tui --dev`) | run a playbook whose carried server is down -> node fails -> host calls `resolve_dag_node continue` -> approval overlay "Approve this action: resolve_dag_node {...}" -> key 1 -> tool runs -> attempt-2 dispatched | pass (before the fix the same step ended in "This call requires user approval (ask tier), but this turn is not interactive") |
| WebUI (`raven serve`) | same flow -> approval sheet "Approve this action: resolve_dag_node {...}" -> "Allow once" -> tool runs (the model chose abandon this time, so no second attempt; the point under test is that the approval reaches the person and the gated tool executes) | pass |

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (no user-facing docs exist for the approval gate)

## Risk

- Permission semantics: an ask-tier tool called in a sub-agent relay turn now reaches the person watching the conversation instead of being refused. It still fails closed with nobody watching (`has_subscribers` false), on timeout, and on every other origin. This is the same reachability rule the asker has had since the relay widening; approval was the one binding that had not followed it.
- Backward compatibility: `USER` and `CRON` turns are unchanged. The channel gateway runner is unchanged.
- Rollback: revert the squash commit; no data or config involved.

## Related Issues

N/A
2026-09-10 18:07:36 +00:00
gloryfromca
231a31c641 test(*): the import cycles among subpackages may not reach further
## Summary

These are aggregate ceilings on how far import cycles reach, not a proof that no
cycle was deepened: an edge added between two members of the existing component
leaves every total unmoved, and that case is left to the contracts and to review.
The module docstring says so too.

`acyclic_siblings` is the import-linter contract that would forbid import cycles
among raven's top-level subpackages outright. It is not in `pyproject.toml`
because it is BROKEN: **18 of the 40 top-level subpackages sit in a single
strongly connected component**. Added as a gate it would paint CI red on arrival
and be switched off within the week.

This states the same rule as a budget instead of an absolute. Three ceilings, set
to what the tree measures today, and they may only go down:

```
PACKAGES_IN_CYCLES_CEILING = 18
MUTUAL_PAIRS_CEILING = 13
MODULES_IN_CYCLES_CEILING = 42
```

This branch carries the vocabulary refactor (MR 610) underneath it, because the
ceilings are the numbers that hold once that lands. If 610 merges first this
becomes a single-commit change; either way the ceilings match the tree the guard
runs against.

A change that draws a package or a module into a cycle for the first time, or
makes a new pair import each other, fails here. An edge added between two
members of the existing component does not: every total stays put, and that
case is left to the contracts and to review. Unpicking one lowers a ceiling,
which is a reviewed edit to the test file rather than a drive-by -- the same
shape `tests/test_kernel_budget.py` already uses, and for the same reason its
docstring gives.

Three numbers, because each is blind where another sees. Mutual pairs (a imports
b and b imports a) are the edges people actually add, and the failure message
names them. A cycle three packages long shows up in no pair at all, so a
component measure carries the general case.

The third exists because the package measure is an artifact of granularity as
much as of design. Collapsing a package to one node turns "config's writer
reaches agent, which reaches config's loader" into a cycle even where no module
imports itself back; measured per module, 42 of 635 modules sit in a cycle
rather than 18 of 40 packages. The two answer different questions -- whether a
package can be lifted out on its own, and whether a reader has a real knot to
walk around -- and they move independently: the refactor underneath took the
package count from 22 to 18 while moving the module count by three.

The numbers were still moving while this was written. `agent <-> permissions`
became the fourteenth mutual pair during the drafting, on a `permissions`
package that had just arrived. Nothing reported it, because nothing was
watching -- which is the case for pinning them.

`grimp` becomes an explicit dev dependency rather than staying a transitive one
through import-linter, since a test now imports it directly.

This changes no production code and unpicks no cycle. It fixes the number in
place so that the unpicking, when it happens, is not undone by the next branch.

### Both graph options are chosen, not defaulted

`exclude_type_checking_imports` is read out of `[tool.importlinter]` rather than
hardcoded, so the guard measures the same graph the contracts are checked
against. Without it, `gateway` counts as cyclic on a type-checking-only import
and the module count moves by three -- the guard would have been ratcheting a
graph no contract enforces.

`cache_dir=None` because grimp's default is a shared `.grimp_cache` that
outlives a branch switch. A guard whose job is to report the graph of the
checked-out tree must not serve one an earlier revision left behind: a stale hit
turns it red or green for reasons absent from the diff. Building fresh costs
about a second.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Other: an architecture guard, in the shape of a budget. Closest to `test`, which
is the commit type used.

## Verification

```
$ uv run --frozen --python 3.12 pytest tests/test_import_cycle_budget.py -q
4 passed in 1.06s

$ uv run --frozen --python 3.12 pytest tests/test_import_cycle_budget.py \
    tests/test_kernel_closure.py tests/test_kernel_budget.py -q
10 passed in 1.51s

$ uv run --frozen --python 3.12 lint-imports
Contracts: 8 kept, 0 broken.

$ uv run --frozen --python 3.12 --extra dev ruff check tests/test_import_cycle_budget.py
All checks passed!

$ uv run --frozen --python 3.12 --extra dev ruff format --check tests/test_import_cycle_budget.py
1 file already formatted

$ make check-commits COMMIT_RANGE=origin/main..HEAD
exit 0
```

Checked in the failing direction once per ceiling, since a gate that has never
been seen red is not known to work.

Making `utils` import the subagent presets trips the two package ceilings and
leaves the module count untouched -- which is the granularity point above, seen
from the failing side:

```
AssertionError: 23 subpackages are inside an import cycle, over the ceiling of 18
AssertionError: 15 pairs of subpackages import each other, over the ceiling of 13
```

Making `utils.text` and `importer.skills.hermes` import each other is a real
module-level cycle, and trips the third:

```
AssertionError: 45 modules are inside an import cycle, over the ceiling of 42
```

Both probes were reverted; the diff carries only the guard, the ceilings and the
dependency line.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing surface changes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No runtime behaviour change. The guard reads the import graph and asserts three
counts; it adds about a second to the suite. All three ceilings are met on
arrival,
so nothing that exists today is blocked. Rollback is reverting the one commit.

The known limit of a count-based ratchet is the one the coverage ratchet already
carries: a change that closes one cycle and opens another leaves the totals
unmoved and passes. Naming the pairs in the failure message is what makes that
visible in review when it matters.

## Related Issues

N/A
2026-09-10 17:23:40 +00:00
Handsome-wzw
7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00
forrestjgq
f01231fef6 fix(cli): overhaul trajectory browser interaction ux (#379)
## Summary

Overhaul the interactive `raven trajectory` browser end to end, fixing
every
issue from real-terminal testing:

- Navigation: every screen opens with a help area under its title (page
  purpose plus "[Key] function" hotkey hints, rendered in a new readable
help color). Esc walks one level up (action -> attempt -> session); on
the
top-level session screen Esc stays put and quitting is Ctrl+C only,
which
prints a single exit notice. Esc inside a running action cancels just
that
action through a dedicated exception at the action boundary, so the back
  sentinel can never reach a conversion or the data layer; a cancelled
  report keeps its pre-confirm bundle-and-pin side effects (the existing
  declined-report contract). Back/Exit menu entries are gone; answered
  prompts erase themselves and navigation is echoed as breadcrumb lines
  with markup-escaped untrusted text.
- Tables: the session list is TITLE | ATTEMPTS | LAST ACTIVITY and the
  attempt list is # | STARTED | TURNS | SPANS | VERDICT | PIN | MERGED |
PREVIEW with green check cells. Time cells keep the year. The layout is
  budgeted per screen build from the live terminal width (row indent, a
safety margin, and inter-column gaps are all part of the math); PREVIEW
  drops as a whole column when it no longer fits, and below the dynamic
  minimum width the tightest layout is kept while the renderer clips
  overlong lines at the terminal edge. Every dynamic cell passes a
  single-line sanitization gate before width math (verdict sidecars only
  guarantee a non-empty string), cells are measured and truncated by
terminal display width (CJK is 2 cells), and fixed column widths always
  fit their headers.
- Space preview: Space on an attempt row prints that attempt's per-turn
input/output previews, collected in the same snapshot scan and sorted by
a fully-stringified key so ordering never follows log source order (the
  table PREVIEW cell derives from the same sorted collection). Any key,
arrow keys included, returns with the cursor kept on the row; the waiter
  passes the terminal's CPR reply through to the renderer instead of
  treating it as a keypress, and Ctrl+C keeps the browser-wide cancel.
- Merge hotkey: with two or more attempts, m/M opens the multi-select
merge
  (advertised in the help line; the old "Merge attempts" menu entry is
gone). The checkbox screen carries its own help line for toggle,
confirm,
  and cancel. A single attempt neither binds nor advertises the key.
- Theme and feel: new success (green check) and help (readable hint
text)
colors in both palettes, all WCAG AA on their backgrounds; a lone ESC is
flushed after 50ms instead of prompt_toolkit's 0.5s default, removing
the
  laggy Esc feel.

The data layer is untouched: changes are confined to
raven/cli/trajectory_browse.py, raven/cli/_theme.py, and their tests.
The
id-taking subcommands (raven trajectory save/list/...) behave as before.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `COLORTERM=truecolor uv run pytest tests/test_cli_theme.py
tests/test_cli_trajectory_browse.py
tests/test_cli_trajectory_commands.py -q` -> 220 passed
- `COLORTERM=truecolor uv run pytest tests/ --ignore=tests/integration
-q` -> 6917 passed; the 10 failed + 20 errors are the pre-existing
environment-dependent set, identical file-by-file on a clean main
checkout (cron, everos server, config loader, tracing viewer)
- `uv run ruff check` and `uv run ruff format --check` on the four
touched files -> clean
- PTY smoke on a CPR-answering pseudo terminal at 100 and 58 columns:
full flow (preview, merge cancel, action screen, staged Esc backout,
Ctrl+C quit) and the narrow-width degradation contract
- New coverage includes real pipe-input key-binding tests (Esc on all
four prompt kinds, Space/m/M hits, waiter key paths with CPR
pass-through), table budget/clipping boundaries at the computed minimum
width, and corrupt verdict/timestamp corpora

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The interactive browser's look and key protocol change wholesale (that
is
  the point of this PR): Esc no longer quits from the top level (Ctrl+C
  does), Back/Exit entries are gone, and lists render as tables. The
machine-facing subcommands and the trajectory data layer are untouched.
- Rollback: revert the squash commit; the browser returns to the
previous
  interaction wholesale, no data migration involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-05 23:58:20 +08:00
0xKT
66cc5b5964 fix(design-engine): settle the task-state ledger before the reply ships
## Summary

A turn ends on the first reply without a tool call, and the design engine's `completion_notice` rides that reply out. In a direct chat that is a nudge with a next turn to act on it. Dispatched as a DAG node there is no next turn, so the same text reaches the caller's `dag_verdict` judge as a verdict.

The run that prompted this wrote a 147 KB page, took ten desktop screenshots and three narrow ones, reported the absolute path, and was failed. Its ledger's last revision was written 37 minutes before the reply, with two items still `in_progress` and `pending` -- the work in them had been done and never ticked. The notice was accurate; where it landed made it a verdict on work that existed.

`DesignEngineHook.after_iteration` now sends such a turn back **once**, carrying the notice plus an instruction to settle the record, using the `rollback` + `rollback_inject` seam the hook contract documents for exactly this ("bouncing a draft back with feedback").

Two decisions worth stating:

- **Decided from the ledger, not from the prose.** Sniffing whether a reply "reads as finished" is a language judgement and would need a marker list per language. Settling the record is an action the agent can always take, and both honest outcomes -- ticking what is done, or naming in the reply what is not -- are reachable from one nudge.
- **`after_send` is untouched.** If the second answer still leaves items open, the notice is appended exactly as before and the judge fails the node. That case is a fact the judge should see, so the fallback is the unchanged path rather than a new branch.

This is the guard `plugins-dist/ppt-engine` has had at the same seat (`UNFINISHED_NUDGE`, `hook.py`). The design engine was the one of the two without it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks
                            or hook_contract or hook_adapters"
  217 passed in 33.68s         (re-run at the rebased head)

python -m ruff check   plugins-dist/design-engine/raven_design/plugin/hook.py \
                       tests/test_design_engine_plugin_face.py
  All checks passed!
python -m ruff format --check <same two files>
  2 files already formatted

python scripts/check_commit_messages.py origin/main..HEAD
  exit 0, 1 commit
```

The new test was watched red before the fix and again after it was strengthened, both times against this repo's own unmodified copy of the engine:

```
FAILED tests/test_design_engine_plugin_face.py::test_stale_ledger_sends_the_turn_back_once
  assert False is True     (the base class returns a pass-through decision)
```

It drives the real `AgentHookContext`, the real `LLMResponse` and a real `TaskStateManager`, matching `tests/test_agents_oncall_flow_hooks.py` rather than the `SimpleNamespace` its neighbours in the same file use for `after_send`.

The instruction's means of compliance was checked rather than assumed: `make_update_task_state` returns `None` unless `shared.manager` is set, and `make_hook` is handed that same `shared.manager`, so the nudge cannot arrive on an install where the `update_task_state` tool that satisfies it is absent.

`lint-imports` is the gate GitLab CI does not run, and it is the one that has produced a blocker on this project before:

```
lint-imports
  Contracts: 9 kept, 0 broken.
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**User-visible change.** A design-engine turn that would have ended on an unfinished ledger now costs one extra model call. It fires on both lanes: in a direct chat the reader waits one call longer and gets a reply whose ledger agrees with it; on a dispatch the caller's judge stops failing finished work.

Bounded three ways: once per turn (a flag in `ctx.metadata`), then the loop's own `_MAX_HOOK_ROLLBACKS = 8`, and a rollback consumes no iteration.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security surface: the change adds no capability, reads no new input, and the injected text is engine-authored.

Rollback is deleting the `after_iteration` method -- the base class's pass-through restores today's behaviour exactly, and `after_send` is unmodified.

## Related Issues

N/A

## Review round 1

One blocking finding from @chandler.zhang, reproduced and fixed.

`after_iteration` fires after `stream_llm_call` has delivered every content delta, so the rollback popped history the reader had already seen, and the ACP collector concatenated the rejected draft with its replacement. Worse than reported in one respect: the retry did not tick the ledger either, so `after_send` appended the notice again -- the caller received draft, reply and notice, strictly worse than the stale notice this change was written to prevent.

```
before: ['FIRST DRAFT', 'FINAL REPLY', '<the task-state notice>']
after:  ['FINAL REPLY', '<the task-state notice>']
```

`DraftGate` (in `recovery.py`, beside `ContinuationGate`) holds a response's deltas until something keeps it. Released as soon as a response carries tool calls, so a preamble is not parked for a whole tool run; where both gates would apply the draft gate owns the continuation head cut, because holding the whole text cuts it exactly rather than on a prefix.

**Two deviations from the remedy as asked, both deliberate.**

Asked for it buffered "on this lane". There is no lane to scope to: `raven/rpc/spine.py` runs the human TUI turn and the ACP turn through the same `run_turn(..., stream=True)`, and only a CRON turn takes `stream=False`.

Scoped by capability instead -- the gate installs only where an installed hook overrides `after_iteration`. That is narrower than a lane check, and a regression is what found it. Buffering coalesces the reply into one delta, and with the predicate on `hook_ctx is not None` two existing tests went red on the incremental sequence (`assert ['Hello'] == ['Hel', 'lo']`). `_Announce` there overrides only `after_send`, so under the narrowed predicate it installs no gate and both tests stand unchanged. They are the guard on that half; nothing was weakened to make room.

Also corrected: the comment above that hook site claimed a rollback "discards-and-re-samples it", true of history and false of the stream.

**Scope this closes beyond the finding.** `plugins-dist/ppt-engine`'s `after_iteration` rolls back a text response the same way and had the identical exposure. This closes it too, the defect being a property of the seam rather than of either hook.

**Disclosed, not fixed.** A hook's `short_circuit_result` still streams the draft it replaces. That path is untouched here and predates this branch.

Re-verified at the rebased head (`origin/main` moved twice during review; both replays pure, `range-diff` all `=`). The base's own commits touched the ACP question surface, so those suites are in the re-run rather than only mine:

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks or hook_contract
                            or hook_adapters or run_emit or agent_loop_stream or acp_questions
                            or acp_elicitation or acp_dialects or ask_user"
  665 passed in 34.09s

python -m pytest tests/            19954 passed, 138 skipped in 519.84s
python -m ruff check / format --check   clean
lint-imports                            9 kept, 0 broken
scripts/check_commit_messages.py        exit 0
```

Covered at the loop level rather than by calling the hook. Disabling the gate turns it red:

```
FAILED test_a_rolled_back_draft_never_reaches_the_stream
  AssertionError: the caller saw the rejected draft: ['FIRST DRAFT', 'FINAL REPLY']
```

## Review round 3

@chandler.zhang's round-two blocker: the draft gate was scoped by whether a hook *overrides* `after_iteration`, which only says the hook watches the phase. `eval_engine`'s `AfterIterationHook` overrides it, awaits an LLM judge inside it, and its own docstring says an evaluator should never interrupt the user's reply chain -- so enabling the Eval Engine took incremental output away from a supported configuration and held the answer until the judge returned, for a draft that was never going to be rejected. Reproduced: `['Hel', 'lo']` arrived as `['Hello']`.

`AgentHook` now declares `rolls_back_iterations` (default `False`) and the loop reads the declaration. The design engine's hook and the deck engine's hook declare it -- the latter because this description claims the deck engine's exposure is closed, and the declaration is what keeps that true.

The observer path is pinned, and the pin asserts the await completed as well as the chunk sequence: `CompositeHook` swallows a hook exception, so a probe that raised past its await would otherwise pass while proving half of what it claimed. The first draft of that test did exactly that.

**Disclosed here, closed in round four.** Six research-flow gates under `agents/raven-research` (`evidence_floor`, `finalize`, `plain_first`, `report_shape`, `spin_breaker`, `verify`) answer `rollback` while their contributed hook does not declare it, so at this revision their rejected drafts still stream as they do on `main` today.

```
pytest tests/                                     19960 passed, 139 skipped
ruff check + format --check over the job's targets clean, 1849 files
lint-imports                                       9 kept, 0 broken
```

## Review round 4

@chandler.zhang's nonblocking note from round three, closed. `ResearchFlowHook` forwards every phase to a composite it builds per session, and six of that chain's gates answer `rollback`. The loop reads `rolls_back_iterations` off the contributed hook object once per turn, before any session chain exists, so nothing could derive the flag from the members that carry the behaviour -- and nothing in `flow.py` hinted the declaration was needed. The hook now declares it, with the reason recorded at the attribute.

**Enumerated rather than taken on the note.** Every `HookDecision` built with `rollback=True` across `agents`, `plugins-dist`, `raven` and `evolver` was located by AST; the research flow was the only tree whose contributed hook did not declare. The deck and design engines declare it in the same class that rolls back. `raven-code` and `raven-oncall` contribute hooks that never roll back and correctly leave the flag `False`, so this is not a case of one lane opting out of a rule. The research manifest declares exactly one hook contribution and its other three are tools, so no gate is registered directly and the contributed hook is the only level the declaration can sit at.

**Not a regression.** Draft suppression did not exist before this branch, so the research flow never held its deltas either. It gains the guarantee rather than losing one, which is why the note was graded nonblocking and why it is closed here rather than earlier.

The new test derives that producing set instead of listing it, so the omission cannot recur silently: a tree that grows a rollback and is not mapped fails as an unmapped root, and two further cases guard the derivation itself against passing vacuously. It reads the source by AST rather than importing, because `raven_design` and `raven_ppt` are editable installs rooted at the main checkout, and importing them from a worktree reads the main checkout's code instead of the revision under test.

Watched red before the fix and green after, with the derivation guards passing both times:

```
FAILED test_a_tree_that_rolls_back_declares_it_on_its_contributed_hook[agents/raven-research/...]
  AssertionError: ResearchFlowHook can roll back an iteration but does not declare
  rolls_back_iterations = True, so the loop streams a draft it may then retract
  1 failed, 4 passed
```

Read back through the loop's own predicate rather than off the class, in both directions: a real `ResearchFlowHook` instance now makes the loop hold drafts, and `eval_engine`'s judge -- which overrides the phase and only annotates -- still does not. The declaration is also not a `dataclass` field, so the hook's constructor signature is unchanged; an annotated spelling would have added a parameter and could not have sat above the fields without a default-ordering error.

Re-verified at the rebased head. `origin/main` moved six commits during review; the replay is pure (`range-diff` reports every commit `=`) and the base's three changes touch no file this branch touches, with the repo-wide rollback-producer set identical on both sides.

```
pytest tests/ -k research                            917 passed
pytest (loop, hook contract, iteration hooks, design face)   139 passed
ruff check + format --check over the job's targets   clean, 1850 files
lint-imports                                         9 kept, 0 broken
check_commit_messages.py / check_source_language.py  exit 0
make check-large-files                               exit 0
```
2026-09-09 12:31:40 +00:00
LivXue
42796ed28e fix(design-engine): settle the task-state ledger before the reply ships
## Summary

A turn ends on the first reply without a tool call, and the design engine's `completion_notice` rides that reply out. In a direct chat that is a nudge with a next turn to act on it. Dispatched as a DAG node there is no next turn, so the same text reaches the caller's `dag_verdict` judge as a verdict.

The run that prompted this wrote a 147 KB page, took ten desktop screenshots and three narrow ones, reported the absolute path, and was failed. Its ledger's last revision was written 37 minutes before the reply, with two items still `in_progress` and `pending` -- the work in them had been done and never ticked. The notice was accurate; where it landed made it a verdict on work that existed.

`DesignEngineHook.after_iteration` now sends such a turn back **once**, carrying the notice plus an instruction to settle the record, using the `rollback` + `rollback_inject` seam the hook contract documents for exactly this ("bouncing a draft back with feedback").

Two decisions worth stating:

- **Decided from the ledger, not from the prose.** Sniffing whether a reply "reads as finished" is a language judgement and would need a marker list per language. Settling the record is an action the agent can always take, and both honest outcomes -- ticking what is done, or naming in the reply what is not -- are reachable from one nudge.
- **`after_send` is untouched.** If the second answer still leaves items open, the notice is appended exactly as before and the judge fails the node. That case is a fact the judge should see, so the fallback is the unchanged path rather than a new branch.

This is the guard `plugins-dist/ppt-engine` has had at the same seat (`UNFINISHED_NUDGE`, `hook.py`). The design engine was the one of the two without it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks
                            or hook_contract or hook_adapters"
  217 passed in 33.68s         (re-run at the rebased head)

python -m ruff check   plugins-dist/design-engine/raven_design/plugin/hook.py \
                       tests/test_design_engine_plugin_face.py
  All checks passed!
python -m ruff format --check <same two files>
  2 files already formatted

python scripts/check_commit_messages.py origin/main..HEAD
  exit 0, 1 commit
```

The new test was watched red before the fix and again after it was strengthened, both times against this repo's own unmodified copy of the engine:

```
FAILED tests/test_design_engine_plugin_face.py::test_stale_ledger_sends_the_turn_back_once
  assert False is True     (the base class returns a pass-through decision)
```

It drives the real `AgentHookContext`, the real `LLMResponse` and a real `TaskStateManager`, matching `tests/test_agents_oncall_flow_hooks.py` rather than the `SimpleNamespace` its neighbours in the same file use for `after_send`.

The instruction's means of compliance was checked rather than assumed: `make_update_task_state` returns `None` unless `shared.manager` is set, and `make_hook` is handed that same `shared.manager`, so the nudge cannot arrive on an install where the `update_task_state` tool that satisfies it is absent.

`lint-imports` is the gate GitLab CI does not run, and it is the one that has produced a blocker on this project before:

```
lint-imports
  Contracts: 9 kept, 0 broken.
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**User-visible change.** A design-engine turn that would have ended on an unfinished ledger now costs one extra model call. It fires on both lanes: in a direct chat the reader waits one call longer and gets a reply whose ledger agrees with it; on a dispatch the caller's judge stops failing finished work.

Bounded three ways: once per turn (a flag in `ctx.metadata`), then the loop's own `_MAX_HOOK_ROLLBACKS = 8`, and a rollback consumes no iteration.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security surface: the change adds no capability, reads no new input, and the injected text is engine-authored.

Rollback is deleting the `after_iteration` method -- the base class's pass-through restores today's behaviour exactly, and `after_send` is unmodified.

## Related Issues

N/A

## Review round 1

One blocking finding from @chandler.zhang, reproduced and fixed.

`after_iteration` fires after `stream_llm_call` has delivered every content delta, so the rollback popped history the reader had already seen, and the ACP collector concatenated the rejected draft with its replacement. Worse than reported in one respect: the retry did not tick the ledger either, so `after_send` appended the notice again -- the caller received draft, reply and notice, strictly worse than the stale notice this change was written to prevent.

```
before: ['FIRST DRAFT', 'FINAL REPLY', '<the task-state notice>']
after:  ['FINAL REPLY', '<the task-state notice>']
```

`DraftGate` (in `recovery.py`, beside `ContinuationGate`) holds a response's deltas until something keeps it. Released as soon as a response carries tool calls, so a preamble is not parked for a whole tool run; where both gates would apply the draft gate owns the continuation head cut, because holding the whole text cuts it exactly rather than on a prefix.

**Two deviations from the remedy as asked, both deliberate.**

Asked for it buffered "on this lane". There is no lane to scope to: `raven/rpc/spine.py` runs the human TUI turn and the ACP turn through the same `run_turn(..., stream=True)`, and only a CRON turn takes `stream=False`.

Scoped by capability instead -- the gate installs only where an installed hook overrides `after_iteration`. That is narrower than a lane check, and a regression is what found it. Buffering coalesces the reply into one delta, and with the predicate on `hook_ctx is not None` two existing tests went red on the incremental sequence (`assert ['Hello'] == ['Hel', 'lo']`). `_Announce` there overrides only `after_send`, so under the narrowed predicate it installs no gate and both tests stand unchanged. They are the guard on that half; nothing was weakened to make room.

Also corrected: the comment above that hook site claimed a rollback "discards-and-re-samples it", true of history and false of the stream.

**Scope this closes beyond the finding.** `plugins-dist/ppt-engine`'s `after_iteration` rolls back a text response the same way and had the identical exposure. This closes it too, the defect being a property of the seam rather than of either hook.

**Disclosed, not fixed.** A hook's `short_circuit_result` still streams the draft it replaces. That path is untouched here and predates this branch.

Re-verified at the rebased head (`origin/main` moved twice during review; both replays pure, `range-diff` all `=`). The base's own commits touched the ACP question surface, so those suites are in the re-run rather than only mine:

```
python -m pytest tests/ -k "design_engine or loop_hooks or iteration_hooks or hook_contract
                            or hook_adapters or run_emit or agent_loop_stream or acp_questions
                            or acp_elicitation or acp_dialects or ask_user"
  665 passed in 34.09s

python -m pytest tests/            19954 passed, 138 skipped in 519.84s
python -m ruff check / format --check   clean
lint-imports                            9 kept, 0 broken
scripts/check_commit_messages.py        exit 0
```

Covered at the loop level rather than by calling the hook. Disabling the gate turns it red:

```
FAILED test_a_rolled_back_draft_never_reaches_the_stream
  AssertionError: the caller saw the rejected draft: ['FIRST DRAFT', 'FINAL REPLY']
```

## Review round 3

@chandler.zhang's round-two blocker: the draft gate was scoped by whether a hook *overrides* `after_iteration`, which only says the hook watches the phase. `eval_engine`'s `AfterIterationHook` overrides it, awaits an LLM judge inside it, and its own docstring says an evaluator should never interrupt the user's reply chain -- so enabling the Eval Engine took incremental output away from a supported configuration and held the answer until the judge returned, for a draft that was never going to be rejected. Reproduced: `['Hel', 'lo']` arrived as `['Hello']`.

`AgentHook` now declares `rolls_back_iterations` (default `False`) and the loop reads the declaration. The design engine's hook and the deck engine's hook declare it -- the latter because this description claims the deck engine's exposure is closed, and the declaration is what keeps that true.

The observer path is pinned, and the pin asserts the await completed as well as the chunk sequence: `CompositeHook` swallows a hook exception, so a probe that raised past its await would otherwise pass while proving half of what it claimed. The first draft of that test did exactly that.

**Disclosed here, closed in round four.** Six research-flow gates under `agents/raven-research` (`evidence_floor`, `finalize`, `plain_first`, `report_shape`, `spin_breaker`, `verify`) answer `rollback` while their contributed hook does not declare it, so at this revision their rejected drafts still stream as they do on `main` today.

```
pytest tests/                                     19960 passed, 139 skipped
ruff check + format --check over the job's targets clean, 1849 files
lint-imports                                       9 kept, 0 broken
```

## Review round 4

@chandler.zhang's nonblocking note from round three, closed. `ResearchFlowHook` forwards every phase to a composite it builds per session, and six of that chain's gates answer `rollback`. The loop reads `rolls_back_iterations` off the contributed hook object once per turn, before any session chain exists, so nothing could derive the flag from the members that carry the behaviour -- and nothing in `flow.py` hinted the declaration was needed. The hook now declares it, with the reason recorded at the attribute.

**Enumerated rather than taken on the note.** Every `HookDecision` built with `rollback=True` across `agents`, `plugins-dist`, `raven` and `evolver` was located by AST; the research flow was the only tree whose contributed hook did not declare. The deck and design engines declare it in the same class that rolls back. `raven-code` and `raven-oncall` contribute hooks that never roll back and correctly leave the flag `False`, so this is not a case of one lane opting out of a rule. The research manifest declares exactly one hook contribution and its other three are tools, so no gate is registered directly and the contributed hook is the only level the declaration can sit at.

**Not a regression.** Draft suppression did not exist before this branch, so the research flow never held its deltas either. It gains the guarantee rather than losing one, which is why the note was graded nonblocking and why it is closed here rather than earlier.

The new test derives that producing set instead of listing it, so the omission cannot recur silently: a tree that grows a rollback and is not mapped fails as an unmapped root, and two further cases guard the derivation itself against passing vacuously. It reads the source by AST rather than importing, because `raven_design` and `raven_ppt` are editable installs rooted at the main checkout, and importing them from a worktree reads the main checkout's code instead of the revision under test.

Watched red before the fix and green after, with the derivation guards passing both times:

```
FAILED test_a_tree_that_rolls_back_declares_it_on_its_contributed_hook[agents/raven-research/...]
  AssertionError: ResearchFlowHook can roll back an iteration but does not declare
  rolls_back_iterations = True, so the loop streams a draft it may then retract
  1 failed, 4 passed
```

Read back through the loop's own predicate rather than off the class, in both directions: a real `ResearchFlowHook` instance now makes the loop hold drafts, and `eval_engine`'s judge -- which overrides the phase and only annotates -- still does not. The declaration is also not a `dataclass` field, so the hook's constructor signature is unchanged; an annotated spelling would have added a parameter and could not have sat above the fields without a default-ordering error.

Re-verified at the rebased head. `origin/main` moved six commits during review; the replay is pure (`range-diff` reports every commit `=`) and the base's three changes touch no file this branch touches, with the repo-wide rollback-producer set identical on both sides.

```
pytest tests/ -k research                            917 passed
pytest (loop, hook contract, iteration hooks, design face)   139 passed
ruff check + format --check over the job's targets   clean, 1850 files
lint-imports                                         9 kept, 0 broken
check_commit_messages.py / check_source_language.py  exit 0
make check-large-files                               exit 0
```
2026-09-09 12:31:39 +00:00
0xKT
2b6c4d495c fix(*): break a loop that keeps succeeding, and name a clobbered helper
## Summary

One livelock, three causes, all three closed here. A run read one file 127 times over an hour, exit 0 on every call, and ended its turn exactly where it started -- at iteration 343 of 600 when it was killed, with the file's md5 unchanged throughout.

**The guard did not arm.** `failure_streak` counts only hard failures and `turn_path` resets the streak on any success, so a call that keeps working and keeps answering the same thing is structurally invisible to it. `raven/agent/loop/no_progress.py` counts the other shape.

It counts **per turn rather than consecutively**, and that is the load-bearing decision: the run alternated two spellings of one read -- a plain read, and the same read behind an `md5sum` -- so no two neighbouring calls were identical and a consecutive streak would have stayed at one forever. The cost is that two spellings each reach the threshold on their own, so an alternating pair fires at about fifteen calls rather than eight. Against 127 that is the trade worth taking. A test pins the arithmetic so the design is not "simplified" back to consecutive.

Threshold 8 (higher than the failure threshold's 2, because repeating a working call is ordinary -- a poll, a re-read after an edit), capped at 2 per turn like the failure nudge, and it appends a nudge and nothing else.

**The cause was silent.** `provision` rewrites every helper beside the build script on every build. That is deliberate and its docstring says why (a quietly edited helper fails in a way that reads as a deck bug), but it said nothing -- so a run whose own `themes.json` went back to the engine's copy could not tell a reverted write from one that never happened. `ppt_build` now names what it put back, and says nothing when it put nothing back.

**The documentation contradicted the data.** The generated `ppt_theme` docstring promised a single entry and called a named theme a `KeyError`. That holds on the template route, where `with_template_helpers` narrows `themes.json` to the template's own palette. Without a template the same file carries the ten reviewed themes and naming one is how an author picks -- which `tests/test_ppt_engine_assets_script_helpers.py` itself does (`THEMES["indigo-scholar"]`). The text now follows the payload.

`ppt_charts` and `ppt_layout` opened with the same `T = THEMES[...]` line and carried the template route's claim beside it, so fixing `ppt_theme` alone left two generated modules telling a catalog-route author the opposite. They now point at the module that owns the rule, and a test pins that they do not restate it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
python -m pytest tests/ -k "ppt_engine or agent_loop or turn_path
                            or failure_streak or no_progress"
  3956 passed, 94 skipped in 93.34s      (re-run at the rebased head)

python -m ruff check raven/agent/loop/ plugins-dist/ppt-engine/raven_ppt/backends/script/ \
                     plugins-dist/ppt-engine/raven_ppt/services/assets/script_helpers.py \
                     tests/test_agent_loop_tool_loop_break.py \
                     tests/test_ppt_engine_assets_script_helpers.py \
                     tests/test_ppt_engine_script_backend.py
  All checks passed!
python -m ruff format --check <the eight changed files>
  8 files already formatted

python scripts/check_commit_messages.py origin/main..HEAD
  exit 0, 1 commit
```

The 94 skips are the pre-existing `templates.manifest.json` payload skips; they are identical on `origin/main`.

Each of the three was watched red first, and the loop guard was mutation-tested rather than trusted:

```
# the nudge computed and then withheld (one line replaced by `pass`)
FAILED test_an_unchanging_successful_call_is_nudged_and_bounded
  AssertionError: an unchanging successful call was never nudged

# the one-home test against this repo's unmodified charts/layout
FAILED test_only_the_theme_module_states_the_theme_rule
  AssertionError: ppt_charts restates the template route's rule

# the restore note, before ppt_build carried it
FAILED test_a_restored_helper_is_named_in_the_build_note
  AssertionError: the build names the helper it put back
```

The nudge was then driven through a real `AgentLoop` to see it arrive rather than inferred from the counter:

```
THRESHOLD=8  MAX=2
[loop] `list_dir` has returned the same result for the same arguments 8 times in
this turn. Calling it again will not answer differently. ...
```

Both directions of the new guard were probed:

```
identical call and result -> one key          (fires: correct)
a poll whose answer moves -> 8 distinct keys  (never fires: correct)
```

`lint-imports` is the gate GitLab CI does not run, and it is the one that has produced a blocker on this project before:

```
lint-imports
  Contracts: 9 kept, 0 broken.
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**User-visible change.** Three, all additive:

1. A turn that repeats one exact call with one exact answer eight times gets one extra line appended to that tool result, at most twice per turn. Nothing is blocked, refused or rerouted.
2. A `ppt_build` that restored a helper the author had edited says so in its `note`. A build that restored nothing is unchanged.
3. A build directory with no template now ships a `ppt_theme` docstring that says to pick a theme by name. The code in that module is byte-identical; only the text differs.

**Disclosed, and deliberately not fixed.** The no-progress key includes the result, which is what separates a poll making progress from one that is not. A result carrying a clock or a per-call nonce therefore keys differently every time and never accumulates, so the guard is too tight in that direction rather than too loose. Widening it (hashing a prefix, stripping digits) would start firing on genuine progress, which is the worse failure for a nudge that tells the model to stop looking.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security surface: no new input is read, no capability is added, and all three texts are engine-authored.

Rollback is per cause and independent -- delete the `elif` in `turn_path` for the guard, drop the `if put_back:` block in `runner` for the note, or pass no flag to `theme_module_source` for the docstring.

## Related Issues

N/A

## Review round 1

Two blocking findings from @chandler.zhang, both reproduced and fixed, one commit each.

**The restore note was dropped on a failed build** (`02ef9aa4`). `outcome.note` was read only after the tool's success return, and two post-provision exits in `run_script` dropped it as well -- the missing-script return and the timeout. The livelock this note exists for was **nine failed `ppt_build` calls**, so as first submitted it would not have fired once in the incident that motivated it. The early exits above `stage.run` are pre-provision and correctly carry nothing. Now tested through `ppt_build`, on the tool's own JSON.

**The no-progress key hashed `model_text` alone** (`5a318c2e`). An image read answers with path, dimensions and a token estimate -- stable across a repaint at the same size -- while the pixels ride in the routed blocks, so a render-edit-inspect pass read "the same result" eight times in exactly the workflow the guard was added to help. Reproduced through the loop's own path:

```
what the loop counts on   : '[image: /tmp/.../slide.png] | 400x300px | ~165 tokens'
model_text identical      : True      # two 400x300 images, different pixels
image blocks identical    : False
no_progress key identical : True
```

Of the two remedies offered I took the first, a stable identity for the routed blocks, rather than excluding multimodal results -- so the guard stays live for an image loop that genuinely is stuck. Covered at loop level with a real repainted image and vision forced on, paired against the same loop over an unchanging one.

Re-verified at the rebased head (`origin/main` moved twice during review; both replays pure, `range-diff` all `=`):

```
python -m pytest tests/ -k "ppt_engine or agent_loop or turn_path or failure_streak or no_progress"
  3958 passed, 94 skipped in 87.74s

python -m ruff check / format --check   clean
lint-imports                            9 kept, 0 broken
scripts/check_commit_messages.py        exit 0
```

Each fix was watched red first:

```
FAILED test_a_restored_helper_is_named_on_a_failed_build
  AssertionError: a failed build does not name the helper it put back
FAILED test_changing_pixels_behind_one_metadata_line_are_progress
  AssertionError: a repainted image counted as no progress
```

## Review round 2

`1a601700`. @chandler.zhang's clearing note flagged a formatter-only failure and it was real: `ruff format --check` wanted the three-line `BuildOutcome(...)` added by `02ef9aa4` on one line, and the CI lint job reported it. Now clean over every changed `.py` file, with `ruff check` and the three ppt suites (208 passed) re-run after.

Cause on my side: `ruff check` was re-run over that edit and `ruff format --check` was not, and this repo's `core.hooksPath` points at a directory that does not exist, so nothing catches it at commit time.
2026-09-09 12:26:51 +00:00
LivXue
a54627a566 fix(*): break a loop that keeps succeeding, and name a clobbered helper
## Summary

One livelock, three causes, all three closed here. A run read one file 127 times over an hour, exit 0 on every call, and ended its turn exactly where it started -- at iteration 343 of 600 when it was killed, with the file's md5 unchanged throughout.

**The guard did not arm.** `failure_streak` counts only hard failures and `turn_path` resets the streak on any success, so a call that keeps working and keeps answering the same thing is structurally invisible to it. `raven/agent/loop/no_progress.py` counts the other shape.

It counts **per turn rather than consecutively**, and that is the load-bearing decision: the run alternated two spellings of one read -- a plain read, and the same read behind an `md5sum` -- so no two neighbouring calls were identical and a consecutive streak would have stayed at one forever. The cost is that two spellings each reach the threshold on their own, so an alternating pair fires at about fifteen calls rather than eight. Against 127 that is the trade worth taking. A test pins the arithmetic so the design is not "simplified" back to consecutive.

Threshold 8 (higher than the failure threshold's 2, because repeating a working call is ordinary -- a poll, a re-read after an edit), capped at 2 per turn like the failure nudge, and it appends a nudge and nothing else.

**The cause was silent.** `provision` rewrites every helper beside the build script on every build. That is deliberate and its docstring says why (a quietly edited helper fails in a way that reads as a deck bug), but it said nothing -- so a run whose own `themes.json` went back to the engine's copy could not tell a reverted write from one that never happened. `ppt_build` now names what it put back, and says nothing when it put nothing back.

**The documentation contradicted the data.** The generated `ppt_theme` docstring promised a single entry and called a named theme a `KeyError`. That holds on the template route, where `with_template_helpers` narrows `themes.json` to the template's own palette. Without a template the same file carries the ten reviewed themes and naming one is how an author picks -- which `tests/test_ppt_engine_assets_script_helpers.py` itself does (`THEMES["indigo-scholar"]`). The text now follows the payload.

`ppt_charts` and `ppt_layout` opened with the same `T = THEMES[...]` line and carried the template route's claim beside it, so fixing `ppt_theme` alone left two generated modules telling a catalog-route author the opposite. They now point at the module that owns the rule, and a test pins that they do not restate it.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
python -m pytest tests/ -k "ppt_engine or agent_loop or turn_path
                            or failure_streak or no_progress"
  3956 passed, 94 skipped in 93.34s      (re-run at the rebased head)

python -m ruff check raven/agent/loop/ plugins-dist/ppt-engine/raven_ppt/backends/script/ \
                     plugins-dist/ppt-engine/raven_ppt/services/assets/script_helpers.py \
                     tests/test_agent_loop_tool_loop_break.py \
                     tests/test_ppt_engine_assets_script_helpers.py \
                     tests/test_ppt_engine_script_backend.py
  All checks passed!
python -m ruff format --check <the eight changed files>
  8 files already formatted

python scripts/check_commit_messages.py origin/main..HEAD
  exit 0, 1 commit
```

The 94 skips are the pre-existing `templates.manifest.json` payload skips; they are identical on `origin/main`.

Each of the three was watched red first, and the loop guard was mutation-tested rather than trusted:

```
# the nudge computed and then withheld (one line replaced by `pass`)
FAILED test_an_unchanging_successful_call_is_nudged_and_bounded
  AssertionError: an unchanging successful call was never nudged

# the one-home test against this repo's unmodified charts/layout
FAILED test_only_the_theme_module_states_the_theme_rule
  AssertionError: ppt_charts restates the template route's rule

# the restore note, before ppt_build carried it
FAILED test_a_restored_helper_is_named_in_the_build_note
  AssertionError: the build names the helper it put back
```

The nudge was then driven through a real `AgentLoop` to see it arrive rather than inferred from the counter:

```
THRESHOLD=8  MAX=2
[loop] `list_dir` has returned the same result for the same arguments 8 times in
this turn. Calling it again will not answer differently. ...
```

Both directions of the new guard were probed:

```
identical call and result -> one key          (fires: correct)
a poll whose answer moves -> 8 distinct keys  (never fires: correct)
```

`lint-imports` is the gate GitLab CI does not run, and it is the one that has produced a blocker on this project before:

```
lint-imports
  Contracts: 9 kept, 0 broken.
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

**User-visible change.** Three, all additive:

1. A turn that repeats one exact call with one exact answer eight times gets one extra line appended to that tool result, at most twice per turn. Nothing is blocked, refused or rerouted.
2. A `ppt_build` that restored a helper the author had edited says so in its `note`. A build that restored nothing is unchanged.
3. A build directory with no template now ships a `ppt_theme` docstring that says to pick a theme by name. The code in that module is byte-identical; only the text differs.

**Disclosed, and deliberately not fixed.** The no-progress key includes the result, which is what separates a poll making progress from one that is not. A result carrying a clock or a per-call nonce therefore keys differently every time and never accumulates, so the guard is too tight in that direction rather than too loose. Widening it (hashing a prefix, stripping digits) would start firing on genuine progress, which is the worse failure for a nudge that tells the model to stop looking.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security surface: no new input is read, no capability is added, and all three texts are engine-authored.

Rollback is per cause and independent -- delete the `elif` in `turn_path` for the guard, drop the `if put_back:` block in `runner` for the note, or pass no flag to `theme_module_source` for the docstring.

## Related Issues

N/A

## Review round 1

Two blocking findings from @chandler.zhang, both reproduced and fixed, one commit each.

**The restore note was dropped on a failed build** (`02ef9aa4`). `outcome.note` was read only after the tool's success return, and two post-provision exits in `run_script` dropped it as well -- the missing-script return and the timeout. The livelock this note exists for was **nine failed `ppt_build` calls**, so as first submitted it would not have fired once in the incident that motivated it. The early exits above `stage.run` are pre-provision and correctly carry nothing. Now tested through `ppt_build`, on the tool's own JSON.

**The no-progress key hashed `model_text` alone** (`5a318c2e`). An image read answers with path, dimensions and a token estimate -- stable across a repaint at the same size -- while the pixels ride in the routed blocks, so a render-edit-inspect pass read "the same result" eight times in exactly the workflow the guard was added to help. Reproduced through the loop's own path:

```
what the loop counts on   : '[image: /tmp/.../slide.png] | 400x300px | ~165 tokens'
model_text identical      : True      # two 400x300 images, different pixels
image blocks identical    : False
no_progress key identical : True
```

Of the two remedies offered I took the first, a stable identity for the routed blocks, rather than excluding multimodal results -- so the guard stays live for an image loop that genuinely is stuck. Covered at loop level with a real repainted image and vision forced on, paired against the same loop over an unchanging one.

Re-verified at the rebased head (`origin/main` moved twice during review; both replays pure, `range-diff` all `=`):

```
python -m pytest tests/ -k "ppt_engine or agent_loop or turn_path or failure_streak or no_progress"
  3958 passed, 94 skipped in 87.74s

python -m ruff check / format --check   clean
lint-imports                            9 kept, 0 broken
scripts/check_commit_messages.py        exit 0
```

Each fix was watched red first:

```
FAILED test_a_restored_helper_is_named_on_a_failed_build
  AssertionError: a failed build does not name the helper it put back
FAILED test_changing_pixels_behind_one_metadata_line_are_progress
  AssertionError: a repainted image counted as no progress
```

## Review round 2

`1a601700`. @chandler.zhang's clearing note flagged a formatter-only failure and it was real: `ruff format --check` wanted the three-line `BuildOutcome(...)` added by `02ef9aa4` on one line, and the CI lint job reported it. Now clean over every changed `.py` file, with `ruff check` and the three ppt suites (208 passed) re-run after.

Cause on my side: `ruff check` was re-run over that edit and `ruff format --check` was not, and this repo's `core.hooksPath` points at a directory that does not exist, so nothing catches it at commit time.
2026-09-09 12:26:51 +00:00
0xKT
7af4931185 fix(agent): dispatch a continued dag node without waiting for its siblings
## Summary

A `continue` answer to a suspended DAG node was applied only once the scheduling round
it belonged to had drained. Both signals a round could see -- `cancel` and the replan
`interrupt` -- end it by cancelling its in-flight tasks, so `continue` had no way to
shorten the wait: the re-dispatch sat behind every sibling dispatched alongside the
suspended node, however long those ran and however unrelated they were. Observed as a
node that was told to continue and did not for 45 minutes, while an independent sibling
in the same graph kept working.

This adds the missing third shape. `AdjudicationDesk` gains a `continued` event, fired
by `resolve` for a CONTINUE decision, and `_run_ready_groups` gains a `resume` parameter
that ends the round early and returns the tasks still running instead of reaping them.
The scheduler carries those into the next round through a new `carried` argument, so the
re-dispatch runs alongside them under the same semaphore. The two signals cannot share a
parameter: `interrupt` means stop these nodes, `resume` means stop waiting for them.

Handing tasks back moves their ownership out of the round's `finally`, which is where
every in-flight task was reaped. The two loop exits reachable while any are outstanding
-- the replan branch and the cancellation handler -- now reap through `_reap_carried`.
Without that an uncancelled group task outlives its run, holding a semaphore slot and a
sub-agent process nothing will collect.

Ownership has a second consequence, in the ready set. `_run_node` sets `running` inside
the concurrency gate on purpose, so a node still queued for a slot reads `pending` --
which, from status alone, cannot be told apart from a node nobody has taken. The
scheduler therefore creates each round's group tasks itself and records which node ids
each one owns, and a carried task's nodes stay out of the ready set until it ends.
Without that a sibling carried before it had acquired a slot was dispatched a second
time, doing its work twice and racing its own files.

Three of the commits come from the pre-submit sweep rather than the defect: the twin
assertions the new desk event was missing, an `_await_adjudications` docstring still
saying nothing else in the graph can run during the wait, and the replan design stating
the drained-round rule as absolute when it now holds only for the signals that stop work.

Scoped deliberately to `continue`. `abandon` shares the same wait, but shortening it buys
nothing: an abandoned node's dependents cascade to `skipped` and none could have been
dispatched sooner. A test pins that as a decision rather than an omission.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Full Python suite at the branch head, rebased onto the current `main`:

    uv run --frozen --python 3.12 --all-extras pytest tests/ -q
    19961 passed, 139 skipped, 54 warnings in 386.56s

Zero failures. `--all-extras` is load-bearing: without it the deck-engine tests are not
collected, and running with only `raven` on `PYTHONPATH` resolves `raven_ppt` out of a
different checkout, which reports failures belonging to neither side.

The rebase took on twenty commits and none of them touch the five files changed here;
`git range-diff` against the pre-rebase range reports all four commits replaying
unchanged.

Two regression tests carry the behaviour, both in `tests/test_subagent_dag_runner.py`:

`test_a_continued_node_runs_again_while_a_sibling_is_still_running`
deadlocks without the fix, because only the continued node's second attempt releases the
sibling. Red before it (asyncio TimeoutError at the 15s guard), green after, in 1.97s.

`test_a_carried_group_queued_for_a_slot_is_not_dispatched_twice`
drives two independent nodes through a single slot and asserts the queued sibling runs
once. Without the ready-set guard it records the backend calls `['a', 'b', 'b', 'a']`.

Two more pin the disposal of a carried task, the property that has no owner but the two
`_reap_carried` calls:

`test_an_outer_cancellation_reaps_a_handed_on_task`
`test_a_replan_reaps_a_handed_on_task_before_it_records_them_cancelled`

Both drive one node to a continue and another to an unanswered suspension, so the loop
parks in `_await_adjudications` holding the carried task alone -- outside any round, which
is the only state in which those calls are what cancels it. Each reddens for its own call
site and neither for the other, checked by deleting them one at a time and together. They
are the siblings of `test_run_dag_cancel_skips_unfinished_and_reaps_in_flight_task`, which
proves the same property for a task inside the round.

Gates the GitLab pipeline does not run:

    ruff format --check raven/ tests/                          1487 files already formatted
    ruff check raven/ tests/                                   All checks passed
    lint-imports                                               Contracts: 9 kept, 0 broken
    python scripts/check_source_language.py origin/main...HEAD exit 0
    python scripts/check_commit_messages.py origin/main..HEAD  exit 0
    make check-large-files                                     exit 0

Six commits: the fix, three sweep follow-ups, the ready-set guard, and the reap tests.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing docs: the change is internal to the DAG scheduler and alters no tool
description, prompt asset or RPC shape.

## Risk

Behaviour change: a continued node is re-dispatched as soon as its answer lands rather
than after its round drains. Nothing else about the decision moves -- the continuation
budget, the message handed to the node, and the node's status transitions are untouched.
`dag_node_updated` for the continued node fires earlier; those events already interleave
across concurrent nodes, so no consumer gains an ordering it did not already tolerate.

Known gaps, disclosed rather than fixed:

- One bounded latency replaces the unbounded one: when a handed-on task is running and a
  second node is suspended without an answer, the loop parks in the adjudication wait and
  does not observe that task completing until an answer lands or the wait times out.
  That state was previously unreachable, and the wait it replaces had no bound at all.

Rollback: revert the four commits. No migration, no persisted state, no configuration.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-09 12:26:20 +00:00
LivXue
afc81a083c fix(agent): dispatch a continued dag node without waiting for its siblings
## Summary

A `continue` answer to a suspended DAG node was applied only once the scheduling round
it belonged to had drained. Both signals a round could see -- `cancel` and the replan
`interrupt` -- end it by cancelling its in-flight tasks, so `continue` had no way to
shorten the wait: the re-dispatch sat behind every sibling dispatched alongside the
suspended node, however long those ran and however unrelated they were. Observed as a
node that was told to continue and did not for 45 minutes, while an independent sibling
in the same graph kept working.

This adds the missing third shape. `AdjudicationDesk` gains a `continued` event, fired
by `resolve` for a CONTINUE decision, and `_run_ready_groups` gains a `resume` parameter
that ends the round early and returns the tasks still running instead of reaping them.
The scheduler carries those into the next round through a new `carried` argument, so the
re-dispatch runs alongside them under the same semaphore. The two signals cannot share a
parameter: `interrupt` means stop these nodes, `resume` means stop waiting for them.

Handing tasks back moves their ownership out of the round's `finally`, which is where
every in-flight task was reaped. The two loop exits reachable while any are outstanding
-- the replan branch and the cancellation handler -- now reap through `_reap_carried`.
Without that an uncancelled group task outlives its run, holding a semaphore slot and a
sub-agent process nothing will collect.

Ownership has a second consequence, in the ready set. `_run_node` sets `running` inside
the concurrency gate on purpose, so a node still queued for a slot reads `pending` --
which, from status alone, cannot be told apart from a node nobody has taken. The
scheduler therefore creates each round's group tasks itself and records which node ids
each one owns, and a carried task's nodes stay out of the ready set until it ends.
Without that a sibling carried before it had acquired a slot was dispatched a second
time, doing its work twice and racing its own files.

Three of the commits come from the pre-submit sweep rather than the defect: the twin
assertions the new desk event was missing, an `_await_adjudications` docstring still
saying nothing else in the graph can run during the wait, and the replan design stating
the drained-round rule as absolute when it now holds only for the signals that stop work.

Scoped deliberately to `continue`. `abandon` shares the same wait, but shortening it buys
nothing: an abandoned node's dependents cascade to `skipped` and none could have been
dispatched sooner. A test pins that as a decision rather than an omission.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Full Python suite at the branch head, rebased onto the current `main`:

    uv run --frozen --python 3.12 --all-extras pytest tests/ -q
    19961 passed, 139 skipped, 54 warnings in 386.56s

Zero failures. `--all-extras` is load-bearing: without it the deck-engine tests are not
collected, and running with only `raven` on `PYTHONPATH` resolves `raven_ppt` out of a
different checkout, which reports failures belonging to neither side.

The rebase took on twenty commits and none of them touch the five files changed here;
`git range-diff` against the pre-rebase range reports all four commits replaying
unchanged.

Two regression tests carry the behaviour, both in `tests/test_subagent_dag_runner.py`:

`test_a_continued_node_runs_again_while_a_sibling_is_still_running`
deadlocks without the fix, because only the continued node's second attempt releases the
sibling. Red before it (asyncio TimeoutError at the 15s guard), green after, in 1.97s.

`test_a_carried_group_queued_for_a_slot_is_not_dispatched_twice`
drives two independent nodes through a single slot and asserts the queued sibling runs
once. Without the ready-set guard it records the backend calls `['a', 'b', 'b', 'a']`.

Two more pin the disposal of a carried task, the property that has no owner but the two
`_reap_carried` calls:

`test_an_outer_cancellation_reaps_a_handed_on_task`
`test_a_replan_reaps_a_handed_on_task_before_it_records_them_cancelled`

Both drive one node to a continue and another to an unanswered suspension, so the loop
parks in `_await_adjudications` holding the carried task alone -- outside any round, which
is the only state in which those calls are what cancels it. Each reddens for its own call
site and neither for the other, checked by deleting them one at a time and together. They
are the siblings of `test_run_dag_cancel_skips_unfinished_and_reaps_in_flight_task`, which
proves the same property for a task inside the round.

Gates the GitLab pipeline does not run:

    ruff format --check raven/ tests/                          1487 files already formatted
    ruff check raven/ tests/                                   All checks passed
    lint-imports                                               Contracts: 9 kept, 0 broken
    python scripts/check_source_language.py origin/main...HEAD exit 0
    python scripts/check_commit_messages.py origin/main..HEAD  exit 0
    make check-large-files                                     exit 0

Six commits: the fix, three sweep follow-ups, the ready-set guard, and the reap tests.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing docs: the change is internal to the DAG scheduler and alters no tool
description, prompt asset or RPC shape.

## Risk

Behaviour change: a continued node is re-dispatched as soon as its answer lands rather
than after its round drains. Nothing else about the decision moves -- the continuation
budget, the message handed to the node, and the node's status transitions are untouched.
`dag_node_updated` for the continued node fires earlier; those events already interleave
across concurrent nodes, so no consumer gains an ordering it did not already tolerate.

Known gaps, disclosed rather than fixed:

- One bounded latency replaces the unbounded one: when a handed-on task is running and a
  second node is suspended without an answer, the loop parks in the adjudication wait and
  does not observe that task completing until an answer lands or the wait times out.
  That state was previously unreachable, and the wait it replaces had no bound at all.

Rollback: revert the four commits. No migration, no persisted state, no configuration.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-09 12:26:20 +00:00
0xKT
c726d2d81a feat(ppt-engine): compaction on for deck runs, with a deck ledger under each summary
## Summary

Raven-PPT shipped with no `compaction` section, so the host's context compaction was off for deck runs: a 20-page deck once reached 450k tokens a call (62M input tokens over 137 turns), because nothing shrank the transcript short of a ceiling it never met.

Two changes, both in the product and the plugin; nothing on the host moves.

1. `agents/raven-ppt/config.json` turns the host's compaction on: `compaction.enabled: true`, `triggerRatio: 0.85`. Every other threshold (reserved tokens, tail budget, image byte window, tool-result handling) stays the host's default. The authoring skill is `always: true` and lives in the protected system prefix, so it is never compacted away. The swap ledger in the launcher tests records the row.
2. The ppt-engine plugin's `before_iteration` hook appends a deck ledger under each compaction summary through the `append_note` grant, once per summary (the ledger's first line is the marker; a later compaction gets its own). The ledger holds what a generic, code-shaped summary loses for a deck:
   - the user's own words, verbatim: every user message and every `ask_user` answer from the session record (the host's untrusted-data wrapper stripped), because they are the deck's requirements and a summary paraphrases them; capped at 24k characters with an explicit note when the cap is hit;
   - the deck's state as the tools left it on disk, each with its path: brief (language, audience, page budget, notes, forbidden list), template, sources count, figure catalogue, every outline page's claim and layout, the reader's still-open findings (up to 30, with seen counts), published count and the last refusal.

Design decision D34 in `plugins-dist/ppt-engine/docs/ppt-raven-design.md`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run --frozen ruff check plugins-dist/ppt-engine/raven_ppt/plugin/ledger.py plugins-dist/ppt-engine/raven_ppt/plugin/hook.py tests/test_ppt_engine_ledger.py tests/test_agents_ppt_launcher.py
uv run --frozen pytest tests/test_ppt_engine_ledger.py tests/test_ppt_engine_plugin.py tests/test_agents_ppt_launcher.py -q
# 75 passed
```

Replay on a recorded deck session (the 34-page self-evolving-skills deck from 2026-09-07: 314 messages, about 267k estimated tokens, one `ask_user` with two questions, three later user turns). The host's own `select_split` / `render_transcript` / `build_compacted` were run as the loop runs them, the summary written by `z-ai/glm-5.3-flash` through OpenRouter with the host's `SUMMARY_INSTRUCTIONS`, then the ledger built from the session record and the deck folder on disk:

| requirement in the transcript | in the host's summary alone | in the ledger |
|---|---|---|
| page limit 25-30 (later 33 accepted) | lost | verbatim, plus the brief note the tool wrote |
| all simplified Chinese | lost | verbatim |
| minimal template style (white, one accent, no gradients) | lost | verbatim, plus `brief forbids` lines |
| ask_user answer "33 pages is fine" | lost | verbatim |
| ask_user answer "compact matrix layout" | lost | verbatim |
| source HTML path | lost | verbatim |
| feedback "the cover illustration is ugly" | kept | verbatim |
| output path | kept | verbatim |
| outline (34 pages), 90 open reader findings, template, 17 sources, figure catalogue | not present | listed with paths |

The summary the model produced was 314 -> 22 messages (about 267k -> 13k estimated tokens) and read partly as an echo of the transcript's tail rather than a brief, which is the host's summary quality with this model and outside this MR; the ledger is what makes the deck safe against it.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Behaviour change: deck runs now compact at about 85 percent of the window; a summary call is paid for when that happens. The ledger adds one note (a few thousand characters, bounded) after each compaction.
- The ledger reads only files the engine's own tools write, every read guarded; a half-written deck yields a shorter ledger, never a failed turn.
- Rollback: revert the squash commit; no persisted state is involved.

## Related Issues

N/A
2026-09-09 12:25:42 +00:00
Tchen
898b23e0f7 feat(ppt-engine): compaction on for deck runs, with a deck ledger under each summary
## Summary

Raven-PPT shipped with no `compaction` section, so the host's context compaction was off for deck runs: a 20-page deck once reached 450k tokens a call (62M input tokens over 137 turns), because nothing shrank the transcript short of a ceiling it never met.

Two changes, both in the product and the plugin; nothing on the host moves.

1. `agents/raven-ppt/config.json` turns the host's compaction on: `compaction.enabled: true`, `triggerRatio: 0.85`. Every other threshold (reserved tokens, tail budget, image byte window, tool-result handling) stays the host's default. The authoring skill is `always: true` and lives in the protected system prefix, so it is never compacted away. The swap ledger in the launcher tests records the row.
2. The ppt-engine plugin's `before_iteration` hook appends a deck ledger under each compaction summary through the `append_note` grant, once per summary (the ledger's first line is the marker; a later compaction gets its own). The ledger holds what a generic, code-shaped summary loses for a deck:
   - the user's own words, verbatim: every user message and every `ask_user` answer from the session record (the host's untrusted-data wrapper stripped), because they are the deck's requirements and a summary paraphrases them; capped at 24k characters with an explicit note when the cap is hit;
   - the deck's state as the tools left it on disk, each with its path: brief (language, audience, page budget, notes, forbidden list), template, sources count, figure catalogue, every outline page's claim and layout, the reader's still-open findings (up to 30, with seen counts), published count and the last refusal.

Design decision D34 in `plugins-dist/ppt-engine/docs/ppt-raven-design.md`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run --frozen ruff check plugins-dist/ppt-engine/raven_ppt/plugin/ledger.py plugins-dist/ppt-engine/raven_ppt/plugin/hook.py tests/test_ppt_engine_ledger.py tests/test_agents_ppt_launcher.py
uv run --frozen pytest tests/test_ppt_engine_ledger.py tests/test_ppt_engine_plugin.py tests/test_agents_ppt_launcher.py -q
# 75 passed
```

Replay on a recorded deck session (the 34-page self-evolving-skills deck from 2026-09-07: 314 messages, about 267k estimated tokens, one `ask_user` with two questions, three later user turns). The host's own `select_split` / `render_transcript` / `build_compacted` were run as the loop runs them, the summary written by `z-ai/glm-5.3-flash` through OpenRouter with the host's `SUMMARY_INSTRUCTIONS`, then the ledger built from the session record and the deck folder on disk:

| requirement in the transcript | in the host's summary alone | in the ledger |
|---|---|---|
| page limit 25-30 (later 33 accepted) | lost | verbatim, plus the brief note the tool wrote |
| all simplified Chinese | lost | verbatim |
| minimal template style (white, one accent, no gradients) | lost | verbatim, plus `brief forbids` lines |
| ask_user answer "33 pages is fine" | lost | verbatim |
| ask_user answer "compact matrix layout" | lost | verbatim |
| source HTML path | lost | verbatim |
| feedback "the cover illustration is ugly" | kept | verbatim |
| output path | kept | verbatim |
| outline (34 pages), 90 open reader findings, template, 17 sources, figure catalogue | not present | listed with paths |

The summary the model produced was 314 -> 22 messages (about 267k -> 13k estimated tokens) and read partly as an echo of the transcript's tail rather than a brief, which is the host's summary quality with this model and outside this MR; the ledger is what makes the deck safe against it.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Behaviour change: deck runs now compact at about 85 percent of the window; a summary call is paid for when that happens. The ledger adds one note (a few thousand characters, bounded) after each compaction.
- The ledger reads only files the engine's own tools write, every read guarded; a half-written deck yields a shorter ledger, never a failed turn.
- Rollback: revert the squash commit; no persisted state is involved.

## Related Issues

N/A
2026-09-09 12:25:41 +00:00
TongLi31
7b0af21ac6 fix(agent): bundle ripgrep and report incomplete grep searches
## Summary

Fix the shared grep tool on main, without Raven-Code migration or workspace-gate changes.

- Add ripgrep-bin >=15.2.0,<16.0.0 as a normal runtime dependency and lock its binary distributions.
- Resolve rg from the running Python environment's scripts directory, then PATH, so direct virtualenv/ACP launches do not require shell activation.
- Keep the Python fallback. When its existing traversal deadline expires, report an incomplete search and retain partial matches in content, files_with_matches and count modes. Never report an unfinished empty search as "No matches found."
- Preserve the tool schema, path restrictions, system-root guard and completed-search behavior. Existing fallback tests still explicitly exercise the fallback after rg is installed.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

On a fresh uv environment from this branch:

- `uv sync --locked --extra dev`
- `uv run --locked --extra dev pytest tests/test_search_tools.py tests/test_file_search_traversal_guard.py tests/test_filesystem_tools.py tests/test_filesystem_write_modes.py tests/test_tool_registry_execute.py tests/test_agents_research_tools.py -q`: 158 passed.
- `uv run --locked ruff check raven/agent/tools/file_search.py tests/test_search_tools.py tests/test_file_search_traversal_guard.py`: passed.
- `uv run --locked ruff format --check raven/agent/tools/file_search.py tests/test_search_tools.py tests/test_file_search_traversal_guard.py`: passed.
- `uv lock --check`, `git diff --check`, and the commit-message, source-language and large-file checks passed.

Regression evidence: before the implementation fix, the new assertions produced 7 failures (the empty-PATH binary lookup and incomplete empty/partial results in three output modes). After the fix they pass. The empty-PATH test executes the real installed rg and forbids the Python fallback; deadline tests use a controlled clock rather than sleeping.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Adds the ripgrep-bin binary distribution from PyPI; installation and execution were tested on Linux x86_64. Other platforms require a compatible wheel or source-build support. The bundled executable takes precedence over system rg; the existing rg arguments and ignore behavior are unchanged. The fallback deadline remains cooperative, not an interruptible I/O timeout.

uv regenerated some transitive platform-marker placement in uv.lock; no existing package version was upgraded. Rollback is a revert of this single commit, including the dependency and lockfile changes.

## Related Issues

N/A
2026-09-09 12:25:06 +00:00