51 Commits

Author SHA1 Message Date
Dizhan Xue
14edd7bdb2
feat(*): run the host as an agent manager that hands work, coding included, to sub-agents (#1010)
## Summary

Raven's host acts as the user's agent manager: it delegates production
work, including small edits, to the agent whose ownership or description
fits the task -- or, on a turn bound to a Worker Table, to the worker
whose brief covers it -- keeps track of the task and its instance,
routes follow-ups back to that instance, and gathers the results for the
user. Its own tools support briefing, checking and coordination.

- Add the resident `multi-agent-coding` skill for choosing coding
agents, keeping one instance per task, passing results by file,
isolating concurrent edits in worktrees, and handing over work when an
agent exhausts its quota.
- Keep the new host role authoritative in both the DAG skill's resident
description and its full guide. A single task uses `spawn`, or a
one-node graph when only the DAG dispatch path is available.
- Rename the generic sub-agent to `Raven-General`, with `Raven` and
`raven` retained as compatibility aliases. Instance records, handle
locks, live activity, cancellation, preferences and RPC lookups use the
same canonical identity. Historical spawn and DAG readers normalize
names before liveness checks and serialization, so tasks and transcripts
join the current instance roster. Concurrent alias calls preserve both
turns instead of overwriting one conversation snapshot. The instance
registry reads former-name rows under the current name and, on its next
write, saves every loaded row under it; transcripts, direct chats and
stored graphs stay as written.
- Resolve built-in configuration aliases before merging fields:
`Raven-General` wins over `Raven`, then `raven`, regardless of file
order. Discard older rows wholesale with a warning, preventing stale
model, permission or transport settings from leaking into the selected
row.
- Resume the most recently written message state and instance log across
former names. Saved messages take precedence over an empty chat
directory; equal timestamps prefer the current name and then the most
recent former name. Existing files stay in place. Legacy spawn records,
completed DAG manifests and unfinished graphs remain readable through
the current name.
- Emit the agent roster once at the end of `spawn`'s description. The
registry places `run_subagent_dag` and then `spawn` last, including
after late MCP or session tools. The graph tool renders the roster
itself when `spawn` is unavailable. With a Worker Table bound, dispatch
guidance selects worker labels from the live enum; the roster explains
underlying capabilities. The tool descriptions and resident DAG guide
follow the same rule.
- On a turn bound to a Worker Table, such as a Persona session, `spawn`
and the DAG nodes offer worker labels alone, and a generated Persona
never includes the generic agent. The host role on such a turn routes
production work to the worker whose brief covers it, by label, and puts
work no worker covers to the user, who can add a worker or give the
conversation another Harness, instead of doing it itself or naming an
agent outside the enum. Ordinary turns render byte for byte as before. A
Harness that `load_playbook` binds mid-turn takes effect after the
identity was assembled, so that call's result carries the same routing
as this system's trusted note, after the untrusted fence. The coding
skill picks worker labels on such a turn, for its first pick and for a
quota takeover.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Measured at this head on Linux/Python 3.12.13.

- `UV_NO_SYNC=1 uv run --no-sync pytest -q -n 12`: **33,767 passed, 178
skipped, 2 xfailed; 9 failed and 1 error.** Eight of the failures are
bound to this box and fail the same way on current main, measured the
same day: five provider tests that read the box's proxy environment, a
Node.js path case, a LibreOffice conversion, and a fixture that root can
read anyway. The ninth, with its teardown error, is the `remote_runs`
race tracked in #990; that test passed 5 of 5 runs alone, and its file
58 of 58.
- 13 new regression tests for Worker Table routing: 12 fail against the
previous head's source and all pass now; the thirteenth checks that a
Harness loaded mid-turn leaves the next turn's role plain, and passes on
both by design. The end-to-end case drives a real Persona-bound turn:
both enums offer `research-a` alone, dispatching `Raven-General` is
refused, and the role must name neither that agent nor the roster
fallback.
- Nine single-construct mutants, each applied to an isolated copy of the
tree, each fail at least one new test: the role ignoring the table, the
default path not reading the turn's table, the uncovered-work rule
removed, the generic fallback kept in the worker routing, the coding
skill reverted, and the mid-turn note never added, added on every call,
added without a dispatch path, or phrased apart from the role.
- The host identity on ordinary turns is byte-identical to the previous
head in all 16 rendered shapes: a launched sub-agent process or not,
with or without ownership lines, and every combination of the two
dispatch tools.
- Instance registry probe, run with this head and with main on one
synthetic file: reads leave the file byte-identical; one unrelated
`upsert_spawn` saves every former-name row as `Raven-General`, keeping
the more recently updated of two colliding spellings and the later one
in the file on a tie; main then finds none of those rows under `Raven`
or `raven`.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage.json> diff --base-ref origin/main --threshold 90`, over a
coverage run of 225 test files (8,952 passed): **99.40%**, 167/168
executable changed lines covered.
- `UV_NO_SYNC=1 make lint-types lint-imports check-large-files
check-source-language COMMIT_RANGE=origin/main...HEAD`: passed; all 10
import contracts kept.
- `UV_NO_SYNC=1 make check-commits`: both commit-message validators
passed; each also refused a dangling control commit with a malformed
subject.
- `ruff check` and `ruff format --check` over the branch's Python files
passed. `node ui-tui/scripts/gen-i18n.mjs --check` passed after the
rebase onto #1015, which also changed the i18n catalogue.
- `uv run --no-sync pre-commit run --from-ref origin/main --to-ref
HEAD`: every hook passed except the whole-TUI Prettier hook, which
reports only `newInstancePicker.tsx` and `subagentsHub.tsx`; both are
byte-identical to current main.
- Live-model replay and native Windows/macOS testing were not repeated
for these fixes.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The host delegates work it previously performed itself, including
small scripts. Task-to-instance routing remains prompt-based; a task can
still stay with the host after it has already answered a follow-up
itself.
- In a conversation bound to a Persona or another Harness with workers,
work no worker's brief covers now waits for the user to add a worker or
change the conversation's Harness, rather than being carried out by the
host.
- Old agent spellings remain accepted. The instance registry's first
write after the upgrade, of any row for any session, saves every
former-name row in `subagent_instances.json` as `Raven-General`, and
keeps only the more recently updated row where two spellings of one
instance collide. Builds from before the rename cannot look those rows
up, so reverting the squash commit alone does not restore their instance
bindings: that needs the registry file as it stood before that first
write. Renaming rows back by hand cannot recover a colliding row the
write dropped. Transcripts, direct chats and stored graphs are not
rewritten.
- When multiple generic-agent configuration spellings coexist, only the
newest row contributes overrides. Fields supplied exclusively by an
older row are intentionally discarded.
- Moving the dispatch tools changes the tool-array cache prefix once
after upgrade.
- A plugin participant that removes `spawn` through `select_tools` can
still leave the graph tool pointing at an absent roster, because that
selection occurs after the registry's offer check. No bundled
participant does this.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-10-20 17:56:05 +08:00
Handsome-wzw
615256bf90
fix(*): show a machine's switched-off agent entries, and reconnect them from its sheet (#1019)
## Summary

Reported by the owner on 2026-10-20, on the first day with #1007 on
their site: the Ravenx card said
"No agent connected here yet" while the machine's connect form greyed
Claude Code as "already connected
on this machine". Both were telling the truth by different rules. The
card counted connected entries
only (`agents_on_machine(only_enabled=True)`); the form, and
`subagents.add`'s duplicate rule, counted
every entry of the preset on the machine, switched off or not -- and the
owner had just switched
`Claude Code@ravenx` off. So a switched-off entry was counted nowhere on
the card, blocked the form,
and had no way back from the machine's side.

What changes:

- `machines.list` rows carry `agents_off_here`, the entries on the
machine that are switched off
(the connected ones stay in `agents_here`; both empty when the agents
section or the config file
cannot be read). The card line and the sheet field show them marked
*switched off*.
- `subagents.presets` now means by `on_machines` the machines a
*connected* entry runs on, and adds
`off_on_machines` (machine + entry name) for the switched-off ones. Both
additive.
- The connect form keeps greying a preset the machine has connected, but
offers one it has switched
off back under its own entry -- each switched-off entry as a choice of
its own, by name, since a
machine may hold several explicitly named ones: the option says
*switched off*, the working-directory
field is left out (the entry keeps the one it had), and the button reads
**Reconnect**, which runs the
agents page's own `subagents.toggle enabled=true` on that entry -- same
proof ping, no second entry. The
add is offered only while the machine holds nothing of the preset (the
server refuses it otherwise). The
Reconnect goes through the agents page's own `act`, so the row is marked
joining for the length of the
ping: that page draws it pending and cannot press its own Connect
meanwhile, and the form, reading
`joining` too, greys the entry the other way round -- one ping whichever
side pressed. The card line and
the sheet field read *N on this machine* / *Agents on this machine*,
since they list switched-off
  entries as well. The
write is guarded like the add (opening generation, drawer still current)
and marked pending so a
form reopened meanwhile does not offer it twice; a refusal stays on the
form in the agent's words.
- Docs: machines guide (en + zh), CHANGELOG, CONTEXT.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest tests/test_rpc_machines.py tests/test_rpc_schema_match.py
tests/test_rpc_registration.py -n 4`: 580 passed; `pytest
tests/test_rpc_subagents.py tests/test_ops_connection_add.py
tests/test_raven_config_tool.py -n 4`: 365 passed. The two extended
cases (`agents_off_here` listed apart; `off_on_machines` carries the
entry name while `on_machines` leaves it out) fail with the two handler
changes stashed.
- `ruff check` / `ruff format --check` on the changed Python files:
clean.
- ui-web: `vitest run src/features/extAgents scripts/gates`: 483 passed
(5 new page cases: the third for two switched-off entries of one preset
on one machine, each reconnectable in turn; the fourth and fifth for a
toggle in flight from either side -- the agents page offers no Connect
while the machine's Reconnect runs, and the form greys the entry and
says why while the agents page's Connect runs -- both fail with the
previous store/form; the switched-off preset is pickable and labelled,
no directory field, Reconnect toggles the existing entry and lands back
on the sheet with the lists re-read; a refused Reconnect stays on the
form in the agent's words -- both fail with
`MachineCards.tsx`/`store.ts` stashed; the card/sheet case extended with
a switched-off entry). `tsc --noEmit`, `eslint` on the changed files,
`gen:check` clean; `ui-tui` `lint:rpc` / `lint:i18n` clean.
- `pytest tests/test_docs_site.py`: 16 passed; `mkdocs build --strict`:
clean.
- Full suite: see the comment on this PR.
- Real page: see the comment on this PR.

## Risk

- [x] Security impact considered: the new fields carry entry names
already shown on the agents page; Reconnect is the existing toggle, with
its proof ping and refusals.
- [x] Backward compatibility considered: `agents_off_here` and
`off_on_machines` are new fields (the page maps both defensively);
`on_machines` narrows to connected entries, which only this page reads.
- [x] Rollback path is clear for risky changes: revert the commit;
nothing stored changes shape.

## Related Issues

N/A (follow-up to #1007, from the owner's report on 2026-10-20)

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-20 16:48:09 +08:00
Handsome-wzw
2e390adff6
feat(*): say the next step on a ready machine and connect an agent from its sheet (#1007)
## Summary

Follows #1005, which is now on main; main was merged in after it landed,
so the diff against main is this change alone.

A registered machine whose node is up left the owner with nothing to do:
the card said
"Link up, no agent" and the Agents tab did not mention the machine. The
way to connect
an agent on it (the preset sheet's *Runs on*) is hidden exactly where it
is needed most:
a preset already connected on this computer is withheld from the roster,
and a preset not
installed here shows only install instructions. Measured on the owner's
own setup
(2026-10-19): two machines registered from the page, both reached and
with the node,
and no path on the page to put Claude Code on either.

What changes:

- `machines.list` rows carry `agents_here`, the connected agent entries
that run on the
machine (`agents_on_machine(only_enabled=True)`; a malformed agents
section lists none).
The card says "N connected here: ..." or "No agent connected here yet";
the state line
now reads "Link up, no agent running", so "running now" and "connected
here" are two lines.
- A ready machine's sheet says the next step and offers **Connect an
agent on this machine**:
a form in the same drawer with the preset, its working directory there,
and Connect. It
calls the existing `subagents.add` with `machine`, `remote_cwd` and the
name fixed to
`<preset name>@<machine id>`, so the proof ping and every refusal are
the add's own.
- New read-only `subagents.presets`: the presets a machine's node can
start (acp, cli),
listed whether or not an entry already holds them, with `connected_here`
and
`on_machines` so the form greys the one this machine has and says when
this computer
has it too. Built-in agents, Mirasim shims and openai endpoints are not
listed.
- The Agents tab's "not installed" sheet also offers *Runs on* when ssh
machines are
registered, with a Connect button once a machine is picked: running on a
machine does
  not need the agent on this computer.
- New read-only `machines.local_keys`: the private keys this computer
has, as `~/.ssh/<name>`
(ssh's default identities in ssh's order, then any file with a `.pub`
beside it). The
register form fills the key path with the first and offers the rest,
instead of a
placeholder naming `id_ed25519` to an owner whose key is `id_rsa`; with
none found it
  says so and points at the password login.

- One preset once per machine, held by the server whichever way the add
arrives: `subagents.add`
without a `name` now refuses a preset the machine already has (judged on
preset + machine, before
the ping), naming the entry there and whether it is connected or
switched off; a named add is a
second entry on purpose (two working directories, say). The page's
connect form leaves the name to
the server's default (`<preset>@<machine id>`) rather than fixing its
own spelling, so the page and
a chat can no longer put one agent on one machine twice between them
(the owner's setup had the two
  routes spelling the default differently, 2026-10-19).

Contract (`rpc-schema/openrpc.json`), models, both generated clients and
`i18n/messages.json`
are updated; the offline fixture library answers the two new methods and
the new field, and refuses
the duplicate the way the server does.
Docs: the machines guide (en + zh), CHANGELOG, CONTEXT.

**Fifth round, after the review draft on LivXue's board (four findings,
all reproduced) and the bot's inline note.** `machines.list` read the
agents section for every row and caught only the pydantic error, so a
`config.json` that does not parse at all (a `//` comment, a trailing
comma) failed the whole listing with `internal_error` while the gateway
ran on: `_agents_here` now treats the loader's `ConfigReadError` the
same way (nothing listed; the agents page says why), and the test is a
real unparseable file. The register form proposed `id_ed25519` before
`id_rsa` and called that ssh's order; `ssh -G` lists `id_rsa` first on
OpenSSH 8.9, 9.6 and 9.9, so an owner with both was proposed the key ssh
itself tries second: `DEFAULT_IDENTITIES` follows the measured order, a
test pins it against the ssh of the computer running it, and the guide's
two sentences say so. A row that exists, switched off, with its command
missing here, was offered **Connect on <machine>** from its sheet while
the press sent the row's own `toggle`, which carries no machine: only a
preset not yet on the roster is offered one, as the other *Runs on*
already did. An add still running on a machine was unknown to a form
opened again on that machine, so the preset could be pressed twice (a
second proof prompt, then the duplicate refusal, a toast saying
connected and a form saying not): the adds in flight are kept by the
name each lands under (`mconnectPending`) and the form greys that preset
(*being connected on this machine right now*). The contract's
`subagents.presets` description names the add's default
`<preset>@<machine id>` now, as the bot noted. The branch also carries
#1005's fourth commit (merged) and main up to #1001.

**Sixth round: merged onto #1005's head (`9dbef0874`) and LivXue's F1
fixed.** The merge had three conflicts, all the same shape -- this PR's
addition beside what #1005 changed (the `auth` field in the machine
sheet, `agents_here` in the fixtures, the new register assertions) --
and keeps both sides; the superseded password-used-once code
(`generate_key`, `install_public_key`, `revoke_public_key`,
`forget_key`, the remote scripts) is gone from the diff, so a review of
this PR no longer grades that design. F1: a connect form open across a
write reads `presets` again when the add lands -- whichever opening it
is -- so a preset just connected on that machine reads as taken instead
of inviting a second add the server refuses; the case that pinned the
wrong end state moved with it. F2 (four ui-web files reformatted to the
root prettier style) is withdrawn in 1834c68ee: each file is the #1005
text with this PR's own hunks replayed onto it (a three-way merge of the
base, the base formatted the way the reformat had been, and this
branch's head; ten overlaps resolved by hand, the maintainer edits'
lines included), so what remains in them is this PR's change -- the four
files' lines against #1005 go from 173/211/281/1694 to 137/44/206/590.
Check used: formatting the result with the root config reproduces the
previous head except where that head itself kept hand style, so nothing
but wrapping changed.

Verification, sixth round: related Python `1068 passed` over the twelve
files this PR touches or affects, diff coverage 96.13% (348/362
executable changed lines); full suite `-n 6` 33386 passed, 3 failed --
the LibreOffice case and the two simulation cases this Mac always fails
(run before the F1 commit, whose change is ui-web only); ui-web `vitest
run` 4121 passed (250 files, gates included), `tsc --noEmit` and
`eslint` clean, `gen:check` 247 methods; ui-tui `lint:i18n` and
`lint:rpc` clean; the three range gates over `origin/main..HEAD`.

**Also merged: #1005's c0c822294** (Oncall campaigns run on a password
machine through trunk's askpass; the helper answers a password prompt
only; `--password-stdin` implies `--auth password`) and the three
maintainer edits on this branch (68ea01189, e5884ddf1, dac563aab), all
clean.

**Seventh round: two findings from the review of 91952baed.** (1) A
configless Raven held a kept machine password unscrubbed: `held_secrets`
returned early when `config.json` was missing, before it read
`keys/<id>.password` -- and those live under the Raven home
independently of any config, so a tool printing that file passed the
password to the model verbatim (gloryfromca, inline). An absent config
is now an empty config-credential set rather than the end of the call;
the case is pinned with no config file at all, and the existing "no
config holds nothing to scrub" case still holds for the config's own
credentials. (2) CI shard 3/4 failed on
`tests/test_node_stderr_replay.py::test_a_reason_written_on_stderr_while_the_link_was_down_reaches_the_run_watching_it`
("did not happen in time", then a leftover agent in its teardown) and
`tests/test_subagent_cli_remote.py::test_...` -- both stand-in-machine
timing cases this branch does not touch, the same class as the ones
recorded for #965/#979/#983; 135 passed over those three files locally
in one run, and the job passes on rerun.

Verification: `tests/test_held_secrets_machine_passwords.py` and
`tests/test_security_untrusted_context.py` 42 passed; the other three
files that read the scrubber 223 passed; `ruff check`, `ruff format
--check` and `ty check` clean on the two changed files; the three range
gates over `origin/main..HEAD`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest tests/test_rpc_machines.py tests/test_ops_connection_add.py
tests/test_rpc_subagents.py tests/test_rpc_schema_match.py
tests/test_rpc_registration.py -n 4`: 768 passed (7 new: `agents_here`
incl. the malformed-section case, `local_keys` order and exclusions at
both layers, `subagents.presets` membership and placement).
- Full suite `pytest -n 6`: see the last comment on this PR for the
count (run at the head).
- `ruff check` / `ruff format --check` on the changed Python files:
clean.
- Duplicate-on-machine refusal: 2 new cases in
`tests/test_rpc_subagents.py` (default-named second add refused before
any ping, naming the entry and its state; named add allowed; another
machine allowed; switched-off entry named), both fail with the handler
change stashed; related groups 928 passed after it.
- ui-web: `vitest run` 4006 passed (250 files, page gates included; 7
new page cases: the card line and the sheet's next step, the connect
form end to end incl. a refusal and the nothing-to-connect case, key
prefill with and without keys, an absent agent connected onto a machine
from its own sheet, and no machine offered when none is registered);
`tsc --noEmit` clean; `eslint` clean on the changed files; `gen:check`
matches the contract.
- ui-tui: `gen:rpc` and `gen:i18n` regenerated; `pytest
tests/test_docs_site.py` 16 passed; `mkdocs build --strict` clean.
- Real machine, isolated gateway (own Raven home and registry copy,
channels off, the branch's node installed in its own directory on the
test machine), real browser: the ready machine's sheet shows the next
step and the button; the form lists 12 startable presets; Connect of
Claude Code on the machine landed in 9 s and the card then reads "1
connected here: Claude Code@<id>"; the register form was prefilled with
this computer's `~/.ssh/id_ed25519` with `id_rsa` as the other choice;
an agent absent here (OpenClaw) connected onto the machine from its own
sheet was refused in 1 s with the agent's own sentence ("the agent's
command was not found on machine ...") shown in place. Gateway stopped
by pid, the node directory and daemon on the machine removed afterwards.
- Previews of the five surfaces were shown to the owner before the
change and approved.
- Fifth round, red before and green after: `machines.list` over a real
`config.json` that is not JSON lists the machine with no agent
(`test_list_lists_no_agent_over_a_config_file_that_does_not_parse_rather_than_failing`;
through the real `Dispatcher` with a scratch home and registry, the
previous handler answered `-32603 internal_error`, this one `[('box',
[])]`); `local_keys` order at both layers, plus
`test_the_default_identities_are_in_the_order_this_computer_s_ssh_offers_them`
against `ssh -G -F /dev/null` (this Mac, OpenSSH 9.9: `id_rsa, id_ecdsa,
id_ecdsa_sk, id_ed25519, id_ed25519_sk, id_xmss, id_dsa`; the same on a
registered Linux machine, OpenSSH 9.6); the page: a configured,
switched-off, missing row gets no machine field and no *Connect on*
(`offers a row that exists no machine...`), and a form opened again on a
machine while an add still runs there greys the preset, keeps the other
one pressable and sends no second add (`greys a preset whose add on the
machine is still running...`) -- both fail with the three page sources
stashed. The six Python files 821 passed, diff coverage of the changed
lines 97.50%; ui-web `vitest run` 4119 passed (page gates 216 included),
`tsc`, `eslint` on the feature, `gen:check` (247 methods), `npm run
build` with both boot goldens; ui-tui `gen:rpc`/`gen:i18n`, `lint:i18n
-- --check`; `ty check` on the changed modules; `pytest
tests/test_docs_site.py tests/test_living_docs.py` 18 passed, `mkdocs
build --strict` clean; the three range gates; full suite (`-n 6`) 33353
passed, 4 failed: the three this Mac always fails (a LibreOffice case,
two simulation cases) and one stand-in-machine timing case in
`tests/test_subagent_cli_remote.py` (the record not yet written when the
test reads it) that passes 3 of 3 runs alone and touches no file this
branch changes.

## Risk

- [x] Security impact considered: `machines.local_keys` returns key
*paths* on this computer, to this computer's own page, never key
material; `subagents.presets` carries no credential; nothing new leaves
the host. The connect form reaches the machine only through the existing
`subagents.add` and its proof ping.
- [x] Backward compatibility considered: `agents_here` is a new required
field on `machines.list` rows (the page maps it defensively); the two
new methods are additive; an older page against this server ignores
both.
- [x] Rollback path is clear for risky changes: revert the commit; no
stored data changes shape.

## Related Issues

N/A (follow-up to #983 and #1005, from the owner's walkthrough on
2026-10-19)

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: MiniMax (minimax-m3.1-flash-free) <noreply@minimax.io>
2026-10-20 14:24:04 +08:00
Handsome-wzw
7376de6828
feat(*): a machines hub tab, and registering a machine with a password Raven keeps (#1005)
## Summary

Add a Machines tab to the Agent Hub and let owners register an SSH
machine with either a key or a password.

- The Machines tab reuses the existing machine cards, details and
registration form. The hub remembers the last selected module. Machine
details show the login method, and removal explains that the locally
stored password is deleted.
- Password registrations keep a 0600 credential file under the Raven
home's 0700 keys directory. Registration, node installation, file tools,
tunnels and Oncall use the shared SSH askpass transport. Passwords are
excluded from command lines, registry rows and responses, and included
in held-secret scrubbing.
- Password connections authenticate independently, disable inherited
jump hosts, and answer only the registered account's password prompts.
Oncall cannot redirect a stored password to another endpoint. Windows
rejects this mode before creating credentials because its OpenSSH cannot
execute Raven's POSIX helper. Key authentication remains supported as
before.
- Concurrent registrations preserve the successful request's password.
Each probe uses its own temporary 0600 credential file, which is
scrubbed and removed afterwards. Password writes receive a file
identity; cleanup checks ownership under the writer's lock. Successful
registration checks duplicates again and publishes its password with the
registry row under that lock. Both completion orders, RPC cancellation
and CLI aborts are covered.

The RPC contract adds optional `MachineRow.auth` and password
registration parameters. English and Chinese machine documentation, the
runtime context and changelog describe the resulting behavior.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

After rebasing onto the latest main:

```bash
uv run --no-sync pytest tests/test_ops_connection_add.py tests/test_ops_connections.py tests/test_ops_transport.py tests/test_cli_ops_connection_commands.py tests/test_rpc_machines.py tests/test_held_secrets_machine_passwords.py tests/test_acp_remote.py tests/test_node_client.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_import_cycle_budget.py -q -n 0 --tb=short --cov=raven.ops.connection_add --cov=raven.ops.transport --cov=raven.cli.ops_connection_commands --cov=raven.rpc.methods.machines --cov-report=json:/tmp/pr1005-fix-coverage.json
```

Result: 932 passed, 6 skipped. The skipped cases require zsh, tcsh or
csh, which are absent on this Linux host.

- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
/tmp/pr1005-fix-coverage.json diff --base-ref HEAD^ --threshold 99`:
100% (94/94 executable changed lines) for the concurrency fix.
- `uv run --no-sync pre-commit run --from-ref HEAD^ --to-ref HEAD`:
passed, including formatting, file checks and private-key detection.
- `uv run --no-sync ty check raven/ops/connection_add.py
raven/ops/transport.py raven/cli/ops_connection_commands.py
raven/rpc/methods/machines.py`: passed.
- `uv run --no-sync lint-imports`: all 10 contracts passed.
- `uv run --no-sync python scripts/check_source_language.py
origin/main..HEAD`, `uv run --no-sync python
scripts/check_commit_messages.py origin/main..HEAD`, `make
check-large-files`, and `git diff --check origin/main...HEAD`: passed.

The original RPC failure/cancellation and CLI-abort reproductions failed
before the fix. Additional regressions cover the reverse completion
order, probes with competing passwords, temporary-secret scrubbing,
duplicate registration and failed registry writes. The complete rebased
source was compared with the original PR plus main, and the concurrency
fix retained its patch identity.

## Risk

- [x] Security impact considered: stored and temporary passwords are
owner-readable and scrubbed from tool output. Unknown SSH host keys are
accepted on the first connection, as before; the guides explain
verifying the host from a terminal first on an untrusted network.
Passwords shorter than six characters or containing CR/LF are rejected.
- [x] Backward compatibility considered: key registration remains the
default and `MachineRow.auth` is optional. Password publication and
cleanup now use file locks and atomic replacement; temporary probe files
are removed when the probe ends.
- [x] Rollback path is clear: revert this PR to restore the earlier
registration flow. Removing a machine deletes the password stored under
its id. Existing password rows require migration to key authentication
if reverting the feature.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: MiniMax (minimax-m3.1-flash-free) <noreply@minimax.io>
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 12:23:40 +08:00
江国庆/Forrest
55c145ceb8
feat(*): probe the bug report receiver and gate uploads with a challenge (#1015)
## Summary

Adds two things to bug reporting, on the Bug Report Receiver and on the
Raven clients that send to it.

**A hello probe.** The receiver answers `GET /v1/hello` with its service
name and protocol version, and Raven asks it before a report starts, so
a person learns that the receiver is down (or that the URL leads to
something else) before spending a redaction review on the report:

- `raven trajectory report-bug` and the trajectory browser's *Report a
bug* probe before collecting anything. On a TTY without `--yes` they ask
whether to continue (default no; declining collects and pins nothing).
With `--yes` they say they are continuing and keep the package if the
upload then fails. Without a TTY and without `--yes` the existing rule
still refuses first.
- The WebUI's *Report a problem* wizard asks the new RPC
`bugreport.probe` when it opens. When the receiver does not answer, the
report does not start: the wizard shows the reason and *Check again*,
and the primary button becomes *Continue anyway*, so going on is the
reader's explicit choice and never a dead end. A start pressed while the
check runs waits for it (at most 8 s) and then checks the form again.
- The probe runs under one deadline for the whole exchange (5 s), does
not follow redirects, and the new requests never echo credentials or
query tokens from a redirect target.
- `/healthz` stays as the operator's liveness check. It was not reused
for the probe because anything alive answers ok there, and it is not
under a base URL's path prefix.

**An upload challenge.** The receiver now takes only uploads that answer
its Upload Challenge: the client gets a challenge from `POST
/v1/challenge` and sends back an HMAC, under a constant built into
Raven, of the challenge and the package's report id, sha256 and size.
Key decisions:

- Shallow by design. The goal is that an HTTP client that is not Raven,
written without reading Raven's source, cannot upload. The constant is
published with the (public) source and is not configurable, so there are
no credentials to manage. The docs say so, with a table of what the gate
stops and what it does not.
- Stateless. Challenges are sealed with a per-process random key, are
valid for 300 s and are not consumed; a restart retires them all.
Replaying a captured upload as it was only re-delivers the same package,
which the receiver already answers as a duplicate; replaying it with
another package fails because the answer is bound to the package.
- Checked from the headers alone, after the existing header checks and
before the daily budget is reserved: a refused upload gets 401
(`unauthorized` or `challenge_expired`) with `WWW-Authenticate:
Raven-Challenge`, writes nothing, and still spends a rate-limit token.
Package checks, rate limits, budgets and timeouts are unchanged.
- The client fetches the challenge inside the upload's existing total
deadline. A 404 there means a receiver older than this Raven; a 401 is
retryable only when the challenge expired.

Uploading has not shipped in a release yet (it is under `Unreleased` in
the CHANGELOG), so no released client breaks. A receiver deployed from
main must be upgraded together with its clients, and a proxy in front of
it must forward all of `/v1/`; the docs and the CHANGELOG entry say so.

Commits, in order: the hello route and the client probe; the challenge
end to end (new `raven/trajectory/receiver_auth.py`, and the Upload
Challenge term in `CONTEXT.md`); the CLI probe; the WebUI probe (RPC,
wizard, i18n, regenerated ui-web and ui-tui bindings); a fix that checks
the form again after the wizard waits on the probe; the docs in both
languages; integration tests against real processes.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run --no-sync pytest tests/test_trajectory_receiver_auth.py
tests/test_trajectory_receiver.py tests/test_trajectory_upload.py
tests/test_rpc_bugreport.py tests/test_cli_trajectory_commands.py
tests/test_cli_trajectory_browse.py
tests/test_trajectory_report_session.py tests/test_rpc_schema_match.py
-q --idle-ceiling-strict --cov=raven --cov-branch
--cov-report=json:coverage.json`: 1190 passed.
- `make coverage-diff COVERAGE_BASE_REF=origin/main`: 99.53% of changed
lines (213/214), threshold 90%.
- `uv run --no-sync pytest
tests/integration/test_bug_report_upload_e2e.py
tests/integration/test_web_bugreport_e2e.py -m integration -n 0 -q`: 10
passed, three runs in a row.
- `uv run --no-sync pytest tests/ -q`: 102 failed, 33527 passed. All 102
are in 14 files this PR does not touch (code tools, code flow, subagent
probe and backends, a2a, whatsapp, provider config and rates, security
network, everos, tracing command), and the same 102 fail on the base
commit (checked on a worktree of `main`; the two failure sets are
identical).
- `make lint-python`, `make lint-imports` (10 contracts kept), `deptry
raven`: pass. `make lint-types`: only two `pyflakes.checker` unresolved
imports in
`agents/raven-code/plugins/code-flow/code_flow/tools/diagnostics.py`,
because pyflakes is not installed in the local environment (CI installs
all extras); every changed Python file passes `ty check`.
- `mkdocs build -f docs-site/mkdocs.yml --strict`: pass.
- ui-web: `gen:check`, `lint` (0 errors), `type-check`, `npx vitest run
--maxWorkers=8` (251 files, 4176 tests), `build` + `build.py` +
`check-page` / `check-css` / `check-class-namespace`: pass.
- ui-tui: `lint`, `lint:rpc`, `lint:i18n`, `type-check`, vitest (2242
tests), `build`: pass.
- `make check-source-language check-large-files check-commits`: pass.
- Manual, on real processes: a receiver plus curl (hello and challenge
answer; an upload without an answer, or with a forged one, gets 401 and
nothing is stored); the CLI (an upload while the receiver runs; while it
is down, the warning, continuing and the kept package; declining on a
TTY leaves nothing); a real gateway walked with Playwright in English
and Chinese at 1280 and 360 px (the warning while the receiver is down,
Check again after it starts, a report through to its receipt, no page
errors, no overflow).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes:

- With an upload URL configured, `report-bug` and the browser's *Report
a bug* start with a probe of at most 5 s, and ask one more question when
the receiver does not answer. The WebUI wizard shows a warning and
*Continue anyway* in that case.
- The receiver refuses uploads that do not answer the challenge. A
receiver deployed from main must be upgraded together with its clients
(no released client uploads yet). A proxy that only forwarded
`/v1/bugreports` must forward all of `/v1/`.
- No change to package formats, `record.json`, or anything stored on
either side.

Rollback: revert the squash commit; there is nothing to migrate.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-20 10:22:53 +08:00
Dizhan Xue
6fbc573876
feat: enable MCP lending on Windows hosts (#972)
## Summary

Windows hosts can lend granted MCP servers to local and machine-based
ACP agents through authenticated loopback endpoints. Hosts with Unix
sockets retain that transport. Server definitions, upstream credentials,
sandbox policy and disabled-tool enforcement stay on the host.

- Share one connector between the local bridge and node client. Bind TCP
endpoints only to 127.0.0.1 and authenticate with a fresh per-endpoint
token before opening an upstream.
- Read inherited Windows stdin pipes through a daemon worker with
backpressure. Acknowledge each chunk on the event loop before reading
another, so neither the reader buffer nor queued callbacks can grow
without bound.
- Cancel both bridge pumps on exit, allow up to one second for the
endpoint connection to flush, then abort the transport if it still holds
unsent bytes. A peer that stops reading cannot hold failure or
cancellation cleanup indefinitely.
- Resolve this installation's raven.exe, preserve task and session
isolation, hide endpoint credentials from grant representations, and
redact them in bridge diagnostics and session-start journal frames.
Remote dispatches do not receive the host's loopback token.
- Update capability reporting, CLI help, the remote ACP entry's
`machine` docstring, CONTEXT.md and the English/Chinese collaboration
guide. Add Windows CI coverage and exercise both Unix and TCP endpoints
on Linux.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Docker Linux, Debian Bookworm with procps and git, Python 3.12.12,
--init, a native Linux source copy, and a frozen environment: 783
related unit tests and all 3 real-server integration tests passed. The
source and test files were compared with the committed versions before
publication.

```sh
uv sync --frozen --extra dev
uv run --frozen --no-sync pytest tests/test_mcp_transport.py tests/test_mcp_bridge.py tests/test_mcp_endpoint.py tests/test_cli_mcp_commands.py tests/test_subagent_acp_mcp_lifecycle.py tests/test_node_client.py tests/test_acp_journal.py tests/test_subagent_acp_remote.py tests/test_subagent_manager.py tests/test_subagent_mcp_grant.py tests/test_subagent_third_party.py -q -n 4 --dist loadfile --tb=short --cov=raven --cov-report=json:/out/coverage.json --junitxml=/out/units.xml
uv run --frozen --no-sync pytest tests/integration/test_mcp_bridge_e2e.py -m integration -q -n 0 --tb=short --junitxml=/out/integration.xml
```

PR diff coverage is 98.26% (169/172 changed executable lines), above the
unchanged 90% threshold. On the host checkout, `$coverageReport` points
to the JSON exported by that Linux run:

```powershell
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $coverageReport diff --base-ref origin/main --head HEAD --threshold 90
```

Native Windows, Python 3.13.12, a separate frozen environment and
PYTHONUTF8=1: all 21 bridge unit tests passed; all 8 integration cases
passed, comprising 3 MCP bridge cases and 5 native command-line cases
retained in the Windows CI job.

```powershell
uv run --frozen --python 3.13 --no-sync pytest tests/test_mcp_bridge.py -q -n 0 --tb=short
uv run --frozen --python 3.13 --no-sync pytest tests/integration/test_mcp_bridge_e2e.py tests/integration/test_acp_mirasim_real_process.py -m integration -q -n 0 --tb=short
```

Seven regression cases cover paused input, resuming consumption, closing
a paused worker, a busy event loop, and teardown with pending TCP
writes: while the endpoint stays stalled, while it drains those writes,
and when a second cancel arrives after that drain finished. Aborting the
endpoint after a close that finished was wrong: the selector transport
has already run connection_lost and dropped its loop, and its
_force_close returns early only on _conn_lost, which a drained close
does not raise, so the bridge exited 1 at teardown on Linux and macOS.
The abort now depends on whether the endpoint's write buffer still holds
bytes, since a closing transport empties it before calling
connection_lost. A flag set once the close returned would miss the
second cancel, and that case fails with one. The stalled-endpoint
teardown case pauses the endpoint's reading before the frames are fed:
it used to wait for the socket to stop reading through its own flow
control, which took 1.87 s of the case's 2 s budget on Linux, so a busy
runner timed out before the teardown ran. It is now 0.17 s per case, and
20 consecutive runs under 32 busy loops on 16 cores all passed. The
original teardown and buffering cases fail without the fixes. At this
head, making the stdin worker read again without waiting for its
previous chunk to be delivered fails 3 cases, removing the close
deadline fails 2, and never aborting fails 2; aborting unconditionally
fails both drained cases. These mutations ran on a disposable copy of
the source tree, not on the committed files.

The unreachable-upstream integration test still uses its original
1.5-second deadline. Its Windows fixture bypasses the venv redirector
while preserving the venv, so an inherited stdout handle cannot mask
EOF.

Full-PR pre-commit, commitlint, commit-message, source-language, asset
and whitespace checks passed. The bridge type check passed and all 10
import contracts were kept:

```powershell
uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD
npx --no-install commitlint --from origin/main --to HEAD --config commitlint.config.cjs
uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD
uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD
uv run --frozen --no-sync ty check raven/mcp/bridge.py
uv run --frozen --no-sync lint-imports
git diff --check origin/main...HEAD
```

These are focused local results. Remote dispatch tests use the
repository's SSH stand-in and real node processes, not an external
machine. POSIX-dependent paths are covered on Linux; no full-suite
Windows pass, live LLM run or sandbox VM run is claimed. Existing
Windows path, shell and journal portability outside this MCP change
remains out of scope.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Windows dispatches can now use granted MCP servers. Each fallback
endpoint opens an ephemeral loopback listener with a bounded
authentication handshake and its own random token. Upstream definitions
and credentials remain on the host, and remote dispatches receive no
loopback token.

Input now pauses when a bridge cannot forward requests. During shutdown,
a connection that cannot flush within one second is aborted and its
remaining queued bytes can be discarded. This bounds cleanup when a peer
has stopped reading.

Unix sockets remain the preferred transport, and the Raven node wire
protocol is unchanged. No dependencies, configuration fields or
migrations are added. Reverting the change restores the previous Windows
MCP limitation.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #962

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-19 19:58:55 +08:00
userName20260323
259d1f52fc
fix(*): regenerate sub-agent prompt files on every launch (#987)
## Summary

The research, code, oncall and ppt agents keep prompt files in their
workspace that the model reads on every turn:
`agent_memory/profile/soul.md`, `agent_memory/profile/agent.md` and
`TOOLS.md`. They were written only when absent (`seed_once`), so after a
raven upgrade or a config change the model kept reading the text of the
first launch while the gates enforced the current release. A
Raven-Research install seeded before the reader layout still asked the
model for the `## Answer` / `## Findings` / `## Limitations` report
while its shape gate checked the reader layout.

These files are now generated output:

- `write_generated()` in `raven/config/product_render.py` rewrites a
product-owned file from the release and the rendered config on every
launch.
- An operator's own lines go in the sibling `*.local.md` file
(`agent_memory/profile/agent.local.md`, `soul.local.md`,
`TOOLS.local.md`) and are appended after the product text at the next
launch. They must be UTF-8: one in another encoding (an editor's legacy
code page, GBK for example) raises a decode error that names the
`*.local.md` and asks for UTF-8, rather than one naming the generated
file.
- A receipt beside each file (`.<name>.generated.sha256`) records
`<owner> <digest>` for the last text written. A file the caller's own
receipt lines do not name (edited in place, written by another product,
or left by an older release) is backed up to `<name>.bak-<time>-<pid>`
before it is replaced, and the log says which of those it is when the
receipt can tell. A backup is never readable by more accounts than the
original: on POSIX an owner-only copy; on Windows, where a new file
takes its folder's inherited ACL whatever mode it is created with, a
hard link to the file itself, so it keeps the file's security descriptor
and no bytes are written. A Windows volume that cannot hard-link refuses
the backup, and a failed backup leaves the file untouched. While the
target changes, the receipt names both texts, so a failure between the
two writes never leaves a file the next launch cannot recognise.
- `product_text=None` (raven-code's `CODE_CONDUCT=off`) removes only a
file this product wrote; an edited one, or another product's file and
receipt, is left alone.
- The launchers pass `adopt=True`, since their home is the product's own
ACP home, and refuse to start with the file named when a prompt cannot
be written. On a raven without `write_generated` they fall back to
seeding once and log that the file will not be refreshed.
- The ppt engine writes into whatever home its plugin is granted, which
can be a host's or another product's, so it passes `adopt=False`: it
replaces only a seat its own receipt, or the digest of a prompt it has
shipped, shows it wrote. A test fails when a ppt prompt changes without
its digest being added. It marks its identity written only once the
write lands and retries on the next turn otherwise. A seat that fails
does not stop the other seats; the host's template sync waits for the
retry, since it would fill the failed seat and a seat the engine did not
write is never replaced. A seat it leaves alone is logged at DEBUG, not
WARNING: in a host's home those are the host's own, every process, with
nothing to act on. The notice says how to hand a seat back (move the
lines into `*.local.md`, delete the file), which is what a seat edited
in Raven-PPT's own home before this release needs; the docs name that
exception.
- raven-code's earlier per-file receipts and legacy digests are still
honoured, so untouched copies are refreshed without a backup.
- `atomic_replace()` takes a `newline` argument so generated files stay
LF on every platform. It also removes its temp file when the replace
fails, and removes a temp file already present before it writes,
clearing the read-only attribute first: Windows refuses to replace a
read-only target once the temp file has taken its mode, and the
read-only temp file left behind used to fail every later launch even
after the target was made writable again.
- The temp file `atomic_replace()` writes is created no more open than
the target from its first byte. On Windows a new file takes its folder's
inherited ACL and `os.replace` (`MoveFileExW`) moves that descriptor
onto the target, so a prompt kept private in a folder other accounts may
read came back readable by them, `*.local.md` lines included. When the
target exists, the temp file is now made by `CreateFileW` with the
target's DACL as `GetFileSecurityW` reads it, and the save is refused
when that DACL cannot be read. On POSIX it is created with the target's
mode rather than the umask default, closing the span in which the new
text sat in a wider file until the chmod. This holds for every caller
that passes no explicit mode, `config.json` included. The Win32 bindings
behind it are declared once per process, behind a cache: a structure
defined per call left a new ctypes pointer type, about 6 KB, alive for
every save.

Docs: `agents/README.md` (the seed-once contract becomes "Regenerated
prompts", with rollback steps), `docs-site` agent integrations (en and
zh), `CONTEXT-MAP.md`, and `CHANGELOG.md` (an unreleased entry that
promised the old behaviour, plus a Fixed entry).

Tests whose meaning changed: seven existing tests asserted that an
edited prompt is never overwritten; they now assert it is backed up and
regenerated. The raven-code launcher test fixture now pins
`CODE_CONDUCT` and `CODE_CONDUCT_DISCIPLINE` to `on`: a deleted switch
fell back to the developer's `.env`, and two conduct tests on main
already failed with `CODE_CONDUCT_DISCIPLINE=off` for that reason.

Checked and deliberately not changed (documented in `agents/README.md`):

- The processes of one product (one per model binding) share these
files, so a launch can change what an older process still running reads
on its next turn. Closing that needs a per-process snapshot or
in-process injection of the prompt, which is a separate change.
- Backups are never deleted automatically.
- Host workspace templates (`USER.md`, `HEARTBEAT.md` and the rest) are
the user's files and stay write-if-missing.
- A file removed from a release is not removed from an installed
`~/.raven/agents` copy. Nothing loads such a leftover today: modes are
read by label, tools are imported by name, and no release has removed a
skill directory.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run at the head of this branch, rebased on the current main:

- `uv run pytest -q`: 32739 passed, 17 failed, 147 skipped, 2 xfailed.
The same 17 test IDs fail on the current main in a clean worktree with
the same interpreter and the same fetched tokenizer dictionary (proxy
redaction, tokenizer data, network-safety resolution, install script,
document converters, node runtime, trajectory time zone); none is
introduced here.
- `uv run pytest tests/test_agents_code_launcher.py -q` with
`CODE_CONDUCT` and `CODE_CONDUCT_DISCIPLINE` each set to `off` in the
environment, alone and together: 79 passed in all four runs.
- `make lint-python lint-imports lint-types lint-deps`: all pass, 10
import contracts kept, no dependency issues.
- `scripts/check_commit_messages.py`, `scripts/check_source_language.py`
and `scripts/check_large_files.py` over `origin/main..HEAD`, and `git
diff --check`: pass.
- Each fix was reverted on its own and its test went red: the receipt
written after the target, a failed target write, the backup mode, the
ppt `adopt=False` guard, receipts without an owner, an undeletable
earlier receipt, the `*.local.md` read-back from the assembled system
prompt, the temp file a failed replace leaves behind, and the temp
file's mode while it holds the new text.
- Windows review findings:
`test_a_windows_backup_is_the_edited_file_itself_so_its_acl_comes_with_it`
and `test_a_windows_backup_the_volume_cannot_link_is_refused_not_copied`
force the link path on Linux;
`test_a_failed_replace_takes_its_tmp_with_it` and
`test_a_read_only_tmp_a_dead_writer_left_does_not_block_the_replace`
cover the temp file on every platform;
`test_the_tmp_is_never_more_open_than_the_target_while_it_holds_the_new_bytes`
pins the POSIX mode from creation;
`test_a_read_only_prompt_regenerates_once_it_is_writable_again`
(read-only retry) and
`test_a_private_windows_prompt_keeps_its_own_dacl_when_regenerated_and_backed_up`
(a private prompt in a folder every local user may read keeps its own
DACL through regeneration and backup, read back with `icacls /save`) run
on Windows only.
- The `Windows ACP stdio` CI job gains a step that runs
`tests/test_config_product_render.py` and the two temp-file tests
natively. On the Windows runner: 158 passed, 2 skipped (the two
POSIX-mode tests), the DACL, read-only retry and hard-link backup tests
among the passes.
- Diff coverage over `raven/` with the related suites: 94.12%, above the
90% gate.
- Second-round review findings, each test red without its fix:
`test_an_addendum_that_is_not_utf8_is_named_and_nothing_is_written`,
`test_a_local_file_that_is_not_utf8_is_named_when_the_launch_stops` and
`test_an_addendum_that_is_not_utf8_names_itself_and_holds_back_no_other_seat`
(the decode error names the addendum; the ppt engine writes the other
seats and holds the sync);
`test_a_hosts_own_seats_are_left_alone_without_a_warning_in_every_process`
and `test_a_seat_left_alone_says_how_to_hand_it_back` (DEBUG level, and
the notice's instructions);
`test_the_windows_bindings_are_declared_once_not_per_save` (Win32
stand-ins on Linux, two saves load each DLL once).

Windows is covered by the CI step above, not by a local run. The CRLF
fold of an older copy is reasoned, not run. `npx commitlint` is not
installed on the machine this was run on; the Python commit check above
was run instead.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- User-visible: on the first launch after upgrading, each agent rewrites
its prompt files. A file edited in place, or left by an older release
without a receipt, is copied to `<name>.bak-*` and the launcher log
names the copy; an in-place edit stops taking effect until its lines are
moved into `*.local.md`. A `*.local.md` that is not UTF-8 stops the
launch with the file named. In Raven-PPT's own home a seat edited before
this release keeps the edit, and its `*.local.md` is not applied, until
the seat is deleted. A Raven-Research install seeded before the reader
layout switches to it at that launch.
- Security: a backup is never readable by more accounts than the file it
keeps (an owner-only copy on POSIX, a hard link keeping the file's ACL
on Windows), and a replaced file keeps its own mode, and on Windows its
DACL, from the temp file's first byte; receipts hold product names and
digests only; `*.local.md` is operator-owned and carries the same trust
as editing `agent.md` did before.
- Compatibility: a launcher running on a raven without `write_generated`
seeds once and logs it; the ppt engine on such a raven keeps
write-if-missing. On a Windows volume that cannot hard-link (FAT32,
exFAT, some network shares), an edited prompt file stops the launch with
the file named, until its lines move to `*.local.md` and the file is
deleted. On Windows, a save through `atomic_replace()` whose target's
DACL cannot be read is refused rather than made with the folder's ACL.
- Rollback: revert this change, then stop the product's processes,
delete each generated file whose `.<name>.generated.sha256` receipt
still names its current text, and relaunch so the older code seeds its
own. `*.local.md` files, backups and edited files are left as they are.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: userName20260323 <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-19 17:36:03 +08:00
Dizhan Xue
1b9cbf85e5
feat(*): let a dag decision node choose which of its branches run (#1001)
## Summary

A sub-agent DAG node can now decide which of its downstream branches
run. Before this, a background run could not turn on a result: every
node ran, and choosing a path meant the host winding the run down and
starting another.

How it works:

- A node that declares `routes` (route name to condition) is a decision
node; `select` is `"one"` (the default) or `"many"`. Raven's per-node
judge, the existing verdict call, picks the route or routes the node's
answer satisfies. The node's own text never selects a route. A route
that cannot be determined fails the node, with no fallback route.
- A downstream node carries `when`, a guard over `<node>.<route>` and
`<node>.done` atoms joined with `not`, `and`, `or` and parentheses,
evaluated with three-valued logic: a route of a decision node that was
never executed is unknown. Every decision node in a node's `depends_on`
must appear in its `when`; such graphs are refused at submission
otherwise, along with guards that can never hold and a route named
`done` in any letter case. Validation keeps negation parity and the
surrounding boolean expression: a double-negated `done` and a
disjunction with a satisfiable arm are accepted, without expanding
disjunctions into combinations.
- A node that does not run is `not_executed`, with a reason:
`when_false`, `when_unknown` or `upstream_not_executed`. A node whose
dependencies all went unexecuted is not executed either; a failed or
skipped dependency still skips its dependents first.
- A merge node reads a missing upstream as a fixed sentence, `<This task
is not executed and the output is missing, please go on without this
output.>`, written outside every untrusted-content fence, plus a note
listing the inputs that may be missing. A file reference to the files of
a node that may not execute is refused at submission however its path is
spelled: `@nodes/`, an absolute path into the node root, or a workdir
path that lands there, symlinks followed. Windows case, separator and
drive-relative aliases are recognized, and the advice keeps the declared
node ID.
- Node judgement is always on: the `subagentDag.verdictEnabled` switch
is retired. Config migration 14 removes it with a notice, and drops it
again on every later load should an older build sharing the config write
it back. `raven playbook run` now judges its nodes too, on the same
`subagentDag` settings and a pinned `verdictModel`'s own provider; with
nobody there to answer, a node judged not to have accomplished its task
fails on the judge's reason. A host with no judge at all refuses graphs
that declare routes.

Where it shows:

- Run events, the manifest, `dag.get`, `tasks.list` and
`dag.run_completed` carry `reason`, `selected_routes` and
`route_evidence` per node, and `routes`, `select` and `when` from the
graph. The instance registry's DAG-node rows (and `InstanceRow`) carry
the outcome too, so a client that reloads before the run finalizes shows
what a branch decided. `TaskCounts.not_executed` is a new counter.
- TUI: a not-executed node shows a hollow marker and its reason in
words; a decision node shows its selection on the node's second line,
and the guard and route evidence in the detail and in `/dag <node>`. The
Agents Overlay and the live agents strip leave not-executed nodes out,
as the backend's agent lists do.
- Web: the desk's tasks tab names not-executed steps and shows the same
fields.
- Playbooks pass the three fields through, namespace guard atoms with
the run prefix, and keep them through compilation.
- The `run_subagent_dag` schema, the DAG orchestration skill and the
docs site (English and Chinese) describe routes, guards and the merge
rule; CHANGELOG has the entries.

A graph with no `routes` and no `when` behaves exactly as before: the
same judge requests, events, records, summary and finish text.
`route_evidence` is a quote the judge picks from the sub-agent's output;
it is cut at 500 characters, keeps a lone surrogate from a malformed
escape as the escape's own text, and never enters model-facing text
(finish text, `dag_status`, exception reports). When a judge answer does
not report a usable route, the judge is asked again at most twice, each
time with one correction. The shared instance registry drops its cache
when a write fails on the data rather than the disk, so a row the file
cannot hold fails its own write and no other.

Two Windows CI steps run the alias and finish-text regressions; the
latter exposes the installed Git `cat` executable required by the
existing fixture.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Linux, Python 3.12.13, on this revision rebased onto the current main
(`$CHECKS` is a temporary evidence directory):

```sh
uv run --frozen --no-sync pytest -q -n 4
# 23 failed, 33229 passed, 162 skipped, 1 xfailed, 3 errors
uv run --frozen --no-sync pytest $(git diff --name-only origin/main...HEAD -- tests/) -q -n 4 --idle-ceiling-strict --cov=raven --cov-branch --cov-report=json:$CHECKS/coverage.json
# the 20 test files this branch changes: 2622 passed, 1 skipped (a POSIX permission case that does not apply to root)
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json "$CHECKS/coverage.json" diff --base-ref origin/main --head HEAD --threshold 90
# Diff coverage: 99.48% (764/768 executable changed lines), passed the 90.00% threshold
```

The full run's 23 failures and 3 errors are 25 tests. 24 of them fail
the same way on the current main in this environment: the venv lacks
`nltk` and `datrie` (16), a proxy is set in the environment (5), this
box has its own `npm` in `/usr/bin` (1 install-script case), LibreOffice
writes no `.xls` here (1), and the run is root (1). The 25th,
`test_a_cli_run_left_on_the_machine_when_this_computer_stops_is_picked_up_by_the_next_start`,
failed with its teardown in the full run and in one of two runs of its
file, and passed in five runs on its own: it reads the remote-run record
the moment its pid file appears, a race in code this branch does not
change (#840 makes that test wait for both). The four uncovered changed
lines are the gateway's `_verdict_provider` body, whose helper is tested
on its own; a guard atom over a node that did not complete, which the
runner never evaluates because it reads a guard only once every
dependency completed or was not executed; and the cut of an over-long
terminal output.

Ruff check and format, `ty check` over the changed modules, the import
contracts, `deptry raven`, commitlint, and
`scripts/check_commit_messages.py`, `check_large_files.py` and
`check_source_language.py` over this branch's commits pass. ui-web:
`vitest run src/features/tasks/BoardCard.test.tsx scripts/gates` passes
50 files and 244 tests, and `tsc --noEmit` is clean. In Chromium a
not-executed title whose first word is wider than the room beside its
status now breaks that word instead of starting under a blank line, and
every other tasks-board card renders pixel-identical, English and
Chinese, light and dark. Firefox still starts such a title under a blank
line and cuts it without an ellipsis. CI's `TUI checks` covers ui-tui.

Mutation checks: removing any one of these makes its own test fail: the
every-load strip of the retired switch, the quote's escape before the
cut, the registry's cache drop, the node-root check in the tool, the
runner and the replan path, the physical path through a symlink, the
CLI's judge, its settings and its no-answer failure, and the pinned
judge provider. A model reading only the DAG skill's catch-all paragraph
answers that without a catch-all route the judge may pick the nearest
route or fail the node; an earlier wording led it to answer that the
graph always stops.

A live run on `openrouter/z-ai/glm-5.3-flash`, two graphs with synthetic
messages, selected the expected route at each decision node, left the
unselected branch `not_executed` with `when_false`, and put the fixed
sentence outside the fences. Earlier revisions of this change passed the
native Windows selections (aliases, finish text, guards; Windows build
26300, Python 3.13.12). The final revision has not been rerun on native
Windows or macOS.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Security: sub-agent text stays fenced or out of model-facing text on
every new path. The routes block in the judge request sits after the
last untrusted fence and is labelled as the graph author's; the
missing-output sentence is written outside every fence; `route_evidence`
is bounded and reaches records, events, the registry and the UIs only.
In the TUI, route evidence is drawn through the same path as a node's
error text; neither strips terminal control sequences, which this PR
leaves as it is.

Behaviour changes: a config that set `subagentDag.verdictEnabled: false`
now gets judged nodes; migration 14 removes the key and says so once.
`raven playbook run` judges every node, one more model call per node,
and a node judged not to have accomplished its task fails there instead
of completing. New wire fields are optional except
`TaskCounts.not_executed`, which is always present (0 when nothing
branched). `InstanceRow` gains three optional fields, and a decision
node's registry row now also stores its selection and up to 500
characters of route evidence.

Conditional-artifact validation matches node IDs using the session
namespace's case-insensitive identity on every host, including macOS.
This conservatively rejects case-only spellings even on case-sensitive
filesystems. Separator and drive normalization remain native, and
unconditional file resolution keeps its existing behaviour.

Rollback: revert the squash commit. A config already stamped 14 loads
unchanged on the previous build, since the stamp only gates migrations;
a user who had set `verdictEnabled: false` can set it again there, and a
later build drops it again on load.

## Related Issues

Related, found while building this and not changed here: #998, #999,
#1000. Follow-ups from its review, not changed here: #1008.
2026-10-19 15:17:04 +08:00
Dizhan Xue
b602de6bbf
feat(*): stream Mirasim turns over the running host's WebSocket (#996)
## Summary

The Mirasim adapter keeps one WebSocket connection to the Mirasim host
already running on the machine. It replaces the per-request `mirasim
ui-cli` processes and streams the host's session snapshots through ACP.

- Find the host through `RAVEN_MIRASIM_HOST` (`unix:<path>`, or a
  loopback port as `<port>`, `127.0.0.1:<port>`, `localhost:<port>` or
  `[::1]:<port>`, connected to as named), else the remote host's Unix
  socket (Linux and macOS), else the desktop app's loopback port and its
  token file. Tokens travel in headers. The adapter never requests
  `ready` or `getConfig`, whose answers contain settings and
  credentials. The WebSocket upgrade is bounded by the connect timeout.
- Stream text, thoughts, tools, sub-agents, plans, delivered files,
  usage and accepted steers. Reconnect with backoff, resubscribe
  sessions, and take a fresh snapshot when patches have a sequence gap.
- Live patches carry no segments; only settled history turns do. Within
  one patch a call already shown settles before the text after it, and
  a new call follows the text before it. A snapshot taken whole, on
  subscribing, after a gap or when a queued turn is dispatched, puts its
  calls before its text, so the answer after a call stays one message.
  History replays in its segments' order, using the host's UTF-16
  offsets.
- Support new, load with complete history replay, resume without
  replay, list, close and delete. A turn running in the app while a
  session loads is forwarded after the replay. Lists cover this agent's
  conversations in the given directory, ones begun in Mirasim included.
  Delete also removes the Mirasim conversation. A conversation the host
  refuses to continue, deleted or not one of its own, is started again
  once, with a note saying why. A list or history request the host
  fails is refused at once with its reason, and history paging stops if
  the host hands back a page it already gave.
- Cancel running tasks with `stop` and queued prompts with `recall`.
  Prompts accepted after timeout remain tracked for cleanup and session
  persistence. Failed or unconfirmed steers are recalled; if they
  started their own task, that task is stopped by id. A prompt still in
  the host's queue when the connection comes back is recognised from
  the queue the host lists on resubscribe.
- Relay approvals and questions of turns Raven started through ACP
  permission and elicitation requests; questions in turns Mirasim starts
  are left to the app. Claude receives inline images; other agents get
  an image linked as a local file as its path, and refuse inline ones.
  Claude, Codex, DSH, Pi and ZCode support steering. Turns started in
  Mirasim are forwarded while the adapter holds their session.
- Refuse queued prompts after session close or adapter shutdown. Every
  local ACP agent whose tree a Windows Job or POSIX process group owns
  gets an EOF grace that ends as soon as the agent exits. An agent may
  ask for a longer one under `agentCapabilities._meta` as
  `raven.shutdownGraceS`, granted up to five seconds; the Mirasim
  adapter asks for five, so a first prompt the host accepts late is
  still stopped. A close cancelled during the grace still releases the
  connection. PID-only cleanup keeps the launcher alive for the
  process-tree walk.
- The question of which agents Mirasim has runs in the environment the
  presets' adapters get: the login shell's on Linux and macOS, Raven's
  own on Windows. With no host reachable it lists none, and the boot
  handshakes no Mirasim preset except a row that sets
  `RAVEN_MIRASIM_HOST` or `MIRASIM_HOME` in its own `env`.
  `readyTimeoutMs` is 150000, above the adapter's own connect and menu
  limits.

The old `uicli.py` and `guard.py` transport is removed. Existing binding
files remain readable. The agent-integration documentation is updated
in English and Chinese. The Windows ACP job exercises the loopback
transport, real preset subprocesses and teardown. The live suite claims
a slot directory of its own by a lock, sweeps only that slot, and checks
the model each turn ran on.

`scripts/check_mirasim_frames.py` checks a host bundle for the frames
and fields the adapter uses. Run against the bundle that last worked, it
also reports each of those names that now occurs fewer times, which is
how a frame renamed in one of several places shows.

Protocol details verified against the host bundles 0.0.537, 0.0.547 and
0.0.558: `listSessions` filters by workdir, `getHistory` pages
backward, any prompt can set the session's route, there is no
unsubscribe frame, a request whose handler throws is answered with an
error carrying its `reqId` as `clientRef`, and live patches never carry
`segments`. Deleting a Mirasim conversation leaves the underlying
agent's transcript files intact.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Final local verification ran on Linux with Python 3.12.13 at the head
of this branch. Temporary report paths are abbreviated as `$CHECKS`.

```sh
PYTHONDONTWRITEBYTECODE=1 uv run --frozen --no-sync pytest \
  tests/test_acp_mirasim_catalog.py tests/test_acp_mirasim_host.py \
  tests/test_acp_mirasim_input.py tests/test_acp_mirasim_interactions.py \
  tests/test_acp_mirasim_server.py tests/test_acp_mirasim_store.py \
  tests/test_acp_mirasim_stream.py tests/test_check_mirasim_frames.py \
  tests/test_subagent_mirasim.py tests/test_acp_client_teardown.py \
  tests/test_acp_windows.py tests/test_acp_client_launch.py \
  tests/test_acp_client_resume.py tests/test_subagent_probe.py \
  tests/test_subagent_third_party.py tests/test_subagent_acp.py \
  tests/test_acp_protocol.py \
  -n 0 -q --tb=short --idle-ceiling-strict -p no:cacheprovider \
  --cov=raven --cov-branch \
  --cov-report=json:$CHECKS/coverage.json \
  --junitxml=$CHECKS/regression.xml
# 1060 passed; no failures, errors, skips, or strict-idle violations

uv run --frozen --no-sync python scripts/coverage_gate.py \
  --coverage-json $CHECKS/coverage.json \
  diff --base-ref origin/main --threshold 90
# 99.20%: 1362 of 1373 executable changed lines

UV_NO_SYNC=1 make lint-python lint-types
# Ruff check/format: 2384 files; ty passes
uv run --frozen --no-sync lint-imports
# 10 contracts kept
uv run --frozen --no-sync deptry raven
# No dependency issues
uv run --frozen --no-sync pre-commit run --files <changed files>
# All applicable hooks pass
UV_NO_SYNC=1 make check-commits check-large-files check-source-language
# commitlint, commit messages, assets and source language pass
git diff --check origin/main...HEAD
# Exit 0
UV_PROJECT_ENVIRONMENT=$CHECKS/docsenv uv run --frozen --group docs \
  --no-dev mkdocs build -f docs-site/mkdocs.yml --strict -d $CHECKS/site
# Built

for v in 0.0.537 0.0.547 0.0.558; do
  uv run --frozen --no-sync python scripts/check_mirasim_frames.py \
    ~/.mirasim-remote/servers/$v/server.cjs \
    --against ~/.mirasim-remote/servers/0.0.537/server.cjs
done
# Each: 13 sent frames, 11 read frames and 64 fields present,
# none fewer than in 0.0.537

PYTHONDONTWRITEBYTECODE=1 uv run --frozen --no-sync pytest \
  tests/integration/test_acp_mirasim_real_llm.py \
  -m integration -n 0 -q -p no:cacheprovider -rA --tb=short
# 10 passed in 346 s against the Mirasim host 0.0.537 running
# on this machine; every turn ran on glm-5.3-flash
```

Each new regression test was observed failing before its repair: the
refused unknown conversation, the reconnect with a prompt still queued,
the tool that ran while the connection was down, the turn running in
the app during a load, the repeated history page, the failed list and
history, the upgrade that never answers, the descendant holding the
agent's stdout, the close cancelled during the grace, the first prompt
the host accepts after the grace has passed, the empty agent list with
no host, the ready budget, and the frame check's count drop. The
stand-in host now hands frames to its dispatcher concurrently, as the
host does; removing the adapter's late recall makes three tests fail.
The frame check reports, against 0.0.537, copies with the dispatcher's
`stop` case renamed (6 occurrences, then 5), the six patch frames
renamed, and `coalesceMs` renamed; presence alone passed all three.

The local pass used the loopback stand-in host and real subprocess
tests; the live-model suite then ran against the Mirasim host already
running on the machine, across claude, codex, pi and zcode. Native
Windows and macOS were not run for this revision; the Windows ACP CI
job runs the four adapter test files natively.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

- Mirasim presets need a reachable host. Preset discovery still checks
  for `mirasim` on the login PATH; a desktop-only install without that
  executable does not offer the rows. The adapter's old `--executable`
  argument is removed.
- The adapter is one more client of the user's own host: Raven's turns
  show live in the Mirasim app and on paired phones, queue behind the
  user's own turns on the same conversation, and end if the desktop app
  closes or the host restarts. Turns the user starts in Mirasim on a
  conversation Raven holds are forwarded to Raven as turns nobody
  prompted.
- Removing an ACP instance now deletes its Mirasim conversation as well.
  Loading a session replays its complete history. The host has no
  unsubscribe operation.
- The desktop token is sent to the configured loopback endpoint, as
  with `ui-cli --port`; discovery relies on the local token file and
  port.
- Every local ACP agent's EOF grace now ends when the agent exits
  rather than when its pipes close; an agent can ask for up to five
  seconds. An unreachable host, abrupt parent death, or an adapter that
  cannot finish within its grace can still prevent cancellation of work
  owned by the shared host.
- Known limit: a queued prompt the host dispatches while the connection
  is down is not recognised as this adapter's after it reconnects; it
  is given up after 60 seconds and its turn runs on, forwarded as one
  nobody prompted.
- Rollback: revert the squash commit to restore the per-request
  transport. No binding migration is required. Conversations already
  deleted from Mirasim are not restored by reverting code.

## Related Issues

N/A
2026-10-19 14:28:09 +08:00
Handsome-wzw
edb00f54f5
feat(*): machines tab in the Agent Hub: register, install the node, remove (#983)
## Summary

The machine registry behind a page. The Agent Hub gets a **Machines**
tab:

- every registered machine as a card, with the state of this computer's
link to it (`NodePool.status`) and its Raven node (live version while a
link is up, else the one recorded on the row), and the register card at
the end;
- **Register a machine**: name, id, address, port, user, private-key
path and work directories; the server reaches the machine once (the same
`connection_add.probe` as `raven ops connection add`), writes the row
and its ssh alias, and a refusal comes back with its kind (unreachable,
login refused, changed host key, timeout, no ssh here, duplicate id,
invalid field, registry file) so the page says in the reader's language
what to do, with the server's sentence under it;
- **Install node** / **Update node**: `node_install.plan` + `install`
run in a background task; `machines.list` reports it on the row
(`install.running/ok/detail`) and the page polls until it ends; the
sheet's *Install into* field is the command's `--dir`;
- **Remove machine** (after a confirmation): refused while an agent
entry names the machine (the entries are listed) or agents run on it
over the link; otherwise the row and the managed `Host <id>` block go,
the link is closed, the node on the machine stays;
- **Test link** (`machines.list refresh=true id=...`) and **Check**
(`machines.doctor`);
- a preset's sheet picks where it runs (*Runs on*) among the machines
with this Raven's node, with the working directory there for acp/cli,
and the connect carries `machine` / `remote_cwd` to `subagents.add`.

Nothing a row or a refusal carries is the way onto the machine: no
address, account or key path (the pool's and the probe's sentences are
scrubbed of them).

Also: `raven ops connection remove <id> [--yes] [--force]` as the
command behind the removal; `install-node` and the page record the node
they put on, or found on, the machine as `node_install` on the row,
which is how a listing says a machine has this Raven's node without
reaching it. New RPC group
`machines.{list,add,install_node,doctor,remove}` in the contract, the
Python models and both generated TS clients. Docs: machines guide
(en/zh) sections for the page and the removal, CHANGELOG, CONTEXT.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest -n 6` (full suite): 32094 passed, 174 skipped, 3 failed -- the
same three that fail on main on this Mac (libreoffice absent, two
simulation tests).
- `pytest tests/test_rpc_machines.py tests/test_ops_connection_add.py
tests/test_cli_ops_connection_commands.py tests/test_update_subagents.py
tests/test_rpc_registration.py tests/test_rpc_schema_match.py
tests/test_docs_site.py tests/test_living_docs.py -n 0`: all green (632
+ 18).
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`:
94.29% of changed lines (the uncovered lines are since covered by one
more CLI test).
- `ruff check`, `ruff format --check`, `ty check` on the touched files,
`lint-imports` (10 kept, 0 broken), `tests/test_import_cycle_budget.py`,
`pre-commit run --files ...`: all pass.
- ui-web: `vitest run` 3959 passed (249 files), including the new
machines-tab tests and the five page gates (fixture shape, i18n
namespace, store shape, class namespace, domain registration); `tsc
--noEmit`; `eslint .` (0 errors, 4 pre-existing warnings in
SubagentsPage); `npm run gen:check`; `npm run build` with the boot
golden OK. ui-tui: `gen:rpc`, `gen:i18n`, `tsc --noEmit`.
- Real machine, from the page of an isolated gateway on this Mac (own
RAVEN_HOME, channels off), against one registered ssh machine:
registered through the form (probe + row + alias, no address shown
afterwards), node installed from the sheet (background task, card
reported the end, `node_install` recorded on the row, node directory
present on the machine), link tested (idle), Claude Code connected onto
the machine from its preset sheet with a working directory (entry
`claude_code@ravenx2` written, agent started there by the node), removal
refused while that entry was connected (the refusal named it), and, once
the entry was disconnected from the Connected tab, removed from the
sheet: the row is gone from the registry, the managed `Host` block is
gone from `~/.ssh/config` with the owner's own lines untouched, and the
node directory is still on the machine.
- The tab, the two sheets, the confirmation, a refused register (type
the id `locked`, `down`, `changed` or `slow`) and a refused removal (the
row agents run on) can all be walked through offline: build the page and
open `ui-web/dist/index.html?stub=1`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

- Follow-up commit 3248a83cb, after the Windows review: the fixtures
that set `HOME` around the ssh-alias
helpers set `USERPROFILE` too, through `tests/_home.py`, and a test pins
the Windows reading of `~` with
`ntpath` on every platform (it fails against a `HOME`-only fixture). The
three files: 113 passed on macOS;
  native Windows was not run here.
- Follow-up commit e9b150bc6, after the agent review's F1: the suite's
autouse `_no_real_raven_home` sets
`USERPROFILE` beside `HOME` through the same helper, so the two
ssh-alias CLI tests that bring no home of
their own (`store` and `reached` only) stop writing `Host gpu` into the
operator's `~/.ssh/config` on
Windows; `tests/test_conftest_no_real_home.py` pins the Windows reading
of `~` from a test without a home
fixture (red against the `HOME`-only fixture). The Windows reading
emulated on macOS (`os.path.expanduser`
-> `ntpath.expanduser`, outer `USERPROFILE` at an emptied sentinel, over
the three files): the two tests
wrote into the sentinel before, none after. Full suite on this Mac (`-n
6`): 32113 passed, 3 failed, the same three that fail on main here (a
LibreOffice case and two simulation cases). Native Windows was not run
here.
- Merge commit b58a6048d brings in main up to #971, whose family picker
lives in the same ext-agents page
files as the machines tab: five files conflicted (`ExtAgentsPage.tsx`,
its test, `source.ts`, `store.ts`,
`styles.css`), all resolved by keeping both sides. The tab strip counts
cards for the agent tabs
(`entries`, a family counts once) and registry rows for the machines
tab; `connect` carries `model` next to
`machine`/`remote_cwd`; the agent sheet, the machine sheet and the
family picker share the one drawer, so
opening any of the three clears the other two and `sheetDismissed` drops
all three. Checks on the merged
tree: ui-web `vitest run` 3997 passed (250 files), `tsc --noEmit`,
`eslint` on the feature, `gen:check`,
`npm run build` with the boot golden; the offline page (`?stub=1`) shows
the five tabs with their counts
and the machines tab as before; Python: the machines, connection,
contract and subagents files 366 passed,
the full suite (`-n 6`) 32480 passed, 5 failed: the three above plus two
`tests/test_acp_mirasim_uicli.py` cases that fail the same way on a
pristine `origin/main` checkout on this Mac (`mirasim ui-cli agents` did
not answer within 4s).

## Risk

- [x] Security impact considered -- the page never receives an address,
an account or a key path: the row is `connections.shown`-style data, the
probe's and the pool's sentences are scrubbed of the row's host,
user@host and key, and the register form's fields travel once in
`machines.add`. The install's package index stays on the step's stdin as
before. Removal needs a confirmation and is refused while entries or
running agents depend on the machine.
- [x] Backward compatibility considered -- new RPC methods and one new
optional registry field (`node_install`); `install-node` records it as a
warning-only step, every existing row and command keeps working,
`connections.shown` is unchanged.
- [x] Rollback path is clear for risky changes -- revert the commit; the
registry rows keep an extra `node_install` key that nothing else reads.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-17 21:08:46 +08:00
Handsome-wzw
b31eeccbf4
fix(node): an attach replays the agent's stderr past the line it last read (#977)
## Summary

Follow-up to LivXue's review note on #957 (the pick-up's binding at
`cli_agent.py:1393`). The machine's daemon kept an agent's stderr in the
task's record (`stderr.log`) but sent it only to the connection holding
the
task when a line was written, and `agent/attach` replayed `frames.jsonl`
alone. So what an agent said only on stderr while no connection held it
never
reached the host:

- **The session id a CLI names only on stderr** (hermes does). The
pick-up
#957 added binds a create's handle to the id in its output; for such a
CLI
it found none, warned "not resumable", and the next turn to the instance
  opened a new session.
- **The reason a command failed.** A run picked up after this computer
  stopped was reported as `exited 3` where the dispatch would have said
  `exited 3: token expired`. This part predates #957.
- **The stderr written while a link was down**, on a run still being
  watched, the same way.

The fix is in the node and the host's link to it (`raven/node/`):

- The daemon numbers stderr lines as it numbers frames (`agent/stderr`
  carries `n`), and `agent/attach` takes an optional `stderr_after`: the
stderr lines past that one are replayed after the frames, under the lock
that forwards a new line, so none is sent twice or missed at the
handover.
It works the same for a live task, a finished one, and a record a daemon
that never ran the task reads. An attach that does not ask hears no
stderr,
as before; a `stderr_after` that is not a whole number is refused like a
  bad `after`.
- The host's `ChannelEndpoint` counts the stderr lines it got
(`stderr_received`). A cli pick-up (`NodePool.attach`, a process that
read
none of them) asks from the first line; an acp pick-up from the line the
  run's record kept as the prompt went out (`RemoteRun.stderr_received`,
  follow-up commit below); a re-attach after a lost link
  (`NodePool._reattach`) asks from the last one it read.

The cli dispatch and pick-up read stderr off the endpoint as before and
now
get all of it, so the reason and the session id are there. An acp
pick-up's
connection drains the same endpoint into the stderr tail it quotes in
its
errors and into its journal, so what the agent wrote during the turn
while
this computer was away reaches them too; the acp run record keeps the
stderr
line count as the prompt goes out, so the earlier prompts' lines are not
read again (follow-up commit below). No protocol bump:
the new fields are optional, and a node runs exactly the host's code
anyway
(protocol and digest are checked at every connection).

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `tests/test_node_stderr_replay.py`, new, 22 tests: the daemon numbers
and
keeps stderr lines; an attach replays exactly the lines past
`stderr_after`
(from a live task, a finished one, and a record a daemon that never ran
it
reads), none when not asked, none from a record with no `stderr.log`,
and
refuses a bad `stderr_after`; the endpoint counts by the node's numbers;
a
  re-attach and a pick-up ask from the right line. End to end on the
stand-in machine (real daemon, stand-in ssh): a left cli run's reason on
stderr reaches its pick-up (said before and after this computer
stopped),
a left create that names its session only on stderr is bound to it by
the
  pick-up (both orders), and a reason written while the link was down
reaches the run watching it. 18 fail on main (2c2718122); the 4 that
pass
  there pin what does not change (no `stderr_after`, past the last line,
  past the end, no `stderr.log`).
- Changed assertions: the four attach-parameter assertions in
  `tests/test_node_client.py` and the forwarded-stderr one in
  `tests/test_node_server.py` include the new fields.
- Full suite: 32065 passed, 173 skipped, 2 xfailed, 3 failed -- the same
3
that fail on main (2c2718122) on this computer when run alone there: a
host
  without LibreOffice, and two simulation cases.
- Diff coverage: 100.00% (30/30 executable changed lines; `coverage
json`, then
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`).
- `ruff check`, `ruff format --check`, `ty check`, `lint-imports`,
  pre-commit on the diff; `scripts/check_commit_messages.py`,
`scripts/check_source_language.py` and `scripts/check_large_files.py` on
  `origin/main..HEAD`: clean.
- Docs: CONTEXT.md's Raven node entry, the protocol module's notes, and
the
machines guide's "Disconnects and restarts" section in English and
Chinese
now say the daemon keeps and replays stderr; the changelog has an entry
under Fixed. `tests/test_docs_site.py` and `tests/test_living_docs.py`
pass
  (18), `mkdocs build --strict` passes.
- Real machine, ravenx2, an isolated node directory per tree. The host
process that dispatched was killed with SIGKILL and the run picked up by
a
new process; the link was dropped by killing the local ssh. The command
  writes `token expired` (or `session slow-A1`) on stderr:

  | Probe | main (2c2718122) | this branch |
  |---|---|---|
| reason written after the host process was killed | `exited 3` |
`exited 3: token expired` |
| reason written before it was killed | `exited 3` | `exited 3: token
expired` |
| reason written while the link was down | `exited 3` | `exited 3: token
expired` |
| create left before its commit, session id only on stderr | "not
resumable", nothing bound | bound to `slow-A1`, no warning |

The machine was cleaned after: the probe's two daemons ended by pid, its
  directory removed.
- The runs above are on main 2c2718122 (#957 merged). The branch was
then
  rebased onto 400b30dfa, whose two new commits touch only TUI copy and
i18n; the node, docs and range checks were run again there (238 passed,
1
skipped). It merges cleanly with the open #964 and #965 (`git
merge-tree`).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

- Follow-up commit e0468fa88, after the agent review's F1 (an acp
pick-up asked
for the stderr from the agent's first line, across every earlier prompt
of
its process): `RemoteRun.stderr_received`, taken in
`Tracker.prompt_sent`
  off the endpoint's count, and passed through `NodePool.attach(...,
stderr_after=...)` by the pick-up; a cli pick-up keeps asking from 0; a
  record from before the field reads with 0. Regression on the stand-in
machine
(`test_a_pick_up_reads_the_turn_s_own_stderr_and_not_what_the_prompts_before_it_wrote`):
two turns of 150 stderr lines, a third interrupted after its prompt, the
next start's pick-up -- before the change its tail held 200 lines, 50 of
them the second turn's; after it, the third turn's 150 and nothing else.
Unit tests pin the record, the pick-up's attach arguments and the
default
of 0, and the tracker's wiring. The related files 259 passed; full suite
32067 passed with the three failures this Mac always has, plus one test
whose stand-in `Tracker` lacked the new keyword, fixed in the same
commit.
  Diff coverage of the changed lines: see CI.

## Risk

The replayed lines are the agent's own stderr, already kept in the
task's
owner-only record and already sent live to whichever connection held the
task; nothing new leaves the machine. `stderr_after` and `n` are
optional: an
attach without `stderr_after` behaves as before, and a line without a
number
counts on from the last. A cli pick-up replays the whole `stderr.log`,
as it
already replays every frame of the command; an acp pick-up replays the
lines
past the count its record kept as the prompt went out, so the replay is
bounded to the turn, and the connection keeps a bounded tail of it.
Rollback
is reverting the commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Follow-up to #957, for this review note:
https://github.com/EverMind-AI/Raven/pull/957#discussion_r4282448689

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-17 21:08:32 +08:00
Dizhan Xue
547c1aba5e
feat(*): reach Mirasim's agents as ACP sub-agent presets (#971)
## Summary

Connect the coding agents Mirasim runs as Raven ACP sub-agents, with one
bundled preset per agent and no separate adapter download.

- Serve Claude Code, Codex, Kimi Code, Antigravity, Qwen Code, Grok
  Build, DSH, Pi and ZCode CLI through `mirasim ui-cli`. The adapter
  runs on Raven's interpreter, so a Mirasim preset is not offered on a
  registered machine, and a stored row whose interpreter is gone is
  pointed at the running one when the config is next read. It withholds
  session MCP because ui-cli does not accept per-session servers. The
  boot capability backfill asks Mirasim once which agents it has
  installed and handshakes only those.
- Read model and effort choices from Mirasim. Offer the own models of
  Claude Code, Codex, Kimi Code and ZCode CLI on both their own-account
  and Mirasim platform routes. Persist conversation bindings and switch
  routes before continued turns. A menu read that fails is tried again
  after a short pause rather than kept for the cached copy's lifetime,
  an effort the menus in hand do not offer is read again before it is
  refused, and a failed menu read still reports the agents installed
  since the last one. A connect or test whose handshake could not read
  the menus keeps the ones already measured, and connecting an agent
  with a model refuses one its measured menu does not offer, as editing
  one already did.
- Retain the last model catalog when a refresh fails. Deleting a session
  prevents later writes, prompts, loads and route switches, waits for an
  already-running store write, and does not wait for unrelated catalog
  work. A turn, and its cancel, uses the catalog in hand instead of
  queueing behind another session's read.
- Relay questions and tool approvals through ACP. Content returned by
  another agent retains that agent's redaction policy, including values
  matching Raven's configured credentials. The rule covers Mirasim,
  shared sub-agent records and announcements, delegated tool results,
  A2A replies, logs, main-agent context, live ACP updates and session
  replay. Provenance is decided per result: a tool's declaration admits
  it to carrying another agent's content, and each result that does
  claims it, so a delegating tool's own errors, receipts and status
  lines stay Raven's text under the local scrub, and no result can claim
  provenance its tool never declared. Forwarding through `tool_call`
  carries a result whole for every tool, so a refusal reached that way
  ends the turn as a direct call does. The contract surface version is
  incremented for the added provenance fields. Local tool-result
  protections, outbound credential handling, permission checks and
  untrusted-content fencing retain their existing roles.
- Group unconfigured Mirasim presets into one card. Connect each member
  with its selected model in one write. Missing members open the
  existing installation and recheck flow, and a member whose handshake
  was refused for want of a sign-in is named in the drawer as it is on
  its card. A family whose members sit in two sections of the agents
  page draws one card in each. Narrow drawers keep the agent name and
  place the model and action on a second row.
- Guard command processes and clean up their descendants on
  cancellation, including ones a tool starts in a session of its own
  while the agent ignores SIGTERM: on Linux the guard adopts them as a
  child subreaper, and the process tree is read again before SIGKILL. A
  guard that outlasts its drain no longer replaces the command's own
  error with a shutdown traceback. Stage oversized prompts within the
  full Windows command-line budget, counting the venv launcher's second
  launch, refuse a `mirasim` that resolves to a `.bat` or `.cmd` shim,
  and preserve percent escapes and UNC authorities in file attachments.
  The guard starts without the ACP package, polls its command's exit
  quickly at first and then slowly, and the adapter's stderr log carries
  no local variables.
- Share the existing ACP stdin reader with the adapter, and update the
  integration docs, domain vocabulary, translations and agent-mark
  attribution.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Rebased onto current main and reran validation on Linux with Python
3.12.13 and Node 22.23.2. The isolated interpreter resolves this
checkout even when a subprocess clears PYTHONPATH. Existing dependencies
were reused without changing their declarations or lockfile. In the
commands below, `$CHECKS` denotes the temporary result directory used
for the run. Both Python selections use the same
`COVERAGE_FILE=$CHECKS/post-rebase.coverage`.

```sh
uv run --frozen --no-sync pytest tests/test_agent_loop_*.py tests/test_acp_mirasim_*.py tests/test_tool_registry_*.py tests/test_subagent_acp*.py tests/test_subagent_activity.py tests/test_subagent_history.py tests/test_subagent_manager.py tests/test_subagent_probe_state.py tests/test_security_untrusted_context.py tests/test_tool_admission.py tests/test_tool_search.py tests/test_cli_log_file.py tests/test_acp_journal.py tests/test_a2a_client_tool.py tests/test_subagent_mirasim.py tests/test_subagent_probe.py tests/test_subagent_third_party.py tests/test_ui_agent_marks.py tests/test_acp_replay.py tests/test_acp_updates.py tests/test_rpc_console.py tests/test_rpc_system.py -n 0 -x -q --tb=short --show-capture=no --cov=raven --cov-branch --cov-report=json:$CHECKS/post-rebase-coverage.json --cov-report=xml:$CHECKS/post-rebase-coverage.xml --junitxml=$CHECKS/post-rebase-python.xml
# 2871 passed
uv run --frozen --no-sync pytest tests/test_contracts_*.py tests/test_tool_admission.py tests/test_tool_registry_execute.py tests/test_security_untrusted_context.py tests/test_config_loader.py tests/test_raven_config_tool.py tests/test_rpc_subagents.py tests/test_subagent_name_vocabulary.py tests/test_tool_refusal_continuation.py -n 0 -q --tb=short --show-capture=no --cov=raven --cov-branch --cov-append --cov-report=json:$CHECKS/post-rebase-coverage.json --cov-report=xml:$CHECKS/post-rebase-coverage.xml
# 487 passed, 1 skipped (POSIX directory permissions are not enforced for root)
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $CHECKS/post-rebase-coverage.json diff --base-ref origin/main --head HEAD --threshold 90
# 93.33%, 1594/1708 executable changed lines; passed
```

The full Python suite, `uv run --frozen --python 3.12 --all-extras
pytest -q`: 9 failed, 32315 passed, 176 skipped, 2 xfailed. Each failing
id is then run by id against a clean archive of the merge base, through
the same interpreter and extras and with the imported `raven` checked to
resolve inside the archive; every id that fails here also fails there.

The catalog-only failure and resumed-session deletion regressions failed
before their fixes. Additional tests exercise deletion during catalog
reads, store writes and route lookups. The content-policy tests failed
against the former redaction behavior and now check exact content
through direct calls, tool_call, the main provider, recorded frames,
persisted transcripts and ACP replay. Local file reads and outgoing
credential records remain covered by controls.

In ui-web:

```sh
npm test -- --no-file-parallelism
# 250 files, 3977 tests passed
npm run gen:check
npm run type-check
npm run lint
npm run build
# passed; lint has four warnings in unchanged files
```

Chromium checks used the actual page and styles with synthetic agent
rows. Model selection travels in the connect request; Escape closes only
the picker; missing members reach the download link and Check again. At
390px and 320px the name columns measure 252px and 182px, respectively,
with no horizontal overflow in the drawer. No page errors were reported.

```sh
make check-commits check-large-files check-source-language
# passed
npm run lint:i18n --prefix ui-tui
npm run lint:rpc --prefix ui-tui
# passed
uv run --frozen --no-sync python -m pre_commit run --from-ref origin/main --to-ref HEAD
```

The full-PR pre-commit run passes except prettier (ui-tui), which flags
`src/components/newInstancePicker.tsx` and
`src/components/subagentsHub.tsx`. Both files are byte-identical to
main; running the formatter against their main contents fails too. The
fixes do not touch either file. Commit-scoped hooks for the new changes
pass. Ruff check and format, ty on changed production files, all ten
import contracts, and whitespace checks pass. Every replayed commit
retains its patch, and the PR title and body pass the repository's ASCII
and commit-message grammar checks.

The earlier native Windows verification covers prompt launch limits and
file-URI conversions. This follow-up has no new native Windows, real SMB
or live vendor/provider run.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Credential authorization and scope belong at the point where access is
granted. Redacting a later reply cannot revoke access the receiving
agent already has. Under this product policy, Raven preserves returned
content unchanged and leaves redaction decisions to the sending agent.
Credentials an agent includes in its reply can therefore remain in
Raven's retained logs and transcripts.

Agent-returned content is preserved according to the sender's policy
across all supported delegation paths. The adapter returns each reply
when the turn settles rather than streaming it. Short prompts use
command-line arguments; oversized prompts use temporary attachments.
Connecting an agent probes it: with a model chosen, on that model's own
route, so the pick a connect carries is proved before the row is stored;
with none, on the agent's default. Compatibility was originally measured
against Mirasim 0.0.465; older CLI versions have not been verified.

A stored Mirasim row whose interpreter no longer exists is rewritten in
the config file, once, by the first Raven to read the config after that
interpreter is gone. A row naming another interpreter that still exists
is left as written, so two installs reading one config do not rewrite it
back and forth.

Rollback is a revert. Configured Mirasim rows then need removal because
their bundled adapter is no longer present, and until they are removed
Raven refuses to load the config at all: an unknown preset name fails
validation, so the host does not start and the file has to be edited by
hand.

## Related Issues

N/A
2026-10-17 14:31:43 +08:00
Handsome-wzw
baf76335af
fix(node): an old uv no longer skips the index; the port check is an install's (#965)
## Summary

Three follow-ups to #963 (installing a Raven node on a registered
machine),
from LivXue's review of it. Only `raven/node/install.py` and its test
change.

- **An old uv no longer skips the configured index.** #963 hands uv the
index
pip is configured with on the machine through `UV_DEFAULT_INDEX`, which
uv
  reads since 0.4.23 (uv's changelog adds it there together with
`--default-index`). A machine's own uv older than that ignores the
variable
and installed from PyPI while the plan named the mirror -- before #963
such a
uv refused the `--default-index` argument instead, loudly. The probe now
reports the found uv's version (`UV_VERSION`, from `uv --version`). With
an
  index configured, a uv older than 0.4.23, or one that does not say its
version, is passed over: the pinned uv is fetched under the node dir as
for
a machine with none, and the plan says why ("the machine has uv 0.4.22,
and
only uv 0.4.23 or later installs from the index pip is configured
with").
With nothing to fetch it with, the install says so in one sentence
naming
  the version. Without an index configured an old uv is used as before.
- **The port check is an install's, not the look's.** #963 refused an
index
whose port is not a number from 0 to 65535 in `plan()`, which
`install-node`
  calls before it reads whether the node is already there and before
`--prune`; neither needs the index, so a machine that had its node and a
bad
port in pip's configuration could not be told so or pruned. The check is
now in `refusal()`, with the same sentence; the look takes the machine
as it
  is.
- **The real install test no longer reaches the network.** Its `run`
stand-in stood in for `uv pip install`; since #963 the two steps that
run it
go through `push` when an index is configured, and the `push` stand-in
ran
  them for real -- on a computer with a system-wide `pip.conf` the test
  installed packages, against the module docstring's promise. The `push`
stand-in now stands in for them too, and the test also runs with an
index
  configured, checking both steps are fed it on stdin.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Follow-up commit, after the review of this PR:

- A uv an earlier install fetched under `node_dir/.uv` is used in place
of a
machine uv that would ignore the index. The probe reports that uv on its
  own (`NODE_UV`, `NODE_UV_VERSION`) beside the one it found first, and
`plan()` swaps it in when the first is older than 0.4.23 (or silent) and
the fetched one is not; without an index the machine's own uv serves as
before, and one under `node_dir` that is itself too old does not help.
Two
tests failed first: the plan from such a probe (uses the fetched uv, no
  `uv` step, 60 MB, no refusal without curl), and the probe under a real
  `/bin/sh` reporting both uvs (then the one, then none).
- The machines guide, English and Chinese: the "What lands on the
machine"
row for `uv` names the old-uv case, and the troubleshooting table has a
row
  for the sentence an old uv earns. `tests/test_docs_site.py` and
  `tests/test_living_docs.py` pass (18), `mkdocs build --strict` passes.
- The docstring of
`test_a_plan_whose_index_port_cannot_be_read_still_describes_itself_and_its_failures`
says the install, not the look, refuses the port. The real install
test's
`uv_version` is pinned to `UV_VERSION` rather than read off this
computer's
uv, so a test host with a uv older than 0.4.23 does not reach astral.sh
  through the stand-in that stubs only `pip install`.
- `tests/test_node_install.py` -- 82 passed; with the cli, bundle,
client and
import-cycle files -- 204 passed; `.venv/bin/python -m pytest -n 6` --
31781
  passed, 3 failed (the libreoffice and two simulation cases, which fail
the same way on main on this computer); diff coverage against main 100%
  (30/30 changed executable lines).
- Real machine (a registered Linux machine; everything under this
check's
own directory, removed after): uv 0.4.22 from uv's release page was put
first on the PATH the check's own runner gives the probe, with an index
  configured. Before this commit: the first install passed it over and
fetched uv 0.13.0 under `node_dir` (85 MB planned), and the second look
found 0.4.22 first again, planned to fetch 0.13.0 again (85 MB, an `uv`
  step), and with curl and wget taken away was refused with "upgrade uv
there" while 0.13.0 sat under `node_dir`. After: the same first install,
then the second look found the fetched 0.13.0, planned 60 MB with no
`uv`
  step, and went ahead without curl or wget.

- `.venv/bin/python -m pytest tests/test_node_install.py` -- 80 passed.
Each
new or changed case failed on main first: with an index configured, the
real install test's packages step really ran `uv pip install` against
the
test's mirror and failed resolving it (now stood in for, both steps fed
the
index on stdin); `plan()` of a machine whose index names a port that is
not
  a port now succeeds, `refusal()` gives the sentence, `install()` runs
nothing, and a machine that has this node is found so and pruned; for a
uv
at 0.4.23 and later, older, and silent about its version, with and
without
curl or wget, the plan uses the machine's uv or fetches the pinned one
and
says why, or `refusal()` gives the exact sentence; an old uv still
serves a
machine with no index; under a real `/bin/sh` the probe reports the
version
  a stand-in uv prints and nothing for one that prints none.
- `.venv/bin/python -m pytest tests/test_node_install.py
tests/test_cli_ops_connection_commands.py tests/test_node_bundle.py
tests/test_node_client.py tests/test_import_cycle_budget.py -n 6`
  -- 202 passed.
- `.venv/bin/python -m pytest -n 6` -- 31780 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
simulation cases, which fail the same way on main on this computer);
diff
  coverage against main 100% (26/26 changed executable lines).
- `ruff check`, `ruff format --check`, `ty check raven/node/install.py`
  clean; `check_commit_messages.py`, `check_source_language.py` and
  `check_large_files.py` on `origin/main..HEAD` clean.
- Real machine (a registered Linux machine whose pip is configured with
an
HTTPS mirror; this check's own node dir, removed after): uv 0.4.22 from
uv's
release page was put where the probe finds a uv, and the install was
given
an index pointed at a port nothing listens on, so whether uv used it
shows.
With main's code the old uv was used and the install succeeded -- uv
went to
  PyPI behind the plan. With this change the probe read 0.4.22, the plan
fetched uv 0.13.0 and said why, the packages step failed with
"Connection
  refused" (the index was used), and the node dir held uv 0.13.0 after.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The install module's docstring now says an old uv with an index
configured
is passed over; the new `Plan` field and the moved check are documented
where they are. `CONTEXT.md` (the node entry) does not describe uv
versions or
the index and still holds.

## Risk

A machine whose own uv is older than 0.4.23 and whose pip is configured
with
an index now gets the pinned uv fetched under the node dir (about 25 MB,
as
for a machine with no uv), or one sentence asking to upgrade uv when it
has
no curl or wget; every other machine is unchanged. A machine with a bad
port
in pip's configuration can again be told it has the node and be pruned;
an
install there is still refused with the same sentence. The change is to
the
install plan and one probe line; rollback is a plain revert.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Follows the review of #963 (the observations F1, F2 and F3 left on it).

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 18:36:02 +08:00
江国庆/Forrest
c18278b8af
feat(*): import exported and reported conversations to read on the web page (#960)
## Summary

The web page can now import a Trajectory Export, a Bug Report Package or
a trajectory report and read it as a read-only conversation, transcript
and trajectory both. It is meant for developers reading a bug report
they received, and for looking at a conversation from another machine.
It builds on #948 (export package v1, the trajectory view on by default,
and the conversation menu's availability predicate).

- **Import store** (`raven/trajectory/imported_package.py`,
`imported.py`). Three kinds are accepted: a `raven_trajectory_export` v1
export, a `bug_report_package` v1, and a `raven trajectory report`
tarball. A file that is not gzip is "not a package Raven can import", an
unknown version asks for an update, anything else is "damaged or
unsafe". Packages are unpacked member by member on `package_check`'s
bounded layer, with one budget shared by both layers: 100 MiB per
package, 1 GiB unpacked for an export and 256 MiB for the report kinds,
100k members; traversal, absolute names, backslashes, symlinks, hard
links, devices, FIFOs, sparse members, repeats, directories with data
and oversized extended headers are refused, and every file is created
with `open(..., "xb")`. The package's session record is held to 32 MiB
and 50,000 records before anything reads it (the import and every open
load it whole), or the package is refused as `transcript_too_large`. The
result is normalized into one trace store: export v2 shells stay as they
are, report v1 model inputs become v2 shells that resolve to exactly the
original payload (so long conversations read whole), and a pointer that
maps to nothing in the package becomes null and reads as not recorded.
Each import lives in `<raven home>/imports/<uuid>/` with a `meta.json`
(source file name, import time, title, kind, sha256); it is unpacked
under `.staging/` and renamed into place, removed whole on failure or
cancel, and stale staging is swept at startup. Default titles: an export
takes the original conversation's title or its first user input, a bug
report the first line of its description or its report id, a trajectory
report the original title or its id. One import runs at a time on a
dedicated worker thread that owns the slot; there is no queue, a closed
page does not free the slot early, and gateway shutdown stops and waits
for it.
- **RPC and server-side guard.** New `imported.upload` (base64, under
the same 100 MiB limit as `fs.upload`) answers `imported` or `duplicate`
(by sha256), and imports another copy with `force`. New error codes:
-32029 `imported_read_only`, -32030 `imported_package_invalid`
(`data.reason` carries a stable refusal code), -32031 `imported_busy`.
An Imported Conversation's key is `imported:<uuid>`. `session.list`
appends imported rows when asked for the `imported` channel (never in
the archived list); `session.resume`, `session.title` and
`session.delete` read and write `imports/` directly and never touch the
SessionManager. A dispatcher guard allows only a vetted list of read
methods on an imported key (any of the four session parameter names, or
`channel: imported`) and refuses every other method with -32029,
including methods added later. The SessionManager refuses to save or
fork an imported key and `/file` answers 404 for one, so an imported
conversation never lands in `<workspace>/sessions/`. The trajectory
methods read an import from its own trace store; the indexer registry
gains an LRU cap (four foreign stores), idle eviction covers every
indexer, and deleting an import drops its indexer.
- **Web page.** "Import conversation" in the left menu picks a file and
uploads it; while it runs the button shows the file name and is
disabled, every refusal has its own message, and a repeated package
offers "Open existing" or "Import anyway" (the confirm dialog gains an
optional second action). An "Imported" group below Recent (hidden when
empty) lists the imports with only Rename (inline) and Delete, and the
source file name in the tooltip; search mode matches, and "delete all"
in settings leaves imports alone. An open import is read-only: no input
box (and no transcript mask left behind for it), no title editing in the
header, no Branch, the conversation menu hidden with its slot kept, no
working-directory tag. Sub-agent panes, DAG runs, attachments, delivered
files and generated images show as unavailable without sending requests
the server would refuse; a package without a transcript says so and the
trajectory view still opens; `[REDACTED:...]` placeholders are shown as
they are.
- **Docs and tests.** The webui page (English and Chinese) gains "Import
a conversation"; `CONTEXT.md` (Imported Conversation) and
`ui-web/CONTEXT.md` define the term. Unit tests cover every refusal of
the unpacker, normalization, meta and titles, duplicates and `force`,
runner admission and cleanup, the guard (checked against every
registered method), and the imported branches of `session.*` and
`trajectory.*`; vitest covers the import entry, the rail and the
read-only mode. A new integration test imports packages made by the real
writers into a second raven home and reads them back, and covers
duplicates, rename, delete, refused writes, and damaged or hostile
packages (an oversized upload and a decompression bomb built at the
shipped limits).

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- The branch merges the current `main` (merge commits, no conflicts; the
generated RPC bindings and i18n catalogue still match their sources);
everything below ran on the merged tree unless noted.
- `uv run pytest tests/ -q` (full suite): 10 failed / 31918 passed / 188
skipped. All 10 (proxy environment, IPv6 prefix resolution, node
runtime) also fail on `main` on this machine; none is introduced by this
branch.
- `uv run pytest -m integration
tests/integration/test_imported_conversation_e2e.py
tests/integration/test_web_traj_export_e2e.py
tests/integration/test_web_bugreport_e2e.py -q`: 16 passed.
- Changed-line coverage over the feature's tests (before the merge),
`scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`:
97.75% (955/977).
- `uv run ruff check`, `uv run ruff format --check`, `make
lint-imports`, `make lint-types`: pass.
- ui-web (the merge touches no frontend, i18n or RPC contract file):
`npm run gen:check` (240 methods), `npm run lint`, `npm run type-check`,
`npx vitest run` (248 files / 3923 passed), `npm run build` and `python3
ui-web/build.py` (both boot snapshots match), `check-page.mjs`,
`check-css.mjs`, `check-class-namespace.mjs`: pass.
- ui-tui: `npm run lint`, `npm run lint:rpc`, `npm run lint:i18n`, `npm
run type-check`, `npm test` (144 files / 2111 passed), `npm run build`:
pass.
- `uv run --group docs mkdocs build -f docs-site/mkdocs.yml --strict`:
pass.
- `make check-source-language check-large-files check-commits`: pass.
- Headless Chromium against a real gateway (`raven web --foreground` on
a fresh raven home, at 1280 and 360 px): imported a real export and a
real bug report through the page, opened both read-only, read a
122-message model input to its end in the trajectory view, used both
duplicate choices, renamed and reloaded, deleted, and uploaded a text
file; no page errors. The same flow on the built page with offline
fixtures.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- `imported.upload` takes untrusted archives. The unpacker is bounded
and refuses unsafe members, the guard is an allow-list, and the
SessionManager refuses imported keys as a backstop; the import directory
is under the raven home, which `fs.*` and `/file` already refuse to
serve.
- An export is not redacted, so importing someone else's export gives
you its raw contents; the docs say so.
- Two processes sharing one raven home (for example `raven serve` and
`raven web`) do not see each other's import slot, so importing the same
package in both at once can leave two copies; the extra one can be
deleted.
- Rollback: revert the squash commit. Imported data stays in `<raven
home>/imports/`, which older versions ignore and which can be removed by
hand.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-16 06:54:34 +08:00
Handsome-wzw
c43006bf0f
feat(*): lend the host's MCP servers to an acp agent on a machine through its node (#947)
## Summary

Adds MCP lending for remote ACP agents on top of the machine and
recovery
services already merged in #920, #922 and #934. The branch includes
current
main and its default-on registered-machine behavior.

What this does: an acp entry on a registered machine is handed the MCP
servers its row grants, the way an entry on this computer is, and the
servers themselves stay here with their definitions and keys.

- The agent on the machine runs the node's own bridge as a stdio MCP
  server (`python -I -m raven.node --mcp --dir <node dir> --task <task>
  --lend <dispatch> --server <name>`, `raven/node/bridge.py`): two byte
  pumps around a connection to the daemon's socket, no MCP parsed, no
  credential held, the standard library only.
- The daemon carries the bridge's frames as a *stream* to the host
  holding the task (`mcp/open`, `mcp/frame`, `mcp/close`; protocol 6,
`raven/node/protocol.py`, `raven/node/server.py`). The daemon tells
which
  task a bridge is by the process that connected -- the peer pid off the
socket (`SO_PEERCRED` on Linux, `LOCAL_PEERPID` on macOS), walked up the
parent chain to the agent the daemon started, which the bridge is under
and no other task's process is; the kernel supplies that pid and a peer
cannot forge it. `mcp/open` carries only `{task, lend, server}`, and the
task it names must be the one the connection is a process of. Two tasks
on
  one machine run as one user, so no name or file a bridge could present
would tell them apart; the connecting process does. One stream per
bridge, at most 16 per task; a frame
  travels in pieces of at most 256 KiB characters, the last marked
(`protocol.pieces`), up to the 32 MiB the host's own bridge allows; the
  daemon relays and keeps nothing. A stream ends with either end, the
  host's link or the task, and the other end is told.
- The host opens the endpoints for a machine row exactly as for a row
here
(`McpEndpoints`: disabled tools withheld on the wire, stdio upstreams in
the host's sandbox, HTTP auth added here), binds them to the machine,
dispatch and acquired agent task
(`NodePool.lend`/`unlend`) before handing out bridge stanzas. On
`mcp/open`
the host checks the daemon-authenticated task against that binding
before
  connecting; it rechecks the grant and link after the socket opens. An
unlend, replacement grant or replaced link cannot authorize that pending
connection. The host then pumps the frames both ways
(`raven/node/client.py`).
`node/hello` names the bridge; a node serving without a daemon
(`--stdio`)
  names none, and the dispatch then goes without MCP and says so.
- The reply closes with the servers lent and the machine's node, or with
why none were. Remote lending requires Unix-socket support on the host:
on Windows it is withheld before endpoint setup, the task continues, and
the reply explains why. The roster's "injectable MCP" matches that
support
  (`session_mcp_delivered`). The cli lane on a machine still
  withholds its servers (the next change); the openai lane has no tools.

Not in this change: the cli lane's MCP file on the machine; a stream
that
survives the link going (a bridge sees its server end, the agent reports
the call failed, and the next dispatch works again); MCP for a run
picked
up after this computer restarted (#934): it reports its calls fail.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Follow-up fix 05eaba67a, after the c9de53947 review:

- The nine tests the Windows review named now pin the host fact through
one
  fixture, `unix_socket_host` (`tests/conftest.py`), which sets
  `raven.mcp.bridge.unix_sockets_available` to true and re-points every
  `raven.*` module that imported it by name; their bodies are unchanged.
- Found complete, not just the nine: the whole suite was run with
  `asyncio.start_unix_server` and `asyncio.open_unix_connection` deleted
before collection (a pytest plugin, controller and xdist workers), at
the
  merge base `6b938a426` and at `c9de53947`, and the two failure sets
compared. 17 node ids failed at the head and not at the base: the nine,
and eight this change added that run the transport for real (the bridge
module, and two lending tests through the stand-in machine). Those eight
are guarded by host capability instead of pinned: the bridge module
skips
when `sys.platform == "win32" or not unix_sockets_available()`, the two
lending tests carry the existing `skipif(not unix_sockets_available())`.
After the fix the same run fails the same 66 node ids at the head as at
  the base: nothing this change adds or touches fails on a host without
  unix sockets.
- A bridge whose daemon side goes ends as streams end. The host can
close a
  stream while a frame of the agent's is on its way; the daemon writes
`mcp/close` and hangs up, and the frame meets the closed socket, or the
  read side is reset first. That exited 1 with `BrokenPipeError` or
`ConnectionResetError`; a connection-level end is now exit 0 with
nothing
said, a frame past the cap still exits 1 with its reason, both pumps are
  awaited before the socket closes, and a daemon gone before the stream
opens is one sentence, not a traceback. Four deterministic cases with a
  stand-in connection fail on `c9de53947` and pass here.
- The bridge test's stand-in agent outlives its bridge, as an agent
outlives
  one of its MCP servers, so the later parts of

`test_a_stream_the_host_closes_ends_the_bridge_and_a_bridge_of_no_task_or_of_another_s_never_begins`
no longer race its exit: 1 failure in 80 runs on `c9de53947`, 0 in 80
here.
- The focused set (node, MCP, remote, manager and roster files) -- 935
  passed; `.venv/bin/python -m pytest -n 6` -- 31602 passed, 3 failed
(`test_rpc_files.py::test_a_host_without_libreoffice_says_so` and the
two
  simulation cases, which fail the same way on main on this computer);
  diff coverage against main 97.67% (420/430). Real machine, node
reinstalled with this tree in a directory of the check's own: a stdlib
ACP
  agent's bridge reached a host-held server and 1 MiB and 8 MiB results
  crossed whole, bridge exit 0; nothing was left on the machine.

Follow-up fix c9de53947, after the same-user token finding:

- The daemon no longer uses a token. `agent/launch` mints and writes
none,
  `mcp.token` and the `hmac` check are gone, and `Daemon.open_stream`
  identifies the bridge by the connecting process instead (`_peer_pid` +
  `Daemon.task_of`). The same-user path the reviewer kept open -- a peer
reading another task's `mcp.token` and opening its stream -- is closed,
  because identity is the connection the kernel reports, not a readable
  secret.
- Tests, each failing on the unfixed tree first: a bridge placed under
task
  B that names task A is refused (`the connection is a process of task
  'agent@box-2', not 'agent@box-1'`), as are a process under no task, an
  unplaced connection, and a task the daemon does not run
(`tests/test_node_server.py`); a real-OS test launches an agent, takes
its
real child, and asserts the child resolves to the task while the test's
own
  process resolves to none (runs on Linux in CI); `_peer_pid` is checked
against a real socketpair; the in-process and subprocess bridge tests
now
identify by the connecting process, and a bridge this test spawns
itself,
  or another task's agent spawns, never opens the stream.
- Windows withholding now covers a row on this computer too, not only a
  machine row: `session_mcp_delivered` drops the `machine and` guard and
answers no whenever `unix_sockets_available()` is false, so a local acp
row is no longer advertised as injectable and no longer raises at
dispatch
on a host without the transport; `_mcp_endpoints` yields nothing before
opening an endpoint and `_mcp_note` explains it. The two tests the
Windows
review named now guard on `unix_sockets_available()`, and the
withholding
  test is parameterized over a machine row and a local row.
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over the node/MCP/remote suite -- 97.63% changed-line coverage
(412/422).
  `ruff`, `ty` on the changed modules, and the import-cycle budget pass.
- Real machine (ravenx2, this head, node reinstalled, digest
fdbe132e76f0):
a stdlib ACP agent's bridge reached a host-held MCP server and a 1 MiB
and
  an 8 MiB tool result crossed whole (`CALL_OK`, bridge exit 0), so the
  connecting-process identity admits the legitimate bridge end to end on
  real Linux; the cross-task refusal on Linux is the real-OS test above,
  which CI runs there.

Earlier review fixes (4dc5b3e1f, the task-to-grant binding). Its
host-side
binding of each dispatch's endpoints to the acquired agent task is kept;
the
daemon-side token this run verified was since replaced by the
connecting-process
identity above (c9de53947), so the token-named tests and reproduction
below no
longer describe the current tree.


- `uv run --no-sync pytest tests/test_node_client.py
tests/test_node_server.py tests/test_node_bridge.py
tests/test_node_relay.py tests/test_node_bundle.py
tests/test_node_install.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py
tests/test_subagent_third_party.py -n 4 --idle-ceiling-strict
--cov=raven --cov-branch --cov-report=json:<coverage-json> --cov-report=
--tb=short` -- 1495 passed and one minimal-node stdio response test
failed;
  strict idle accounting also flagged the assembly test. Both cases pass
together under `-n 0 --idle-ceiling-strict` (2 passed). No assertions or
timing limits were weakened. The new authorization, revocation and host
  compatibility tests all pass.
- The five focused cases failed on the unfixed implementation first: a
  different authenticated task selecting another grant, a stream with no
task, each missing Unix socket API, and an astral Unicode chunk over the
node's byte limit. Regressions also cover revocation, grant replacement
and link replacement while an endpoint connection is opening, and a real
  remote turn continuing when both host Unix socket APIs are absent.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
  -- 97.94% (381/389 changed executable lines); this fix alone is 100%
  (40/40). Python lint, type, dependency
  and import checks pass; all 10 import contracts hold. The scope checks
  for commit messages, source language and file sizes pass. Both RPC
  generators pass `--check` after integrating current main.
- A real Linux node with two concurrent tasks and a host loopback
endpoint:
  before, the attacker's own task token plus the victim's public grant
  names returns the victim endpoint's marker; after, it returns no data
  and the endpoint is never reached. Its own grant works, unknown and
revoked grants fail, and the same attack remains denied after the relay
  is killed and an ordinary ping reattaches both tasks. A full chunk of
  262144 astral characters was lost/corrupted before and arrives intact
after. The final documented source tree was reinstalled and checked
again.
- Real ACP dispatch through the node's bridge reaches a host-held stdio
MCP tool and returns its marker; its grant is removed when the turn
ends.
  With the host Unix socket APIs removed to simulate Windows, the remote
  agent still completes, receives no MCP, explains the host limitation,
  and the host MCP ledger records no extra call. This is not a native
  Windows run. Dedicated test tasks, host endpoints and bridge processes
  are cleaned up.
- CONTEXT.md, module/field docstrings, both collaboration guides, the
  external design notes and this description were checked against the
  implementation. Main's no-debug-switch behavior and full-turn recovery
  documentation are retained.

Full combined-tree verification: `uv run --no-sync pytest -n 4
--tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
returned 31594 passed, 230 skipped, 2 xfailed and 4 failed. Each failure
reproduces on current main in a separate checkout: host CLI color
rendering,
LibreOffice detection and the two simulation cases. No other test fails.
The minimal-node response test that failed in the focused parallel
selection
passes in this full run. Diff coverage: 97.94% (381/389 executable
changed lines).

Earlier author verification:

- `uv run pytest tests/test_node_bridge.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_bundle.py tests/test_node_install.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_acp_per_session_mcp.py
tests/test_subagent_acp_mcp_lifecycle.py tests/test_mcp_endpoint.py
tests/test_subagent_mcp_grant.py tests/test_mcp_bridge.py
tests/test_acp_remote.py tests/test_remote_file_tools.py
tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_import_cycle_budget.py tests/test_resume_remote.py
tests/test_remote_resume.py tests/test_subagent_manager.py`
  -- 1182 passed. Covered: on the daemon, `node/hello` naming the bridge
  and a stdio node naming none, the token written owner-only beside the
  record, a stream opened with the task's token and refused with another
  or for a task not running, refused with no host holding the task,
  refused past 16 per task, one per bridge connection, the pieces of a
  frame crossing both ways as they are and a stranger carrying nothing,
  and every way a stream ends (the bridge closing, the host closing, the
  bridge's connection ending, the host's link ending, the agent ending)
  with the other end told; the bridge as a process against a real daemon
  over its socket: a frame to the host and one back whole, a large frame
  each way in pieces, the agent closing its stdin ending the stream and
the bridge leaving clean, the host closing the stream ending the bridge,
  a wrong token and a missing daemon each refused on stderr with exit 1;
  the stand-in machine with the real daemon and the real bridge: an acp
stub there that connects to its `mcpServers` and calls a tool reaches a
  stdio server here and answers with its result, the stanza handed to it
  naming the node's bridge, the task, the dispatch and the server and
  nothing of the server's definition or socket, the ledger here showing
  one call, no bridge outliving the dispatch; the reply's note and the
  roster predicate for a machine row.
- `uv run pytest` on this tree -- 31288 passed, 230 skipped, 2 xfailed,
4 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`,

`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`
and
`test_subagent_kimi_code.py::test_an_ask_that_runs_out_takes_what_its_kimi_started`,
which fail
  the same way on main on this machine.
- Real machine (a registered Linux machine with Claude Code signed in,
  the node reinstalled with the code before and after):
  A28, before (the
  code of #934's head and its node): the turn closed with "MCP servers
  'probe' were not lent", Claude Code said no tool of that name was
  connected, and the server here saw no call; after: `PROBE_OK:r2-lend`
  in 9.4 s, the server here recorded the call, and the turn closed with
"lent to acp agent ... through the Raven node on machine ...". A34:
after
the turn the machine's process list shows no bridge, a grep of its node
  directory finds nothing of the server's command or of this computer's
  socket directory, and no endpoint socket is left here. A29: with the
tool switched off here, Claude Code answered UNAVAILABLE and the server
saw no call. A30: a 40 s tool call under way, the bridge visible in the
  machine's process list, then cancelled: 3 s later no bridge there and
  no socket here. A31: the relay ssh killed during a 20 s call: the turn
ended 4.1 s later with "CALL_FAILED ... Connection closed", and the next
dispatch reached the server again in 7.6 s. A32: five calls in one turn
  all reached the server and came back; an 8 MiB answer: the server here
  produced it, and a plain ACP agent on the machine (a stdlib stub that
speaks the stanza's bridge directly) received it whole through the node
-- 1 MiB in 1.3 s, 8 MiB in 3.5 s, the bridge exiting 0 -- while Claude
  Code itself closed the server on that answer and reported the call
  failed: the agent's own cap on a tool result, not the link's.
- `ruff`, the commit-message and PR-body checks -- clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The English and Chinese collaboration guides describe lending,
task-bound authorization,
Windows withholding, and the existing CLI/restart limitations.

## Risk

Registered-machine support is enabled by configuration, without the
removed
trial environment switch. An ACP entry on a machine receives its granted
MCP servers on a Unix-socket-capable host; the host's normal MCP policy
still
applies. This intentionally exposes those tools to the selected remote
agent.
The server definition, credentials and endpoint paths remain on the
host.

The daemon identifies a bridge's task by the connecting process (the
peer
pid the kernel reports, walked to the agent it started), not by any
token or
name the bridge presents, so a peer task sharing the OS account cannot
open
another task's stream; the host then authorizes that task against the
dispatch/server grant. A system that reports no peer pid opens no
stream.
On Windows hosts remote lending is withheld explicitly, without aborting
the
remote task. Protocol 6 remains the MCP wire version; host and node must
also
have the same code digest, so updated nodes must be reinstalled.
Rollback is
a revert; the machine then carries no stream and receives no bridge
stanza.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#934 (merged); #920 and #922 (merged).

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-15 15:37:50 +08:00
Handsome-wzw
132c89b91b
docs(docs): a guide to working on registered machines, in English and Chinese (#956)
## Summary

The sub-agent page's "Work on a registered machine" section said in six
sentences that the feature exists; a reader who had not seen the code
could not get from there to an agent running on their own GPU box. This
adds `docs-site/docs/machines.md` and `machines.zh.md`, a full
walk-through, and puts the page in the nav under "Working with
Sub-agents" (with its Chinese nav title). The old section is now two
sentences and a link.

What the page covers, in order: what the machine needs; registering it
with `raven ops connection add` (what is recorded and where, the key
path rather than the key, host keys on first connection, the managed ssh
alias); installing the node (what lands where, the space check, the
package index, `--dir`, upgrading and `--prune`, the daemon's lifetime);
naming the machine in an acp, cli or openai entry, and why a builtin one
cannot; adding one from a conversation through `subagents.add`; what a
dispatch does (attachments named by address, MCP not lent, files left on
the machine, the reply's closing line); the scp spelling of a path on a
machine; reading, editing and fetching by address with the permission
rules; disconnects and restarts; a table of limits and defaults; a
troubleshooting table of the sentences the tools say; what never leaves
this computer. MCP lending is written as coming in #947, not as present.

Every number and sentence was read from the code rather than remembered:
`raven/node/protocol.py` and `raven/node/client.py` (agent limit, record
retention, heartbeat, backoff, idle close), `raven/node/install.py`
(what an install needs, puts there and refuses),
`raven/agent/tools/deliver.py` (fetch budget and landing path),
`raven/acp_client/remote.py` and `raven/ops/remote_path.py` (address
forms, failure sentences, reading roots), `raven/config/schema.py` (the
`machine` and `remoteCwd` fields of each kind),
`raven/rpc/methods/subagents.py` (the add's refusals and default name).
The three config examples validate against
`ThirdPartyAcpSubagentConfig`, `ThirdPartyCliSubagentConfig` and
`ThirdPartyOpenAISubagentConfig`. Example addresses use the
documentation-reserved 203.0.113.0/24 range. The review's four findings
were folded back in: the installer needs `uv` (or `curl`/`wget` to fetch
it) whatever Python the machine has; `doctor` exits non-zero only when
no entry is usable; a link idle for 10 minutes is closed at the next use
of another machine, not by a timer; and the pull request is named in
text, since site pages link nowhere external.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run --frozen --group docs mkdocs build -f docs-site/mkdocs.yml
--strict`: builds clean on the committed tree, both languages; the
page's internal anchors and the link into `oncall.md` resolve in the
built HTML.
- Both pages rendered in a browser (full-page screenshots reviewed): nav
entry, right-hand TOC, tables and code blocks in both languages.
- `pytest tests/test_docs_site.py -n 0`: 16 passed (both pages have both
languages, no external links, Chinese headings pin the English anchors).
- `python -m scripts.check_source_language`, `python -m
scripts.check_large_files`, `python -m scripts.check_commit_messages
origin/main..HEAD`: pass.
- The English and Chinese pages have the same sixteen headings; every
Chinese heading carries the English anchor, so a link written against
one language lands on the other.
- The three JSON examples were loaded through the config schema classes.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Documentation only; no code changes. The page describes the security
boundary as the code draws it: the key and the assembled ssh line never
reach a model, the user, address and port do as part of a path on the
machine. Rollback is reverting the commit.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-15 14:35:25 +08:00
江国庆/Forrest
6b938a4267
feat(*): export conversations and report problems from the web page (#948)
## Summary

The web page gains two things a conversation could only get from the
terminal before: exporting the selected turns as a download, and
reporting a problem through the bug report pipeline. The trajectory view
is now on for everyone.

- **Trajectory view without `--dev` (R0).** `raven`, `raven web`, `raven
serve` and `raven gateway` no longer take `--dev`; the trajectory policy
starts enabled. `raven tui --dev` (running the TUI from source) is
unchanged.
- **Turn selection.** From this version each turn's opening user message
records its trace id (the Turn Trace Mark, never sent to the model).
`raven/trajectory/selection.py` lists a session's main-line turns,
closes a selection over the sub-agent and delegated traces it
dispatched, collects their spans, and cuts the session record down to
the selected turns; older records are matched conservatively and an
unmatched turn stays in the trajectory only.
- **Export.** The Trajectory Export package v1 (`schema:
raven_trajectory_export`, `format_version: 1`) is the input contract of
the coming `web-traj-import`; it is defined in `CONTEXT.md` and pinned
by tests. It holds the raw spans, artifacts and v2 message blobs of the
selected traces plus the cut session record, unredacted. Exports run as
background jobs (at most two) and the browser downloads them from the
same-origin `/trajectory/export/download` route with a one-time token.
New RPCs: `trajectory.turns`, `trajectory.export`,
`trajectory.export_status`, `trajectory.export_cancel`.
- **Bug reports.** `freeze_export` is split into staging and finishing
so a report can be prepared from selected turns; the CLI's behaviour is
unchanged. A Report Session holds a web report in the gateway between
page requests: at most two at once, all blocking I/O on worker threads,
a cancel checked after every step (a report cancelled while packaging is
never sent), a failed or uploaded report releases its slot and keeps its
last status for ten minutes, and an idle one is cancelled after thirty
minutes. Seven `bugreport.*` RPCs, the `bugreport-v1` capability
(announced only while the receiver URL is valid), and error codes -32026
/ -32027 / -32028. Every decision made on the web page, defaults and
bulk buttons included, is remembered in the redaction history with `web`
as its source, unlike the CLI's bulk flags.
- **Web page.** A three-dot conversation menu in the header's corner
(the desk launcher moves one slot in) opens the new `export` wizard.
Export: pick turns, name the file under an "unredacted" warning, follow
the job, download once automatically with a link to download again.
Report a problem: pick turns, describe the problem, decide the uncertain
values in two columns (hits highlighted at code points), upload, and get
a receipt. Closing before the upload starts discards the report being
prepared; once the upload starts the report is saved on the machine, a
failure marked retryable offers a retry, and the saved report can be
sent later from the command line by its id. Every answer is checked
against the wizard's epoch and the job or report it was asked about, and
a write step whose answer is lost is settled from the report's status
instead of being sent again. Layouts were measured at 360 and 390 px in
English and Chinese.
- **Docs and tests.** The webui and trajectory-debugging pages (English
and Chinese) describe both flows. Two integration tests drive them end
to end: the export through the real download route, and the bug report
against a real receiver process, including a stopped receiver, a retry,
the redaction history's replay and a cancel at each step.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- The branch merges the current `main` (one merge commit, no conflicts);
everything below ran on the merged tree.
- `uv run pytest tests/ -q` (full suite): 57 failed / 31491 passed / 177
skipped. Every one of the 57 also fails on the current `main` on this
machine (environment-specific, pre-existing); none is introduced by this
branch.
- `uv run pytest -m integration
tests/integration/test_web_traj_export_e2e.py
tests/integration/test_web_bugreport_e2e.py
tests/integration/test_bug_report_upload_e2e.py -q`: 10 passed (three
consecutive runs of the two new files).
- `uv run ruff check`, `uv run ruff format --check`, `uv run ty check`
on the changed Python: pass.
- ui-web: `npm run gen:check` (239 methods), `npm run lint`, `npm run
type-check`, `npx vitest run` (246 files / 3868 passed), `npm run build`
and `python3 ui-web/build.py` (both boot snapshots match),
`check-page.mjs`, `check-css.mjs`, `check-class-namespace.mjs`: pass.
- ui-tui: `npm run lint:rpc`, `npm run lint:i18n`: pass.
- `uv run --group docs mkdocs build -f docs-site/mkdocs.yml --strict`:
pass.
- `make check-source-language check-large-files check-commits`: pass.
- Headless Chromium on the built page (offline fixtures): both flows
walked to the download and to the receipt; the header corner measured in
five layouts (desk panel open and closed, workspace panel open, shell
mode, narrow split) and the wizard at 360 and 390 px in both languages.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- A process started with the old `--dev` flag that runs `system.upgrade`
relaunches itself with `--dev`, which the new version refuses; the user
runs `raven web` once by hand. This was accepted as the cost of dropping
the flag.
- The trajectory view is on by default: each open page calls the light,
in-memory `trajectory.state` every five seconds, and the index is built
only when the view is opened. No switch to turn it off is added here;
the policy keeps a replaceable source for a settings entry later.
- The export is the raw trajectory and may contain secrets. The page
says so in a warning; the download is served only on the same origin,
behind the session guard, by an unguessable one-time token, and no path
ever reaches the page.
- Rollback: revert the squash commit. Nothing is migrated; the
`trace_id` field already written into session records is ignored by
older versions.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-14 21:34:46 +08:00
Handsome-wzw
aa023212a0
feat(*): pick up a turn left on a registered machine when this computer stops (#934)
## Summary

Registered machines are now available without a debug environment
switch.
`RAVEN_DEBUG_MACHINES` previously defaulted to off: it kept
machine-backed
agents out of the dispatch table, refused node/installer access, hid
remote
preset destinations, and treated file paths locally while off. It was a
trial feature gate, not a verbose-logging option. At the owner's
request,
this PR removes the variable and all of those checks. Normal startup and
restart recovery use the configured machine entries; even a leftover
value
of `0` is ignored. Machine registration, SSH authentication, file
permission
prompts, and concurrency limits still apply.

Fourth change in the series that lets a Raven dispatch work to the
machines in its machine registry. #920, #921 (merged through #922) and
#922 are on main; this change stands on main alone now, and the diff is
its own.

What this does: a turn running on a registered machine survives this
computer stopping, and the next start picks it up and reports it.

- While a turn runs on a machine, this computer remembers what the
  machine does not know (`raven/acp_client/remote_runs.py`): one
  owner-only file per run under `<raven home>/machines/runs/`, with the
machine's registry id, the task id its daemon runs the agent under, the
  agent, the session key, the instance handle, the dispatch id, the ACP
  session, the id of the prompt whose answer is awaited, the ids this
  side has used, how many of the agent's frames were read here, the
  agent's answer to `initialize` and the connection's launch key. Kept
  current after every frame the connection has dealt with, removed when
  the turn ends here. No address is in it.
- When this process stops -- the gateway, `serve`, the TUI, the page
  stack -- the
  agents on machines stay with their daemons
  (`remote.leave_agents_running()` before the drain): a cancelled turn
on a machine sends no `session/cancel`, closing its connection ends the
endpoint only, and the manager leaves the dispatch's record open rather
  than finishing it as cancelled. A hard kill or `/restart` leaves the
  same record, written as the turn ran.
- At the next start -- the gateway once its generation is bound; the
`serve`, TUI and page stacks once --
`raven/agent/subagent/resume_remote.py`
picks every run up: `task/list` on the machine, `agent/attach` from the
  frame last read (`NodePool.attach`), the connection adopted into the
  pool with request ids carried on past the other process's
  (`AcpConnectionPool.adopt`, `AcpClient.continue_ids_from`), the
  prompt's answer awaited (`AcpClient.await_response`), the reply built
  as a turn ended here is -- the partial notice when the agent stopped
  early, the machine note saying it was picked up after this computer
  restarted -- then the dispatch's record finished, the registry row and
  the node claim closed, and the result announced into the conversation
  that asked, as the manager announces a turn that ended here
  (`raven/acp_client/remote_resume.py`). A machine that cannot be asked
  keeps its record for a later start; a run the machine no longer has,
  one whose agent ended while this computer was away, one whose agent
  left the agent table, and one whose turn says nothing are announced as
  errors that say so. The adopted agent process serves later dispatches
  of its entry, as one started here would.
- Seams: `AcpClient.request(sent=)` hears the id of what goes out;
  `ChannelEndpoint.watchers` hear the frames the connection has dealt
  with (`consumed`, called by the read loop); `ChannelEndpoint.stays`
  reads the leaving flag.
- A cli entry's run on a machine is recorded while it runs (kind `cli`,
  no ACP part) and left the same way, the manager reading the dispatch
  as left; the entry picks it up (`CliAgentBackend.pick_up`): the daemon
  replays the command's whole output from its first line, the exit code
  ends it, and the reply closes with the machine note.
- The record carries the task as the model wrote it, and -- once the
  manager has read the run as left -- the dispatch's origin and summary
  (`remote_runs.annotate`): the announce after a restart names the
  instance, node and tool call as the announce here does and quotes the
  task as authored, never the rendered prompt on disk; a host killed
  outright gets an origin made up from the spawn record and the run.
- The record goes last, once the outcome has reached its conversation:
  a conclusion that fails, a stop under it, or an announce the scheduler
  refuses (`_announce_result` says so now) leaves the record, its claim
  released, for the next start to try again.
- A run that returned here keeps its record the same way, until the
  manager's announce of the reply was taken: the manager dispatches with
  `reported_by_caller=True`, the backend then leaves the record behind
  with the reply instead of removing it as it returns, and the manager
  removes it after the announce (`remote_runs.forget_dispatch`) -- or,
  when the draining scheduler refused the announce, annotates it with
  the dispatch's origin and summary for the next start to report. A host
  killed between the agent's answer and the announce therefore still has
  the way back to the run. A caller that takes the reply as its own
  result (a graph node, a probe, a direct chat) dispatches without the
  keyword, and the record goes as it did. The record's frame count stops
  at the prompt: the turn's own frames, the chunks before the answer as
  much as the answer, are never checkpointed, so a pick-up replays the
  turn whole and the reply it reports is complete; a record that had
  moved past the answer would have resumed after it and waited forever.

Not in this change: a run from the DAG lane has no spawn record to
finish; it is announced from the origin the manager added to its record
when it was left, or, for a host killed outright, from the run's record
alone. The Tasks page reads the finished record on its next load rather
than live.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Machine availability and restart checks after removing the trial switch:

- New default-behavior regressions were run before the removal: 8
failed,
  with the enabled-value control passing. After removal, all 9 passed,
  including ACP/CLI recovery through the real AgentRegistry with the
  variable absent or set to `0`.
- `uv run --no-sync pytest tests/test_acp_remote.py
tests/test_node_client.py tests/test_node_server.py
tests/test_node_install.py tests/test_ops_remote_path.py
tests/test_permissions_remote_read.py tests/test_remote_file_tools.py
tests/test_deliver_files_tool.py tests/test_raven_config_tool.py
tests/test_rpc_subagents.py tests/test_shell_machine_channel.py
tests/test_subagent_acp_remote.py tests/test_subagent_cli_remote.py
tests/test_resume_remote.py tests/test_remote_resume.py
tests/test_remote_runs.py tests/test_subagent_manager.py
tests/test_cli_tui_commands.py tests/test_rpc_schema_match.py
tests/test_config_schema.py -n 4 --idle-ceiling-strict --cov=raven
--cov-branch --cov-report=json:<coverage-json> --cov-report= --tb=short`
  -- 1799 passed on the combined tree, including the latest published
result-retention fix. An earlier run had a missing stdio response in the
minimal-node dependency test and an idle-gate hit; both passed a focused
recheck, and this final complete selection passed with the strict gate.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main --threshold 90`
-- 92.92% (656/706 changed executable lines) for this PR; the
gate-removal
commit alone covers 2/2 new executable lines (the remainder deletes
gates
  and updates descriptions).
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`, the
commit/source-language/large-file scope gates and `git diff --check`
pass.
Both RPC generators' `--check` runs pass; `npm test --
scripts/gates/fixture-shape.test.mjs`
  from `ui-web` passes all five tests.
- Real Linux machine, isolated node installations, separate dispatch and
restart processes: before removal, a task started with the trial switch
on was reported as an absent agent and its recovery record deleted when
  restarted without it. After removal, both absent and old `0` settings
  recover the full CLI result and read its output file by address. The
absent-variable case kills the relay before host shutdown. The combined
  final tree was reinstalled and both scenarios repeated successfully.
- With the variable absent, 12 further timing/boundary probes pass:
early
  CLI exit and blank lines, relay loss before heartbeat with normal-call
recovery, tunnel loss, ACP/CLI/OpenAI cancellation during disconnection,
  11 launches admitting 8 and refusing 3, occupied tunnel port, and an
unwritable prompt file. Three attachment, POSIX-command and IPv6 checks
  also pass; Windows command parsing is simulated, not native Windows.
No test task or tunnel remains; the dedicated node directories are
removed.
- CONTEXT.md, configuration and RPC descriptions, transport/path
docstrings,
  both collaboration guides and the owner's design notes were checked
against the code. The guides explain the former trial switch and normal
startup without it. The existing SSH, permission and limit tests
continue
  to exercise their checks with no test-only switch setup.

- `env -u RAVEN_DEBUG_MACHINES uv run --no-sync pytest -n 4 --tb=short
--cov=raven --cov-branch --cov-report=json:<full-coverage-json>
--cov-report=`
-- 31345 passed, 230 skipped, 2 xfailed, 5 failed. A separate
current-main
  checkout reproduces four failures: CLI color rendering on this host,
LibreOffice detection, and the two simulation cases. The remaining
failure
  was the direct-chat test stub rejecting `reported_by_caller` from the
latest result-retention change; the published wrapper fix now forwards
only keywords declared by its fake backend. The full
suite was not rerun for that test-only correction. Final direct-chat and
manager recheck: 277 passed with strict idle accounting. Full-PR
changed-line coverage from this
  run is 96.18% (679/706).

Earlier verification:

- `uv run pytest tests/test_remote_runs.py tests/test_remote_resume.py
tests/test_resume_remote.py tests/test_acp_client_resume.py
tests/test_node_client.py tests/test_subagent_acp_remote.py
tests/test_subagent_cli_remote.py tests/test_cli_tui_commands.py
tests/test_subagent_manager.py tests/test_rpc_bootstrap.py
tests/test_cli_gateway_commands.py tests/test_acp_client_launch.py
tests/test_subagent_acp.py tests/test_scope_canon.py
tests/test_l4_entrances.py`
-- all passed (605 passed). Covered: the record written owner-only, kept
current
per frame and per prompt, removed when the turn ends here and kept when
  this process leaves; a dispatch left at shutdown neither finished nor
  announced by the manager; an endpoint ended while leaving leaves the
  agent with the daemon (real daemon); a task picked up from the frame
  last read, with the watchers hearing each frame; a pick-up the machine
  refuses or cannot carry; an answer awaited for a request another
  process sent, ids carried on, no `session/cancel` while leaving; the
  reply read off replayed frames with the other sessions' frames
  ignored, a turn cut short marked partial, an unreachable machine
  keeping the record, a run the machine no longer has, a prompt never
  sent, an agent that ended while this computer was away, an answer
  waited out, a turn that said nothing, a pick-up beside a live
  connection closing its own; the manager-side conclusion: record
  finished and announced, an error announced and the record failed, an
  unreachable machine keeping the record, an agent off the table, a
  failure inside one pick-up not stopping the next, a run with no
  conversation, the once-per-process scheduling; a cli run recorded
  while it runs and left at shutdown, picked up whole on the real daemon
  behind the stand-in ssh, its pick-up's errors said (a non-zero exit, a
  machine that no longer has it, a link down, this computer leaving), a
  cancel that is not a leave ending the command with no record left; the
  TUI's stop saying the agents stay before the cancel and its start
  scheduling the pick-up; the record annotated with the dispatch's
  origin and summary when the manager reads it as left, the announce
  quoting the task as authored and naming instance, node and tool call,
  with and without the manager's fields; the record outliving a refused
  announce and a failed conclusion, claim released; a cli record read
  without an ACP session, an older record read with defaults; the record
  moving with the frames before the prompt and not past it, a dispatch's
  record forgotten by the manager's id with its claim, the manager
  dispatching with `reported_by_caller` and forgetting the record after
  an announce that was taken and annotating it after one that was
  refused; on the real daemon behind the stand-in ssh, an acp reply and
  a cli output returned with the record kept and the host "stopped"
  before the announce, the next start reading the same reply again
whole, and a dispatch without the keyword leaving no record. End to end
on the real
  daemon behind the stand-in ssh: a turn dispatched, this process
  "stopped" the way the gateway stops, the agent still running on the
  daemon, then a fresh pick-up reading its answer, finishing the record
  and announcing it, with the agent's process the pool's connection
  again.
- `uv run pytest` on this tree, at an earlier revision of this change --
29270 passed, 121 skipped, 3 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
  which fail the same way on main on this machine; diff coverage of this
  change against its base: 97.45% (153 of 157 executable changed lines),
  and 94.67% for the four changes against main; `lint-imports` 10 kept,
  0 broken; `ty` clean.
- Real machine (a registered Linux machine with Claude Code signed in,
  node reinstalled with this tree): process A dispatched a 25 s task,
  saw the prompt out after 2.4 s and the record keep up with the frames
  (6, then 10), then stopped the way the gateway stops 5.4 s after the
  dispatch, leaving the record and the agent running there; process B,
  started fresh, found the record and the running task, picked it up
  in 2.4 s, announced the agent's reply to the conversation with the
  machine note, finished the record, removed the run's record, held the
  agent's process as the pool's connection, and closing the pool ended
  the agent on the machine. This revision, the cli lane with the code
  before and after, the node reinstalled for each: a cli entry
  `sh -c 'echo $$ > pid.txt; sleep 20; echo done; cat'` was dispatched
  and the host stopped the gateway's way 2.7 s later. Before: no
  record, the command left running on the machine, and the next start
  found nothing to pick up. After: the record kept with the task as
  authored and the manager's origin, the command running on, and the
  next start picked it up in 16.8 s (the sleep), announcing `done` and
  `hi` with the instance, node and tool call and the machine note. The
  same with the link killed before the stop: recorded, left, picked up.
  With the machine's daemon ended between the stop and the pick-up, it
  ended its groups on its way out, the task's record there said exit
  -9, and the pick-up announced that as the run's error in 0.1 s,
  leaving nothing on the machine.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
  and `scripts/check_large_files.py` on the branch range, `commitlint`,
  `ruff` -- all clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The English and Chinese agent-collaboration guides explain normal
startup,
registered-machine setup and recovery, and the removed trial switch.

## Risk

Configured machine entries are available by default; entries without a
`machine` remain local. A remote-looking file path is resolved against
the
registry even without an environment variable; use an explicit local
path
such as `./a:b` for a local filename containing a colon. A formerly
hidden
machine entry is now available for dispatch, and an old variable set to
`0`
no longer disables it. Stopping Raven no longer cancels a turn
running on a machine; it goes on there until the agent finishes or is
cancelled, and its answer arrives in the conversation after the next
start, as a sub-agent result that triggers a turn. One record per run in
the raven home, owner-only, removed when the run is concluded; a machine
that cannot be reached keeps it, so a run is reported at most once and
never lost quietly. The adopted connection answers requests with ids
past the ones the stopped process used, so a replayed frame resolves
nothing sent after the restart. Rollback is a plain revert; a record
left
behind is then an inert file.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#920 and #922 (merged); #921 was carried by #922.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-14 21:34:19 +08:00
Handsome-wzw
e3d0135049
feat(*): a Raven daemon on a registered machine, reached over one watched ssh link (#920)
## Summary

First of three changes that let a Raven dispatch work to the machines in
its machine registry on a design where the machine owns the work: a
Raven
daemon runs there, the ssh is only the way to it, and an ssh that goes
mid-task does not end the task. This one puts the daemon on a machine
and
keeps the link to it; nothing dispatches to a machine yet. The next
change
runs acp sub-agents through it, the one after reads and writes files
there
by address.

It follows #893 (the ssh line and the remote working directory) and
replaces #897 and #901, closed when the design changed: their code is in
here, reworked to the daemon.

After the review's second round, four more things hold: a page read
(`read_file`, here and on a machine) reads the file a chunk at a time
and decodes only the page's own lines, so no one step holds the
interpreter for the whole file and the file is never whole in memory;
the whole-file calls on a machine go through one gate (readers
together, a writer alone, a waiting writer first), so a read never
answers with the old page and the new digest; the install reads the
index pip is configured with on the machine the way pip does, names it
in the plan and hands it to `uv pip install`; and a machine without uv
gets one pinned version as an unmanaged install, which writes no
receipt under the home directory and changes no PATH, while a node
directory that was there keeps its mode.

What a daemon is (`raven/node/`):

- This Raven's own code, running as a daemon on a registered machine.
  The host's ssh runs a relay there (`python -I -m raven.node --attach
  --dir <node_dir>`, `raven/node/relay.py`): it bridges the ssh session
to the daemon's unix socket under the node directory, starts the daemon
  when none answers (`--serve`, detached, its words in `log/serve.log`),
and leaves with the ssh. The daemon leaves on its own after ten minutes
with no connection and no running agent; the next relay starts it again,
  so nothing is registered as a service and a reboot costs one connect.
  It opens no network port; the socket is the registered user's own
  (mode 600) in `run/` (0700), or, when that path is too long for a
  socket, in a directory of the user's own under the temp directory
  (0700, checked before use: a directory of the user's, never a link,
  never another user's, narrowed when wider). The relay forwards only to
  a socket of the user's and starts no daemon beside anything else; a
  daemon whose socket's place holds a thing it cannot remove says so in
  one line and leaves rather than listen beside it. ssh login is the
  authentication.
- It speaks newline-framed JSON-RPC 2.0, the framing ACP uses, so the
  host drives it with `AcpClient` (over an `Endpoint` seam this change
  adds: a process, or a channel of a link). Protocol 3.
- File calls (`fs/read`, `list`, `find`, `grep`, `stat`, `write`,
`edit`,
  `pull`) run Raven's own file tools there, fenced to the `roots` each
  call names, so a remote read comes back exactly as a local one would.
  A write may carry `expect`, the digest the file had when it was read,
  and is refused with `CHANGED` when it no longer matches. A page read
  with its digest, a stat with one, a chunk and a checked write run in a
  thread of the daemon's, writers one at a time under a lock, so the
  loop that answers the heartbeat and forwards every agent's frames is
  not held for the length of a large file.
- Agents (`agent/launch`, `frame`, `stderr`, `exit`, `cancel`, `close`):
  the command named is started in the machine user's login environment
  (captured once by the daemon itself, under `bash` or `zsh` `-lic`),
  in its own process group, under a task id of the host's. Its ACP
  frames travel on a numbered channel of the connection that holds it,
  and are kept in the task's record on disk (`<node_dir>/tasks/<task>/`:
`task.json`, `frames.jsonl`, `stderr.log`; the directories 0700 and the
  files 0600, as the host's own journal is) as they arrive. A connection
  that goes leaves the agent running; `agent/attach` on a later
  connection replays the record from the frame the host last read and
  takes the task over, and a finished task is replayed from its record
  alone, even by a daemon that never ran it. `agent/cancel` ends the
  whole process tree: the tree is read off `ps` before the first signal,
so a command the agent put in a session of its own (Claude Code's shell
tool does) goes too. `task/list` names the tasks running or remembered;
  records are kept seven days. `node/shutdown` ends agents only under
  `--stdio`, the one mode that ties them to the connection (tests, and a
  fallback).
- Probes (`probe/machine`, `probe/agent`) and `session/dir`, which makes
  a session directory and says where it is as the machine resolves it.
- Small on purpose: it imports the file tools and nothing that talks to
a
  model, reads a config or opens a port; beside Raven's code the machine
  needs uv (or curl or wget, to fetch one with), a Python 3.12 if it has
  one (else uv brings one), loguru and pillow (about 55 MB, more with a
  fetched uv or Python), ripgrep-bin when its index has it. A test runs
  the node with nothing importable beyond the standard library, its own
  code and those two packages, and asks every file call, so the next
  import added under `raven/agent/tools` is caught here rather than on
  every installed node.

The same code at both ends (`raven/node/bundle.py`): the code is named
by
a digest of the package's files, and `node/hello` compares it, so a
daemon of other code is refused with the command that updates it. An
install lives in `<node_dir>/<version>-<digest>`.

The host side (`raven/node/client.py`):

- One link per machine, shared by every caller in the process; a link
  idle for ten minutes with no agent on it and no call on its way to it
  is closed the next time any link is asked for. A heartbeat every 5 s;
  one unanswered for 15 s means the link is lost.
- A lost link no longer fails the agents on it. Each endpoint waits,
holding what the host writes meanwhile (up to 1000 frames); the link is
  tried again on a backoff (2, 4, 8, 16 s) for as long as any agent
  waits; a link that comes back attaches every task again from the frame
  the host last read, sends what waited and any cancel asked in between.
  Only a task the machine no longer knows ends an endpoint, in one
  sentence. A link closed on purpose just closes: the daemon keeps its
  agents, and `tasks()` on a later link lists them. The listing says
  "link lost: ...; N tasks continue on the machine; reconnecting".
- Every failure is one sentence that names the machine and what to do,
  never its address: not installed (with the install command),
  unreachable, login refused, other code, a refused handshake, a call
  that ran out of time, or the daemon's own last word.

The registry's package (`raven/ops/remote_path.py`) tells a remote path
from a local one (a UNC or extended Windows path, and on a Windows host
a
drive-relative one, are this computer's), resolves it against the
registry, and lends the tools the pool through one seam (`use_nodes`,
`nodes`) -- the tools must not import the node package, which imports
them. `raven/acp_client/remote.py` keeps the ssh command, the machine
lookup, the posix wrapping of a remote command and the one-sentence
reading of an ssh failure from #893, and loses the launch command, the
env preamble and the session-directory ssh that #897 would have used:
the daemon does those. The ssh command is handed to the connection as an
argv, and `AcpClient.launch` takes one as it is, so no host's own
grammar
reads it again (a Windows host's parser split the line into twelve
words). What reaches a model is a path on the machine written the way
scp writes it, address included, because that is how a file there is
named to the tools; the way in -- the key path, ssh's options, ssh's own
messages -- reaches no config file, log line or model, and an error
names
a machine by id and name only. The lane stays
debug-only (`RAVEN_DEBUG_MACHINES=1`, ruled on #897): with it off no
daemon is started and no install is planned, each saying why.

Installing (`raven ops connection install-node <id>`,
`raven/node/install.py`): one look at the machine (the install, its uv
or
a curl or wget to fetch one with, a Python 3.12 if it has one, the free
space), then a question, then everything through uv, as the repository's
dependency rule asks: a uv fetched under the node directory from its
installer when the machine has none, a virtualenv uv makes of the
machine's Python 3.12 or of one it brings, the pinned packages with
`uv pip`, the code as a tarball on ssh's stdin, each step one ssh call
and a failed step removing the half-made install; the node directory is
made 0700. A machine with neither uv nor a way to fetch one is refused
with the installer's address. `--dir` keeps installs elsewhere and
records it on the row; `--prune` removes other versions. Nothing else on
the machine changes. `raven ops connection doctor` reports the registry
as before.

Also: `CONTEXT.md` defines the Raven node and seats `raven.node` inner
in
the import contract (`pyproject.toml`, `lint-imports`); the repo layout
table gets its row; the gateway, the TUI and the RPC entrance close the
node pool with the ACP pool on shutdown.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_node_bundle.py tests/test_node_server.py
tests/test_node_client.py tests/test_node_relay.py
tests/test_node_install.py tests/test_acp_client_teardown.py
tests/test_acp_remote.py tests/test_ops_remote_path.py
tests/test_cli_ops_connection_commands.py
tests/test_ops_connection_add.py tests/test_ops_transport.py
tests/test_subagent_acp.py tests/test_acp_client*.py
tests/test_cli_gateway*.py`
  -- 859 passed on this tree; diff coverage of this change's lines from
  these groups alone passes the 90% gate. Covered, beside what #901's
  tests covered (parity of every file call with the tool's own answer,
  the fence, each malformed field, the handshake, the stdio loop, the
  install's scripts and one real install on this computer, through uv):
  - The socket's directory is the user's own or not used (a file, a
    link and another user's directory refused; wider bits narrowed); a
    daemon refused its socket's place says so and leaves; the relay
    forwards to nothing that is not the user's and starts no daemon
    beside it; a task's records are 0700 and 0600; a ping is answered
    while a file's digest is taken; a call on its way keeps its link;
    the ssh argv reaches the connection as it is and `AcpClient.launch`
    takes an argv; the installer fetches uv when the machine has none
    (curl or wget) and refuses a machine with neither; a drive-relative
    path is this computer's on a Windows host and scp's elsewhere, an
    extended or UNC path this computer's anywhere; the node answers
    every file call with only its requirements importable.
  - A task is recorded as it runs and goes on when its connection ends;
a second connection attaches from the frame the host had not read and
    hears the exit; a finished task is replayed from its record by a
    daemon that never ran it; a record left "running" by a daemon that
    died reads as lost; an attach for a task nobody knows is refused by
code; the task list and the sweep of old records; shutdown from a host
    leaves a daemon's tasks running.
- The daemon on its socket: two connections in turn, the agent surviving
    the first, the daemon leaving when idle and leaving at once when
    another serves the same directory; `--serve` and `--attach` need
    `--dir`.
  - The link, over a stand-in `ssh` that hands the line to a real login
    shell and starts the real relay and daemon: a request made while the
    link is down is answered after the link is back, on the same daemon;
    a daemon told to stop takes its agents and the turn says so; a link
    closed on purpose leaves the agent for the next link to list, attach
    and cancel; a heartbeat lost parks the endpoint and the reconnect
    attaches it again; a task the machine no longer knows ends its
endpoint in one sentence; an attach the link cannot carry parks every
    waiting endpoint again; held frames are capped; a file fetched whole
    and checked, and refused outside the roots.
  - The relay in this process: it starts a daemon when none answers,
    bridges bytes both ways, reaches the same daemon again without
    starting another, and gives up in one sentence when none comes up.
  - Cancel ends a child the agent put in a session of its own; the tree
    is read off `ps`; the daemon's own group is never signalled.
- `uv run pytest --collect-only` -- no import error; `lint-imports` --
  10 kept, 0 broken. The daemon's two test modules skip on win32 before
  importing it (fcntl, unix sockets), and the two POSIX teardown tests
  lend a SIGKILL where the platform has none.
- `uv run pytest` on the final tree of the three changes, which includes
  this one -- 29217 passed, 121 skipped, 3 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
and
`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
  which fail the same way on main on this machine. The groups above
  also pass under `--idle-ceiling-strict`, CI's idle ceiling.
- Real machines (the final tree of the three changes, two registered
  Linux machines, a scratch registry and `RAVEN_HOME`):
  - Install through uv on both: on the machine without uv, uv fetched
    with the machine's curl under the node directory, then a virtualenv
    of the machine's own Python 3.12, 16 s in all; on the machine whose
    home disk was full, under `/tmp`, with a Python uv brought, 11 s.
    Both nodes answered the handshake with this code's digest.
  - Connect and handshake 0.7 to 0.9 s; `session/dir` 11 to 18 ms;
    `agent/launch` 13 ms; a request's round trip 12 ms.
  - With a real Claude Code running a 25 s shell loop: the host's ssh
killed mid-turn, the link read as lost in 5 s, the turn finished 27 s
    after the kill with the agent's reply, 25 of 25 lines written, the
    same daemon on both sides of the gap; a cancel left 0 processes 3 s
later; three sessions at once answered in 8 s; with every link closed
    the daemon stayed with its records and no agent process remained.
- The second round's four notes (`tests/test_filesystem_tools.py
  tests/test_node_install.py tests/test_node_server.py
  tests/test_node_client.py tests/test_read_file_image.py` -- 337
  passed, 100% diff coverage of that commit's lines): a page is the
  lines `splitlines` gives whatever the chunk edge, over every break it
  knows; a 120 MiB file pages with every ping answered within 0.25 s; a
  read with a slowed digest beside a write of the same file answers with
  the old page and the old digest and the write lands after; a waiting
  writer goes before new readers; the probe reads `pip.conf` and
  `PIP_INDEX_URL` as pip does under a real shell, the plan names the
  index and the install hands it to uv; the uv step fetches one pinned
  version unmanaged; a node directory that was there keeps its mode and
  one the install makes is owner-only. On a registered Linux machine
  with the node reinstalled: a page read of a 1 GiB file there beside
  pings every 50 ms kept the longest wait of a ping at 0.34 s; the reads
  sent during a write all answered with a page and a digest of one
  version; the probe found the mirror that machine's pip is configured
  with; an install into a fresh node directory, forced down the fetched
  path, put uv 0.13.0 under it unmanaged in 24 s, with the receipt under
  the home directory untouched and the directory at 700.
- The full review's five findings (`tests/test_node_client.py
  tests/test_node_server.py tests/test_node_relay.py` -- 223 passed, the
  seven new tests failing on the code before the fix, diff coverage of
  that commit's lines past the gate): a link the heartbeat has not yet
  seen die, replaced by an ordinary call, brings its parked agents along
(`agent/attach` on the new link, nothing left parked); a replay the new
link cannot carry keeps every frame it did not take, parks the endpoint
  being replayed and every one not yet reached, and the link is tried
  again; a cancel asked on a dead link, or failing on the new one, stays
  wanted until a link delivers it, and a refusal clears it; two installs
  on one long node directory get sockets apart; a launch still capturing
  the login environment is counted, so a second one at the limit is
  refused before the first spawns. On a registered Linux machine, one
  script run before and after the fix (the local relay killed with the
  heartbeat kept out of the way, then a plain call; a cancel asked in
  that gap; eleven launches at once against the limit of eight): before,
  the agent stayed parked and never answered, the cancelled task ran on
with its record saying running, nine launches started and ten tasks ran
  there; after, the call took 0.3 s and the agent answered through the
  new link, the task ended within the reattach with its record saying
  so, eight started and three were refused with eight running, and
  nothing was left running on the machine. The other half of the first
finding, a relaunch that fails under the call with agents parked: on the
machine, the relay killed and the relaunch made to fail once, before the
agent was still parked 30 s after the machine was reachable again and no
  reconnect was under way; after, the reconnect the heartbeat loss
schedules runs here too, the agent was attached again on the first retry
  after the machine answered and answered through the restored link.
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and `scripts/check_large_files.py` on the branch range, `commitlint` on
  the commit and the prospective squash header, `ruff` -- all clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The command describes itself in `--help`; user docs follow with the
change that runs sub-agents on a machine.

## Risk

Opt-in and inert: nothing starts a daemon unless the debug lane is on
and
a caller asks for a machine, and an install runs only when the owner
asks,
after saying what it will do. On a machine it adds one directory under
`~/.raven-node` (or the row's `node_dir`): the install, a `run/` with
the
socket, lock and pid of this install, a `tasks/` with the records, a
`log/`; removing the directory uninstalls it. The daemon runs as the
registered user, listens on a socket in a directory only that user can
open (and the relay forwards to a socket of that user's only), reads and
writes only inside the roots each call names, and leaves when idle. A
task's record holds the agent's ACP frames, which may quote what the
agent read: it is the user's alone (0700, 0600) and is removed after
seven days. The way in stays in the ssh argv, in memory; no error names
the address, and a model sees it only as part of a path on the machine,
written the way scp writes it. Rollback is a plain revert; an installed
daemon is then an inert directory that leaves once idle.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Follows #893; replaces #897 and #901.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 13:54:16 +08:00
zhanghui
d106a5c6ef
feat(plugins): add the everos-cloud-memory plugin and its host wiring (#883)
## Summary

Adds `everos-cloud-memory`, a separately distributed memory plugin for
EverOS Cloud, and wires the host so a second memory plugin is a
first-class choice.

The plugin (`plugins-dist/everos-cloud-memory`) is a thin REST client
over httpx: one API key (the config slice
`plugins.config.everos-cloud-memory.api_key`, or
`EVEROS_CLOUD_API_KEY`), owner ids from the host's `ServiceLocator`
(never from the slice), every add in `agent` mode, and a flush only when
a caller marks the conversation over (`metadata.flush` /
`metadata.is_final`: the sub-agent handoff and the importer's final
batch) and once at stop. It ships its own onboard screen (env or on-file
key used without a prompt; 401 / 403 / unreachable get their own
sentences with Re-enter or Skip), `delete` answers False (the cloud
deletes by scope, nothing calls it), and `health()` classifies seven
answers for `raven doctor` and the settings card.

Host changes, each guarded so `memory.backend` `everos` and `null`
behave exactly as before:

- `raven onboard` step 4 asks "Which memory backend?" when more than one
plugin contributes a screen (preselects the current backend, `Off`
clears it, cancel exits 1 and writes nothing); with one plugin the step
is byte-identical to today.
- The memory page reads whichever backend `memory.backend` names: the
cloud slice's key goes out as a Bearer header and the `/health` probe is
skipped for the cloud; a missing distribution is named in the page's
note.
- `settings.everos` reports `reason: "other_backend"` for any non-EverOS
backend, so the Model page hides the three EverOS-only roles (embedding
stays: knowledge bases use that pin); `settings.everosSet` refuses those
three sections for another backend; `settings.set` admits
`plugins.config.<id>.<field>` against the plugin's manifest through the
same admission door channels use; new `settings.everosCloud` feeds the
card (key source, endpoint, health).
- Provider-key and main-model saves no longer restart the local EverOS
when another backend is active.
- ui-web: an EverOS Cloud card (status chip, key row, endpoint) and the
`settings.everosCloud` data behind it. No page mounts the card: it drew
on Model settings until the owner ruled that a memory backend's endpoint
and credential answer neither question that page asks, and where it
belongs is the Memory section's to say once that section has a design.
Until then the key is entered through `raven onboard` or
`EVEROS_CLOUD_API_KEY`. The fixture gate pins the four optional fields
the offline library never sends.
- Gates: the plugin boundary test scans both memory plugins, coverage
and the release workflow build the second wheel, deptry's ignore list
names the plugin beside `raven_everos`, and a conftest fixture keeps the
second plugin out of entry-point discovery for the existing onboarding
tests (they describe a one-plugin install).

A second topic rides along, because it is what makes the hosted backend
reachable from an agent product at all. Every product under `agents/`
pinned its memory at the stock EverOS address and only `raven-design`
read the host's, so the other four addressed whatever answered on 18791
-- the wizard moves a managed server when the port is taken, and a
server the user runs themselves is wherever they put it.
`inherit_memory_backend` in `raven/config/product_render.py` now carries
`memory.backend`, the endpoint, and (only when the product holds none
itself) the credential from the host into every render, beside the host
denials and plugin opt-outs that already travel there; `raven-design`'s
private copy is deleted and the five product configs stop naming an
address. Identity never travels, which is what keeps five products apart
inside one store. `memory.backend: null` travels like any other value,
because that null is how an operator turns long-term memory off and a
product that kept writing would be the one surface ignoring them; the
product's own default stands only when the host carries no key at all.
Which store memories live in is not a product's choice either: the host
reads a sub-agent's memories back through its own backend, so a product
writing anywhere else left that record empty with nothing reporting it.
`owned` is forced off whenever the local backend is in force, because a
product infers ownership from a slice naming no root -- the shape all
five ship -- and so answered a host that was merely not serving by
starting a second server against a root of its own, stranding every
memory written so far.

Decisions worth a reviewer's eye: the fake cloud
(`tests/_everos_cloud_fake.py`) honours or refuses every field the
backend can send and serves both an in-process transport and an HTTP
face for the real-host acceptance; the live tests
(`tests/integration/test_everos_cloud_real_cloud.py`) skip without
`EVEROS_CLOUD_API_KEY`; run with a real key on 2026-10-09 all six pass
against api.evermind.ai (round trip, get by session, empty flush, 401
wording, probe with and without a key, and extraction without a flush
within two minutes), so the documentation and the service agree on
everything the plugin sends. Design, acceptance cases and plan are under
`docs/specs/2026-10-08-everos-cloud-memory-*.md` and `docs/plans/`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Rebased onto main `182dacac0` on 2026-10-13. What that rebase and the
four commits on top were verified with:

- Full suite, `uv run --frozen --python 3.12 --extra dev pytest -q` ->
**29748 passed, 8 failed** in 4m10s. All eight fail identically with
these four commits' sources reverted in-tree at the same base, so none
of them is this branch's: the two `test_the_products_tool_face_*` cases
in the code and oncall launchers (an extra `web_search` in the visible
tool set), `test_agent_loop_token_budget`,
`test_rpc_files::test_a_host_without_libreoffice_says_so`,
`test_ppt_engine_image_search`, `test_shell_background`, and the two
`test_simulation_*` cases.
- Real-host before and after for the inheritance, driving the launcher
the host spawns (`agents/raven-research/run.py`) against an isolated
`RAVEN_HOME` whose EverOS address is `19701`, not the stock port.
Before: the product rendered `base_url: http://localhost:18791` with no
`owned` key, ignored the host, and started an EverOS of its own -- an
orphaned `everos server start --root <its own state root>/everos`
holding 18791, which is the stranding this change exists to stop. After:
it rendered `{"base_url": "http://localhost:19701", "owned": false}` and
said so itself, "the EverOS you manage is not running at
http://localhost:19701 ... Raven does not start or stop it", with `ps`
and `lsof` confirming nothing was started. On the cloud the product
rendered `memory.backend: everos-cloud` with the host's key and left the
out-of-force local slice untouched, and `raven doctor` over that
rendered config answered `everos-cloud: ok https://api.evermind.ai`, so
the credential reached the service. Identity stayed `raven-research-ng`
throughout, and a fingerprint of the operator's own EverOS root was
identical before and after.
- The first CI round on this head found one that the local gates did
not: `tests/test_import_cycle_budget.py`. `inherit_memory_backend` read
the backend-to-distribution pairing from `raven.core.plugin_stack`, and
that edge from `raven.config` into `raven.core` -- which imports
`raven.config` back -- took mutual pairs from 13 to 14 and packages
inside a cycle from 18 to 22. `make lint-imports` keeps ten contracts
and none of them covers that pair, so it was green throughout. The
pairing now lives in `raven/config/raven.py` beside the two fields it
joins (`MemoryConfig.backend` names a contribution,
`PluginsConfig.config` is keyed by the distribution), which `raven.core`
may import and which imports nothing back; `plugin_stack` re-exports the
two names its own callers read from it. An intermediate commit put it in
`raven.contracts` instead, which broke the cycle but took the papers
ceiling from 3700 to 3719 lines -- `tests/test_kernel_budget.py` says a
new paper is a reviewed change to that ceiling, so it moved again rather
than raising it.
- The review round on this head raised one non-blocking finding, fixed
rather than carried forward: the inheritance read `memory.backend` with
`.get()`, which collapses an explicit `null` into an absent key, so a
host with long-term memory off left every packaged agent on its own
shipped backend and still writing. Reproduced by calling the function
directly before the change and after, now read on presence, and pinned
by a case whose mutation (restoring the `.get()`) turns it and nothing
else red.
- One ordering problem found while checking the above, pre-existing and
not fixed here: `tests/test_rpc_settings.py` run before
`tests/test_cli_onboard_commands.py` leaves
`raven.home._current_config_path` pointing at a dead tmp config, so the
onboard cases write somewhere else. With main's own copies of both files
and main's conftest in the tree it is 57 failures; with this branch's,
19. Which cases turn red changes run to run as xdist distributes them
differently.
- `cd ui-web && npx vitest run` -> 876 files, 3728 passed, 0 failed;
`npm run lint` -> 0 errors (4 warnings, all in files this change does
not touch); `npm run gen:check` -> in sync (228 methods); `npm run
type-check` -> clean; `cd ui-tui && npm run lint:i18n && npm run
lint:rpc` -> in sync
- `make lint-python` / `lint-imports` (10 contracts kept) / `lint-types`
/ `lint-deps` -> all pass; `uv lock --check` -> exit 0 with both sides'
new packages present; `scripts/check_source_language.py` and
`scripts/check_large_files.py` on `github/main..HEAD` -> exit 0;
commitlint over every commit -> ok
- Semantic-conflict audit of the rebase, beyond what the suite answers:
21 of the branch's 66 files are also in main's 499, and every one of
main's changes in them is additive and elsewhere in the file. The one
that could have broken silently is main's new `vision.model` /
`vision.provider` simple keys meeting this branch's new
`plugins.config.<id>.<field>` branch in `settings_set`; nothing in the
suite exercises `vision.*` through that method, so it was run directly
-> `{'applied': True}` and `{"model": "..."}` on disk. The subtraction
decisions were re-checked against the new base: no new caller of a
memory backend's `delete`, main's new onboard cases are about TLS and
not the memory step, `inherit_everos_address` exists nowhere, and
`DEFAULT_EVEROS_BASE_URL` is still defined.
- Mutation runs on the new code, restored afterwards: dropping `owned:
false`, letting the host's credential overwrite the product's own, not
writing the host's backend name, and carrying every memory slice instead
of the one in force each turn their own cases red; making the cloud card
render nothing turns 4 of its 5 new cases red and leaves the fifth
("draws nothing when not selected") green.

From the earlier round, on base `6d509a8c5`:

- `uv run --frozen --python 3.12 --extra dev pytest -p no:cacheprovider
-q tests/test_everos_cloud_fake.py tests/test_everos_cloud_backend.py
tests/test_everos_cloud_backend_contract.py
tests/test_everos_cloud_onboard.py tests/test_everos_cloud_discover.py
tests/test_cli_onboard_commands.py tests/test_rpc_memory.py
tests/test_rpc_settings.py tests/test_plugin_boundary.py
tests/test_rpc_schema_match.py tests/test_rpc_registration.py
tests/test_release_plugin_list.py
tests/integration/test_everos_cloud_real_cloud.py` -> 1001 passed, 6
skipped (the six live tests, no key)
- `cd ui-web && npx vitest run` -> 214 files, 3211 passed; `npm run
lint` -> 0 errors (4 warnings, all in files this change does not touch);
`npm run gen:check` -> in sync (207 methods); `cd ui-tui && npm run
lint:rpc && npm run lint:i18n` -> in sync
- `make lint` (ruff, ruff format, ui tsc, deptry) -> ok; `make
lint-types` -> All checks passed; `make build-ui` -> boot snapshots
match; `scripts/check_large_files.py` and
`scripts/check_source_language.py` on `6d509a8c5..HEAD` -> exit 0;
commitlint and `scripts/check_commit_messages.py` on each commit -> ok
- Mutation runs per change (restored afterwards, `git diff` clean): a
host import of the plugin turns the boundary test red; `import
raven_everos` in the plugin turns the import-isolation test red;
removing the chooser's cancel exit, the Bearer header, the restart
guard, the slice-admission guard, the Roles filter and the fake's two
fixes each turn their own test red
- Real-host acceptance (64 cases,
`docs/specs/2026-10-08-everos-cloud-memory-acceptance.md`) run against a
fake EverOS Cloud: the wizard in tmux (chooser, key typed / from env /
on file, 401 / 403 / unreachable wording, back, Off, Ctrl-C,
non-interactive, base-tree transcript byte-identical with one plugin),
`raven agent` turns (user and agent-track recall, agent-mode adds, 4 s
recall bound, a 300 KB message dropped with a warning, a sub-agent trace
stored under its own ids, flushed and read back, one exit flush per
unflushed session), `raven import run` (3 adds then one flush; a hung
flush retried 4x and the exit sweep defeated), `raven doctor` under
seven cloud conditions (seven sentences, exit 2 only for the five
faults) and with the distribution hidden, and the web page through
playwright-cli (memory page counts and the fake's episode, empty and
refused stores, the Model page hiding the three EverOS-only roles while
the embedding row stays, the cloud card rotating a key and showing the
401 wording, three refused settings.set frames over the gateway
WebSocket, the local everos / null / mem0 backends drawing the page as
before, and a provider-key save that restarts the local EverOS on the
base tree and does not on this branch). Evidence and the per-case
verdicts are kept outside the repo. The live service was reached with a
real key after the acceptance run: six tests, six passes, the service's
replies archived with the evidence.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible changes: the wizard's step 4 chooser (only when two memory
plugins are installed); Model settings hides the three EverOS-only role
rows when `memory.backend` is not `everos`/`null` and draws no cloud
card; the memory page sends a Bearer header for the cloud backend.

Two behaviour changes for agent products, both on the `agents/` lane
only. An agent product now follows the host's `memory.backend` and
endpoint instead of its own, which is a fix for the four products that
never followed the host and a change for `raven-design`, which followed
the endpoint but not the backend. And a product no longer starts an
EverOS server of its own: where it used to answer a host that was not
serving by starting one against an empty root of its own, it now reports
long-term memory as unavailable, which is what the host does. Rollback
for this part is the one commit; the product configs it edits are
shipped files, not user config. The plugin is not installed by default
(its wheel is built by the release workflow; nothing changes for an
install without it). Rollback: revert the squash commit; the config
schema gains no new key (the slice is a plugin slice), so an older raven
reads a config written by this one.

Credentials: the cloud key is written only by `set_plugin_config_fields`
into the plugin's slice; the settings page's key row goes through
`settings.set` with a manifest-declared field, and
`settings.everosCloud` reports `api_key_set` / `key_source`, never the
value.

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 11:08:11 +08:00
江国庆/Forrest
15e6549842
feat(trajectory): remember redaction decisions across bug reports (#941)
## Summary

The bug report review now remembers the user's redaction decisions.
Every keep
or replace answer on a suspected token is stored on this machine and
replayed
the next time the same value shows up, so the user is not asked twice.
The
store is a standalone module so the future web UI bug report can use it
as-is.

Data layer: `raven/trajectory/redaction_history.py` (new, no CLI
imports)

- Stored in `<raven home>/redaction_history.json` (mode 0600), keyed by
the
sha256 of the token value. Only the hash and a masked sample are kept,
never
the plaintext: a value the user chose to replace is a confirmed secret.
- `resolve()` decides an item, or a whole linked group, only when every
member
has a recorded and agreeing decision. Confirmed-sensitive (private key)
items
  are always asked.
- Each human decision gets a fresh random revision. A replay only counts
a
reuse while the record still carries the revision it read, so a stale
replay
from another process never overrides a newer answer or revives a
forgotten
  record.
- Writes run under the `atomic_io` write lock but do their own read,
backup
and replace (the shared `atomic_update` reads before its callback and
does
not clean up temp files on failure). An unreadable file reads as empty
and
is backed up with an exclusive create before it is replaced; a file from
a
newer schema is never overwritten. Every file-system or lock failure
raises
`HistoryStorageError`, and results are returned only after they are
saved.
- `forget()` checks and deletes in one transaction and removes nothing
when any
  id is unknown or ambiguous.

Review flow: `report-bug` and the browser's "Report a bug"

- Precedence is explicit flag, then history, then asking. Replayed items
are
listed (with or without a TTY); only the rest are asked. Human decisions
are
  recorded once the decision set is valid, including when the user then
  declines to ship; a cancelled review records nothing.
- The blanket `--keep-findings` / `--redact-findings` flags neither read
nor
  write the history: the user never looked at those values one by one.
- Without a TTY, items decided from history no longer need keep/redact
flags.
`--accept-risk` is still required and is checked before any decision is
  applied or any history write happens.
- New `report-bug --review-all` ignores the history for one run and
replaces
  the remembered answers.
- Storage failures and unreadable files only print a warning; the report
  proceeds.

Management: `raven trajectory redaction-history list|forget|clear`, thin
wrappers that only show what the module returns; `clear` needs `--yes`
without a TTY.

`RAVEN_BUGREPORT_REDACTION_HISTORY=off` disables replay and recording.
Docs
(English and Chinese trajectory debugging guide), `CONTEXT.md` (new term
Redaction History) and `CHANGELOG.md` are updated. Package and record
formats,
`freeze_export` and the review module are unchanged.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_trajectory_*.py
tests/test_cli_trajectory_*.py tests/test_rpc_trajectory.py
tests/test_living_docs.py -q`: 1422 passed (108 new tests: 72 data
layer, 36 CLI and browser).
- `uv run pytest tests/ -q`: 29843 passed, 54 failed, 2 errors. Four of
them (`test_core_tokenizer*` collection errors and two
`test_cli_resources_commands` failures) came from a coverage run
re-syncing the virtualenv at the same time and pass when rerun alone.
The other 52 are this dev machine's known environment failures (a2a
interop, whatsapp websockets, subagent probes and backends, provider
proxy settings, NAT64 network checks, everos server, tracing viewer
health); the ones I reran on main fail there too, and none touch the
changed code.
- `uv run pre-commit run --from-ref origin/main --to-ref HEAD`: passed.
- `make lint-imports`: 10 contracts kept, 0 broken. `uv run ruff check`
and `ruff format --check` on the changed files: passed. `uv run ty
check` on the changed modules: passed.
- Diff coverage (`scripts/coverage_gate.py diff --base-ref origin/main
--threshold 90` over the related tests): 97.27% (498/512 changed lines).
- `make docs-build` (mkdocs `--strict`, English and Chinese): passed.
- `make check-source-language` and `make check-large-files`: passed.
- `uv run raven trajectory regression validate --all`: 2 cases, 0
problems.
- Manual: `raven trajectory redaction-history list|forget|clear` under a
temporary `RAVEN_HOME` (table output, all-or-nothing forget, `clear`
refusing without `--yes` and without a TTY, file mode 0600).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes: a bug report asks fewer questions and prints a
block
listing the decisions taken from history; `report-bug` gains
`--review-all`;
`raven trajectory` gains the `redaction-history` subcommands; a new file
`redaction_history.json` appears in the Raven home after the first
review.

The known trade-off: a real secret the user once kept is kept silently
in later
reports. It is never silent on screen (every replayed decision is
listed), and
`--review-all`, `redaction-history forget|clear` and the environment
switch undo
it.

Rollback: set `RAVEN_BUGREPORT_REDACTION_HISTORY=off` to restore the
previous
behaviour, or revert the PR; deleting `redaction_history.json` clears
the
history. No package, record or RPC format changed, so nothing needs
migrating.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-14 11:05:07 +08:00
江国庆/Forrest
ef3b4bf1b2
feat(trajectory): receive and upload bug report packages (#926)
## Summary

Bug Report Packages can now leave the machine and be received on the
other end.

**Receiver.** `raven trajectory receiver --save-dir DIR` runs a
standalone aiohttp
service that takes packages in, checks them in a stream without
extracting
anything (bounded decompression with a per-header cap, sparse members
refused,
member rules, manifest hashes, the `needs_review` / `risk_accepted`
invariant),
stores the original bytes under `<save-dir>/<UTC day>/` next to a
receipt
sidecar, and answers idempotently by report id (same bytes: the stored
receipt;
different bytes: 409). The sidecar is written before the package is
renamed into
place, so a package in a date directory is always a committed one; a
damaged
sidecar is rebuilt from the package. There is no authentication by
design: abuse
is bounded by a per-address token bucket, a concurrency cap, daily byte
and
package budgets, a free-disk reserve and idle/total timeouts, and the
budget is
settled by what the commit task actually did, not by the HTTP outcome.
`X-Forwarded-For` is believed only behind a `--trusted-proxy` network.

**Upload module.** `raven/trajectory/upload.py` carries no user
interface, so the
CLI, the interactive browser and a future web page call the same
functions. The
destination is one string: `DEFAULT_UPLOAD_URL` as shipped (empty here,
meaning
no upload) overridden by `RAVEN_BUGREPORT_URL` (set it empty to switch
uploading
off). A URL must be https or http on this machine unless
`RAVEN_BUGREPORT_ALLOW_HTTP=1`. The whole exchange runs under one
deadline that
cancels the request, the response is read as raw bytes under a cap
(compressed
answers refused), a receipt counts only when every field agrees with the
request, and the outcome is written into `record.json.upload`.

**Wiring.** `report-bug` and the browser's *Report a bug* resolve the
URL before
any side effect, disclose `Will upload to <origin>` on the confirmation
page,
upload once the package lands, and on failure keep the package and name
the
retry command. The browser's *Bug reports* screen gains an *Upload*
action and
`raven trajectory upload-bug-report REPORT_ID` sends a filed report
again.

Docs: a "Send and receive bug reports" section on the trajectory
debugging page
(en + zh), two CONTEXT.md terms (Bug Report Receiver, Upload Receipt),
CHANGELOG.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_trajectory_package_check.py
tests/test_trajectory_receiver.py tests/test_trajectory_upload.py
tests/test_cli_trajectory_commands.py
tests/test_cli_trajectory_browse.py tests/test_trajectory_bugreport.py
tests/test_docs_site.py -q` -> all pass (about 480 new tests across
them)
- `uv run pytest tests/integration/test_bug_report_upload_e2e.py -q -o
addopts=""` -> 3 passed (a real receiver process, a real `report-bug`
run, a real retry, SIGTERM shutdown)
- `uv run pytest tests/ -q -p no:cacheprovider` -> 28620 passed, 52
failed, 144 skipped; the 52 failures are pre-existing on this machine
(network/proxy/NAT64-dependent tests) and identical to the `main`
baseline
- `uv run ruff check . && uv run ruff format --check .` -> clean
- `make lint-imports` -> 10 contracts kept; `make
check-source-language`, `make check-large-files` -> pass
- `make docs-build` (mkdocs `--strict`) -> pass
- Manual: receiver + `report-bug --yes` (201), repeat (200), different
content under the same id (409), the interactive browser driven through
a pty, failure with the receiver down and retry with
`upload-bug-report`, a 128 MiB upload refused with 413 after ~2 MB both
directly and through an nginx container (`client_max_body_size`). The
nginx-to-receiver hop itself could not be exercised on this machine
(rootless docker cannot reach host services).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Off by default: with `RAVEN_BUGREPORT_URL` unset and the shipped
default empty, `report-bug` and the browser behave exactly as before;
the receiver is a separate process nobody runs by accident.
- The receiver authenticates nobody on purpose; the docs require a TLS
proxy for anything beyond loopback and say not to expose the port
publicly. Packages are redacted for credentials, not anonymized, so
`--save-dir` is sensitive.
- Rollback: unset the variable and stop the receiver. The new
`record.json.upload` fields are optional and older readers ignore them;
the receiver's on-disk layout is plain files by day.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-13 17:52:12 +08:00
Kevin Hu
82c76a82a9
feat(knowledge): read every format, show what the index holds, and let an agent search it (#489)
<img width="2318" height="1448" alt="image"
src="https://github.com/user-attachments/assets/bca71d69-4972-420c-b14e-c8cf74a17de9"
/>

## Summary

Everything a knowledge base could not read, and the pieces it cut them
into
saying what they are.

**Reading a PDF.** A PDF records glyphs at coordinates; whether they are
a
heading, a table cell or a running footer is inferred from how they
look. There
are now two readers. The geometric one infers structure from type size
and
position and needs nothing but the file. The other asks deepdoc's
models, ported
into `raven/knowledge/parser/deepdoc`: a layout model that says what
each region
is, a detector and recognizer that read a page carrying no text at all,
and a
table-structure model that rebuilds a grid from a picture of one. 
 Detecting boxes around text the file already spells
out exactly costs an inference pass per page to learn less, and
recognising it
costs accuracy.

The layout graph answers with a shape its own postprocessing
raises on, and its classes are DocLayout-YOLO's order rather than the
list
Both were established by running the model, and the label order is
pinned by a test.

**Tables go in as tables.** `<table>` is the only shape that keeps what
a grid
states -- which column a cell stood under, which cells are merged, which
row is
the header -- and it is the shape a model reads a table in, which is
what these
chunks are for. PDF, Word and PowerPoint all state their merges exactly,
and
they now survive as `colspan` and `rowspan`. The flat form this replaces
had to
repeat a header standing over two columns across both of them, because
there was
nowhere to record a span; it had no callers left afterwards and is gone.
Two
deviations from upstream, both about what the string is for: every
cell's text
is escaped, where upstream interpolates it raw and a cell holding a `<`
produces
markup that is no longer a table, and the span attributes are written
only where
there is a span, where upstream emits `<td  >` for every ordinary cell.

**A picture of where each chunk came from.** A chunk is a few hundred
words with
a page number on it, which is enough to cite and not enough to answer
whether
the parser read the page correctly. So every PDF chunk that knows where
it sits
carries one image: the regions it covers, cropped out of the rendered
pages and
stacked. here they are already structured in the parser's element spans,
so
the markers would only put text nobody wrote into the thing being
embedded.
The panel draws a chunk's tables as tables, parsed and
rebuilt as elements rather than injected: a person can rewrite any piece
from
that same panel, so what comes back from the store is user input however
it got
there.

**The page that reads them.** Upstream retired the knowledge page while
this
branch was out; it is rebuilt against the new architecture, and the half
that
never existed is the one a reader needs -- what a document actually
became once
it was cut and embedded. Documents sit in one-level folders and arrive
by four
routes: a file, a folder, a URL, a written note. Adding and indexing are
two
calls on the contract and nothing drains that queue on its own, so a
page that
stopped after the first left every upload saying Queued for good.

Opening a file takes the whole page: the document on the left, the
pieces the
index holds of it on the right. The left half is the file drawn as its
own
pages rather than framed by the browser's PDF viewer, which reads where
to open
once and answers to nothing afterwards -- and which nothing can be drawn
on.
Pages are rendered once and kept, and the region a piece was cut from is
washed
over the page it came from, placed in fractions of the page so the
picture's
own scale never enters the arithmetic. Clicking a piece takes the file
there;
double-clicking opens it for rewriting.

The pieces can be searched, filtered by state, turned on and off,
deleted,
rewritten and added to, one at a time or in a batch. Every one of those
was
already on the contract with no reader for it. A base's settings are
back and
fuller: how many pieces a search returns, how documents are cut, and how
much
prose comes with a table or a figure. The embedding model is a picker
while the
base holds no pieces and a stated fact once it does -- moving it then is
the
collection made again at the new width and every document re-cut, which
is not
a thing a panel does by being saved.

**An agent can read them.** The retrieval engine was there and nothing
could
reach it: no agent had a tool that opened a library. A reader now points
a
conversation at bases from the composer, the way they pin a working
directory,
and the selection rides in the same place -- the session's metadata,
bound
around the turn.

The tool takes a query and never a base. The scope is the reader's
choice, so a
model cannot reach material nobody offered it, and it is withheld where
nothing
was picked rather than offered and refused. A hit says which document
and page
it came from, because a hit a reader cannot place is a hit they cannot
check --
and the passages are pressable in the transcript, walking into the base,
the
document, the folder it is filed under and then the piece. A citation a
reader
cannot follow is a claim they have to take on trust, and the whole point
of
searching their own material is that they can go and look.

**The word segmenter and the resources.** `raven.core.Tokenizer` is
Infinity's
`rag_tokenizer`, ported: a forward and backward maximum match over a
dictionary
trie, with stemming and lemmatisation on the latin path. XGBoost is not
carried
over. None of the data lives in git -- the dictionary, the NLTK corpora
and the
ONNX graphs run to about a hundred megabytes, so `make fetch-resources`
on an
install and an optional Docker layer behind `RAVEN_RESOURCES` both point
at one
script. A checkout that could not reach the network still works, and
every
feature that reads this data says so when asked.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on macOS against Python 3.12, with the deepdoc weights fetched (the
model
tests skip without them):

```
uv run pytest                        27450 passed, 111 skipped, 3 failed
npm run test --prefix ui-web          3276 passed
npm run test --prefix ui-tui          2106 passed
npm run type-check --prefix ui-web    clean (tsc)
npm run lint --prefix ui-web          clean (4 pre-existing hook warnings)
make lint-python                      clean (ruff check + format)
make lint-types                       clean (ty)
make lint-imports                     10 contracts kept, 0 broken
make check-source-language            clean
make check-large-files                clean
make build-ui                         built, both boot snapshots match
node ui-web/scripts/check-css.mjs     clean
node ui-web/scripts/check-class-namespace.mjs  clean
```

The three Python failures are not this branch's and were each confirmed
on the
base alone: one asks a host without LibreOffice to refuse a render and
this host
has LibreOffice installed, and two simulation cases fail on an
unmodified
checkout of the base commit.

The RPC contract was regenerated for both consumers and both match it
(`npm run gen --prefix ui-web`, `npm run gen:rpc --prefix ui-tui`, then
`gen:check`). The page's boot snapshots and region goldens were
refreshed where
the composer bar gained a button, and each records that button and
nothing else.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

A PDF's tables, and a Word or PowerPoint file's, are now indexed as HTML
rather
than as header-prefixed lines. Chunk text changes shape for those
documents, so
a base indexed before this and searched after it holds both forms until
its
documents are reindexed; retrieval keeps working either way, because
both forms
are text. The same applies to how much prose a table or a figure carries
with
it: the setting now reaches as far as it says, where before it stopped
at the
nearest paragraph, so a chunk cut before this keeps the narrower context
until
its document is rebuilt. Rolling back is reverting the branch and
reindexing.

The deepdoc models are optional. An install that never ran `make
fetch-resources` reads PDFs geometrically, exactly as before, and says
so when
the models are asked for by name. Nothing here reaches the network at
request
time.

Chunk crops and rendered pages are written at index time and on first
view, both
under the state directory rather than in the repository, and both are
deleted
with their document. A page is immutable for the life of its document
id, so
the route serving them caches; the document route does not. All three
routes
take ids and no path, and the two image routes answer under the
strictest
sandbox in the tree.

What an agent may search is the reader's choice and nothing else. The
tool takes
no base argument, so a model cannot name one, and it is absent from the
schema
entirely for a conversation that named none -- a lane nobody pointed at
a base
is unchanged by this branch. The bases a session carries are stored
beside its
working directory and travel no further.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-12 19:05:16 +08:00
江国庆/Forrest
fa29b27449
feat: add the developer-only trajectory view to the web page (#877)
## Summary

`raven --dev` and `raven web --dev` now serve a trajectory view in the
web page: an icon toggle in the chat header, left of the workspace panel
button, once a conversation has content. The view lists every recorded
step of the conversation with its timing, opens the details of any step,
and draws a duration bar above the list. A normal launch is unchanged:
no capability, no toggle, `/health` reports `trajectory_view: false`,
and `raven web --dev` against a running normal service refuses to attach
and says how to proceed.

Backend (`raven/trajectory`, `raven/observability`, `raven/rpc`,
`raven/cli`):
- `entries.py` projects the span log into trajectory entries with stable
ids and a single Timing Owner per logical span, so the bar never charges
a duration twice; broken records keep unique ids and in-progress spans
charge nothing. A turn opens with `user.input` and closes with
`turn.end`, which owns the turn's whole clock. The projection marks the
rows the list need not show (`meta.hidden`: a turn's end that repeats
the turn's last model output word for word, a turn's end of a turn that
finished ok with nothing recorded in it, an internal step that recorded
nothing) and what every model input adds to the conversation before it
(`meta.delta` first / continued / independent / unknown, with
`new_from`, `echo_at` and `message_count`, proven by an ordered prefix
over the v2 message references within one chain and purpose, the
predecessor's own output written back into the history not counted as
news). Tool results that trip an error rule (an error key, a true
is_error, a false ok / success, an error status word) fail the output
entry with the rule as evidence. A model input's row previews the first
message with text that is not a system one of what the call brought,
read from an inline payload or learnt by bounded repair reads the index
queues, which keep at most five probes per input and treat a failed read
as terminal. A shell that a long tool list pushes past the 64 KiB
preview read still gives its message references, which precede the
tools; an input whose references cannot be read at all leaves unknown
only the input it may have preceded directly. A span whose preview pass
raises (a result nested too deep to parse, a pointer that cannot name a
file) marks its own entry unreadable and never stops the session's
index.
- `observability/purpose.py` names the purpose of each model call
through a context variable that the `llm.call` span records as
`llm.purpose`; the main loop (its wrap-up when the iterations run out
included), watch work, titles, the permission judge, DAG verdicts,
memory extraction, the skill gate and playbooks each set theirs around a
call that records an `llm.call` span, so side questions are told apart
from the main line.
- `index.py` indexes one session's spans incrementally across the
archive chain and the active file, verifies log identity on open,
detects same-size rewrites, serves change snapshots with cursors bound
to an epoch and a revision (up to four list walks at once), and queues
the one blob read a continued input needs for its preview. One index
refreshes one scan at a time, even when the request that started a scan
was cancelled; a spilled sub-agent trace whose first log file was
removed is recovered from the files still there, and the gap is counted.
- `details.py` describes an entry's blocks and reads them in pages under
bounded limits, with integrity findings (missing, outside the store,
oversized, unreadable artifacts) built from the same pass. A
memory.feedback step's skills carry the names its own turn's
skill.inject step recorded. The usage block is attributes only, the
timing block is left out when nothing was charged, empty optional blocks
are left out, the tool block is gone (its fields live in timing and
error), an outline block lists a message list page by page (role, size,
first words and the cursor of the page each row starts, read from the
first 8 KiB of a large message) so a long prompt can be folded before it
is read, and two blocks under the raw record serve the files the span
names: a directory with the recorded size and a cursor per file, and one
file's content by its cursor, as JSON when it parses and text otherwise,
saying when it was cut at the 512 KiB read limit or to fit one response.
- Five RPC methods, `trajectory.state`, `trajectory.list`,
`trajectory.changes`, `trajectory.detail` and `trajectory.block`, with
error codes -32020 (disabled), -32021 (entry not found), -32022 (cursor
expired) and -32023 (revision changed); the `trajectory-v1` capability
is announced only by a process launched with `--dev`, which `web`,
`serve` and `gateway` all accept, and which the page-triggered upgrade
carries into the relaunch; a root `--dev` ahead of a subcommand is
refused with the form that works. Detail and block reads refresh the
index first, and while a rebuilt index is still scanning, an entry it
has not reached yet is answered with its new epoch rather than as gone;
response pages are measured in the UTF-8 bytes the transports send.

Frontend (`ui-web/src/features/trajectory`):
- The icon toggle and the parked conversation scroller; the fixed-row
virtual list over the index's visible rows, with live appends followed
near the tail (a selection from the bar or a link brings its row into
view once and leaves the following alone), stable selection, and
per-session memory of the place, the selection and the zoom. Hidden rows
stay out unless "show hidden entries (n)" is on or a link pointed at
one, which reveals it with a marked edge. That switch and the bar's
duration threshold (a slider over detents from 0 to 1000 ms, applied to
the bar only) are viewer preferences in localStorage, where an earlier
build's switches are migrated.
- The details pane: tabs per block with arrows when they overflow,
on-demand paged reads, an 8 MiB cache budget and a page window that lets
its stalest page go but never the one just asked for, recovery through
revision and epoch changes with bounded retries, and a place in the
page's Escape order. Relations resolve to links by (trace, span); the
usage block is six labelled rows with unrecorded cache counters said so;
an unreadable artifact reads as missing data in red. The messages tab
lists the outline on its own, folds the messages the model had already
seen and opens the new ones, draws only the rows near the viewport and
reads bodies for the open rows in that window; a row whose page the
cache let go folds again until asked for, a row a page was cut short of
to fit one response is read by its own cursor, a page that failed to
read is said in place with a retry, and an outline walk cut short is
taken up again on return. A JSON node draws its children two hundred at
a time. Under the raw record every file the span names is headed by its
attribute and shown as a JSON tree or text; contents are read by their
own cursor only for open files near the viewport, placed by where they
stand on the page, two at a time, kept apart from block pages, and let
go stalest first when the budget is short, to be read again only on
request.
- The duration bar on a canvas above the list: a fit that goes dense
when the rows outnumber the minimum slots, a spread with a frozen unit,
wheel zoom about the pointer, drag to pan with the anchor carried along,
a delayed click that checks the world it was made in, a pick list for
dense blocks placed within the window, a hidden turn end's time as the
turn's band under its blocks, a green dot above the selected entry's
block and a red one below a failed one, the summary in the band's hover
and the tools' title, a sideways wheel or swipe (or Shift with the
wheel) panning a zoomed bar, and a zoomed view that keeps its place on
the next entry when the threshold takes its anchor away.
- 223 `gui.trajectory.*` catalogue keys in both languages, the kind
vocabulary covering every kind the index emits with zh labels of at most
four characters; the TUI catalogue regenerated.

Docs: `commands.md`, `webui.md` and `webui.zh.md` describe the developer
launch; `CONTEXT.md` and `ui-web/CONTEXT.md` define Trajectory Entry,
Timing Owner, Message Delta, LLM Purpose and Trajectory View.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Python, compared with the main commit this branch is based on (61269654)
by test nodeid, same command and environment in both trees (`uv sync
--frozen --python 3.12 --extra dev --dev`, then `uv run --frozen
--python 3.12 --extra dev --dev pytest tests/ -q -p no:cacheprovider
--junitxml=...`):
- main: 52 failed, 27541 passed, 143 skipped (all environment-dependent:
subagent probes, a2a interop, whatsapp, everos, ...)
- branch: 52 failed, 27786 passed, 143 skipped, the same 52 as main, so
no test that passes on main fails here; tests collected only on the
branch: 245 (all added by this PR)
- `uv run ruff check .`: All checks passed; `uv run ruff format --check
.`: already formatted; `make lint-types`: All checks passed

Web UI (`ui-web/`):
- `npm run gen:check`: generated.ts matches the contract (215 methods)
- `npm run type-check`: 0 errors; `npm run lint`: 0 errors, 4
pre-existing warnings outside this change
- `npx vitest run --maxWorkers=24`: 231 files, 3493 tests passed (the
`scripts/gates/*.test.mjs` gates included)
- `npm run build && python3 build.py`: the build matches the boot
goldens, which this PR changes by one node each, `div#trajHost`
(`boot-stub.txt` 344, `boot-live-noserver.txt` 343,
`src/test/__golden__/region-app.txt` 189 lines)

TUI (`ui-tui/`): `npm run lint:rpc` in sync, `npm run lint:i18n` up to
date, `npm test`: 144 files, 2098 passed, 13 skipped.

Repository gates: `make check-source-language`, `make check-large-files`
and `git diff --check` exit 0; `uv run --frozen --group docs mkdocs
build -f docs-site/mkdocs.yml --strict`: built with no warnings.

End to end, on a real service with headless Chromium (scripts, logs and
screenshots kept outside the repository; run on this branch at its head,
every review change in): `raven web --foreground --dev` on a freshly
built fixture home with thirteen sessions (slow, failed, parallel,
nested sub-agent trace with injected skills, rotated archive, broken
artifacts, a 45-message prompt continued twice and a side question, a
240-message prompt, a turn whose end recorded nothing, a tool round
trip, a span naming thirteen small and another naming twenty large
files, empty, 10002 entries). The expected entries, with their hidden
marks, deltas and failure rules, are derived from the fixture definition
by the recording rules; the indexer and the paged `trajectory.list` walk
are checked against that list first, and the page is checked against it
after. 185 checks passed: the toggle and the parked scroller, the
visible rows' ids and order with hidden replies left out and listed
again under the switch, failure marks including a result that carried an
error key, integrity wording per broken artifact, the virtual window
bound and both ends of the long list, the details tabs, relation links
across traces, the usage rows, the messages tab folding the old rows and
opening the new ones as a window over the pane's viewport, a 240-message
list reading only the pages in view, letting the stalest pages go on the
way down and reading a folded row back on request, a feedback step's
skills under their own turn's names, a turn whose end recorded nothing
hidden while a failed one stays, the call after a tool round trip
previewing the first result with its predecessor's echo folded, the
duration threshold at 200 ms and 1 s, sideways panning, every file of a
span headed and read as it comes into view, large files each read once
with the earliest released and read back on request, a file's tree
folding to bring the next file in, the bar above the list, the turn's
band and its hover, the green and red dots read from the canvas, the
short filter, a slot doubling under a x2 wheel, drag by the pointer's
travel with the anchor kept across live appends, dense slots in
proportion at the scale ceiling, the summary hover in the gap between
blocks, pick list placement in tall and short windows, measured column
widths per language, tab arrows under overflow, theme and language
switches, viewports of 1200, 800 and 320 px; then a normal `raven web`
with no toggle, and `raven web --dev` against it exiting 1 with the
stop-first message while the service kept running.

## Risk

- [x] Security impact considered: the view is served only under `--dev`;
the RPC methods answer -32020 otherwise, artifact reads stay inside the
trace store with the 512 KiB cap, and the page reads through the
existing authenticated channel.
- [x] Backward compatibility considered: no existing RPC method, session
file or span format changes; the one new span attribute (`llm.purpose`)
is additive; a normal launch behaves as before. Known recorder gap, left
for a follow-up: `semconv.turn` reads the turn's reply from a tuple and
records `{"content": null}`; this PR only hides such empty turn ends.
- [x] Rollback path is clear for risky changes: revert this PR; it adds
a feature behind a flag and no migration.

## Related Issues

N/A

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-12 11:33:11 +08:00
knqiufan
27636d1337
fix: keep approval Escape from stopping the running turn (#868)
## Summary

Pressing Esc on an approval card previously denied the request and then
reached the page's stop handler, cancelling the running turn. Clicking
the Deny button only denied the request, even though that button
displays Esc as its shortcut.

- Consume Escape in both permission and confirmation cards after
answering deny. The turn can continue to respond to the refusal; a later
Escape follows the existing page close order and stops only when no
higher layer is open.
- Cover all five permission layouts, confirmation cards, editable
fields, the running-turn handler, layer priority, and approvals parked
in another conversation.
- Update the keyboard contract, stale comments, and both permission
guides. Use the existing portable path helper in the CSS gate so the
complete WebUI suite can run on Windows.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Commands below ran in ui-web unless prefixed with uv or git.

- `npm test -- src/features/composer/approve.test.ts
src/state/escapeOrder.test.ts src/features/composer/credential.test.ts
scripts/gates/settings-drawer-css.test.mjs --maxWorkers=4`: 160 passed.
- `npm test -- --maxWorkers=4`: all 3194 tests in 214 files passed. Nine
denial/bubbling cases failed against the original handlers before the
fix.
- `npm run lint`: passed with four existing React hook warnings in
unchanged files.
- `npm run type-check` and `npm run gen:check`: passed.
- `npm run build`: passed.
- `uv run python ui-web/build.py`: passed; both stub and
live-without-server boot snapshots matched.
- `node scripts/check-page.mjs`: passed; the assembled page and its two
scripts validated.
- `uv run python scripts/check_large_files.py origin/main`, `uv run
python scripts/check_source_language.py origin/main`, and `git diff
--cached --check`: passed.
- `uv run pre-commit run --files docs-site/docs/permissions.md
docs-site/docs/permissions.zh.md ui-web/CONTEXT.md
ui-web/scripts/gates/settings-drawer-css.test.mjs
ui-web/src/features/composer/approve.test.ts
ui-web/src/features/composer/approve.ts
ui-web/src/features/composer/credential.ts
ui-web/src/state/escapeOrder.test.ts ui-web/src/state/escapeOrder.ts`:
all applicable hooks passed.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Esc now has the same deny-only behavior as the card's button. Users can
press Esc again to close the next layer or stop the turn. Denial still
refuses the action; the permission decision and allow shortcuts retain
their existing behavior.

Reverting the squash commit restores the previous combined deny-and-stop
shortcut.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #852

Co-authored-by: Codex <noreply@openai.com>
2026-10-10 17:40:18 +08:00
Dizhan Xue
9c893bb65e
fix(*): trust the system certificate store and stop unverified Codex retries (#878)
## Summary

Behind a TLS-inspecting corporate proxy, or against a company-hosted
gateway signed by a private CA, `raven onboard` failed with
`CERTIFICATE_VERIFY_FAILED` even with the corporate root installed in
the operating system (#637). httpx, LiteLLM and the OpenAI SDK verify
against certifi's bundled roots unless told otherwise, so the
operating-system store was never consulted on the paths that reach a
model. aiohttp, urllib and websockets read OpenSSL's defaults instead,
so one process trusted two different sets of roots depending on which
client made the call.

What changes:

- `raven/security/tls.py` (new): `use_system_ca()` injects truststore,
so every TLS context built afterwards is verified by the operating
system (CryptoAPI on Windows, the Security framework on macOS, OpenSSL's
default paths on Linux). A bundle a caller loads into a context is still
honoured: on Linux it joins the store, on macOS its roots become extra
anchors, on Windows it is tried when the system roots fail. So every
root trusted before stays trusted. `RAVEN_NO_SYSTEM_CA=1` opts out,
parsed like `RAVEN_NO_UPDATE_CHECK`.
- `raven/cli/entry.py` (new) is now the entry of both the console script
and `python -m raven`, the form the web supervisor starts its gateway
with. It switches trust before importing the CLI, because the CLI's
imports reach aiohttp (through the sub-agent backends), and aiohttp
builds its default context when it is imported. `raven/cli/commands.py`
no longer starts the CLI when run as a script, which skipped the switch;
pointing it at the entry would not have helped, since by then its own
imports have built aiohttp's context.
- The provider probe names a certificate that fails verification
`certificate_untrusted` instead of `network_error`. It checks for that
ahead of the proxy branch: through a proxy that answered the tunnel, a
refused certificate used to be reported as `proxy_unreachable`. The
Codex check (its backend's list of the account's models), MiniMax's
token renewal (which reported `oauth_token_missing`) and the sub-agent
key prompt's probe name it too. LiteLLM's Copilot token exchange and its
ChatGPT renewal drop the handshake error before Raven sees it, so those
still read as the credential.
- The three surfaces that read the probe say so. `raven onboard` and
`raven provider test` no longer send the reader to check the network,
proxy or VPN; their hints name the three causes: a CA to add to the
system certificate store; macOS and Windows refusing a certificate by
their own rules even when `SSL_CERT_FILE` trusts its CA, which
`RAVEN_NO_SYSTEM_CA=1` turns off; and an expired certificate, or one for
another host, which is the server's to fix. `raven onboard` offers the
menu a network error gets (re-entering the key cannot fix a
certificate); the web Settings page has its own line, in English and
Chinese, in `i18n/messages.json` and the TUI's catalogue generated from
it.
- The RPC contract's description of the model catalogue's `status`
(`rpc-schema/openrpc.json` and both generated clients) now lists the
probe's real words. It named `unauthorized`, which the probe never
returns (a 401 or 403 is `invalid_key`), and missed `proxy_unreachable`
as well as the new status.
- The Codex provider no longer retries a request whose certificate
failed verification with verification turned off (`verify=False`). The
second attempt carried the ChatGPT OAuth bearer token and the whole
conversation to whoever presented the certificate (a captive portal, a
hostile network, a proxy that re-signs TLS), and on a network like the
one in #637 it is what would keep Codex working unverified while every
other provider fails. The retry dates from the initial import;
`_request_codex` loses the `verify` parameter that existed only to carry
it.
- `raven upgrade` and the page's Update verify against the same store as
the release lookup. The upgrade helper runs on the base interpreter,
outside the tool environment that holds truststore, and fetched the
release lists with urllib's default context; uv fetched the rest with
its bundled roots. When the spawning process itself verifies against the
system store, it now copies truststore into a fresh temporary directory
and hands the helper its path, as it hands over the marker path; one
that opted out, or never went through `cli.entry`, hands over nothing
and needs no truststore to say so. The helper loads the copy before its
first download and deletes it, and gives `UV_SYSTEM_CERTS` (and
`UV_NATIVE_TLS`, which older uv reads) to uv's own calls only, so the
Raven it relaunches inherits neither. A copy, because
`inject_into_ssl()` imports requests and urllib3 wherever it finds them,
and the tool environment's belong to what uv is about to rewrite.
- Quick Start, in both languages, gains a section on corporate proxies
and private CAs: WSL2 does not see the Windows store, and the EverOS
memory server and Node-based agents run as separate programs that need
`SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` in the shell profile.
`SSL_CERT_FILE` must name a complete bundle that exists, since httpx
refuses to start with a missing one:
`/etc/ssl/certs/ca-certificates.crt` on Debian, Ubuntu and WSL, and one
built from `/etc/ssl/cert.pem` on macOS. A note says macOS and Windows
apply their own certificate policy and names `RAVEN_NO_SYSTEM_CA=1` as
the way back.

The LiteLLM warning in the issue ("Failed to fetch remote model cost map
... CERTIFICATE_VERIFY_FAILED", with `SSL_CERT_FILE` exported) is not
LiteLLM building its own store. That fetch is a plain `httpx.get`, which
reads `SSL_CERT_FILE`, and the variable replaces the whole trusted set:
with a corporate-only PEM the gateway verifies while
raw.githubusercontent.com, signed by a public root, no longer does.
Measured with a private CA: corporate-only PEM, warning; certifi plus
the corporate root, remote fetch succeeds.

Not covered, by construction: the EverOS server is a separate `everos`
executable and Node-based agents are not Python, so an injection in this
process cannot reach them. The new docs section gives them their
environment variables.

`uv.lock`: adding truststore re-resolved the lock (uv 0.11.25, revision
3 kept). Besides the truststore entries, five packages' dependency edges
lose a marker: matrix-nio, h2, aiohttp-socks, aiologic and culsans. Each
dropped marker is implied by the only path that reaches its package
(matrix-nio, and through it h2 and aiohttp-socks, only where
`sys_platform != 'win32'`; culsans and aiologic only below Python 3.13),
so the resolved set is unchanged on every platform. uv 0.12.23 produces
the same edges and also rewrites the lock to revision 5, so it was not
used; both versions accept the committed lock under `uv lock --locked`.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Red first, on the base: the end-to-end `raven provider test custom`
against a gateway whose root only the stand-in system store holds exits
1 with `network_error ... unable to get local issuer certificate` under
both launch forms, and its opted-out twin prints "Check network /
firewall / VPN settings"; the probe test returns `network_error`, and
through a CONNECT proxy `proxy_unreachable`; the onboarding menu offers
`rekey` for a certificate failure; the web note falls to
`probe_unknown`; and the Codex test's endpoint receives `Bearer
tok-secret`, with the captured log line `SSL certificate verification
failed for Codex API; retrying with verify=False`.

Mutations, each applied and then restored byte for byte:

- the entry no longer switching trust: both end-to-end trust cases fail;
- the switch moved after the CLI import: only
`test_trust_is_settled_before_the_cli_builds_a_context` fails;
- the classifier accepting any `ssl.SSLError`: the plain-HTTP case
fails;
- the certificate check placed after the proxy branch: the proxy case
fails.

Review round two, red first at the round-one head: against an endpoint
signed by a root no store holds, the Codex check returned
`network_error`, MiniMax's token renewal `oauth_token_missing` and the
sub-agent key probe `network_error`; both hint tests failed on the
missing opt-out; the guard scan found `raven/cli/commands.py`; and the
upgrade tests found nothing handed over. Round-two mutations, each
applied and then restored byte for byte:

- the classifier matching `CERTIFICATE_VERIFY_FAILED` in the message:
the new macOS and Windows cases fail (282 of 282 stayed green before
them);
- 17 single-construct mutants of the upgrade hand-over (not switching,
not putting the copy on the path, keeping it, hiding a failed load,
leaving the path in the environment, not telling uv, overwriting a uv
setting the user made, dropping the older uv name, telling uv without a
copy, uv not given its environment, passing on an inherited path,
importing truststore instead of reading whether this process switched,
handing over without that check, handing over always, handing over no
copy, and keeping a copy no helper started, at each spawn site): each
fails at least one test;
- the copy failure's warning and its cleanup: each fails the
copy-failure test.

Commands and results, at the head after rebasing onto `main`. The
commits `main` gained since round one touch none of this branch's lines:
`git range-diff` shows every commit unchanged, and the aggregate
patch-id is the same before and after.

- The touched Python test files (`tests/test_security_tls.py`,
`tests/test_cli_system_ca.py`, `tests/test_config_update_providers.py`,
`tests/test_cli_onboard_commands.py`,
`tests/test_cli_provider_commands.py`,
`tests/test_cli_subagent_setup.py`, `tests/test_updates_upgrade.py`,
`tests/test_openai_codex_provider.py`), the Codex provider's neighbours
(`tests/test_providers_first_byte_budget.py`,
`tests/test_provider_protocol.py`) and the i18n tests, in full: 1010
passed.
- Full suite with branch coverage, `pytest -q -p no:randomly --cov=raven
--cov=raven_everos --cov-branch` on the all-extras environment with the
box proxy variables unset: 2 failed, 27750 passed, 119 skipped. Both
failures fail the same way on a clean tree of the new merge base under
the same interpreter, so neither is introduced; they are local to this
box (an `npm` on its PATH, and running as root):
-
`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`
-
`tests/test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing`
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over that run: 100.00% (57/57 executable changed lines).
- `tests/integration/test_cli_upgrade_real_uv.py`, which CI runs only in
the Windows self-upgrade job (`--noconftest`): 3 passed locally. Its
fixture calls the handoff without `cli.entry`; the `system store`
variant switches first, as `cli.entry` does, so that job runs the whole
hand-over on Windows, and the fixture prints what it handed over so
neither variant passes without its branch running.
- `ruff check`, `ruff format --check`, `ty check` over the changed
modules, `lint-imports` (10 contracts kept), `deptry raven`: clean.
- Repository gates over `origin/main...HEAD`:
`check_source_language.py`, `check_large_files.py` and
`check_commit_messages.py` exit 0.
- Generated files on the rebased tree, where `main` had also changed the
RPC schema, the message catalogue and their generated copies:
`gen-i18n.mjs --check` and `gen-rpc-types.mjs --check` (ui-tui) and
`gen-rpc-client.mjs --check` (ui-web) exit 0.
- Round two changes no file under `ui-web`, `ui-tui` or `rpc-schema`;
these were measured at round one:
- ui-web, `node node_modules/vitest/vitest.mjs run
src/features/settings/SetupBodies.test.tsx
scripts/gates/i18n-keys.test.mjs`: 33 passed; `scripts/gates`: 46 files,
205 passed; `tsc --noEmit` clean, its `--listFiles` resolving the edited
files inside this tree; ESLint clean on the three files.
- ui-tui, the steps of TUI checks: ESLint 0 errors; `lint:rpc` and
`lint:i18n` pass; `tsc --noEmit` clean; the build passes; `vitest run`
has 9 failures in 4 files (renderer and typing-burst tests), the same 9
that fail on a clean worktree of the merge base on this box, so none is
introduced.
- RPC contract: `node scripts/gen-rpc-client.mjs --check` (ui-web) and
`node scripts/gen-rpc-types.mjs --check` (ui-tui) exit 0, each generated
file differing from the base by the one description line; the 9 Python
test files that read the schema, 798 passed.
- Docs: `mkdocs build --strict` clean, with the new paragraphs on the
English and Chinese pages; `tests/test_docs_site.py` and
`tests/test_living_docs.py` pass in the full suite.

The end-to-end cases run on Linux only. They stand the system store in
with OpenSSL's directory store through `SSL_CERT_DIR`, and name a bundle
of public roots in `SSL_CERT_FILE` so that httpx cannot pass by reading
`SSL_CERT_DIR` itself. Verification on macOS and Windows was read from
truststore 0.10.4's source, not run. The upgrade helper's tests share
the limit: on Linux truststore and the standard library read the same
store, so they show the hand-over (the copy loads on the base
interpreter under `-I` before the first download, and uv alone is told),
not the Keychain difference it exists for.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Every Raven process started as `raven` or `python -m raven` now trusts
the operating system's roots in addition to what each library bundles.
For those processes `SSL_CERT_FILE` adds to the trusted set instead of
replacing it; someone using it to narrow trust to one private root keeps
that root but also trusts the system store.
- On macOS and Windows the system verifier applies its own certificate
policy as well. A private certificate that OpenSSL accepted from
`SSL_CERT_FILE` can be refused there if it breaks that policy;
`RAVEN_NO_SYSTEM_CA=1` restores OpenSSL's verification.
- On macOS and Windows truststore also verifies the client's chain on
server-side contexts, refusing a client that sends none. Raven builds no
server-side TLS context: it serves plain HTTP on localhost.
- Upgrades verify their downloads the way the release lookup does: on
macOS that includes Apple's certificate policy, which the helper's
OpenSSL did not apply. uv is told to use the system store only for the
upgrade's own calls, and a `UV_SYSTEM_CERTS` or `UV_NATIVE_TLS` the user
set is kept.
- Behind a proxy that re-signs TLS, Codex requests now fail with the
certificate error instead of continuing unverified. Trusting the proxy's
root, in the system store or in a complete bundle named by
`SSL_CERT_FILE`, is the remedy.
- Startup: about 100 ms warm and 270 ms cold, measured on Linux, mostly
truststore importing urllib3 and requests to patch their references.
- Rollback: `RAVEN_NO_SYSTEM_CA=1` restores the previous trust per
process, and an upgrade started under it hands its helper and uv
nothing; the Codex change has no switch, by design. Reverting the squash
commit removes both.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #637

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-10 10:40:12 +08:00
Dizhan Xue
3c88206b56
fix(*): resolve the latest release under windows powershell 5.1 (#863)
## Summary

Native Windows installs fail from the shell Windows ships with. Both
commands the README gives for Windows break under Windows PowerShell
5.1:

- `irm https://raven.evermind.ai/install.ps1 | iex` stops with `(308)
Permanent Redirect`. PowerShell 5.1 follows 301, 302 and 307 redirects
but not 308, which is what that URL answers with. This is the error #148
reported; the README already offered the direct URL for it.
- The direct URL downloads the script, which then stops at "Could not
resolve the latest Raven release wheel from GitHub".
`Resolve-RavenLatestVersion` requests the release page without following
the redirect and reads the Location header off the exception that
raises. Only PowerShell 7 attaches the response to that exception.
PowerShell 5.1 returns the unfollowed redirect as the response and
reports `MaximumRedirectExceeded` as an error with no response, so the
lookup always came back empty. This came in with #299 and has been on
main since.

Changes:

- `install.ps1`: the release-page request uses `-ErrorAction Ignore`
instead of `-ErrorAction Stop`, so PowerShell 5.1 reads Location from
the returned response. PowerShell 7 still raises regardless of
`-ErrorAction` and goes through the existing catch. The Location is
still matched against the same stable-tag pattern.
- CI: the installer job ran `install.ps1` under `pwsh` only, which is
how this shipped. It now also runs the piped install under Windows
PowerShell 5.1 (`shell: powershell`), into its own tool, bin and home
directories, sharing the uv cache.
- Docs: README, README.zh-CN, the docs-site quick start (en, zh) and the
release notes template now say that Windows PowerShell 5.1 stops on the
short URL with `(308) Permanent Redirect` and to switch to the direct
URL when that appears. The short URL's redirect itself is unchanged.
- Tests: tripwires for the lookup (`-ErrorAction Ignore`, catch kept)
and for the CI gate (both shells, and separate tool, bin and home
directories, since the version check runs the `raven.exe` in the bin
directory).

Second topic, Windows test hygiene: three tests failed when the suite
ran on a Chinese-locale Windows checkout. CI runs the unit tests on
Linux only, so it never saw them.

- `test_no_workflow_step_enters_the_removed_page_directory` read
workflows in the locale encoding (GBK there) and stopped on the UTF-8 in
`release.yml`. It now reads UTF-8.
- The font and LibreOffice harness tests run `install.sh` code under
`sh`, which on Windows is Git Bash: its curl sits outside `/usr/bin` and
paths mix separators, so two failed and others passed for the wrong
reason (the digest-mismatch test passed because curl was missing). The
ten harness tests now carry the POSIX-only skip the other three
sh-driven tests already had, shared as one `POSIX_SH_ONLY` marker. Linux
runs are unchanged.

#861 merged first, and this branch is rebased onto it. The one conflict
was in `tests/test_install_script.py`. The sh-driven tests #861 added,
its two Node harness tests and
`test_a_download_that_cannot_be_hashed_is_not_installed`, now carry
`@POSIX_SH_ONLY` like their neighbours, so the marker stays the one
spelling of that skip.

Third topic, test infrastructure: Python's locale-dependent text I/O
uses the system code page when no `encoding=` is passed. On a GBK-locale
Windows checkout, any test that reads or writes a file containing
non-ASCII bytes will fail or silently corrupt data. The class of bug was
found above; there are ~790 `read_text()` call sites without `encoding`
in `tests/`. Measured on Linux under a zh_CN.GBK locale, main without
UTF-8 mode is not one test short: 87 tests fail only there, across 25
files, and `tests/test_rpc_schema_match.py` (418 tests) does not
collect, mostly on `UnicodeDecodeError` from reading UTF-8 files.

- Makefile: `export PYTHONUTF8 := 1`, so every recipe runs in UTF-8
mode, `check-core-wheel` (part of `make ci`) included.
- `tests/conftest.py` raises at import time on any interpreter whose
default text encoding is not the UTF-8 codec, telling the caller to set
`PYTHONUTF8=1` or use `-X utf8`. It compares codecs rather than names,
so Windows with the system-wide UTF-8 option, which reports `cp65001`,
passes. `uv run pytest`, which AGENTS.md documents and which does not
read the Makefile, now stops here on a GBK checkout, as does an IDE
runner that does not set the variable. CI's `--noconftest` self-upgrade
job (its tests all name `encoding` explicitly) is unaffected.
`tests/test_conftest_utf8_guard.py` pins the codec comparison and drives
the refusal in an ASCII-locale child.
- CI sets `PYTHONUTF8` in no job. The jobs that load `tests/conftest.py`
run on ubuntu-latest, where the locale is UTF-8 already, and the `unit`
job gets the variable from the Makefile through `make coverage-shard`.
The Windows self-upgrade job runs its one file with `--noconftest`, and
that file names its encodings. The installer job runs no tests at all:
its steps are raven installing itself and answering `--version`, so the
variable there would only change how raven runs, and that gate exists to
run it the way users do.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on Windows 11 (zh-CN) with Windows PowerShell 5.1.26100 and
PowerShell 7.6.6, before the rebase onto #861 and the second review
round. Every install below was isolated: `UV_TOOL_DIR`,
`UV_TOOL_BIN_DIR` and `RAVEN_HOME` in a temp directory,
`RAVEN_MINIMAL=1`, `RAVEN_NO_LAUNCH=1`.

- Before the fix, under 5.1: `irm https://raven.evermind.ai/install.ps1`
fails with `(308) Permanent Redirect`; `irm
https://raw.githubusercontent.com/EverMind-AI/Raven/refs/heads/main/install.ps1
| iex` stops at `Could not resolve the latest Raven release wheel from
GitHub`.
- `Invoke-WebRequest
https://github.com/EverMind-AI/Raven/releases/latest -MaximumRedirection
0 -UseBasicParsing -ErrorAction Stop`: 5.1 throws
`InvalidOperationException` (`MaximumRedirectExceeded`) with no
`Response`; 7.6.6 throws `HttpResponseException` whose
`Response.Headers.Location` is the tag URL. With `-ErrorAction Ignore`,
5.1 returns the 302 with Location set, and 7.6.6 still throws into the
catch.
- Redirects under 5.1, via httpbin `redirect-to`: 301, 302 and 307 are
followed; 308 is not.
- After the fix, `Get-Content install.ps1 -Raw | Invoke-Expression`
under 5.1 and under 7.6.6, with `uv tool update-shell` disabled in a
temp copy of the script to leave the user PATH alone: both resolved
v0.2.4, installed raven with everos-memory, design-engine and
ppt-engine, and `raven.exe --version` printed `Raven v0.2.4`.
- `uv run pytest tests/test_install_script.py
tests/test_release_plugin_list.py
tests/test_constraints_export_contract.py` plus the six installer tests
in `tests/test_cli_onboard_commands.py`, on Windows: 48 passed, 13
skipped (before: 55 passed, 3 skipped, 3 failed). The two new tripwires
fail against the previous `install.ps1` and `ci.yml`. With the UTF-8
gate added, a bare `uv run pytest --co -q tests/test_install_script.py`
on the GBK checkout raises RuntimeError naming `PYTHONUTF8=1`, while the
same invocations with the variable set pass (45 passed, 13 skipped).
- `uv run ruff check` and `uv run ruff format --check` on both test
files pass; `ty` does not cover `tests/`, the only Python this changes.
`PYTHONPATH=. uv run python scripts/check_source_language.py
origin/main` exits 0.

Run on Linux (Ubuntu 22.04, Python 3.12.13) at this head. A zh_CN.GBK
locale compiled into a private `LOCPATH` gives a non-UTF-8 interpreter
(`locale.getpreferredencoding(False)` is `GBK`, UTF-8 mode off).

- Full suite, `uv run --frozen --python 3.12 --all-extras pytest -q`: 7
failed, 27567 passed, 119 skipped. On main the same command gives 7
failed, 27556 passed, 119 skipped with the identical 7 failing IDs, this
machine's local baseline (five proxy tests, a root-only permission test
and an npm found on PATH), so 0 introduced; the 11 extra passes are this
PR's new tests.
- Main under GBK without UTF-8 mode: 86 failed, 27059 passed, 119
skipped, 11 errors. 87 IDs fail only there, and
`tests/test_rpc_schema_match.py` does not collect. One of the 87, a
non-ASCII file name, is specific to Linux, where GBK also becomes the
filesystem encoding.
- This head under GBK: a bare `uv run pytest --co -q
tests/test_install_script.py` stops at the conftest RuntimeError naming
`PYTHONUTF8=1`. `make test-python`, which exports it: 7 failed, 27567
passed, 119 skipped, the same 7 IDs as under UTF-8. `make
check-core-wheel`: 1 passed, where with the per-target exports it exited
2 at the guard.
- The harness tests marked `@POSIX_SH_ONLY` run on Linux in the full
suite above.
- Mutations: with the old name tuple back in the guard's predicate, only
the `cp65001` case of `tests/test_conftest_utf8_guard.py` fails; with
the guard deleted, its end-to-end refusal test fails. Giving the 5.1
step pwsh's `UV_TOOL_BIN_DIR`, or its `RAVEN_HOME`, now fails the CI
gate test naming the shared variable, where before it passed.
- `make lint-python` passes, and the commit-message, source-language and
large-file gates exit 0 over `origin/main..HEAD`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible: the one-line Windows install works again from Windows
PowerShell 5.1 through the direct URL. The short URL still answers with
a 308 there, and the docs now name that error. `install.ps1` is served
from main, so the fix reaches users on merge, and reverting the squash
commit rolls it back the same way. The resolved Location is still
matched against the same stable-tag pattern before anything is
downloaded. The installer CI job gains one Windows install, which reuses
the uv cache.

Developer-visible: on a Windows checkout whose code page is not UTF-8, a
pytest run that loads `tests/conftest.py` without UTF-8 mode stops at
import with a message naming `PYTHONUTF8=1`; the `make` targets set it,
and Linux and macOS runs are unaffected. On Windows the sh-driven
harness tests report as skipped: the ten this PR marks and
`test_a_download_that_cannot_be_hashed_is_not_installed`, beside the
five that already skipped there.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#148

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-08 17:30:59 +08:00
knqiufan
3ca6d283fa
fix(core): prevent heartbeat from repeating cron replies (#843)
## Summary

A Cron reminder could reply through Spine delivery and then enqueue a
completion event that made Heartbeat send the reminder again. Use
`TurnOutcome.explicit_reply` to skip that redundant event and wake,
including replies without captured readback text.

Successful runs clear their pending failure event; explicit replies also
clear earlier silent completion events for the same job. Silent
completions, failures, missed reminders, and automatic disabling retain
their notification paths. Add a regression using the real Scheduler,
DeliveryHub, event queue, and HeartbeatService, plus event cleanup tests
and updated English/Chinese documentation.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The delivery regression failed for both readback variants before the
fix. After rebasing onto latest official main, 167 focused tests passed
with 94.90% combined line/branch coverage for `cron_stack.py`; changed
executable lines have 100% coverage.

```powershell
$env:PYTHONUTF8 = '1'
$env:COVERAGE_FILE = Join-Path $env:TEMP 'raven_issue164_coverage'
uv run --frozen --no-sync pytest tests/test_core_cron_stack.py tests/test_core_cron_stack_ledger.py tests/test_event_wake.py tests/test_gateway_spine.py tests/test_spine_delivery.py tests/test_living_docs.py tests/test_docs_site.py -q --cov=raven.core.cron_stack --cov-branch --cov-report=term-missing
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json $ravenCoveragePosixJson diff --base-ref FETCH_HEAD --threshold 90
```

`$ravenCoveragePosixJson` is the generated coverage JSON with Windows
path separators normalized for the diff checker.

Broader Cron service/tool/CLI/TUI coverage passed 336 tests after
excluding two failures reproduced on clean official main: Windows
timestamp resolution in
`test_reload_detects_rewrite_within_float_mtime_collision` and terminal
wrapping in `test_run_test_fire_on_recurring_job`.

```powershell
$ravenCronTestFiles = @((rg --files tests) | Where-Object { $_ -match '^tests[\\/]test_cron_service_.*[.]py$' })
uv run --frozen --no-sync pytest tests/test_core_cron_stack.py tests/test_core_cron_stack_ledger.py tests/test_event_wake.py tests/test_gateway_spine.py tests/test_spine_delivery.py @ravenCronTestFiles tests/test_cron_tool.py tests/test_cron_wake_grant.py tests/test_cli_cron_commands.py tests/test_tui_cron_tool_wired.py tests/test_tui_cron_delivered_event.py tests/test_living_docs.py tests/test_docs_site.py -q -k 'not test_reload_detects_rewrite_within_float_mtime_collision and not test_run_test_fire_on_recurring_job'
```

The following passed: Ruff over the full Python tree, all 10 import
contracts, type checking of the changed module, diff whitespace, asset
limits, and source language checks.

```powershell
uv run --frozen --no-sync ruff check raven evolver agents plugins-dist tests scripts docs-site
uv run --frozen --no-sync ruff format --check raven evolver agents plugins-dist tests scripts docs-site
uv run --frozen --no-sync lint-imports
uv run --frozen --no-sync ty check raven/core/cron_stack.py
git diff --check FETCH_HEAD...HEAD
uv run --frozen --no-sync python scripts/check_large_files.py FETCH_HEAD..HEAD
uv run --frozen --no-sync python scripts/check_source_language.py FETCH_HEAD..HEAD
```

`make ci` could not run because this Windows environment has no `make`.
Full-suite collection and wheel smoke testing also encounter existing
Unix-only assumptions (`os.geteuid` and `venv/bin/python`); full
dependency/type checks require platform-excluded `nio`/`boxlite`
packages. Full Linux CI remains to be checked on the PR.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Explicit Cron replies no longer trigger Heartbeat completion follow-ups.
Existing Spine/DeliveryHub delivery and retry handling remains
responsible for those replies; this change adds no transport
acknowledgement or exactly-once guarantee. No schema, permission, or
dependency changes. Revert this PR to restore the previous event
behavior.

## Related Issues

Fixes #164

Co-authored-by: Codex <noreply@openai.com>
2026-10-03 23:28:52 +08:00
Dizhan Xue
f7d9e69054
fix(*): key browser grants on the page a call lands on and prune owner state (#839)
## Summary

Fixes the three defects #600 reports in the browser tools, and the test
fixture that failed six real-browser tests on Linux.

**A permission key could name a site the call would not act on.** The
acting tools write the site into the call before the gate reads it, and
took it from `url_for(owner)`, which fell back to the front tab for an
owner with no live binding. When another owner held that tab, the call
opened a blank tab of its own instead: a person approved `bank.test` for
a key press that landed on `about:blank`, and banked a grant that any
later click, type or press on `bank.test` presents, since the three
verbs share one key per site.

`url_for` now reports the page `_page_for` will bind, through one rule
both use (`_landing`): the owner's live tab, else the front tab unless
another live owner holds it, else nothing. In the last case the call
opens a tab of its own, which no site describes yet, so no site is
written and the grant is keyed on the call itself.

The issue offered two directions, and neither is taken as written. An
empty answer for every unbound owner fixes the reported case but breaks
one that worked: with the front tab the reader's own, the call lands on
it while the prompt names no site, and its per-call grant can then be
presented by an identical call on whatever front tab comes next.
Resolving the site after binding would make `cast_params` asynchronous,
and that hook is part of the contract every tool shares.

**`_BrowserTool._acted` was never pruned.** Keyed by `run:<uid>`, it
kept one entry per delegated run for the life of the process: 50,000
runs left 50,000 entries and 6.6 MiB after `gc.collect()`. The driver
now ends every binding through one `_drop_owners` (a reap, a release,
the owner's tab closing, the browser closing) and tells a listener the
tools register on their first stamp. The map also keeps only the 512
owners that acted most recently, for owners whose binding is never
dropped. The same 50,000 runs now leave 512 entries.

**`browser_tabs` did not say that close refuses a tab nobody holds.**
Such a tab is the user's and may hold a login they were asked to
finish, and the listing marks it neither `yours` nor `held`. The
description now says so, as does the browser guide in both languages.

**`test_browser_real_web.py` aimed Playwright at the macOS cache on
every platform.** The suite redirects `HOME`, and Playwright resolves
its cache from the home directory, so the fixture points it back at the
login's cache, but only at `~/Library/Caches`. The look-up now lives in
`tests/_browser_cache.py`, shared by both real-browser suites, and
mirrors playwright-core's `registryDirectory`: XDG_CACHE_HOME on Linux,
LOCALAPPDATA on Windows, `PLAYWRIGHT_BROWSERS_PATH=0` for an install
inside the package, and a relative path. Both suites skipped only when
the package was missing, so a missing browser failed them; they now
check for the browser. The driver called that failure "not installed"
even when the browser sat where the process did not look; the message
now quotes the path it looked for and names PLAYWRIGHT_BROWSERS_PATH,
on one line.

Known gaps, left as they are:

- `url_for` is a prediction made before the prompt. Anything that moves
  while a person decides (the reader switches tabs, another owner takes
  the front tab, the binding idles past ten minutes) moves the call with
  it. Closing that needs the gate to see the binding the call will use;
  the issue thread sketches one shape.
- The skip checks for a Chromium directory, not the revision this
  Playwright needs. A cache holding only an older build still fails the
  launch, and the message then names the path it looked for.
- `browser_tabs` stamps an owner after closing that owner's own tab;
  that stamp leaves through the cap rather than through a release.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run with the project environment's interpreter (all extras installed):

- `python -m pytest` over `tests/test_browser_cache.py`,
  `tests/test_browser_driver.py`, `tests/test_browser_tools.py`,
  `tests/test_browser_policy.py`,
  `tests/integration/test_browser_real_web.py` and
  `tests/integration/test_browser_tools_real_web.py`: 172 passed. CI
  does not collect `tests/integration/`, so those two files ran here
  only.
  With PLAYWRIGHT_BROWSERS_PATH pointed at a directory that does not
  exist, all 12 skip with the new reason instead of failing.
- The `url_for` table, one case per branch of the landing rule, fails
  against main in the two cases where another owner holds the front
  tab, and against the empty-answer direction in the three where the
  front tab is free.
- Deleting each of the 22 constructs that carry this change, one at a
  time, turns at least one test red.
- `python -m pytest -q -p no:randomly`: 27132 passed, 7 failed, 116
  skipped. The same 7 IDs fail on a complete tree of origin/main: five
  provider-proxy tests that read this machine's proxy settings, and two
  Node-runtime tests that depend on its node install and on root
  ignoring permission bits.
- `ruff check` and `ruff format --check` on the changed files,
  `lint-imports` (10 kept, 0 broken), and `ty check raven evolver agents
  plugins-dist scripts` (the `make lint-types` set): all pass.
- `scripts/check_source_language.py` and `scripts/check_large_files.py`
  over origin/main...HEAD, and `scripts/check_commit_messages.py` and
  commitlint over origin/main..HEAD: all pass.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

An acting call that would open a tab of its own is no longer keyed on
the front tab's site: its prompt names no site, and a session grant for
it covers only an identical call, so that state prompts more often. A
call landing on the user's front tab keeps the site key it had. The
browser-unavailable message the panel shows is reworded; nothing parses
it. No config, schema or stored state changes, and reverting the squash
commit restores the previous behaviour.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #600

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-03 12:18:35 +08:00
Zuyi Zhou
bcbc295c0d
fix: record unanswered questions an unattended turn could not ask (#807)
## Summary

An unattended turn could ask a question and leave no trace a person can
read.

`raven agent -m` already lists calls the permission gate refused (#708).
It said nothing about a question nobody could answer. On the web, a
sub-agent turn with no page subscribed to the session binds no asker
(`start_ask_turn(None)` in `raven/rpc/spine.py`). The host `ask_user`
tool still tells the model "user did not answer; proceed with best
judgment". That sentence is the tool result the model reads. Opening the
session later showed no event for the question.

Questions that were never put to anyone are now recorded on the turn: a
sub-agent `ask_user` or elicitation with no asker, a question that gives
up waiting for the conversation lock, a round trip that is structurally
unavailable, and a host `ask_user` call whose answer comes back empty. A
one-shot call has no broker at all, so it returns at that guard; the
questions `_prepare` accepts are recorded there too, and the same error
string is what the model still reads. A call `_prepare` rejects is not
recorded. The turn's save writes one assistant entry with a
`question_unanswered` notice, which is what a reopened session draws. A
one-shot prints the same questions after the reply. That list does not
change the exit status: the turn continued, which is what the model was
told to do. A refusal is still the run that exits 3.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

```
uv run --frozen --python 3.12 --extra dev ruff check raven/agent/loop/turn_path.py raven/agent/tools/ask_user.py tests/test_ask_user_tool.py tests/test_cli_one_shot_spine.py tests/test_agents_research_flow_report_shape.py
  All checks passed

uv run --frozen --python 3.12 --extra dev ruff format --check <same files>
  already formatted

uv run --frozen --python 3.12 --extra dev pytest -q \
  tests/test_ask_user_tool.py::test_a_call_with_no_broker_records_the_normalized_questions \
  tests/test_ask_user_tool.py::test_error_paths_return_plain_strings \
  tests/test_cli_one_shot_spine.py::test_a_one_shot_ask_user_call_with_no_broker_is_what_gets_reported \
  tests/test_agents_research_flow_report_shape.py::test_the_bounce_scaffolding_never_becomes_history \
  tests/test_agent_loop_session_stamps.py::test_an_unanswered_question_is_stored_as_a_notice
  5 passed

cd ui-tui && npm run lint:i18n
  i18n: generated catalogue is up to date

PYTHONPATH=. uv run --frozen --python 3.12 --extra dev python scripts/check_source_language.py origin/main...HEAD
  exit 0

make check-commits && PR_TITLE="fix: record unanswered questions an unattended turn could not ask" make check-pr-title
  exit 0

git diff --check origin/main...HEAD
  exit 0

git merge-tree --write-tree HEAD origin/main
  clean
```

Reverting `raven/agent/tools/ask_user.py` to its parent makes both new
no-broker tests fail with an empty record. Reverting
`raven/agent/loop/turn_path.py` to the parent, where filing was a
method, makes `test_the_bounce_scaffolding_never_becomes_history` raise
`AttributeError: '_Loop' object has no attribute '_file_unanswered'`.
Both files were restored afterwards. The earlier reverts (notice not
stored, ACP ask and elicitation not recorded, one-shot report not
printed) still stand for those paths.

The full suite was not run. A run of the touched CLI modules in this
terminal also failed existing assertions that search Rich output for a
plain substring (help flags, and `1 action(s) were refused`). Those
assertions fail the same way with this diff's refusal printer left
unchanged. The type checker was not rerun locally; CI's python lint job
runs it on the head.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

The model is still told to continue. Nothing new is approved. The
one-shot exit status is unchanged: unanswered questions are reported and
do not by themselves exit 3. The notice is one more assistant entry on
the session, with `notice.kind` `question_unanswered`, so a client that
already draws runtime notices draws this one. Rollback is reverting the
commit.

## Related Issues

#708
2026-09-27 10:30:48 +08:00
arelchan
b27126ef39
feat(*): upgrade from the page with progress, open it for bare raven (#806)
## Summary

The upgrade button on the page could never work for a normal install,
and when an upgrade did run, neither the page nor (on a slow link) the
terminal showed anything that moved. This PR makes the page upgrade
work, shows its progress, and fixes what an end-to-end run of it turned
up, plus two entry-point changes from the same pass.

**The page upgrade works on `raven web`.** `raven web` runs `raven
gateway`, so nearly every page is gateway-hosted, and `system.upgrade`
refused all of them (`gateway_hosted`): its relaunch brought back a bare
`raven serve`, which builds no ChannelManager, so the IM channels would
have vanished. Now:

- On every page mount the gateway hands the page its own graceful stop
(the path `gateway.shutdown` takes), its busy check, and its supervisor
pid. Every mount, because a config swap tears the page down and disarms
it.
- On upgrade the gateway exits 0. The `raven web` supervisor already
reads a zero exit as "stand down" (its docstring names
`system.upgrade`), so it exits too.
- The helper waits for the **supervisor**, not the gateway, then
installs and relaunches `raven web --supervise` on the same port with
the same token and cookie. Waiting on the supervisor matters: its
cleanup removes `web.json` after the gateway exits, and a supervisor
started before that would lose its state file, inviting the next `raven
web` to start a second one.
- Refused, each with its own reason, when nothing would bring the
gateway back (started by hand, or `raven web --foreground`), when work
the page cannot see is running (an IM turn, a sub-agent, a pending
question), or when there is no executable to relaunch from. The config
swap and the upgrade now share one busy check.

This takes effect for upgrades that start from a release containing it;
a gateway on an older release still refuses with the old message.

**The page shows the upgrade's progress.** The card said "Upgrading
Raven..." and nothing else until the new Raven answered, because the
process that served the page is gone for the whole install.

- The helper downloads the release itself, where the bytes can be
counted, and hands uv the local files so uv only resolves dependencies.
All or nothing: if any asset cannot be fetched, uv gets the original
URLs and downloads them as before. Files live under
`~/.raven/cache/upgrade/<version>`; earlier versions' are cleared.
- Once the old Raven has exited, the helper answers on the page's port
(recorded in the upgrade marker): `/upgrade/status` with phase, bytes
done and total, and rate; 503 to everything else, so the page's "is it
back?" probe keeps waiting and a reload gets a self-refreshing wait
page. It answers only the page's own Host, and lets the port go before
the relaunch binds it.
- A failure is held until the page has read it (at most five seconds).
Before, a failed install relaunched the old version and the page
reloaded onto it silently.
- The page polls the status, measures the bar while bytes are counted
and slides it otherwise, and says when the new Raven is starting. Its
twenty-minute ceiling now counts silence, not the whole upgrade, so a
slow download that is visibly moving is not given up on.

**Two bugs the end-to-end run exposed**, both fixed here:

- *A red "the service is not running, start it with raven web" bar
during the upgrade.* The transport's health probe counted any answer but
401/403 as "the gateway is back and refused this session", so the
helper's 503 made it give up. A 5xx now means "not back yet". Following
that bar's advice mid-upgrade is exactly what would start a second
supervisor.
- *The old gateway took 46 seconds to exit, blocked on the page's own
open WebSocket.* aiohttp's cleanup waits for live handlers instead of
closing websockets. The WS gateway now closes its sockets from
`on_shutdown`; the same run dropped to 4 ms. This also speeds up `raven
web --stop` with a page open.

**`raven gateway` refuses a half-written install.** `raven serve` has
guarded this since the upgrade marker existed, but the gateway never
did, and the gateway is what `raven web` supervises. A restart inside an
upgrade window came up on the half-written tree and, because the page
route is chosen once, served the placeholder page until restarted by
hand. The guard runs before the instance lock. It is also a precondition
for the restart above.

**Refusals are localized.** The page pasted the server's English
`detail` into an otherwise Chinese card. Refusals it can name come from
the catalogue; refusals whose detail carries facts from this run keep
them. A server-side `busy` gets the same wait notice as the page's own
busy turn, not the "run this in a terminal" card.

**The terminal upgrade names its download size.** uv draws one bar per
package and no total, so a large release reads like a stalled install (a
real report: a 72 MiB upgrade at about 50 KB/s looked frozen). The
helper prints each asset's size and the total before downloading, from
HEAD requests bounded at 5 seconds, headline first so a hanging probe
adds no silence; the terminal then shows the helper's own live progress
line. Output is flushed so it keeps its place in `web.log`.

**Bare `raven` opens the page.** The installer ends in `raven web`, but
the command typed the next day landed in the TUI. Bare `raven` now does
what `raven web` does; where no browser can be opened it runs the TUI
instead, decided before anything starts. `raven web` and `raven tui` are
unaffected.

**Composer placeholders.** Empty: "Describe your task, or have Raven
build an agent for it" becomes "Describe your task". While a turn runs:
"it is sent once this turn ends" was stale (mid-turn sends merge into
the running turn; the queue is only a fallback), so it now says Raven
receives the new instruction directly.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

End to end, three times, in a sandboxed uv tool install (own
`UV_TOOL_DIR` and `RAVEN_HOME`, no plugins, so nothing reached a real
memory server), upgrading 0.2.90 to 0.2.91 from a local stand-in for the
GitHub release API, clicking the real rail-foot upgrade row in a
browser:

- First run (fast local download): page back after 17 seconds, reloaded
by itself, still signed in; served by `python -P -m raven gateway` (not
`serve`) on the same port; `web.json` names the new supervisor, which is
the new gateway's parent; old supervisor and gateway gone.
- Second run, with the release trickled at ~200 KB/s: the progress
showed, and the run found the two bugs above (the red bar, and the old
gateway taking 46 seconds to exit).
- Third run, same trickle, after the fixes: the card read "Downloading
the new version: 0.6 / 6.1 MiB, 191 KB/s" five seconds after the click,
then 2.4 / 6.1 MiB with the bar measuring; no red bar; the old gateway's
page socket closed 4 ms after the control plane stopped; uv prepared the
package from the local file in 157 ms; the page reloaded itself onto
0.2.91.
- `raven upgrade` in a real pseudo-terminal against the same trickle:
the live line redrew in place 98 times from 0.1 to 6.1 of 6.1 MiB, then
uv installed from the local file.

Commands, all on this branch rebased onto current main:

- `make coverage` - 26,955 passed. 21 failed, and they are the same 21
that fail on unmodified main on this machine (research-tool and
simulation tests, and one that assumes LibreOffice is absent); a 22nd
was a load-sensitive test added here, fixed in this PR and then 25 of 25
under CPU load
- `scripts/coverage_gate.py diff --base-ref github/main --threshold 90`
- 100% (77 of 77 changed lines)
- `npm test --prefix ui-web` - 3096 tests pass
- `make lint-python` - pass; `uv run lint-imports` - 10 contracts kept,
0 broken
- `make lint-ui` - 0 errors (4 pre-existing `exhaustive-deps` warnings
in files this PR does not touch); `npm run lint:i18n --prefix ui-tui` -
up to date
- `node ui-web/scripts/check-page.mjs`, `check-css.mjs`,
`check-class-namespace.mjs` after `npm run build` and `build.py` - OK
- `mkdocs build --strict` - pass
- `npx commitlint --from github/main --to HEAD`,
`scripts/check_commit_messages.py`, `make check-source-language` and
`make check-large-files` with `COMMIT_RANGE=github/main..HEAD` - pass

New tests were mutation-checked; each edit below turns the test written
for it red. Server: relaunching `serve` instead of the supervisor,
waiting on the gateway instead of the supervisor, dropping the
unsupervised refusal, the busy refusal, the no-relaunch guard, or the
page-mount hand-over; not closing sockets on shutdown. Helper: never
downloading, not closing the status server before the relaunch, dropping
the Host check, not holding a failure, dropping the manifest or its
flush. Page: counting the ceiling from the start instead of from
silence, ignoring a held failure, dropping the "starting" line, a bar
that ignores or overshoots the fraction, the old 5xx probe rule, server
sentence instead of catalogue, `busy` routed to the failure card. Entry
point: never opening the page, opening it on the wrong port.

Note for reviewers running the gates locally: `make check-commits`
hardcodes `--from origin/main`, and `check-source-language` /
`check-large-files` default `COMMIT_RANGE` to `origin/main..HEAD`. Where
`origin` is not GitHub those ranges fail on unrelated commits; pass the
GitHub base explicitly.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The quick-start's update section (English and Chinese) now covers
updating from the page.

## Risk

User-visible changes:

- Bare `raven` opens the browser page instead of the TUI, leaving the
resident gateway `raven web` always leaves. Headless machines keep the
TUI; `raven tui` still names it outright.
- The page upgrade restarts the gateway, briefly disconnecting IM
channels. It is refused while any turn, sub-agent or pending question is
running.
- During a page-started upgrade the page's port is answered by the
upgrade helper (status and 503s) instead of nothing.
- The helper keeps the current version's wheels under
`~/.raven/cache/upgrade/<version>` (about 75 MB for a release with its
engines), because the tool's receipt names them as the install source.
- A `raven gateway` that starts inside an upgrade window waits it out
and exits instead of serving; the supervisor restarts it once the
install is whole.

Known edge: running `raven web` by hand in the seconds between the old
supervisor exiting and the new one starting can start a second
supervisor beside the relaunched one. The install itself stays protected
by the upgrade marker, and the page no longer tells the reader to do
this.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Security: the status endpoint is unauthenticated but loopback-only,
answers only the page's own Host (so a rebound host name gets 421), and
exposes version numbers and byte counts; the relaunch carries the same
token and cookie the `raven serve` path already did. Rollback: revert
the squash commit. Upgrades started from a release without this change
keep the old refusal, so there is no mixed-version state to unwind.

## Related Issues

N/A

---------

Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-27 00:36:05 +08:00
Zuyi Zhou
a5ead03898
fix(permissions): a deny rule stops its program however the command runs it (#783)
## Summary

A user exec deny rule could be stepped around by running the program it
names any other way than as the first word of the command.
`raven/permissions/rules.py` matched every exec rule against the command
as written, token by token. That is right for a rule that grants (`git
*` must not allow `sudo git push`), and the module docstring says so,
but it left a deny rule blind to wrappers, and three kinds of syntax (a
redirection outside the cwd, a substitution, a heredoc) sent the whole
command to the `*` rule without asking the deny rule at all.

Reproduced on origin/main 7db17b0cf through `PermissionGate.check`, with
`curl *: deny` beside `*: allow`, in ask and in smart mode:

| command | before | after |
|---|---|---|
| `curl https://x` | deny | deny |
| `bash -c "curl https://x"`, `sh -c`, nested `bash -c` | allow | deny |
| `env curl`, `FOO=1 curl`, `sudo -u bob curl`, `command curl`, `nohup
curl`, `timeout 5 curl`, `xargs curl` | allow | deny |
| `doas curl`, `pkexec curl`, `flock /tmp/lock curl`, `exec curl`,
`busybox curl`, `watch -n 5 'curl x'`, `su -c 'curl x'`, `eval 'curl
x'`, `env -S 'curl x'` (and its attached spellings), `find ... -exec
curl` | allow | deny |
| `curl https://x > /tmp/out` | allow | deny |
| `echo $(curl x)`, backticks, `echo "$(curl x)"`, `page="$(curl x)"`,
`cat <(curl x)`, `bash <<EOF` with curl in the body | allow | deny |
| `then curl x`, `do curl x`, `! curl x`, `coproc curl x` | allow | deny
|
| `/usr/bin/curl`, `curl.exe`, `"C:\Windows\System32\curl.exe"` | allow
| deny |

With only `curl *: deny` (no `*: allow`) the same commands asked instead
of being refused, and smart mode's judge could approve them. In full
mode, where a user deny rule is the one check that still refuses
(`test_user_deny_holds_in_full_mode`), any wrapper was enough to run the
command.

The change:

- `exec_rule_tier` asks the deny rules first, before any of the
narrowing, about every command the string can be seen to run
(`_commands_run`). That walk is `_iter_argv` from `shell_policy`, the
one the builtin classifier trusts (compound segments, sudo, env,
command, nohup, `sh -c`, xargs and the other runners), plus what a deny
rule must not be blind to beside it:
  - each segment as written;
- `$(...)` and backtick substitutions read off the raw text, so a
double-quoted one counts and a single-quoted one does not. The closing
`)` is found by a walk that treats a quoted or escaped parenthesis as
text and skips a double-quoted `$(...)` inside the body whole, and
counts no parenthesis between `case` and `esac` standing where a command
starts (a pattern's `)` is not the body's; `echo esac` is an argument),
so `echo "$(echo ')'; curl x)"` reads the curl (the first review
finding);
- a shell keyword in front of a command (`then`, `do`, `!`, `coproc`,
...);
  - `env -S`;
  - `find -exec` / `-execdir` / `-ok` / `-okdir`;
- for a wrapper in `_WRAPPERS` whose command position `_iter_argv` does
not know, every later word, with a word holding whitespace read as a
command.
- The program is also compared by its bare name: a path prefix and a
Windows `.exe` / `.com` / `.cmd` / `.bat` suffix are dropped.
- Comments and heredoc bodies that no shell reads are dropped first,
through the existing `executable_text`, so `cat <<EOF > notes.md` with a
curl line in the body is not refused.
- Allow and ask rules are unchanged: `sudo git push`, `env git status`,
`bash -c 'git status'` and `/usr/bin/git status` still ask under `git *:
allow`.
- `busybox`, `eval` and `exec` join `_WRAPPERS`, so a prefix starting
with one is never suggested and a typed one is refused, as for `sudo`
and `bash`.
- The sub-agent path (`raven/acp_client/permissions.py`, which refuses a
sub-agent's command when `user_tier` is deny) and commands on a
registered machine go through the same `exec_rule_tier` and gain the
same reach.

Each piece between shell operators is lexed on its own, so a piece the
lexer refuses drops only itself rather than every command beside it. The
walk errs towards reading too much, on purpose: a deny rule that fires
on a word which only looked like a command costs one refusal the model
can see and reword, and one that misses runs the command the rule was
written to stop. Commands that only mention the program stay allowed
(`grep -rn curl notes/`, `find . | xargs grep curl`, `rg -l 'curl -s'
src`, a commit message, a comment, `echo '$(curl x)'`, `man curl`,
`curlie`); those are pinned as tests. Known over-read: behind a wrapper
whose command position is unknown, an argument spelled exactly like the
program counts (`flock /tmp/curl make` is refused under `curl *: deny`).

What a deny rule still cannot see, and this PR does not claim: a command
a program builds for itself (`python -c "..."`, a script file, an alias
or shell function), and an unquoted Windows path, whose backslashes the
POSIX lexer every rule here uses has already removed (quoted, it
matches). The module docstring now says both.

One existing assertion pinned the old reading and is changed, not
weakened: in `test_exec_rules_compound_and_opaque`, `git status && echo
$(rm -rf /)` under `rm *: deny` asserted `ask`; it now asserts `deny`,
the point of the change.

Pre-existing and out of scope: the builtin unconditional guard shares
the gap for some of these shapes (`echo "$(rm -rf /)"`, `x="$(rm -rf
/)"`, `doas rm -rf /` and `exec rm -rf /` classify allow). It is left to
its own change, because widening a check with no approval path needs its
own look at false positives.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Head ccafbddba, on origin/main 588d2d41b.

```
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q -p no:cacheprovider tests/test_permissions_gate.py
  307 passed
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q -p no:cacheprovider tests/test_permissions_gate.py tests/test_acp_permissions.py tests/test_shell_approval.py tests/test_shell_policy_reasons.py tests/test_permissions_session.py tests/test_permissions_judge.py tests/test_agent_loop_approval.py tests/test_rpc_approval.py
  693 passed
COLORTERM=truecolor make coverage
  26362 passed, 108 skipped, 1 failed: tests/test_rpc_files.py::test_a_host_without_libreoffice_says_so, which fails on origin/main on this machine too
  (LibreOffice is installed here; CI has none). Run at ccafbddba.
COVERAGE_BASE_REF=origin/main make coverage-diff
  Diff coverage: 98.32% (117/119 executable changed lines)
uv run --frozen --python 3.12 --extra dev ruff check raven tests / ruff format --check raven tests
make lint-imports lint-deps lint-types / git diff --check
PYTHONPATH=. uv run --frozen --python 3.12 --extra dev python scripts/check_source_language.py origin/main...HEAD
make check-commits / PR_TITLE=... make check-pr-title / COMMIT_RANGE=origin/main...HEAD make check-large-files
git merge-tree --write-tree HEAD origin/main
  all clean
```

Revert-to-red: with `raven/permissions/rules.py` restored to origin/main
and the tests kept, `tests/test_permissions_gate.py` gives `63 failed,
244 passed`: all 39 cases of
`test_a_deny_rule_reaches_the_command_however_it_is_run`, the 12 of
`test_a_parenthesis_that_is_text_does_not_end_a_substitution`, the 4 of
`test_only_the_shell_words_case_and_esac_suspend_the_count` and
`test_a_piece_the_lexer_refuses_drops_only_itself` (the helpers do not
exist there), the three modes of
`test_the_gate_refuses_a_wrapped_command_a_deny_rule_names`, the three
new `validate_exec_pattern` rows, and the changed compound assertion.
The "only mentions the program" cases, the three
`test_the_text_after_a_closed_substitution_is_not_read_as_a_command`
controls and the allow-rule test pass on both sides; they pin that
nothing new is refused or allowed.

Mutation matrix over the new walk at 53c8c1a3a, one branch disabled at a
time (selection: the exec-rule tests in
`tests/test_permissions_gate.py`; the two `env -S` cases added after it
would each add one kill to the `env -S` row):

| mutant | result |
|---|---|
| drop `_iter_argv` from the walk | 14 failed (`bash -c`, `env`,
`FOO=1`, `command`, ...) |
| drop the segments as written | 1 failed (`env -S`) |
| drop substitutions | 2 failed (`echo "$(...)"`, `page="$(...)"`) |
| drop the keyword strip | 3 failed (`if/then`, `while/do`, `!`) |
| drop `env -S` | 1 failed |
| drop `find -exec` | 1 failed |
| drop the scan behind an unknown wrapper | 5 failed (`doas`, `pkexec`,
`flock`, `exec`, `busybox`) |
| drop reading a whitespace word as a command | 3 failed (`watch`, `su
-c`, `eval`) |
| drop the bare-name comparison | 3 failed (`/usr/bin/curl`, `curl.exe`,
quoted Windows path) |
| drop the Windows suffix | 2 failed |
| ask the deny rule after the narrowing, as before | 38 failed |
| treat single quotes as not quoting | 1 failed (`echo '$(curl x)'`
refused) |
| skip `executable_text` | 1 failed (`cat <<EOF > notes.md` refused) |
| control: break `_suggest_pattern` | 13 failed
(`test_exec_approval_shape`) |

And over the closing-parenthesis walk added for the review finding, at
3fbd2515a and 8f7a3b8fc (selection: the deny, parenthesis,
closed-substitution and case-word tests):

| mutant | result |
|---|---|
| single quotes do not quote | 2 failed (`')'`, `'('`) |
| no backslash escape | 1 failed (`\)`) |
| double quotes do not quote | 1 failed (`")"`) |
| no skip of a nested double-quoted `$(...)` | 1 failed (`"$(echo
")")"`) |
| no `case`/`esac` tracking | 3 failed |
| `case` matched inside a word | 2 failed (`showcase`, `case_1`) |
| control: drop `env -S` / drop the backslash escape | 3 failed / 1
failed |
| no `coproc` keyword (at ccafbddba) | 2 failed |
| `case`/`esac` counted anywhere, not only where a command starts | 3
failed |
| lex the whole text at once instead of per piece | 1 failed
(`test_a_piece_the_lexer_refuses_drops_only_itself`) |
| control: drop `find -exec` | 1 failed |

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Docs: `docs-site/docs/permissions.md` said substitution, backticks,
heredocs or unconfined redirection may leave only the fallback rule
applicable. That now holds for `allow` and `ask` patterns only, so the
bullet says so, and a new bullet states what a `deny` pattern reaches
and what it still cannot see. `permissions.zh.md` carries the same two
bullets. The module docstring of `raven/permissions/rules.py` carries
the same contract.

## Risk

User-visible: a command that runs a program a deny rule names is now
refused however it is spelled, where before it was allowed (with `*:
allow`, or in full mode) or asked. A command that only mentions the
program is not refused. The known over-read is described above; it is
one refusal the model can reword. Allow and ask rules behave exactly as
before.

Rollback: revert the squash commit; no config or state format changes.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-24 21:07:06 +08:00
silverLXT
afa469d4ac
refactor(*): one registry reader and one writer, and the coding agent can add a machine (#690)
## Why

In a research -> code -> oncall chain run on 2026-09-22, Raven-Code
called
`exec(machine="conn_cpu_32c")` and got "No connection is registered", on
a home
whose `connections.json` listed seven machines. It then read
host/port/key out
of the config and ran a raw `ssh root@<ip> -p <port> -i ~/.ssh/id_rsa`,
putting
the way in into the transcript.

Two gaps:

1. **Read side.** `raven.ops.connections.store_path()` looked only
beside the
instance's own config. A sub-agent runs on a rendered config in its own
state
directory, so it looked in an empty directory. The host already hands
every
sub-agent `RAVEN_HOME`; the registry just was not looked for there. Only
the
on-call launcher worked around this (it sets `RAVEN_CONNECTIONS`
itself).
2. **Write side.** `ops_connection_add` (#545) lived in the on-call
plugin. In the
usual chain the coding agent runs first, meets the empty registry first,
and
   had no way to take the owner's answer.

## What changes

- `store_path()`: env var -> `raven_home()/connections.json` if it
exists ->
beside the config if THAT exists -> the home (where a first add lands).
An
  install that kept its own list beside its config keeps working.
- New `raven/ops/connection_add.py`: `probe`, `ssh_defaults`,
`parse_probe`,
`write`, `write_ssh_alias`, and `ASK_OWNER` (the questions relayed to
the owner).
The tool calls these functions directly; `raven ops connection add` (the
terminal command for a person) calls the same ones, so the logic exists
once.
- New `raven/agent/tools/connection_add.py`: `ops_connection_add` as a
trunk tool,
**opt-in** via `tools.connectionAdd` (on in raven-code and raven-oncall,
off in
  the other three products, whose pinned tool faces are unchanged).
- `raven.ops.transport.make_ssh_runner(identities_only=)` +
`ISOLATE_IDENTITY`,
moved from the plugin: a key the tool picked itself is probed with that
key
  alone (see #545 review).
- On-call plugin: `oncall_flow/connections.py` imports trunk's reader
instead of
keeping a hand-aligned copy (758 -> 212 lines); keeps only campaign
lookups
(`get`, `display_name`, `resolve_into`, `describe`). Its own tool file,
  factory and manifest row are removed.
- Raven-Code guide (`TOOLS_CODE.md`) gains when to leave this computer:
write
here; verify where the software is; ask the owner for a machine only
when
installing the software here is wrong (GPU, memory) or heavy (compiled
solver,
toolchain, long download); never ssh from an address in the task
statement.
- `SHOWN` made public in `raven.ops.connections` (the plugin may not
import an
  underscored trunk name; `_SHOWN` stays as an alias).
- Docs: `docs-site/docs/oncall.md` / `oncall.zh.md` updated for the new
lookup
  and the tool.

## Not in scope

- The pre-dispatch registry gate removed in a44664d1b is NOT restored.
Where a
job runs is still decided inside the agent that runs it; this only makes
that
  agent find the owner's registry and able to add to it.
- #552 (exec refuses a typed ssh to a registered machine) is
complementary: it
only fires once the registry is visible, which this makes true for
Raven-Code.

## Tests

- `tests/test_ops_connection_add.py` (moved from the plugin suite,
patches trunk
modules), including the ssh `-G` measurement that only the candidate key
is
  offered under isolation.
- Five new cases in `tests/test_ops_connections.py` for the lookup
order.
- Plugin parity tests replaced by an identity pin (the plugin's names
ARE trunk's
  objects).
- Code launcher face ledger gains a machine lane; oncall face unchanged.
- Full suite: see CI.

@LivXue -- review requested: this touches `raven/ops` and the
`RAVEN_CONNECTIONS`
handoff described in a44664d1b.

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>

Generated with Claude Code (https://claude.com/claude-code)

---------

Co-authored-by: xiaotian.luo <xiaotian.luo@thetahealth.ai>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-24 18:36:28 +08:00
KT
7c10b6b295
docs: align the release notes template and quick start with the web-first install flow (#760)
## Summary

The release notes template that CI prefills into every release draft,
the structure sketch of the same sections in RELEASING.md, and the quick
start's upgrade recipe still described the 0.1.13 finish line. Since
then both installers end by opening Raven in the browser, where
first-run setup now happens; `raven upgrade` installs the plugin wheels
too; rerunning the installer cannot pick up a pre-release, since it
resolves `releases/latest`; and a release ships three plugin wheels plus
`raven-constraints.txt` and `raven-plugins.txt` beside the wheel and
sdist. Both files now say so.

The Upgrade section of the template and the "Update a one-line install"
recipe in docs-site/docs/quick-start.md and quick-start.zh.md are
written to match the upgrade code: stop the page, run `raven upgrade`,
and start Raven again only after the upgrade has finished. On Linux and
macOS the command execs the helper in the foreground and returns when
the install is done; on native Windows it spawns the helper and returns
at once, and a `raven web` started before the completion message runs
into the install marker and exits 75. `raven upgrade` installs the
latest stable release unless the install has joined the beta channel, in
which case it installs that channel's pre-release.

Each claim was checked against the v0.2.0 tree: the launch sections of
install.sh and install.ps1, the `web` command and
`_refuse_incomplete_install` in raven/cli/serve_commands.py,
`_handoff_upgrade` and `fetch_latest_for_channel` in
raven/updates/upgrade.py, the first-run gate in ui-web/src/app/boot.ts,
and the asset list in .github/workflows/release.yml.

The 0.2.0 draft is edited in place; this change only reaches the drafts
CI creates from now on. The `__VERSION__` and `__TAG__` placeholders and
the two TODO markers the release job and the notes step rely on are
unchanged.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `git diff --stat origin/main..HEAD`:
.github/release-notes-template.md, RELEASING.md,
docs-site/docs/quick-start.md, docs-site/docs/quick-start.zh.md
- `npx commitlint --from origin/main --to HEAD --config
commitlint.config.cjs`: exit 0
- `PYTHONPATH=. python scripts/check_commit_messages.py
origin/main..HEAD`: exit 0
- `PYTHONPATH=. python scripts/check_source_language.py
origin/main..HEAD`: exit 0
- Non-ASCII scan of the template and RELEASING.md: nothing found (the zh
quick start page is Chinese by design)
- No tests run: markdown only; the one code consumer is the sed in
release.yml, whose placeholders are unchanged. The docs build runs in
CI.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Documentation only. Rollback is reverting the commits; the next tagged
release then prefills the old text again.

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-24 02:17:40 +08:00
Zuyi Zhou
34ed138cc7
fix(plugins): a plugin that fails to load no longer takes every other plugin with it (#703)
## Summary

One plugin that failed to activate dropped every plugin.
`PluginRegistry.activate` raised on the first plugin whose factory would
not import (or whose contribution name collided), and
`build_plugin_registry` (`raven/core/plugin_stack.py`) answered by
returning an empty registry. A single broken user plugin therefore
silently removed research-flow, ppt-engine, design-engine, the memory
backend and the bundled playbook tools, with one stderr warning as the
only trace. Reproduced with two mock plugins under a scratch
`RAVEN_HOME`: `aa-broken` (factory raises `ImportError`) beside
`zz-good`; before this change the registry came back with `activated:
[]`, after it with `design-engine, everos-memory, playbook, ppt-engine,
zz-good` activated and `aa-broken` named as failed.

Changes:

- **Per-plugin activation** (`raven/plugins/registry.py`). Each admitted
plugin activates on its own. One that fails is rolled back whole (no
partial contribution in any table, and the `sys.path` entry
`_ensure_importable` appended for it is withdrawn), recorded as a
`PluginActivationFailure` (`activation_failures()`), and the plugins
after it still activate. In a name conflict the plugin activated first
(discovery order, by id) keeps the name and the other is recorded as
failed. A module `__getattr__` that raises something other than
`AttributeError` is now a recorded failure rather than an uncaught
crash.
- **Said to the user** (`plugin_stack.build_plugin_registry`,
`runtime.build_runtime`). Each failure is one notice naming the plugin,
its cause and the `plugins.disabled` escape, through the host's notifier
(`HostWiring.notify`), else stderr - the same channel the
missing-memory-backend notice already uses. When the configured memory
backend belongs to a plugin that failed, the memory notice names that
plugin and its cause instead of "no installed plugin provides it".
- **`raven plugins`** shows a `failed` status and the cause under the
table (the command called `activate` directly and previously crashed on
such a plugin).
- **Agents smoke check** (`raven/cli/agents_commands.py`) matched the
old log line "plugin activation failed", which no longer exists; it now
reads the shared `PLUGIN_FAILURE_MARKER`, and its test prints the notice
the real producer writes, so the two cannot drift.
- **Sub-agents inherit the host's opt-outs**
(`product_render.write_rendered`, `inherit_plugin_opt_outs`). A product
engine inherits `RAVEN_HOME`, so it scans the host's `<home>/plugins`
and the shared interpreter's entry points, but it read
`plugins.disabled` only from its own rendered config; a plugin disabled
on the host came back in every product. Every render now merges the
host's list after the product's own. The product's own engine plugin
never travels (`own_plugins=`, passed by all five launchers and the
scaffold): the PPT agent is `ppt-engine`, and a host turning it off for
its own agent is not a request to run the PPT agent without it. A
contract test reads every launcher's `write_rendered` call and requires
it to name its `*_PLUGIN_ID`.
- `CONTEXT.md` (Plugin Registry) and
`docs-site/docs/building-plugin{,.zh}.md` describe the new behaviour.

Not in this PR: the Web UI plugin list (`ext.list`) is discovery-only
and does not show a failed plugin; adding it is a wire change
(`ExtListResult`, openrpc, generated TS, the page) and is left for a
follow-up. The TUI, gateway and `raven agent` hosts show the notice
through their notifiers; the rpc stack (serve/acp) prints it to stderr,
which is also what the agents smoke check reads.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

**Rebased**: head 7cd000dbc on origin/main 8ee5ae0d8, without conflicts.
`git range-diff` shows every commit replayed unchanged. The branch's own
test files rerun at this head: 395 passed. Numbers elsewhere in this
description are from earlier heads.

Head b4d51aee8, rebased onto origin/main 6551f8259 after #698 and #708
merged. #698 conflicted in `raven/config/product_render.py` and
`tests/test_config_product_render.py`: both PRs added a function at the
same spot (`inherit_host_denials` there, `inherit_plugin_opt_outs` here)
and a call in `write_rendered`. The resolution keeps both functions and
all their tests; `write_rendered` now reads `host_config()` once and
passes it to both. #708 touches none of these files. At this head:

```
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q -p no:cacheprovider
  25948 passed, 109 skipped
uv run --frozen --python 3.12 --all-extras pytest -q tests/test_config_product_render.py
  44 passed
ruff check / ruff format --check / make lint-imports lint-deps lint-types / git diff --check origin/main...HEAD
check_source_language.py origin/main...HEAD / make check-commits / check-large-files
  all clean
cd ui-tui && npm run lint:rpc && npm run type-check && npm test
  144 files, 2106 passed
```

The earlier `TUI checks` failure (run 35834019105, head 5fc374035) was
`newInstancePicker.test.tsx > quick-selects by the number it printed`.
This PR changes nothing under `ui-tui/`; the test passed 5 of 5 runs
locally at this head and in the full ui-tui run above.

The numbers below are from the earlier heads and are kept for the
record.

```
uv run --frozen --python 3.12 --extra dev ruff check raven evolver agents plugins-dist tests scripts
  All checks passed
uv run --frozen --python 3.12 --extra dev ruff format --check raven evolver agents plugins-dist tests scripts
  2083 files already formatted
make lint-imports && make lint-deps && make lint-types
  Contracts: 10 kept, 0 broken / No dependency issues found / All checks passed
git diff --check origin/main...HEAD
  clean
COLORTERM=truecolor make coverage        (full suite, head before rebasing onto 7fdf296f4)
  25847 passed, 109 skipped
COVERAGE_BASE_REF=origin/main make coverage-diff
  Diff coverage: 100.00% (71/71 executable changed lines)
PYTHONPATH=. uv run --frozen --python 3.12 --extra dev python scripts/check_source_language.py origin/main...HEAD
  exit 0
make check-commits
  pass
git merge-tree --write-tree HEAD origin/main
  clean
```

After rebasing onto 7fdf296f4 (its two new commits touch none of these
files), the touched suites at this head:

```
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q tests/test_plugin_*.py tests/test_core_plugin_stack.py tests/test_cli_plugin_commands.py tests/test_cli_agents_commands.py tests/test_cli_tui_commands.py tests/test_config_product_render.py tests/test_agents_*launcher*.py tests/test_everos_plugin_discovery.py tests/test_release_plugin_list.py tests/test_cli_gateway_commands.py
  727 passed
uv run --frozen --python 3.12 --all-extras pytest -q tests/test_subagent_vendored_agents.py tests/test_rpc_subagents.py tests/test_subagent_acp.py
  452 passed
```

Revert-to-red, one production file restored to origin/main at a time
with the tests kept:

```
raven/plugins/registry.py        -> 2 errors (plugin_stack imports PluginActivationFailure)
raven/core/plugin_stack.py       -> 6 failed
raven/core/runtime.py            -> 1 failed (build_runtime hands the host's notifier to the registry)
raven/config/product_render.py   -> 42 failed
raven/cli/agents_commands.py     -> 1 failed (smoke reds a handshake whose plugin did not load)
raven/cli/plugin_commands.py     -> 1 failed (failed row and escaped cause)
agents/raven-ppt/run.py          -> 2 failed (engine exemption, launcher contract)
```

Ten existing tests pinned "activation raises" (`test_plugin_registry`,
`test_plugin_bootstrap`, and the conflict tests for tools, hooks, tool
gates and session observers). They now assert the new contract instead:
the later plugin is recorded with the same reason text, the earlier one
keeps the name, and nothing of the loser is registered. Two stubs in
`test_cli_tui_commands.py` took `build_plugin_registry(cfg)` and now
accept the `notify` keyword.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- A plugin conflict used to disable all plugins; it now keeps the plugin
that activates first and skips the other with a notice. Two plugins
contributing the same name therefore run with one of them instead of
none.
- A product sub-agent now starts with the host's `plugins.disabled`
merged into its config (its own engine excepted). A plugin that was
disabled on the host but relied on inside a product stops loading there;
the remedy is to enable it on the host.
- `PluginRegistry.activate` no longer raises `PluginConflictError` /
`PluginFactoryImportError`; the only in-tree callers were
`assemble_plugin_registry` and `raven plugins`. The exception classes
stay exported.
- Security: the change only narrows what loads. A failed plugin's
directory leaves sys.path, and the inherited opt-outs can only remove
plugins from a product, never add one.
- Rollback: revert the squash commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-23 22:41:21 +08:00
Dizhan Xue
6a038a67df
fix(docs): light the contents rail between headings and use raven.evermind.ai's favicon (#734)
## Summary

Two fixes to the documentation site.

**The contents rail went dark between headings.** The right-hand table
of contents lit an entry only while its heading was inside the viewport.
Partway through a section longer than the screen every heading sits
above or below it, so nothing was lit, the gold stretch collapsed to
zero height, and the column stopped following the reader. On the built
site before this change, 18 of 28 English pages had such stretches: 114
of 1264 scroll positions sampled every 120px. `sync()` in
`docs-site/docs/javascripts/toc-progress.js` now falls back to the
nearest heading above when none is in view and lights that entry alone.
The lit stretch and the column follow read the same list, so both pick
it up with no further change. Above the first heading nothing is lit,
since no section has begun there. The design spec states the rule, and
the rail's section comment in `evermind.css`, which still credited
Material with marking the section, now names what does.

**The tab icon differed from raven.evermind.ai.** The site drew its own
simplified EverMind mark, a single heavy lemniscate without the rings.
raven.evermind.ai links one 192px PNG of the full mark for its light,
dark and apple-touch icons alike. `theme.favicon` in
`docs-site/mkdocs.yml` now inlines that PNG byte for byte as a data URI,
since AGENTS.md section 7 keeps image files out of `docs-site/`. For
whoever refreshes it: that site's CDN answers any `Accept` naming
`image/avif` with a lossy AVIF re-encode (5178 bytes), and only a plain
`Accept: */*` returns the original PNG (5184 bytes). The comment above
the value says so.

Checked and deliberately left alone:

- The mark at the top of the navigation rail (`--em-mark` in
`evermind.css`) is a separate drawing and still the simplified one. This
change is the tab icon only.
- raven.evermind.ai also sets an apple-touch icon. Material emits no
such link, and adding one needs a template override, which is an `.html`
file that section 7 blocks.
- The favicon's identity is verified by the measurements below rather
than pinned by a test: a pinned hash would only detect change, and
comparing against the live site would put the network in the suite.
- `tests/integration/` is outside every CI job (`norecursedirs`), so the
e2e results below are local runs.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run --frozen --extra browser --group docs pytest
tests/integration/test_docs_site_controls_e2e.py
tests/integration/test_docs_site_search_e2e.py -n 0 -q`: 63 passed.
- `uv run --frozen --extra browser --group docs pytest
tests/test_docs_site.py -q`: 16 passed.
- The new
`test_table_of_contents_keeps_the_section_above_lit_between_headings`
computes, from the Sandbox page's own layout, every stretch between two
headings (or after the last) with none on screen, and checks the middle
of each. Against the unfixed script it fails (`at 1354px: assert [] ==
[2]`), and with the fix it passes.
- A sweep of every built page in headless Chrome at 1440x900, one sample
every 120px, checking the rule at each position: 0 violations over 1264
English, 1145 Chinese and 21 Overview positions after the change, and
wherever an entry is lit, the first one sits inside the contents column.
The unfixed script gives 114 violations over the same English positions.
- Favicon: each of the 63 built HTML files carries exactly one icon
link, and its decoded bytes match the PNG raven.evermind.ai links
(sha256 `4a5051e1781fac64...`). Drawn in Chrome at 16, 32 and 192px, the
docs icon and that PNG differ in 0 pixels.
- `mkdocs build --strict`, `scripts/check_source_language.py
origin/main...HEAD`, `scripts/check_large_files.py origin/main...HEAD`,
`scripts/check_commit_messages.py origin/main..HEAD`, commitlint,
`pre-commit run --from-ref origin/main --to-ref HEAD`, ruff and `git
diff --check`: all pass.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The last box is left blank because nothing needed it: no documentation
page describes the rail or the icon, and the design spec is updated in
this change.

## Risk

- Between headings, the entry of the section above now stays lit and the
contents column keeps it in view. While any heading is on screen the
behaviour is unchanged.
- The tab icon changes to the full mark. Each page's head now carries
about 6.9 KB of base64 for it, against about 0.6 KB for the previous
SVG.
- The new code relies on no browser API newer than
`Array.prototype.includes` (2016).
- The two halves touch disjoint files, so either can be reverted on its
own.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

The inlined PNG was read chunk by chunk: standard chunks plus three
ImageMagick timestamps, and nothing after IEND. Inlining it adds no
third-party request, which hotlinking the CDN copy would have.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-09-23 22:09:48 +08:00
KT
a7f102c292
refactor(*): retire the deep_research tool in favour of the research agents (#717)
## Summary

Retires the built-in `deep_research` tool: the MiroThinker HTTP tool,
its
unconfigured stand-in of the same name, the async channel manager, the
`raven deep-research` command group, `tools.deepResearch` /
`DeepResearchToolConfig`, the loop's registration / mid-session
promotion /
broker and submit wiring, the `inline_tool_stream` flag (its only
consumer
was the tool's stream callback) and the `delivered_sink` plumbing from
#705
(its only writer was that callback). The WebUI settings page loses the
row
and its MiroThinker key panel, the ACP command menu loses
`deep-research`,
and the three transcript i18n keys go (readers fall back to the raw
name).

Research keeps two surfaces: the Raven-Research agent and the
MiroThinker
sub-agent preset; neither shared code with the tool.
`TurnRequest.deliver_text`
stays as a spine contract with its own tests, although nothing in-tree
produces it any more; the comments that used deep research as its
example
now name the sub-agent relay.

Config: a load-time migration (floor 10) drops `tools.deepResearch` /
`tools.deep_research`, one notice per spelling it finds, silent and
idempotent when nothing changes (it
re-runs on every load for configs parked by the research rename, and
`load_raven_config` calls it a third time without writing the stamp). A
`deep_research` entry in `tools.disabledTools` is left as written: that
list
is the general name denylist and a plugin tool may carry the name
(plugin
tools register last so one can shadow a built-in), so dropping the entry
would put such a tool back on offer. The first revision stripped it; the
review caught that and the strip is gone.

Kept on purpose (see the grep gate below): the MiroThinker sub-agent
preset
and its fixtures, `agents/raven-research/**`, the four trajectory
cassettes
that recorded the tool's schema (the replay serves the recording's own
tool
list), `benchmarks/skill_evals/queries.jsonl`, and the frozen
vendored-fork
fixtures, and the `deep_research` deny rows in the five
`agents/*/config.json`
files (the same denylist argument as the migration; the launcher parity
tests
are unchanged).

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

- `uv run ruff check raven/ tests/` and `uv run ruff format --check
raven/ tests/`: clean.
- `uv run pytest -q` (whole suite, with coverage, at the rebased head):
25840 passed, 109 skipped; `python scripts/coverage_gate.py ratchet
--tolerance 0.05`: line 90.10% vs baseline 87.30%, branch 83.73% vs
79.43%, passed.
- `uv run pytest tests/test_config_loader.py
tests/test_config_update_tools.py tests/test_agent_loop_run_emit.py
tests/test_cli_one_shot_spine.py tests/test_rpc_bootstrap.py
tests/test_rpc_spine.py tests/test_rpc_turn_send.py
tests/test_turn_wire_shape_conformance.py tests/test_core_cron_stack.py
tests/test_cli_gateway_commands.py tests/test_cli_smoke.py
tests/test_acp_methods.py tests/test_rpc_console.py
tests/test_tool_registry_timeout.py tests/test_question_broker.py
tests/test_provider_auth_method.py tests/test_agents_ppt_launcher.py
tests/test_agents_design_launcher.py
tests/test_agents_research_launcher.py tests/test_tool_capabilities.py
tests/test_l4_entrances.py tests/test_settings_tool_groups.py
tests/test_trajectory_regressions.py tests/test_living_docs.py
tests/test_cli_onboard_commands.py tests/test_cli_theme.py -q`: 1500
passed.
- Mutation check on the migration: with the `_migrate_config` dispatch
neutered, three of the four new `tests/test_config_loader.py` cases
fail; with the `_persist_migrations` dispatch neutered,
`test_the_deep_research_section_also_leaves_the_file_on_disk` fails;
with the pre-review stripping behaviour restored, all four fail.
Restored: 74 passed.
- `npm test --prefix ui-web` (208 files, 2884 tests), `npm run lint
--prefix ui-web`, `npm run type-check --prefix ui-web`, `npm run
gen:check --prefix ui-web`: pass.
- `npm run lint:i18n --prefix ui-tui`, `npm run type-check --prefix
ui-tui`, `npm test --prefix ui-tui` (144 files, 2106 tests): pass;
`messages.generated.ts` regenerated with `npm run gen:i18n`.
- Grep gate: `grep -rniE 'deep[ _-]?research' .` (excluding
node_modules, .venv, dist) hits only the kept surfaces listed above plus
CHANGELOG history.
- Real machine, isolated `RAVEN_HOME` whose config carried
`tools.deepResearch` and `disabledTools: ["deep_research", "exec"]` with
a version-9 stamp: `raven agent -m "..."` printed the `Config updated:`
notice for `tools.deepResearch` once, answered, and left the file
without that section, `disabledTools` unchanged (`["deep_research",
"exec"]`), stamp 10; `raven deep-research` answers `No such command`;
the served WebUI (`raven serve`) Settings > Tools page lists 36 rows
with no `deep_research` and `exec` still off, and `ext.list` carries no
`deep_research`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- User-visible: the `deep_research` tool, the settings row and `raven
deep-research` are gone; a config that still carries the section loses
it once at load with a notice; a `deep_research` entry in
`tools.disabledTools` stays as written. Reports the tool wrote under
`<workspace>/deep_research/` stay on disk. Archived transcripts that
recorded a `deep_research` call replay with the generic icon and the raw
tool name instead of the `researched` verb.
- Coverage: the two deleted modules were well covered, but the live
totals sit several points above the committed baseline, so the ratchet
passes with room to spare (measured locally, see Verification).
- Rollback: revert the PR; the migration is additive on the stamp, so a
reverted build reads a migrated config unchanged (the section it would
have read is simply absent).

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#156 (its deep-research half no longer applies; tracing half unaffected)

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-23 18:19:42 +08:00
Zuyi Zhou
6551f8259e
fix(*): a one-shot run names the calls it refused and counts its sub-agents' spend (#708)
## Summary

Two things `raven agent -m` reported were not true.

**Refused calls were silent.** A one-shot has no approval responder, so
under the ask tier (or smart, when the reviewer escalates) every
`write_file`, `edit_file`, spawn, DAG and mutating `exec` is refused.
The model is told to carry on without it, and the run still exited 0
with nothing on screen. A driver reading the status code could not tell
that run from one that did the work.

- `PermissionGate` now records every refusal on the turn it belongs to
(`PermissionTurn.refusals`): the tool, the same action line an approval
prompt shows, the reason the model was given, and the decision source.
All eight refusal paths go through one `_refuse` helper, so none can
skip the record.
- `start_permission_turn` returns the object it binds. The scheduler
builds the turn's task, so an entrance cannot read that context back
afterwards; the one-shot runner keeps the handle and passes the refusals
to `build_one_shot_spine(on_refusals=...)` once the turn ends.
- `agent -m` lists the refused actions after the reply, deduplicated by
action. If any needed approval nobody could give, the run exits **3**.
It is not 1, because the turn did not fail, and a driver needs to tell
"could not run" from "ran without its mutations". A builtin or user deny
rule is listed but leaves the status at 0: that is the operator's own
policy doing its job.

**The cost line left out sub-agents.** The summary read only this
process's in-memory tracker. A product sub-agent
(Raven-Code/PPT/Design/Research/Oncall) is a separate ACP process that
records its calls with its own tracker, under its own session, billed to
our root. The line now adds `delegated_usage(root, since, until)`:
telemetry rows billed to this root by *other* sessions within the turn's
window. The root's own rows are excluded, since its tracker already
counted them. The line says `incl. N sub-agent calls` when there are
any.

- Checked against real telemetry on the author's machine: for a Web UI
session that delegated to an ACP agent, delegated plus own rows equal
the `session.usage` panel exactly (35 + 11 = 46 calls). The old line
would have shown about 0.11 of the actual 0.41 USD.
- `session.usage` and the new reader share one row parser and one
attribution rule (`read_usage_rows`, `row_root`), now in
`token_wise/usage_tracker.py` next to the writer, so the CLI and the
panel cannot drift apart.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest -q tests --ignore=tests/integration`: 25602 passed, 120
skipped.
- New tests in the existing files: `test_permissions_gate.py` (every
refusal path leaves a record, an allow leaves none, answered refusals
keep their source), `test_cli_one_shot_spine.py` (refusals reach the
caller through the real scheduler, a second turn does not re-report the
first, the summary adds delegated usage and advances its window,
tolerates an unreadable ledger), `test_cli_agent_commands.py` (exit 3
with the report, exit 0 for a deny rule, no report when nothing is
refused), `test_token_wise_usage_tracker.py` (reads what a child tracker
actually writes; window, root and reported-price filters).
- Mutation check: removing each of `_report_refusals(...)`, the gate's
`note_refusal(...)`, the root-row exclusion, the delegated read, and the
window advance makes the matching test fail.
- Real run against the configured provider, `raven agent -w <tmp>
--permission-mode ask -m "Create hello.txt ... using write_file"`: the
reply, then `1 action(s) were refused in this run: - write_file:
write_file content='hi' path='hello.txt'` with the reason and the
`--permission-mode full` hint; exit status 3; no file created. Before
this change the same run exits 0.
- `uv run ruff check` / `ruff format --check`, `make
check-source-language`, `make check-large-files`: clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- User-visible: a one-shot whose ask-tier calls were refused now exits 3
instead of 0 and prints the refusal list. A script that treated such a
run as success will now see a non-zero status; that is the point of the
change, and `--permission-mode full` restores a clean exit for work
meant to mutate. `--help` and `docs-site/docs/permissions{,.zh}.md` say
so.
- The summary line reads the telemetry directory once per reply. A read
failure falls back to the host-only numbers (logged at debug); it never
fails the turn.
- Rollback: revert this commit; nothing is persisted in a new format.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-23 17:48:34 +08:00
Zuyi Zhou
89a24c5769
fix(*): a command the host denies stays denied once it is handed to a sub-agent (#698)
## Summary

A host deny rule bound the host's own tool calls and nothing it
dispatched.
The host answered every `session/request_permission` from an ACP
sub-agent
with the most permissive option (`raven/acp_client/permissions.py`), and
a
product launcher rendered its own config without the host's permissions
(Raven-Code pins `mode: ask` and carries no rules). Measured end to end
before this change: with `permissions.tools.exec {"curl *": "deny"}` and
`tools.exec.extraDenyPatterns ["\\bcurl\\b"]` on the host, a curl handed
to
Raven-Code asked the host, was answered "Allow for this session", and
hit a
loopback sink.

Refusals now travel, on both sides, because a refusal needs nobody to
answer:

- **Launcher side.** `product_render.write_rendered` merges the host's
`deny`
entries of `permissions.tools` and `tools.exec.extraDenyPatterns` into
every
product render (strictest wins; a product exec tier written as one
string
becomes the `*` fallback beside the host's patterns; a pattern that does
not
compile is left behind, as the host's own live reader leaves it). Every
launcher and the scaffold end in that call, and a test pins that. This
is
  the half that matters for raven's own products: a call their own gate
  allows never produces a request the host could refuse.
- **Host side.** The approver reads the command a request names
  (`toolCall.rawInput.command`, as sent and unquoted, and every codex
  `commandActions` entry) against the host's builtin deny list with
`extraDenyPatterns` and its `deny` rules, read live, and answers a match
with the agent's reject option (cancel only when none was offered). A
check
  that raises refuses. A parse error is not a refusal. This also covers
  third-party agents, for what they ask about.
- **Server side.** Raven's own ACP broker now sends a shell call's
command as
`rawInput.command` (redacted like the title), so a raven host can read
it.

- **Logs.** Both approver log lines (the INFO refusal, and the DEBUG
approval, which logged the agent's own title) go through the ACP
redactor,
since the command is the sub-agent's to author and the log is retained.
The
redactor moves from `raven/acp/redact.py` to `raven/security/redact.py`
(content unchanged but one docstring paragraph), because
`raven.acp_client`
  may not import `raven.acp`, and one table beats a second copy.

Deliberately not changed: the ask tier and the mode. What no deny rule
names
is still approved, as before; routing the ask tier to the host's human
is a
separate decision. A rule added on the host reaches an already running
raven
product at its next launch (the host-side check is live, the render is
not).
Agents launched with a never-ask setting never send a request, so only
their
own configuration binds them. Docs (permissions, protocol backends, both
languages) and the `Unattended Approval` entry in `CONTEXT.md` say this.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Head 703c7ceaf, rebased onto main 3a463d6cd (`git range-diff` shows both
commits `=` against the reviewed 4f8e0af95; the head is linear again,
the two
"Update branch" merge commits are gone). At this head: the six focused
suites
480 passed, the e2e 2 passed, `git diff --check` clean, `make
lint-imports`
10 kept. The full suite was run at the same two commits on earlier
bases:
25878 passed / 109 skipped at ca5e5dc15 (shown below), 25881 passed /
110
skipped at 4f8e0af95, diff coverage 100% at both. Main's newer commits
touch
`CONTEXT.md` in a different entry and none of the other files here:

```
uv run pytest -q tests/test_subagent_acp.py tests/test_acp_redact.py tests/test_acp_permissions.py tests/test_acp_server.py tests/test_config_product_render.py tests/test_agents_code_launcher.py
  480 passed
uv run pytest -q -n0 -m integration tests/integration/test_subagent_host_deny_e2e.py
  2 passed   (control run hits the sink; denied run does not, reply names the deny rule)
COLORTERM=truecolor make coverage   (fresh worktree at ca5e5dc15)
  25878 passed, 109 skipped in 619s
COVERAGE_BASE_REF=origin/main make coverage-diff
  Diff coverage: 100.00% (120/120 executable changed lines)
uv run --frozen --python 3.12 --extra dev ruff check raven evolver agents plugins-dist tests scripts   -> All checks passed
uv run --frozen --python 3.12 --extra dev ruff format --check raven evolver agents plugins-dist tests scripts   -> 2084 files already formatted
make lint-imports   -> 10 kept, 0 broken
make lint-deps && make lint-types   -> no issues / All checks passed
git diff --check origin/main...HEAD   -> clean
PYTHONPATH=. uv run --frozen --python 3.12 --extra dev python scripts/check_source_language.py origin/main...HEAD   -> exit 0
make check-commits; make check-pr-title; COMMIT_RANGE=origin/main...HEAD make check-large-files   -> pass
```

Revert-to-red, one production file at a time restored to origin/main
(the
focused suites plus the e2e where it applies; measured on the first
commit
before the rebases, which touched none of these files):

| reverted | unit | e2e |
|---|---|---|
| raven/config/product_render.py | 12 failed | 1 failed (sink not hit:
the host approver refused it, reply "User denied this action"; only the
wording assertion fails) |
| raven/acp_client/permissions.py | 2 failed, 14 errors | - |
| raven/acp/permissions.py | 2 failed | - |
| all three | 15 failed, 14 errors | 2 errors |

The first row is the two halves covering each other: with the launcher
half
reverted the host half still stops the command. For the second commit,
each
log line's redaction reverted alone turns
`test_the_logged_refusal_and_approval_carry_no_credential` red (1 failed
each).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible: an ACP sub-agent (Raven products, Claude Code, codex) now
has a
command refused when it matches the host's deny rules or the builtin
catastrophe list, where it used to run. Hosts with no deny rules see no
difference except the builtin list. Rollback: revert this commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-23 17:46:08 +08:00
Handsome-wzw
d4ff0088e6
docs: add an Update Raven section to the quick start (#711)
## Summary

The quick start page on the documentation site says how to install Raven
and
not how to update it. The two answers differ by how Raven was installed,
and the
command a reader reaches for first is wrong for a source checkout:
`raven
upgrade` refuses an editable install with "Editable Raven installations
cannot
be upgraded automatically. Pull the source checkout and rebuild Raven."

This adds an **Update Raven** section after **Install Raven** on the
quick start
page, in English and Chinese, with one subsection per way Raven was
installed.
The README is unchanged.

- **Update a one-line install**: run the same installer again, or use
`raven
upgrade` between `raven web --stop` and `raven web`. The stop and start
are
needed because the CLI's upgrade hands the install to a helper that
relaunches
nothing (`raven/updates/upgrade.py`: "The CLI passes 4 args and keeps
the old
exec-in-place behaviour"). The one path that does relaunch is the served
page's own update, and it refuses when the page is hosted by `raven
gateway`
(`system.upgrade`: "Run `raven upgrade`, then restart the gateway"),
which is
  what `raven web` runs.
- **Update a source checkout**: `git pull`, then the installer run as a
file
  again (`./install.sh`, or `.\install.ps1` in PowerShell). Both scripts
reinstall the checkout in editable mode, rebuild the TUI and WebUI only
when a
source file is newer than the built one (`is_stale` in `install.sh`, the
`LastWriteTimeUtc` check in `install.ps1`), stop a running WebUI, and
end in
  `raven web --foreground`.
- The section opens with what a reader is left with either way: a
foreground
WebUI that Ctrl-C stops and `raven web` restarts in the background, and
the
settings and conversations in `~/.raven` untouched -- neither script
writes
  there beyond the Node runtime.

The subsections are named "Update a one-line install" and "Update a
source
checkout" rather than reusing Install's "From a source checkout", whose
anchor a
second heading of that name would collide with. The zh page gives each
heading
the same explicit anchor the English page generates.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Every sentence was checked against the code it describes, not against
earlier
docs: `raven/cli/upgrade_commands.py` (the editable refusal),
`raven/updates/upgrade.py`
(the helper's relaunch argument and the CLI's four-argument handoff),
`raven/rpc/methods/system.py` (`system.upgrade` refusing under `raven
gateway`),
and the closing steps and staleness checks of `install.sh` and
`install.ps1`.
`raven upgrade --help` lists `--check` as described. On this machine a
re-run of
`./install.sh` from a checkout stopped the running WebUI, skipped the
page build
because the page was newer than its sources, and ended in `raven web
--foreground`.

- `uv run --frozen --python 3.12 --only-group docs mkdocs build -f
docs-site/mkdocs.yml --strict`
  -- exit 0, no warnings
- In the built site, the English page's generated anchors and the zh
page's
explicit ones are the same (`update-raven`, `update-a-one-line-install`,
`update-a-source-checkout`), and both pages list the three headings in
their
  table of contents
- `scripts/check_source_language.py`,
`scripts/check_commit_messages.py`,
  `scripts/check_pr_title.py` -- all clean

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Docs only; no behaviour changes. The section describes two facts that
live in
code and could drift from it: that `raven upgrade` does not restart a
running
WebUI, and that both installers end in a foreground `raven web`. If
either
changes, this section should change with it. Rollback is a revert.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-23 17:03:25 +08:00
Dizhan Xue
5ef7b8302f
feat(*): give every channel its app icon and a name in both languages (#696)
## Summary

The channels section of the settings dialog drew a hashed letter tile
for every
entrance. It now draws each app's own icon, in the list row and in the
picked
channel's header, and every entrance is named in both of the web UI's
locales.

**Icons.** `ui-web/src/assets/channels/` holds eleven files:

- Nine are the icons the vendors publish on their App Store listings
(China
  storefront for Feishu, DingTalk, WeCom, WeChat and QQ; US for Slack,
Telegram, Discord and WhatsApp), resampled from the 512px artwork to
96x96
PNG. For the five Chinese apps the listing is where a colour icon was
found:
the permissively licensed icon sets checked (lobe-icons,
`@cherrystudio/ui`,
SVG Logos, Simple Icons and the brand sets on Iconify) carry them only
as
  single-colour glyphs.
- `matrix.svg` and `mochat.svg` are the files matrix.org and MoChat
serve as
  their own marks, copied byte for byte.
- Mail is a protocol rather than a brand, so it gets a neutral envelope
drawn
  in the component.

`scripts/refresh_channel_marks.py` names the listing or URL behind every
file and
regenerates the directory; `LICENSES/README.md` records the basis for
showing
them (the vendors' marks, identifying the row they sit on, as with
AMD's).
`ChannelMark` is keyed by channel id, never by the row's name, and falls
back to
the letter tile for an id with no mark and for a file that fails to
load. The
asset digest helper moves out of `ProviderMark.tsx` into
`lib/assetStamp.ts` so
both marks share it and a refreshed icon is not served from cache.

**Names.** Six brands bypassed the message catalogue with one verbatim
spelling,
and `gui.chan.mochat` held the Chinese name in its English slot. Every
entrance
now has a `gui.chan.<id>` entry in both locales (Slack, QQ, Telegram,
Discord,
WhatsApp and Matrix keep their Latin names in Chinese, as their own
Chinese
builds do), the verbatim path is gone (`ConnChannel.key` is required),
and the
English name is MoChat. The adapter's `display_name`, which the CLI
prints, and
the two channels docs pages also say MoChat. The TUI's generated
catalogue is
regenerated.

**Guards.** `tests/test_ui_channel_marks.py` holds the mark table equal
to the
adapters, the shipped files and the refresh script, and the PNGs to
96x96.
`catalogue.test.ts` holds every entrance to both locales and to English
in the
English slot. `tests/test_channels_display_names.py` holds the CLI and
the web UI
to one English name per channel.
`scripts/gates/channel-mark-css.test.mjs` reads
the expected box off `.pmtile`'s own rule, since the mark stands in for
that
per-container-sized tile.

**Also in this PR.** `.pre-commit-config.yaml` adds `channels/` to the
end-of-file fixer's exclude beside `providers/` and `agents/`:
matrix.org writes
`matrix.svg` without a trailing newline, and appending one would end the
byte-for-byte copy.

**Existing tests whose meaning changed.** `test_channels_mochat.py` now
expects
the MoChat display name; `source.test.ts` now expects every row, a brand
included, to be named through the message catalogue; the
`ConnectionsPage`
fixtures move from `name:` to `key:` (the stand-in translator hands a
key back
as its text, so no query changed); the DOM snapshot shows the mark in
place of
`.pmtile`.

**Deliberately unchanged.** Adapter docstrings, log lines and class
names keep
"Mochat" (identifiers and developer text, not a name a reader is shown).
The
TUI draws no channel list, so it gets no icons. The refresh script needs
the
network and is run by hand; neither CI nor the build runs it.

**Weight.** Eleven files, about 66 KB, copied into `dist/assets` and the
wheel.

Nine commits: the feature, then one per finding of the pre-submit sweep.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Suites ran before the last commit, which changes only
`.pre-commit-config.yaml`
and was checked with pre-commit.

- `npx vitest run --no-file-parallelism` in `ui-web/` (with
`NODE_OPTIONS=--localstorage-file=<file>`, needed under node 26): 202
files,
2790 tests, 2788 passed, 2 failed. The two are the "when storage
refuses"
cases in `src/features/desk/palette.test.ts` and `store.test.ts`, which
fail at
every revision under this machine's node 26 and pass under CI's node 22;
this
  branch touches neither.
- `npm run gen:check`, `npm run lint` (0 errors, the 4 existing
warnings),
  `npm run type-check` in `ui-web/`: pass. `npm run type-check` and
  `node scripts/gen-i18n.mjs --check` in `ui-tui/`: pass.
- `npm run build`, `python3 ui-web/build.py`, `check-page.mjs`,
`check-css.mjs`,
  `check-class-namespace.mjs`: all OK.
- `uv run pytest -q` with all extras: 25778 passed, 114 skipped, 1
failed:

`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`,
which fails the same way on a checkout without this branch (this machine
has
  npm on four PATH entries, which defeats its no-npm case).
- `ruff check` and `ruff format --check` over the Makefile's lint
targets: clean.
`lint-imports`: 10 kept, 0 broken. `ty check` on the changed modules:
clean.
- `uv run --isolated --frozen --group docs mkdocs build -f
docs-site/mkdocs.yml --strict`:
  exit 0; both channels pages render MoChat.
- `pre-commit run --from-ref origin/main --to-ref HEAD`: every hook
passes except
prettier (ui-tui), which flags only
`src/__tests__/instanceModeCommand.test.ts`,
unchanged here (last touched by #669); CI's pre-commit job skips that
hook,
  having no `node_modules`.
- `scripts/check_commit_messages.py origin/main..HEAD` and commitlint:
pass.
- Red first and mutants: every new case failed before its
implementation.
Mutants, each reverted with the tree clean after: no `onError` fallback,
a
failure kept as a boolean, marks keyed by name, the asset digest
dropped, `qq`
dropped from the table, the English name set back to "Mochat",
`.channel-mark`
resized, and a slot-specific `.pmtile` box. Each turned the tests
written for it
red: 1, 1, 3, 1, 3 Python guards, the display-name guard, and the CSS
gate twice.
- Browser, headless Chrome over the built page with `?stub=1`: all
twelve rows
draw a 38x38 mark with its file loaded and the digest in its URL, and no
letter
tile is left. The letter tile measures the same 38x38 with an 11px
radius in
both slots. English reads Feishu, Email, WeChat, WhatsApp, Telegram,
Discord,
MoChat, WeCom, Slack, DingTalk, QQ, Matrix. After Settings > General >
Chinese,
MoChat shows its Chinese name and the six brands keep their Latin names.
  Matrix's mark follows the theme switch on a light OS.
- `raven channels list` from this tree: `mochat | MoChat`.
- `scripts/refresh_channel_marks.py` run twice: the second run
reproduced every
PNG byte for byte, and the two SVGs are `cmp`-identical to a separate
fetch.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible: the channels section shows app icons where it showed
letter tiles,
and MoChat's English name changes from its Chinese name (web UI) and
from
"Mochat" (CLI). The other eleven names read as before. No config, RPC or
stored
data changes; `ConnChannel.name` was internal to `ui-web`. Rollback is
reverting
the squash commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-09-23 14:11:10 +08:00
Ethan Wang
78e41b2f8d
feat(*): a playbook mode that takes many rounds (#499)
## Summary

A third playbook mode. A `mode: stint` playbook declares roles instead
of nodes, and the driver compiles one sub-agent graph per round, so a
stint is not a sibling of a `dag` run but a layer over it: one fan-out
is one graph, and many rounds are one graph each. Thirty rounds is
thirty graphs on the shared dispatch path, which is what makes the
validation, the dispatch budget, the approval gate and the detached-run
bookkeeping the ones already there rather than a second set.

The stint is a file. Nothing depends on the process that started it
still being alive: a gateway restart, a machine restart, or a `raven
playbook stints resume` all pick it up, and the resume is node-granular,
so a round that died with two roles done re-runs the third.

**One noun, singular and plural.** The mode is `mode: stint`; one run of
such a
playbook is a stint, with its own id, checkout and record; the plural is
the
command family and the RPC namespace. `round` keeps its own meaning as
the unit
inside -- one pass of every role -- and `stage` as one role's turn in
it, so
"the stint is on round 7 of 30" says three different things with three
different
words. The whole surface sits under `raven playbook`: `stints` answers
what has
run on this machine, `stint` acts on one project's layout and the
backlog its
roles share.

**Handover is files, not a conversation.** Every role opens a fresh
conversation each round, which is a decision rather than a limitation:
which instance serves which role changes, processes restart, transcripts
get compacted. What a role learned this round reaches the next one only
by being written down. That makes the journal the contract -- an
append-only file, windowed by recent rounds and by characters, because a
forty-round journal would otherwise fill the prompt it exists to inform.

**The boundary is three layers.** `owns` and `appends` are declared in
the playbook, rendered into the role's prompt from the same declaration
the enforcement reads, measured by the role's own checks, and undone
afterwards with the file the role wrote kept under `violations/`.
Declaring without telling is a trap; telling without undoing is a
suggestion; and the undo reads the stage's own commits as well as the
worktree, because a role that commits a stray write disappears from the
working tree and a boundary any role can step around by committing is
not one. Nothing refuses a write before it lands: a charter narrows a
role's tools only where the playbook declares a `playbook:` block, and
`owns` is not derived into one, so the shipped playbook's roles have no
tool gate and the undo is the boundary rather than its backstop.

Two shapes the boundary cannot hold are refused where they are written
rather than left to surface as violations in round four:

- **Two roles that can run at once where either is enforced.** They
share one checkout, so the first judged is graded against the difference
the whole tree shows and quarantines the other's work. Observed before
it was refused: two roles with disjoint `owns`, each writing only what
it owned, and the first judged came back as `dev-a wrote 1 path(s) it
may not write: src/b/work.py`, with that file gone. The message names
both ways out, because which one is right is the author's call. What
would make it work is a checkout per concurrent role, which is a merge
problem, not a validation rule.
- **`enforce.read: hard`.** It reaches the guard text, which then tells
the role that reads outside its paths are refused at the tool gate, and
no gate refuses one. The wording stays in `rounds_prompt` for whoever
wires the gate; until then the grade is not offered.

**Checks come before any model's opinion.** `verify[]` runs real
commands, and a failure goes back to the role that caused it with the
failure text, up to `maxHandbacks`. When that budget runs out the round
records the failure and carries on rather than failing the node: a
failed node skips every role downstream, and a failed build is exactly
what the reviewer downstream has to see.

**One approval, and it names what it is approving.** The whole stint is
confirmed once, at the start, and the text states the round budget and
every command the stint may run -- a playbook is a distribution unit, so
a file somebody handed you does not get to run shell thirty times on one
unexamined yes. Every round is charged against the hourly dispatch
budget like any graph; a round the budget turns down pauses the stint
rather than failing it, and `stints resume` takes it up once the
allowance recovers. From the CLI the same approval is put to the
terminal, and with no terminal to ask at the run is refused rather than
approved on nobody's behalf.

A stint works a checkout of its own, cut from the project's head, so
hours of its commits do not collide with the conversation that started
it. `stop.maxRounds` defaults to 10 and is capped at 99. `stop.until`
lets a role end a stint early by reporting a marker on a line of its own
-- a whole line, so a role quoting the marker mid-sentence does not end
the plan -- and only a terminal role is told the marker, because a role
that knows how to stop the stint has a reason to reach for it.

**Surfaces.** `raven playbook stints
list/get/extend/pause/stop/resume/sweep/answer`, where `resume` and
`extend` are relayed to the raven serving the page when one is up, so a
round runs in the process that owns the conversation;
`playbooks.stints.*` RPC methods, the read-only ones reading the record
off disk so opening the page does not cost what a dispatch costs; a
Stint tab on the playbook page showing the roles, what each owns, the
checks with their full commands, and what carries between rounds; and a
Runs tab listing every stint with its standing, how far into its budget
it is, each round's checks and what was undone, with pause, stop and
resume.

`raven playbook stint` lays a project out for one. The shipped playbook
is `rounds`: a planner, a builder and a verifier, on any project that
has a specification. `init` writes `.stint/`: a guard file per role, the
shared prose, and a link to the project's own specification, chosen by
two sieves -- the file name first, then how much of the document says
something is required, so an empty `SPEC.md` stub does not send thirty
rounds off a blank page. `task`, `ask` and `confirm` work the backlog
the roles share.

New package `raven/stint/` carries the parts a stint needs that are not
the playbook: the record, its checkout, the ownership grades, the
journal window and the check runner. `CONTEXT.md`, `CHANGELOG.md` and
the two `docs-site` repo-layout tables gain their rows; `README.md` is
not touched.

Three unrelated changes ride along. Two are the same shape -- a web
change that does not show:

- `index.html` went out with no caching header, so browsers fell back to
heuristic freshness and served a rebuilt page from cache without asking,
which reads exactly like a server that never picked the change up.
`no-cache` keeps the entry and revalidates; the ETag still saves the
transfer.
- `make ui` is one target for the two-step page build, because running
only vite leaves the served one-file page unchanged with nothing to say
so.

The third is `raven agent` (`raven/cli/agent_commands.py`):
`--message-file` reads the prompt from a file, `--permission-mode` picks
the tier for the run, and `--no-markdown` prints a soft-wrapped `Text`
rather than a markdown render; 92 lines of tests in
`tests/test_cli_agent_commands.py`.

Designed and deliberately not built, so a reviewer knows where the edges
are: a checkout per concurrent role with each role merging itself in
when it is judged (`docs/plans/2026-09-18-stint-acceptance.md`), and the
backlog-writing round zero. `raven playbook stint confirm` records a
person's word on the plan and nothing in the rounds path reads the flag
yet, which that document also says.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run ruff check --no-cache raven evolver agents plugins-dist tests scripts
uv run ruff format --check --no-cache raven evolver agents plugins-dist tests scripts
uv run lint-imports
make lint-types
make lint-ui
make lint-tui
uv run pytest -q
uv run python scripts/check_commit_messages.py main..HEAD
uv run python scripts/check_source_language.py main..HEAD
uv run python scripts/check_large_files.py main..HEAD
```

Ruff clean over 2,004 files. Import contracts: 10 kept, 0 broken. `ty`
clean, which caught one real defect on the way in: `StintStore.list`
shadowed the builtin for every annotation after it in the class body, so
`-> list[StintRecord]` on it and on `live()` resolved to the method. The
UI gates include the generated-file drift checks (`gen:check`,
`lint:rpc`, `lint:i18n`) and both type-checks; eslint reports 23
warnings and 0 errors, none of them in a file this branch touches. `make
lint-bridge` was not run: this checkout has no `bridge/node_modules`,
and nothing here touches `bridge/`.

Suite: the whole `uv run pytest -q` runs in CI as four shards, and every
shard is green at this head, with the import-cycle budget (44 modules,
the same as `main`), the coverage gate and the three content gates over
the commit range. Locally the suites this branch touches are run per
change; the full run is CI's.

Behaviour, driven rather than asserted:

- A stray write by a concurrent role was reproduced end to end before
the validation refused the shape: `dev-b's own file still there after
dev-a was judged: False`, `violations recorded against dev-a: ['dev-a
wrote 1 path(s) it may not write: src/b/work.py']`.
- The check runner's timeout kills the process group, not just the
direct child. Proved as a regression guard: red with `process.kill()`,
green with the group teardown.
- The round budget refuses rather than wrapping: extending a real stint
past the ceiling answered `that would give it a budget of 104, and 99 is
the most any stint may have`, and the file was not written.
- The spec picker's two sieves, over a real project: `KEPT docs/PRD.md |
weight 12`, `dropped SPEC.md | len 12 | weight 0`, `dropped notes.md |
len 868 | weight 0`.
- Only a terminal role is told the stop marker: `planner=False
builder=False verifier=True` on the shipped example.
- The shipped example is discoverable with no user layer: `listed:
['rounds']`, `origin: builtin`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Additive. `mode: stint` is a new value; `dag` and `prompt` playbooks
load, validate, dispatch and report exactly as before, and the sections
the new mode reads are withheld from the generator's emit schema, so no
generated playbook can reach them. That withholding is pinned by a test
that fails when a section is added to the spec without being added to
the refusal.

Nothing outside the new mode changes behaviour: `spawn` and the DAG path
keep the per-session in-memory dispatch window they have on `main`.

A plan runs shell from `verify[]`. It is gated on the approval, the
approval text lists every command, and a playbook carrying `verify`
cannot be generated -- only written by hand or shipped. The builtin
layer ships exactly one playbook, `rounds`, pinned by name in a test,
for that reason: withheld from the generator and not shipped either, the
mode would be reachable only by transcribing YAML out of prose.

Rollback is a revert of the merge commit. A plan's on-disk record is
under the sub-agent history root and is read by nothing else; nothing
else's on-disk shape changes.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Ethan Wang <ethan.wang@evermind.ai>
Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-23 12:24:08 +08:00
userName20260323
6099b8c75e
docs: refine native agent guidance (#647)
## Summary

Refine the bilingual Agent Integrations documentation so the
native-agent section is easier to scan. Keep the five-agent overview,
calling paths, minimal ACP example, and concise task templates together.
Move Raven-Code workspace and verification guidance under the
native-agent section, keep both the new and legacy anchors, clarify that
turn-local worker labels cannot be stored in Playbooks, and update the
builder guide references.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [ ] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Commands and results:

```text
PYTHONDONTWRITEBYTECODE=1 TMPDIR=/root/tmp uv run --frozen --group docs mkdocs build -f docs-site/mkdocs.yml --strict --site-dir /root/tmp/raven-docs-site-fix
Passed: bilingual strict build.

PYTHONDONTWRITEBYTECODE=1 TMPDIR=/root/tmp uv run --frozen --group docs pytest -p no:cacheprovider tests/test_docs_site.py tests/test_living_docs.py tests/integration/test_docs_site_controls_e2e.py -q -o addopts=''
Passed: 77 tests.

git diff --check origin/main...HEAD
Passed.

uv run python scripts/check_commit_messages.py origin/main..HEAD
Passed.

make check-large-files
Passed.
```

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

This is a documentation-only change. The existing Raven-Code deep link
is preserved, and reverting the commit restores the previous
documentation structure.

## Related Issues

N/A

---------

Co-authored-by: userName20260323 <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Codex (gpt-6-astra) <noreply@openai.com>
2026-09-22 19:25:31 +08:00
userName20260323
8a1ebdc577
docs: organize bilingual documentation navigation (#635)
## Summary

Reorganize the Raven documentation landing page and expand the bilingual
documentation site.

- Add bilingual guidance for agent collaboration, ACP and A2A protocols,
DAG orchestration, channels, browser collaboration, knowledge bases,
Playbooks, proactive follow-ups, Oncall, permissions, tracing,
trajectory replay, and Evolver.
- Group daily usage under `Use Raven` and `Using Raven`.
- Group `Technical Reference` and `Developer Guide` under `Explore the
documentation` while preserving separate cards.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [ ] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Exact checks:

- `uv run --frozen --group docs mkdocs build -f docs-site/mkdocs.yml
--strict` - passed.
- `uv run --frozen --group docs pytest tests/test_docs_site.py
tests/test_living_docs.py
tests/integration/test_docs_site_controls_e2e.py -q -o addopts=''` - 77
passed.
- `uv run --frozen python scripts/check_commit_messages.py
origin/main..HEAD` - passed.
- `git diff --check origin/main...HEAD` - passed.
- `make check-large-files` - passed.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

This is a documentation-only change. It does not change runtime
behavior, configuration semantics, or public APIs. Reverting the
documentation commits restores the previous site content and navigation.

## Related Issues

N/A

---------

Co-authored-by: userName20260323 <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Claude (claude) <noreply@anthropic.com>
2026-09-22 16:14:44 +08:00
userName20260323
72ee555972
fix: restore missing overview card icons
Restore the missing icon mappings for the overview cards in both locales.
2026-09-21 12:59:46 +08:00
userName20260323
e708f34a91
docs: align bilingual documentation with runtime behavior (#551)
## Summary

- Align English and Chinese documentation for Docker/Compose, sandbox
execution, and the in-tree tracing API with verified runtime behavior.
- Separate proactivity usage guidance from developer design and
implementation details, with configuration, troubleshooting, cost, and
safety boundaries.
- Update navigation and remove redundant section separators. Keep page
URLs unchanged.
- Preserve published proactivity section and title anchors in both
languages. Use native aliases for same-page content and visible
migration links for moved sections, with regression tests.
- Clarify Sentinel delivery fallback and menu-selection requirements,
tracing-on/off argument handling, headless CLI menu output, and viewer
descriptor lookup. Add focused regression coverage without changing
runtime behavior.
- Clarify the initial frozen baseline and improve Evolver wording in the
bilingual site and repository mapping. Keep its structure, navigation,
implementation terms, and technical claims unchanged; broader Evolver
documentation work is excluded.
- No runtime changes.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Passed on the rebased revision:

```bash
uv run --frozen --no-sync pytest tests/test_docs_site.py tests/test_tracing_api.py tests/test_sandbox_unit.py tests/test_sentinel_runner.py tests/test_sentinel_planner.py tests/test_sentinel_fast_path.py tests/test_nudge_policy.py tests/test_proactive_spawn.py tests/test_core_sentinel_stack.py tests/test_core_cron_stack_ledger.py tests/test_cli_sentinel_commands.py tests/test_decision_router.py -q -n 0 -x
# 395 passed

uv run --frozen --no-sync --with 'mkdocs-material>=9.5' --with 'mkdocs-static-i18n>=1.0' --with 'mkdocs-git-revision-date-localized-plugin>=1.6.0' --with 'jieba>=0.42.1' mkdocs build -f docs-site/mkdocs.yml --strict --site-dir /tmp/raven-pr551-push.9ZfOEi/site
# Strict English and Chinese builds passed

UV_NO_SYNC=1 make lint-python PYTHON_LINT_TARGETS='tests/test_docs_site.py tests/test_tracing_api.py tests/test_decision_router.py tests/test_cli_sentinel_commands.py'
UV_NO_SYNC=1 make check-large-files check-source-language COMMIT_RANGE=origin/main..HEAD
PYTHONPATH=. uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
npm exec --yes --package=@commitlint/cli@21.1.0 -- commitlint --from origin/main --to HEAD --config commitlint.config.cjs
git diff --check origin/main...HEAD
# All passed
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Documentation checks also covered bilingual examples and the proactivity
JSON configuration schema. A built-HTML audit verified all 110
historical proactivity section fragments and four title fragments,
including same-language migration destinations, and found zero broken
internal fragment links across 29 HTML pages (672 links). Replaying
pre-fix content made all four page compatibility cases fail as expected.
The new descriptor-directory test failed on all three documents before
their stale paths were corrected. Added coverage exercises tracing
on/off, bare-number versus explicit menu selection with incomplete
classifier configuration, and discovery-menu dispatch through the real
CLI stack's headless sink. Runtime type checks and the full repository
test suite were not run; no runtime code changes. No browser click test,
live LLM, real-VM, or container end-to-end run was performed.

## Risk

Documentation-only, with regression tests. Proactivity navigation
changes, but page URLs and historical heading fragments are preserved.
An old link to content moved between pages lands on a visible migration
entry and requires one click to reach the new section; no JavaScript
redirect is needed. Existing queued-nudge policy and external-agent
isolation limitations are described, not fixed. Roll back by reverting
the documentation and associated test changes; no data migration is
involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: zhao.wang <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-20 19:20:27 +08:00
Dizhan Xue
7faa2060e2
feat(*): raven speaks Agent2Agent 1.0, inbound and outbound (#446)
## Summary

Raven speaks the Agent2Agent protocol, version 1.0, in both directions.
The host
agent is the peer: nothing here makes a sub-agent addressable from
outside, and a
raven launched as a sub-agent refuses to serve the face at all, because
it is
already reached over ACP by the host that started it.

Inbound is an aiohttp face carrying the two cards and a JSON-RPC
endpoint. It
mounts on the gateway when `a2a.server.enabled` is set, and `raven a2a
serve` runs
it standalone for a headless deployment, where running the command is
itself the
opt-in. Outbound is a single tool, `a2a_send`, that takes a card URL and
a message;
it enters the tool list only when `a2a.peers` is non-empty, so a host
with nowhere
to send a message does not carry the schema. The two switches are
deliberately
independent: calling out and being called are separate deployments.

Onboarding mints `a2a.server.token` and stops there. The face ships
closed:
finishing an unrelated install is not consent to open a second network
surface, so
`raven a2a enable` is what opens it, and it is the only writer of that
flag.
Enabling mints a missing token in the same command, because a face
switched on with
an empty token answers every caller with a 401 and reads as a broken
deployment
rather than the closed door it is. `raven a2a disable` closes it again
and keeps the
token, so reopening does not re-issue a credential callers already hold.
The wizard
never writes the flag at all, not even as false, so a repeat onboard
leaves a face
the operator opened by hand alone. A config nobody onboarded, and any
`A2aConfig()`
assembled in process, stays inert.

What that opens is narrower than a network face. The app carrying it
binds loopback
at both call sites, hardcoded rather than read from config, and every
request is
checked against a freshly minted 256-bit bearer token. The minting write
is also
what fixes `config.json` to owner-only, from the temporary file's first
byte rather
than with a chmod afterwards; a later enable or disable adds no secret
material, so
it leaves whatever mode the operator chose. Nothing else narrows that
file and it
already held provider API keys, so the obligation predates this work and
is merely
discharged here.

There are two cards because the protocol splits one description across
two
authentication states. The public card answers an unauthenticated GET,
and has to:
a caller reads it to learn which scheme to authenticate with, so
requiring the
credential first would be circular. It therefore names no installed
agent. The
extended card is a `GetExtendedAgentCard` RPC riding the same bearer
check as every
other method, and is the only one of the two that may say what this host
can
actually do.

On that extended card the sub-agent roster rides
`capabilities.extensions`, not
`skills`. A skill is the protocol's word for something an agent can be
asked to do,
and a peer cannot ask for one named sub-agent: it can only send a
message to this
host, which then decides what to dispatch. One skill per sub-agent would
advertise
call targets that do not exist. What a peer can genuinely request is the
orchestration, so that is a single skill, and the roster moves to the
protocol's
own extension point under a versioned URI. That is the only place a
conformant card
may carry a payload the spec does not define, because `AgentCard` is a
closed set
of fourteen protobuf fields: a custom top-level key is rejected by a
strict parser
and silently dropped by a lenient one. The entry declares `required`
false, so a
reader that has never heard of the URI ignores it and still talks to
this host.

Both cards derive `supportedInterfaces[].url` from the origin the caller
arrived
on. One face answers under every name that routes to it, and a URL fixed
when the
face is mounted is right for at most one of them. This matters more than
it looks:
a caller that finds an interface off the card's own origin must refuse
it, since a
card fetched from a trusted origin would otherwise redirect the
credential
elsewhere, and a relative URL reads as off-origin to that check.

## Type

- [x] Feature

## Verification

- `make test-python` at this head: 23729 passed, 112 skipped, 1 failed,
in 288 s.
  The one failure is

`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`,
which fails with the same assertion on `origin/main` by itself; this
branch changes
  no install or node file.
- `uv run --frozen --all-extras pytest
tests/integration/test_a2a_protocol_e2e.py -q`:
6 passed. That directory sits outside the default pytest scope, so it is
run by
  hand rather than reached by the line above.
- `make lint-imports`: 10 contracts kept, 0 broken. `make lint-types`:
all checks
  passed.
- `uv run ruff check raven/ tests/`: clean. `ruff format --check`:
clean.
- `make check-commits` over all 68 commits, `make
check-source-language`,
  `make check-large-files`: all exit 0.
- Live, over a real network boundary rather than in process: a raven in
a Docker
container and a raven on the host, each running this branch, called each
other in
both directions. Nineteen assertions per direction covering both cards,
the four
refusal paths, and a completed turn. The container publishes its face on
a
different port than it binds, so the card has to name the port the
caller used and
not the socket the server opened; both cards did, in both directions.
That run
predates the default-off change described above, which touches no file
under
`raven/a2a/` or `raven/a2a_client/`: both packages are byte-identical to
the head
  it ran against.
- Reverting each fix turns its own tests red, checked one fix at a time
rather than
all together, including the interface-URL fix mutated out inside the
running
  container.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Behaviour changes for an operator who runs `raven onboard` after this
lands: a
token is minted into `a2a.server.token`, and `config.json` is narrowed
to
owner-only. No listening face is opened. Opening one takes `raven a2a
enable` and a
gateway restart, so the second network surface only ever exists because
somebody
typed for it.

Rolling an opened face back is `raven a2a disable` and a restart. The
token stays in
place and is never rotated, so callers already holding it keep working
when the face
is reopened, and a later onboard will not reopen it by itself. Nothing
else in the
install changes.

Costs and limits, stated rather than left to be discovered:

- A gateway that serves A2A imports `a2a-sdk`, measured at roughly 570
ms and 391
modules on top of the gateway's own stack, sqlalchemy and the
OpenTelemetry api
among them. It is paid once at mount, not per request, and an operator
who
switches the face off pays none of it. Moving it to the first request is
recorded
  as deferred in the design note rather than done here.
- `a2a_send` adds 157 tokens to the tool array of every turn, measured
off a
  captured provider call, and only on a host with a peer configured.
- The task store is in memory and unbounded, and inbound concurrency is
not capped.
A local face behind a bearer token is the deployment this is written
for; a
  host exposed more widely should bound both first.
- Pulling in `a2a-sdk` brings the OpenTelemetry api into the gateway's
import
graph. This is a trade accepted for the SDK's protocol types and
handler, not an
  adoption of OpenTelemetry by this project.

Checked during review and deliberately not fixed here: config validation
accepts an
agent row with an empty or whitespace name, which would reach the
extended card's
roster as an empty entry. It is not introduced by this work, the
previous shape
carried the same value, and closing it belongs with the agent schema
rather than
with the card.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-20 14:58:15 +08:00
Dizhan Xue
3837fa3d03
fix(docs): seat the search dialog's clear button on its own centre line (#521)
## Summary

The search dialog stacked two controls in one corner and seated neither
on
the line the query is typed on.

The theme places its clear button at a fixed distance from the form's
top,
an offset that centres the button only in a form of the theme's own
height.
This dialog's form is taller, with two consequences from the one cause:
the
button sat 10px above the magnifier and the text beside it, and it
landed in
the same corner as the chip naming the key that opens search,
overlapping it
by 21px.

The button is centred against the form box and against its own container
now, so it follows whatever height the form takes rather than assuming
one.
The chip is confined to the closed state, where it is still true: a
reader
already inside the dialog has nothing to press that key for, and the
field
left in the rail keeps its chip and is unchanged in both states.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Clicking always reached the button, overlap and all, so a guard written
around the click would have passed while the fault was plainly visible
in a
screenshot. The guard measures the boxes instead: the chip and the
button
must not intersect, and the button's centre must be within a pixel of
the
magnifier's. Run against the unchanged site it reports a 21px overlap
and a
10px drift, which is how the fault was confirmed before it was fixed.

Measured in a real browser at 1560x820, vertical centres in the open
dialog:

| element | before | after |
| --- | --- | --- |
| form box | 195 | 195 |
| magnifier | 195 | 195 |
| query text | 194.5 | 194.5 |
| clear button | 185 | 194.5 |

- `uv run pytest tests/integration/test_docs_site_controls_e2e.py
tests/integration/test_docs_site_search_e2e.py` -> 30 passed
- `make lint-python`, `make lint-imports`, `make lint-deps`, `make
lint-types`
  -> exit 0, against an environment synced the way CI installs
- `pre-commit run --from-ref origin/main --to-ref HEAD`, `make
check-commits`,
  `make check-source-language`, `make check-large-files` -> exit 0
- `mkdocs build --strict` -> built
- captured the rail at rest, the rail while the dialog is open, and the
  dialog itself: the rail is identical in both states and the dialog's
  corner holds one control

## Risk

User-visible and confined to the open search dialog above the desktop
breakpoint: the clear button moves down onto the centre line, and the
key
chip no longer appears while the dialog is open. Narrow viewports keep
the
theme's own full-screen search untouched.

Rollback: revert the commit. The change is two rules in one stylesheet,
and
either can be dropped on its own.

## Related Issues

N/A

Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-20 10:09:22 +08:00
Dizhan Xue
15cadbd80b
fix(docs): make the Chinese pages searchable by word (#518)
## Summary

Chinese runs no spaces, so the indexed text carried no word boundary at
all.
The search separator broke on whitespace and hyphens, which left each
Chinese
run as a single token, and a reader who typed a word rather than a whole
heading found nothing.

Two things had to line up, and neither works without the other:

- the build segments the text with jieba, which marks every boundary
with a
  zero-width space. jieba was reaching the build only as a transitive
dependency of another group, so it was absent wherever the docs group
was
installed on its own, which is every build that ships. It is declared
now.
- the separator breaks on that mark. The whitespace class the theme
defaults
  to does not cover it, so the marks stayed buried inside the token.

The guard for this passed while the site was broken, which is worth
saying
plainly. It took its query from the page's own heading, boundary marks
and
all, and a query carrying those marks matches the stored token exactly.
No
reader can type that string. The query strips the marks now, and a
second
guard searches a word taken from the middle of a title, which can only
be
found if the run was indexed as words rather than kept whole.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Measured on a site built inside an environment holding only the docs
group,
served locally and driven in a real browser, searching the Chinese site.
Repeated three times with identical counts:

| query | before | after |
| --- | --- | --- |
| a word from the middle of a heading | 0 | 36 |
| a single-word heading | 5 | 12 |
| another single-word heading | 5 | 10 |
| a compound the segmenter splits | 5 | 0 |

The same measurement against the published site reproduced the "before"
column exactly, which is what says the local environment matches the one
that builds it.

- `uv run pytest tests/integration/test_docs_site_search_e2e.py
tests/integration/test_docs_site_controls_e2e.py
tests/test_docs_site.py` -> 34 passed
- both new guards were mutation-tested: with the separator reverted, the
build guard reports that the separator does not break on the boundary
and
  the browser guard finds nothing for its word
- `make lint-python`, `make lint-imports`, `make lint-deps`, `make
lint-types`
-> exit 0, run against an environment synced the way CI installs, not
the
  working one
- `make check-commits`, `make check-source-language`, `make
check-large-files`,
  and `pre-commit run --from-ref origin/main --to-ref HEAD` -> exit 0
- `mkdocs build --strict` -> built, 146 of 146 Chinese entries carry
word
  boundaries

## Risk

User-visible: Chinese search returns results for ordinary words where it
returned none. One case gets worse: a compound the segmenter splits into
parts is no longer found, because the theme does not segment the query,
so a
run that is not itself a word no longer matches any token. A custom
segmenter dictionary would recover those; it is not in this change.

Rollback: revert the commit. The two halves are one line of mkdocs.yml
and
one dependency entry, and dropping either one restores the previous
behaviour on its own.

## Related Issues

N/A

Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-19 23:40:58 +08:00
Dizhan Xue
f10ac9f116
fix(docs): give each language its own search index and settle the site shell (#516)
## Summary

The documentation site built one search index for both languages, so a
reader on
the English pages was offered Chinese ones: results they cannot read,
behind
links that leave the language they chose. A build hook now splits the
merged
index in two and points the Chinese pages at their own copy, which the
theme
also resolves every result link against.

Where that split runs is the whole fix. The merge lands in a plugin's
own
`on_post_build`, so the split has to be the last thing that event does.
Running
it on `on_shutdown` looks equivalent and is not: `mkdocs serve` never
shuts
down, so the preview went on serving the merged index however often it
rebuilt.
The new guard makes the same assertion against `mkdocs build` and
against
`mkdocs serve`, because passing one says nothing about the other.

The rest settles the shell against the reference layout:

- the rail's foot is its own box the navigation cannot show through, and
the tab
  icon is the EverMind mark;
- the footer keeps to the content column instead of covering the fixed
columns
  once the reader reaches the page foot;
- the contents column reaches the screen foot with its scrollbar hidden
behind a
  fade, and follows the reader by centring the first lit entry;
- search opens as a centred dialog over the page. The form itself
travels into
that dialog, so what stays in the rail is drawn from the field's own
metrics
and its shape does not change on a click. Neither state animates: the
theme
transitions the two shapes into each other, and leaving search played
that out
  in the corner of the page after the reader had moved on.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/integration/test_docs_site_search_e2e.py` -> 2
passed.
Without this change the preview case fails: the Chinese index is HTTP
404 and
  the English index carries 146 Chinese pages.
- `uv run pytest tests/integration/test_docs_site_controls_e2e.py` -> 25
passed
- `uv run pytest tests/test_docs_site.py
tests/test_source_language_check.py` ->
  31 passed
- `make check-commits`, `make check-source-language`, `make
check-large-files` ->
  exit 0; `ruff check` on the added files -> no findings
- `make docs-build` -> the root index holds 149 entries and no Chinese
page; the
  Chinese index holds 146, none of them addressed from the site root

Driven in a real browser at 1560x820 in both languages: a word that
appears in
both trees returns 10 results and 0 from the other language, and the
corner of
the page is pixel-identical to its never-opened state at every frame
after
leaving search. Every page of the built site was checked at the page
foot (28
pages, 0 faults) for a footer covering a column, for a column whose last
entry
cannot be reached, and for the rail's foot icons being clickable.

No page content changed, so nothing else needed updating with it.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible: search no longer returns pages from the other language,
and the
search control, the footer and the two fixed columns change shape. The
field
left in the rail while the dialog is open is a drawn replica, and every
rule
behind it sits above the theme's desktop breakpoint, so narrow viewports
keep
the theme's own full-screen search untouched.

Rollback: revert the commit. The hook is one file, and dropping it from
the
`hooks:` list in `docs-site/mkdocs.yml` restores the previous merged
index on
its own.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-19 21:22:24 +08:00