282 Commits

Author SHA1 Message Date
Dizhan Xue
1b9cbf85e5
feat(*): let a dag decision node choose which of its branches run (#1001)
## Summary

A sub-agent DAG node can now decide which of its downstream branches
run. Before this, a background run could not turn on a result: every
node ran, and choosing a path meant the host winding the run down and
starting another.

How it works:

- A node that declares `routes` (route name to condition) is a decision
node; `select` is `"one"` (the default) or `"many"`. Raven's per-node
judge, the existing verdict call, picks the route or routes the node's
answer satisfies. The node's own text never selects a route. A route
that cannot be determined fails the node, with no fallback route.
- A downstream node carries `when`, a guard over `<node>.<route>` and
`<node>.done` atoms joined with `not`, `and`, `or` and parentheses,
evaluated with three-valued logic: a route of a decision node that was
never executed is unknown. Every decision node in a node's `depends_on`
must appear in its `when`; such graphs are refused at submission
otherwise, along with guards that can never hold and a route named
`done` in any letter case. Validation keeps negation parity and the
surrounding boolean expression: a double-negated `done` and a
disjunction with a satisfiable arm are accepted, without expanding
disjunctions into combinations.
- A node that does not run is `not_executed`, with a reason:
`when_false`, `when_unknown` or `upstream_not_executed`. A node whose
dependencies all went unexecuted is not executed either; a failed or
skipped dependency still skips its dependents first.
- A merge node reads a missing upstream as a fixed sentence, `<This task
is not executed and the output is missing, please go on without this
output.>`, written outside every untrusted-content fence, plus a note
listing the inputs that may be missing. A file reference to the files of
a node that may not execute is refused at submission however its path is
spelled: `@nodes/`, an absolute path into the node root, or a workdir
path that lands there, symlinks followed. Windows case, separator and
drive-relative aliases are recognized, and the advice keeps the declared
node ID.
- Node judgement is always on: the `subagentDag.verdictEnabled` switch
is retired. Config migration 14 removes it with a notice, and drops it
again on every later load should an older build sharing the config write
it back. `raven playbook run` now judges its nodes too, on the same
`subagentDag` settings and a pinned `verdictModel`'s own provider; with
nobody there to answer, a node judged not to have accomplished its task
fails on the judge's reason. A host with no judge at all refuses graphs
that declare routes.

Where it shows:

- Run events, the manifest, `dag.get`, `tasks.list` and
`dag.run_completed` carry `reason`, `selected_routes` and
`route_evidence` per node, and `routes`, `select` and `when` from the
graph. The instance registry's DAG-node rows (and `InstanceRow`) carry
the outcome too, so a client that reloads before the run finalizes shows
what a branch decided. `TaskCounts.not_executed` is a new counter.
- TUI: a not-executed node shows a hollow marker and its reason in
words; a decision node shows its selection on the node's second line,
and the guard and route evidence in the detail and in `/dag <node>`. The
Agents Overlay and the live agents strip leave not-executed nodes out,
as the backend's agent lists do.
- Web: the desk's tasks tab names not-executed steps and shows the same
fields.
- Playbooks pass the three fields through, namespace guard atoms with
the run prefix, and keep them through compilation.
- The `run_subagent_dag` schema, the DAG orchestration skill and the
docs site (English and Chinese) describe routes, guards and the merge
rule; CHANGELOG has the entries.

A graph with no `routes` and no `when` behaves exactly as before: the
same judge requests, events, records, summary and finish text.
`route_evidence` is a quote the judge picks from the sub-agent's output;
it is cut at 500 characters, keeps a lone surrogate from a malformed
escape as the escape's own text, and never enters model-facing text
(finish text, `dag_status`, exception reports). When a judge answer does
not report a usable route, the judge is asked again at most twice, each
time with one correction. The shared instance registry drops its cache
when a write fails on the data rather than the disk, so a row the file
cannot hold fails its own write and no other.

Two Windows CI steps run the alias and finish-text regressions; the
latter exposes the installed Git `cat` executable required by the
existing fixture.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Linux, Python 3.12.13, on this revision rebased onto the current main
(`$CHECKS` is a temporary evidence directory):

```sh
uv run --frozen --no-sync pytest -q -n 4
# 23 failed, 33229 passed, 162 skipped, 1 xfailed, 3 errors
uv run --frozen --no-sync pytest $(git diff --name-only origin/main...HEAD -- tests/) -q -n 4 --idle-ceiling-strict --cov=raven --cov-branch --cov-report=json:$CHECKS/coverage.json
# the 20 test files this branch changes: 2622 passed, 1 skipped (a POSIX permission case that does not apply to root)
uv run --frozen --no-sync python scripts/coverage_gate.py --coverage-json "$CHECKS/coverage.json" diff --base-ref origin/main --head HEAD --threshold 90
# Diff coverage: 99.48% (764/768 executable changed lines), passed the 90.00% threshold
```

The full run's 23 failures and 3 errors are 25 tests. 24 of them fail
the same way on the current main in this environment: the venv lacks
`nltk` and `datrie` (16), a proxy is set in the environment (5), this
box has its own `npm` in `/usr/bin` (1 install-script case), LibreOffice
writes no `.xls` here (1), and the run is root (1). The 25th,
`test_a_cli_run_left_on_the_machine_when_this_computer_stops_is_picked_up_by_the_next_start`,
failed with its teardown in the full run and in one of two runs of its
file, and passed in five runs on its own: it reads the remote-run record
the moment its pid file appears, a race in code this branch does not
change (#840 makes that test wait for both). The four uncovered changed
lines are the gateway's `_verdict_provider` body, whose helper is tested
on its own; a guard atom over a node that did not complete, which the
runner never evaluates because it reads a guard only once every
dependency completed or was not executed; and the cut of an over-long
terminal output.

Ruff check and format, `ty check` over the changed modules, the import
contracts, `deptry raven`, commitlint, and
`scripts/check_commit_messages.py`, `check_large_files.py` and
`check_source_language.py` over this branch's commits pass. ui-web:
`vitest run src/features/tasks/BoardCard.test.tsx scripts/gates` passes
50 files and 244 tests, and `tsc --noEmit` is clean. In Chromium a
not-executed title whose first word is wider than the room beside its
status now breaks that word instead of starting under a blank line, and
every other tasks-board card renders pixel-identical, English and
Chinese, light and dark. Firefox still starts such a title under a blank
line and cuts it without an ellipsis. CI's `TUI checks` covers ui-tui.

Mutation checks: removing any one of these makes its own test fail: the
every-load strip of the retired switch, the quote's escape before the
cut, the registry's cache drop, the node-root check in the tool, the
runner and the replan path, the physical path through a symlink, the
CLI's judge, its settings and its no-answer failure, and the pinned
judge provider. A model reading only the DAG skill's catch-all paragraph
answers that without a catch-all route the judge may pick the nearest
route or fail the node; an earlier wording led it to answer that the
graph always stops.

A live run on `openrouter/z-ai/glm-5.3-flash`, two graphs with synthetic
messages, selected the expected route at each decision node, left the
unselected branch `not_executed` with `when_false`, and put the fixed
sentence outside the fences. Earlier revisions of this change passed the
native Windows selections (aliases, finish text, guards; Windows build
26300, Python 3.13.12). The final revision has not been rerun on native
Windows or macOS.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Security: sub-agent text stays fenced or out of model-facing text on
every new path. The routes block in the judge request sits after the
last untrusted fence and is labelled as the graph author's; the
missing-output sentence is written outside every fence; `route_evidence`
is bounded and reaches records, events, the registry and the UIs only.
In the TUI, route evidence is drawn through the same path as a node's
error text; neither strips terminal control sequences, which this PR
leaves as it is.

Behaviour changes: a config that set `subagentDag.verdictEnabled: false`
now gets judged nodes; migration 14 removes the key and says so once.
`raven playbook run` judges every node, one more model call per node,
and a node judged not to have accomplished its task fails there instead
of completing. New wire fields are optional except
`TaskCounts.not_executed`, which is always present (0 when nothing
branched). `InstanceRow` gains three optional fields, and a decision
node's registry row now also stores its selection and up to 500
characters of route evidence.

Conditional-artifact validation matches node IDs using the session
namespace's case-insensitive identity on every host, including macOS.
This conservatively rejects case-only spellings even on case-sensitive
filesystems. Separator and drive normalization remain native, and
unconditional file resolution keeps its existing behaviour.

Rollback: revert the squash commit. A config already stamped 14 loads
unchanged on the previous build, since the stamp only gates migrations;
a user who had set `verdictEnabled: false` can set it again there, and a
later build drops it again on load.

## Related Issues

Related, found while building this and not changed here: #998, #999,
#1000. Follow-ups from its review, not changed here: #1008.
2026-10-19 15:17:04 +08:00
Dizhan Xue
ccf1e2d08e
feat(*): let a failed turn's row open to its whole error (#978)
## Summary

A failed turn's Web row can expand to the provider's full diagnostic
text, up to 8,000 characters, while the existing bounded reason stays
unchanged. The optional `full_detail` field travels through the runner,
lane, RPC event and session marker, so live display and replay expose
the same account. Copy includes the expanded account. Quiet runtime
notices use "Show details" instead of the error label.

Failure details and call-record bodies remove known credential shapes
before the final cap. An unterminated private-key block is removed
through the end of the input, including when an earlier scan or call
record cut away its closing armour. Known cut boundaries also remove
incomplete JWTs, vendor tokens, URL passwords and labelled credential
tails; each of those patterns is searched only from behind the run of
its own value's characters that ends the input and the label in front of
it, so the scan stays linear whatever the text holds and still reads a
label gap of any width. The authorization and JWT patterns avoid
repeated backtracking, and the turn path and outer runner reuse the
computed detail. Attachment and lane reads use the base exception's
instance storage, so a foreign read-only property or throwing getter
cannot replace the scrubbed value or swallow the terminal failure event.

The optional fields are mirrored in the RPC schema and both generated
clients. The design document separates this delivery from the subsequent
Turn Continuation feature; Retry still starts an ordinary turn.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

The automated suites and static/build checks below were re-run after
synchronizing with main, using the existing local dependencies. The
independent behavioral and timing probes were measured before that
synchronization. The last three commits change only
`raven/security/redact.py`, two test files and the design document; on
that final tree the Python suite, the ui-web tests, the type, generation
and lint checks, `ty`, the import contracts, the range gates and
pre-commit were run again, and the ui-web build checks were not, since
none of their inputs changed.

- `uv run --no-sync pytest -q tests/test_agent_loop_failure_text.py
tests/test_llm_call_record.py tests/test_agent_loop_broken_turn.py
tests/test_agent_loop_llm_error_patience.py
tests/test_agent_loop_run_emit.py tests/test_spine_scheduler_lane.py
tests/test_spine_events.py tests/test_rpc_spine.py
tests/test_rpc_schema_match.py tests/test_kernel_budget.py
tests/test_acp_redact.py tests/test_security_untrusted_context.py -x
--tb=short --show-capture=no`: 904 passed.
- From `ui-web`: `npm test -- --no-file-parallelism
src/features/transcript/TranscriptPage.test.tsx
src/state/session/pipeline.events.test.ts
scripts/gates/i18n-keys.test.mjs`: 225 passed.
- From `ui-web`: `npm run type-check`, `npm run gen:check`, `node
../ui-tui/scripts/gen-i18n.mjs --check`, `node
../ui-tui/scripts/gen-rpc-types.mjs --check`, and `node
node_modules/eslint/bin/eslint.js
src/features/transcript/TranscriptPage.tsx
src/features/transcript/TranscriptPage.test.tsx`: passed.
- `uv run --no-sync lint-imports`: 10 contracts kept. `uv run --no-sync
ty check raven evolver agents plugins-dist scripts`: passed. Ruff check
and format check passed on the 20 Python files this PR changes.
- From `ui-web`: `npm run build`, `uv run --no-sync python build.py`,
`node scripts/check-page.mjs`, `node scripts/check-css.mjs`, and `node
scripts/check-class-namespace.mjs`: passed, including both boot
snapshots.
- `UV_NO_SYNC=1 make check-large-files check-source-language
COMMIT_RANGE=origin/main..HEAD`: passed. `PYTHONPATH=. uv run --no-sync
python scripts/check_commit_messages.py origin/main..HEAD`, commitlint
over the same range and `git diff --check origin/main...HEAD`: passed.
- `uv run --no-sync pre-commit run --from-ref origin/main --to-ref
HEAD`: all applicable hooks passed except the whole-TUI Prettier hook,
which reports `newInstancePicker.tsx` and `subagentsHub.tsx`. Both files
and the formatter config are unchanged from main; an isolated main-file
control with the same locked Prettier version reproduces both failures.
`node ui-tui/node_modules/prettier/bin/prettier.cjs --check
ui-tui/src/i18n/messages.generated.ts`: passed.
- Regression coverage includes truncated private keys, incomplete JWT
and credential tails, terminal-event delivery with foreign exception
attributes, scan reuse and CPU cost, the cut-tail scan's cost on
label-shaped runs and long gaps, its agreement with a whole-text search,
a label gap of any width, the JWT boundary and private-key END rules,
and quiet-notice expansion. The five independent boundary probes passed
before the base synchronization; the committed regression suite above
passed again afterwards.
- On the same 32,267-character synthetic crash input, the queued
event-loop callback delay fell from 15.351 seconds to 0.00951 seconds.
The regression checks use thread CPU time with a 0.5-second ceiling.
- Cut-tail scan: a failed turn whose crash text holds 31,000 spaces
before its last two words took 32.6 seconds while the region came from a
regex search for the last whitespace, and 0.016 seconds now; 64,000
characters of `pwd=` ending in a comma took 10.8 seconds through the
whole-text tail search and 0.003 seconds now, and a 300,000-character
call-record body of either shape takes under 0.3 seconds. These and the
figures below were measured with the machine's load average between 36
and 132. Across 3,728 timed texts, every label or prefix unit repeated
to 64,000 characters behind a cut and to 300,000 past the scan cap, the
slowest measured alone took about 0.2 seconds against about 0.1 seconds
on the base revision, and a text past the cap takes the same time at
twice the size. Searching the whole text again turns six timing cases
red; a smaller name room, a skipped gap or scheme step, or any value
class narrowed below its pattern's turns the property or gap test red,
as does removing the JWT boundary or restoring the END-only private-key
rule. On 12,008 generated texts with gaps up to 5,000 characters, the
output equals the whole-text version's.
- Chromium with synthetic local data: full text retained and HTML
escaped, expanded tooltip removed, internal scrolling and Retry
preserved, neutral notice label displayed, no page errors. No live
vendor call or native Windows rerun is claimed for these fixes.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered: the long text is scrubbed of
recognized credential shapes before persistence and publication. It
remains diagnostic text and may contain hostnames or paths. Model
history and chat-channel replies keep their existing bounded content. A
session export can include the long text. `raven.security.redact` is
also the scrubber for every ACP surface (tool titles, result previews,
permission prompts, replay, error messages) and for the outbound
client's permission log, so two of its rules reach those surfaces too: a
private-key header without END hides the rest of that text, and a JWT
glued to a preceding word character, such as `/verify-eyJ...`, is left
unredacted. Tests pin both.
- [x] Backward compatibility considered: the new fields are optional,
older markers fall back to their bounded reason, and the TUI can ignore
the extra field. Incomplete private-key blocks are conservatively
removed to the end, which can also omit trailing diagnostic text. Custom
runners attach long details in the exception's instance dictionary;
descriptors are ignored.
- [x] Rollback path is clear for risky changes: revert the squash
commit. Older code ignores the optional marker field.

## Related Issues

N/A

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-17 14:28:35 +08:00
zhanghui
16784d3a1e
build(*): upgrade everos and the memory plugin to 1.4.2 (#950)
## Summary

EverOS 1.4.2 shipped today. It fixes what a memory server that stays up
for
days runs into: a LanceDB index reader leaked file descriptors until
searches
took seconds, an interactive search queued behind the cascade's
background
embedding (vector search p50 46 s under three concurrent writers, 0.60 s
now),
and a cascade pass sent one embedding request per entry instead of one
per
file. The memory plugin's exact pin moves from
`everos[multimodal]==1.4.1` to
`==1.4.2`, and the plugin's own version from `1.4.0` to `1.4.2` -- the
same
number as the EverOS it ships. Both EverOS releases declare the same 36
requirements, so nothing else in the lock follows everos down.

**No adapter change.** Every internal everos symbol the adapter reaches
is
present in 1.4.2 with the same signature, the package layout churn
underneath
included: `infra.persistence.lancedb.tables` became
`infra.persistence.records` and `infra.persistence.backends` became
`lancedb.LanceIndexBackend`, but the adapter imports `episode_repo` and
`agent_skill_repo` from the `lancedb` package, which re-exports both the
way
it did before.

**The derived index repairs itself, once.** 1.4.2 makes a table's
primary key
`(app_id, project_id, id)` rather than `id` alone, and the first full
cascade
pass after the upgrade re-reconciles every indexed markdown file against
the
index, re-embedding only the rows that are missing and recording that it
ran
in `.index/.index_reconcile_version`. A healthy root pays one read pass
and no
embedding; markdown is the source either way and was never affected by
the bug
this fixes. The two roots EverOS says need `everos cascade rebuild` by
hand are
a Milvus derived index and a root where reflection already ran across
projects;
raven writes one project into a LanceDB root and reaches neither, though
a user
who pointed raven at a root of their own might.

**The plugin version now equals the EverOS it pins.** The only precedent
was
plugin `1.2.0` pinning EverOS `1.2.3` and plugin `1.4.0` pinning EverOS
`1.4.1`: the plugin tracked EverOS major.minor and kept its patch for
itself,
and nothing said so anywhere. The result reads as a version one release
behind
the thing it ships, so step 5 of the upgrade SOP now says to hold the
two
equal, and this upgrade skips `1.4.1` to close the gap. That step also
named a
test that no longer exists for the literal assertion
(`test_bundled_shadows_lower_priority_source`); it is
`TestPackageSurface::test_imports_clean`, and the step now lists all
three
files that carry the version rather than the manifest alone.

Recorded in `docs/memory-plugin-architecture.md` section 7.5, which the
upgrade
SOP in section 7.2 asks for.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other (external dependency upgrade)

## Verification

```
uv run --frozen --python 3.12 --all-extras pytest -q \
  tests/test_everos_plugin_discovery.py tests/test_everos_backend.py \
  tests/test_everos_backend_contract.py tests/test_everos_http_adapter.py \
  tests/test_everos_config.py tests/test_everos_server.py \
  tests/test_everos_discover.py tests/test_memory_backend_protocol.py \
  tests/test_memory_backend_contract.py tests/test_cli_plugin_commands.py \
  tests/test_config_update_everos.py tests/test_dependency_constraints.py \
  tests/test_release_plugin_list.py tests/test_skill_router_everos_source.py \
  tests/test_agent_loop_everos_pipeline.py
-> 567 passed, 1 skipped (the skip is Linux-only inotify)

uv run --frozen --python 3.12 --all-extras pytest -q \
  tests/test_everos_plugin_discovery.py tests/test_cli_plugin_commands.py \
  tests/test_release_plugin_list.py tests/test_plugin_manifest.py \
  tests/test_plugin_discover.py tests/test_everos_discover.py
-> 91 passed (re-run after the plugin version moved to 1.4.2)

make lint-types                           -> all checks passed

make lint-python lint-imports lint-deps   -> clean, 10 import contracts kept
make check-source-language check-large-files COMMIT_RANGE=github/main..HEAD -> clean
```

Against a copy of a real 55 MB store (568 episodes, seven LanceDB
tables)
written under 1.2.x and last served by 1.4.1, booted on a scratch root:

- 1.4.2 opens it with no schema complaint, `/health` answers
`version 1.4.2` with `cascade.healthy true`, and the reconcile pass
writes
  `.index/.index_reconcile_version` with no error in 441 log lines.
- The same keyword search (`POST /api/v2/memory/search`, the body the
adapter
sends) returns the same five episode ids in the same order under 1.4.1
and
under 1.4.2; BM25 scores drift in the third decimal, as they did on the
  1.2.3 to 1.4.1 move.
- Raven's own `installed_everos_version()` reads 1.4.2, and every
internal
  symbol the adapter imports resolves with the signature it had.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible change: memory searches stop queueing behind background
writes,
and a long-lived server stops leaking file descriptors. An existing
index pays
one read pass on the first start after the upgrade.

Rollback is `git revert`. The reconcile pass only re-embeds rows the old
primary key had lost, and `lancedb` does not move, so 1.4.1 reads an
index
1.4.2 has written. The new `.index/.index_reconcile_version` file is
ignored by
1.4.1; a re-upgrade later would skip the pass it records, which is the
intent.

Not exercised here: a Milvus derived index (raven configures LanceDB), a
multi-project root, and a real embedding or LLM round trip (the scratch
root
ran without provider keys, so the checks above are the keyword leg and
the
cascade's own reconcile).

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-15 11:01:45 +08:00
zhanghui
d106a5c6ef
feat(plugins): add the everos-cloud-memory plugin and its host wiring (#883)
## Summary

Adds `everos-cloud-memory`, a separately distributed memory plugin for
EverOS Cloud, and wires the host so a second memory plugin is a
first-class choice.

The plugin (`plugins-dist/everos-cloud-memory`) is a thin REST client
over httpx: one API key (the config slice
`plugins.config.everos-cloud-memory.api_key`, or
`EVEROS_CLOUD_API_KEY`), owner ids from the host's `ServiceLocator`
(never from the slice), every add in `agent` mode, and a flush only when
a caller marks the conversation over (`metadata.flush` /
`metadata.is_final`: the sub-agent handoff and the importer's final
batch) and once at stop. It ships its own onboard screen (env or on-file
key used without a prompt; 401 / 403 / unreachable get their own
sentences with Re-enter or Skip), `delete` answers False (the cloud
deletes by scope, nothing calls it), and `health()` classifies seven
answers for `raven doctor` and the settings card.

Host changes, each guarded so `memory.backend` `everos` and `null`
behave exactly as before:

- `raven onboard` step 4 asks "Which memory backend?" when more than one
plugin contributes a screen (preselects the current backend, `Off`
clears it, cancel exits 1 and writes nothing); with one plugin the step
is byte-identical to today.
- The memory page reads whichever backend `memory.backend` names: the
cloud slice's key goes out as a Bearer header and the `/health` probe is
skipped for the cloud; a missing distribution is named in the page's
note.
- `settings.everos` reports `reason: "other_backend"` for any non-EverOS
backend, so the Model page hides the three EverOS-only roles (embedding
stays: knowledge bases use that pin); `settings.everosSet` refuses those
three sections for another backend; `settings.set` admits
`plugins.config.<id>.<field>` against the plugin's manifest through the
same admission door channels use; new `settings.everosCloud` feeds the
card (key source, endpoint, health).
- Provider-key and main-model saves no longer restart the local EverOS
when another backend is active.
- ui-web: an EverOS Cloud card (status chip, key row, endpoint) and the
`settings.everosCloud` data behind it. No page mounts the card: it drew
on Model settings until the owner ruled that a memory backend's endpoint
and credential answer neither question that page asks, and where it
belongs is the Memory section's to say once that section has a design.
Until then the key is entered through `raven onboard` or
`EVEROS_CLOUD_API_KEY`. The fixture gate pins the four optional fields
the offline library never sends.
- Gates: the plugin boundary test scans both memory plugins, coverage
and the release workflow build the second wheel, deptry's ignore list
names the plugin beside `raven_everos`, and a conftest fixture keeps the
second plugin out of entry-point discovery for the existing onboarding
tests (they describe a one-plugin install).

A second topic rides along, because it is what makes the hosted backend
reachable from an agent product at all. Every product under `agents/`
pinned its memory at the stock EverOS address and only `raven-design`
read the host's, so the other four addressed whatever answered on 18791
-- the wizard moves a managed server when the port is taken, and a
server the user runs themselves is wherever they put it.
`inherit_memory_backend` in `raven/config/product_render.py` now carries
`memory.backend`, the endpoint, and (only when the product holds none
itself) the credential from the host into every render, beside the host
denials and plugin opt-outs that already travel there; `raven-design`'s
private copy is deleted and the five product configs stop naming an
address. Identity never travels, which is what keeps five products apart
inside one store. `memory.backend: null` travels like any other value,
because that null is how an operator turns long-term memory off and a
product that kept writing would be the one surface ignoring them; the
product's own default stands only when the host carries no key at all.
Which store memories live in is not a product's choice either: the host
reads a sub-agent's memories back through its own backend, so a product
writing anywhere else left that record empty with nothing reporting it.
`owned` is forced off whenever the local backend is in force, because a
product infers ownership from a slice naming no root -- the shape all
five ship -- and so answered a host that was merely not serving by
starting a second server against a root of its own, stranding every
memory written so far.

Decisions worth a reviewer's eye: the fake cloud
(`tests/_everos_cloud_fake.py`) honours or refuses every field the
backend can send and serves both an in-process transport and an HTTP
face for the real-host acceptance; the live tests
(`tests/integration/test_everos_cloud_real_cloud.py`) skip without
`EVEROS_CLOUD_API_KEY`; run with a real key on 2026-10-09 all six pass
against api.evermind.ai (round trip, get by session, empty flush, 401
wording, probe with and without a key, and extraction without a flush
within two minutes), so the documentation and the service agree on
everything the plugin sends. Design, acceptance cases and plan are under
`docs/specs/2026-10-08-everos-cloud-memory-*.md` and `docs/plans/`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Rebased onto main `182dacac0` on 2026-10-13. What that rebase and the
four commits on top were verified with:

- Full suite, `uv run --frozen --python 3.12 --extra dev pytest -q` ->
**29748 passed, 8 failed** in 4m10s. All eight fail identically with
these four commits' sources reverted in-tree at the same base, so none
of them is this branch's: the two `test_the_products_tool_face_*` cases
in the code and oncall launchers (an extra `web_search` in the visible
tool set), `test_agent_loop_token_budget`,
`test_rpc_files::test_a_host_without_libreoffice_says_so`,
`test_ppt_engine_image_search`, `test_shell_background`, and the two
`test_simulation_*` cases.
- Real-host before and after for the inheritance, driving the launcher
the host spawns (`agents/raven-research/run.py`) against an isolated
`RAVEN_HOME` whose EverOS address is `19701`, not the stock port.
Before: the product rendered `base_url: http://localhost:18791` with no
`owned` key, ignored the host, and started an EverOS of its own -- an
orphaned `everos server start --root <its own state root>/everos`
holding 18791, which is the stranding this change exists to stop. After:
it rendered `{"base_url": "http://localhost:19701", "owned": false}` and
said so itself, "the EverOS you manage is not running at
http://localhost:19701 ... Raven does not start or stop it", with `ps`
and `lsof` confirming nothing was started. On the cloud the product
rendered `memory.backend: everos-cloud` with the host's key and left the
out-of-force local slice untouched, and `raven doctor` over that
rendered config answered `everos-cloud: ok https://api.evermind.ai`, so
the credential reached the service. Identity stayed `raven-research-ng`
throughout, and a fingerprint of the operator's own EverOS root was
identical before and after.
- The first CI round on this head found one that the local gates did
not: `tests/test_import_cycle_budget.py`. `inherit_memory_backend` read
the backend-to-distribution pairing from `raven.core.plugin_stack`, and
that edge from `raven.config` into `raven.core` -- which imports
`raven.config` back -- took mutual pairs from 13 to 14 and packages
inside a cycle from 18 to 22. `make lint-imports` keeps ten contracts
and none of them covers that pair, so it was green throughout. The
pairing now lives in `raven/config/raven.py` beside the two fields it
joins (`MemoryConfig.backend` names a contribution,
`PluginsConfig.config` is keyed by the distribution), which `raven.core`
may import and which imports nothing back; `plugin_stack` re-exports the
two names its own callers read from it. An intermediate commit put it in
`raven.contracts` instead, which broke the cycle but took the papers
ceiling from 3700 to 3719 lines -- `tests/test_kernel_budget.py` says a
new paper is a reviewed change to that ceiling, so it moved again rather
than raising it.
- The review round on this head raised one non-blocking finding, fixed
rather than carried forward: the inheritance read `memory.backend` with
`.get()`, which collapses an explicit `null` into an absent key, so a
host with long-term memory off left every packaged agent on its own
shipped backend and still writing. Reproduced by calling the function
directly before the change and after, now read on presence, and pinned
by a case whose mutation (restoring the `.get()`) turns it and nothing
else red.
- One ordering problem found while checking the above, pre-existing and
not fixed here: `tests/test_rpc_settings.py` run before
`tests/test_cli_onboard_commands.py` leaves
`raven.home._current_config_path` pointing at a dead tmp config, so the
onboard cases write somewhere else. With main's own copies of both files
and main's conftest in the tree it is 57 failures; with this branch's,
19. Which cases turn red changes run to run as xdist distributes them
differently.
- `cd ui-web && npx vitest run` -> 876 files, 3728 passed, 0 failed;
`npm run lint` -> 0 errors (4 warnings, all in files this change does
not touch); `npm run gen:check` -> in sync (228 methods); `npm run
type-check` -> clean; `cd ui-tui && npm run lint:i18n && npm run
lint:rpc` -> in sync
- `make lint-python` / `lint-imports` (10 contracts kept) / `lint-types`
/ `lint-deps` -> all pass; `uv lock --check` -> exit 0 with both sides'
new packages present; `scripts/check_source_language.py` and
`scripts/check_large_files.py` on `github/main..HEAD` -> exit 0;
commitlint over every commit -> ok
- Semantic-conflict audit of the rebase, beyond what the suite answers:
21 of the branch's 66 files are also in main's 499, and every one of
main's changes in them is additive and elsewhere in the file. The one
that could have broken silently is main's new `vision.model` /
`vision.provider` simple keys meeting this branch's new
`plugins.config.<id>.<field>` branch in `settings_set`; nothing in the
suite exercises `vision.*` through that method, so it was run directly
-> `{'applied': True}` and `{"model": "..."}` on disk. The subtraction
decisions were re-checked against the new base: no new caller of a
memory backend's `delete`, main's new onboard cases are about TLS and
not the memory step, `inherit_everos_address` exists nowhere, and
`DEFAULT_EVEROS_BASE_URL` is still defined.
- Mutation runs on the new code, restored afterwards: dropping `owned:
false`, letting the host's credential overwrite the product's own, not
writing the host's backend name, and carrying every memory slice instead
of the one in force each turn their own cases red; making the cloud card
render nothing turns 4 of its 5 new cases red and leaves the fifth
("draws nothing when not selected") green.

From the earlier round, on base `6d509a8c5`:

- `uv run --frozen --python 3.12 --extra dev pytest -p no:cacheprovider
-q tests/test_everos_cloud_fake.py tests/test_everos_cloud_backend.py
tests/test_everos_cloud_backend_contract.py
tests/test_everos_cloud_onboard.py tests/test_everos_cloud_discover.py
tests/test_cli_onboard_commands.py tests/test_rpc_memory.py
tests/test_rpc_settings.py tests/test_plugin_boundary.py
tests/test_rpc_schema_match.py tests/test_rpc_registration.py
tests/test_release_plugin_list.py
tests/integration/test_everos_cloud_real_cloud.py` -> 1001 passed, 6
skipped (the six live tests, no key)
- `cd ui-web && npx vitest run` -> 214 files, 3211 passed; `npm run
lint` -> 0 errors (4 warnings, all in files this change does not touch);
`npm run gen:check` -> in sync (207 methods); `cd ui-tui && npm run
lint:rpc && npm run lint:i18n` -> in sync
- `make lint` (ruff, ruff format, ui tsc, deptry) -> ok; `make
lint-types` -> All checks passed; `make build-ui` -> boot snapshots
match; `scripts/check_large_files.py` and
`scripts/check_source_language.py` on `6d509a8c5..HEAD` -> exit 0;
commitlint and `scripts/check_commit_messages.py` on each commit -> ok
- Mutation runs per change (restored afterwards, `git diff` clean): a
host import of the plugin turns the boundary test red; `import
raven_everos` in the plugin turns the import-isolation test red;
removing the chooser's cancel exit, the Bearer header, the restart
guard, the slice-admission guard, the Roles filter and the fake's two
fixes each turn their own test red
- Real-host acceptance (64 cases,
`docs/specs/2026-10-08-everos-cloud-memory-acceptance.md`) run against a
fake EverOS Cloud: the wizard in tmux (chooser, key typed / from env /
on file, 401 / 403 / unreachable wording, back, Off, Ctrl-C,
non-interactive, base-tree transcript byte-identical with one plugin),
`raven agent` turns (user and agent-track recall, agent-mode adds, 4 s
recall bound, a 300 KB message dropped with a warning, a sub-agent trace
stored under its own ids, flushed and read back, one exit flush per
unflushed session), `raven import run` (3 adds then one flush; a hung
flush retried 4x and the exit sweep defeated), `raven doctor` under
seven cloud conditions (seven sentences, exit 2 only for the five
faults) and with the distribution hidden, and the web page through
playwright-cli (memory page counts and the fake's episode, empty and
refused stores, the Model page hiding the three EverOS-only roles while
the embedding row stays, the cloud card rotating a key and showing the
401 wording, three refused settings.set frames over the gateway
WebSocket, the local everos / null / mem0 backends drawing the page as
before, and a provider-key save that restarts the local EverOS on the
base tree and does not on this branch). Evidence and the per-case
verdicts are kept outside the repo. The live service was reached with a
real key after the acceptance run: six tests, six passes, the service's
replies archived with the evidence.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible changes: the wizard's step 4 chooser (only when two memory
plugins are installed); Model settings hides the three EverOS-only role
rows when `memory.backend` is not `everos`/`null` and draws no cloud
card; the memory page sends a Bearer header for the cloud backend.

Two behaviour changes for agent products, both on the `agents/` lane
only. An agent product now follows the host's `memory.backend` and
endpoint instead of its own, which is a fix for the four products that
never followed the host and a change for `raven-design`, which followed
the endpoint but not the backend. And a product no longer starts an
EverOS server of its own: where it used to answer a host that was not
serving by starting one against an empty root of its own, it now reports
long-term memory as unavailable, which is what the host does. Rollback
for this part is the one commit; the product configs it edits are
shipped files, not user config. The plugin is not installed by default
(its wheel is built by the release workflow; nothing changes for an
install without it). Rollback: revert the squash commit; the config
schema gains no new key (the slice is a plugin slice), so an older raven
reads a config written by this one.

Credentials: the cloud key is written only by `set_plugin_config_fields`
into the plugin's slice; the settings page's key row goes through
`settings.set` with a manifest-declared field, and
`settings.everosCloud` reports `api_key_set` / `key_source`, never the
value.

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 11:08:11 +08:00
zhanghui
efe0f46aed
feat(agent): annotate the conversation tail into the episodic log (#924)
## Summary

`user_memory/episodic/episodes.md` had no writer on any surface people
use.
Its only one, the annotator, runs inside the token-pressure compaction
loop,
and all three calls into that loop are gated on `owns_compaction`, which
the
single shipped context engine hardcodes to `True`. The annotator
survived as
live code with one caller: `/new` typed as message text in a chat
channel,
which the web and TUI "new session" buttons never send. Six readers read
the
empty file: the prompt line, two attention producers and three predictor
modules.

This gives the log a writer that does not depend on compaction. After a
user
turn, once `memory.annotateEveryTurns` user turns have accumulated since
the
last annotation, the unannotated tail goes to the same annotator, in the
background, under the consolidator's per-session lock so it cannot
interleave
with `/new`.

Key decisions, each with the alternative it rejected:

- **Its own cursor, not `last_consolidated`.** The `owns_compaction`
gate
  exists to keep a second writer off the Curator's pointer, and
  `undo_last_turn` reads that pointer as a floor. The cursor is
  `session.metadata["annotated_through"]`.
- **Patched, not saved.** A detached task that calls `sessions.save`
with its
  own copy of the metadata drops keys the next turn wrote;
  `append_metadata_patch` exists for exactly this writer.
- **Seeded at the length before the first seen turn.** The old loop fed
the
annotator chunks bounded by a token budget; this step has no chunking,
so an
unseeded long session would go to the provider in one call. Seeding also
covers forks, which copy messages but not metadata. Seeding at the
*current*
  length instead would skip the first turn of every new session.
- **Truncation moves the cursor where it happens.** A lazy clamp at the
next
turn leaves the stale index on disk and swallows every turn written
after an
undo or a clear until the count climbs past it. `Session.clear` drops
the
cursor and `Session.undo_last_turn` pulls it back to the cut, beside the
same
  methods' handling of `last_consolidated`.
- **Only `Origin.USER` turns.** Two of the file's readers are Sentinel
producers; writing Sentinel's own turns into it would feed the layer its
own
  output.

`memory.annotateEveryTurns` ships at 0, which turns the step off: the
log has
had no writer for the life of the project, so no install is waiting on
one,
and nobody is billed for a side provider call to fill a file they may
never
read. Set it to 5 (or any positive cadence) to turn it on.

### The second commit, and why it is here

Driving this on a live gateway showed the annotator could not write at
all on
the default model. It sends `tool_choice="required"`, and DeepSeek's
thinking
mode answers `HTTP 400: Thinking mode does not support this
tool_choice`. The
Responses adapter returned that as the call's result, so the annotator
logged
"LLM did not call annotate_conversation" and gave up -- correct
behaviour, over
a feature that could never work.

Nine call sites force a tool choice today: two in the memory
consolidator,
three in the proactive layer, four in the playbook compiler. Each
already
handles a model that declines under `auto`. So the adapter now does what
its
Anthropic sibling already did: a 400 that names `tool_choice` while the
body
forces one relaxes it to `auto` and retries once. Measured against the
live
vendor three ways -- `required` answered 400, `auto` and omitted both
produced
the tool call.

The two commits are separable: the provider repair stands on its own and
fixes
eight other latent call sites; without it the first commit is inert on
this
model.

## Type

- [x] Feature

## Verification

Unit, on this branch:

- `uv run --frozen --python 3.12 pytest
tests/test_agent_loop_episodic_annotate.py
tests/test_providers_forced_tool_choice.py tests/test_session_manager.py
tests/test_segments.py tests/test_agent_loop_backend_dispatch.py
tests/test_context_engine_factory.py tests/test_providers*.py -q` -> 344
passed
- Broader sweep over the loop, session, provider, context-engine, memory
and
  playbook suites -> 2919 passed, 103 skipped, 1 failed:

`test_agent_loop_token_budget.py::test_a_ceiling_as_large_as_the_window_still_leaves_room_for_history`
asserts `reserved_tools < 7_500` and measures 7557. Pre-existing: a
clean
`github/main` worktree at `829eb28e1` fails it with the same 7557. This
  change touches no tool schema.
- `make lint-types` -> All checks passed
- `uv run --frozen --all-extras lint-imports` -> 10 contracts kept, 0
broken
- `ruff check` / `ruff format --check` on every touched file -> clean
- `scripts/check_source_language.py github/main...HEAD` -> exit 0
- `scripts/check_large_files.py github/main` -> exit 0
- commitlint per sha, with the repo config -> both ok

Every new guard was mutation-verified: deleted, the suite run, the
matching
test confirmed red on its own symptom, restored, md5-compared. Nine
guards,
nine reds -- the knob clause, the origin check, the session lock, the
cursor-on-decline return, the seed, the undo pull-back, the clear drop,
and
the provider repair's two halves.

Live gateway, isolated `RAVEN_HOME`, real provider, `memory.backend =
null`:

| What | Result |
|---|---|
| 12-message pre-existing session, one turn | cursor seeded at 12,
`episodes.md` 0 bytes |
| two turns past the cursor | no annotator call, cursor unmoved |
| three substantive turns | `annotate done for 6 messages -> 3
episode(s)`; three lines carrying a token minted at run time |
| those three lines through `parse_episode_line` | all three parse,
tagged `#project-<token>` the way `project_rhythm` ranks by |
| `annotateEveryTurns: 0` | file unchanged, no annotator call, cursor
not even seeded |
| `session.undo` of two turns | `removed: 4`, cursor pulled 26 -> 22 in
the record the undo wrote |
| kill by pid, restart | cursor still 26 |
| trivia-only turns | `annotate done for 8 messages -> 0 episode(s)`,
cursor advanced -- process-only episodes are dropped by design |
| identity prompt, both branches | no-backend unchanged; with a backend
the log is named read-only and the profile path stays out |

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes:

- Nothing changes for an install that leaves `annotateEveryTurns` at its
  default of 0: the step short-circuits before the cursor is seeded. An
install that sets a positive cadence runs one annotator LLM call per
that
  many user turns, plus a profile refresh when a tag crosses the hot
threshold; with a memory backend wired that means two extractors over
the
same conversation. The knob is read at boot, so a change needs a gateway
  reload.
- With a backend wired the identity prompt gains one read-only line
naming the
  episodic log.
- A forced tool choice that a vendor refuses now costs one extra request
instead of failing the call. Callers that relied on the error are none:
all
  nine already handle a declining model.

Rollback: back to `memory.annotateEveryTurns: 0` disables the first
commit's
behaviour without reverting it. The provider repair only changes a path
that previously
ended in an error response.

Not covered: a crash between the file write and the metadata patch
re-annotates
one tail, duplicating lines. Harmless to grep and over-counts a tag by
one
batch at most; making it atomic means writing the cursor into the same
file
under the same lock.

## Related Issues

#122

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-13 20:10:59 +08:00
Handsome-wzw
829eb28e1d
feat(*): build the ssh launch line and remote workdir for an agent on a registered machine (#893)
## Summary

First of a series of changes that let an ACP sub-agent run on a machine
in the
machine registry (`connections.json`), reached over ssh, so one turn can
dispatch work to agents on several machines at once.

The approach was measured first, on two registered Linux machines and
without changing code: raven's own `AcpClient`, started on an `ssh`
line,
already drives a remote `claude-agent-acp` and `qwen --acp` (handshake,
`session/new`, `session/cancel`, 10 connections at once), and a remote
agent exits once the local ssh is killed -- idle within 15 s, and mid
tool call within 2 s, its child command included. What breaks is
everything that assumes the agent shares this disk; the first of those
is that an agent refuses a `cwd` that does not exist where it runs
(`-32602 cwd does not exist on the machine running the agent`).

This change adds the pieces only a remote agent needs, with nothing
calling them yet:

- `raven/acp_client/remote.py`
  - `machine(id)`: the registered ssh machine. Refuses an unknown id, an
    unreadable registry, an entry with a blocking problem, and a local
    row, each in one sentence. Looked up in the caller, the way the
    machine channel looks one up, since main keeps the registry's
    readers to what its callers need.
  - `launch_command()`: the ssh line, as the one string the pool and
    `AcpClient` already take as a command (`shlex.join`, so their
    `shlex.split` gives back the argv). Built from the registry at
    launch and never stored, so no address, port or key path reaches a
    config file, log line or model. The agent starts in the machine
    user's login shell (`$SHELL -lic`): measured, agents installed
    through nvm or into `~/.local/bin` are on that PATH and no other.
    The remote command is one argv element after `--`, quoted once per
    shell that reads it -- the hand-written form
    `ssh host -- bash -lic 'qwen --acp'` loses `--acp` to the far
    shell's re-parse and hangs to the initialize timeout.
  - `ssh_log_path()` / `read_ssh_log()`: ssh's own messages name the
    address ("connect to host <ip> port <n>", and on a first connect
    "Permanently added '[<ip>]:<n>'"). The launch line sends them to an
    owner-only log per agent (`ssh -E`) instead of to stderr, which the
    connection journals and quotes in its errors; they are read back
    only to classify a failure. Measured: with `-E`, stderr carries the
    far side's output and nothing of ssh's.
  - `session_dir()`: `<root>/<handle>` made on the machine and resolved
    to an absolute path in one round trip (0.17 s measured). A directory
    the machine will not make is reported with the machine's own last
    word -- measured on a registered machine whose disk had filled, "No
    space left on device" -- skipping any line that carries the address.
  - `explain()`: one sentence for an ssh-level failure (timed out,
    unreachable, login refused, host key changed, agent not installed),
    naming the machine and never its address.
- `raven/ops/transport.py`: the ssh options move into `ssh_argv()` and
  the row's address into `ssh_target()`, so the one-shot runner and a
  remote agent share one spelling. The runner sends the same argv as
  before; a new test pins it.
- `docs/specs/2026-08-11-acp-local-agent-mesh-design.md`: its "remote
  agents" non-goal carries a dated amendment pointing at `remote.py`.
  ACP still runs over stdio only; the local process is ssh.

The next change (#897) wires this into the ACP backend behind a
`machine`
field on an agent's entry. Later changes add a small Raven process on
each registered machine that
reads and writes files there for this one, so the main agent can read a
remote file's lines without copying the file; scp-style remote paths for
the file tools; and `machine` on `subagents.add`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_acp_remote.py tests/test_ops_transport.py
tests/test_ops_connection_add.py tests/test_shell_machine_channel.py
tests/test_ops_connections.py`
  -- all passed. The quoting cases run the remote command through real
  shells here (`sh -c` standing in for sshd, `$SHELL -lic` for the login
  shell, `HOME` in a temp dir): the agent's own arguments arrive intact,
  a variable valued ``a b 'c' "d" $HOME `id` \e`` arrives unexpanded,
  a `~` root with a space in it is made and entered, a relative root
  comes back absolute from the login's home, and an agent on the PATH
  only a login shell's `.profile` sets is found.
- The same generated command run under dash on a registered Linux
  machine: same argv, cwd and value.
- Real machines, through `remote.py` and raven's `AcpClient`, against a
  scratch registry (not `~/.raven`): the session directory was made and
  resolved on the machine, `claude-agent-acp` answered `initialize` and
  accepted `session/new` in that directory; a machine that does not
  answer, an unregistered id, a wrong key and an agent that is not
  installed each came back as one sentence with no address in it. The
  remote test directory was removed afterwards and no process was left.
- `uv run pytest --ignore=tests/integration --idle-ceiling-strict` --
  27800 passed, 119 skipped, 3 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
  and

`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
  which fail the same way on main on this machine.
- Diff coverage over the changed lines: 99% (the gate asks for 90).
- Twenty mutations, each reverting one part of the change, each
  turning a test red: the machine's reason dropped, the line carrying
  the address kept, the inner command not quoted for the outer shell,
  the variable assignments not quoted, `-T` dropped, `--` dropped, the
  extra options placed after the destination, the runner's host-key
  option dropped, ssh's own words passed on, the host-key failure not
  recognised, an unsafe handle kept without its digest, a slash allowed
  in a handle, a local row accepted as a remote machine, an entry with a
  blocking problem accepted, the session directory echoed instead of
  resolved on the machine, a failed `mkdir` taken as success, a `~` root
  quoted so the shell cannot expand it, the agent not started in the
  root, a bad variable name let through, and the agent started outside
  the login shell
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
  and `scripts/check_large_files.py` on the branch range, and the
  prospective squash header and the commit through `commitlint` -- all
  clean

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Nothing user-facing changes yet: no entry can name a machine until the
next change.

## Risk

No behaviour change. `remote.py` has no caller yet, and the one-shot
ssh runner sends the same command line as before, pinned by a test. The
launch line carries the machine's address, which is why it is built at
launch and held only in memory; variables travel on that line too, so
the design keeps secrets off it (no lent keys for a remote agent).
Rollback is a plain revert.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-12 16:09:09 +08:00
userName20260323
8f42a1ec03
feat: ship research reports in a reader layout, measured and link-checked (#890)
## Summary

raven-research reports were written for a grader: fixed `## Answer` /
`## Findings` / `## Limitations` headings, HTTP codes beside links, and
a
chat reply that drifted from the file saved with `write_file`. This
branch
ships them in a reader layout and closes the gaps that product runs on
DeepSeek showed between the brief, the reply and the saved file.

Reader layout (`finalShape.reportLayout = "reader"`, on for
raven-research):

- The report opens with the answer as a blockquote, with no label or
bold
  lead-in. Anything the brief asks the report to open with (a retrieval
date, the kinds of sources) follows in the blockquote. Delivery facts
the
brief asks back (path, counts) go on its last line, which the chat reply
  keeps and the saved file leaves out.
- The body sits under its own `##` headings, using the brief's section
  names word for word, numbering included, when it gives them. The last
  section names what could not be verified (`## Limitations` in English;
  the reply's language otherwise, with no English word in that heading).
The shape gate reads the last `##` section as the limits when no word it
  knows names them, since no word list covers every language, and when
  only the first heading has such a word with no body above it (a body
  heading on export limits is body, not limits). The audit reads it the
  same way and counts a body written as prose under the answer.
- Prose cites through markdown links, with no fetch status beside them.
A
brief that asks for link checks gets one table before the limits,
listing
  every cited URL as fetched, with its status, marked blocked where the
  site blocked the fetch.
- The layout is derived from the deep clause, so the evidence rules stay
  one text. The shape gate, its reminder, rewrite prompt and interrupted
  fallback, and `scripts/research_report_audit.py` read it as the same
  three parts. A test pins the audit's limits matcher to the gate's.

Chat and file are one text: at turn end the delivered reply is written
over
the report file the turn wrote. Which file that is:

- A markdown file the turn wrote whole, or appended to or edited when
the
turn wrote it whole first or it still holds exactly what an earlier turn
  of the session left in it. The session record keeps the sha256 the
session left in each report file, checked at turn start before any tool
  runs. Any other file (a log the user keeps, a report the user changed
  between turns) holds someone else's text, so it is never the report. A
later turn that only edits a report nobody else touched still syncs it.
- Whose `##` headings are at least half the reply's. On the
reader-layout
  product runs a report draft shared 75-100% of its headings with the
reply, and a notes file saved beside the report shared none, so a notes
file or a source index is never overwritten, alone or beside the report.
- Of several such files, the one the reply names; when it names none or
  more than one, every file is left alone.

The file keeps its `#` title when the reply has none. It is also left
alone
when the reply is not a report, or when the reply is under 80% of the
file's length, so a summary reply never replaces the report. The path is
read from the first line of the tool's result only, matched against the
tool's whole fixed wording, and the file tools now
refuse a path that carries a line break or other control character: they
echo the path into their results, so such a path could forge a second
result line naming a file outside the workspace.

Length and counts, measured rather than estimated:

- Every markdown write or edit gets a `[report length: ...]` line on its
  tool result, carrying:
  - CJK characters and other words, with prose and tables apart;
  - the same counts per `##` section;
  - the number of distinct URLs cited, extracted the way the research
    trail extracts them.
- The report checks a brief's length against that line, and takes any
  count it states (sources included) from it.
- `write_file` now reports bytes. Its success line said "N bytes" and
  counted characters, so a Chinese report was sized at about a third of
  its real length.

Fetch status and the research trail:

- The research fetch tool reports the target's own status from the Jina
reader's header instead of the reader's 200, and marks a CAPTCHA wall as
  `blocked`.
- The trail counts such pages as refused by the site, not as read, per
  fetch: a page refused once and read on a retry counts as read. When a
  thin page is re-read at another address, each address keeps its own
  status: the stub its own, each fallback the one in its own header.
- The trail drops click-tracking parameters (`utm_*`, `srsltid`,
`gclid`,
  `fbclid`, `msclkid`) before matching a citation.
- Its `<details>` folds are now plain labelled lists, since a terminal
  shows the tags raw.
- `scripts/research_report_audit.py` reads the refused-page clause and
  note. It also reads the reviewer verdict when the trail names the
  reviewing model. That second break predates this branch: a model id
  carries `/` and `-`, so the verdict read as missing on every product
  trail.

Other:

- The research reviewer now runs on DeepSeek in thinking mode. That
  provider refuses a named `tool_choice` with HTTP 400, so every review
  failed open and each report shipped unreviewed. The gate retries with
  `tool_choice=auto`.
- `measuredGuidance` is off in the shipped config. Its numbers were
  measured on a short-answer benchmark and pull a long report toward a
  terse answer.
- The appendix twin's guard names the twin's lead over the frozen dr@3.5
  record instead of requiring the two to be equal: 4 added and 5 changed
  definitions, with `ResearchTrail` compared member by member so its
  untouched members stay pinned. The record itself is unchanged from
  `main`.
- The product label moves to the `-reader` suffix. Its digest was
  re-stamped several times on this branch only; `main` never shipped an
  intermediate digest.

Checked and deliberately not changed here:

- The main agent's own `web_fetch` (`raven/agent/tools/web.py`) also
  reads through Jina and still reports the reader's 200 for a target
  error. It is a separate module, not a twin of the research tool, so it
  needs its own change.
- `fetch_result_ok` is unchanged, so a page the site refused still
releases the fetch gate, counts toward the evidence floor and enters the
  conversation memo as a source. Changing it would move when the gate
  releases.
- The report sync writes the file directly, not through `write_file`, so
it
  emits no file-change event and a client's diff panel does not show it.
raven-research runs with no write approval, so no approval is bypassed.
- Every CAPTCHA flag in 36 local fetches that carried a reader warning
was
  on a page of 862 characters or fewer. Target errors appear on pages of
  up to 18k characters, and each one checked was a real error page. No
  false positive was found.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run after rebasing onto the current `origin/main`. The range-diff
against
the pre-rebase branch is `=` for all 15 commits.

- `uv run pytest tests/test_agents_research_*.py
tests/test_research_*.py tests/test_filesystem_write_modes.py
tests/test_agent_tools_web_fetch.py tests/test_history_trimmer.py
tests/test_context_engine*.py -q`: 1070 passed.
- Full suite before the rebase, `uv run pytest tests -q`: 27745 passed,
  118 skipped, 12 failed. None of the 12 were introduced by this branch:
  - 11 fail identically in clean worktrees of `origin/main` and of the
    branch head. They are environment-bound: proxy redaction, NAT64
    addresses, the node path.
  - `test_agents_ppt_launcher` passes in the clean branch worktree and
    failed only because of an uncommitted local edit to
    `agents/raven-ppt/subagent.json`.
- Mutation checks: each of these, reverted alone, turns its new tests
red:
  - the target-status override;
  - the refused-page split;
  - the ledger classification;
  - choosing the named report file (3 tests fail with `paths[-1]`);
  - the audit's refused clause;
  - the audit's ` via ` terminator.
- After the second review round:
`uv run pytest tests/test_agents_research_*.py tests/test_research_*.py
tests/test_filesystem_write_modes.py tests/test_filesystem_tools.py
tests/test_deliver_files_tool.py tests/test_rpc_files.py
tests/test_rpc_turn_send.py tests/test_tool_registry_execute.py
tests/test_tools_workdir.py tests/test_agent_tools_web_fetch.py -q`:
  1238 passed. Each fix, reverted alone, turns its new test red: the
  control-character refusal, the first-line read, the append rule, the
  last-section limits (gate and audit apart), the fallback verdicts, and
  the URL count, the per-fetch refusal, the draft-heading filter and the
session-wide edit rule. The twin guard fails on an undeclared definition
and on a
  stale lead entry.
- After the third review round, on the rebased head:
`uv run pytest tests/test_agents_research_*.py tests/test_research_*.py
tests/test_filesystem_write_modes.py tests/test_filesystem_tools.py
tests/test_deliver_files_tool.py tests/test_rpc_files.py
tests/test_rpc_turn_send.py tests/test_tool_registry_execute.py
tests/test_tools_workdir.py tests/test_agent_tools_web_fetch.py
tests/test_history_trimmer.py -q`:
  1278 passed. Mutants each turn the new tests red: the unanchored
unchanged-file match; the control-character guard narrowed to LF, or to
  `Cc` only; each of the four success lines reworded in the real tools;
  a twin-only change to an untouched `ResearchTrail` default.
- After the fourth review round, same command: 1288 passed. Mutants each
  turn the new tests red: held files taken without the digest check, the
  digest never recorded, held files ignored, the first-heading rule
  reverted in the gate and in the audit apart, the gate's body-above
  condition dropped, and the audit's prose-body count dropped.
- `make lint-python`, `make lint-imports`, `make lint-deps`,
  `make lint-types`: all pass.
- `scripts/check_commit_messages.py origin/main..HEAD`,
  `scripts/check_source_language.py origin/main...HEAD`,
  `make check-large-files`: all exit 0.
- Product runs through the TUI host, on reports about personal agents,
AI
  glasses, AI coding agents, office agents and edge AI:
  - The saved file equals the reply, less the delivery line, with its
    title kept.
  - The stated length and source count match the `[report length]` line.
  - The link-check table has one row per counted URL.
- A Gartner 403 and an HBS 405 behind a CAPTCHA came through with their
    real status.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes:

- raven-research replies and saved reports change form: an answer
blockquote, no fixed headings, no HTTP codes beside links, and a single
  link-check table.
- The report file a research turn wrote is overwritten with the
delivered
  reply at turn end, except in the cases above.
- `read_file`, `write_file`, `edit_file`, `list_dir`, `deliver_files`
and
the RPC file reads refuse a path containing a line break or other
control
  character (they share `resolve_path`).

Existing installs keep the contract they seeded into
`agent_memory/profile/agent.md`. The new layout reaches them only after
that file is deleted; until then the gate still accepts the old
three-heading replies. The changelog says so.

Not covered by this branch:

- The research subagent pins `contextWindowTokens` to 65536. On a host
  model with a 1M window, the budget note reaches 80-136% early and cuts
  research short. Left as is on purpose; it is a separate decision.
- Post-delivery behaviour of the host agent is unchanged. It may recount
  or edit the report after delivery.

Rollback: revert the squash commit. To keep the code but drop the
layout,
set `finalShape.reportLayout` to `"sections"` in
`agents/raven-research/config.json`.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: userName20260323 <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-12 15:41:52 +08:00
Dizhan Xue
6b2609ad66
fix(acp_client): end the reply's message where the live view does (#892)
## Summary

ACP replies and live output need the same message boundaries. A steer
arriving during a report must not drop its earlier half, and a held
prompt's heartbeat must not split that report. Return the last nonblank
message while retaining narration in the transcript.

- A tool opening, terminal or status-less tool update, or plan opening
ends a message. Updates marked `pending` or `in_progress` do not;
Raven's held-prompt heartbeat can arrive mid-report.
- A steer stays inside its message with one separator. That separator
spends the message's budget when it is inside returned text; a separator
between messages does not.
- Leading whitespace remains visible but spends no capped-message budget
because `reply` strips it. Blank chunks and repeated steers before the
first nonblank text no longer silently truncate a reply that fits.
Whitespace inside substantive message text still counts.
- Preserve the distinct closing transcript row and document the
intentional report/save/sign-off behavior. Code comments and the ACP
parsing design name both nonterminal statuses explicitly.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Linux, Python 3.12, after rebase onto current main:

```text
uv run --frozen --all-extras pytest tests/test_subagent_acp.py tests/test_acp_methods.py tests/test_subagent_turn_rows.py tests/test_subagent_history.py tests/test_rpc_dag.py tests/test_rpc_subagent_calls.py tests/test_subagent_openai_steps.py tests/test_subagent_openai_backend.py -q -x --tb=short
```

618 passed. Coverage includes the real held-prompt heartbeat, tool/plan
boundaries, persisted replies, transcript rows, and downstream DAG
consumers.

Native Windows, Python 3.12:

```text
uv run --frozen pytest tests/test_subagent_acp.py -k "blank_prefixes or steered_message or steer_inside or a_steer_sets" -n 0 -q -x --tb=short
```

23 passed. Nine blank-prefix cases failed before the fix; the final
matrix also covers prefixes after a tool boundary and explicitly
preserves visible whitespace. Four isolated mutations were caught:
charging stripped whitespace, treating every chunk as leading, ending
the leading-prefix state on a blank chunk, and dropping visible leading
whitespace. Sources were restored after each run.

Checks passed:

```text
uv run --frozen pre-commit run --from-ref origin/main --to-ref HEAD
uv run --frozen ty check raven/acp_client/acp_agent.py raven/agent/subagent/backends/base.py raven/agent/subagent/history.py
uv run --frozen lint-imports
uv run --frozen python scripts/check_commit_messages.py origin/main..HEAD
commitlint --from origin/main --to HEAD
uv run --frozen python scripts/check_source_language.py origin/main..HEAD
uv run --frozen python scripts/check_large_files.py origin/main..HEAD
git diff --check origin/main...HEAD
```

All 10 import contracts kept. Full repository tests, native macOS, and
the TUI suite were not rerun for this follow-up. The full ACP subprocess
selection ran on Linux; Windows verification covers the collector
directly.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

ACP callers receive the last nonblank message, so a report followed by a
save step and "Saved." returns the sign-off; the earlier report remains
in the transcript. Leading display-only whitespace and separators
between messages can make the raw stream longer than the
returned-message cap. Internal separators and substantive text remain
capped together. No stored format or configuration changes.

Rollback: revert the squash commit.

## Related Issues

#889

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-11 23:06:09 +08:00
Handsome-wzw
b56da140aa
fix(ui-web): say there is no new key instead of sending an empty save (#884)
## Summary

On a connected provider's sheet (Settings > Model providers), Update --
or
Enter in the key field -- with no new key in the field sent
`model.save_key`
with the slug alone. That happens with the field empty, and since #881
with
the stored key the eye revealed, unedited, which is deliberately not
sent
back. A key-shaped provider draws no address in that card, so the
request
carried nothing to save: the server refused it -- or, for Bedrock's
ambient
credential, accepted it and changed nothing -- and the page toasted its
English refusal ("OpenRouter requires an API key") while the key sat on
screen.

- `ProviderDetail.tsx`: when the provider is connected, the field holds
no
new key and the card has no address to send, the page sends nothing and
  says so in the pane's own error line (where "Enter the API key first"
already appears for a provider not yet connected): there is no new key
to
  save; to replace the saved one, type a new key and press Update. A
provider whose address is drawn in that card (a key-less vendor reached
by
  address) still saves the address, as before.
- Endpoint-shaped providers (custom, Azure OpenAI, MiniMax CN) are
covered
too: the page groups them with local servers, but `model.save_key`
refuses
their save without a key, the address included. Not yet connected,
"Enter
the API key first"; connected with nothing changed, the same "no new
key"
line; connected with only the address edited, the address is saved on
its
own through `model.set_fields` -- the route the settings design gives
any
field but the key, and the one the Advanced address field already uses
--
so no key is asked for again, even with the revealed key unedited in the
  field. A save that carries a key goes out as before.
- `i18n/messages.json`: one new string,
`gui.settings.providers.key_unchanged`
(en and zh), naming the API key field since the line is drawn at the
foot
  of the pane; the TUI catalogue is regenerated.
- `docs/specs/2026-09-20-web-settings-model-section-acceptance.md`:
T3.2's
observation for pressing Update over the revealed key now reads "sends
no
  frame and the pane says there is no new key to save", and its red-when
  column fails any frame from that press.

No layout change: the message uses the existing error line.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `ui-web`: `gen:check` clean, `type-check` clean, `eslint .` 0 errors
(the
same 4 warnings as main, in other files), `npx vitest run` -- 214 files,
3211 tests passed; build and both boot snapshots match their goldens.
New
  cases in `ProviderDetail.test.tsx`: Update and Enter over an unedited
revealed key send nothing and show the message, and so does Update over
the empty field; for a custom row, nothing changed shows the same
message,
an edited address is saved alone through `model.set_fields` (also with
the
revealed key unedited in the field), an edited address with a key typed
saves both, and Connect on one not yet connected asks for the key first;
  the existing case that saves a key-less vendor's address still passes.
- `ui-tui`: `lint:i18n` and `type-check` clean; `npm test` -- 144 files,
  2111 tests passed
- Mutations, each turning a test red: dropping the new check (the empty
save
  is sent again), dropping its address condition (a key-less vendor's
address is no longer saved), leaving endpoint providers out, not asking
an
  unconnected endpoint provider for the key first, refusing an edited
  endpoint address again, and writing an unchanged one anyway
- Server side, for the claims above: `credential_status` refuses custom,
azure_openai and minimax_cn_api without a key whether or not an address
is
sent, and accepts each with one; `model.set_fields` with `api_base`
alone
  is how the Advanced address field already saves
- Real browser, on an isolated gateway running this branch (its own home
holding only a made-up key, its own ports, channels off): Settings >
Model
providers > the connected provider; eye pressed, then Update -- the page
sent only the `model.reveal_key` call, no `model.save_key`, and the pane
  showed the new message; eye pressed again, then Update over the empty
field -- no frame, the same message; the config file's hash was
unchanged;
  no console errors
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and
  `scripts/check_large_files.py` on the merge base, and the prospective
  squash header through `commitlint` -- all clean

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing doc describes the provider key field.

## Risk

User-visible change: on a key-shaped or endpoint-shaped provider, Update
or
Connect with nothing new shows a sentence in the reader's language
instead of
the server's English refusal, and no request is made; a connected
endpoint's
edited address now saves without the key. Nothing that used to save
stops
saving: `model.save_key` refused every one of these saves (checked with
the
provider's environment variable set as well: refused, stored key
unchanged),
except Bedrock's, which it accepts and which leaves the config
unchanged. A
provider whose address is drawn in the key card saves exactly as before.
Rollback is a plain revert.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-09 16:35:03 +08:00
Handsome-wzw
6d509a8c51
feat(*): show a provider's saved API key when the eye beside it is pressed (#881)
## Summary

On a connected provider's sheet (Settings > Model providers), the API
key
field is empty by design -- its placeholder says to type a new key to
replace
the saved one -- so the eye beside it had nothing to show. The one key a
reader could not check was the one in use. The page is never sent a key:
`model.options` reports whether one is configured and nothing more.

This reverses a rule the web settings specs record: "the eye toggle
reveals
typed text, never a stored key" (model-section design C10 and A8,
acceptance
T3.2, and the parent design's C10 and "masked" term). The owner asked
for the
key in use to be readable from the eye; each of those rules now carries
a
dated amendment (2026-10-09) saying what it was and what it is. What
still
holds: the stored key never goes back to the wire unedited, and it is
never
part of `model.options` or any other snapshot.

- `model.reveal_key` (new, `raven/rpc/methods/model.py`) reads back the
flat
`api_key` of one provider's section -- the value `model.save_key` writes
and
the field edits -- only when it is the key requests carry: an
`endpoints`
  list or Gemini's `apiKeyList` replaces the flat key outright
(`provider_endpoints`), and a flat key left behind under one answers
null.
It answers that key alone: not the rest of the section, not an
endpoint's
key, not a key the environment supplies. Unknown providers are refused
like
the other `model.*` handlers. Schema, Pydantic models and both generated
  clients carry it.
- The sheet asks for it only when the eye is pressed over an empty key
field
of a connected provider. The key fills the field as text. Pressed again
with the key unedited, the field empties back to its placeholder rather
than holding a "new" key that is the old one; an edited key is the
reader's
and stays. A key the reader typed is read back as typed, with no fetch.
A
  provider that is not connected never asks. Typing, or pressing the eye
again, retires a read still in flight, so an answer that lands after the
reader started typing a replacement does not overwrite it. Update or
Enter
with the revealed key unedited sends what the empty field sends -- no
key
-- rather than writing the stored key back (which rewrote the config and
  could restart EverOS for no change).
- `KeyInput` gains an optional `onPeek(shown)`, called before the eye
turns
and waited for, so the field does not flash empty and then fill. A
failed
  read still turns the eye (the source has already shown the error). Its
  other callers (plugins, tools, agent keys, the add-provider form) pass
  nothing and behave as before.
- The offline fixture library answers the new method with a made-up key
per
  connected row, so the demo page's eye has something to show.

No new strings and no layout change: the same field and the same eye.
`docs/specs/` carries the amendments above.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_rpc_model.py tests/test_rpc_schema_match.py
tests/test_rpc_registration.py --idle-ceiling-strict`
-- 662 passed (eight new cases: the saved key read back, null when none
is
saved or after a disconnect, the key alone from a section that also
holds
an address and a secret header, null for a flat key left behind under an
endpoint or under Gemini's key list, a lone endpoint carrying the same
key
  still revealed, the environment's key not read, an unknown provider
  refused)
- `uv run pytest --ignore=tests/integration --idle-ceiling-strict` after
  `uv sync --all-extras` -- 27691 passed, 115 skipped, 3 failed:
  `test_rpc_files.py::test_a_host_without_libreoffice_says_so`,

`test_simulation_scenario.py::test_a_fresh_trial_refreshes_the_subagent_homes_before_it_runs`
  and

`test_simulation_suite.py::test_a_stopped_run_is_marked_in_its_record_and_finished_with_its_evidence`,
  which fail the same way on main on this machine. The one test over the
  idle ceiling is the failing simulation-suite one.
- `ui-web`: `gen:check` clean, `type-check` clean, `eslint .` 0 errors
(the
same 4 warnings as main, in other files), `npx vitest run` -- 214 files,
  3205 tests passed, including the fixture gates; build and both boot
  snapshots match their goldens
- `ui-tui`: `lint:rpc`, `lint:i18n` and `type-check` clean; `npm test`
-- 144
  files, 2111 tests passed
- Real browser, on an isolated gateway running this branch (its own home
holding only a made-up key, its own ports, channels off): Settings >
Model
providers > the connected provider, eye pressed -- the field showed the
saved key as text; pressed again -- the field was empty and masked, with
  its placeholder; no console errors
- Fifteen mutations, each reverting one part of the change, each turning
a
test red: the handler reading the key redacted, an empty key sent as an
empty string, a flat key that requests do not carry revealed, an
unedited
revealed key saved again, an unknown provider not refused, the method
not
  registered,
the source calling another method, an unconnected provider asked, a
typed
key replaced by the saved one, the revealed key kept when the eye shuts,
an
edited key emptied when it shuts, the eye opening before the key
arrives,
  an answer landing after typing overwriting the typed key, typing not
  retiring the read in flight, and a failed read keeping the eye shut
- `scripts/check_source_language.py`, `scripts/check_commit_messages.py`
and
  `scripts/check_large_files.py` on the merge base, and the prospective
  squash header through `commitlint` -- all clean

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing doc describes the provider key field or its eye.

## Risk

The page can now receive a stored provider key in plaintext, which it
could
not before. The limits on that:

- one provider per call, only on an explicit press, never in
`model.options`
  or any other snapshot or poll, so the key does not ride along on every
  refresh;
- the answer is the key alone, and the RPC dispatcher logs no results;
- it adds no reach: a signed-in page could already read a stored key in
  plaintext through the CLI bridge (`provider get <slug> --show-secrets`
over `cli.dispatch`), and could replace the key or change the address
the
provider sends it to. It is behind the same sign-in and Origin check as
  every other page RPC, and it is an RPC method only, not an agent tool.

Compatibility: the method is additive. The page and the gateway ship
together; a page talking to an older gateway would get method-not-found
when
the eye is pressed over an empty field, show that error, and open on the
empty field, as before.

Rollback is a plain revert.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-09 11:53:25 +08:00
Dizhan Xue
2f0a09dcfc
feat(*): let the image role run on an OpenAI-compatible provider (#866)
## Summary

The image role in Settings offered OpenRouter alone, although
`image_generate` already speaks the OpenAI Images API
(`/images/generations`, `/images/edits`) on any other base. A user with
an OpenAI or OpenAI-compatible provider could reach it only by
hand-editing `tools.media.image.apiKey` and `apiBase`.

`tools.media.image` gains `provider`. A section naming one runs on that
provider's address, key and headers as one group, taken from the
endpoint the key comes from and resolved at call time the way the
embedding pin is (`provider_connection`, factored out of
`resolve_provider_credentials`); the section's own `apiKey`, `apiBase`
and `extraHeaders` stand aside. Every media request sends the section's
`extraHeaders` (a new field, secret like a provider's own) after the
bearer key, the order the chat route gives a provider's `extra_headers`.
The registry declares which providers the image tool can run on
(`ProviderSpec.image_api`: OpenRouter, OpenAI, Custom), `model.options`
carries the flag per row, the roles card offers exactly those for the
image slot, and `settings.set` refuses any other. An OpenRouter pick
writes an empty provider, so existing sections keep their current
meaning. Speech and video keep to OpenRouter: their sections have no
`provider` field, so one written there is ignored on every reader, and
`settings.set` and `set_media` refuse it.

Two fixes on the same path:

- The OpenRouter key, `providers.openrouter.apiKey` or
`OPENROUTER_API_KEY`, is borrowed only by a section that calls
OpenRouter's address. A keyless section pointed at another endpoint used
to send that endpoint the OpenRouter key as its bearer token.
- Off OpenRouter, every image model goes to the Images API. The chat
route with output modalities is OpenRouter's own; an OpenAI-compatible
relay answered it with its own 404 (`model is not found`), which the
OpenRouter-only fallback did not recognise, so a model outside the
Images API name list could not generate there.

The live config reader resolves the named provider's section exactly as
boot does: every spelling, former name and field alias of it goes
through `ProvidersConfig`, which folds them into one section, and no
other provider's section is validated. A section header replaces the
key's header of the same name in any case.

raven-ppt and raven-design inherit the host's image section through the
same resolver. When raven-ppt pins an image setting, the section becomes
a snapshot read as written: it drops the provider name and takes the
Wire Model that provider uses. A pinned key or base leaves the inherited
headers behind, since they complete the host's key at the host's
address. A pinned base also clears whatever key the host resolved, so
the deck's key slot decides afresh: `PPT_IMAGE_API_KEY`, the host
section's own key, or, toward OpenRouter only, the key that pays for the
words. That closes a leak every 0.2 release has, the OpenRouter key the
host borrowed for OpenRouter's address going to a pinned deck base. On a
named provider's own address the deck sends that provider's key or none.
The launcher can outlive its raven, since a stamped home tree outranks a
checkout's own `agents/`, so it calls only what every 0.2 release has:
the reader's two-argument form, and `media_provider` looked up where it
exists.

The settings dialog's design and acceptance specs (C14, A22, A24 and
their decisions) are amended to the new rule.

Overlap: open #432 and #433 (MiniMax images and video) edit
`_generate_one`, the video tool's `execute`, a `_resolve_key` override
on each tool, and the lines beside the removed `borrows_openrouter_key`.
Their MiniMax routes return before the routing this change touches and
their overrides delegate to the base resolver, so the conflicts are
textual.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run --frozen --python 3.12 --all-extras pytest -q`: 27601 passed,
7 failed, 115 skipped. The 7 fail identically on `origin/main` 3632e6040
on this machine (five proxy tests in `test_config_update_providers.py`
under the shell's proxy variables,
`test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`,
and
`test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing`
run as root); 0 introduced.
- ui-web, `node node_modules/vitest/vitest.mjs run
--no-file-parallelism` (both test roots, 214 files): 3188 passed, 0
failed, 0 unhandled errors. Measured before the review commits, which
touch no TypeScript, generated file or RPC contract.
- ui-tui, the same command (144 files): 2089 passed, 9 failed, 0
unhandled errors. The 9 are rendering tests in four files that fail with
identical ids on `origin/main`'s ui-tui on this machine; 0 introduced.
- `tsc --noEmit` in ui-web and ui-tui, `gen-rpc-client.mjs --check`,
`gen-rpc-types.mjs --check`, eslint over the touched ui-web directories,
and `ruff format --check` / `ruff check` over the changed Python files:
all clean.
- `scripts/check_commit_messages.py origin/main..HEAD`, commitlint over
the same range, `scripts/check_source_language.py origin/main...HEAD`,
`scripts/check_large_files.py origin/main...HEAD`: all exit 0.
- The new tests were written first and watched fail for the missing
behaviour, the review rounds' tests on the head before each fix. A
mutation pass then removed each construct this change adds, one at a
time: 40 Python and 3 TypeScript mutants, all killed but one, controls
green. The survivor matches a provider on its name instead of its
`route_names`, which is equivalent today because every former name in
the registry is also a field alias.
- Each commit added in review passes the touched test files on its own.
- Live, against an OpenAI-compatible relay serving
`sensenova-u1.5-lite`: before, `image_generate` with the section's own
`apiKey`/`apiBase` posted to `/chat/completions` and got 404. After, on
an isolated `raven web` in Chromium, the image row offered Custom,
OpenAI and OpenRouter (speech and video OpenRouter only), the typed id
was added to Custom as an image model and saved as `{model, quality,
provider: "custom"}`, and one call through the live config reader
returned a 2048x1536 PNG from `/images/generations` with the Custom
provider's key.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No user-facing documentation covers the media tools; the settings
dialog's specs carry the rule.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

- Security: credentials travel to fewer places. A section borrows only
from the provider it names, address and key together, and a provider
with no known address supplies no key. `tools.media.image.provider`
joins the `settings.set` whitelist with the reach the subsystem pins
already have: it moves the tool to a provider the operator configured,
and names no address and carries no key.
- Provider headers can carry credentials: they travel with the key to
the provider's own address only, a video poll or content URL on another
origin gets neither, and a media section's `extraHeaders` is masked like
a provider's. The OpenRouter default lane still borrows only the
OpenRouter key, not that provider's headers.
- A keyless image, speech or video section whose `apiBase` is not
OpenRouter's now reports a missing key instead of borrowing the
OpenRouter key. A relay in front of OpenRouter at another address needs
`apiKey` on the section.
- On such a base, a model outside the Images API name list (a Gemini
image model, say) now goes to `/images/generations` instead of
`/chat/completions`.
- Choosing a provider other than OpenRouter in Settings sets aside a
hand-written `apiKey`/`apiBase` on the image section; they stay in the
file and apply again once an OpenRouter model is picked.
- A `provider` hand-written into `tools.media.speech` or
`tools.media.video` is now ignored: that section runs on OpenRouter,
with the OpenRouter key.
- raven-ppt: a deck pinning `PPT_IMAGE_API_KEY` no longer sends the
host's `extraHeaders`, so an endpoint that also routes by header cannot
take the deck's own key; before this change no header travelled at all.
A deck pinning `PPT_IMAGE_API_BASE` without a key of its own no longer
receives the OpenRouter key the host borrowed; it needs
`PPT_IMAGE_API_KEY`, the host section's own key, or a base on
OpenRouter's address.
- The raven-ppt launcher still renders under every 0.2 release, which
lacks `media_provider`; a test runs it against a reader with the 0.2
signature.
- Rollback: revert the squash commit. The previous config models ignore
a saved `provider`, so the image tool returns to OpenRouter with the
earlier key rules.

## Related Issues

Fixes #828

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-08 17:33:39 +08:00
Dizhan Xue
b963862e50
fix(*): keep the browser touch note, explain launch failures, isolate the test profile (#848)
## Summary

Follow-ups to #839 in the browser driver, the browser tools and the
real-browser tests.

**The touch note survives the end of an owner's binding (regression from
#839).** #839 had the driver tell the tools' `_acted` store whenever it
dropped an owner's tab binding, and the store then deleted that owner's
last-act stamp. The readback promises `note: the user interacted with
the browser since your last action` until the owner acts again, and the
end of a binding is not the end of the owner: `Browser.release()` has no
production caller, so the listener fired only on a reap after
`OWNER_IDLE_S`, on the owner's tab closing and on the browser closing.
When the reader closed an agent's tab from the panel, or the binding
idled out, the agent's next read lost the note. The listener is removed
and the driver's owner removal is back to its pre-#839 form;
`_ACTED_MAX` still bounds the store.

**A host missing a system library is told so.** The driver now answers
`Chromium is installed but cannot start here: the system library
libatk-1.0.so.0 is missing. Installing system libraries needs root, so
it is the user's step: on Debian or Ubuntu, <python> -m playwright
install-deps chromium`. Before, on x64 the loader's refusal came back as
Playwright's whole launch log, after a throwaway relaunch that could not
help; where Playwright's own dependency check runs (its linux-arm64
Chromium builds), its report suggests `playwright install-deps`, which
the driver read as "Chromium is not installed". The message says whose
step it is because tool results reach the model too.

**Only failures a fresh profile cannot fix skip the throwaway
relaunch.** The relaunch was gated on the substring "profile", and every
Playwright launch failure quotes the command line,
`--user-data-dir=<home>/.raven/browser-profile` included, so every
failure relaunched and logged "profile is busy". Every failure is still
retried once except a missing system library, Playwright's host check
and a missing executable, and the warning says the launch failed rather
than that the profile is busy. Matching Chromium's lock words instead
would be narrower and wrong under another locale: with
`LANG=zh_CN.UTF-8` a second headful launch on a held profile prints its
line in Chinese, Playwright's English rewrite of it does not fire, and
the error says only "Target page, context or browser has been closed".

**The profile path is read at launch.** `PROFILE_DIR` was computed at
import, before `tests/conftest.py` gives each test its own HOME, so the
real-browser tests used the developer's own `~/.raven/browser-profile`,
with parallel workers opening it at once; the cookie
`test_browser_real_web.py` sets turned up in a developer's own cookie
jar. `_profile_dir()` now resolves the path on each launch.

**The real-browser suites skip on a host that cannot load Chromium.**
`tests/_browser_cache.py` gains `chromium_launch_failure()`: one bare
headless launch per process, with Playwright alone. When it fails for a
missing system library, both
`tests/integration/test_browser_*real_web.py` modules skip with the
library named: the twelve `libatk-1.0.so.0` launch failures reported on
#839 become twelve skips. Any other launch failure, such as an older
build left in the cache or a timeout under load, is not a reason to
skip, and the suites fail on it as before.

`docs/browser-and-desktop.md` said a second process would get a
throwaway profile. The headless shell takes no profile lock, so a second
headless Chromium opens the same profile; the sentence now says so.

Not in this PR:

- `url_for` still predicts the landing page before the permission
prompt, so whatever moves while a person decides moves the call (the
binding-generation idea in the #600 discussion).
- Two headless Ravens share one profile with no lock; documented above,
not changed.
-
`tests/integration/test_browser_tools_real_web.py::test_two_owners_two_tabs_and_neither_takes_the_other`
hung in 2 of 73 runs of both real-browser files on main's tree -- both
times with one profile shared by every worker, never in the 24 runs with
a profile per test -- and in none of 78 runs on this branch or 240 runs
of that test alone. The cause is unknown, and counts this small cannot
credit any change here with a fix. One of the first 45 branch runs, made
before the profile change, also failed one test outright; it was not
seen again and is not explained.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Python 3.12, the repo environment with all extras, run from this tree.

- `python -m pytest tests/test_browser_driver.py
tests/test_browser_tools.py tests/test_browser_cache.py -q`: 129 passed.
With `raven/browser/driver.py` and `raven/agent/tools/browser.py` taken
from origin/main, 15 of them fail: the touch-note test for a reader's
close and for an idle binding, the launch-time profile test, the four
host-library cases, the empty-message retry, and the seven probe tests,
which call a helper main lacks.
- Each guarded construct was mutated on its own (14 mutations, among
them: no stamp after a close, a stamp pruned on a tab close, each of the
three no-retry clauses dropped, the host-library check moved after the
not-installed one, the profile path fixed at import, the probe skipping
on any failure, the probe launching on the caller's loop); each turns at
least one test red.
- Full suite, `python -m pytest -q`: 27482 passed, 116 skipped, 2
failed. Both failures are local to this box and touch nothing this
branch changes:
`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`
(an npm elsewhere on PATH leaks past the fixture) and
`tests/test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing`
(the suite runs as root, which reads the file the test makes
unreadable).
- `python -m pytest tests/integration/test_browser_real_web.py
tests/integration/test_browser_tools_real_web.py -q` with a real
Chromium: 12 passed in each of 3 runs. A guard around `_profile_dir()`
logged 39 resolutions, none under the real home, and no file under
`~/.raven/browser-profile` changed during the runs.
- Report shapes: the host-library fixtures follow real launches against
a binary built to need an absent library, with Playwright's dependency
check both skipped (the loader's refusal) and forced (both of its Linux
reports). Two headless launches on one profile: the second starts. Two
headful launches on one profile under `xvfb-run`: with
`LANG=en_US.UTF-8` the error says the profile "is already in use"; with
`LANG=zh_CN.UTF-8` it carries none of Chromium's English lock words.
- `ruff check`, `ruff format --check`, `lint-imports` (10 kept), `ty
check raven`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`: all pass.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible: the touch note is reported again after the reader closes
an agent's tab or the binding idles out; a launch that fails for a
missing system library reads as one line naming it, with no throwaway
relaunch; every other launch failure still gets one throwaway relaunch,
under a different warning.

- [x] Security impact considered: the new message names a root command,
and it is worded as the user's step because tool results reach the
model.
- [x] Backward compatibility considered: `Browser.on_owner_released`
(added in #839) and the module constant `PROFILE_DIR` are removed; `git
grep` finds no reader of either outside the two files changed here.
- [x] Rollback path is clear for risky changes: revert the squash
commit; nothing persisted changes shape.

## Related Issues

Follows up #839; refs #600.

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-03 22:09:53 +08:00
Dizhan Xue
e84152c52f
docs: link the technical report on arxiv and hugging face (#836)
## Summary

The technical report is now on arXiv as 2609.33439, with a paper page
on Hugging Face. This points the READMEs at those two pages instead of
the `tech-report-v1` Release PDF, and switches the citation to the
arXiv entry.

- Both READMEs: the report link that leads the navigation line under
  the badges becomes two links, arXiv and the Hugging Face paper page.
  The Citation sentence now links the arXiv abstract page.
- Badge row, both READMEs: an arXiv badge (2609.33439) joins it beside
  the X badge, and the HuggingFace badge is relabelled Paper and points
  at the paper page instead of the organisation page.
- The navigation line under the badges drops its two report links, now
  that the badges carry them, and keeps Website, Documentation and the
  language switch.
- BibTeX, in all three copies (both READMEs and
  `docs/technical-report/README.md`, still byte-identical):
  `@techreport` becomes arXiv's `@misc` form with `eprint`,
  `archivePrefix` and `primaryClass`. `institution` and `month` are
  dropped, as arXiv's export carries neither.
- Two deliberate departures from arXiv's generated export. The key
  stays `evermind2026raven`, so existing citations keep resolving.
  Author and title keep their double braces: arXiv parses "EverMind
  AI" as a person (its `citation_author` reads "AI, EverMind", which
  is where its generated key `ai2026ravenharnessharnessescomposable`
  comes from), so the verbatim export cites as "AI [2026]" under
  natbib author-year, and as "E. AI" with a lowercased title under
  `abbrv`.
- `docs/technical-report/README.md` keeps its Release download and
  adds the two links. The Release PDF and arXiv v1 are the same
  document: 82 pages each, and a word-level diff of their text is
  empty apart from arXiv's margin stamp.

Not changed in git: the `tech-report-v1` Release notes held a fourth
copy of the old `@techreport` entry. It is outside the repository; its
notes were updated to the same entry, and `/releases/latest` was
re-checked afterwards and is still the software release.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run at the branch head, with the repository's Python 3.12 environment:

```
PYTHONPATH=. python scripts/check_large_files.py origin/main..HEAD
PYTHONPATH=. python scripts/check_source_language.py origin/main..HEAD
PYTHONPATH=. python scripts/check_commit_messages.py origin/main..HEAD
commitlint --from origin/main --to HEAD --config commitlint.config.cjs
pre-commit run --from-ref origin/main --to-ref HEAD
python -m pytest -q tests/test_living_docs.py \
  tests/test_cli_onboard_commands.py::test_readme_quickstart_matches_the_installer \
  tests/test_docker_runtime.py::test_container_exposes_no_engine_selector
```

The four gates exit 0; every pre-commit hook that applies passes and
modifies no file; pytest reports 4 passed: the living-docs scan and
the two tests that read the root READMEs.

- Links: `https://arxiv.org/abs/2609.33439` and
  `https://huggingface.co/papers/2609.33439` both return HTTP 200, and
  both pages carry the report's exact title.
- BibTeX: the entry compiles without warnings under bibtex with
  `plain`, `abbrv` and `plainnat`, and under biber with biblatex
  `numeric`, which prints `arXiv: 2609.33439 [cs.AI]`.
- Rendering: GitHub's own render of all three files at the branch
  head; every new link carries the intended href, and each file
  renders one bibtex code block.
- The three BibTeX blocks share one sha256.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Documentation only: three Markdown files, no code. The README
navigation line and Citation section change for readers. Existing
citations keep their key, and the Release and its download URL are
untouched, so old links still resolve. Roll back by reverting the
squash commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-01 11:37:18 +08:00
Dizhan Xue
25c576859a
docs: add the technical report page and link it from the readmes (#812)
## Summary

Adds `docs/technical-report/README.md`, the landing page for the Raven
technical report: title, abstract, BibTeX entry, and a download link for
the PDF. It replaces the zero-byte `.gitkeep` that created the folder,
and `docs/README.md` gains the index entry that the placeholder change
deferred until real content landed.

Both root READMEs link the PDF itself rather than that page: a Technical
Report link leads the navigation line under the badges, and a closing
Citation section, after License, carries the same BibTeX entry and links
the same file. The English and Chinese READMEs change in the same
places, and the three BibTeX blocks are byte-identical.

The PDF itself (4,035,271 bytes, 82 pages) is not committed. AGENTS.md
section 7 keeps report assets and files over 1 MiB out of git, and
`scripts/check_large_files.py` rejects this file on both counts. It is
attached to the `tech-report-v1` release instead, published with
`--latest=false`: `/releases/latest` is what `install.sh`, `install.ps1`
and `raven/updates/upgrade.py` resolve, and all three accept only a
`vX.Y.Z` tag there. The report tag points at the v0.2.3 commit. A later
revision of the report gets its own release rather than replacing this
asset.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Every range below starts at the branch base, the #810 squash on `main`.

- `scripts/check_large_files.py <base>..HEAD`: exit 0.
- `scripts/check_source_language.py <base>..HEAD`: exit 0.
- `scripts/check_commit_messages.py <base>..HEAD`: exit 0.
- `commitlint --from <base> --to HEAD --config commitlint.config.cjs`:
exit 0.
- `pre-commit run --from-ref <base> --to-ref HEAD`: exit 0. The hooks
that match these files passed and modified nothing.
- `pytest -q tests/test_living_docs.py tests/test_docs_site.py
tests/test_large_file_check.py tests/test_docker_runtime.py`: 41 passed.
`test_docker_runtime.py` is the one test that reads both root READMEs.
- All three pages rendered through the GitHub markdown API. The landing
page shows the abstract's en dash, both emphasized terms, the two links
and the BibTeX block; each root README shows the Technical Report link
first in its navigation line, both of its report links resolve to the
PDF download URL, and it ends with the Citation section and its BibTeX
block.
- The release asset's digest equals the local file's SHA-256
(`14832f50...`). After publishing,
`https://github.com/EverMind-AI/Raven/releases/latest` still redirects
to `v0.2.3`, and the release page and the PDF download URL both return
200 to an anonymous request.

`make lint` was not run: `PYTHON_LINT_TARGETS` and the TypeScript lint
trees do not include `docs/` or the root READMEs.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Docs only: nothing reads these files at runtime. The release is the part
with reach: it is public, publishing it notified release watchers, and
it must never become Latest. If it has to be withdrawn, `gh release
delete tech-report-v1 -R EverMind-AI/Raven --cleanup-tag` removes it and
its tag, and the landing page's links then 404 until the page is
reverted. Rolling back this PR is a revert of the squash commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

#804 created the folder this page fills.

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-09-27 20:07:07 +08:00
Dizhan Xue
cd2968131b
docs: add a placeholder for the technical report folder (#804)
## Summary

Adds `docs/technical-report/.gitkeep`, a zero-byte placeholder that
creates the `docs/technical-report/` folder for the Raven technical
report. Git tracks files rather than directories, so the folder cannot
exist in the repository without one.

`docs/README.md` gains an index entry when the report's own files land;
an entry for an empty folder would describe nothing. The placeholder can
be deleted in that same change.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Every range below starts at the branch base, the #803 squash on `main`.

- `scripts/check_large_files.py <base>..HEAD`: exit 0. Its
`changed_paths` lists `docs/technical-report/.gitkeep`, so the gate read
the new file.
- `scripts/check_source_language.py <base>..HEAD`: exit 0.
- `scripts/check_commit_messages.py <base>..HEAD`: exit 0.
- `commitlint --from <base> --to HEAD --config commitlint.config.cjs`:
exit 0.
- `pre-commit run --from-ref <base> --to-ref HEAD`: exit 0. The five
hooks that match the file passed and modified nothing; every other hook
skipped with no files to check.
- `pytest -q tests/test_living_docs.py tests/test_docs_site.py
tests/test_large_file_check.py`: 30 passed.
- `git grep -E "technical[-_]report"` finds no hit in the tree, so
nothing references the folder yet.

`make lint` was not run: `PYTHON_LINT_TARGETS` and the TypeScript lint
trees do not include `docs/`. The docs box is unchecked because there is
no user-facing change.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

The file is empty and nothing references it, so no behaviour changes.
Rollback is a revert of the squash commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-09-26 22:33:28 +08:00
Dizhan Xue
9b3ca145b6
docs: refresh bilingual readme showcases (#795)
## Summary

The README showcase now highlights three complete projects: THRESHOLD,
Raven RSI, and Raven's product launch kit. Each project pairs a concise
description with hosted videos, posters, presentations, and website or
README previews.

- Polish the English and Chinese titles, descriptions, and media labels.
- Preserve the previous 11 examples in `docs/showcase.md` and
`docs/showcase.zh-CN.md`, with links from both READMEs and the
documentation index.
- Keep all media hosted externally; this change contains five Markdown
files.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Commands run after rebasing onto the current target:

- `rtk proxy uv run --no-sync --all-extras pytest
tests/test_living_docs.py tests/test_docs_site.py
tests/test_docker_runtime.py
tests/test_cli_onboard_commands.py::test_readme_quickstart_matches_the_installer
-q` - 28 passed.
- `rtk proxy uv run --no-sync pre-commit run --from-ref origin/main
--to-ref HEAD` - all applicable hooks passed, including private-key
detection.
- `rtk proxy env UV_NO_SYNC=1 make check-large-files
check-source-language COMMIT_RANGE=origin/main..HEAD` - passed.
- `rtk proxy env PYTHONPATH=. uv run --no-sync python
scripts/check_commit_messages.py origin/main..HEAD` - passed.
- `rtk proxy git diff --check origin/main...HEAD` - passed.

The migrated showcase bodies match their originals byte for byte. GitHub
Markdown rendering produces three project headings, three tables, 11
images, and two video players per README, with no unrendered bold
markers. Relative links and domain terminology were checked.

All five project website links returned HTTP 200. Requests to the 19
GitHub attachment URLs returned HTTP 403 or timed out; a direct GET
retry also timed out, so media payload delivery could not be revalidated
from this environment.

Validation is scoped to documentation. Type checking is not applicable
to these Markdown-only changes. The full application and integration
suites were not run, and the showcase experiments were not rerun.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

This changes the README presentation and adds navigation to the full
galleries. Runtime behavior, configuration, and dependencies are
unchanged. Media remains dependent on its external hosting. Revert this
documentation change to restore the previous layout.

## Related Issues

N/A
2026-09-26 01:18:48 +08:00
zhanghui
2b3cce70fa
feat(*): upgrade everos to 1.4.1, lift the windows gate, guard the embedding width (#791)
## Summary

EverOS 1.4.0 runs natively on Windows and 1.4.1 pins `openai` below 3,
so the memory plugin's exact pin moves from `everos[multimodal]==1.2.3`
to `==1.4.1`. The lock follows: `pyarrow` 24.0.0 -> 25.0.1 (EverOS's new
floor), `everalgo-boundary` 0.2.0 -> 0.2.1, `everalgo-core` 0.4.0 ->
0.3.0 (EverOS now pins its everalgo layer with `==`), and `msvc-runtime`
on `win32` only. `lancedb` stays at 0.34.0. Every internal everos symbol
the adapter reaches is present in 1.4.1 with the same signature, so the
adapter itself is unchanged. The plugin manifest and package version
move to 1.4.0.

**The Windows gate is gone.** `everos_platform_note` (#737) and its five
call sites, `ServiceState.UNSUPPORTED`, the import.scan refusal (#750),
the wizard's WSL notice and its zh catalogue line are removed;
`_everos_executable` looks for `everos.exe` on `win32`. Spawning,
probing and reusing the server were already portable. Identifying and
stopping a stale server was not: `_cmdline_of` asked `ps`, and the
marker `everos server start` never matched a Windows command line
(`...\Scripts\everos.exe server start`). The command line now comes from
WMI through PowerShell on `win32` and the marker accepts both shapes;
`_listening_port` asks the TCP table through PowerShell there, walking
the launcher's descendants: the socket belongs to the base interpreter
two launchers below `everos.exe`. The server is spawned in a process
group of its own (the Windows counterpart of `start_new_session`, which
also keeps a Ctrl-C at the gateway off it) and stopped with Ctrl-Break,
which uvicorn takes as a shutdown; one that ignores it for ten seconds,
or one no group can be addressed to, gets `TerminateProcess` as before.
And because Windows cannot replace a running executable, the upgrade
helper stops whatever still runs from under the tool environment before
it installs (the server outlives the gateway by design); without that,
`uv` failed on `Scripts\everos.exe` with os error 32. Recorded in
`docs/memory-plugin-architecture.md` section 7.4.

**Two guards, found while verifying against a copy of a real 1.2.x
store:**

- `set_embedding_endpoint` measures the model's vector width before
writing the pin and refuses anything narrower than the 1024-wide memory
index (`REQUIRED_EMBEDDING_DIMENSIONS`), but only where an EverOS
backend will read the pin: the `everos-memory` distribution is installed
and the config names `everos` as the memory backend (an absent key is
the schema default; an explicit `null` or another backend is not gated,
and a knowledge base sizes itself to whatever width the model returns).
The wizard's own check imports the same constant and probe. A
768-dimension model pinned through the settings page had left every
memory store and search answering 500 about a mismatched width, with
nothing on the page saying why. A probe that cannot reach the provider
is not a verdict: the pin is written and a warning logged. The two RPC
writers run the (now network-touching) writer in a thread.
- `ensure_everos_server` reads the running server's `/health` version
and, for a root raven owns, sends a mismatch with the installed everos
through the same precheck / stop / spawn chain a rotated credential
takes (`stale_reason`). Raven used to reuse whatever answered on the
port, so an upgrade left the old server serving until something
unrelated restarted it. A root the user manages is never touched.

**The wire contract is pinned.** Every request body the plugin and the
memory page send (`/add` with a tool exchange, `/flush`, the four
`/search` shapes, the four `/get` kinds) is validated against everos
1.4.1's own request models in `TestRequestBodiesMatchEverosModels`, so a
schema move on the next upgrade fails in the suite rather than as a 422
in a gateway log.

**A six-angle review then hardened the degraded paths** (turn survival
under every EverOS failure, Windows lifecycle, upgrade paths, the
embedding guard, the wire contract, concurrency). The rule: memory
degrades with a notice and never blocks a turn, a start or an upgrade.

- A gateway starts the server again when it finds nothing listening (or
a probe that timed out) and nothing holding the OME lock: only for a
root raven owns, never over a child of its own, at most once per thirty
seconds. The only spawn used to be in `start()`, so a server that went
away left a running gateway without memory until restarted by hand. A
stop that was sent and did not finish is not adopted (uvicorn closes the
port at once and finishes what it had), so the version-mismatch restart
after an upgrade cannot leave the gateway on a dying server.
- A command-line lookup that failed (no `ps`, PowerShell blocked, WMI
wedged, timeout) is distinct from a process that is gone: the stop keeps
waiting, the lock names nothing, and a settings save over a server that
answers but cannot be identified is reported as not applied. A losing
spawn no longer overwrites the live server's pidfile. `stop_pid` counts
wall time and takes a grace; on Windows a backend drains the server it
started at `stop()`.
- The upgrade helper asks again after `Stop-Process` and refuses to run
`uv` while anything from the environment survives (an elevated or
another user's process): `--force` removes the environment before
writing and would have left every file but the one that could not go.
- An embedding pin narrower than the index is measured at backend start
(once per pin per process) and withheld from the spawn: EverOS runs
keyword recall and keeps storing, and the notice names the model and
width. This covers a pin written around the write-time check (before
memory was on, unreachable at save time, edited by hand). The probe no
longer raises on an odd response, times out at ten seconds, honours
`plugins.disabled`; `migrate_roles` runs off the event loop it had put a
provider round-trip onto.
- The memory page uses the `/api/v2` routes (v1 is EverOS's legacy
alias), asks `/health` before a search the way the chat adapter does,
and shows the server's own sentence on a refusal. The recalled profile
is rendered as lines rather than a `namespace(...)` repr with its
evidence fields; a tool-call-only row stores empty content, not
`"None"`; `top_k` stays within 1..100; an empty query asks nothing; a
failed capability probe is not cached. The wizard accepts a model wider
than 1024 (EverOS keeps the first 1024, as the settings page already
allowed) and re-prompts on the host's refusal instead of a traceback.

`docs/memory-plugin-architecture.md` section 7 records the upgrade (7.4)
the way 7.3 recorded 1.2.1 -> 1.2.3.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Unit and RPC tests over every touched area:

```
env -u SERPER_API_KEY uv run --frozen --python 3.12 --extra dev pytest -q tests/test_everos_server.py tests/test_everos_backend.py tests/test_cli_onboard_commands.py tests/test_everos_discover.py tests/test_rpc_settings.py tests/test_everos_config.py tests/test_everos_plugin_discovery.py tests/test_cli_plugin_commands.py tests/test_core_plugin_stack.py tests/test_rpc_memory.py tests/test_rpc_import_sync.py tests/test_everos_http_adapter.py tests/test_memory_backend_protocol.py tests/test_memory_backend_contract.py tests/test_plugin_boundary.py tests/test_i18n_boundary.py tests/test_config_update*.py
# 1312 passed, 1 skipped (inotify is Linux-only) on the first commit; 960 passed after the review round; full suite 26214 passed, 2 failed, 120 skipped, the 2 failing on an untouched github/main checkout too (LibreOffice path, background child env)
uv run --frozen --python 3.12 --extra dev ruff check <changed files>   # All checks passed!
uv run --frozen --python 3.12 --extra dev ruff format --check <changed files>   # 20 files already formatted
```

Against a 46 MB copy of a store written by everos 1.2.x (568 episodes,
5528 atomic facts, seven LanceDB tables), never touching the live one:

- `everos server start` on 1.4.1 opens it with no schema complaint;
`/health` reports `version 1.4.1`, `cascade.healthy true`.
- The same four `/api/v2/memory/search` requests on 1.2.3 and 1.4.1
return the same ids in the same order; BM25 scores drift in the third
decimal place.
- `raven agent -m` on this branch spawns everos 1.4.1, a "remember this"
turn lands in `episodes/episode-2026-09-24.md` and `.atomic_facts/`, a
fresh one-shot session recalls it, and a third session recalls episodes
written under 1.2.x months earlier.
- With a 1.2.3 server left on the port, the next `raven agent -m` logs
`runs everos 1.2.3 while this raven installs 1.4.1; restarting`, the old
pid exits and `/health` reports 1.4.1. The 1.2.3 server had opened the
index 1.4.1 wrote with no complaint.
- Windows 11 box, this branch checked out and synced: `raven serve`
spawns `everos.exe` natively, the Memory page shows its four tabs
instead of the platform sentence, a chat turn lands as an episode
(`/add` 200, `/flush` 200, cascade `upserted=1`), and after a memory
role change the next start logs `holds credentials raven has since
changed; restarting`, the old everos pid exits and the new one answers
`/health` 1.4.1.
- The settings page refuses `BAAI/bge-base-en-v1.5` on deepinfra with
`returns 768-dimension vectors and the memory index is 1024 wide` (real
probe), and the pin stays unchanged.
- Adapter import smoke against 1.4.1: all 13 internal symbols present,
signatures unchanged.
- Windows 11 box, second round: with the server up, `uv pip install
--reinstall --no-deps everos==1.4.1` fails on `Scripts\everos.exe` (os
error 32, exit 2); after the helper's sweep (`Stopped what was still
running from the old install (pid ...)`) the same command succeeds and
the relaunched gateway spawns a fresh 1.4.1. A stop from the spawning
console runs the full uvicorn shutdown (`Shutting down` ... `Finished
server process`) in 3 s; a stop from another console falls back and the
process is gone in 2 s. `lock_holder(root).port` reads 18997 (launcher
-> venv python -> base python).
- `tests/test_everos_server.py tests/test_updates_upgrade.py`: 261
passed, 1 skipped; `tests/test_everos_backend.py -k RequestBodies`: 7
passed.
- Review round (`env -u SERPER_API_KEY uv run --frozen --python 3.12
--extra dev pytest` over the everos, config, settings, memory page,
import, onboard, plugin-stack, boundary, cycle-budget and contract
files): 1225 passed, 1 skipped; `test_everos_server.py` 124,
`test_everos_backend.py` 188.
- Windows 11 box, review round: with the gateway serving, the everos
launcher was terminated by hand (`Stop-Process`, nothing on 18997); the
next turn answered without memory (`recall failed ...
state=unresponsive`), and six seconds after it began the gateway logged
`started everos server` / `everos server ready`, the pidfile named the
new pid and `/health` answered 1.4.1. The first attempt did not fire
because a dead port there takes 2-4 s to be refused, longer than the 1 s
probe; the respawn now fires on a timed-out probe as well, guarded by
the lock holder. The memory page's four counts and a search on the Cases
tab went through the v2 routes.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Windows installs that used to read "long-term memory is not available
on Windows yet" now spawn EverOS, and a stale server is identified and
replaced there too. The stop there is Ctrl-Break with a ten-second
fallback to `TerminateProcess`. An upgrade on Windows first stops every
process whose executable lives under the tool environment: raven's own
gateway, sub-agents and server, nothing outside that directory.
- An embedding pin narrower than 1024 dimensions is refused at the
settings page and the wizard, with the width in the message, only for an
install whose memory runs on EverOS (plugin installed, backend `everos`
or the default). Knowledge-only installs, memory off, and other backends
keep any width. Existing pins are not touched.
- An owned EverOS server whose version differs from the installed
package is restarted on the next session start. A root the user manages
is never restarted.
- A gateway may now start an EverOS server on its own when its probe
finds nothing listening and nothing holding the lock (owned roots only,
at most every thirty seconds). On Windows a backend stops the server it
started when it shuts down (Ctrl-Break, sixty seconds to drain), so a
settings change or an upgrade there restarts the server rather than
leaving it. An upgrade on Windows refuses to proceed while a process
from the tool environment cannot be stopped, and says which pid.
- An embedding pin narrower than 1024 that reached the config around the
settings check is no longer handed to EverOS: recall runs on keywords
with a notice naming the model, until a 1024-dimension model is pinned.
- Rollback: revert the commit and `uv sync`. `lancedb` did not move, so
an index written by 1.4.1 opens under 1.2.3 (exercised on the copy).

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-25 19:33:18 +08:00
Dizhan Xue
6a038a67df
fix(docs): light the contents rail between headings and use raven.evermind.ai's favicon (#734)
## Summary

Two fixes to the documentation site.

**The contents rail went dark between headings.** The right-hand table
of contents lit an entry only while its heading was inside the viewport.
Partway through a section longer than the screen every heading sits
above or below it, so nothing was lit, the gold stretch collapsed to
zero height, and the column stopped following the reader. On the built
site before this change, 18 of 28 English pages had such stretches: 114
of 1264 scroll positions sampled every 120px. `sync()` in
`docs-site/docs/javascripts/toc-progress.js` now falls back to the
nearest heading above when none is in view and lights that entry alone.
The lit stretch and the column follow read the same list, so both pick
it up with no further change. Above the first heading nothing is lit,
since no section has begun there. The design spec states the rule, and
the rail's section comment in `evermind.css`, which still credited
Material with marking the section, now names what does.

**The tab icon differed from raven.evermind.ai.** The site drew its own
simplified EverMind mark, a single heavy lemniscate without the rings.
raven.evermind.ai links one 192px PNG of the full mark for its light,
dark and apple-touch icons alike. `theme.favicon` in
`docs-site/mkdocs.yml` now inlines that PNG byte for byte as a data URI,
since AGENTS.md section 7 keeps image files out of `docs-site/`. For
whoever refreshes it: that site's CDN answers any `Accept` naming
`image/avif` with a lossy AVIF re-encode (5178 bytes), and only a plain
`Accept: */*` returns the original PNG (5184 bytes). The comment above
the value says so.

Checked and deliberately left alone:

- The mark at the top of the navigation rail (`--em-mark` in
`evermind.css`) is a separate drawing and still the simplified one. This
change is the tab icon only.
- raven.evermind.ai also sets an apple-touch icon. Material emits no
such link, and adding one needs a template override, which is an `.html`
file that section 7 blocks.
- The favicon's identity is verified by the measurements below rather
than pinned by a test: a pinned hash would only detect change, and
comparing against the live site would put the network in the suite.
- `tests/integration/` is outside every CI job (`norecursedirs`), so the
e2e results below are local runs.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run --frozen --extra browser --group docs pytest
tests/integration/test_docs_site_controls_e2e.py
tests/integration/test_docs_site_search_e2e.py -n 0 -q`: 63 passed.
- `uv run --frozen --extra browser --group docs pytest
tests/test_docs_site.py -q`: 16 passed.
- The new
`test_table_of_contents_keeps_the_section_above_lit_between_headings`
computes, from the Sandbox page's own layout, every stretch between two
headings (or after the last) with none on screen, and checks the middle
of each. Against the unfixed script it fails (`at 1354px: assert [] ==
[2]`), and with the fix it passes.
- A sweep of every built page in headless Chrome at 1440x900, one sample
every 120px, checking the rule at each position: 0 violations over 1264
English, 1145 Chinese and 21 Overview positions after the change, and
wherever an entry is lit, the first one sits inside the contents column.
The unfixed script gives 114 violations over the same English positions.
- Favicon: each of the 63 built HTML files carries exactly one icon
link, and its decoded bytes match the PNG raven.evermind.ai links
(sha256 `4a5051e1781fac64...`). Drawn in Chrome at 16, 32 and 192px, the
docs icon and that PNG differ in 0 pixels.
- `mkdocs build --strict`, `scripts/check_source_language.py
origin/main...HEAD`, `scripts/check_large_files.py origin/main...HEAD`,
`scripts/check_commit_messages.py origin/main..HEAD`, commitlint,
`pre-commit run --from-ref origin/main --to-ref HEAD`, ruff and `git
diff --check`: all pass.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The last box is left blank because nothing needed it: no documentation
page describes the rail or the icon, and the design spec is updated in
this change.

## Risk

- Between headings, the entry of the section above now stays lit and the
contents column keeps it in view. While any heading is on screen the
behaviour is unchanged.
- The tab icon changes to the full mark. Each page's head now carries
about 6.9 KB of base64 for it, against about 0.6 KB for the previous
SVG.
- The new code relies on no browser API newer than
`Array.prototype.includes` (2016).
- The two halves touch disjoint files, so either can be reverted on its
own.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

The inlined PNG was read chunk by chunk: standard chunks plus three
ImageMagick timestamps, and nothing after IEND. Inlining it adds no
third-party request, which hotlinking the CDN copy would have.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-09-23 22:09:48 +08:00
KT
f64f4c43e9
fix(ui-web): re-read a running dag node's record on a beat (#684)
## Summary

The desk task pane's context tab froze for a running dag node: it drew
the dispatch prompt and nothing else until the node settled, and the
subtitle's token and tool counts stood still with it (a user screenshot
shows a Raven-Design node at 4m17s with only the request on screen).

`useNodeRecord` re-read a dag node's record only on a status transition
or a `dag.node_updated` frame, on the premise that the frame fires once
per tool call. It does not: the runner emits it when a node starts and
when it ends (`raven/agent/subagent/dag_runner.py`), and the
`tool_call_id` on it names the parent turn's `run_subagent_dag` call,
not a step of the node's own. The server's live account already carried
every step (`dag.node` falls back to the run activity the ACP collector
republishes on every frame; `tasks.list` overlays live usage), and the
TUI already polls `dag.node` for a running node (`useDagNodePoll`). Only
the page's refresh rule was wrong.

Fix: extend the one-second beat a running spawn already reads on (#567)
to any running node, and correct every copy of the old premise: the
hook's comments, the store's `nodeVersions` note, `NodeRecord`'s answer
caret note, the design spec
(`docs/specs/2026-09-18-desk-tasks-list-design.md` section 4) and the
test that asserted a dag node is not re-read on the beat (now asserts
that it is, and that the beat stops on the terminal frame). No backend
or wire change.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Unit tests (worktree, `ui-web/`):

```
npx vitest run src/features/tasks     # 8 files, 219 tests passed
npx vitest run                        # 192 files, 2675 tests passed
npm run type-check                    # exit 0
npx eslint src/features/tasks         # exit 0
```

Mutation check: with the old gate restored (`row.kind === 'spawn' &&
node.status === 'running'`), the two new dag-node tests fail (2 failed /
73 passed in `TasksPage.test.tsx`); with the fix they pass.

Real gateway A/B, same gateway and backend code (`raven serve` from this
branch's venv on an isolated `RAVEN_HOME`), only
`ui-web/dist/index.html` swapped between the two runs. Each run: the
main agent (deepseek-v4-flash) dispatches one `run_subagent_dag` with a
single node on the `Raven-Code` ACP agent doing three sequential `sleep
15` shell steps; a Playwright driver opens the node's context tab once
the node is running and samples the DOM every 3 s without closing the
panel, beside `dag.node` / `tasks.list` for the same node:

| run | node ran | panel tool rows while running | server `dag.node`
tool rows | server `tool_call_count` | answer after settle |
| --- | --- | --- | --- | --- | --- |
| before (main) | 49 s, 17 samples | 0 throughout | 0 -> 1 -> 2 -> 3 | 1
-> 2 -> 3 | shown |
| after (this branch) | 46 s, 16 samples | 1 -> 2 -> 3, in step with the
server | 0 -> 1 -> 2 -> 3 | 1 -> 2 -> 3 | shown |

The server served the live steps in both runs; only the page's refresh
rule differed.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered (none: the page issues the same two
reads it already issues for a running spawn)
- [x] Backward compatibility considered (no wire change; one `dag.node`
and one `tasks.list(kind, id)` read per second per open running dag node
pane, the same cadence a running spawn pane already has)
- [x] Rollback path is clear for risky changes (revert the commit; the
page returns to reading a dag node on transitions only)

## Related Issues

#461 (the agents page's live view for a running sub-agent instance is a
different surface and stays open)

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-23 13:35:21 +08:00
Ethan Wang
78e41b2f8d
feat(*): a playbook mode that takes many rounds (#499)
## Summary

A third playbook mode. A `mode: stint` playbook declares roles instead
of nodes, and the driver compiles one sub-agent graph per round, so a
stint is not a sibling of a `dag` run but a layer over it: one fan-out
is one graph, and many rounds are one graph each. Thirty rounds is
thirty graphs on the shared dispatch path, which is what makes the
validation, the dispatch budget, the approval gate and the detached-run
bookkeeping the ones already there rather than a second set.

The stint is a file. Nothing depends on the process that started it
still being alive: a gateway restart, a machine restart, or a `raven
playbook stints resume` all pick it up, and the resume is node-granular,
so a round that died with two roles done re-runs the third.

**One noun, singular and plural.** The mode is `mode: stint`; one run of
such a
playbook is a stint, with its own id, checkout and record; the plural is
the
command family and the RPC namespace. `round` keeps its own meaning as
the unit
inside -- one pass of every role -- and `stage` as one role's turn in
it, so
"the stint is on round 7 of 30" says three different things with three
different
words. The whole surface sits under `raven playbook`: `stints` answers
what has
run on this machine, `stint` acts on one project's layout and the
backlog its
roles share.

**Handover is files, not a conversation.** Every role opens a fresh
conversation each round, which is a decision rather than a limitation:
which instance serves which role changes, processes restart, transcripts
get compacted. What a role learned this round reaches the next one only
by being written down. That makes the journal the contract -- an
append-only file, windowed by recent rounds and by characters, because a
forty-round journal would otherwise fill the prompt it exists to inform.

**The boundary is three layers.** `owns` and `appends` are declared in
the playbook, rendered into the role's prompt from the same declaration
the enforcement reads, measured by the role's own checks, and undone
afterwards with the file the role wrote kept under `violations/`.
Declaring without telling is a trap; telling without undoing is a
suggestion; and the undo reads the stage's own commits as well as the
worktree, because a role that commits a stray write disappears from the
working tree and a boundary any role can step around by committing is
not one. Nothing refuses a write before it lands: a charter narrows a
role's tools only where the playbook declares a `playbook:` block, and
`owns` is not derived into one, so the shipped playbook's roles have no
tool gate and the undo is the boundary rather than its backstop.

Two shapes the boundary cannot hold are refused where they are written
rather than left to surface as violations in round four:

- **Two roles that can run at once where either is enforced.** They
share one checkout, so the first judged is graded against the difference
the whole tree shows and quarantines the other's work. Observed before
it was refused: two roles with disjoint `owns`, each writing only what
it owned, and the first judged came back as `dev-a wrote 1 path(s) it
may not write: src/b/work.py`, with that file gone. The message names
both ways out, because which one is right is the author's call. What
would make it work is a checkout per concurrent role, which is a merge
problem, not a validation rule.
- **`enforce.read: hard`.** It reaches the guard text, which then tells
the role that reads outside its paths are refused at the tool gate, and
no gate refuses one. The wording stays in `rounds_prompt` for whoever
wires the gate; until then the grade is not offered.

**Checks come before any model's opinion.** `verify[]` runs real
commands, and a failure goes back to the role that caused it with the
failure text, up to `maxHandbacks`. When that budget runs out the round
records the failure and carries on rather than failing the node: a
failed node skips every role downstream, and a failed build is exactly
what the reviewer downstream has to see.

**One approval, and it names what it is approving.** The whole stint is
confirmed once, at the start, and the text states the round budget and
every command the stint may run -- a playbook is a distribution unit, so
a file somebody handed you does not get to run shell thirty times on one
unexamined yes. Every round is charged against the hourly dispatch
budget like any graph; a round the budget turns down pauses the stint
rather than failing it, and `stints resume` takes it up once the
allowance recovers. From the CLI the same approval is put to the
terminal, and with no terminal to ask at the run is refused rather than
approved on nobody's behalf.

A stint works a checkout of its own, cut from the project's head, so
hours of its commits do not collide with the conversation that started
it. `stop.maxRounds` defaults to 10 and is capped at 99. `stop.until`
lets a role end a stint early by reporting a marker on a line of its own
-- a whole line, so a role quoting the marker mid-sentence does not end
the plan -- and only a terminal role is told the marker, because a role
that knows how to stop the stint has a reason to reach for it.

**Surfaces.** `raven playbook stints
list/get/extend/pause/stop/resume/sweep/answer`, where `resume` and
`extend` are relayed to the raven serving the page when one is up, so a
round runs in the process that owns the conversation;
`playbooks.stints.*` RPC methods, the read-only ones reading the record
off disk so opening the page does not cost what a dispatch costs; a
Stint tab on the playbook page showing the roles, what each owns, the
checks with their full commands, and what carries between rounds; and a
Runs tab listing every stint with its standing, how far into its budget
it is, each round's checks and what was undone, with pause, stop and
resume.

`raven playbook stint` lays a project out for one. The shipped playbook
is `rounds`: a planner, a builder and a verifier, on any project that
has a specification. `init` writes `.stint/`: a guard file per role, the
shared prose, and a link to the project's own specification, chosen by
two sieves -- the file name first, then how much of the document says
something is required, so an empty `SPEC.md` stub does not send thirty
rounds off a blank page. `task`, `ask` and `confirm` work the backlog
the roles share.

New package `raven/stint/` carries the parts a stint needs that are not
the playbook: the record, its checkout, the ownership grades, the
journal window and the check runner. `CONTEXT.md`, `CHANGELOG.md` and
the two `docs-site` repo-layout tables gain their rows; `README.md` is
not touched.

Three unrelated changes ride along. Two are the same shape -- a web
change that does not show:

- `index.html` went out with no caching header, so browsers fell back to
heuristic freshness and served a rebuilt page from cache without asking,
which reads exactly like a server that never picked the change up.
`no-cache` keeps the entry and revalidates; the ETag still saves the
transfer.
- `make ui` is one target for the two-step page build, because running
only vite leaves the served one-file page unchanged with nothing to say
so.

The third is `raven agent` (`raven/cli/agent_commands.py`):
`--message-file` reads the prompt from a file, `--permission-mode` picks
the tier for the run, and `--no-markdown` prints a soft-wrapped `Text`
rather than a markdown render; 92 lines of tests in
`tests/test_cli_agent_commands.py`.

Designed and deliberately not built, so a reviewer knows where the edges
are: a checkout per concurrent role with each role merging itself in
when it is judged (`docs/plans/2026-09-18-stint-acceptance.md`), and the
backlog-writing round zero. `raven playbook stint confirm` records a
person's word on the plan and nothing in the rounds path reads the flag
yet, which that document also says.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run ruff check --no-cache raven evolver agents plugins-dist tests scripts
uv run ruff format --check --no-cache raven evolver agents plugins-dist tests scripts
uv run lint-imports
make lint-types
make lint-ui
make lint-tui
uv run pytest -q
uv run python scripts/check_commit_messages.py main..HEAD
uv run python scripts/check_source_language.py main..HEAD
uv run python scripts/check_large_files.py main..HEAD
```

Ruff clean over 2,004 files. Import contracts: 10 kept, 0 broken. `ty`
clean, which caught one real defect on the way in: `StintStore.list`
shadowed the builtin for every annotation after it in the class body, so
`-> list[StintRecord]` on it and on `live()` resolved to the method. The
UI gates include the generated-file drift checks (`gen:check`,
`lint:rpc`, `lint:i18n`) and both type-checks; eslint reports 23
warnings and 0 errors, none of them in a file this branch touches. `make
lint-bridge` was not run: this checkout has no `bridge/node_modules`,
and nothing here touches `bridge/`.

Suite: the whole `uv run pytest -q` runs in CI as four shards, and every
shard is green at this head, with the import-cycle budget (44 modules,
the same as `main`), the coverage gate and the three content gates over
the commit range. Locally the suites this branch touches are run per
change; the full run is CI's.

Behaviour, driven rather than asserted:

- A stray write by a concurrent role was reproduced end to end before
the validation refused the shape: `dev-b's own file still there after
dev-a was judged: False`, `violations recorded against dev-a: ['dev-a
wrote 1 path(s) it may not write: src/b/work.py']`.
- The check runner's timeout kills the process group, not just the
direct child. Proved as a regression guard: red with `process.kill()`,
green with the group teardown.
- The round budget refuses rather than wrapping: extending a real stint
past the ceiling answered `that would give it a budget of 104, and 99 is
the most any stint may have`, and the file was not written.
- The spec picker's two sieves, over a real project: `KEPT docs/PRD.md |
weight 12`, `dropped SPEC.md | len 12 | weight 0`, `dropped notes.md |
len 868 | weight 0`.
- Only a terminal role is told the stop marker: `planner=False
builder=False verifier=True` on the shipped example.
- The shipped example is discoverable with no user layer: `listed:
['rounds']`, `origin: builtin`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Additive. `mode: stint` is a new value; `dag` and `prompt` playbooks
load, validate, dispatch and report exactly as before, and the sections
the new mode reads are withheld from the generator's emit schema, so no
generated playbook can reach them. That withholding is pinned by a test
that fails when a section is added to the spec without being added to
the refusal.

Nothing outside the new mode changes behaviour: `spawn` and the DAG path
keep the per-session in-memory dispatch window they have on `main`.

A plan runs shell from `verify[]`. It is gated on the approval, the
approval text lists every command, and a playbook carrying `verify`
cannot be generated -- only written by hand or shipped. The builtin
layer ships exactly one playbook, `rounds`, pinned by name in a test,
for that reason: withheld from the generator and not shipped either, the
mode would be reachable only by transcribing YAML out of prose.

Rollback is a revert of the merge commit. A plan's on-disk record is
under the sub-agent history root and is read by nothing else; nothing
else's on-disk shape changes.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Ethan Wang <ethan.wang@evermind.ai>
Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-23 12:24:08 +08:00
KT
e5f5558347
feat(*): step through an ask_user batch as one form, with back, multi-select and one submit (#672)
## Summary

The web page renders an `ask_user` batch as one form the reader steps
through, instead of one sheet per question with no way back.

Today the question broker emits one `clarify.request` per question and
the tool loop awaits them one at a time, so the answer to question 1 is
settled on the server before question 2 is even shown: the page could
not offer "back" no matter how it drew the sheet. The page also ignored
`header`, `recommended`, `index`, `total` and `batch`, so a
three-question call arrived as three unrelated sheets.

What changes, mirroring Claude Code's `AskUserQuestion`:

- A lone single-select question still answers on the pick. A
multi-select question (new per-question `multi_select` flag on the tool)
shows check boxes and needs Submit. A batch is a form: step chips (the
question headers) at the top, Back / Skip / Next per step, a
single-select pick advances to the next step, the last step holds its
pick and reports the whole form with Submit. The recommended option is
marked. The form survives a conversation switch.
- Wire contract, all backward compatible: `clarify.request` carries
`multi_select`, and the `batch` entries the `ask_user` tool builds carry
`choices` / `recommended` / `multi_select` so a surface can draw every
step up front; `clarify.respond` accepts an optional `answers` list
aligned with `batch`. The broker resolves the pending question from
`answer` as before and stashes the later answers; the tool loop's later
`await_question` calls for the same batch are answered from the stash
with no further frame. A producer whose batch entries carry no `choices`
(an ACP elicitation form) is still asked one question at a time, and the
TUI and the chat channels keep answering one question at a time. Every
in-repo `QuestionResponder` (the RPC broker, the routing shim and the
A2A broker) takes the widened keyword set, and a test walks the tree so
the next implementation is held to it.
- `ask_user` gains `multi_select` on each question; the answer reaches
the model as the chosen labels joined with `, `.

Not in this PR: check boxes in the TUI, a multi-select hint in chat
channels, and richer batch entries from the ACP elicitor.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Python (worktree venv):

```
uv run pytest tests/test_ask_user_tool.py tests/test_question_broker.py tests/test_a2a_asking.py tests/test_rpc_schema_match.py tests/test_rpc_contract_shapes.py tests/test_acp_ask_user.py tests/test_acp_elicitation.py tests/test_cli_gateway_commands.py tests/test_agents_research_flow_ask_user_surface.py -q
make lint-python && make lint-imports && make lint-types
make test-python
```

- targeted files: 649 passed (plus 9 in
test_contracts_two_tier_ledger.py); lint, import and type gates exit 0;
`make test-python`: 25110 passed, 109 skipped.

Web and TUI:

```
cd ui-web && npx vitest run && npm run type-check && npm run lint && npm run gen:check
cd ui-tui && npm run lint && npm run lint:rpc && npm run lint:i18n && npm run type-check && npm test
```

- ui-web: 193 files / 2678 tests passed including the gates under
scripts/gates (class namespace, css ownership, i18n keys, notification
table); type-check clean; lint 0 errors (4 pre-existing warnings in
files this PR does not touch); gen:check in sync.
- ui-tui: lint, lint:rpc, lint:i18n, type-check and the test suite all
exit 0; the regenerated i18n table and rpc types are committed.

Real gateway, driven with playwright against an isolated `RAVEN_HOME` on
a `deepseek-v4-flash` session that really calls `ask_user`:

- before: a three-question call docked three sheets one after another,
no chips, no way back;
- after: one sheet for the whole batch (a MutationObserver counted one
dock), pick on step 1 advances, Back returns with the pick still held,
the multi-select step joins `cheese, onion`, Submit on the last step,
and the model echoed `ANSWERS: red | cheese, onion | Zed`; a lone
single-select answers on the pick; a lone multi-select needs Submit and
echoed `monday, wednesday`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible: a multi-question `ask_user` on the page becomes one
stepped form; digit shortcuts on the sheet no longer fire while any text
field has the caret (they used to fire from the composer). Old frames
and old clients keep working: every new field is optional or additive,
`clarify.respond` without `answers` behaves exactly as before, and the
TUI reads the same frames unchanged. A batch whose entries do not carry
`choices` falls back to one question at a time. Rollback is a revert of
the squash commit; nothing is persisted in a new shape.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-23 10:32:44 +08:00
KT
da13bddd48
fix(*): the desk lists a task's files as changes and hands over only what was delivered (#656)
## Summary

A spawned sub-agent that wrote ten markdown files showed up on the desk
as ten "Task output" rows on the deliverables tab and a diff tab saying
"No changes this turn". Two things were wrong at once: the deliverables
tab listed files nobody delivered, and the diff tab hid every whole-file
write a task made (it only took `op === 'edit'` task files).

- The diff tab now lists every file a task's nodes wrote or edited,
under a "Task changes" heading; its badge and the tab the desk opens on
count them too, under the same id `openDeskDiff` marks read.
- The deliverables tab lists only what `deliver_files` handed over. A
sub-agent has no `deliver_files`; the main agent decides what is a
product, which is what the system prompt already tells it.
- A file the turn created draws `+` (kind `add`) instead of `M`: live
from `tool.complete.file_change` without `before`, on reload from the
stored diff's `@@ -0,0` header. A whole-file write over an existing file
stays `M`, and the transcript's "New" badge follows the same rule
instead of calling every whole-file write new. A created file rewritten
in the same turn stays a creation.
- Task-file glyphs: an `edit_file` is always a modification (the
prototype rule from #509 drew `+` for an edit that deleted nothing); a
whole-file write that replaced nothing is a creation.
- Labels: "N products" on a task row becomes "N files changed"; "Task
output" becomes "Task changes". The TUI i18n table is regenerated; the
desk tasks spec (section 7) says where task files go now.
- Second commit: `prettier --write` on two ui-tui files that arrived on
main unformatted. The pre-commit prettier hook checks the whole ui-tui
tree whenever a file under it changes (this PR regenerates the i18n
table), so it fails until those two are formatted.

Not in this PR: a file deleted by `exec rm` still leaves no row
anywhere. A follow-up branch (`feat/track_file_deletions`) adds deletion
tracking on the backend and the red `D` row; it is stacked on this one.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run from the worktree after rebasing onto `a48c4edf`:

- `npm test --prefix ui-web` -- 192 files, 2599 tests passed (the whole
suite, gates included)
- `npm run lint --prefix ui-web` -- 0 errors (4 pre-existing warnings);
`npm run type-check --prefix ui-web`; `npm run gen:check --prefix
ui-web` -- generated.ts matches the contract
- `npm run lint --prefix ui-tui` -- 0 errors (23 pre-existing warnings);
`npm run type-check --prefix ui-tui`; `npm run lint:i18n --prefix
ui-tui` -- generated catalogue is up to date
- `uv run pre-commit run --from-ref origin/main --to-ref HEAD` -- passes
with the second commit (fails on `prettier (ui-tui)` without it)
- `make lint-python`; `make test-python` -- no Python file changes; both
pass
- Real gateway: `raven serve` from this branch with an isolated
`RAVEN_HOME` (channels off, default model `deepseek/deepseek-v4-flash`),
one conversation that spawns the in-process `Raven` sub-agent to
`write_file` two files and `edit_file` one, one conversation whose main
agent `write_file`s a new file. Before: diff tab "No changes this turn",
deliverables tab listing both task files under "Task output", the main
agent's new file drawn `M`. After: diff tab lists both task files with
`+` under "Task changes", deliverables tab "Nothing produced yet", the
main agent's new file drawn `+`. Same two conversations reopened after a
reload read the same.
- Mutation check on the new tests: dropping task ids from
`idsOf('diff')`, reverting `taskChangeKind` to the deletion-count rule,
dropping the `add` mark in `record.ts`, and widening the transcript's
"new" back to every write each turn at least one new test red.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

UI only; the wire is untouched. Files a task wrote leave the
deliverables tab (they were never delivered) and appear on the diff tab,
so a reader who relied on the shelf to open a sub-agent's output now
finds it under diff, or in the task pane's file chips as before. Two
i18n keys are renamed (`gui.ws.task_output` -> `gui.ws.task_changes`,
`gui.tasks.artifacts_n/_1` -> `gui.tasks.files_n/_1`); no other consumer
of them exists. Rollback: revert the two commits.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 20:06:46 +08:00
KT
ecda77313e
fix(ui-web): a settled node's result-less call reads as no result, not never run (#655)
## Summary

Follow-up to #636, addressing the review thread left open on it
(NodeRecord.tsx, the settled result-less call).

#636 drew a settled node's result-less calls as "not run". That claimed
more than the record knows: `result == null` only says no result frame
was recorded. On the acp lane a `tool_call` frame means the call was
initiated (Raven's live emitter sends it with `status: in_progress`), so
a node cancelled after an `exec` or a write began may have finished it,
side effects included, before the cancel landed; on the in-process lane
a round's calls are advertised before they are reached, so some really
never ran. The wire carries no pending or not-started state that would
tell the two apart, and the transcript's shape is the same for both.

The settled label now says only what is known: `no result` on the row,
`{n} with no result` on a folded step's summary, faint and not breathing
as before. The classes and catalogue keys are renamed to match
(`tknoresult`, `gui.tasks.call_no_result`,
`gui.tasks.call_no_result_n`), the TUI's generated catalogue follows,
and the design note's node-context sentence states the uncertainty.
Carrying an explicit per-call state so the two cases could be told apart
would be a wire change and is not attempted here.

One more commit formats two `ui-tui` files
(`src/__tests__/approvalRoundTrip.test.ts`,
`src/components/prompts.tsx`) that #650 left unformatted: the `prettier
(ui-tui)` pre-commit hook checks the whole tree whenever a PR touches
any `ui-tui/src` file, so regenerating the catalogue here would
otherwise fail on drift this PR did not cause. Formatting only.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run in the fix worktree, base `origin/main` at a2db9ac6:

- `npm test --prefix ui-web -- src/features/tasks`: 8 files, 206 passed
(the settled-call tests now assert the new class and words; the two that
read the label fail against the pre-change component, the third is the
still-running guard and is unchanged).
- `npm run lint --prefix ui-web`: 0 errors, 4 pre-existing warnings in
other domains. `npx tsc --noEmit -p ui-web`: clean.
- `npm test --prefix ui-web -- scripts/gates`: 38 files, 164 passed.
`npm run lint:i18n --prefix ui-tui`: generated catalogue up to date.
`npm run build --prefix ui-web && python3 ui-web/build.py`: boot
snapshots match their golden (241 nodes).
- `grep -rn "call_not_run\|tknotrun" ui-web/src i18n ui-tui/src
docs/specs`: no stale references.
- `cd ui-tui && npx prettier --check "src/**/*.{ts,tsx}"`: all matched
files use Prettier code style (it failed on the two #650 files before
the formatting commit).
- `make lint-python`: clean. `make test-python` (`uv run --frozen
--python 3.12 --all-extras pytest -q`): 24965 passed, 109 skipped. In
`ui-tui`: `npm run lint`, `npm run lint:rpc`, `npm run type-check`, `npm
test` (2071 passed), `npm run build`. `npx commitlint --from origin/main
--to HEAD`, `scripts/check_commit_messages.py`,
`scripts/check_large_files.py`, `scripts/check_source_language.py`,
`pre-commit run --from-ref origin/main --to-ref HEAD`: all pass.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
(docs/specs/2026-09-18-desk-tasks-list-design.md section 4)

## Risk

- [x] Security impact considered: none; a label and class rename.
- [x] Backward compatibility considered: no wire change. The two
catalogue keys introduced by #636 are replaced, with their only
consumers updated in the same change; nothing else read them.
- [x] Rollback path is clear for risky changes: revert the squash
commit.

## Related Issues

N/A. Closes the open review thread on #636.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 19:08:34 +08:00
KT
a2db9ac6e8
fix(*): make the context tab's process trajectory agree with its tool count (#636)
## Summary

Opening a spawned sub-agent in the desk tasks pane, the context tab's
process fold did not agree with the tool count, and a run whose model
returned no reasoning text read as one opaque fold. Four causes, one
commit each; the folded summary row (one row per step, calls behind it)
is the prototype's design and is unchanged.

1. ACP lane, while running: the collector republished the live
transcript on every update but counted the calls only in `_record`, at
the end of the turn. `tasks.list` reads a running node's tool count off
the live account, so the board chip stayed blank while the context tab
already listed the calls, then jumped to N. `activity.set_tool_calls` is
the named-run counterpart of `set_transcript` (replaced, not appended,
because the opening `tool_call` frame names a call by its title and a
later frame carries the arguments its label is built from); the
collector publishes it on every tool frame and `_record` writes the
settled list over it.
2. ACP lane, settled: the whole reply joins every note the agent said on
the way, and the transcript already carries each note on the step it
preceded. `subagent.context` and `dag.node` appended the whole reply as
the answer row, so each progress note read twice. The record now keeps
the closing message (what the run said after its last step, CONTEXT.md)
as `<node_id>.closing.md` beside `out.md`: `SpawnRecord.finish` writes
it for a spawn when the lane reported one, the dag runner on every
attempt (empty when none, since the id is reused across attempts). Both
context reads draw it as the answer row when present and fall back to
the whole output otherwise. `out.md` is unchanged; `dag.node` reads the
field into the answer row and keeps it off the detail, so the wire is
unchanged.
3. In-process lane, stopped mid-round: the round's assistant message
advertises every call before they execute, so a run cancelled or aborted
mid-round leaves a record with calls it never reached. The process fold
keyed its breathing icon on the call having no result and drew those
rows in flight forever. A call with no result is in flight only while
the node runs; on a settled node it reads as not run (plain verb, faint
row, a `not run` chip, no detail card) and a folded step counts them
beside its failures. Two catalogue keys, the TUI's generated copy, the
design note's node-context section.
4. A thought-free run folded into one group. `groupSteps` opened a new
step only on a thought, so a run whose model returned no reasoning text
(reproduced on a real gateway: deepseek-v4-flash through the provider
bills reasoning tokens and hands no reasoning text back, 0 thought
frames over 15 tool calls) folded every call into one group with all its
narration piled above them: a 16-call research run read as one summary
row and nothing else. Narration that follows calls now opens a step too,
so such a run reads as one step per thing it said before calling;
narration after a thought stays in that thought's step, as before. The
design note says so.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run in the fix worktree, base `origin/main` at 096f1b03:

- `uv run pytest tests/test_subagent_acp.py
tests/test_subagent_activity.py -q`: 231 passed (2 new; the live-count
test fails on the pre-fix source).
- `uv run pytest tests/test_subagent_history.py
tests/test_rpc_subagent_calls.py tests/test_rpc_dag.py
tests/test_subagent_dag_runner.py tests/test_rpc_tasks.py -q`: 427
passed; 3 failed only in the plain venv without the everos extra
(`test_a_node_leaves_a_memory_record`,
`test_a_node_passes_its_instance_to_the_record`,
`test_an_outer_cancellation_reaps_this_runs_memory_pollers`),
identically on the unmodified base, and they pass under `make
test-python` below. Eight tests are new; six of them fail on the pre-fix
source.
- `npm test --prefix ui-web -- src/features/tasks`: 8 files, 201 passed
(5 new; three fail on the pre-fix component: the two not-run tests and
the narration-opens-a-step test).
- `npm run lint --prefix ui-web`: 0 errors, 4 pre-existing warnings in
other domains. `npx tsc --noEmit -p ui-web`: clean.
- `npm test --prefix ui-web -- scripts/gates`: 38 files, 164 passed.
`npm run lint:i18n --prefix ui-tui`: generated catalogue up to date.
`npm run build --prefix ui-web && python3 ui-web/build.py`: boot
snapshots match their golden (242 nodes).
- `make lint-python`: clean. `make test-python` (`uv run --frozen
--python 3.12 --all-extras pytest -q`): 24920 passed, 109 skipped. In
`ui-tui`: `npm run lint`, `npm run lint:rpc`, `npm run type-check`, `npm
test` (143 files, 2068 passed), `npm run build`. `npx commitlint --from
origin/main --to HEAD`, `scripts/check_commit_messages.py`,
`scripts/check_large_files.py`, `scripts/check_source_language.py`,
`pre-commit run --from-ref origin/main --to-ref HEAD`: all pass.
- Real gateway (`raven serve` from this branch on an isolated
`RAVEN_HOME`, one Raven-Code spawn over acp asked to narrate before each
of three `sleep 12` commands, `tasks.list` and `subagent.context` polled
every 2 s over RPC): while the row was running the node's
`tool_call_count` read null, 1, 2, 3 in step with the context's call
rows (2, 4, 6 messages), and settled at 3 with 3 call rows in the
transcript. Settled, the answer row was the closing sentence plus `FINAL
alpha beta gamma`; none of the three narration rows opened it;
`<node_id>.closing.md` equalled it and `out.md` still held narration and
answer. The not-run rendering (3) is covered by the component tests and
the boot snapshot only; no real-gateway pass exercised a mid-round
cancel.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
(docs/specs/2026-09-18-desk-tasks-list-design.md section 4, CONTEXT.md
Closing Message)

## Risk

- [x] Security impact considered: none; the change publishes a count
already collected into an in-memory index the same readers already
serve, writes one more file beside a record's existing ones, and changes
how the page draws a row it already drew.
- [x] Backward compatibility considered: no wire change (`TaskNode`,
`SubagentContextResult`, `DagNodeDetail` unchanged). A record written
before this change has no `closing.md` and reads exactly as before. The
two new catalogue keys have consumers in the same change.
- [x] Rollback path is clear for risky changes: revert the squash
commit; the `closing.md` files it wrote are ignored by the reverted
reader.

## Related Issues

N/A. #461 (a running sub-agent's tool calls are stored but never
streamed) is the instance surface's version of the same class of gap and
is not closed by this.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 18:35:11 +08:00
zhanghui
77f5ffc4a7
fix(plugins): the role rules hold at every door, not one each (#633)
## Summary

Follow-up to #614, which merged with a blocking finding open. Two
consequences
of one shape, both confirmed by reproducing them on the merged branch
before
anything was touched: a rule enforced at one door while another door
reaches
the same operation.

**`clear_role` never consulted `REQUIRED_ROLES`.** The RPC door did, and
refused
to clear `llm` or `embedding`. The wizard reached the same function
through its
own older answer, whose table says `optional: true` for embedding --
which
answers "may be left unset", not "may be erased". Clicking Skip on a
configured
embedding role called `set_embedding_endpoint({"model": "", "provider":
""})`
and emptied raven's own top-level block, the one endpoint every
knowledge base
embeds with. Reproduced:

```
before                     {'model': 'text-embedding-3-small', 'provider': 'openrouter'}
CONTROL  clear_role(rerank)      block unchanged
SUBJECT  clear_role(embedding)   not refused; block = None
                                 log: "config/update: embedding endpoint cleared"
```

The rule now lives with the operation. `clear_role` raises
`RoleRequiredError`;
the RPC translates it instead of restating it, so there is one rule
rather than
two copies that drifted; the wizard stops offering Skip for a role that
is set
and required, while still offering it for one that is merely unset. A
test that
asserted the old behaviour is replaced -- it pinned the defect as the
contract.

**`restart_for_config_change` never asked `everos_owned`.**
`_require_owned`
sits on the write primitives "so a new caller cannot opt out"; the
restart is a
new caller and is not a write primitive, so it did. The embedding role
is exempt
from the ownership gate by design, so a save of it on a user-managed
root fell
through to `stop_for_reload` -> `stop_pid`, which SIGTERMs whatever
`lock_holder`
names for that root: the user's own server. Reproduced as far as the
exemption,
which is the part that can be driven without harming a running service:

```
owned=False   llm  rerank  multimodal -> REFUSED (EverosRootNotOwnedError)
              embedding                -> got past the ownership gate
```

That contradicts this feature's own written rule three times:
`everos_owned`'s
docstring ("never start or stop the process"), the file header, and C13
in the
design document. The chain now refuses before it stops anything, and
says the
root is theirs to restart.

Also, smaller, all named in the same review: the design document used
"role
block", which the CONTEXT entry it ships beside puts on its `_Avoid_`
list; the
vendor table spelled the rerank protocol `rerank_provider`, the word
that entry
says to avoid, while EverOS's own `rerank.provider` field and the
`EVEROS_RERANK__PROVIDER` variable keep their names; and A18 stated a
criterion
that is false about the code, since the protocol legitimately reads
`deepinfra`
and `dashscope` where it belongs.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run --frozen --python 3.12 --all-extras pytest
tests/test_everos_config.py tests/test_everos_backend.py
tests/test_everos_server.py tests/test_everos_discover.py
tests/test_config_update_everos.py tests/test_config_update_providers.py
tests/test_plugin_boundary.py tests/test_rpc_contract_shapes.py
tests/test_rpc_settings.py tests/test_cli_onboard_commands.py -q` --
1018 passed, 1 skipped
- `ruff check raven/ tests/ plugins-dist/` -- all checks passed
- `ruff format --check raven/ tests/ plugins-dist/` -- 1839 files
already formatted
- `make check-source-language` over this range -- clean

Each fix was mutated back to the defect to confirm the new cases bite:

- `clear_role` stops consulting `REQUIRED_ROLES` -- reddens the config
case and
the RPC case, which is what says the RPC now depends on the one rule
rather
  than carrying a second copy
- the wizard offers Skip for a required role again -- reddens the wizard
case
- the restart stops asking `everos_owned` -- reddens the unowned-root
case

Each mutation was reverted and the file hash compared afterwards.

## Risk

User-visible: Skip disappears from the wizard menu for an embedding role
that is
already configured, and a save of the embedding role against a
user-managed root
now reports that the server was not restarted instead of restarting it.
Both are
the behaviour the feature's own documents already describe.

Rollback: one commit, reverts cleanly.

## Related Issues

Follow-up to #614.

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-22 16:29:02 +08:00
userName20260323
8a1ebdc577
docs: organize bilingual documentation navigation (#635)
## Summary

Reorganize the Raven documentation landing page and expand the bilingual
documentation site.

- Add bilingual guidance for agent collaboration, ACP and A2A protocols,
DAG orchestration, channels, browser collaboration, knowledge bases,
Playbooks, proactive follow-ups, Oncall, permissions, tracing,
trajectory replay, and Evolver.
- Group daily usage under `Use Raven` and `Using Raven`.
- Group `Technical Reference` and `Developer Guide` under `Explore the
documentation` while preserving separate cards.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [ ] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Exact checks:

- `uv run --frozen --group docs mkdocs build -f docs-site/mkdocs.yml
--strict` - passed.
- `uv run --frozen --group docs pytest tests/test_docs_site.py
tests/test_living_docs.py
tests/integration/test_docs_site_controls_e2e.py -q -o addopts=''` - 77
passed.
- `uv run --frozen python scripts/check_commit_messages.py
origin/main..HEAD` - passed.
- `git diff --check origin/main...HEAD` - passed.
- `make check-large-files` - passed.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

This is a documentation-only change. It does not change runtime
behavior, configuration semantics, or public APIs. Reverting the
documentation commits restores the previous site content and navigation.

## Related Issues

N/A

---------

Co-authored-by: userName20260323 <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Claude (claude) <noreply@anthropic.com>
2026-09-22 16:14:44 +08:00
zhanghui
96d3b888ad
refactor(plugins): serve the four EverOS memory roles from raven's own config (#614)
## Summary

EverOS's four memory roles -- llm, embedding, rerank and multimodal --
lived in
`~/.everos/raven/everos.toml`, a file raven wrote and EverOS read once
at startup.
Three defects followed from that arrangement and none was fixable in
place:

- a model changed on the settings page did not take effect until
something
  unrelated restarted the service, with nothing on screen saying so;
- the rerank section needs a request shape (`deepinfra` / `vllm` /
`dashscope`)
that only the wizard's own vendor table knew, so reranking configured
from the
settings page posted to the wrong path against DeepInfra and
SiliconFlow;
- an EverOS root the user manages is read-only to raven, so those users
had no
  way to configure a role at all.

raven now records a **pin** per role -- a model and the vendor serving
it, never a
credential -- and resolves the address and key from that vendor at the
moment they
are needed. Rotating a key is one edit, and every role on that vendor
follows.
The values reach EverOS as `EVEROS_*` variables, emitted on every spawn
and bound
into raven's own process so the in-process multimodal tool reads the
same models
the service does. `everos.toml` keeps its runtime knobs, which EverOS
merges per
key; raven writes only `[api]` there now.

**A save applies itself.** The spawn precheck runs before anything is
stopped, so a
restart that cannot succeed leaves the old server serving; a stop that
does not
finish never falls through to the spawn, which would otherwise adopt the
old
server and report success for a configuration that never took. The
outcome
reaches every session waiting on it as `memory.health` -- success
included, since
that frame is the only thing that clears the standing banner.

**Upgrades migrate themselves.** An install upgrading into this still
has its roles
in the file. The migration runs on the first memory start: it names the
vendor
from the address and the key, creates the provider row when raven
carries none,
and leaves a role it cannot name unset with a notice rather than
guessing. It is
idempotent and never edits `everos.toml`. Measured against the real
shape on a
maintainer's machine, where two of the four roles point at DeepInfra, a
vendor
raven carries no provider spec for.

Key decisions, recorded in
`docs/specs/2026-09-21-everos-role-config-design.md`:

- the pin holds no credential, so nothing on the settings wire carries
one in
  either direction;
- an operator who exports `EVEROS_<ROLE>__*` still outranks raven, and
the slot
  says so rather than accepting a save that would never take effect;
- the rerank request shape comes from the vendor table where it is
known, and is
recorded on the role only for a self-hosted endpoint no table can answer
for;
- the migration is scheduled from the plugin, not from raven's config
migrations,
because the host may know this plugin only through the plugin contract.

Rebased onto `main` after #613, which settles the lint and contract
drift the
branch replay left behind. This branch briefly carried its own partial
fix for
that; it is dropped, and #613's is the one in force.

## Type

- [x] Refactor

## Verification

```
uv run --all-extras pytest -q
# 24517 passed, 4 failed, 111 skipped -- the 4 are pre-existing, confirmed by
# checking out the branch base in the same environment and re-running them there

uv run pytest <the 28 files that reference the changed modules> -q
# 1516 passed, 1 skipped

npm --prefix ui-web test          # 2496 passed, 191 files
npm --prefix ui-web run gen:check # generated.ts matches the contract (192 methods)
npm --prefix ui-tui run lint:rpc  # in sync
npm --prefix ui-tui run lint:i18n # catalogue up to date
uv run ruff check raven/ plugins-dist/ tests/        # All checks passed
uv run ruff format --check raven/ plugins-dist/ tests/  # 1835 files already formatted
make check-large-files            # exit 0
```

A real gateway was started on a fixture built from a maintainer's own
config in
the pre-upgrade shape. The migration moved all four roles, created the
`deepinfra` provider row, resolved reranking to `/v1/inference` rather
than the
chat endpoint, and left `everos.toml` byte-identical (sha256 unchanged).
That run
is also what found the defect described in the next section.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The remaining real-host acceptance cases in
`docs/specs/2026-09-21-everos-role-config-acceptance.md` have not been
run yet;
they are the next step and will be reported on this PR.

## Risk

User-visible changes:

- the four role slots on the settings page now show the vendor as stored
rather
than deriving it from an address, and offer every vendor that can serve
the
  role;
- a save restarts the memory service, which takes a few seconds and
reports its
  outcome in the standing banner;
- a role whose endpoint comes from exported `EVEROS_*` variables is
read-only,
  and so are all four on a root the user manages;
- reranking on a self-hosted endpoint asks for its request shape once.

Rollback: revert the branch. The migration only ever adds to raven's
config and
never edits `everos.toml`, so an install that has migrated still has its
original
file and falls back to it.

One defect worth naming, because it was invisible to every unit test: an
embedding role with no width pinned emitted
`EVEROS_EMBEDDING__DIMENSIONS=""`,
which EverOS's own settings model refuses as an integer -- the server
exited
before serving anything. Fixed, and the case that catches it now builds
EverOS's
real `Settings` from the emitted environment in a subprocess, which is
the layer
that owns the fact.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-22 15:43:40 +08:00
zhanghui
15c93d3989
feat(*): type the approval request and let no approval wait forever (#617)
## Summary

The approval prompt used to reach the page as one English sentence plus
a truncated argument line, with a 30 s countdown the web never drew and
a 35 s deadline that silently retired the sheet. This makes the request
typed and lets the sheet be drawn from facts instead of prose:

- `approval.request` now carries `kind` (shell.exec / file.write /
mcp.call / unknown), `family` (the shell command family that words the
prompt), `origin` (the main agent or a named sub-agent) and `evidence`
(the tool's own account of the call: the command and its directory, a
path and the diff the write would make, an MCP tool and its input). Each
tool describes itself through the new `Tool.approval_kind` /
`Tool.approval_evidence`; the gate only reads them, and caps what they
return, so no tool can send more than a person reads. `created_at`,
`expires_at` and `action_digest` leave the frame: nothing read them.
- No pending approval can block forever, and none is lost silently. The
broker serializes requests per conversation (a background sub-agent
asking while the main agent's prompt is up waits its turn instead of
replacing it), keeps a day-long floor instead of the 35 s ceiling, and
answers `approval.pending` so a reloaded or reconnected page draws the
open requests again; a request in another conversation lights the line
above the composer. No sweep takes a pending approval down unanswered: a
clarify ends on its own after ten minutes and a confirm on its own hard
limit, so both may be replaced, while the gate's ask has only that floor
under it. The TUI countdown and its expiry state are removed.
- The runtime and the ACP editor declare the same seven shell command
families through one call, so the terminal, the page and the channels
word a delete as a delete.
- Web sheet redesign: title and why-line by family, evidence block by
kind, three answers (deny, always with the suggested rule shown and
where it applies, allow once), a landed line after the answer, undo for
a saved rule (`approval.revoke`), a sentence field after a refusal that
goes on as the next message, and a line above the composer when another
conversation is waiting.
- A rule that reached the config file gets no session grant beside it,
so taking the rule back really does mean being asked again -- unless the
user's own stricter rule outranks it (the strictest matching rule wins),
in which case the session grant stays and the identical call is not
asked twice.
- Recovery and undo are scoped to the surface that asked.
`approval.pending` and `approval.revoke` each answer only for the
conversations a connection owns, plus the unclaimed ones whose frames
are broadcast anyway, so an id alone cannot authorize a command or
delete a rule on a conversation that socket was never shown. The undo
names the answer rather than the rule's text and waits for the gate to
report what its write did, because `approval.respond` answers the page
before the gate persists. A removal that fails on the way to disk leaves
the receipt unspent, so it can be pressed again; the line that reports
the failure names the rule.
- The landed line says so when the engine did not take the answer
(`approval.respond` returned `ok: false` or the socket dropped); a digit
typed into a field is text, not an answer; and the sentence field stays
up while it is still being typed into.

This replaces PR #608, which targeted `refactor/ui_web_architecture`.
That work is now on `main`, so the five commits were replayed onto it
and #608 was closed.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- Rebased from `refactor/ui_web_architecture` onto `main` (`git rebase
--onto`), then onto the newer `main` tip again. One conflict, in
`raven/core/runtime.py`: `main` had rewritten the hooks block this
change appends after, resolved by keeping both. The generated web client
was regenerated rather than hand-merged.
- `uv run --frozen --python 3.12 --all-extras pytest -q` -- full suite.
3 failed, 24615 passed, 111 skipped in 4:40. The three are pre-existing:
a `git worktree` at the `main` tip with none of this change runs the
same three files and fails the same three (`3 failed, 95 passed`).
- `cd ui-web && npx vitest run` -- 189 files / 2482 tests pass; `npx tsc
--noEmit -p tsconfig.json` clean; `npm run gen:check` in sync (194
methods); `node scripts/check-class-namespace.mjs` clean; `npm run build
&& python3 ui-web/build.py` green (both booted-DOM goldens match at 236
nodes).
- `cd ui-tui && npx vitest run` -- 143 files / 2068 tests pass; `npm run
type-check`, `npm run lint:rpc`, `npm run lint:i18n` clean.
- `scripts/check_source_language.py` and `scripts/check_large_files.py`
clean over this PR's range; `npx commitlint` over the same range exits
0.
- Five independent review passes over the finished branch (completeness,
correctness, appropriateness, maintainability, evidence). Every
behavioural finding is fixed and listed under Risk; each fix was re-run
against the code with the fix removed, and fails there on the symptom it
names.
- Real host: `raven serve` on an isolated `RAVEN_HOME` in ask mode with
deepseek-v4-pro, driven with playwright-cli; the TUI from source in tmux
against the same home. Delete: titled as a delete, allowed once, files
gone. Edit: diff shown, refused, note typed after the refusal arrived as
the next message, file untouched. Always-allow wrote
`permissions.tools.exec["glab mr list *"]`, undo removed it, and a rule
the reader had written themselves was left alone. A pending approval in
another conversation shows the waiting line; clicking it switches back;
a reload draws the sheet again. A main-agent request and a background
sub-agent request two seconds apart arrived one after the other, never
side by side.
- The real-host runs found two things the unit tests could not: the dock
passes pointer events through, so the waiting line needed
`pointer-events: auto` before a real click reached it; and the waiting
line filtered out a conversation whose very first turn was blocked,
which is not on disk yet and so not in the list.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- A pending approval no longer expires in 35 s; it waits for the person,
up to a day. Reloading the page redraws it; another conversation's
request is announced above the composer. Two requests in one
conversation are answered one at a time, in order.
- Wire: `approval.request` gains four fields and drops three; the TUI
reads none of the dropped ones any more. `approval.revoke` and
`approval.pending` are new methods, both scoped to the calling
connection. ACP's `session/request_permission` is unchanged.
- A saved rule that a stricter user rule outranks no longer re-asks
within the session (the session grant stays); before this change the
same click would have re-asked on the next identical call.
- One tool's evidence is cut at 16 KiB per value and marked when cut, so
a very large write or MCP payload now shows a prompt that says it was
shortened rather than sending the whole thing.
- Contract tier version 28 -> 30 (`Tool` and `ApprovalResponder` shape,
then `ApprovalOutcome` carrying the id of the request it answered).
- Rollback: revert the commits; no data migrates.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
Co-authored-by: KT <74288668+0xKT@users.noreply.github.com>
2026-09-21 20:57:06 -07:00
KT
94d01131bb fix(*): drop the playbook chip from the desk task row (#610)
## Summary

The desk's tasks list row drew `TaskRow.playbook` -- the playbook
library name the backend recovered from the first node id's prefix -- as
a grey monospaced chip between the task name and the error tag. Next to
a task it reads as a node id (`ai-competitor-compare`), and the
prototype's row never had it: a dot, the summary, the duration line and
the error tag.

Three commits:

1. `fix(ui-web)`: remove the span, its `.tksrc` rule and the
never-rendered `.tsrc` rule in `page.css` the chip's styling was
borrowed from; the test that asserted on the chip becomes a guard that a
playbook run's row shows the summary alone.
2. `refactor(*)`: with the chip gone the `playbook` field on
`tasks.list` had no reader in either UI, while every call still listed
the playbook library to derive it. The schema property, the pydantic
field, the derivation (`_derive_playbook`, `_playbook_names`, the tag
regex) and its four tests go; both generated clients are regenerated
from the contract; the offline fixtures stop sending the field. A
playbook run still tells itself apart on the wire by its node ids, which
the executor prefixes with the playbook's name.
3. `test(ui-web)`: the guard asserts the name line holds a single
element, so any second element beside the name fails it, whatever feeds
it.

Which playbook a run came from stays readable on the `load_playbook`
receipt in the conversation, and on the instance handle the task pane's
work-order tab shows.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_rpc_tasks.py tests/test_rpc_schema_match.py
-q` -> 434 passed
- `uv run ruff check` and `uv run ruff format --check` on the three
edited Python files -> clean
- `npm test --prefix ui-web` -> 191 files, 2492 passed (domain suites
and the gate suite)
- `make lint-ui` -> generated.ts matches the contract (192 methods);
eslint 0 errors (4 pre-existing warnings in connections / cron /
subagents); tsc clean
- `npm run --prefix ui-web build`, `python3 ui-web/build.py`, `node
ui-web/scripts/check-page.mjs`, `node ui-web/scripts/check-css.mjs`,
`node ui-web/scripts/check-class-namespace.mjs` -> boot-snapshot
235/235, every check OK
- `npm run lint:rpc --prefix ui-tui`, `npm run type-check --prefix
ui-tui`, `npm run lint --prefix ui-tui`, `npm test --prefix ui-tui`,
`npm run build --prefix ui-tui` -> generated.ts in sync; tsc clean; 0
errors; 144 files, 2074 passed; built
- `uv run pre-commit run --files <the nine files of commit 2>` -> all
hooks passed
- Mutation check on commit 1: with the chip line put back, the rewritten
test fails (`expected 'Cross-check quotesnightly-checks' not to contain
'nightly-checks'`)
- Real gateway (`raven web` on this branch, the user's home): after
commit 1 the row of the playbook-dispatched E2E run reads `Turn the
incident note into an archived record... / 1m32s / 6 files` with no
`.tksrc` element; after commit 2 `tasks.list` for that session answers
rows without a `playbook` key and the row still draws without a chip

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (the
design note's field list drops `playbook`; no doc names the chip)

## Risk

User-visible: the playbook name no longer shows on the task list row.
Wire: `tasks.list`'s `TaskRow` loses the optional `playbook` property;
the only clients of that contract are the two UIs in this repo, both
regenerated here, and the ACP surface does not expose `tasks.list`.
Rollback: revert the two commits.

- [x] Security impact considered (display and a dropped read-only field)
- [x] Backward compatibility considered (see Wire above)
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
zhanghui
a8db28b093 feat(ui-web): split model providers from model settings, one picker (#565)
## Summary

The settings dialog's Model page answered two questions at once -- which
accounts this install holds, and which model each job runs on -- and it
listed the connected providers only, so the fifty-odd vendors Raven
supports were reachable through a dropdown inside an "add" form and
nowhere else. This splits the page in two, gives every model a kind, and
merges the two model pickers into one.

**Model providers** is a two-column page. The left column is every
provider `model.options` returns (fifty-five on a stock registry,
`hosted_vllm` and `custom` among them) with a search box and six filters
-- connected, direct vendors, aggregators, browser sign-in, local --
connected rows first and each carrying its vendor mark. The right column
is that provider's connection, beside the list rather than paged into.

**Model settings** keeps the roles card alone, and each of its eleven
slots offers its own kind: the embedding slot lists embedding models,
image lists image models, and a chat model appears in neither.

**Adding a model** is a popover under its button. It fetches the
vendor's list, groups a gateway's ids by vendor, counts the kinds
present over the current search result and filters to the one pressed,
and writes one model per click rather than collecting ticks for a
confirming press. An id the vendor does not list is typed into the same
box with a chip saying what kind it is.

**One picker.** `components/ModelPicker` (164 lines, the roles card's)
is deleted; the roles card opens the composer's. What an opening lists
is now an `Offer`: a kind, the providers the caller allows, a title, the
pair it marks, and what a pick means.

### Two decisions worth reading

**The kind narrows each provider's column, never the provider list.** A
tester reported that a provider with a working key but no model list
built vanished from the composer's picker, and the model the chip named
-- set by onboarding, never "added" -- was in no column to be marked. So
a provider with nothing of the offered kind is listed with a count of
zero and a column that says so plus a row to type an id into; and the
model a slot or the conversation currently holds is listed in its
provider's column even when that column does not carry it. Verified on a
real host: with `agents.defaults.model` set to an id absent from
`providers.deepseek.models`, the picker opens on DeepSeek and lists it
first, marked, and writes nothing.

**One classifier, in Python.** `registry_data.kind_of` files a model by
what it writes, and `ModelLabel.kind` carries that answer to the page;
nothing in TypeScript derives a kind from capabilities or modalities.
The one exception is the chip on a typed id, which guesses from the name
-- a translation of `inferred_tags`, with a nine-row table asserted on
both sides so the copy cannot drift.

### Backend

Two read-only, additive fields on `model.options`: `ModelLabel.kind` and
`ModelOptionProvider.gateway` (the registry's `is_gateway`, which no
client can derive from a slug). No new method, no new config key.
Contract first: `openrpc.json`, then pydantic, then the handlers, then
`npm run gen`.

### Not in this change

The three navigation entries the prototype also adds (channels, cron,
memory); the prototype's OAuth pane, which tells the reader to run a CLI
command -- the browser device flow stays; backend enforcement of kinds,
which remains a filter the page applies; the onboarding wizard, which
keeps rendering `providers/Providers.tsx` untouched; the TUI.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Rebased onto `refactor/ui_web_architecture` at 6062b3678 and re-run
after the rebase.

```
uv run pytest tests/test_rpc_model.py tests/test_provider_registry_data.py tests/test_rpc_schema_match.py -q
630 passed in 12.44s

(cd ui-web && npx vitest run)
Test Files  186 passed (186)
     Tests  2552 passed (2552)

(cd ui-web && npx tsc --noEmit && npm run gen:check && npm run build && python3 build.py)
generated.ts matches the contract (189 methods)
boot-snapshot: OK (250 nodes match golden)
boot-snapshot: OK (251 nodes match golden)

make check-source-language; make check-large-files; npm run lint:i18n --prefix ui-tui
i18n: generated catalogue is up to date

uv run ruff check raven tests
All checks passed!
```

Mutation checks on the two new backend fields: commenting out the
`"gateway"` line fails `test_options_rows_carry_the_gateway_flag` on
`KeyError: 'gateway'`; commenting out the `"kind"` key fails
`test_model_labels_carry_a_kind` on `KeyError: 'kind'`. Restored, `git
diff` clean.

Driven on a real host (`raven serve` on its own home and port 18899,
real keys for DeepSeek / OpenRouter / Gemini, a real browser):

- Model providers draws 55 rows in two columns of 286px and 462px, the
three connected ones first with a status dot, DeepSeek carrying the
default badge, 54 vendor marks and one lettered tile (`custom`).
- The embedding slot's picker lists all three connected providers with
counts 1 / 0 / 0 and exactly one model. It is at the body, not inside
the dialog, at z-index 46 over the veil's 40.
- The add-model popover against OpenRouter's live catalogue: 534 models,
kind tabs reading all 534 / text 445 / image 52 / embedding 33 / audio
4, five vendor groups, the four seeded models checked. Typing
`my-team/bge-custom` offers it with an "embedding" chip, which is what
`inferred_tags` answers for that name.
- The composer's chip lists the same three providers with OpenRouter
counted 1: its embedding, reranker and image models are not offered for
chat.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Composer users see two changes: embedding and reranker models leave the
model chip's picker, and typing an id there now adds it to the provider
before switching. Settings users learn two navigation entries where
there was one; `settingsTab.id` values written elsewhere still land on a
model page, since the roles page keeps the id `model`.

A model the registry classifies wrongly -- an embedding model with no
`embedding` tag whose name the two patterns miss -- is invisible to the
embedding slot until an overlay states its kind. The add-model popover's
chip is the way to state it.

Rollback: the two wire fields are additive and optional on the reader's
side; the section split is a data change in `SECTIONS`; the deleted
picker is one `git revert` away.

One gate pin moved: `fixture-shape`'s `UNSENT` list loses its six
`model_labels` entries, because the offline fixtures now send them (a
page that filters by kind needs a kind offline too). The gate's own
message asks for exactly this.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
KT
900eb7615c fix(*): key a running spawn's live account by its record, not its id alone (#577)
## Summary

While a spawned sub-agent runs, its live account (transcript, usage,
tool calls) is held in a process-wide index. A spawn was keyed into it
by the node id the model chose, which is unique for one conversation
only, so two conversations running a spawn under the same id shared one
entry: the later `collecting` overwrote it, so the earlier
conversation's context panel showed the other run's transcript (and,
since #567, its live usage), and the first run to finish popped the
entry, so the other run's live view went blank until it ended. This has
been possible since a92514e75 (2026-09-07), when the spawn record id
became the model's own `node_id`; before that the key was a minted,
globally unique call id.

The key is now the record's own address, the conversation's node root
plus the id (`history.spawn_live_key`, the spawn side of
`dag_store.node_live_key`). The writer holds the root as the record's
directory and the readers as the node files' root, both resolved from
the same session directory, so the two sides agree without depending on
how a session key is spelled. `subagent.context` and `tasks.list` read
by the same helper; the dag side already carried its run id. No wire
change.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run in the fix worktree, base `refactor/ui_web_architecture` at
9656dd91:

- `uv run pytest tests/test_rpc_tasks.py
tests/test_rpc_subagent_calls.py tests/test_subagent_activity.py`: 104
passed. Two tests are new: two conversations running a spawn under the
same id each read their own usage through `tasks.list`, and each read
their own transcript through `subagent.context` driven through a real
`SubagentManager.spawn` with a backend that waits to be released, with
the first run to finish taking nothing of the other's. The second test
was run against the pre-fix source and failed there (conversation A read
conversation B's transcript).
- `uv run pytest tests/test_subagent_manager.py`: 159 passed.
- `make lint-python` and `make test-python`: 23989 passed, 109 skipped.
- Relevant Ruff check and format check: passed.
- Real gateway (`raven serve` from this branch on the user's home,
Raven-Code lane): two conversations dispatched a spawn under the same
`node_id` at once, one saying ALPHA in two steps and one saying BRAVO in
five; polled every 3 s, each conversation's `subagent.context` carried
only its own word while both ran, and after the ALPHA run settled the
BRAVO conversation's live view stayed up and kept growing (6 to 10
messages) instead of going blank. The #567 regression driver on the same
gateway still passed its trace and subtitle checks (tool rows 0, 1, 3, 4
while running; answer landed; subtitle `Raven-Code / done / 1m02s`).
- Real gateway on an isolated `RAVEN_HOME` (in-process `Raven` lane):
the `tasks.list` reader, which now reads by the same key, still served
the live usage -- subtitle `Raven / running / 5s / 1,881 tokens` up to
`Raven / done / 49s / 8,601 tokens`, board chip 1, 2, 3 tools.

## Risk

- [x] Security impact considered: this closes a cross-conversation leak
of a running sub-agent's transcript and usage inside one gateway
process; nothing new is exposed.
- [x] Backward compatibility considered: the key is in-memory only, so
no record on disk changes shape; a reader and a writer of one process
always run the same code.
- [x] Rollback path is clear for risky changes: revert the squash
commit.

## Related Issues

Fixes #580. #567 added the second reader of this index (`tasks.list`)
and gated it to running nodes; the collision itself predates it.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
KT
30d1c2155c fix(*): live spawn record, live usage and a leaner node panel subtitle (#567)
## Summary

Two defects in the desk tasks tab's node panel, both reported from the
page.

1. A spawn task's node panel read its record once, when opened, and
never again. `useNodeRecord` re-read only on `store.nodeVersion`, which
only a dag's `node_updated` frame bumps, and its key held no status; so
a panel left open on a running spawn kept the first read's single step
while the transcript's own spawn card moved on, and the final answer
never landed either. The panel now re-reads a running spawn's record on
the one-second beat the transcript card already reads `subagent.context`
on (skipping a beat while a read is still out, and only while a tasks
source is wired), and re-reads once on any status transition, which also
covers a dag node settled by a `run_completed` frame that no
`node_updated` announced.
2. The subtitle read `Raven-Code @raven-code-db8bbb / done / 2m55s /
usage not reported / 14 tools / 1 failed`. It now reads agent, status
word, duration and, only when the lane reported usage, the token total:
`Raven / running / 1m50s / 4,910 tokens`. The handle stays on the board
card and the work-order tab; the tool count stays on the board card, and
the calls themselves are in the process fold (whose folded multi-call
rows still say how many failed). The two catalogue keys nothing reads
any more are removed, the TUI's generated copy is regenerated, and the
design note says what the header shows and how often a record is
re-read.

3. A running node's usage was invisible until it settled: `tasks.list`
read tokens, tool counts and files from the record on disk, which the
run writes when it finishes. The reader now overlays those facts from
the activity being collected for the node in this process (the same
in-memory account `subagent.context` and `dag.node` serve a transcript
from), keyed by the record id for a spawn and by `node_live_key` for a
dag node, only for a node that is running and only for what the lane has
reported so far. A dag node that finishes while its run is still going
keeps the account the runner set aside for it
(`activity.record_settled`) until the manifest is written, so its total
does not vanish between the two. On the page, each read of a running
node's record also re-reads its row through `tasks.list(kind, id)`, so
the subtitle's token total moves during the run (one row read out at a
time, so a dag node's per-tool-call frames do not stack requests), and a
dag node's own terminal frame re-reads the row too, so the node's final
total lands even while a sibling keeps the run going; `reconcile` keeps
a frame that landed while such a read was out, so a read started on the
beat cannot put a settled row back to running.

No wire change: `TaskNode`'s fields and their null semantics are
unchanged; a lane that has not reported usage yet still reads null. The
in-process lane reports usage after every model call, so its total moves
during the run; the acp lane reports once at the end of its turn, so its
total appears when the run settles.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run in the fix worktree, base `refactor/ui_web_architecture` at
6062b367:

- `npm test --prefix ui-web -- src/features/tasks`: 8 files, 189 tests
passed. Seven tests are new (the beat, the skipped beat while a read is
out, no beat for a dag node, the settle re-read and the beat stopping,
the beat stopping on close, the row re-read on the beat reaching the
subtitle, `reconcile` keeping a frame that landed during its read) and
three changed for the subtitle; the tests that reproduce each defect
were run against the pre-fix source and failed there.
- `npm run lint --prefix ui-web`: 0 errors, 5 pre-existing warnings in
other domains.
- `npx tsc --noEmit -p ui-web`: clean.
- `npm test --prefix ui-web -- scripts/gates`: 40 files, 158 tests
passed.
- `npm run lint:i18n --prefix ui-tui`: generated catalogue up to date.
- `make lint-python` and `make test-python`: 23980 passed, 109 skipped.
- In `ui-tui`: `npm run lint`, `npm run lint:rpc`, `npm run type-check`,
`npm test` (2074 passed), `npm run build`.
- `uv run pytest tests/test_rpc_tasks.py`: 43 passed (3 new: a running
spawn's usage, counts and files off the live activity and nothing once
the block closes; a live lane that has not spoken keeps its nulls; a
running dag node's usage off `node_live_key`).
- `python3 ui-web/build.py`: both boot snapshots match their golden (250
and 251 nodes).
- Real gateway (`raven serve` from the worktree with the rebuilt page):
a Raven-Code spawn dispatched over RPC, its node panel opened while
running and left open. Sampled every 3 s, the process fold's tool rows
went 1, 3, 4 without the panel being closed; the answer appeared when
the run settled; the subtitle read `Raven-Code / running / 6s` and then
`Raven-Code / done / 51s` (the acp lane reports usage once, at the end
of its turn).
- Real gateway on an isolated `RAVEN_HOME` whose default model is
deepseek, so the in-process `Raven` lane runs: a three-step spawn (each
step a `sleep 15` shell command that needs the previous output). Sampled
every 3 s with the panel left open, the subtitle read `Raven / running /
6s / 1,812 tokens`, then `3,769 tokens`, then `5,852 tokens`, and `Raven
/ done / 49s / 8,021 tokens` once settled; the board card's tool chip
went 1, 2, 3; `tasks.list` reported the same growing usage over RPC.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
(docs/specs/2026-09-18-desk-tasks-list-design.md, sections 4 and 7)

## Risk

- [x] Security impact considered: none; the change is confined to
rendering and to calling an existing read-only method
(`subagent.context`) once a second per open panel while a spawn runs,
the cadence the transcript card already uses.
- [x] Backward compatibility considered: the two removed catalogue keys
had one consumer, removed in the same change; `gui.tasks.tools_n` stays
in use by the board card.
- [x] Rollback path is clear for risky changes: revert the squash
commit.

## Related Issues

N/A for closing. #461 is the instance surface's version of the same
class of gap (a running sub-agent's tool calls not streamed live) and is
not closed by this.

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
zhanghui
076b47b253 fix(rpc): keep the error sentence when a refusal also carries data (#560)
## Summary

A handler that raised an `RpcError` with both a sentence (`detail`) and
structured `data` lost the sentence on the wire: `Dispatcher` wrote
`data` in place of `{"detail": ...}`. The web page reads
`error.data.detail` for its toasts, so every refusal from the plugins
page arrived as `config_validation_error` and nothing else --
`plug.configure` on a plugin that takes no credential, `plug.configure`
on a server with no ledger, `plug.auth` on an unknown server, and
likewise every `model.*` refusal that names its provider slug.

The frame now carries the sentence beside the fields, unless the handler
already put one there (the auth rollback in `plug.install` does). The
hermes-only stubs stop passing their message as the detail: it is
already the data's `error` field, and `StubResult` forbids extra keys,
so their frames are unchanged.

Before, on a `raven serve` host built from the base:

```
{"code": -32011, "message": "config_validation_error", "data": {"field": "name", "name": "context7"}}
```

After:

```
{"code": -32011, "message": "config_validation_error", "data": {"field": "name", "name": "context7", "detail": "this plugin takes no credential"}}
```

The second commit records what was missing from the settings page's
acceptance list: no requirement said a refused write must tell the
person why, so none of its cases exercised a refusal and a whole pass
went green over this. A56 states the requirement and T9.8 is the case.

Found while tracing a tester's report that the plugins page fails with a
bare `config_validation_error` toast. One thing the toast was hiding is
left as is: a server written into `tools.mcpServers` by hand under a
catalog name (for example `github`) gets the credential panel from the
catalog, but `plug.configure` refuses it ("not installed from the
catalog; edit the config file instead"). With this fix that sentence
reaches the toast.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run --frozen pytest tests/test_rpc_plughub.py tests/test_rpc_stubs.py tests/test_rpc_contract_shapes.py tests/test_acp_protocol.py tests/test_acp_methods.py -q
320 passed

uv run --frozen --extra dev ruff check raven/rpc/dispatcher.py raven/rpc/methods/_stubs.py tests/test_rpc_plughub.py
All checks passed!

uv run --frozen --extra dev ruff format --check raven/rpc/dispatcher.py raven/rpc/methods/_stubs.py tests/test_rpc_plughub.py
3 files already formatted
```

The new test
`test_a_refusal_with_structured_data_still_carries_its_sentence` fails
on the base (the frame's `data` lacks `detail`) and passes here.

T9.8 was run on a real host, in the browser, before and after. A
temporary `RAVEN_HOME` with a hand-written `github` stanza (a catalog
name with no ledger entry), the page built from this branch, then
Settings > Plugins > `github` > type a token > update, and the same for
clear:

    base code (dispatcher and stubs checked out at b63f35087):
      toast: "operation failed: config_validation_error"
    this branch:
toast: "operation failed: not installed from the catalog; edit the
config file instead"

Both toasts were read from the page's accessibility tree and captured as
screenshots. The same two calls over the socket return the frames shown
above.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered: `detail` was already sent whenever
`data` was absent; no new information leaves the server.
- [x] Backward compatibility considered: `data` gains one key and every
existing key is unchanged. The web client already prefers `data.detail`;
the TUI's typed `RpcError` keeps `data` opaque; the stub frames are
byte-identical.
- [x] Rollback path is clear for risky changes: revert the commit.

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
zhanghui
1cc090364b docs(*): record what the settings page shipped, where the design said otherwise (#556)
## Summary

Two statements in the committed settings design read as fact and are
not. This
corrects them in place; no other section changes.

**`features/model/vendors.ts` was never created.** The design names it
four
times as the home of the vendor mark table, with the table re-exported
through
that domain's `types.ts` so the settings page could reach it. None of
that was
needed: the table already lives in `components/ProviderMark.tsx`, which
is the
same layer the shared picker is in, so the picker imports it directly
and no
cross-domain edge arises.

This is the sentence that cost the picker its marks. The shared picker
shipped
drawing provider names as plain text, on the reading that the edge was
forbidden -- a conclusion taken from this document rather than from the
tree.
It was put back in #540. The Decisions entry now states where the table
is and
keeps the planned path as the route not taken, so the next reader does
not
re-derive it.

**The meta-tool entry was true but partial.** It said `tool_search` and
`tool_call` are greyed, not switchable, and left out of the counter,
which
still holds. It did not carry the distinction #540 had to fix -- hidden
from
the schema is not the same as withheld from the model, so the DAG
controls were
drawn as fixed while their switch worked -- and it did not describe a
default
install, where `tools.toolSearch.enabled` is off and a card named after
tool
search holds the one meta-tool that is not tool search.

Two other decisions were deviated from during execution and both were
reversed
back to what this document already said, so neither needed an edit here:
the
picker is a popover again, and the About row offers the upgrade as its
own
step.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
make check-source-language   -> pass
make check-large-files       -> pass
```

Every path this document now names was checked against the tree at the
target
tip: `components/ProviderMark.tsx` exists and holds the table,
`features/model/vendors.ts` does not exist. The four occurrences of the
missing
path are gone except one, which is now explicitly the plan that was not
followed.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Documentation only; no code path changes. Rollback is the one commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
KT
441df6c842 feat(*): draw the desk tasks tab from a tasks.list read (#509)
The desk's tasks tab, end to end: a `tasks.list` RPC that reads one
conversation's
delegated work off disk, and the web desk's tasks tab drawn from it
after the
prototype.

A task is every unit of delegated work a conversation started: a
`spawn`, or a
`run_subagent_dag` graph (a playbook run is one of those). `tasks.list`
answers one
row per task with its nodes, counts, a derived status and the replan
link, composed
server-side from the three stores a task leaves behind (the run dir, the
session node
registry, the instance registry). Bodies stay where they are: a node's
messages,
output and rendered prompt come from the existing `dag.node` /
`subagent.context`;
stopping reuses `subagent.interrupt` (dag) and
`subagent.cancel_instance` (spawn).
Live updates are the five existing events reduced on the client with a
re-read on
settle.

Key decisions (design note:
`docs/specs/2026-09-18-desk-tasks-list-design.md`):

- One structural read, no new stop method, no new events.
- Status ladder: failed > interrupted > running
(pending/running/exception) >
cancelled (cancelled/skipped) > completed; a replan that started reads
cancelled,
  one that never started reads failed with its error.
- A node's status mid-run comes from its instance row, not from the
session registry's
`running` claim (which every node of a run carries from the start); a
node with
neither is pending with no clock. A stopped run reads cancelled from the
registry
  since a hard stop never writes a manifest.
- Two small runtime additions the tab needed: the in-process lane
records the files a
node wrote or edited on its activity (one item per path: op,
added/removed lines
accumulated across touches, last size), and a spawn's delivered mark
carries its
  `node_id`.
- Frontend: the tasks domain re-exports the generated wire types and is
reached from
`state/` and the transcript only through its installed source (optional
verbs on
`TasksSource`), the desk orders its tabs deliverables / tasks / diff
with a
running-count badge, task-written files appear under the deliverables
and diff
tabs, the store drops reads that land after a session switch, and the
prototype's
visual values are ported onto the domain's `tk`-prefixed classes. The
shared hunk
  builders move to `lib/hunks.ts`, so the import-direction lists shrink.

Prototype fidelity, in the last two commits: the tab is drawn one-to-one
after the
prototype where the prototype has a form, and adds nothing the prototype
lacks (an
element-by-element audit of list row, strip, pane, board card and node
panel came to
114 deviations; all are closed). A second sweep listed 55 things the
prototype's task
module has that ours lacked; every one the runtime can back is in:

- Desk: task-written files open as files (extension chip, size), task
diffs lead with
the M/+ chip, name the full path and open the diff; the palette is a
dialog, opens on
the tasks tab, resizes leftward while anchored; Escape peels one layer
at a time
(fullscreen, picked node, pane, desk) through a registered slot in the
escape order.
- Pane and board: a toast confirms a stop request, the first task window
opens at 440px,
pan and zoom survive picking a node, the fit view keeps its margin,
per-status dimming,
the board fills the pane until a node is picked. Board cards are the
prototype's HTML
`runCard` (196x78) through an optional `renderNode` on the shared
`DagGraph`.
- Node panel: the prototype's verb vocabulary and grouping (a thought
folds the calls it
caused), detail cards, markdown answers, per-node fold memory,
hover-only copy and
time, the panel scrolls inside itself with the reply dock pinned, a
nested spawn or dag
call names its run (parsed from the receipt, resolved against the task
list) and opens
  it, a running node's answer carries a caret.
- Diff pane: renders the raw unified patch (headers, patch line numbers,
first-character
colour) with the file bar above it, rebuilt from the stored hunks by
`lib/hunks
  toUnified`. This applies to session-level diffs too.
- Runtime: the in-process lane records a file's path relative to the
run's workspace,
which is what the file endpoint anchors relative paths to; the patch
body matches a
tool call to the recorded file at a segment boundary so older absolute
records still
  find their hunks.

Not in this PR (no runtime primitive): retry / resume of a node (the
prototype's
"retry this step" and "continue" buttons; the runtime only has
`resolve_dag_node` for a
node awaiting adjudication) and files for the acp and cli lanes (those
lanes report
tool counts only). Both to be raised as issues.

Multi-commit PR; the commits walk design -> wire contract -> backend
reader ->
frontend -> styling -> fixes from a real-gateway E2E pass and two
reviews ->
one-to-one redraw -> prototype gap sweep.

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

Backend:

```
uv run pytest tests/test_rpc_tasks.py tests/test_subagent_manager.py tests/test_subagent_activity.py tests/test_rpc_schema_match.py tests/test_rpc_subagent_calls.py tests/test_rpc_dag.py tests/test_rpc_instances.py -x -q
uv run ruff check . && uv run ruff format --check .
uv run python scripts/check_source_language.py
```

Web UI (from `ui-web/`):

```
npm test            # 189 files, 2753 tests passed
npm run type-check
npm run lint        # 0 errors (7 pre-existing warnings in unrelated files)
npm run gen:check   # generated.ts matches the contract (185 methods)
node scripts/check-page.mjs && node scripts/check-css.mjs && node scripts/check-class-namespace.mjs
npm run build && python3 build.py    # boot snapshots OK
```

TUI client (from `ui-tui/`): `npm run lint:rpc`, `npm run lint:i18n`,
`npm run type-check`.

Real gateway: `raven serve` on this branch, spawn and dag cases driven
through the
served page (spawn ok / spawn failed on the acp lane / fork-join dag
writing files /
dag with a crashed node and a skipped dependent / shared-instance dag /
playbook run /
stop a spawn and a dag from the desk / gateway killed mid-run and
restarted / a
main-agent edit, a dag node edit and an edit-only spawn for the
red/green diff). Each
scenario's `tasks.list` answer and screenshots were checked side by side
against the
prototype rendered from its own script and stylesheet at the same
viewport.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

User-visible: the desk's tab order becomes deliverables / tasks / diff
and it opens on
the tasks tab, which lists every spawn and dag run of the conversation
with a stop action
on running ones. Every diff pane (session-level too) now shows the raw
unified patch
text instead of the previous side-by-side line view. Escape closes desk
layers one at a
time. The `tasks.list` method is additive; existing methods and events
are unchanged
except `subagent.list` now reading a dead run's `exception` node as
interrupted, which
`dag.get` already did. The file-recording hook only adds a `files` key
to a node's meta
and manifest when a writing tool ran; paths inside the workspace are now
recorded
relative to it (older records stay absolute and still resolve).

Rollback: revert the PR; no data migration, no stored format changes.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
zhanghui
707152d100 feat(ui-web): rebuild the settings dialog on the v0.2 prototype, with its backend (#494)
Rebuilds the web settings dialog on the v0.2 prototype -- eight sections
(general, usage, models, skills, tools, plugins, archive, about) in
place of
the thirteen tabs -- as a `features/settings/` domain in the shape this
branch
gates, and adds the backend the pages need so every control reads a real
value and writes to the real place.

Backend (one commit, one test per method, each checked by a mutation
run):

- `settings.usage` takes `from`/`to` and answers a per-day series;
- `session.list {archived}` and a lazy auto-archive pass inside the list
(`sessions.autoArchiveAfterDays`: one appended record per stale unpinned
  session, none on the next list);
- `skills.manage inspect` carries the file list, `always` and the
install
  stamp; `skills.manage open` opens a skill file on the gateway host;
  RPC-driven installs are stamped `trigger: "rpc"`;
- `model.add_models` (one write for a batch), `model.set_fields`
(address,
  Azure deployment and API version, header patches), `model.oauth_login`
(device flow for the page, URL + code); `model.options` gains `key_url`
and
  the redacted `extra_headers`;
- `plug.retry`, `plug.revoke`, `plug.configure`;
- `skillForge.blocklist` is read live, so a switch takes effect on the
next
  turn; reload-only keys return a `warning` the toast repeats.

Frontend (one commit): the domain files, the prototype's stylesheet
under the
`settings-` prefix in `styles.css`, a shared props-only
`components/ModelPicker`,
`gui.settings.*` in both locales, the old page and its 295 orphan keys
deleted, offline fixtures for the new methods, and gate pins moved with
the
tree (`check-class-namespace` counts, two import pins, the unsent-field
list).

Then a `fix(ui-web)` commit with nine faults only the real host showed
(each
with a test), a `test(*)` commit pinning `toolGroups.json` to the tools
the
loop registers, and a docs commit correcting two on-disk section names
in
the acceptance cases.

Design record, acceptance cases and plan:
`docs/specs/2026-09-17-web-settings-page-design.md`,
`docs/specs/2026-09-17-web-settings-page-acceptance.md`,
`docs/plans/2026-09-17-web-settings-page.md`.

Reviewer notes:

- The base is this refactor branch; the six `fix(...)` commits at the
bottom
are PR #453 (still open against main) and will fall out once it merges.
- A trial merge of this branch with `main` shows three conflicts, none
from
  this branch's files except one line: main's #464 rewrote
`raven/skill_hub/hub.py::install` and no longer calls
`write_install_meta`
there, so the `trigger="rpc"` stamp this branch adds collides; take
main's
version and re-add the stamp only if a `write_install_meta` call
remains.
`ui-web/build.py` and `ui-web/src/live/120-settings.js` are this
branch's
  base against main, not this PR.
- `tools.media.<kind>` takes the whole `{model, quality}` object and no
null,
so a media role is cleared by writing an empty model; the design said
null.
- The dialog opens on the section the shared slot names (`usage`), as
before;
the prototype opens on general. Left as is: `state/settings.ts` is
shell.

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

Commands run on the rebased branch (base:
`refactor/ui_web_architecture`):

- `cd ui-web && npm run type-check` -- clean.
- `cd ui-web && npx vitest run` -- 175 files, 2536 tests passed, 0
failed,
  0 skipped (includes the seven convention gates).
- `cd ui-web && npm run lint` -- 0 errors (7 pre-existing warnings in
other
  domains).
- `cd ui-web && npm run build && python3 build.py` -- both boot goldens
  unchanged (298 / 285 nodes).
- `uv run pytest tests/test_rpc_model.py tests/test_rpc_schema_match.py
  tests/test_settings_tool_groups.py tests/test_rpc_skills.py
tests/test_rpc_settings.py tests/test_rpc_console.py
tests/test_rpc_session.py
  tests/test_rpc_plughub.py tests/test_providers_oauth_login.py
  tests/test_skill_forge_blocklist_live.py -q` -- 935 passed.
- `scripts/check_source_language.py` and `scripts/check_large_files.py`
over
  the branch -- exit 0.
- Real-host acceptance: a temp `RAVEN_HOME`, `raven serve --port 18899`,
the
built page driven by playwright-cli; 46 of the acceptance cases passed
with
every write read back from `config.json` or the session files, 1 partial
(the offline stub page: boots with no console error, the dialog was not
driven), 9 not run (real OAuth accounts, EverOS plugin install, catalog
credential rewrites, three probes needing extra LLM turns). The record
with
  commands, jq expectations and snapshots is kept outside the repo.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

- New RPCs only add methods and optional fields; existing methods keep
their
shapes (`model.options` gains two optional fields). The settings dialog
is
  replaced whole; the previous one had no consumers outside the dialog.
- `skillForge.blocklist` is now read per turn instead of at start; a
switch
  in the dialog changes the next turn's skills without a reload.
- Rollback: revert the frontend commit to restore the previous dialog;
the
  backend commit can stay, nothing else calls the new methods.

- [x] Security impact considered (the settings whitelist still excludes
the
containment controls; header values and keys are redacted on every read)
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
Tong Li
608cedf180
feat(browser): give the model the shared browser, and a desktop MCP server to reach (#578)
## Summary

The browser existed on every side except the model's: `raven/browser`
drives a
Chromium the panel streams over CDP, and `browser.*` RPC gives the
reader hands
on it, but `raven/agent/tools/` had no browser tool at all -- a model
asked to
open a page had nothing to call. This registers eight `browser_*` tools
over
that same driver, with no MCP hop, so the page the model drives is the
page the
panel shows and a human can take the keyboard mid-task.

Desktop control reuses `macos-mcp` through the MCP client that was
already
there; the only new code for it is a market catalog entry.

What the tools add beyond the driver:

- **Reading back.** Every acting call returns the page as it is
afterwards
(url, title, the actionable refs, the visible text), so the next call
can be
aimed without a separate read. `browser_screenshot` is the one that
costs an
  image and is asked for on purpose; it goes out as an image block.
- **Who is acting.** Each call names an owner -- the sub-agent run in
flight,
else the conversation -- and the driver binds that owner to a tab. Two
agents
work in two tabs, neither can activate or close the other's, and a popup
a
click opens becomes the opener's tab. A reader's own clicks carry no
owner:
they are stamped, and the owner's next readback says the user touched
the
  browser, once.
- **Permissions.** Reading and navigating default to allow, on the
`web_fetch`
  reasoning (opening a page is a GET the agent could already make, and
`raven.browser.policy` refuses what a GET must not reach). Clicking,
typing
and pressing ask, keyed by site rather than by ref, so one approval
covers a
  site for the conversation.
- **Config.** `tools.browser.headfulOnAgentUse` pops the shared browser
out
into a real window on the model's first use. Off by default: a server
has no
  desktop to pop into.

The tool surface grew, so the `reserved_tools` tripwire moved:
descriptions
were trimmed first (~1270 to ~1090 tokens) and the bound raised from
6000 to
7400 against a measured 7115, with the trade written beside it. The
schemas are
only present where the browser extra is installed; without playwright
the tools
report themselves unconfigured and never reach the request.

No ui-web changes are in this PR. The web UI side of this work (a
browser pane
in the desk, a per-tab miniature, docking) is written against surfaces
that
`refactor/ui_web_architecture` removes, so it waits for that branch to
land.

## Type

- [x] Feature

## Verification

- `uv run pytest tests/ -q` -> 23755 passed, 121 skipped, 1 failed. The
failure
is
`test_cli_onboard_commands::test_clearing_does_not_disturb_a_key_the_user_exported_themselves`,
which reads the developer's own shell environment and fails the same way
on
  an unmodified `main` in this checkout.
- `uv run pytest tests/integration/test_browser_tools_real_web.py -q` ->
6
  passed against a real Chromium and a local form: fill and submit, a
screenshot arriving as an image block, two owners in two tabs with
neither
able to take the other's, a read leaving the front tab where the last
actor
put it, a `target=_blank` popup landing its opener on the new tab, and a
scripted model driving `AgentLoop`'s default tools end to end -- the
gate
asking once for the site, the grant covering the click that follows, and
the
  panel's `browser.frame` returning the page the model left.
- `uv run ruff check raven tests`, `uv run ruff format --check raven
tests`,
`uv run lint-imports`, `make check-source-language`, `make
check-large-files`
  -> clean.
- Desktop side exercised by hand through raven's own MCP client, over
stdio and
  over streamable-http with a bearer token: launch Calculator, read its
accessibility tree, click 2 + 3 =, read the result back. Screenshots
arrive
  as image blocks.

Not verified: the desktop flow was driven by a script rather than by a
model in
the loop; sub-agents that run in their own process (the ACP and cli
lanes) have
their own `Browser` singleton and are not given these tools; Windows and
Linux
desktop executors were not chosen.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Notes. The acting verbs sit on the ask tier by default and a grant is
scoped to
a site and a conversation, so an approved click on one host does not
carry to
another. Navigation goes through the existing outbound policy. The
desktop MCP
server runs locally under `uvx` and needs macOS Accessibility and Screen
Recording grants, which the operating system asks for itself; the
catalog entry
is opt-in like every other market entry. Nothing here changes an
existing
default: without the browser extra the tools are absent, and with it the
only
new always-on cost is the tool schema.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-21 20:28:04 +08:00
silverLXT
0cb8b7996a
fix(raven-oncall): a campaign bills only its own jobs under a shared rounds directory (#553)
A campaign asked the shared rounds directory how much had been spent,
and the
directory answered for every job in it, including jobs its sibling
campaigns
had submitted. Rounds that keep one directory therefore bill each other,
and
the bill is cumulative: round three opens already charged for rounds one
and
two.

Measured on a run of 2026-09-11. Five campaigns declared the same rounds
directory. Round two had a budget of 130 and had actually used 34.6, but
the
eighteen jobs the earlier rounds had left in the directory brought the
reading
to 125.6, so the remaining 4.4 would not cover one more trial at 5.9 and
the
round stopped. 142 GPU-minutes went unspent and that round's multi-seed
conclusion was lost. A sibling run that gave each round its own
directory was
not affected, which is why this reads as a directory-hygiene problem
until you
look at the meter.

Two changes. Spend is now counted from the campaign's own ledger, so a
job a
sibling submitted is not billed here; the filter sits above the existing
timeline arithmetic, so same-device overlap is still charged once and
two
cards in parallel are still charged twice. And declare refuses a rounds
directory a live sibling campaign already keeps, naming the sibling, so
the
two rounds do not land in one directory in the first place.

Type: fix

Verification:
- All oncall tests: 775 passed, 1 failed. The one failure
  (test_the_products_tool_face_equals_the_forks_config_intent, a missing
optional a2a_client dependency on this machine) also fails on 0464aa1b1,
so
  this adds six passing tests and no new failures.
- The six new ones pin: a sibling's job is not billed; the campaign's
own two
trials overlapping on one device are still billed 20 minutes once; a job
this executor just submitted is billed before the ledger has caught up
(no
refund window); billing_only leaves a foreign backend untouched; declare
  refuses a directory a live sibling keeps; a concluded sibling, and the
  campaign itself, do not count as a clash.
- The first test also pins the measured number: the same fixture
directory
reads 125.6 unfiltered, which is 34.6 of its own plus 91.0 of its
siblings.

Risk: low for the meter, moderate for declare. The meter can only ever
bill
less than before, and a run that already gives each round its own
directory
reads the same. Declare now refuses something it used to accept, so a
caller
that deliberately shares a directory between live campaigns will see an
error
naming the sibling; concluded siblings and re-declaring the same
campaign are
both still fine.

---------

Co-authored-by: xiaotian.luo <xiaotian.luo@thetahealth.ai>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-09-20 08:32:01 -07:00
userName20260323
e708f34a91
docs: align bilingual documentation with runtime behavior (#551)
## Summary

- Align English and Chinese documentation for Docker/Compose, sandbox
execution, and the in-tree tracing API with verified runtime behavior.
- Separate proactivity usage guidance from developer design and
implementation details, with configuration, troubleshooting, cost, and
safety boundaries.
- Update navigation and remove redundant section separators. Keep page
URLs unchanged.
- Preserve published proactivity section and title anchors in both
languages. Use native aliases for same-page content and visible
migration links for moved sections, with regression tests.
- Clarify Sentinel delivery fallback and menu-selection requirements,
tracing-on/off argument handling, headless CLI menu output, and viewer
descriptor lookup. Add focused regression coverage without changing
runtime behavior.
- Clarify the initial frozen baseline and improve Evolver wording in the
bilingual site and repository mapping. Keep its structure, navigation,
implementation terms, and technical claims unchanged; broader Evolver
documentation work is excluded.
- No runtime changes.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Passed on the rebased revision:

```bash
uv run --frozen --no-sync pytest tests/test_docs_site.py tests/test_tracing_api.py tests/test_sandbox_unit.py tests/test_sentinel_runner.py tests/test_sentinel_planner.py tests/test_sentinel_fast_path.py tests/test_nudge_policy.py tests/test_proactive_spawn.py tests/test_core_sentinel_stack.py tests/test_core_cron_stack_ledger.py tests/test_cli_sentinel_commands.py tests/test_decision_router.py -q -n 0 -x
# 395 passed

uv run --frozen --no-sync --with 'mkdocs-material>=9.5' --with 'mkdocs-static-i18n>=1.0' --with 'mkdocs-git-revision-date-localized-plugin>=1.6.0' --with 'jieba>=0.42.1' mkdocs build -f docs-site/mkdocs.yml --strict --site-dir /tmp/raven-pr551-push.9ZfOEi/site
# Strict English and Chinese builds passed

UV_NO_SYNC=1 make lint-python PYTHON_LINT_TARGETS='tests/test_docs_site.py tests/test_tracing_api.py tests/test_decision_router.py tests/test_cli_sentinel_commands.py'
UV_NO_SYNC=1 make check-large-files check-source-language COMMIT_RANGE=origin/main..HEAD
PYTHONPATH=. uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
npm exec --yes --package=@commitlint/cli@21.1.0 -- commitlint --from origin/main --to HEAD --config commitlint.config.cjs
git diff --check origin/main...HEAD
# All passed
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Documentation checks also covered bilingual examples and the proactivity
JSON configuration schema. A built-HTML audit verified all 110
historical proactivity section fragments and four title fragments,
including same-language migration destinations, and found zero broken
internal fragment links across 29 HTML pages (672 links). Replaying
pre-fix content made all four page compatibility cases fail as expected.
The new descriptor-directory test failed on all three documents before
their stale paths were corrected. Added coverage exercises tracing
on/off, bare-number versus explicit menu selection with incomplete
classifier configuration, and discovery-menu dispatch through the real
CLI stack's headless sink. Runtime type checks and the full repository
test suite were not run; no runtime code changes. No browser click test,
live LLM, real-VM, or container end-to-end run was performed.

## Risk

Documentation-only, with regression tests. Proactivity navigation
changes, but page URLs and historical heading fragments are preserved.
An old link to content moved between pages lands on a visible migration
entry and requires one click to reach the new section; no JavaScript
redirect is needed. Existing queued-nudge policy and external-agent
isolation limitations are described, not fixed. Roll back by reverting
the documentation and associated test changes; no data migration is
involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: zhao.wang <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-20 19:20:27 +08:00
Dizhan Xue
7faa2060e2
feat(*): raven speaks Agent2Agent 1.0, inbound and outbound (#446)
## Summary

Raven speaks the Agent2Agent protocol, version 1.0, in both directions.
The host
agent is the peer: nothing here makes a sub-agent addressable from
outside, and a
raven launched as a sub-agent refuses to serve the face at all, because
it is
already reached over ACP by the host that started it.

Inbound is an aiohttp face carrying the two cards and a JSON-RPC
endpoint. It
mounts on the gateway when `a2a.server.enabled` is set, and `raven a2a
serve` runs
it standalone for a headless deployment, where running the command is
itself the
opt-in. Outbound is a single tool, `a2a_send`, that takes a card URL and
a message;
it enters the tool list only when `a2a.peers` is non-empty, so a host
with nowhere
to send a message does not carry the schema. The two switches are
deliberately
independent: calling out and being called are separate deployments.

Onboarding mints `a2a.server.token` and stops there. The face ships
closed:
finishing an unrelated install is not consent to open a second network
surface, so
`raven a2a enable` is what opens it, and it is the only writer of that
flag.
Enabling mints a missing token in the same command, because a face
switched on with
an empty token answers every caller with a 401 and reads as a broken
deployment
rather than the closed door it is. `raven a2a disable` closes it again
and keeps the
token, so reopening does not re-issue a credential callers already hold.
The wizard
never writes the flag at all, not even as false, so a repeat onboard
leaves a face
the operator opened by hand alone. A config nobody onboarded, and any
`A2aConfig()`
assembled in process, stays inert.

What that opens is narrower than a network face. The app carrying it
binds loopback
at both call sites, hardcoded rather than read from config, and every
request is
checked against a freshly minted 256-bit bearer token. The minting write
is also
what fixes `config.json` to owner-only, from the temporary file's first
byte rather
than with a chmod afterwards; a later enable or disable adds no secret
material, so
it leaves whatever mode the operator chose. Nothing else narrows that
file and it
already held provider API keys, so the obligation predates this work and
is merely
discharged here.

There are two cards because the protocol splits one description across
two
authentication states. The public card answers an unauthenticated GET,
and has to:
a caller reads it to learn which scheme to authenticate with, so
requiring the
credential first would be circular. It therefore names no installed
agent. The
extended card is a `GetExtendedAgentCard` RPC riding the same bearer
check as every
other method, and is the only one of the two that may say what this host
can
actually do.

On that extended card the sub-agent roster rides
`capabilities.extensions`, not
`skills`. A skill is the protocol's word for something an agent can be
asked to do,
and a peer cannot ask for one named sub-agent: it can only send a
message to this
host, which then decides what to dispatch. One skill per sub-agent would
advertise
call targets that do not exist. What a peer can genuinely request is the
orchestration, so that is a single skill, and the roster moves to the
protocol's
own extension point under a versioned URI. That is the only place a
conformant card
may carry a payload the spec does not define, because `AgentCard` is a
closed set
of fourteen protobuf fields: a custom top-level key is rejected by a
strict parser
and silently dropped by a lenient one. The entry declares `required`
false, so a
reader that has never heard of the URI ignores it and still talks to
this host.

Both cards derive `supportedInterfaces[].url` from the origin the caller
arrived
on. One face answers under every name that routes to it, and a URL fixed
when the
face is mounted is right for at most one of them. This matters more than
it looks:
a caller that finds an interface off the card's own origin must refuse
it, since a
card fetched from a trusted origin would otherwise redirect the
credential
elsewhere, and a relative URL reads as off-origin to that check.

## Type

- [x] Feature

## Verification

- `make test-python` at this head: 23729 passed, 112 skipped, 1 failed,
in 288 s.
  The one failure is

`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`,
which fails with the same assertion on `origin/main` by itself; this
branch changes
  no install or node file.
- `uv run --frozen --all-extras pytest
tests/integration/test_a2a_protocol_e2e.py -q`:
6 passed. That directory sits outside the default pytest scope, so it is
run by
  hand rather than reached by the line above.
- `make lint-imports`: 10 contracts kept, 0 broken. `make lint-types`:
all checks
  passed.
- `uv run ruff check raven/ tests/`: clean. `ruff format --check`:
clean.
- `make check-commits` over all 68 commits, `make
check-source-language`,
  `make check-large-files`: all exit 0.
- Live, over a real network boundary rather than in process: a raven in
a Docker
container and a raven on the host, each running this branch, called each
other in
both directions. Nineteen assertions per direction covering both cards,
the four
refusal paths, and a completed turn. The container publishes its face on
a
different port than it binds, so the card has to name the port the
caller used and
not the socket the server opened; both cards did, in both directions.
That run
predates the default-off change described above, which touches no file
under
`raven/a2a/` or `raven/a2a_client/`: both packages are byte-identical to
the head
  it ran against.
- Reverting each fix turns its own tests red, checked one fix at a time
rather than
all together, including the interface-URL fix mutated out inside the
running
  container.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Behaviour changes for an operator who runs `raven onboard` after this
lands: a
token is minted into `a2a.server.token`, and `config.json` is narrowed
to
owner-only. No listening face is opened. Opening one takes `raven a2a
enable` and a
gateway restart, so the second network surface only ever exists because
somebody
typed for it.

Rolling an opened face back is `raven a2a disable` and a restart. The
token stays in
place and is never rotated, so callers already holding it keep working
when the face
is reopened, and a later onboard will not reopen it by itself. Nothing
else in the
install changes.

Costs and limits, stated rather than left to be discovered:

- A gateway that serves A2A imports `a2a-sdk`, measured at roughly 570
ms and 391
modules on top of the gateway's own stack, sqlalchemy and the
OpenTelemetry api
among them. It is paid once at mount, not per request, and an operator
who
switches the face off pays none of it. Moving it to the first request is
recorded
  as deferred in the design note rather than done here.
- `a2a_send` adds 157 tokens to the tool array of every turn, measured
off a
  captured provider call, and only on a host with a peer configured.
- The task store is in memory and unbounded, and inbound concurrency is
not capped.
A local face behind a bearer token is the deployment this is written
for; a
  host exposed more widely should bound both first.
- Pulling in `a2a-sdk` brings the OpenTelemetry api into the gateway's
import
graph. This is a trade accepted for the SDK's protocol types and
handler, not an
  adoption of OpenTelemetry by this project.

Checked during review and deliberately not fixed here: config validation
accepts an
agent row with an empty or whitespace name, which would reach the
extended card's
roster as an empty entry. It is not introduced by this work, the
previous shape
carried the same value, and closing it belongs with the agent schema
rather than
with the card.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-20 14:58:15 +08:00
Dizhan Xue
3dd06f6ea0
feat(docs): add a bilingual documentation site and its deploy pipeline (#507)
## Summary

Stands up a bilingual MkDocs site under `docs-site/`, published to
GitHub Pages
by a workflow that builds on every pull request and deploys only from
`main`.
Fourteen topics ship in English and Chinese: the quick start,
self-hosting,
Docker, the WebUI, the sandbox manual, the runtime architecture, the
command
reference, the repository layout, the developer workflow, the
proactivity design
and reference pair, the self-evolution map and the tracing API.

This pull request is additive. Both READMEs keep their prose and gain
one link
to the site in the header row; trimming them is deliberately a second
pull
request, so the site can be confirmed live before anything depends on
it.

Key decisions:

- **`docs-site/`, not `docs/`.** `docs/README.md` defines `docs/` as
design
notes and dated records, and `docs/plans/` is an archive explicitly not
held
  to today's layout. A user manual has the opposite lifecycle.
- **Full bilingual parity**, via the `mkdocs-static-i18n` suffix
structure
(`page.md` beside `page.zh.md`). Parity decays immediately without a
guard, so
  a test fails when a page has no twin.
- **Anchors are pinned on the Chinese side.** A CJK heading slugifies to
  nothing, so Markdown numbers it `_1`, `_2`, and inserting one heading
renumbers every anchor below it, silently breaking links readers already
hold.
Each Chinese heading therefore states the anchor its English twin gets
for
  free: 133 heading anchors, identical across the two languages, none
  positional.
- **A left rail, not a top bar.** The header carries the brand, the
search box
and the navigation tree down the left; the table of contents on the
right is
  one continuous polyline clipped to the headings currently on screen.
- **No template overrides.** Material customises through
`theme.custom_dir` with
Jinja partials, those partials are `.html`, and
`scripts/check_large_files.py`
  holds `.html` in `BLOCKED_ASSET_EXTENSIONS`. The skin is therefore one
stylesheet remapping Material's own custom properties, plus one script
for the
table-of-contents rail. The brand mark is a `data:` URI for the same
reason.
- **An owner-signed `docs-site/` exemption zone** in AGENTS.md section
1.3. The
i18n config carries the Chinese navigation labels, and `mkdocs.yml` is
not
`*.md`, so the blanket suffix exemption does not reach it. A contract
test
holds the AGENTS.md list and `EXEMPT_PREFIXES` equal, so both moved
together.
- **The site never links out to a file it does not host.** Documents
that stay
in the repository are named as plain paths rather than linked, and a
guard
fails a page that links to an external webpage or a repository-relative
path.

Two defects found and closed here rather than after publication. The
Chinese
site rendered an English sidebar: navigation labels live in
`mkdocs.yml`, not in
any page, so comparing the two languages' Markdown finds nothing and
only the
rendered output shows it. And a control could be present, sized,
animated and
answer `querySelector` while being unclickable, because a popup Material
sized
for a top bar landed outside the viewport once the header became a rail.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

All commands run at the pushed head, rebased onto `origin/main`.

```
make check-large-files                                         exit 0
make check-source-language                                     exit 0
make docs-build                                                exit 0, 0 errors
  docs-site/site/index.html and docs-site/site/zh/index.html   both present
  mkdocs_static_i18n: Translated 17 navigation elements to zh
uv run pytest tests/test_docs_site.py tests/test_source_language_check.py \
  tests/test_readme_scope_canon.py tests/test_docker_runtime.py \
  tests/test_living_docs.py                                    44 passed
uv run pytest tests/integration/test_docs_site_controls_e2e.py 13 passed
pre-commit run --files <changed files>                         all hooks pass
git diff --check                                               clean
```

Anchor parity was measured on the built HTML rather than assumed: 14
page pairs,
133 heading anchors compared at h2 through h6, every one identical
between the
two languages and none positional.

The controls are hit-tested rather than queried.
`document.elementFromPoint` at
the exact point a reader clicks must return the control itself, which is
the
only check that catches an off-viewport popup; the language switcher,
the search
panel, the repository link and the table-of-contents rail each pass it.

The brand lockup is measured, not eyeballed. A failing test was written
first
and reported the wordmark sitting 4.00px below the mark's centre line;
it is
0.00px after the fix, and a 4x screenshot puts the residual optical
offset at
0.25px.

Every guard this branch adds was confirmed to fail first. The
brand-class guard
was mutated from both sides in turn: renaming `.em-eyebrow` in the
stylesheet
turned it red, renaming `class="em-standfirst"` in `index.md` turned it
red, and
it is green again with both restored. The anchor guard was shown to
catch both
failure modes, a missing anchor and a wrong one.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

No behaviour change to any shipped code path. Nothing imports
`docs-site/`, the
new Makefile targets are additive, and the only change outside
`docs-site/` and
its tests is one link added to each README's header row.

Rollback is deleting the branch: the site does not exist until Pages is
enabled
with `Source: GitHub Actions`, and every other change is inert without
it.

Disclosed residue, checked and deliberately not changed:

- The README header now links to `https://evermind-ai.github.io/Raven/`,
which
404s until Pages is enabled. It is the same URL the Documentation
section
  already carries.
- The `deploy` job is skipped on pull requests by design, so a green
check here
does not prove deployment works. The first real evidence is the `deploy`
job's
  own conclusion on the `main` run after merge.
- GitHub Pages is not enabled on this repository. Measured: `GET
/repos/.../pages`
returns 404. Under `Deploy from a branch` the uploaded artifact is
silently
ignored while the workflow still reports success, so the source must be
set to
  `GitHub Actions`.
- `tests/integration/` is excluded by `norecursedirs`, so the browser
suite is a
  local guard and never runs in CI.
- The self-hosting page's Makefile target, docker path and port claims
are
unguarded. They were equally unguarded in the README they came from, and
the
  owner ruled against adding a guard for them.
- The self-evolution SOP and the benchmark contract stay in the
repository and
are named as plain paths from the pages that reference them. The SOP is
a
translation of an upstream project's internal document and carries
redaction
placeholders, which is a publication decision rather than a formatting
one.
- `docs/plans/` still shows the pre-restyle `mkdocs.yml`. CONTEXT.md
defines that
directory as an archive of the tree as it stood, with an explicit rule
against
  updating an old plan to match a later change.
- The Chinese pages drop the brand's italic serif accent word. Cormorant
Garamond
carries no CJK glyphs, so it would fall back to another face without any
sign.
- The code font is Material's default Roboto Mono. The brand names four
  typefaces and no monospace, so none was invented for it.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-09-19 07:06:24 +00:00
zhanghui
e70a868e9c
refactor(*): seal the memory backend seam so a memory plugin needs no host edits (#414)
## Summary

Make `MemoryBackend` the only seam between the host and the memory
subsystem, so a second memory plugin can be installed without editing
host code.

The abstraction is the goal; no second backend is being added.

What the branch does:

- `health()` joins the Protocol. `raven doctor` and `raven import` read
a `BackendHealth(ready, checks)` the backend fills in with its own
vocabulary, instead of reaching into EverOS internals. Doctor prints the
checks verbatim and never interprets a label.
- A new `[[plugin.contributes.onboard]]` kind, with
`raven/contracts/onboard.py` as its paper. The host lends the wizard
shell as one `OnboardUI` and records the result; the screen returns a
`StepOutcome`. The plugin never writes `memory.backend` -- that key
stays the host's. EverOS's step-4 screen and its `everos.toml`
management move into the plugin.
- The skill source and the after-turn feedback prefix are named from
`memory.backend` rather than the literal `everos`.
- `enabled_by_default` is gone from the manifest: an installed plugin
the user has not disabled is active.
- `tests/test_plugin_boundary.py` pins the dependency direction in both
directions, with an equality assertion so the remaining exceptions
cannot grow.

That guard's allowlist started at five host files still speaking
EverOS's wire protocol. The follow-ups below close three of them.

Two things this branch itself introduced were found in manual acceptance
on an install without the plugin and fixed here: a turn printed which
backends were installed -- empty on exactly the install that needs the
sentence -- and `settings.everos` / `settings.everosSet` let
`ModuleNotFoundError` reach the dispatcher as an internal error carrying
a traceback. Both now carry the note doctor and the wizard already use,
naming the distribution to install.

Review on this PR found four blockers, all fixed and covered. A fifth
defect came out of manual acceptance rather than review, and four more
out of the rounds that followed it -- every one of those in the
embedding endpoint's own plumbing, each found by the round after the fix
that introduced it. They are listed under Risk; the blockers are:

1. `onboard` is an optional contribution, but the wizard equated "no
matching screen" with "not configured", so one `raven onboard
--skip-memory` wrote `memory.backend` back to null for a third-party
backend configured by hand. A backend with no screen now answers on the
strength of having been selected; only one whose own screen says it is
unconfigured is cleared, which is the case that rule exists for.
2. The record a sub-agent call leaves behind ran through a backend
nobody had started. The shipped adapter begins in an unknown state, so
the first flushing write only scheduled a readiness probe and answered
false -- the record said "unavailable" while the service was up the
whole time, and each per-record HTTP client was left open. One context
manager now owns start and stop around a record, in both the manager and
the DAG runner.
3. That record's write and the recall after it addressed different
owners: a real config spells its keys in camelCase and the block reaches
the backend as written, while the adapter read only the contract's
snake_case, so a sub-agent's memories were filed under the host
identity. The adapter reads both spellings.
4. Legacy embedding discovery had narrowed to the recorded plugin root,
so an install predating root recording lost its knowledge-base endpoint
with a valid file still on disk.

**The five follow-ups the seam left behind are in this PR too**, rather
than in three stacked ones, because they are one goal and the branch
protection dismisses an approval on every push:

- The embedding endpoint is raven's. A knowledge base indexes inside the
gateway process and never speaks to the memory service, yet it read its
endpoint out of everos.toml, so selecting another backend left it with
no vectors and no message naming the cause. Raven holds the block now
and hands it to the backend at start -- as a shared default, not an
override: a plugin that was given an endpoint of its own keeps it, and
only one that has none takes raven's. Discovery of an existing
everos.toml still checks all three places the plugin itself looks, read
with plain tomllib so it works where the plugin is not installed. No
data migrates: a base is keyed on its model and measured width, not on
the base url.
- `delete()` joins the Protocol. The browser used to delete a row out of
EverOS's index, in the host process, through the plugin's repository
layer -- while the markdown EverOS treats as the source of truth kept
the text, so the memory came back on the next rebuild, after the user
was told it was gone. The adapter now uses only operations EverOS
already performs, for the kinds it performs them for.
- Sub-agent memory goes through the contract. The read half is a new
optional `recall_session`; the write half needed no new surface, only
two metadata conventions written down. `agents.<name>.config.everos`
becomes `config.memory`, opaque to the host except the two keys the host
reads; the old spelling still loads.
- The memory and settings pages say why they are empty -- no plugin, or
a backend this page does not read -- instead of four zeros and a retry
button that cannot help.
- The `everos` package leaves the host's dependencies for the plugin's
own, at the same exact pin; the host reaches the tree through the
plugin. This is the package layer only: the installers already deliver
`everos-memory` as a sibling wheel from the same release, so an ordinary
install still has long-term memory.

The boundary guard's allowlist falls from five entries to two, both GUI
surfaces still shaped around EverOS's own store. Generalising their wire
is a frontend change and is deliberately not in scope.

This branch was reviewed on GitLab as merge request 639 over three
rounds, including a post-rebase range-diff check. It is being moved here
because GitHub is now the source of truth.

## Type

- [x] Refactor

## Verification

- `uv run pytest` on this branch, rebased onto `main`: 22146 passed, 4
failed, 45 skipped. Three of the four (`test_agents_oncall_launcher`,
`test_agent_loop_token_budget`, `test_agents_code_launcher`) were re-run
on a detached checkout of that same `main` commit in this environment
and fail there identically. The fourth (`test_ppt_engine_image_search`)
could not run in that checkout, whose environment lacks the deck
engine's extras; this branch's only change anywhere under
`plugins-dist/ppt-engine` is the removal of the `enabled_by_default`
manifest key, which that test does not reach. So this branch adds no
regression.
- `npx vitest run` in ui-web: 1680 passed across 103 files. `npm run
type-check` and `npm run gen:check` clean, so the generated client
matches the contract.
- Every fix in the follow-up commits was mutated away and its test
confirmed red, then restored and confirmed green. For the record path
that is three separate mutations: dropping `start()`, dropping `stop()`,
and narrowing the owner lookup back to one spelling.
- `uv run ruff check .` and `uv run ruff format --check` on the changed
files: clean.
- `uv lock --check`: consistent.
- `scripts/check_commit_messages.py`, `scripts/check_large_files.py`,
`scripts/check_source_language.py` over the branch range: clean.
- Manual acceptance against a real isolated install (temporary
RAVEN_HOME, isolated EverOS root and port), 34 cases: `raven plugins`,
`raven doctor` across four backend states, `raven onboard` interactive
and non-interactive including all three step-4 lanes, `raven import`
refusing and importing, an end-to-end write and recall, the web UI's
memory and settings pages driven through a real browser, a sub-agent
trace record, and an install with no memory plugin at all. 31 passed; 1
failed and was fixed; 1 partial; 1 carried over from the earlier round.
- The failure is the one worth reading. On a fresh managed install the
wizard reported embedding configured and the service it started the next
moment ran keyword-only: the spawned process carried no
`EVEROS_EMBEDDING__` values, and EverOS logged that the embedding key
was not configured. The rule deciding whether everos.toml already held
an operator's own endpoint asked whether its model name was a `"<...>"`
placeholder; the shipped template seeds every section with a real model
name and an empty key, so every fresh install read as a deliberate
choice and the host's endpoint was never bound. It now asks the
criterion the module already had -- model and key. The unit tests missed
it because each built its "operator's own" section with a key in it; the
regression copies the shipped template itself.
- The partial: the memory page's search returned relevant results
without error, but every memory in the test corpus was on one subject,
so narrowing could not be observed. The carried-over case is the
skill-source prefix, whose path this branch does not touch.
- Findings recorded and not fixed here, none introduced by this branch:
a progress spinner is still drawn after `import` reports it imported
nothing; an unsupported delete surfaces as a generic internal error; the
TUI log and `doctor` word "no memory plugin" differently; a turn that
did reach disk still prints that it was not written.
- The partial case: an induced agent skill is discovered through
`find_skill` with the backend's prefix, but `read_skill` / `use_skill`
on that id fail. Nothing writes the on-disk layer those tools look in,
on this branch or on `main`, and no test covers it. Pre-existing; filed
as a follow-up.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes:

1. User-track recall is now strictly bounded by `top_k`. A profile row
used to ride outside that bound.
2. `raven doctor` builds the plugin factory, so a managed EverOS setup
writes its env var and config templates during a doctor run. It used to
be read-only. The `--json` memory section is reshaped around
`health.checks`.
3. A third-party drop-in plugin without `enabled_by_default = true` used
to be skipped and now loads.
4. The loader no longer back-fills the EverOS config slice's `root` /
`owned`, nor strips a legacy `mode` key; a leftover `mode` now draws an
unknown-key warning.
5. The wizard's recap shows the contribution name `everos` rather than
`EverOS`.
6. `use_skill` with an unknown prefix reports "not found" after a disk
lookup instead of "unknown source".

From the follow-ups:

7. Config gains an optional top-level `embedding` block. Absent, the
knowledge base reads the EverOS file as before and says so once. An
operator who moves the values across changes nothing else; one who names
a *different* model there gets the existing `StaleBaseError` telling
them to rebuild that base. The block is a default for the memory plugin
rather than an override, so an install that had configured the plugin
separately keeps what it had. Note that a config carrying this block
does not load on an older Raven, which rejects unknown top-level keys.
8. `agents.<name>.everos` is written back as `agents.<name>.memory`. The
old spelling still loads; a per-agent `baseUrl` is ignored with a
warning, having never had a user.
9. Deleting a memory now changes what EverOS treats as the source of
truth, so it stays deleted. Profiles and cases report that they cannot
be deleted rather than appearing to succeed.
10. `pip install raven` no longer pulls the memory engine in.
`install.sh` and `install.ps1` are unaffected: both already resolve
`everos_memory-*.whl` from the release they install and pass it as
`--with everos-memory`, and the source path installs it editable, so
those installs keep long-term memory. An install that reaches neither --
a release predating the sibling wheel, or a bare `pip install` -- now
boots without memory and says which distribution to install rather than
failing on an import.

Found after the acceptance run, in review, all fixed here. They share
one cause worth stating once: the endpoint acquired a second home, and
each fix taught one more reader about it while leaving the next one
behind.

11. The endpoint the wizard saved never reached the EverOS process it
then started. The rule for "does everos.toml already hold an operator's
own endpoint" asked whether its model name was a `"<...>"` placeholder;
the shipped template seeds every section with a real model name and an
empty key, so every fresh managed install read as a deliberate choice
and the host's endpoint was never bound. The service came out of the
wizard running keyword-only while the wizard said embedding was
configured. The rule now asks the criterion the module already had:
model and key.
12. The exported `EVEROS_EMBEDDING__*` variables are a third home,
documented by EverOS and ranked above its file. They were being
overwritten. A complete triple is now an operator's choice; an
incomplete one is not an endpoint and is replaced whole rather than
topped up from the host.
13. Raven writes those same variables itself at startup, so a reader
that asked only whether all three were set called raven's own binding an
operator's export -- a blank settings card and a save refused by naming
variables nobody had set. The module now records which of them it set.
14. That record lived in a module-level set, and `raven gateway
--restart` re-launches through `os.execv`: the values survive, module
state does not. The record now travels in a variable beside them, and is
stripped from the environment handed to EverOS.
15. A settings save the environment would outrank is refused with the
variable names, rather than written and silently ignored.

Four of the five arrived one per review round, each in the fix for the
one before it. The tests that missed them all shared a shape: each built
its "an operator configured this" fixture with a key in it, or cleared
the environment before the case, so the states the product actually
produces -- a template with no key, a live process that has just bound
its own endpoint -- were never the input. The regressions now copy the
shipped template, and run the startup sequence and a second interpreter
rather than a prepared state.

Rollback is a revert of the branch: the seam is additive, and
`memory.backend` keeps its shipped default.

- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-16 22:19:50 +08:00
Dizhan Xue
e0d34fd11d
fix(cli): work that belongs to no session still has to show (#406)
## Summary

The tracing viewer keys its entire payload on a session id, and three
readers drop a span that has none. On the measured store that is **683
of 1,956 spans (35%)**, dropped silently: every `plugin.load` ever
recorded, every cron heartbeat, and the titles generated after a turn
ended. A day whose only activity was scheduled reads as a day with no
activity at all -- which is how three consecutive days (09-12 ... 09-14)
came to look empty in the panel while 154 spans sat on disk.

A span with neither `session.id` nor `session.key` now gets a session
derived from its own `startTime`, one per calendar day
(`background:<day>`). The derivation is **span-local on purpose**: the
whole-corpus reader in `server.js` and the per-file shard index are
already pinned against each other by test, and any derivation consulting
global state would let them elect different ids for the same span. A day
is the grain because per-trace would shatter a year of timer ticks into
a session each. A span that has a key but no id is untouched -- the
identity election resolves that one, and still drops it when it cannot.

`SCHEMA` goes 2 -> 3. The sidecar records nothing about the code that
built it, so leaving it at 2 would let every index already on disk keep
answering with the old contents: the fix would ship and change nothing
for the history it was written for. The comment above `SCHEMA` already
describes that failure from the last time it bit.

**This reverses a deliberate decision, and reviewers should weigh it as
such.** `server.js` carried an explicit rationale for the drop (added in
7b45527b5): *"there is nowhere in a session-oriented view to show it,
and on the measured store there are 34,644 of them."* Both halves are
addressed rather than ignored -- the per-day session is the place to
show it, and the per-day grain is what keeps the count in the same order
of magnitude as a real session. That comment is updated rather than left
contradicting the code.

`docs/specs/2026-08-25-tracing-viewer-sharded-reads-design.md` records
the narrowing: its constraint "a span cannot be attributed to a session
by looking at the span" is a statement about spans carrying a
`session.key`. A span with neither has no corpus fact to recover, which
is why attributing it span-locally is sound rather than a shortcut past
the constraint.

## Review round

Three findings, one commit each, all filed as non-blocking and all
fixed.

**The comparison tests could not see a reader disagreeing.** Both tests
that hold
the sharded reader against the whole-corpus one ran on a fixture whose
every span
carried both identity fields, so neither ever built a background
session. Changing
the day grain on one side alone left all 21 green. The fixture now
carries a
sessionless span, and the same change fails both comparisons.

**The panel opened on a background day.** The list is newest-first and
the page
takes the first row; background work is emitted continuously, so the
newest record
in a store is usually one of those. This branch therefore changed the
row the panel
opens on, a default nobody chose. The row now declares `isBackground`
and the page
prefers a conversation. The declaration is on both readers, which the
field-for-field
comparison between a list row and a whole-corpus session then holds
together.

**A sessionless trace across UTC midnight had no single owner.** Its
spans land in
two day rows, and the trace-id lookup returned whichever row iteration
reached
first. It now returns the earliest owner, which is the single-owner rule
the
trace-id union already states.

The day grain is deliberately unchanged. Deriving it per trace would
make one bucket
per sessionless span: measured on a real store, 770 sessionless spans
carried 770
distinct trace ids, which takes one file's pair count from 3 to 341 --
avoiding that
is why the grain is a day. So the straddling trace's spans stay split
across two
rows and only the ownership is settled. Measured incidence of a straddle
on that
store is zero, so its test forces the state rather than reporting a
failure anyone
has seen.

The review also named the `undated` fallback as reached by no test. It
is reachable rather than defensive -- `normalizeSpan` passes `startTime`
through as it finds it -- so it now has one, asserting the pool exists
and the span arrives through it. No writer emits an undated span and
none is on the measured store.

`SCHEMA` is untouched by this round: nothing here changes what a sidecar
holds.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
pytest tests/integration/test_tracing_viewer_e2e.py -m integration     24 passed  (5 new)
pytest tests/test_tracing_api.py tests/test_cli_tracing_commands.py \
       tests/test_tracing_compact.py tests/test_tracing_artifact_v2.py 131 passed
ruff check / ruff format --check                                       clean
scripts/check_commit_file.py                                           exit 0
node --check server.js shard-index.js                                  clean
```

Against the real store (span logs copied to an isolated dir; artifacts
untouched):

| | sessions | distinct spans reachable |
|---|---|---|
| before | 4 | 1,266 |
| after | 9 | 1,950 |

The five new per-day sessions match the previously measured per-day
drops exactly: 188 / 341 / 48 / 48 / 59. All three readers agree
(`/api/data`, `/api/sessions`, `/api/llm-calls`), and
`/api/sessions/background:2026-09-14` returns 200 with 59 traces.

**Five new tests, each watched failing first.** The second of the
original pair matters more than it looks: with `SCHEMA` left at 2 it
fails because a sidecar written by the previous build keeps hiding the
spans -- the fix would have shipped and done nothing for existing
archives. Of the three added this round, the reader-comparison one was
run against the review's own mutant both ways (21 green before the
fixture change, 2 failed after), the trace-owner one against a revert of
its fix, and the undated-pool one against a build with the `undated`
fallback removed.

Rebased onto `56c79edf5` and re-verified there: `git range-diff` maps
the branch one-for-one, the merge commit the previous revision carried
is flattened away, and the counts above are from the new base.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered -- no new input is parsed; the derived
id is built from a span's own `startTime`, already validated upstream.
- [x] Backward compatibility considered -- the `SCHEMA` bump invalidates
existing sidecars, which are a cache: they rebuild on first read, no
manual cleanup. Session ids in the `background:` namespace are new
values in an existing field; no consumer is keyed on the set of ids.
- [x] Rollback path is clear -- revert restores the previous behaviour;
sidecars written at schema 3 are rejected by the old code and rebuilt at
2.

### Two things a reviewer should know

1. **CI does not execute these tests.** `norecursedirs` keeps
`tests/integration` out of the default scope and `addopts` carries `-m
"not integration and not e2e"`; `ci.yml` runs exactly one integration
file (`test_cli_upgrade_real_uv.py`). The 21 tests here -- the 19 that
already existed and the 2 added -- only run locally. Unchanged by this
PR, but it means the regression net is manual.
2. **A known display rough edge, deliberately not fixed here.**
`ui/app.js` truncates with `shortId(sessionId, 18)`, and
`background:2026-09-14` is 21 chars, so the five cards read
`background:2026-09...` -- distinguishable by the start/update times on
the card and by the full id in the tooltip, but not by the title. Fixing
it means either widening a presentation constant or shortening the id;
the UI has no test harness, so it is left as a separate call rather than
changed blind.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-15 19:51:33 -07:00
0xKT
a27559e331 fix(*): the final audit closes consent, tty truth, and the cache lies
An adversarial last pass over the whole branch found what the review
rounds had not. Consent first: the sh decline arm matched only bare n/N,
so a typed "no" installed LibreOffice through sudo; the arm now takes
the word too, and a tripwire pins the default-yes prompt in both
installers so the review-round decision cannot silently flip back. The
tty gate probed existence where openability is the truth -- the node
exists in CI, cron and docker -t without -i -- so the gate now opens it,
and a consoleless run skips instead of defaulting yes or hanging.
install.ps1 finally carries the everos memory wheel through both of its
paths with sh's own rung discipline, so the Windows summary's memory row
can be green at all. The two cache resolvers follow playwright's real
lookups (XDG_CACHE_HOME, LOCALAPPDATA, chrome-win64 layouts), a
non-object registry degrades instead of raising, and the four mouths
that name a fix now say the same runnable thing on every platform.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-12 00:50:35 +08:00
0xKT
d08a416de4 build(*): the three playwright pins read as one specifier
The same library was pinned three ways: the host browser extra said
>=1.49,<2.0 while the design render extra and the ppt-engine carrier
both said >=1.58,<2. One environment resolves one version, so the host
floor rises to match through uv (lockfile re-resolved in the same
change; the resolved version does not move). A contract test now holds
the three declarations byte-identical and each table to exactly one
playwright entry, so a lone bump -- or a cleanup that reads the
unimported carrier as stale -- turns red here before it breaks a
rendering install. dev.md gains the one line naming how a source
checkout gets the chromium binary itself.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-11 23:48:28 +08:00
0xKT
ab8eb7d498 fix(ppt-engine): give a page box the names it promises and rasterise one at a time
## Summary

Three defects a live twenty-page run surfaced, each of which cost whole pages.

`PageBox` said in its own docstring that it reads the same as `ppt_layout.Box`, and `Box`
answers to `.x` and `.y` as well as `.w` and `.h`. `PageBox` answered to two of the four.
A build helper read `round(b.y, 3)` once, in the one place it used for every page, and all
twenty pages failed to draw with `'PageBox' object has no attribute 'y'`. `Box` was given
the same pair after the same crash; the promise was the part that did not travel.

`_check_cut_out` asked its argument for `.name`, and the spelling the skill documents is
`replace_picture(shape_at(slide, n), str(FIGURES / image), fit)`. Every other branch takes
that string -- the ones that open the file hand it to PIL, which does -- so the only thing
that refused it was the warning that exists to explain a mistake. It raised
`'str' object has no attribute 'name'` over the top of the explanation and took the page
with it.

Rasterising now has a gate of its own, and it is one. `pdf()` spends its time in a
LibreOffice subprocess with a profile of its own, so width there costs nothing and buys
wall clock; `pages_of` spends its time in pypdfium2, in a worker thread of this process,
against a library that keeps state per process rather than per document. One gate for both
put them under the same number and let the second run concurrently with itself: over a
nine-template reference set, six of the nine PDFs failed to open with
`PDFium: Data format error` while every one of the nine opened when read one at a time, and
an earlier run on the same path took SIGSEGV. `pages_of` answers a failed read with an
empty dict, so the cost was a reference sheet quietly short of two thirds of its pages
rather than anything that looked like an error.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Commands run on this branch, rebased onto main:

```
uv run --frozen pytest tests/ -k ppt -q
uv run --frozen pytest tests/test_ppt_engine_template.py -q \
  -k "page_box_answers or cut_out_warning_survives or rasterising_runs_one_at_a_time"
uv run --frozen ruff check plugins-dist/ppt-engine/raven_ppt/services/template/compose.py \
  plugins-dist/ppt-engine/raven_ppt/stages/_views.py
```

The three added tests were each confirmed to fail without the source change. The
concurrency one asserts on how many rasterisations are in flight -- the property -- rather
than on the width of a semaphore, which is one way to get it, and its stub renderer carries
the real signatures on purpose: a stub that shapes itself to the caller agrees with a call
the engine cannot make.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Two added properties and one widened type annotation are additive. The third change
narrows rasterising to one at a time while leaving conversion as wide as it was, so a deck
whose pages are read after conversion takes slightly longer in exchange for reading all of
them; reverting the commit restores the shared gate.

## Related Issues

N/A
2026-09-11 07:22:24 +00:00
Tchen
94ed54b78d fix(ppt-engine): give a page box the names it promises and rasterise one at a time
## Summary

Three defects a live twenty-page run surfaced, each of which cost whole pages.

`PageBox` said in its own docstring that it reads the same as `ppt_layout.Box`, and `Box`
answers to `.x` and `.y` as well as `.w` and `.h`. `PageBox` answered to two of the four.
A build helper read `round(b.y, 3)` once, in the one place it used for every page, and all
twenty pages failed to draw with `'PageBox' object has no attribute 'y'`. `Box` was given
the same pair after the same crash; the promise was the part that did not travel.

`_check_cut_out` asked its argument for `.name`, and the spelling the skill documents is
`replace_picture(shape_at(slide, n), str(FIGURES / image), fit)`. Every other branch takes
that string -- the ones that open the file hand it to PIL, which does -- so the only thing
that refused it was the warning that exists to explain a mistake. It raised
`'str' object has no attribute 'name'` over the top of the explanation and took the page
with it.

Rasterising now has a gate of its own, and it is one. `pdf()` spends its time in a
LibreOffice subprocess with a profile of its own, so width there costs nothing and buys
wall clock; `pages_of` spends its time in pypdfium2, in a worker thread of this process,
against a library that keeps state per process rather than per document. One gate for both
put them under the same number and let the second run concurrently with itself: over a
nine-template reference set, six of the nine PDFs failed to open with
`PDFium: Data format error` while every one of the nine opened when read one at a time, and
an earlier run on the same path took SIGSEGV. `pages_of` answers a failed read with an
empty dict, so the cost was a reference sheet quietly short of two thirds of its pages
rather than anything that looked like an error.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Commands run on this branch, rebased onto main:

```
uv run --frozen pytest tests/ -k ppt -q
uv run --frozen pytest tests/test_ppt_engine_template.py -q \
  -k "page_box_answers or cut_out_warning_survives or rasterising_runs_one_at_a_time"
uv run --frozen ruff check plugins-dist/ppt-engine/raven_ppt/services/template/compose.py \
  plugins-dist/ppt-engine/raven_ppt/stages/_views.py
```

The three added tests were each confirmed to fail without the source change. The
concurrency one asserts on how many rasterisations are in flight -- the property -- rather
than on the width of a semaphore, which is one way to get it, and its stub renderer carries
the real signatures on purpose: a stub that shapes itself to the caller agrees with a call
the engine cannot make.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Two added properties and one widened type annotation are additive. The third change
narrows rasterising to one at a time while leaving conversion as wide as it was, so a deck
whose pages are read after conversion takes slightly longer in exchange for reading all of
them; reverting the commit restores the shared gate.

## Related Issues

N/A
2026-09-11 07:22:24 +00:00
gloryfromca
201293cf1c refactor(sandbox): the sandbox is told where its state lives
## Summary

`raven/sandbox/_runtime.py` read `raven.config.paths.get_sandbox_dir` to find the boxlite home, while `raven/config/schema.py` imports `raven.sandbox.config` for `SandboxConfig` -- a two-edge cycle between the two packages. An executor asking the configuration layer where its own data lives is the inverted half of that pair: it can be told instead.

`get_boxlite_runtime` now takes the home. `BoxliteExecutor` and `SandboxDebugServer` take a `sandbox_home` and pass it down. `build_executor` takes `sandbox_dir` -- the resolver, normally `get_sandbox_dir` -- and calls it only inside the boxlite branch.

The resolver rather than an already-resolved path, because resolving one creates the directory and `none` is the default backend: an eager path would leave every install a boxlite home it never uses. That laziness has a test, and the test goes red if the resolve moves above the branch.

`SandboxConfig` gains no field. Its model carries an alias generator, so a field there would become a user-writable knob in `config.json`, and a resolved runtime path is not a declaration. Five production call sites pass the resolver, all in packages that already import `raven.config`, so no new edge appears.

Measured on `origin/main` in the same environment, before -> after: packages in cycles 18 -> 17, mutual pairs 13 -> 12 (`config <-> sandbox` is gone), modules in cycles 42 -> 42. The ratchet ceilings in `tests/test_import_cycle_budget.py` are deliberately left where they are.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_sandbox_unit.py tests/test_sandbox_debug_server.py tests/test_agent_loop_workdir.py tests/test_cli_sandbox_commands.py tests/test_playbook_mcp.py tests/test_subagent_manager.py tests/test_import_cycle_budget.py -q` -> 336 passed
- `uv run lint-imports` -> 9 kept, 0 broken
- `uv run ruff check raven/ tests/` -> All checks passed
- `uv run ruff format --check` on the touched files -> already formatted
- `make check-source-language` -> exit 0
- The laziness guard was mutation-checked: moving `sandbox_dir("boxlite")` above the backend branch turns `test_the_sandbox_home_stays_unresolved_for_a_backend_that_has_none` red; reverting turns it green
- The integration suites under `tests/integration/` are updated to pass the real `get_sandbox_dir("boxlite")`, but were not executed here (they need a real VM)

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No behaviour change on the boxlite path: the same directory reaches the same runtime, and the memo cache is still keyed on it. On the `none` path the directory is no longer created, which is the intended difference. Any caller that constructs an executor without the new argument fails at call time with a `TypeError`, not silently. Rollback is a revert of the single commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 07:19:34 +00:00
Tchen
5b4cce3f43 feat(ppt-engine): deliver where the user named, one route onto a page, and the ten templates
## Summary

This branch folds four merge requests and one unpushed commit into one, so that `main`
takes one review, one approval and one merge instead of four, and so the eight-file
overlap between them is resolved once. It carries, in this order:

- **!602** `feat/ppt_deliver_to_named_path` -- `ppt_build` takes `deliver_to`; every
  publish writes the deck under `out/` and copies the same measured bytes to the path
  the user named, recording it with `role: "delivery"`. The PDF preview rides along only
  where its own `.pdf` name is free: a file already at that stem is the user's and is
  left alone, and the reply says the preview stayed under `out/`.
- **!628** `feat/ppt_copy_route_and_page_isolation` -- `adapt` is removed, so cloning a
  template page and writing into it with `replace_text` is the only route; a page whose
  block raises loses only itself and the pages after it still build; a draft counts
  against the tier's build cap and one is released at it.
- **!632** `feat/ship_templates_and_soffice_dep` -- the ten bundled deck templates are
  tracked in git with per-file sha256 pins, so a fresh clone can build a deck; the
  LibreOffice dependency is declared and resolved through one resolver.
- **!642** `fix/ppt_copy_route_and_clone_typography` -- a copy-route lookup that finds
  nothing names the near miss before the inventory; a text frame with no paragraph at
  all can be written into; a cloned page's title size is said; `row_type_drift` compares
  the same structural slot across a run's units.
- **D57's other-page clause, rewired.** D57 gave five refusals a "did you mean" hint,
  and one of them also answered "the string you named is on page N of this template".
  That clause needs the file a page was cloned out of, and the only refusal holding it
  was `adapt`'s, which !628 removed. It is restored by reading `PPT_TEMPLATE_SOURCE`,
  which the runner already sets in the author's own process and which this module
  already reads for `bundled()`; the two helpers behind the clause are unchanged.
- **D60** `read the ground under each word, not across the whole box` -- the `unreadable`
  contrast gate also measures the render's own word boxes, as extra regions through the
  same guards, so a two-line block crossed by a narrow bright ridge can no longer elect
  the dark ground and read as legible.

Decisions the union forced, none of which drops a behaviour:

- The publish record is built !602's way, with its delivery `role`, and written the way
  D49 requires -- read back off the file, atomic, and refused rather than silent when
  the record cannot be written.
- `clear_region`, `shapes_in`, `page_box` and the refusal of a bare four-number region
  survive `adapt`'s removal; so do `_heading_rows` and its helpers, which the row
  reading loads even though the call they were written for is gone.
- The author-facing guidance names !628's single route and keeps D59's rule that a
  region must declare which reading it is.
**The decision log carries a numbering collision, on purpose.** It is a set union with
no wording changed, and the numbers are left exactly as each branch wrote them, so a
reviewer meets this rather than discovers it:

- !602 and !628 each wrote a **D37** and a **D38**, about different decisions.
- !628's **D45** is not `main`'s D45; they are unrelated entries under one number.
- !628's **D42** is `main`'s **D45** word for word, so the merged log carries one
  decision under two numbers.

Renumbering touches every cross-reference in the log, and doing that inside a
seventy-commit fold immediately before a merge is how a silent break gets in. The
numbering is being fixed on a separate branch, which carries a script and a test that
fails when a citation points at nothing.

## Type

- [x] Feature
- [x] Fix
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on an interpreter carrying `python-pptx`, with `PYTHONPATH` at this worktree and its
`plugins-dist/ppt-engine`, confirmed from a neutral working directory that `raven` and
`raven_ppt` resolve out of the worktree and not the host repo.

- `make test-python` (the whole suite, as CI runs it) -- 20826 passed, 49 skipped, and
  26 failures this machine produces on any branch: it has no `/usr/share/zoneinfo`, so
  every cron timezone case raises `ValueError: unknown timezone 'Asia/Shanghai'`. Checked
  out plain `origin/main` in a clean worktree and ran the same two files: the same 6
  failed and 20 errors, the ids identical character for character. Nothing on this branch
  touches cron or timezones. The stale `test_trunks_executor_would_have_dropped_that_output`
  that failed on the previous revision is retired on the new base and its replacement,
  `test_trunks_executor_preserves_output_on_timeout`, was run on its own and passes.
  The 6 failures and 20 errors are all cron timezone tests failing identically on plain
  `main` in a clean worktree, on a host with no `/usr/share/zoneinfo`.
- `make check-commits` -- exit 0.
- `make check-source-language` -- exit 0.
- `make check-large-files` -- exit 0.
- `make lint-imports` -- exit 0, 9 contracts kept, 0 broken.
- `ruff check` and `ruff format --check` over the 80 changed Python files -- exit 0.
- `pytest tests/test_ppt_engine_templates.py tests/test_utils_office.py
  tests/test_cli_doctor_commands.py` -- 97 passed; the ten templates are tracked and
  satisfy their pins, and the soffice resolver finds the local binary.
- The delivery contract, the record read-back, the page isolation and the near miss were
  each run as a measurement rather than asserted; the `unreadable` gate was run through
  the real LibreOffice render chain on a delivered 12-page deck and reports BLOCKING on
  page 12 under one word at 1.6:1.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes: `ppt_build` gains `deliver_to` and delivers outside `out/`;
`adapt` is gone from the author-facing template module, so a program calling it fails to
import -- `title=` and `subtitle=` have no replacement and the heading is named by the
words the template put there; a draft now counts against the tier's build cap; three
gates that nothing measured against being wrong no longer refuse; and the ten templates
are now bytes in the repository rather than a private registry fetch.

Two pieces here are about not destroying something the user has. The delivery
refuses to write its `.pdf` where a file the user already owns sits at the same
stem, rather than replacing it; and the large-file roster moved out of the manifest
inside the tree being checked and into the gate's own source, so a change can no
longer grant itself the exception the gate exists to withhold.

Rollback is per piece: each of the five sources is a contiguous run of commits on this
branch, in the order listed above, so any one of them can be reverted as a range without
touching the others.

- [x] Backward compatibility considered
- [x] Security impact considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 05:52:36 +00:00
0xKT
8716ec525c feat(docker): run raven in a docker container and make a first run configurable from the page
## Summary

Two things arrive together because the second is what made the first usable.

**Run raven from a container.** A Dockerfile builds the page, the Python
environment and the engine into one image; `cd docker && docker compose up`
serves it. nginx serves the built page and proxies the RPC socket to the engine
over the loopback they share -- the engine binds 127.0.0.1 only, so a proxy in a
second container would have no address to dial. Compose publishes on 127.0.0.1,
and the entrypoint prints a one-time sign-in link on boot.

**A first run is no longer a dead end.** The gateway used to refuse to start
until a provider was configured, while being the thing that serves the page
where a provider gets configured. It now starts, says what is missing, and picks
up a key saved in Settings on the next turn without a restart. The onboarding
overlay no longer takes the page either: New chat, the model chip and Send send
the reader to Settings > Models instead, and an empty model list is marked on
the control that fills it.

Also carried: the provider catalogue registry (capabilities, modalities and
context windows per model), the reworked Settings > Models pane, and four fixes
found in review of that work -- a sibling catalogue answering 200 with a
non-catalogue body no longer crashes the whole fetch; adding a tag no longer
erases the label and description already stored for that model; `authenticated`
no longer demands a bearer token from a local provider reached by address; and
registry rows are keyed by the spelling every lookup produces, which recovers
593 rows that no reader could reach.

## Type

- [ ] Fix
- [x] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Commands run on the branch head, with their results:

- `uv run pytest tests -q -k "provider or registry or catalog or model or rpc or gateway or docker or i18n"` -- 4389 passed, 46 skipped
- `npx vitest run` in `ui-web` -- 1573 passed (101 files)
- `npx vitest run` in `ui-tui` -- 2037 passed (142 files)
- `uv run ruff check raven scripts tests` and `ruff format --check` -- clean
- `npx tsc --noEmit` in `ui-web` and `npm run type-check` in `ui-tui` -- clean
- `npm run lint:rpc` in `ui-tui` -- generated.ts in sync
- `make check-source-language`, `make check-large-files` -- clean

Each of the four review fixes was checked by reverting it and watching a named
test go red, then restoring it.

The container image itself has not been built by CI or by me -- no Docker daemon
was available on the machine this was written on. It has been built and run by
the author of the branch.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Two deliberate security trade-offs in the container deployment, both documented
in `docker/README.md`:

- nginx rewrites the `Origin` header, because the engine accepts only an Origin
  naming its own loopback port and no browser reaching a published port can send
  one. This drops a defence-in-depth layer. The session cookie is
  SameSite=Strict, so an unauthenticated socket is still refused -- but SameSite
  ignores ports, so another page on localhost counts as same-site.
- `RAVEN_AUTO_LOGIN=1` is the default: nginx attaches the session cookie to
  every page load, so anyone who can reach the published port is signed in. It
  is contained by Compose publishing on 127.0.0.1 only. Set it to 0 for any
  deployment reachable from elsewhere. These are two settings that must agree,
  and nothing warns when they do not.

Behaviour changes outside the container: the gateway now starts without a
configured provider rather than exiting 1, and the first-run onboarding overlay
no longer opens by default (`?onboard=1` still forces it).

Rollback is by revert; nothing here migrates state or changes an on-disk format.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 03:27:52 +00:00