33 Commits

Author SHA1 Message Date
Dizhan Xue
39deee89ef
fix(*): stop uv's deprecation warning on upgrades and pin the store hand-over (#896)
## Summary

System-store upgrades passed both certificate-variable names to uv,
causing a deprecation warning on releases that warn even when the
replacement variable is present. Ask the uv used for installation for
its version and add `UV_NATIVE_TLS` only for releases before 0.11 or
when the version cannot be read. Explicit user settings remain intact.

- Keep certificate-store handoff in a fresh private directory, remove it
from the helper's import path after loading, and test both properties.
- Pin ordinary sub-agent connection failures as `network_error`,
distinct from certificate refusals.
- Let `security.tls.system_ca_in_use()` own the query of the actual
injected SSL factory. Upgrade and integration fixtures use the public
switch rather than duplicating its implementation.
- Cover a timed-out `uv --version` in the real helper orchestration. The
existing 30-second timeout falls back to the legacy certificate
variables so installation and relaunch can continue. Removing the
`SubprocessError` catch now fails this test.

This completes review follow-ups to the system-store change. The timeout
follow-up adds coverage without changing the existing fallback or its
duration.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Linux, Python 3.12, after rebasing onto current main:

```text
uv run --frozen --all-extras pytest tests/test_security_tls.py tests/test_updates_upgrade.py tests/test_updates_install_guard.py tests/test_cli_subagent_setup.py tests/test_cli_system_ca.py tests/test_cli_onboard_commands.py tests/test_cli_provider_commands.py tests/test_config_update_providers.py -q -x --tb=short
uv run --frozen --all-extras pytest tests/integration/test_cli_upgrade_real_uv.py -o addopts= -n 0 -q --tb=short
```

Results: 894 passed and 3 passed, respectively. The real upgrades cover
library roots and handed-over system roots. Two existing idle-ceiling
warnings were reported by the integration tests.

Native Windows, Python 3.13:

```text
uv run --frozen --extra dev pytest tests/test_updates_upgrade.py -k older_than_system_certs -n 0 -q -x --tb=short
```

Result: 6 passed, including `subprocess.TimeoutExpired`. An isolated
mutation removing `subprocess.SubprocessError` from the version probe's
catch failed exactly the timeout case; the production source was then
restored and checked against Git.

Additional checks passed:

```text
uv run --frozen --extra dev pre-commit run --from-ref origin/main --to-ref HEAD
uv run --frozen --all-extras ty check raven/security/tls.py raven/updates/upgrade.py
uv run --frozen --all-extras lint-imports
uv run --frozen python scripts/check_commit_messages.py origin/main..HEAD
commitlint --from origin/main --to HEAD
uv run --frozen python scripts/check_source_language.py origin/main..HEAD
uv run --frozen python scripts/check_large_files.py origin/main..HEAD
git diff --check origin/main...HEAD
```

Import-linter kept all 10 contracts. The full repository suite and
native macOS were not rerun for this follow-up. No user-facing
documentation change is needed for the additional timeout test. Existing
integration fixtures can use the user's upgrade cache, and some helper
tests reach GitHub before exercising a fallback; these predate this
follow-up.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

An older or unrecognized uv still receives both certificate-variable
names, preserving compatibility behind a TLS-inspecting proxy. An
explicit legacy setting may still produce uv's own warning. The version
query is bounded, and its failure does not prevent an upgrade attempt.
No certificate verification is disabled.

Rollback: revert the squash commit, restoring the previous warning and
unpinned handoff behavior.

## Related Issues

#878, #637

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-12 16:08:26 +08:00
Dizhan Xue
9c893bb65e
fix(*): trust the system certificate store and stop unverified Codex retries (#878)
## Summary

Behind a TLS-inspecting corporate proxy, or against a company-hosted
gateway signed by a private CA, `raven onboard` failed with
`CERTIFICATE_VERIFY_FAILED` even with the corporate root installed in
the operating system (#637). httpx, LiteLLM and the OpenAI SDK verify
against certifi's bundled roots unless told otherwise, so the
operating-system store was never consulted on the paths that reach a
model. aiohttp, urllib and websockets read OpenSSL's defaults instead,
so one process trusted two different sets of roots depending on which
client made the call.

What changes:

- `raven/security/tls.py` (new): `use_system_ca()` injects truststore,
so every TLS context built afterwards is verified by the operating
system (CryptoAPI on Windows, the Security framework on macOS, OpenSSL's
default paths on Linux). A bundle a caller loads into a context is still
honoured: on Linux it joins the store, on macOS its roots become extra
anchors, on Windows it is tried when the system roots fail. So every
root trusted before stays trusted. `RAVEN_NO_SYSTEM_CA=1` opts out,
parsed like `RAVEN_NO_UPDATE_CHECK`.
- `raven/cli/entry.py` (new) is now the entry of both the console script
and `python -m raven`, the form the web supervisor starts its gateway
with. It switches trust before importing the CLI, because the CLI's
imports reach aiohttp (through the sub-agent backends), and aiohttp
builds its default context when it is imported. `raven/cli/commands.py`
no longer starts the CLI when run as a script, which skipped the switch;
pointing it at the entry would not have helped, since by then its own
imports have built aiohttp's context.
- The provider probe names a certificate that fails verification
`certificate_untrusted` instead of `network_error`. It checks for that
ahead of the proxy branch: through a proxy that answered the tunnel, a
refused certificate used to be reported as `proxy_unreachable`. The
Codex check (its backend's list of the account's models), MiniMax's
token renewal (which reported `oauth_token_missing`) and the sub-agent
key prompt's probe name it too. LiteLLM's Copilot token exchange and its
ChatGPT renewal drop the handshake error before Raven sees it, so those
still read as the credential.
- The three surfaces that read the probe say so. `raven onboard` and
`raven provider test` no longer send the reader to check the network,
proxy or VPN; their hints name the three causes: a CA to add to the
system certificate store; macOS and Windows refusing a certificate by
their own rules even when `SSL_CERT_FILE` trusts its CA, which
`RAVEN_NO_SYSTEM_CA=1` turns off; and an expired certificate, or one for
another host, which is the server's to fix. `raven onboard` offers the
menu a network error gets (re-entering the key cannot fix a
certificate); the web Settings page has its own line, in English and
Chinese, in `i18n/messages.json` and the TUI's catalogue generated from
it.
- The RPC contract's description of the model catalogue's `status`
(`rpc-schema/openrpc.json` and both generated clients) now lists the
probe's real words. It named `unauthorized`, which the probe never
returns (a 401 or 403 is `invalid_key`), and missed `proxy_unreachable`
as well as the new status.
- The Codex provider no longer retries a request whose certificate
failed verification with verification turned off (`verify=False`). The
second attempt carried the ChatGPT OAuth bearer token and the whole
conversation to whoever presented the certificate (a captive portal, a
hostile network, a proxy that re-signs TLS), and on a network like the
one in #637 it is what would keep Codex working unverified while every
other provider fails. The retry dates from the initial import;
`_request_codex` loses the `verify` parameter that existed only to carry
it.
- `raven upgrade` and the page's Update verify against the same store as
the release lookup. The upgrade helper runs on the base interpreter,
outside the tool environment that holds truststore, and fetched the
release lists with urllib's default context; uv fetched the rest with
its bundled roots. When the spawning process itself verifies against the
system store, it now copies truststore into a fresh temporary directory
and hands the helper its path, as it hands over the marker path; one
that opted out, or never went through `cli.entry`, hands over nothing
and needs no truststore to say so. The helper loads the copy before its
first download and deletes it, and gives `UV_SYSTEM_CERTS` (and
`UV_NATIVE_TLS`, which older uv reads) to uv's own calls only, so the
Raven it relaunches inherits neither. A copy, because
`inject_into_ssl()` imports requests and urllib3 wherever it finds them,
and the tool environment's belong to what uv is about to rewrite.
- Quick Start, in both languages, gains a section on corporate proxies
and private CAs: WSL2 does not see the Windows store, and the EverOS
memory server and Node-based agents run as separate programs that need
`SSL_CERT_FILE` and `NODE_EXTRA_CA_CERTS` in the shell profile.
`SSL_CERT_FILE` must name a complete bundle that exists, since httpx
refuses to start with a missing one:
`/etc/ssl/certs/ca-certificates.crt` on Debian, Ubuntu and WSL, and one
built from `/etc/ssl/cert.pem` on macOS. A note says macOS and Windows
apply their own certificate policy and names `RAVEN_NO_SYSTEM_CA=1` as
the way back.

The LiteLLM warning in the issue ("Failed to fetch remote model cost map
... CERTIFICATE_VERIFY_FAILED", with `SSL_CERT_FILE` exported) is not
LiteLLM building its own store. That fetch is a plain `httpx.get`, which
reads `SSL_CERT_FILE`, and the variable replaces the whole trusted set:
with a corporate-only PEM the gateway verifies while
raw.githubusercontent.com, signed by a public root, no longer does.
Measured with a private CA: corporate-only PEM, warning; certifi plus
the corporate root, remote fetch succeeds.

Not covered, by construction: the EverOS server is a separate `everos`
executable and Node-based agents are not Python, so an injection in this
process cannot reach them. The new docs section gives them their
environment variables.

`uv.lock`: adding truststore re-resolved the lock (uv 0.11.25, revision
3 kept). Besides the truststore entries, five packages' dependency edges
lose a marker: matrix-nio, h2, aiohttp-socks, aiologic and culsans. Each
dropped marker is implied by the only path that reaches its package
(matrix-nio, and through it h2 and aiohttp-socks, only where
`sys_platform != 'win32'`; culsans and aiologic only below Python 3.13),
so the resolved set is unchanged on every platform. uv 0.12.23 produces
the same edges and also rewrites the lock to revision 5, so it was not
used; both versions accept the committed lock under `uv lock --locked`.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Red first, on the base: the end-to-end `raven provider test custom`
against a gateway whose root only the stand-in system store holds exits
1 with `network_error ... unable to get local issuer certificate` under
both launch forms, and its opted-out twin prints "Check network /
firewall / VPN settings"; the probe test returns `network_error`, and
through a CONNECT proxy `proxy_unreachable`; the onboarding menu offers
`rekey` for a certificate failure; the web note falls to
`probe_unknown`; and the Codex test's endpoint receives `Bearer
tok-secret`, with the captured log line `SSL certificate verification
failed for Codex API; retrying with verify=False`.

Mutations, each applied and then restored byte for byte:

- the entry no longer switching trust: both end-to-end trust cases fail;
- the switch moved after the CLI import: only
`test_trust_is_settled_before_the_cli_builds_a_context` fails;
- the classifier accepting any `ssl.SSLError`: the plain-HTTP case
fails;
- the certificate check placed after the proxy branch: the proxy case
fails.

Review round two, red first at the round-one head: against an endpoint
signed by a root no store holds, the Codex check returned
`network_error`, MiniMax's token renewal `oauth_token_missing` and the
sub-agent key probe `network_error`; both hint tests failed on the
missing opt-out; the guard scan found `raven/cli/commands.py`; and the
upgrade tests found nothing handed over. Round-two mutations, each
applied and then restored byte for byte:

- the classifier matching `CERTIFICATE_VERIFY_FAILED` in the message:
the new macOS and Windows cases fail (282 of 282 stayed green before
them);
- 17 single-construct mutants of the upgrade hand-over (not switching,
not putting the copy on the path, keeping it, hiding a failed load,
leaving the path in the environment, not telling uv, overwriting a uv
setting the user made, dropping the older uv name, telling uv without a
copy, uv not given its environment, passing on an inherited path,
importing truststore instead of reading whether this process switched,
handing over without that check, handing over always, handing over no
copy, and keeping a copy no helper started, at each spawn site): each
fails at least one test;
- the copy failure's warning and its cleanup: each fails the
copy-failure test.

Commands and results, at the head after rebasing onto `main`. The
commits `main` gained since round one touch none of this branch's lines:
`git range-diff` shows every commit unchanged, and the aggregate
patch-id is the same before and after.

- The touched Python test files (`tests/test_security_tls.py`,
`tests/test_cli_system_ca.py`, `tests/test_config_update_providers.py`,
`tests/test_cli_onboard_commands.py`,
`tests/test_cli_provider_commands.py`,
`tests/test_cli_subagent_setup.py`, `tests/test_updates_upgrade.py`,
`tests/test_openai_codex_provider.py`), the Codex provider's neighbours
(`tests/test_providers_first_byte_budget.py`,
`tests/test_provider_protocol.py`) and the i18n tests, in full: 1010
passed.
- Full suite with branch coverage, `pytest -q -p no:randomly --cov=raven
--cov=raven_everos --cov-branch` on the all-extras environment with the
box proxy variables unset: 2 failed, 27750 passed, 119 skipped. Both
failures fail the same way on a clean tree of the new merge base under
the same interpreter, so neither is introduced; they are local to this
box (an `npm` on its PATH, and running as root):
-
`tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`
-
`tests/test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing`
- `scripts/coverage_gate.py diff --base-ref origin/main --threshold 90`
over that run: 100.00% (57/57 executable changed lines).
- `tests/integration/test_cli_upgrade_real_uv.py`, which CI runs only in
the Windows self-upgrade job (`--noconftest`): 3 passed locally. Its
fixture calls the handoff without `cli.entry`; the `system store`
variant switches first, as `cli.entry` does, so that job runs the whole
hand-over on Windows, and the fixture prints what it handed over so
neither variant passes without its branch running.
- `ruff check`, `ruff format --check`, `ty check` over the changed
modules, `lint-imports` (10 contracts kept), `deptry raven`: clean.
- Repository gates over `origin/main...HEAD`:
`check_source_language.py`, `check_large_files.py` and
`check_commit_messages.py` exit 0.
- Generated files on the rebased tree, where `main` had also changed the
RPC schema, the message catalogue and their generated copies:
`gen-i18n.mjs --check` and `gen-rpc-types.mjs --check` (ui-tui) and
`gen-rpc-client.mjs --check` (ui-web) exit 0.
- Round two changes no file under `ui-web`, `ui-tui` or `rpc-schema`;
these were measured at round one:
- ui-web, `node node_modules/vitest/vitest.mjs run
src/features/settings/SetupBodies.test.tsx
scripts/gates/i18n-keys.test.mjs`: 33 passed; `scripts/gates`: 46 files,
205 passed; `tsc --noEmit` clean, its `--listFiles` resolving the edited
files inside this tree; ESLint clean on the three files.
- ui-tui, the steps of TUI checks: ESLint 0 errors; `lint:rpc` and
`lint:i18n` pass; `tsc --noEmit` clean; the build passes; `vitest run`
has 9 failures in 4 files (renderer and typing-burst tests), the same 9
that fail on a clean worktree of the merge base on this box, so none is
introduced.
- RPC contract: `node scripts/gen-rpc-client.mjs --check` (ui-web) and
`node scripts/gen-rpc-types.mjs --check` (ui-tui) exit 0, each generated
file differing from the base by the one description line; the 9 Python
test files that read the schema, 798 passed.
- Docs: `mkdocs build --strict` clean, with the new paragraphs on the
English and Chinese pages; `tests/test_docs_site.py` and
`tests/test_living_docs.py` pass in the full suite.

The end-to-end cases run on Linux only. They stand the system store in
with OpenSSL's directory store through `SSL_CERT_DIR`, and name a bundle
of public roots in `SSL_CERT_FILE` so that httpx cannot pass by reading
`SSL_CERT_DIR` itself. Verification on macOS and Windows was read from
truststore 0.10.4's source, not run. The upgrade helper's tests share
the limit: on Linux truststore and the standard library read the same
store, so they show the hand-over (the copy loads on the base
interpreter under `-I` before the first download, and uv alone is told),
not the Keychain difference it exists for.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- Every Raven process started as `raven` or `python -m raven` now trusts
the operating system's roots in addition to what each library bundles.
For those processes `SSL_CERT_FILE` adds to the trusted set instead of
replacing it; someone using it to narrow trust to one private root keeps
that root but also trusts the system store.
- On macOS and Windows the system verifier applies its own certificate
policy as well. A private certificate that OpenSSL accepted from
`SSL_CERT_FILE` can be refused there if it breaks that policy;
`RAVEN_NO_SYSTEM_CA=1` restores OpenSSL's verification.
- On macOS and Windows truststore also verifies the client's chain on
server-side contexts, refusing a client that sends none. Raven builds no
server-side TLS context: it serves plain HTTP on localhost.
- Upgrades verify their downloads the way the release lookup does: on
macOS that includes Apple's certificate policy, which the helper's
OpenSSL did not apply. uv is told to use the system store only for the
upgrade's own calls, and a `UV_SYSTEM_CERTS` or `UV_NATIVE_TLS` the user
set is kept.
- Behind a proxy that re-signs TLS, Codex requests now fail with the
certificate error instead of continuing unverified. Trusting the proxy's
root, in the system store or in a complete bundle named by
`SSL_CERT_FILE`, is the remedy.
- Startup: about 100 ms warm and 270 ms cold, measured on Linux, mostly
truststore importing urllib3 and requests to patch their references.
- Rollback: `RAVEN_NO_SYSTEM_CA=1` restores the previous trust per
process, and an upgrade started under it hands its helper and uv
nothing; the Codex change has no switch, by design. Reverting the squash
commit removes both.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #637

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-10 10:40:12 +08:00
knqiufan
d90b6deb7e
fix: allow agents to inherit OAuth-backed host models (#841)
## Summary

Allow all five shipped agents to inherit an OAuth-backed host model,
including OpenAI Codex, by validating the inherited model binding with
the runtime credential check after parent session overrides.

- Reuse the launcher's inheritance decision in setup and update its
English/Chinese message.
- Preserve raw provider settings, routing, reasoning effort and product
limits. OAuth tokens remain in the host credential store.
- Cover OAuth aliases, access/refresh tokens, missing or malformed
credentials, custom token paths, parent bindings and independent agent
keys. Remove the old API-key-only gate exemptions from the auth contract
test.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Selected regression tests: **321 passed, 48 deselected** on Windows with
Python 3.12. The selection focuses on inheritance and excludes untouched
POSIX permission/pruning scenarios.

```powershell
uv run --no-sync pytest tests/test_config_product_render.py tests/test_subagent_vendored_agents.py tests/test_cli_subagent_setup.py tests/test_provider_auth_method.py tests/test_providers_factory.py tests/test_openai_codex_provider.py tests/test_providers_oauth_login.py tests/test_i18n_prompts.py tests/test_i18n_boundary.py -q -o addopts='' -p no:cacheprovider -k '(not test_config_product_render or inherit_llm or launchers_inherit_oauth or launchers_own_keys or secret_slots or env_wins or dig_and_put) and not prune and not backup and not locks_it_down and not test_a_page_sign_in_runs_the_real_driver_and_leaves_the_token_owner_only' --cov=raven.config.product_render --cov=raven.agent.subagent.vendored_agents --cov=raven.cli.subagent_setup --cov=raven.i18n --cov-branch --cov-report=json:C:/Users/Administrator/AppData/Local/Temp/raven_issue809_coverage_final.json --cov-report=term-missing:skip-covered
```

The following checks passed: Ruff across the repository, formatting
(2149 files), relevant type checks, all 10 import contracts, commit
messages, source language and file/asset checks.

```powershell
uv run --no-sync ruff check raven evolver agents plugins-dist tests scripts docs-site
uv run --no-sync ruff format --check raven evolver agents plugins-dist tests scripts docs-site
uv run --no-sync ty check raven/config/product_render.py raven/agent/subagent/vendored_agents.py raven/cli/subagent_setup.py
uv run --no-sync lint-imports
uv run --no-sync python scripts/check_commit_messages.py main..HEAD
uv run --no-sync python scripts/check_source_language.py main..HEAD
uv run --no-sync python scripts/check_large_files.py main..HEAD
```

Changed-line coverage: **100% (22/22 executable lines)** after
normalizing the Windows report's path separators.

```powershell
uv run --no-sync python -c "import json,pathlib; p=pathlib.Path('C:/Users/Administrator/AppData/Local/Temp/raven_issue809_coverage_final.json'); d=json.loads(p.read_text(encoding='utf-8')); d['files']={k.replace(chr(92), '/'):v for k,v in d['files'].items()}; p.with_name('raven_issue809_coverage_posix.json').write_text(json.dumps(d), encoding='utf-8')"
uv run --no-sync python scripts/coverage_gate.py --coverage-json C:/Users/Administrator/AppData/Local/Temp/raven_issue809_coverage_posix.json diff --base-ref main --head HEAD --threshold 90
```

Full local CI remains unverified: `make ci` could not run because make
is unavailable. Existing POSIX file-permission tests fail on Windows.
`uv run --no-sync deptry raven` reports 12 missing optional channel
dependencies in this environment.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Inheritance now refuses an unavailable selected model even when an
unrelated provider has an API key. API-key and independent-agent-key
paths remain covered. Credential validation only reads stored OAuth
credentials; it does not sign in or serialize tokens. Revert this change
to restore the previous inheritance gate.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #809

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: KT <74288668+0xKT@users.noreply.github.com>
2026-10-03 23:43:03 +08:00
Handsome-wzw
7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00
forrestjgq
f01231fef6 fix(cli): overhaul trajectory browser interaction ux (#379)
## Summary

Overhaul the interactive `raven trajectory` browser end to end, fixing
every
issue from real-terminal testing:

- Navigation: every screen opens with a help area under its title (page
  purpose plus "[Key] function" hotkey hints, rendered in a new readable
help color). Esc walks one level up (action -> attempt -> session); on
the
top-level session screen Esc stays put and quitting is Ctrl+C only,
which
prints a single exit notice. Esc inside a running action cancels just
that
action through a dedicated exception at the action boundary, so the back
  sentinel can never reach a conversion or the data layer; a cancelled
  report keeps its pre-confirm bundle-and-pin side effects (the existing
  declined-report contract). Back/Exit menu entries are gone; answered
  prompts erase themselves and navigation is echoed as breadcrumb lines
  with markup-escaped untrusted text.
- Tables: the session list is TITLE | ATTEMPTS | LAST ACTIVITY and the
  attempt list is # | STARTED | TURNS | SPANS | VERDICT | PIN | MERGED |
PREVIEW with green check cells. Time cells keep the year. The layout is
  budgeted per screen build from the live terminal width (row indent, a
safety margin, and inter-column gaps are all part of the math); PREVIEW
  drops as a whole column when it no longer fits, and below the dynamic
  minimum width the tightest layout is kept while the renderer clips
  overlong lines at the terminal edge. Every dynamic cell passes a
  single-line sanitization gate before width math (verdict sidecars only
  guarantee a non-empty string), cells are measured and truncated by
terminal display width (CJK is 2 cells), and fixed column widths always
  fit their headers.
- Space preview: Space on an attempt row prints that attempt's per-turn
input/output previews, collected in the same snapshot scan and sorted by
a fully-stringified key so ordering never follows log source order (the
  table PREVIEW cell derives from the same sorted collection). Any key,
arrow keys included, returns with the cursor kept on the row; the waiter
  passes the terminal's CPR reply through to the renderer instead of
  treating it as a keypress, and Ctrl+C keeps the browser-wide cancel.
- Merge hotkey: with two or more attempts, m/M opens the multi-select
merge
  (advertised in the help line; the old "Merge attempts" menu entry is
gone). The checkbox screen carries its own help line for toggle,
confirm,
  and cancel. A single attempt neither binds nor advertises the key.
- Theme and feel: new success (green check) and help (readable hint
text)
colors in both palettes, all WCAG AA on their backgrounds; a lone ESC is
flushed after 50ms instead of prompt_toolkit's 0.5s default, removing
the
  laggy Esc feel.

The data layer is untouched: changes are confined to
raven/cli/trajectory_browse.py, raven/cli/_theme.py, and their tests.
The
id-taking subcommands (raven trajectory save/list/...) behave as before.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `COLORTERM=truecolor uv run pytest tests/test_cli_theme.py
tests/test_cli_trajectory_browse.py
tests/test_cli_trajectory_commands.py -q` -> 220 passed
- `COLORTERM=truecolor uv run pytest tests/ --ignore=tests/integration
-q` -> 6917 passed; the 10 failed + 20 errors are the pre-existing
environment-dependent set, identical file-by-file on a clean main
checkout (cron, everos server, config loader, tracing viewer)
- `uv run ruff check` and `uv run ruff format --check` on the four
touched files -> clean
- PTY smoke on a CPR-answering pseudo terminal at 100 and 58 columns:
full flow (preview, merge cancel, action screen, staged Esc backout,
Ctrl+C quit) and the narrow-width degradation contract
- New coverage includes real pipe-input key-binding tests (Esc on all
four prompt kinds, Space/m/M hits, waiter key paths with CPR
pass-through), table budget/clipping boundaries at the computed minimum
width, and corrupt verdict/timestamp corpora

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The interactive browser's look and key protocol change wholesale (that
is
  the point of this PR): Esc no longer quits from the top level (Ctrl+C
  does), Back/Exit entries are gone, and lists render as tables. The
machine-facing subcommands and the trajectory data layer are untouched.
- Rollback: revert the squash commit; the browser returns to the
previous
  interaction wholesale, no data migration involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-05 23:58:20 +08:00
0xKT
28299e5f0e fix(*): relays on the page lane, live direct chats reported, continuation heads cut
## Summary

Four sub-agent orchestration defects found on 2026-09-08 while running the research
sub-agent from the TUI and the page; all share one root, the host loop not seeing what
its sub-agents are doing right now.

1. A sub-agent's result relay into a page session ran on the gateway spine while the
   page's own turns to that session ran on the RPC spine, so a relay and a user turn ran
   at once: the same tool calls were drawn twice and the result delivered twice, and
   `turn.send`'s one-turn-per-session guard saw neither. The RPC stack now exposes its
   turn scheduler (`RpcStack.turn_scheduler`, surfaced on `PageMount.submit`), and
   `raven.gateway.submit_router.route_submit` hands turns whose conversation carries
   the page prefix (`tui:`) to the page lane and everything else to the gateway lane.
   The sub-agent manager and the deep-research submit both take the routed callable;
   a gateway without a page keeps the gateway submit unchanged.

2. The watch-work judgement (one model call on a turn's first look) was paid on relay
   and sentinel turns, whose "last user message" is the runtime's re-injected report,
   and ran unbounded: one relay turn spent 1229 seconds in it on a reasoning runaway.
   The turn's origin now reaches the loop; a re-injected origin (SUBAGENT, SENTINEL)
   reads as no request, and the judgement is cut at 60 seconds and treated as no
   verdict. CRON and heartbeat turns are deliberately still judged: their text is the
   owner's own instruction.

3. The direct-chat handoff recorded a turn only when it landed, so an instance the user
   was mid-conversation with rendered to the main agent as "no turns yet"; the model
   read that as free and spawned onto it. The handoff now reads the manager's live
   direct turns (`live_direct_turns`) and renders an answering instance with the time
   it began answering (handle lock held) and a line saying it is the user's conversation. The spawn that lands
   behind such a turn no longer writes `running` before it holds the handle lock (it
   stays the `pending` it was announced as), and no longer takes over the instance's
   live view: `activity.watching_instance` registers only when the slot is free and
   pops only what it registered, so the direct chat's steps stay on screen and the
   spawn's appear once it runs.

4. When reasoning ran to the model's output ceiling with no content and the
   empty-response recovery fed it back as a prefill, the continuation opened on the
   rest of the cut thought and that fragment streamed and was stored as the head of the
   answer. `cut_reasoning_head` drops an opening fragment that starts in lowercase or
   whitespace and ends at the first sentence or line boundary within 200 characters;
   `ContinuationGate` holds streamed deltas until the same rule can decide, so what the
   reader sees matches what the session stores. Only a continuation after
   ceiling-cut reasoning (`finish_reason == "length"`) is treated this way.

Decisions worth knowing:

- The `tui:` prefix is spelled in `raven.gateway.submit_router` and as
  `RoutingQuestionBroker`'s default; the layer contract keeps `raven.gateway` from
  importing `raven.rpc`, so a test holds the two equal instead of one importing the
  other.
- The gateway wiring has no import seam, so its test pins the source of the mounted
  block, the same way the neighbouring cron-partition test does.
- The `subagent.status` payload shape is unchanged; only the `running` frame's timing
  moved (after the handle lock). The page's spawn card reads identity from `pending`
  and the record id from `running`, both still present.
- A CJK boundary class in the head-cut regex is spelled as escapes because the
  source-language gate reads the literal marks as non-English source.

Review round 1 (all findings non-blocking, all taken):

- The lane key is now one definition, `raven.spine.conversation_id`; the scheduler's
  method, the rpc spine and the submit router call it, and the router test pins the
  identity and compares against the scheduler's method instead of literals.
- The loop's wiring of the handoff to the manager's live turns has its own test (a bare
  `DirectChatHandoff()` went green before).
- `activity.live_instances_of` is gone: nothing called it.
- The watch-work judgement now reads any exception as no verdict, not only a timeout.
  The enclosing handler already kept the turn alive, but left the verdict unset, so a
  failing judgement was paid again on every path-touching tool call; the comment that
  claimed a raise was handled is corrected.
- A direct chat's start time is stamped once the handle lock is held, not when the slot
  is claimed, so a chat queued behind another holder is not reported as answering.
- The continuation cut deliberately leaves a continuation that opens directly on CJK
  whole (no case, so no mid-sentence signal); the comment and a test now say that the
  CJK stops end an admitted fragment rather than open one.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests -q -p no:cacheprovider -n 8 --ignore=tests/integration
  -> 7 failed, 19705 passed, 153 skipped. The same 7 fail on a detached origin/main
     worktree through the same interpreter (doctor memories path 1, NAT64 resolution 4,
     design-engine rendering 2): 0 introduced.
uv run pytest tests/test_agent_loop_empty_recovery.py tests/test_watch_work.py \
  tests/test_gateway_submit_router.py tests/test_subagent_activity.py \
  tests/test_subagent_handoff.py tests/test_subagent_direct_chat.py \
  tests/test_rpc_bootstrap.py tests/test_cli_gateway_page.py tests/test_cli_gateway_commands.py
  -> 279 passed (before round 1); round 1 adds 5 tests and removes 1, see the pipeline
uv run ruff check raven tests; uv run ruff format --check raven tests   -> clean
uv run lint-imports                                                       -> 9 kept, 0 broken
make check-source-language                                                -> exit 0
scripts/check_commit_messages.py origin/main..HEAD                        -> exit 0
```

Red-checked (fix reverted, test goes red, fix restored): activity.py alone reverted
-> `a second lane never unseats the first`; the spawn-path hunk of manager.py alone reverted
-> `['pending', 'running'] == ['pending']`; turn_path.py alone reverted -> the continuation
test sees the `Lagg in an token` head. Round 1: the handoff wiring test goes red on a bare
`DirectChatHandoff()`; the queued-direct-chat test goes red with the stamp back on the slot;
the raising-judgement test goes red with the narrow catch (two calls instead of one). The other new tests fail without their change by
construction (missing symbol or signature), which proves less and is noted as such.

Live check of the shipped page: the served `ui-web/dist` bundle booted under happy-dom in
stub and live mode against a gateway running this branch, desk mounted, no console
errors; `subagents.list` and `subagents.instances` answer as before.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed (CHANGELOG, CONTEXT.md
      Handoff Block)

## Risk

- A relay into a page session now waits behind the page's running turn on that session
  instead of running beside it. That is the intended serialisation; the cost is latency
  on a relay while a long page turn runs.
- A judgement that takes over 60 seconds is dropped and the turn runs unwatched for
  that turn only; before, the same turn hung. No configuration knob: the bound is a
  class attribute.
- The head cut only runs on the continuation after a ceiling-cut prefill. A
  continuation whose intended answer begins with a lowercase word and a full stop within
  200 characters would lose that first clause; the test file names the shapes kept.
- Rollback: revert the squash commit. No persisted format changes.

- [x] Security impact considered (no new surface; routing stays in-process)
- [x] Backward compatibility considered (payload shapes unchanged; a gateway without a
      page mount is unaffected)
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-09 06:25:28 +00:00
userName20260323
50ad9377bd fix(*): relays on the page lane, live direct chats reported, continuation heads cut
## Summary

Four sub-agent orchestration defects found on 2026-09-08 while running the research
sub-agent from the TUI and the page; all share one root, the host loop not seeing what
its sub-agents are doing right now.

1. A sub-agent's result relay into a page session ran on the gateway spine while the
   page's own turns to that session ran on the RPC spine, so a relay and a user turn ran
   at once: the same tool calls were drawn twice and the result delivered twice, and
   `turn.send`'s one-turn-per-session guard saw neither. The RPC stack now exposes its
   turn scheduler (`RpcStack.turn_scheduler`, surfaced on `PageMount.submit`), and
   `raven.gateway.submit_router.route_submit` hands turns whose conversation carries
   the page prefix (`tui:`) to the page lane and everything else to the gateway lane.
   The sub-agent manager and the deep-research submit both take the routed callable;
   a gateway without a page keeps the gateway submit unchanged.

2. The watch-work judgement (one model call on a turn's first look) was paid on relay
   and sentinel turns, whose "last user message" is the runtime's re-injected report,
   and ran unbounded: one relay turn spent 1229 seconds in it on a reasoning runaway.
   The turn's origin now reaches the loop; a re-injected origin (SUBAGENT, SENTINEL)
   reads as no request, and the judgement is cut at 60 seconds and treated as no
   verdict. CRON and heartbeat turns are deliberately still judged: their text is the
   owner's own instruction.

3. The direct-chat handoff recorded a turn only when it landed, so an instance the user
   was mid-conversation with rendered to the main agent as "no turns yet"; the model
   read that as free and spawned onto it. The handoff now reads the manager's live
   direct turns (`live_direct_turns`) and renders an answering instance with the time
   it began answering (handle lock held) and a line saying it is the user's conversation. The spawn that lands
   behind such a turn no longer writes `running` before it holds the handle lock (it
   stays the `pending` it was announced as), and no longer takes over the instance's
   live view: `activity.watching_instance` registers only when the slot is free and
   pops only what it registered, so the direct chat's steps stay on screen and the
   spawn's appear once it runs.

4. When reasoning ran to the model's output ceiling with no content and the
   empty-response recovery fed it back as a prefill, the continuation opened on the
   rest of the cut thought and that fragment streamed and was stored as the head of the
   answer. `cut_reasoning_head` drops an opening fragment that starts in lowercase or
   whitespace and ends at the first sentence or line boundary within 200 characters;
   `ContinuationGate` holds streamed deltas until the same rule can decide, so what the
   reader sees matches what the session stores. Only a continuation after
   ceiling-cut reasoning (`finish_reason == "length"`) is treated this way.

Decisions worth knowing:

- The `tui:` prefix is spelled in `raven.gateway.submit_router` and as
  `RoutingQuestionBroker`'s default; the layer contract keeps `raven.gateway` from
  importing `raven.rpc`, so a test holds the two equal instead of one importing the
  other.
- The gateway wiring has no import seam, so its test pins the source of the mounted
  block, the same way the neighbouring cron-partition test does.
- The `subagent.status` payload shape is unchanged; only the `running` frame's timing
  moved (after the handle lock). The page's spawn card reads identity from `pending`
  and the record id from `running`, both still present.
- A CJK boundary class in the head-cut regex is spelled as escapes because the
  source-language gate reads the literal marks as non-English source.

Review round 1 (all findings non-blocking, all taken):

- The lane key is now one definition, `raven.spine.conversation_id`; the scheduler's
  method, the rpc spine and the submit router call it, and the router test pins the
  identity and compares against the scheduler's method instead of literals.
- The loop's wiring of the handoff to the manager's live turns has its own test (a bare
  `DirectChatHandoff()` went green before).
- `activity.live_instances_of` is gone: nothing called it.
- The watch-work judgement now reads any exception as no verdict, not only a timeout.
  The enclosing handler already kept the turn alive, but left the verdict unset, so a
  failing judgement was paid again on every path-touching tool call; the comment that
  claimed a raise was handled is corrected.
- A direct chat's start time is stamped once the handle lock is held, not when the slot
  is claimed, so a chat queued behind another holder is not reported as answering.
- The continuation cut deliberately leaves a continuation that opens directly on CJK
  whole (no case, so no mid-sentence signal); the comment and a test now say that the
  CJK stops end an admitted fragment rather than open one.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
uv run pytest tests -q -p no:cacheprovider -n 8 --ignore=tests/integration
  -> 7 failed, 19705 passed, 153 skipped. The same 7 fail on a detached origin/main
     worktree through the same interpreter (doctor memories path 1, NAT64 resolution 4,
     design-engine rendering 2): 0 introduced.
uv run pytest tests/test_agent_loop_empty_recovery.py tests/test_watch_work.py \
  tests/test_gateway_submit_router.py tests/test_subagent_activity.py \
  tests/test_subagent_handoff.py tests/test_subagent_direct_chat.py \
  tests/test_rpc_bootstrap.py tests/test_cli_gateway_page.py tests/test_cli_gateway_commands.py
  -> 279 passed (before round 1); round 1 adds 5 tests and removes 1, see the pipeline
uv run ruff check raven tests; uv run ruff format --check raven tests   -> clean
uv run lint-imports                                                       -> 9 kept, 0 broken
make check-source-language                                                -> exit 0
scripts/check_commit_messages.py origin/main..HEAD                        -> exit 0
```

Red-checked (fix reverted, test goes red, fix restored): activity.py alone reverted
-> `a second lane never unseats the first`; the spawn-path hunk of manager.py alone reverted
-> `['pending', 'running'] == ['pending']`; turn_path.py alone reverted -> the continuation
test sees the `Lagg in an token` head. Round 1: the handoff wiring test goes red on a bare
`DirectChatHandoff()`; the queued-direct-chat test goes red with the stamp back on the slot;
the raising-judgement test goes red with the narrow catch (two calls instead of one). The other new tests fail without their change by
construction (missing symbol or signature), which proves less and is noted as such.

Live check of the shipped page: the served `ui-web/dist` bundle booted under happy-dom in
stub and live mode against a gateway running this branch, desk mounted, no console
errors; `subagents.list` and `subagents.instances` answer as before.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed (CHANGELOG, CONTEXT.md
      Handoff Block)

## Risk

- A relay into a page session now waits behind the page's running turn on that session
  instead of running beside it. That is the intended serialisation; the cost is latency
  on a relay while a long page turn runs.
- A judgement that takes over 60 seconds is dropped and the turn runs unwatched for
  that turn only; before, the same turn hung. No configuration knob: the bound is a
  class attribute.
- The head cut only runs on the continuation after a ceiling-cut prefill. A
  continuation whose intended answer begins with a lowercase word and a full stop within
  200 characters would lose that first clause; the test file names the shapes kept.
- Rollback: revert the squash commit. No persisted format changes.

- [x] Security impact considered (no new surface; routing stays in-process)
- [x] Backward compatibility considered (payload shapes unchanged; a gateway without a
      page mount is unaffected)
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-09 06:25:28 +00:00
0xKT
f6f51952b0 refactor(*): the vendored subagents tree retires and agents/ is the one home
## Summary

Retires the vendored fork tree (`subagents/`, ~7,500 files) and makes `agents/` the one home for the five agent products. The products were already thin launchers that render a config and exec the installed raven's own `raven acp`; this PR removes the frozen fork checkouts they replaced, migrates every consumer, and keeps the A/B verification power as byte snapshots.

Key decisions:

- **A side becomes a record.** The fork trees are deleted; the files the parity guards compare against live on as byte snapshots under `tests/fixtures/vendored_fork/` (added to ruff's exclude so a record can never be reformatted). The research flow's oracle is the fork's frozen measurement (`research_parity_probe.json`), a constant by construction.
- **Config floors v6 and v7.** Stored roster rows that pointed into the retired tree are migrated in place (floor six): rows whose folder maps to a shipped product are re-aimed at `agents/`, fork-era rows that cannot be re-aimed are left untouched and reported, unparseable rows get a fix-by-hand notice. Verdict-driven, path-root anchored, with a stamped sidecar so each pass runs once.
- **The research product reclaims its name (floor seven).** The `-NG` suffix existed only to coexist with the fork's own row, so it retires with the fork: registered `Raven-Research-NG` rows rename to `Raven-Research` on load; if a fork-era row still holds the name, the advisory repeats and the stamp holds at floor six, so the rename completes on a later load once the name frees (never two rows under one name). Display name only -- the machine id `raven-research-ng` (state root, ACP home, everos identity, the `RESEARCH_NG_*` variables) is a stable key and stays. The v6 name gate maps the research folder to None (never re-aim): the fork's research rows were cli-kind on a different flow, and after the rename a name match would select exactly the rows the gate exists to refuse.
- **Guards retire with their subject.** The vendored-invariant and vendored-subagents guard suites, their Makefile/pre-commit/CI hooks, and the fork-era install stage in `install.sh` are removed. The external-consumer tripwire now derives the promised surface from the five product installers and the charter's uninstall recipe (`add_third_party_subagent` and `remove_third_party_subagent` today) instead of restating a list that goes stale; the fork-era pin count (eight names) retired with its consumers, and `get_third_party_subagents` stays only as a courtesy to fork-era installer copies already carried into raven homes.
- **Wheel and sdist slim down.** The wheel force-includes `agents/` file-by-file from the git index (filled-in `.env` files are refused). The sdist shrinks 93.2 MB -> 2.76 MB (-97%).
- **Docs speak one era.** AGENTS.md, CONTEXT.md, README, CHANGELOG, the demo READMEs, and the RPC field descriptions (models.py -> openrpc.json -> both generated clients) describe the product tree in the present and the fork in the past.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

- Controlled A/B on a clean bench, same machine and env fingerprint, at the current pair: base 19,938 passed / 137 skipped / 0 failed vs branch 19,796 / 137 / 0, skip roster identical line by line. The -142 delta equals the collection delta and reconciles file by file: -159 across the twelve retired or trimmed test files, +9 floor-six migration tests, +8 floor-seven tests. The branch diff is fingerprint-stable across rebases (`git patch-id --stable` verified pairwise), so the reconciliation transfers to later bases. A second fingerprint (dev worktree env) shows 19,555 / 148 / 2 at the final tip; the 2 are `test_design_engine_rendering_outcome` renders hitting a local playwright-home quirk -- on the clean bench both sides skip them identically ("no system chromium"), and the branch does not touch that file.
- `uv run pytest tests/integration/test_research_subagent_smoke.py -m integration`: 2 passed (the smoke now drives the migrated product launcher end to end).
- `make lint-python` (ruff check + format), `make lint-imports` (tail line: `Contracts: 9 kept, 0 broken.` -- the ninth contract landed on main mid-branch), `make check-large-files`: all green.
- `npm run lint:rpc --prefix ui-tui` and `npm run gen:check --prefix ui-web`: generated clients in sync with `rpc-schema/openrpc.json`.
- `bash -n install.sh`: clean.
- Wheel built from the branch, installed into a clean venv: onboarding discovers the five products, and a filled-in `.env` placed in the tree fails the build (secrets refusal probed).
- Real-config upgrade rehearsal on a genuine floor-5 home config (owner present): the migration stamped the sidecar to 6, rewrote nothing (the five product rows were already registered against `agents/`; the one fork-era orphan row got its advisory notice and was left untouched), a second load was silent, and the upgraded config served a live probe plus one full one-shot agent turn. The floor-seven rehearsal on the same config renamed its one transition row (a one-line diff against the byte backup), stamped seven, was silent on the second load, and left the machine ids and the row's command untouched.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- User-visible: on first load of a pre-v6 config, roster rows pointing into the retired tree are re-aimed at the product tree, with a one-line `Migrated` notice per changed row and a cleanup hint for the stale on-disk copy. Rows the migration cannot decide are reported, never rewritten. The floor-seven rename tells once per changed row too; only a blocked rename repeats its advisory, deliberately, until the blocking row is gone. While the block holds, the roster shows the discovered `Raven-Research` row beside the stored `Raven-Research-NG` row -- same launcher, same state root, no state split -- and converges to one row once the name frees.
- Rollback: revert the squash commit. The fork trees return from git history, and a config the migration already re-aimed keeps working on both sides of the revert: the wheel carries `agents/` before and after (that packaging landed separately), so no config rollback is needed. Rows the migration could not decide were never rewritten, and the version sidecar is additive.
- Packaging: a wheel built from an sdist (what a combined `uv build` does) carries no agent products by design -- the sdist has no `.git` for the hook's tracked-files read; the hook warns loudly, and release.yml builds the wheel directly from the checkout and asserts the products are present. Release artifacts are unaffected; do not publish a combined-build wheel.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

BREAKING CHANGE: the `subagents/` source tree is gone. Installs never consumed it directly (the wheel carries `raven/agents`), but any local tooling that read `subagents/` paths must move to `agents/` or the snapshots under `tests/fixtures/vendored_fork/`.
2026-09-09 03:59:19 +00:00
0xKT
231b270dd7 refactor(*): the vendored subagents tree retires and agents/ is the one home
## Summary

Retires the vendored fork tree (`subagents/`, ~7,500 files) and makes `agents/` the one home for the five agent products. The products were already thin launchers that render a config and exec the installed raven's own `raven acp`; this PR removes the frozen fork checkouts they replaced, migrates every consumer, and keeps the A/B verification power as byte snapshots.

Key decisions:

- **A side becomes a record.** The fork trees are deleted; the files the parity guards compare against live on as byte snapshots under `tests/fixtures/vendored_fork/` (added to ruff's exclude so a record can never be reformatted). The research flow's oracle is the fork's frozen measurement (`research_parity_probe.json`), a constant by construction.
- **Config floors v6 and v7.** Stored roster rows that pointed into the retired tree are migrated in place (floor six): rows whose folder maps to a shipped product are re-aimed at `agents/`, fork-era rows that cannot be re-aimed are left untouched and reported, unparseable rows get a fix-by-hand notice. Verdict-driven, path-root anchored, with a stamped sidecar so each pass runs once.
- **The research product reclaims its name (floor seven).** The `-NG` suffix existed only to coexist with the fork's own row, so it retires with the fork: registered `Raven-Research-NG` rows rename to `Raven-Research` on load; if a fork-era row still holds the name, the advisory repeats and the stamp holds at floor six, so the rename completes on a later load once the name frees (never two rows under one name). Display name only -- the machine id `raven-research-ng` (state root, ACP home, everos identity, the `RESEARCH_NG_*` variables) is a stable key and stays. The v6 name gate maps the research folder to None (never re-aim): the fork's research rows were cli-kind on a different flow, and after the rename a name match would select exactly the rows the gate exists to refuse.
- **Guards retire with their subject.** The vendored-invariant and vendored-subagents guard suites, their Makefile/pre-commit/CI hooks, and the fork-era install stage in `install.sh` are removed. The external-consumer tripwire now derives the promised surface from the five product installers and the charter's uninstall recipe (`add_third_party_subagent` and `remove_third_party_subagent` today) instead of restating a list that goes stale; the fork-era pin count (eight names) retired with its consumers, and `get_third_party_subagents` stays only as a courtesy to fork-era installer copies already carried into raven homes.
- **Wheel and sdist slim down.** The wheel force-includes `agents/` file-by-file from the git index (filled-in `.env` files are refused). The sdist shrinks 93.2 MB -> 2.76 MB (-97%).
- **Docs speak one era.** AGENTS.md, CONTEXT.md, README, CHANGELOG, the demo READMEs, and the RPC field descriptions (models.py -> openrpc.json -> both generated clients) describe the product tree in the present and the fork in the past.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

- Controlled A/B on a clean bench, same machine and env fingerprint, at the current pair: base 19,938 passed / 137 skipped / 0 failed vs branch 19,796 / 137 / 0, skip roster identical line by line. The -142 delta equals the collection delta and reconciles file by file: -159 across the twelve retired or trimmed test files, +9 floor-six migration tests, +8 floor-seven tests. The branch diff is fingerprint-stable across rebases (`git patch-id --stable` verified pairwise), so the reconciliation transfers to later bases. A second fingerprint (dev worktree env) shows 19,555 / 148 / 2 at the final tip; the 2 are `test_design_engine_rendering_outcome` renders hitting a local playwright-home quirk -- on the clean bench both sides skip them identically ("no system chromium"), and the branch does not touch that file.
- `uv run pytest tests/integration/test_research_subagent_smoke.py -m integration`: 2 passed (the smoke now drives the migrated product launcher end to end).
- `make lint-python` (ruff check + format), `make lint-imports` (tail line: `Contracts: 9 kept, 0 broken.` -- the ninth contract landed on main mid-branch), `make check-large-files`: all green.
- `npm run lint:rpc --prefix ui-tui` and `npm run gen:check --prefix ui-web`: generated clients in sync with `rpc-schema/openrpc.json`.
- `bash -n install.sh`: clean.
- Wheel built from the branch, installed into a clean venv: onboarding discovers the five products, and a filled-in `.env` placed in the tree fails the build (secrets refusal probed).
- Real-config upgrade rehearsal on a genuine floor-5 home config (owner present): the migration stamped the sidecar to 6, rewrote nothing (the five product rows were already registered against `agents/`; the one fork-era orphan row got its advisory notice and was left untouched), a second load was silent, and the upgraded config served a live probe plus one full one-shot agent turn. The floor-seven rehearsal on the same config renamed its one transition row (a one-line diff against the byte backup), stamped seven, was silent on the second load, and left the machine ids and the row's command untouched.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- User-visible: on first load of a pre-v6 config, roster rows pointing into the retired tree are re-aimed at the product tree, with a one-line `Migrated` notice per changed row and a cleanup hint for the stale on-disk copy. Rows the migration cannot decide are reported, never rewritten. The floor-seven rename tells once per changed row too; only a blocked rename repeats its advisory, deliberately, until the blocking row is gone. While the block holds, the roster shows the discovered `Raven-Research` row beside the stored `Raven-Research-NG` row -- same launcher, same state root, no state split -- and converges to one row once the name frees.
- Rollback: revert the squash commit. The fork trees return from git history, and a config the migration already re-aimed keeps working on both sides of the revert: the wheel carries `agents/` before and after (that packaging landed separately), so no config rollback is needed. Rows the migration could not decide were never rewritten, and the version sidecar is additive.
- Packaging: a wheel built from an sdist (what a combined `uv build` does) carries no agent products by design -- the sdist has no `.git` for the hook's tracked-files read; the hook warns loudly, and release.yml builds the wheel directly from the checkout and asserts the products are present. Release artifacts are unaffected; do not publish a combined-build wheel.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

BREAKING CHANGE: the `subagents/` source tree is gone. Installs never consumed it directly (the wheel carries `raven/agents`), but any local tooling that read `subagents/` paths must move to `agents/` or the snapshots under `tests/fixtures/vendored_fork/`.
2026-09-09 03:59:19 +00:00
0xKT
829bda23be feat(*): a local sub-agent must answer a prompt before it can be switched on
## Summary

Switching a local sub-agent on in settings now sends one real prompt through that
agent's own backend and refuses the enable when the agent does not answer, reporting
the agent's own words as the reason.

The free check that already existed -- `shutil.which` on the launch command plus an ACP
`initialize` + `session/new` handshake -- was treated as proof an agent could work. It is
not. ACP has no authenticated-state field at all: `authMethods` is a capability list, not a
state, and an agent advertises it identically before and after login. Credentials are only
checked at the first *model* call, which `session/new` never reaches. Measured across the 13
registry acp agents on 2026-09-07 with one real prompt each, six returned `ready` from that
exact path and could not complete a single turn -- `kilo` ("You need to sign in to use this
model"), `gemini` and `dirac` (the prompt times out), `factory_droid`, `glm` and
`deepagents` (`-32603`).

So the check is split in two, and they answer different questions:

- **Layer 1**, unchanged and still free: `which` plus the handshake. It answers "is this
  installed, and is its ACP mode on", it is what puts the agent in the listing, and it
  spends nothing. `_test_acp`, `verify_agent`, `_probe_acp` and `run_test` are byte-identical
  on this branch.
- **Layer 2**, new: `ping_agent(cfg)` sends `PROBE_PROMPT` through the row's own backend and
  is reached only from the switch. The verdict rule is the existing one, "returned
  something" -- never "the reply contains PONG", because an agent that answers with a
  preamble is working and an empty reply is itself a signal.

Keeping them apart is the point. Collapsing the ping into layer 1 would make the listing
path spend a model call on every refresh and would remove the honest middle state the
roster needs: installed, not yet proven.

**What is gated, and what is not.** `cli` and `acp` rows are gated. `openai` is a remote
endpoint whose credential the free `/models` probe already settles, so charging a
completion for the switch would buy nothing. `builtin` is this process. Switching off is
never gated -- otherwise a broken agent could not be switched off. `force: true` is the
operator's override, for a working agent whose provider is briefly down.

**The add path is gated too.** `subagents_add` shipped `enabled: true` for a `cli`/`acp`
preset with no ping, and the web Connect button calls it -- so the gate covered one of two
enable verbs and was bypassable by the primary UI path. The add now runs the same ping
itself, ahead of the write: the preset answers and the row is written enabled, or the add
is refused in the agent's own words and **nothing is stored at all**.

Storing nothing is the load-bearing half. The page offers one verb per row
(`XaPage.tsx:16-24`), and a row that exists but is off reads as stage `off`, which offers
**Connect** again -- so a refusal that left a disabled row behind would turn one click into
an undocumented add-then-enable sequence. Leaving no row keeps the stage at `add`, so
Connect stays the one verb whether the ping passes or fails. An earlier revision of this
branch did land such presets disabled; that was reviewed as a blocker and replaced.

`subagents_add` also now takes the write's refusals -- a name already taken, an entry the
schema will not have -- **before** the ping rather than after it, so it never spends an
agent's quota proving a row the write was always going to reject. For a duplicate name it
would have proved the wrong row outright: the schema keeps the first of two, so the ping
would have run against the row already on disk. `validate_agents` is split out of
`set_agents` for that, with no rule duplicated; the write still validates whatever list it
finally holds.

`CONTEXT.md`'s Product agent entry said a missing credential is deliberately not a
readiness reason, which the gate now contradicts. The entry is amended in the same change
per AGENTS.md section 6: readiness asks about the folder, the switch asks about the token,
and neither validates retroactively.

### Decisions a reviewer should know were chosen, not overlooked

- **Cost.** Enabling a `cli` or `acp` agent now spends one LLM call on that agent's own
  quota. `cli` already paid this on an explicit Test; what is new is that `acp` starts to,
  and that the switch triggers it. That is the requirement, not a side effect.
- **No retroactive validation.** Agents already enabled stay enabled, and a discovered
  folder that ships enabled is on the roster having answered nothing. The gate is on the
  act of switching, not on membership. Re-validating the roster at startup would spend a
  call per agent without anyone asking.
- **`force` has no UI affordance.** It is an operator escape hatch. The web UI already
  reads `data.detail` first, toasts it and springs the switch back, so the reason reaches
  the user; a force button is a separate product decision.
- **The ping is bounded at 60s**, half the explicit-Test budget, because a person is
  waiting on the switch. Two measured agents hang rather than refuse, so the cap is what
  turns "no answer" into an answer. Both the run timeout and the handshake window are
  clamped to it, or a preset declaring a longer readiness window (three ship 120s) could
  only be cancelled from outside and never say the agent never became ready.
- **A `migrate` in the web UI is a remove plus an add**, so a migration now proves the new
  transport before it lands -- which is the right question to ask, because the transport
  change is exactly what makes the old evidence worthless. If that ping fails the add is
  refused and the row is already gone, leaving the agent unconfigured until the reader
  clicks Connect again. That non-atomicity is not new: on `origin/main` `subagents_add`
  can already raise `SubagentNotFoundError` on an unknown preset and a config error after
  `set_agents`. This makes an existing failure likelier, not new in kind, and the outcome
  is a reported failure rather than a silent one. Making it atomic wants a `replace` param
  or a `subagents.migrate` method, which is a new wire surface and its own change.
- **The gate pings live rather than reading a stored verdict.** A pass recorded before a
  token expired would put a dead agent back on the roster.

### Known gaps this branch does not close

- The enable ping is neither cancellable nor deduped, unlike `subagents.test`. Two
  simultaneous enables of one agent send two prompts.
- The ping is composed from a config read taken before it runs, so a row whose command is
  edited during the ping is enabled having proven the previous command. The window is one
  ping wide and the gate asks about the agent's health, not the config bytes.
- `SubagentsToggleParams` in `raven/rpc/models.py` is what the schema drift test compares
  against, and the handler is never validated against it -- so the added `force` proves
  schema-to-model agreement, not handler coverage. That is why the truthiness fix below was
  needed at the handler.
- Pre-existing on `main` and deliberately left alone rather than mixed into this branch:
  the TypeScript error map is three codes behind, `ConfigFieldReadonly` is undeclared on
  `subagents.toggle`, and the spec's param table has already drifted.
- The listing surface still cannot distinguish "ready, never run" from "ready, verified".
  This branch gates the switch rather than adding a third status; a UI distinction is a
  follow-up.
- **`subagents.add` is no longer free.** It can hold for up to 60s on a real model call,
  and none of the three surfaces that call it -- the web Connect button, the TUI's
  `/subagents add`, and the TUI add form -- shows anything while it waits. The web store's
  `run` bumps its redraw epoch only after the await, and the button carries no `disabled`,
  so all three block silently. The same cost already applied to the switch at the revision
  reviewed as `008b9840`; what is new is that a second surface pays it.
- Following from that: the web **Connect** button stays clickable during the wait, so two
  clicks spend two of the agent's calls and the loser is refused with
  `duplicate sub-agent name(s): [...]`. Two rows are never written -- the second validation
  refuses -- so this is a wasted call and a poor message, not corruption. Suppressing the
  action while it is in flight means a new in-flight field on the page store, whose redraw
  epoch is documented as remounting the card's uncontrolled form, so it wants its own
  change and its own tests rather than a hurried one here.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run from the branch head, rebased onto `origin/main`, with `PYTHONPATH=.` and
`COLUMNS=200`.

**Tests**

- `pytest tests/ -k "subagent or backend or rpc or living_docs or schema_match" -q`
  -- see the pipeline; locally green with only the two documented root-uid skips
  (`tests/test_subagents_install_script.py` cannot block `chmod 0o500` as root).
- `pytest tests/test_subagent_probe.py tests/test_rpc_subagents.py
  tests/test_subagent_third_party.py tests/test_subagent_acp.py
  tests/test_subagent_manager.py tests/test_rpc_schema_match.py tests/test_living_docs.py -q`
  -- 1034 passed at the first pushed head.
- At the head carrying the review fix: `pytest tests/test_rpc_subagents.py
  tests/test_rpc_schema_match.py tests/test_subagent_probe.py
  tests/test_subagent_third_party.py tests/test_update_subagents.py
  tests/test_living_docs.py -q` -- 738 passed. `npx vitest run src/features/xa` -- 43
  passed. `make lint-ui` and `make lint-tui` both exit 0 with the generators in sync.

**Gates**

- `ruff check raven/ tests/` -- All checks passed
- `ruff format --check raven/ tests/` -- 1491 files already formatted
- `lint-imports` -- Contracts: 9 kept, 0 broken (re-run after the rebase, which brought a
  ninth contract in)
- `scripts/check_source_language.py origin/main..HEAD` -- exit 0
- `scripts/check_commit_messages.py origin/main..HEAD` -- exit 0
- `scripts/check_large_files.py origin/main..HEAD` -- exit 0

**Against real agents, not stubs.** Most tests mock `ping_agent` or the backend, so both
directions were driven against real installed agents:

- Positive: `ping_agent(kimi_code)` returned `ok: True`, "it ran and replied", in 3.76s /
  4.27s / 6.89s across three runs -- 8.7x margin under the 60s cap.
- Negative, and this is the shape the feature exists for: `ping_agent(kilo)` -- one of the
  six agents that reported `ready` and could not complete a turn -- returned `ok: False`
  with `request failed: [-32603] Internal error: You need to sign in to use this model.`
  The full switch refused end to end in 5.60s with that sentence in `data["detail"]`, and
  the row stayed `false` on disk.
- A `cli` row pointing at a nonexistent binary refuses with the `ENOENT` in both the
  message and `data["detail"]`.

**Regression tests proven red before their fix**, by reverse-applying only the source hunk
and re-running: the two add-gate tests (`one Connect must leave the row dispatchable`, and
`DID NOT RAISE SubagentNotReadyError` for the refusal that must store nothing),
the concurrent-write test (`the write made during the ping must survive`),
the truthy-`force` test (`DID NOT RAISE SubagentNotReadyError`), the handshake-clamp
assertion (`KeyError: 'ready_timeout_ms'`), the two stale-row tests
(`SubagentNotFoundError`, and an assertion on the empty command), and the
`data["detail"]` test (`KeyError: 'detail'`). The three exemption tests were checked by
mutation instead: widening the gated-kind tuple with `openai`, dropping the `force` term,
and dropping the `enabled` term each turned exactly one of them red and left the other two
green.

**Rebase.** Replayed onto `origin/main` after it gained ten commits.
`git range-diff` reports all 13 commits as `=`. The only file both sides touched is
`CONTEXT.md`, in regions that do not meet (the base inserted at line 154 and edited around
541; this branch edits at 193). Every gate above was re-run at the new head, not carried
forward.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

**User-visible changes.** Switching a `cli` or `acp` agent on can now fail, with the
agent's reason shown; it spends one LLM call on that agent's quota when it succeeds.
Adding one from a preset can fail the same way, and a failed add stores nothing -- so an
agent nobody has signed into yet cannot be added at all without `force`, and the page has
no force button. That is the correct reading of the page's one-verb contract (Connect is
all-or-nothing), but it is a real behaviour change: previously the row could be stored and
enabled later. A `migrate` can now fail its ping and leave the agent unconfigured until
Connect is clicked again. Nothing already enabled is revisited, `openai` rows are
unaffected, and switching off is unchanged.

**A race this branch introduced and then closed.** The first version of the gate awaited
the ping between `get_agents` and `set_agents`, which turned an atomic read-modify-write
into a window up to 60s wide -- a concurrent `subagents.*` write inside it was silently
reverted, and a row deleted during the ping came back enabled. It was caught in review and
reproduced both ways. The gate now resolves once to compose the row to ping, pings, then
re-reads and re-resolves with no `await` between that read and the write; a regression test
holds a mocked ping open while a second row is written and asserts both writes survive.

**Rollback.** One RPC method's behaviour and one add-path default. Reverting the branch
restores the previous behaviour completely; there is no migration and no stored state whose
shape changed. The `force: true` param is the in-place escape hatch if a gate turns out to
refuse an agent that works.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Security note: the ping runs the agent's own launch command in a temporary directory with
its own instance registry, so a stateful create's handle binding never lands in the file
the running gateway reads. No credential is read, logged or moved by this change; the
refusal text is whatever the agent itself returned, capped at 2000 characters.

Backward compatibility: `force` is an optional param, absent means the pre-existing
behaviour for every exempt kind, and the new error code -32019 is additive. Clients that
never send `force` and never enable a local agent see no change.

## Related Issues

N/A
2026-09-09 02:56:15 +00:00
LivXue
a11309974f feat(*): a local sub-agent must answer a prompt before it can be switched on
## Summary

Switching a local sub-agent on in settings now sends one real prompt through that
agent's own backend and refuses the enable when the agent does not answer, reporting
the agent's own words as the reason.

The free check that already existed -- `shutil.which` on the launch command plus an ACP
`initialize` + `session/new` handshake -- was treated as proof an agent could work. It is
not. ACP has no authenticated-state field at all: `authMethods` is a capability list, not a
state, and an agent advertises it identically before and after login. Credentials are only
checked at the first *model* call, which `session/new` never reaches. Measured across the 13
registry acp agents on 2026-09-07 with one real prompt each, six returned `ready` from that
exact path and could not complete a single turn -- `kilo` ("You need to sign in to use this
model"), `gemini` and `dirac` (the prompt times out), `factory_droid`, `glm` and
`deepagents` (`-32603`).

So the check is split in two, and they answer different questions:

- **Layer 1**, unchanged and still free: `which` plus the handshake. It answers "is this
  installed, and is its ACP mode on", it is what puts the agent in the listing, and it
  spends nothing. `_test_acp`, `verify_agent`, `_probe_acp` and `run_test` are byte-identical
  on this branch.
- **Layer 2**, new: `ping_agent(cfg)` sends `PROBE_PROMPT` through the row's own backend and
  is reached only from the switch. The verdict rule is the existing one, "returned
  something" -- never "the reply contains PONG", because an agent that answers with a
  preamble is working and an empty reply is itself a signal.

Keeping them apart is the point. Collapsing the ping into layer 1 would make the listing
path spend a model call on every refresh and would remove the honest middle state the
roster needs: installed, not yet proven.

**What is gated, and what is not.** `cli` and `acp` rows are gated. `openai` is a remote
endpoint whose credential the free `/models` probe already settles, so charging a
completion for the switch would buy nothing. `builtin` is this process. Switching off is
never gated -- otherwise a broken agent could not be switched off. `force: true` is the
operator's override, for a working agent whose provider is briefly down.

**The add path is gated too.** `subagents_add` shipped `enabled: true` for a `cli`/`acp`
preset with no ping, and the web Connect button calls it -- so the gate covered one of two
enable verbs and was bypassable by the primary UI path. The add now runs the same ping
itself, ahead of the write: the preset answers and the row is written enabled, or the add
is refused in the agent's own words and **nothing is stored at all**.

Storing nothing is the load-bearing half. The page offers one verb per row
(`XaPage.tsx:16-24`), and a row that exists but is off reads as stage `off`, which offers
**Connect** again -- so a refusal that left a disabled row behind would turn one click into
an undocumented add-then-enable sequence. Leaving no row keeps the stage at `add`, so
Connect stays the one verb whether the ping passes or fails. An earlier revision of this
branch did land such presets disabled; that was reviewed as a blocker and replaced.

`subagents_add` also now takes the write's refusals -- a name already taken, an entry the
schema will not have -- **before** the ping rather than after it, so it never spends an
agent's quota proving a row the write was always going to reject. For a duplicate name it
would have proved the wrong row outright: the schema keeps the first of two, so the ping
would have run against the row already on disk. `validate_agents` is split out of
`set_agents` for that, with no rule duplicated; the write still validates whatever list it
finally holds.

`CONTEXT.md`'s Product agent entry said a missing credential is deliberately not a
readiness reason, which the gate now contradicts. The entry is amended in the same change
per AGENTS.md section 6: readiness asks about the folder, the switch asks about the token,
and neither validates retroactively.

### Decisions a reviewer should know were chosen, not overlooked

- **Cost.** Enabling a `cli` or `acp` agent now spends one LLM call on that agent's own
  quota. `cli` already paid this on an explicit Test; what is new is that `acp` starts to,
  and that the switch triggers it. That is the requirement, not a side effect.
- **No retroactive validation.** Agents already enabled stay enabled, and a discovered
  folder that ships enabled is on the roster having answered nothing. The gate is on the
  act of switching, not on membership. Re-validating the roster at startup would spend a
  call per agent without anyone asking.
- **`force` has no UI affordance.** It is an operator escape hatch. The web UI already
  reads `data.detail` first, toasts it and springs the switch back, so the reason reaches
  the user; a force button is a separate product decision.
- **The ping is bounded at 60s**, half the explicit-Test budget, because a person is
  waiting on the switch. Two measured agents hang rather than refuse, so the cap is what
  turns "no answer" into an answer. Both the run timeout and the handshake window are
  clamped to it, or a preset declaring a longer readiness window (three ship 120s) could
  only be cancelled from outside and never say the agent never became ready.
- **A `migrate` in the web UI is a remove plus an add**, so a migration now proves the new
  transport before it lands -- which is the right question to ask, because the transport
  change is exactly what makes the old evidence worthless. If that ping fails the add is
  refused and the row is already gone, leaving the agent unconfigured until the reader
  clicks Connect again. That non-atomicity is not new: on `origin/main` `subagents_add`
  can already raise `SubagentNotFoundError` on an unknown preset and a config error after
  `set_agents`. This makes an existing failure likelier, not new in kind, and the outcome
  is a reported failure rather than a silent one. Making it atomic wants a `replace` param
  or a `subagents.migrate` method, which is a new wire surface and its own change.
- **The gate pings live rather than reading a stored verdict.** A pass recorded before a
  token expired would put a dead agent back on the roster.

### Known gaps this branch does not close

- The enable ping is neither cancellable nor deduped, unlike `subagents.test`. Two
  simultaneous enables of one agent send two prompts.
- The ping is composed from a config read taken before it runs, so a row whose command is
  edited during the ping is enabled having proven the previous command. The window is one
  ping wide and the gate asks about the agent's health, not the config bytes.
- `SubagentsToggleParams` in `raven/rpc/models.py` is what the schema drift test compares
  against, and the handler is never validated against it -- so the added `force` proves
  schema-to-model agreement, not handler coverage. That is why the truthiness fix below was
  needed at the handler.
- Pre-existing on `main` and deliberately left alone rather than mixed into this branch:
  the TypeScript error map is three codes behind, `ConfigFieldReadonly` is undeclared on
  `subagents.toggle`, and the spec's param table has already drifted.
- The listing surface still cannot distinguish "ready, never run" from "ready, verified".
  This branch gates the switch rather than adding a third status; a UI distinction is a
  follow-up.
- **`subagents.add` is no longer free.** It can hold for up to 60s on a real model call,
  and none of the three surfaces that call it -- the web Connect button, the TUI's
  `/subagents add`, and the TUI add form -- shows anything while it waits. The web store's
  `run` bumps its redraw epoch only after the await, and the button carries no `disabled`,
  so all three block silently. The same cost already applied to the switch at the revision
  reviewed as `008b9840`; what is new is that a second surface pays it.
- Following from that: the web **Connect** button stays clickable during the wait, so two
  clicks spend two of the agent's calls and the loser is refused with
  `duplicate sub-agent name(s): [...]`. Two rows are never written -- the second validation
  refuses -- so this is a wasted call and a poor message, not corruption. Suppressing the
  action while it is in flight means a new in-flight field on the page store, whose redraw
  epoch is documented as remounting the card's uncontrolled form, so it wants its own
  change and its own tests rather than a hurried one here.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run from the branch head, rebased onto `origin/main`, with `PYTHONPATH=.` and
`COLUMNS=200`.

**Tests**

- `pytest tests/ -k "subagent or backend or rpc or living_docs or schema_match" -q`
  -- see the pipeline; locally green with only the two documented root-uid skips
  (`tests/test_subagents_install_script.py` cannot block `chmod 0o500` as root).
- `pytest tests/test_subagent_probe.py tests/test_rpc_subagents.py
  tests/test_subagent_third_party.py tests/test_subagent_acp.py
  tests/test_subagent_manager.py tests/test_rpc_schema_match.py tests/test_living_docs.py -q`
  -- 1034 passed at the first pushed head.
- At the head carrying the review fix: `pytest tests/test_rpc_subagents.py
  tests/test_rpc_schema_match.py tests/test_subagent_probe.py
  tests/test_subagent_third_party.py tests/test_update_subagents.py
  tests/test_living_docs.py -q` -- 738 passed. `npx vitest run src/features/xa` -- 43
  passed. `make lint-ui` and `make lint-tui` both exit 0 with the generators in sync.

**Gates**

- `ruff check raven/ tests/` -- All checks passed
- `ruff format --check raven/ tests/` -- 1491 files already formatted
- `lint-imports` -- Contracts: 9 kept, 0 broken (re-run after the rebase, which brought a
  ninth contract in)
- `scripts/check_source_language.py origin/main..HEAD` -- exit 0
- `scripts/check_commit_messages.py origin/main..HEAD` -- exit 0
- `scripts/check_large_files.py origin/main..HEAD` -- exit 0

**Against real agents, not stubs.** Most tests mock `ping_agent` or the backend, so both
directions were driven against real installed agents:

- Positive: `ping_agent(kimi_code)` returned `ok: True`, "it ran and replied", in 3.76s /
  4.27s / 6.89s across three runs -- 8.7x margin under the 60s cap.
- Negative, and this is the shape the feature exists for: `ping_agent(kilo)` -- one of the
  six agents that reported `ready` and could not complete a turn -- returned `ok: False`
  with `request failed: [-32603] Internal error: You need to sign in to use this model.`
  The full switch refused end to end in 5.60s with that sentence in `data["detail"]`, and
  the row stayed `false` on disk.
- A `cli` row pointing at a nonexistent binary refuses with the `ENOENT` in both the
  message and `data["detail"]`.

**Regression tests proven red before their fix**, by reverse-applying only the source hunk
and re-running: the two add-gate tests (`one Connect must leave the row dispatchable`, and
`DID NOT RAISE SubagentNotReadyError` for the refusal that must store nothing),
the concurrent-write test (`the write made during the ping must survive`),
the truthy-`force` test (`DID NOT RAISE SubagentNotReadyError`), the handshake-clamp
assertion (`KeyError: 'ready_timeout_ms'`), the two stale-row tests
(`SubagentNotFoundError`, and an assertion on the empty command), and the
`data["detail"]` test (`KeyError: 'detail'`). The three exemption tests were checked by
mutation instead: widening the gated-kind tuple with `openai`, dropping the `force` term,
and dropping the `enabled` term each turned exactly one of them red and left the other two
green.

**Rebase.** Replayed onto `origin/main` after it gained ten commits.
`git range-diff` reports all 13 commits as `=`. The only file both sides touched is
`CONTEXT.md`, in regions that do not meet (the base inserted at line 154 and edited around
541; this branch edits at 193). Every gate above was re-run at the new head, not carried
forward.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

**User-visible changes.** Switching a `cli` or `acp` agent on can now fail, with the
agent's reason shown; it spends one LLM call on that agent's quota when it succeeds.
Adding one from a preset can fail the same way, and a failed add stores nothing -- so an
agent nobody has signed into yet cannot be added at all without `force`, and the page has
no force button. That is the correct reading of the page's one-verb contract (Connect is
all-or-nothing), but it is a real behaviour change: previously the row could be stored and
enabled later. A `migrate` can now fail its ping and leave the agent unconfigured until
Connect is clicked again. Nothing already enabled is revisited, `openai` rows are
unaffected, and switching off is unchanged.

**A race this branch introduced and then closed.** The first version of the gate awaited
the ping between `get_agents` and `set_agents`, which turned an atomic read-modify-write
into a window up to 60s wide -- a concurrent `subagents.*` write inside it was silently
reverted, and a row deleted during the ping came back enabled. It was caught in review and
reproduced both ways. The gate now resolves once to compose the row to ping, pings, then
re-reads and re-resolves with no `await` between that read and the write; a regression test
holds a mocked ping open while a second row is written and asserts both writes survive.

**Rollback.** One RPC method's behaviour and one add-path default. Reverting the branch
restores the previous behaviour completely; there is no migration and no stored state whose
shape changed. The `force: true` param is the in-place escape hatch if a gate turns out to
refuse an agent that works.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Security note: the ping runs the agent's own launch command in a temporary directory with
its own instance registry, so a stateful create's handle binding never lands in the file
the running gateway reads. No credential is read, logged or moved by this change; the
refusal text is whatever the agent itself returned, capped at 2000 characters.

Backward compatibility: `force` is an optional param, absent means the pre-existing
behaviour for every exempt kind, and the new error code -32019 is additive. Clients that
never send `force` and never enable a local agent see no change.

## Related Issues

N/A
2026-09-09 02:56:15 +00:00
0xKT
d25bdb9c67 fix(providers): an inferred protocol steps aside where the provider has no address for it
## Summary

Fixes the key-only regression reported in #11: 56 curated models on dashscope, moonshot, groq and nvidia_nim stopped constructing a provider after the model-family protocol inference landed, because the family name inferred `responses` and `native_api_base` rightly refused to invent an address for a protocol those providers never declared.

Two rules now govern the choice, both in `raven/providers/protocol.py`:

- A protocol the user configured (`protocol` / `modelProtocols`) is final. Choosing `responses` for a provider with no address for it is still refused, as before.
- A protocol merely inferred from the model family steps aside to `chat` when the provider declares no address for it. `effective_protocol` takes the provider name and the configured address for this; the factory and the model picker pass them. The bare inference (no provider named) is unchanged, so protocol tables and picker labels read as before.
- The gateway fallback to a default address now applies only to a gateway that declares no per-protocol address at all (openrouter, custom). volcengine declares a `responses` address only, so an `anthropic` request no longer receives it: `volcengine/glm-4-7-251222` goes over chat instead of posting `/v1/messages` to a responses endpoint, and the picker can warn where a base is genuinely missing.

The OpenRouter question from the acceptance note is answered: `POST https://openrouter.ai/api/v1/responses` was exercised live (200, `object: response`), so the 35 openrouter rows that route to Responses are served.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- Key-only sweep over every (provider, picker model) pair, base `ce2e0b91` vs this head, same script: `compared: 440  WORKED -> REFUSES: 0  REFUSED -> WORKS: 56` (dashscope 33, moonshot 12, groq 6, nvidia_nim 5); the only other change is `volcengine/glm-4-7-251222` moving from `AnthropicMessagesProvider` to `LiteLLMProvider` (chat). The 68 pairs that fail on both trees are OAuth sign-in providers (github_copilot, minimax) and ollama without a running server.
- `pytest -q tests/test_provider_protocol.py tests/test_providers_factory.py tests/test_rpc_model.py`: 242 passed (three new tests: the four-provider fallback, the explicit-choice refusal, the partial-declaration gateway).
- `pytest -q tests/test_provider*.py tests/test_providers*.py tests/test_rpc_model.py tests/test_cli_doctor_commands.py`: 958 passed, 1 failed (`test_doctor_answers_where_the_memories_are`, environment-dependent on the development box, fails identically on `main`).
- `lint-imports`: 9 contracts kept, 0 broken. `ruff format --check` and `ruff check` over `raven` and `tests`: clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Behaviour change: an inferred native protocol on a provider without an address for it falls back to chat instead of failing; an info log names the fallback. Explicit selections are untouched. Only volcengine is affected by the narrowed gateway fallback (the one gateway declaring a partial protocol map). Rollback is a revert of the squash commit.

## Related Issues

Fixes #11
2026-09-09 02:55:39 +00:00
daoxize1
43f0e2a137 fix(providers): an inferred protocol steps aside where the provider has no address for it
## Summary

Fixes the key-only regression reported in #11: 56 curated models on dashscope, moonshot, groq and nvidia_nim stopped constructing a provider after the model-family protocol inference landed, because the family name inferred `responses` and `native_api_base` rightly refused to invent an address for a protocol those providers never declared.

Two rules now govern the choice, both in `raven/providers/protocol.py`:

- A protocol the user configured (`protocol` / `modelProtocols`) is final. Choosing `responses` for a provider with no address for it is still refused, as before.
- A protocol merely inferred from the model family steps aside to `chat` when the provider declares no address for it. `effective_protocol` takes the provider name and the configured address for this; the factory and the model picker pass them. The bare inference (no provider named) is unchanged, so protocol tables and picker labels read as before.
- The gateway fallback to a default address now applies only to a gateway that declares no per-protocol address at all (openrouter, custom). volcengine declares a `responses` address only, so an `anthropic` request no longer receives it: `volcengine/glm-4-7-251222` goes over chat instead of posting `/v1/messages` to a responses endpoint, and the picker can warn where a base is genuinely missing.

The OpenRouter question from the acceptance note is answered: `POST https://openrouter.ai/api/v1/responses` was exercised live (200, `object: response`), so the 35 openrouter rows that route to Responses are served.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- Key-only sweep over every (provider, picker model) pair, base `ce2e0b91` vs this head, same script: `compared: 440  WORKED -> REFUSES: 0  REFUSED -> WORKS: 56` (dashscope 33, moonshot 12, groq 6, nvidia_nim 5); the only other change is `volcengine/glm-4-7-251222` moving from `AnthropicMessagesProvider` to `LiteLLMProvider` (chat). The 68 pairs that fail on both trees are OAuth sign-in providers (github_copilot, minimax) and ollama without a running server.
- `pytest -q tests/test_provider_protocol.py tests/test_providers_factory.py tests/test_rpc_model.py`: 242 passed (three new tests: the four-provider fallback, the explicit-choice refusal, the partial-declaration gateway).
- `pytest -q tests/test_provider*.py tests/test_providers*.py tests/test_rpc_model.py tests/test_cli_doctor_commands.py`: 958 passed, 1 failed (`test_doctor_answers_where_the_memories_are`, environment-dependent on the development box, fails identically on `main`).
- `lint-imports`: 9 contracts kept, 0 broken. `ruff format --check` and `ruff check` over `raven` and `tests`: clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Behaviour change: an inferred native protocol on a provider without an address for it falls back to chat instead of failing; an info log names the fallback. Explicit selections are untouched. Only volcengine is affected by the narrowed gateway fallback (the one gateway declaring a partial protocol map). Rollback is a revert of the squash commit.

## Related Issues

Fixes #11
2026-09-09 02:55:39 +00:00
daoxize1
4e8475e4d9 fix(cli): the onboarding wizard takes no key for a product that runs on the host's llm
## Summary

Fixes #12. Raven-Design runs on the host's model and credentials since the launcher retired its own-key path, but `raven onboard` was not updated: it still recommended `openai/gpt-5.6-sol via OpenRouter` (a model the product no longer runs), still took a key and wrote `DESIGN_API_KEY` to `.env` (never read again), and with no host provider key it offered only the own-key choice and printed the ready tick for a product whose launcher then exited.

- `agents/raven-design/subagent.json` no longer carries `recommendedLlm`. A folder that recommends no model of its own has nothing to run on but the host's LLM, and the wizard now reads that: no own-key option, no recommended-model line, no key written. It is set up (ticked) when this raven has a key to lend, and reported not ready otherwise. Raven-PPT still recommends its own model and keeps the full menu, so the two products' difference is now stated by the manifest rather than assumed.
- The two smaller items folded into the issue: the product's `tools.web.search.maxResults` leaf is removed from `agents/raven-design/config.json` because the launcher hands the product the host's `tools.web` wholesale (the identity test now asserts the trunk config carries no `tools.web`); the settings usage tile's missing-data note gains the one-line assertion from the !561 acceptance note, verified to fail when the element is removed.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest -q tests/test_cli_subagent_setup.py tests/test_agents_design_launcher.py tests/test_i18n_boundary.py tests/test_cli_onboard_commands.py`: 403 passed. Two new wizard tests: a folder without a model of its own is ticked on a host key with zero prompts and no `.env`; the same folder without a host key is reported not ready, zero prompts, no `.env`.
- Every test file mentioning `recommendedLlm` (code, design, ppt, oncall launchers, wizard, install script): 194 passed, 1 skipped. `tests/test_subagent_raven_launcher.py tests/test_config_update_tools.py`: 57 passed.
- `ui-web`: `vitest run src/features/settings/SettingsPage.test.tsx` 49 passed on the tree; with the `note` element deleted from `SettingsPage.tsx` exactly one test fails (`shows unknown tokens for an image model that only reports money`), so the new assertion pins the element.
- `lint-imports`: 9 contracts kept, 0 broken. `ruff format --check` and `ruff check` over `raven` and `tests`: clean. The two new wizard strings carry `zh` entries in `raven/i18n/zh.py`.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: `raven onboard` no longer asks for a Raven-Design key or recommends a model for it; an existing `DESIGN_API_KEY` line in `.env` is left in place and stays unread, as before this change. `product_state` is unchanged, so the roster still lists Raven-Design; only the wizard's tick now follows whether the host can lend a key. Rollback is a revert of the squash commit.

## Related Issues

Fixes #12
2026-09-09 02:40:02 +00:00
0xKT
3a7cc17aeb fix(*): raven-research takes the host web vendor and key, and the ask_user gate closes
## Summary

Three strands of raven-research work, held together because they all touch the vendored fork and so share one `subagents/TREE_HASHES` line.

**Web search and fetch route through the vendor the deployment picked.** Both tools in the research-flow plugin were pinned to Serper and Jina, so a host on Tavily or Brave still needed a Serper key for the pair. They now carry the kernel's vendor tables (seven search backends, five fetch) with per-vendor request encoding and response normalisation. Serper and Jina wire traffic is byte-identical to before. A vendor that publishes no result offset declares it up front (`SearchProviderSpec.paginates`), so the saturation rule never escalates to a page rung the vendor cannot answer.

The launcher resolves the vendor first and mirrors the SELECTED vendor's key into the plugin slice, never a key by fixed path: a mismatched pair sends one vendor's credential to another's endpoint and every call comes back unauthorized. Every vendor gets an env slot at the same path the host holds it, inherited from the host config when unset, so a deployment configures a vendor once, centrally. The two legacy vendors are also read from the vendor slot, where a host configured after the vendor table landed writes them. `require_search` follows the selection rather than the default.

**The research modes take the host's effort vocabulary**: `deep` becomes `high`, `ultra` becomes `max`, and `medium` is the baseline. This is what lets the companion MR's host tier reach this agent by name.

**The ask_user gate closes after a hook rollback**, and host questions take the conversation lock like every other route, including the `direct_ask` grant. The digest follows its session's mode, a typo in a mode overlay refuses to launch rather than running the base value under the overlay's label, and the reviewer records which model judged.

Three defects found while building it, each with the test that pins it:

- **A credential reached disk.** The retry rows of the search and fetch ledger recorded `str(exc)` for a status error, and httpx puts the whole request URL in that text. SerpApi authenticates by query parameter, so a 429 or 503 wrote the key into the JSONL file. The redaction the search and fetch rows already had is now one function, `_error_text`, used by both writers, in the plugin and in the vendored copy. The regression test drives a retryable status and reads the ledger file off disk; the previous test used 401, which is not retryable and so never reached the leaking write. The key never reached the model or the answer: the appendix partitions on `search` / `fetch` / `verify` / `force_finalize` and reads nothing from a retry row.
- **A keyless fetch backend refused every call.** The kernel's `web_fetch` degrades a keyed backend with no resolvable key to Jina, out loud. The plugin's replacement did not, and it REPLACES the kernel's, so `fetch.provider = tavily` with no Tavily key made a research run the single place that config could not read a page, while a plain raven on it read the page fine. The launcher itself produces that state: it writes the provider unconditionally and the key only when one resolves. Fixed at the build site, where the kernel resolves it, and a test asserts both implementations answer identically for every vendor.

  Reported by chandler.zhang and completed in `ac7d364c`: the degraded backend was still handed the key of the vendor it replaced, which is absent by definition, so a deployment holding a Jina key read pages authenticated through raven's built-in fetch and anonymously through the plugin -- losing the configured quota on the very fallback the rule exists to match. The kernel looks the key up a second time under the EFFECTIVE provider; the slice carries one key per role and no vendor table, so the launcher now mirrors the default reader's key beside the selected vendor's and the build picks by destination, a lookup keyed on the provider the tool will actually call. It is mirrored whenever the selection is not already the default rather than only when the fallback will fire, because the rule that decides that lives in `effective_provider` and a launcher second-guessing it would be the copy that drifts.
- **A typo in a vendor name refused to launch for the wrong reason.** The plugin degrades an unknown vendor to its default, which is right for a slice nothing validates. Carried through the launcher, the same typo had `require_search` demand a key for a vendor that does not exist while the working Serper key sat in the config -- sending the operator to set a credential they already had. The launcher now refuses naming the name and the valid list.

`.env.example` and the plugin docstrings were still describing the fixed Serper and Jina pair; both now cover the provider switch and the per-vendor key slots. The plugin MANIFEST was missed in that pass and is corrected in `18ce29df`, also on chandler.zhang's report: it still called `search.apiKey` a Serper key and `fetch.apiKey` a Jina reader key, and said web_search declines "when no Serper key resolves". A direct plugin user reaching it without the launcher could set `search.provider = tavily` and supply a Serper credential, which this code then sends to Tavily's endpoint.

`subagents/TREE_HASHES` is recorded once, in the final commit, rather than per commit: the hash covers every tracked byte under the fork, so an intermediate value goes stale the moment the next edit or a rebase lands.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch at its current head:

- `uv run pytest` -- 5 failed, 18545 passed, 69 skipped. The five are this host's pre-existing baseline, identical on `origin/main`: `test_cli_doctor_commands.py::test_doctor_answers_where_the_memories_are` and four `test_security_network.py` NAT64 parameters. 0 introduced.
- `make lint-python` -- passed, 1787 files already formatted.
- `uv run lint-imports` -- 8 contracts kept, 0 broken.
- `make check-vendored-subagents` -- exit 0, hash recorded.
- `make check-vendored-invariants`, `make check-large-files`, `make lint-ui`, `make lint-tui`, `scripts/check_commit_messages.py origin/main..HEAD` -- all exit 0.
- The vendored fork's own suite, run with the fork on `PYTHONPATH` so `raven` resolves to its copy: 19 failed, 6223 passed, 4 errors. Reverting only this branch's change to the fork gives the same 19 failures and 4 errors, all in three files that never import the changed module, so 0 introduced. Those failures are an artifact of running the fork against the trunk's venv, not the fork's own.

Reverting each fix and confirming the matching test fails: the ledger redaction (the 429 case red, the 401 case correctly still green), the fetch degradation (4 keyed vendors red, Jina correctly unaffected), the vendor-name refusal (2 red), and the slice key following the selected vendor (2 red).

Three more for the review fix: the build passing the selected vendor's key to the reader that replaced it (1 red), the launcher not mirroring the fallback key (1 red), and the launcher mirroring it for the default vendor too (1 red, the case where a second copy of the same key is one more place for the pair to disagree).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Security: a credential stopped being written to disk. Any research-flow ledger file written before this change by a deployment on SerpApi may contain the key in a `search_retry` or `fetch_retry` row. Those files live under the plugin's state root, one per turn, and are unlinked at the end of the turn that wrote them; a process killed mid-turn leaves one behind. Treat a SerpApi key used on an affected build as exposed and rotate it, then check for leftover files.

Other user-visible changes:

- A research deployment that selected a keyed fetch vendor without its key now reads pages through Jina instead of failing every fetch, and does so on the host's configured Jina key rather than anonymously. Behaviour improves; the log says which vendor is serving.
- That fallback key now travels in the rendered plugin slice as `fetch.fallbackApiKey`, beside the selected vendor's key, wherever the rendered config already holds credentials. It is only ever handed to the default reader: the build looks the key up by destination, so a provider neither key belongs to gets none.
- An unknown vendor name in `search.provider` or `fetch.provider` now refuses to launch. A deployment running on a typo today would have been failing its search calls anyway.
- The research mode ids change: `deep` becomes `high` and `ultra` becomes `max`. A caller that names `deep` by id gets the agent's default instead.
- Ledger `*_retry` rows record `HTTPStatusError: HTTP <status>` where they recorded the full httpx message. The one offline reader, the fork's `bench/rollout.py`, buckets by status either way and already read the search and fetch rows in that shape.

Rollback: revert the commits on this branch. `TREE_HASHES` is recorded in its own final commit, so reverting the set restores the recorded hash with the tree.

## Related Issues

N/A

<!-- The repo guide says to leave this trailer to the commits, on the premise that
     the platform collects it into the squash commit. This GitLab does not, and the
     squash body is taken from this description, so it is repeated here to survive. -->

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-06 04:12:09 +00:00
userName20260323
fec055779d fix(*): raven-research takes the host web vendor and key, and the ask_user gate closes
## Summary

Three strands of raven-research work, held together because they all touch the vendored fork and so share one `subagents/TREE_HASHES` line.

**Web search and fetch route through the vendor the deployment picked.** Both tools in the research-flow plugin were pinned to Serper and Jina, so a host on Tavily or Brave still needed a Serper key for the pair. They now carry the kernel's vendor tables (seven search backends, five fetch) with per-vendor request encoding and response normalisation. Serper and Jina wire traffic is byte-identical to before. A vendor that publishes no result offset declares it up front (`SearchProviderSpec.paginates`), so the saturation rule never escalates to a page rung the vendor cannot answer.

The launcher resolves the vendor first and mirrors the SELECTED vendor's key into the plugin slice, never a key by fixed path: a mismatched pair sends one vendor's credential to another's endpoint and every call comes back unauthorized. Every vendor gets an env slot at the same path the host holds it, inherited from the host config when unset, so a deployment configures a vendor once, centrally. The two legacy vendors are also read from the vendor slot, where a host configured after the vendor table landed writes them. `require_search` follows the selection rather than the default.

**The research modes take the host's effort vocabulary**: `deep` becomes `high`, `ultra` becomes `max`, and `medium` is the baseline. This is what lets the companion MR's host tier reach this agent by name.

**The ask_user gate closes after a hook rollback**, and host questions take the conversation lock like every other route, including the `direct_ask` grant. The digest follows its session's mode, a typo in a mode overlay refuses to launch rather than running the base value under the overlay's label, and the reviewer records which model judged.

Three defects found while building it, each with the test that pins it:

- **A credential reached disk.** The retry rows of the search and fetch ledger recorded `str(exc)` for a status error, and httpx puts the whole request URL in that text. SerpApi authenticates by query parameter, so a 429 or 503 wrote the key into the JSONL file. The redaction the search and fetch rows already had is now one function, `_error_text`, used by both writers, in the plugin and in the vendored copy. The regression test drives a retryable status and reads the ledger file off disk; the previous test used 401, which is not retryable and so never reached the leaking write. The key never reached the model or the answer: the appendix partitions on `search` / `fetch` / `verify` / `force_finalize` and reads nothing from a retry row.
- **A keyless fetch backend refused every call.** The kernel's `web_fetch` degrades a keyed backend with no resolvable key to Jina, out loud. The plugin's replacement did not, and it REPLACES the kernel's, so `fetch.provider = tavily` with no Tavily key made a research run the single place that config could not read a page, while a plain raven on it read the page fine. The launcher itself produces that state: it writes the provider unconditionally and the key only when one resolves. Fixed at the build site, where the kernel resolves it, and a test asserts both implementations answer identically for every vendor.

  Reported by chandler.zhang and completed in `ac7d364c`: the degraded backend was still handed the key of the vendor it replaced, which is absent by definition, so a deployment holding a Jina key read pages authenticated through raven's built-in fetch and anonymously through the plugin -- losing the configured quota on the very fallback the rule exists to match. The kernel looks the key up a second time under the EFFECTIVE provider; the slice carries one key per role and no vendor table, so the launcher now mirrors the default reader's key beside the selected vendor's and the build picks by destination, a lookup keyed on the provider the tool will actually call. It is mirrored whenever the selection is not already the default rather than only when the fallback will fire, because the rule that decides that lives in `effective_provider` and a launcher second-guessing it would be the copy that drifts.
- **A typo in a vendor name refused to launch for the wrong reason.** The plugin degrades an unknown vendor to its default, which is right for a slice nothing validates. Carried through the launcher, the same typo had `require_search` demand a key for a vendor that does not exist while the working Serper key sat in the config -- sending the operator to set a credential they already had. The launcher now refuses naming the name and the valid list.

`.env.example` and the plugin docstrings were still describing the fixed Serper and Jina pair; both now cover the provider switch and the per-vendor key slots. The plugin MANIFEST was missed in that pass and is corrected in `18ce29df`, also on chandler.zhang's report: it still called `search.apiKey` a Serper key and `fetch.apiKey` a Jina reader key, and said web_search declines "when no Serper key resolves". A direct plugin user reaching it without the launcher could set `search.provider = tavily` and supply a Serper credential, which this code then sends to Tavily's endpoint.

`subagents/TREE_HASHES` is recorded once, in the final commit, rather than per commit: the hash covers every tracked byte under the fork, so an intermediate value goes stale the moment the next edit or a rebase lands.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch at its current head:

- `uv run pytest` -- 5 failed, 18545 passed, 69 skipped. The five are this host's pre-existing baseline, identical on `origin/main`: `test_cli_doctor_commands.py::test_doctor_answers_where_the_memories_are` and four `test_security_network.py` NAT64 parameters. 0 introduced.
- `make lint-python` -- passed, 1787 files already formatted.
- `uv run lint-imports` -- 8 contracts kept, 0 broken.
- `make check-vendored-subagents` -- exit 0, hash recorded.
- `make check-vendored-invariants`, `make check-large-files`, `make lint-ui`, `make lint-tui`, `scripts/check_commit_messages.py origin/main..HEAD` -- all exit 0.
- The vendored fork's own suite, run with the fork on `PYTHONPATH` so `raven` resolves to its copy: 19 failed, 6223 passed, 4 errors. Reverting only this branch's change to the fork gives the same 19 failures and 4 errors, all in three files that never import the changed module, so 0 introduced. Those failures are an artifact of running the fork against the trunk's venv, not the fork's own.

Reverting each fix and confirming the matching test fails: the ledger redaction (the 429 case red, the 401 case correctly still green), the fetch degradation (4 keyed vendors red, Jina correctly unaffected), the vendor-name refusal (2 red), and the slice key following the selected vendor (2 red).

Three more for the review fix: the build passing the selected vendor's key to the reader that replaced it (1 red), the launcher not mirroring the fallback key (1 red), and the launcher mirroring it for the default vendor too (1 red, the case where a second copy of the same key is one more place for the pair to disagree).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Security: a credential stopped being written to disk. Any research-flow ledger file written before this change by a deployment on SerpApi may contain the key in a `search_retry` or `fetch_retry` row. Those files live under the plugin's state root, one per turn, and are unlinked at the end of the turn that wrote them; a process killed mid-turn leaves one behind. Treat a SerpApi key used on an affected build as exposed and rotate it, then check for leftover files.

Other user-visible changes:

- A research deployment that selected a keyed fetch vendor without its key now reads pages through Jina instead of failing every fetch, and does so on the host's configured Jina key rather than anonymously. Behaviour improves; the log says which vendor is serving.
- That fallback key now travels in the rendered plugin slice as `fetch.fallbackApiKey`, beside the selected vendor's key, wherever the rendered config already holds credentials. It is only ever handed to the default reader: the build looks the key up by destination, so a provider neither key belongs to gets none.
- An unknown vendor name in `search.provider` or `fetch.provider` now refuses to launch. A deployment running on a typo today would have been failing its search calls anyway.
- The research mode ids change: `deep` becomes `high` and `ultra` becomes `max`. A caller that names `deep` by id gets the agent's default instead.
- Ledger `*_retry` rows record `HTTPStatusError: HTTP <status>` where they recorded the full httpx message. The one offline reader, the fork's `bench/rollout.py`, buckets by status either way and already read the search and fetch rows in that shape.

Rollback: revert the commits on this branch. `TREE_HASHES` is recorded in its own final commit, so reverting the set restores the recorded hash with the tree.

## Related Issues

N/A

<!-- The repo guide says to leave this trailer to the commits, on the premise that
     the platform collects it into the squash commit. This GitLab does not, and the
     squash body is taken from this description, so it is repeated here to survive. -->

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-06 04:12:09 +00:00
0xKT
2f862d8ca2 feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
hylarucoder
c703db6ffd feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
0xKT
9a98c3107f feat(*): subagent discovery switches rails from the forks to agents/
## Summary

Subagent discovery switches rails from the subagents/ fork trees to the
agents/ product tree. This makes the five migrated products (code, oncall,
ppt, research, design) the factory default: a fresh install discovers them
without running any per-product installer, and the fork trees stop being
scanned entirely (they remain on disk, untouched, until their separately
scheduled retirement).

Key decisions:

- Discovery resolves an agents root through the same three levels the fork
  scan used: raven home copy, beside the package, wheel-embedded. The copy
  to raven home is kept folders-only so a product's .env survives upgrades.
- Readiness has new semantics: the launcher's absolute-path command tokens
  must exist, and a product-declared engine distribution must be importable
  (importlib find_spec). A product whose engine wheel is missing is listed
  but disabled, naming the wheel to install. Credential checks leave the
  discovery phase: the launcher itself refuses loudly, which live handshake
  smoke verified for all five products.
- Config rows keep whole-row precedence over discovered rows; the merge
  mechanics are unchanged (verified byte-equivalent modulo renames by an
  independent adversarial review).
- The main wheel now ships the five product launchers under raven/agents
  and no longer embeds raven/subagents (118 files / 1.3 MB, verified by
  building and listing the wheel; zero pyproject changes, the inclusion
  lives in the hatch build hook).
- The design/ppt manifests gain one machine-readable field,
  `engine: {package, wheel}`, consumed by the readiness probe; launch
  behavior is untouched and older readers ignore the field (verified).
- A discovered row whose readiness is clean but whose runtime probe misses
  now renders as attention instead of a dead Install button (fix from the
  adversarial review).

An independent adversarial review (verdict: land after fixes) preceded this
MR; its one HIGH (net test-coverage loss on host_can_lend_a_key edge
branches) and one MED (the Install dead-end above) are fixed in the commit,
with companion tests. Fork-era behaviors it confirmed as bit-identical
inheritances (home-first shadowing, path-with-spaces refusal) are recorded,
not changed.

## Type

- [x] Feature

## Verification

- Full canonical suite on the branch: `uv run pytest tests -q` under
  `uv sync --all-extras` -> 18494 passed, 40 skipped, exit 0 (rehearsal
  tree); 18472 passed, 62 skipped on the drafting stack. Net def-level
  movement vs baseline accounted name-for-name in the wave report.
- Targeted families at landing: tests/test_agents_ppt_launcher.py,
  tests/test_cli_subagent_setup.py, tests/test_hatch_build.py,
  tests/test_rpc_subagents.py, tests/test_subagent_dag_machines.py,
  tests/test_subagent_vendored_agents.py -> 242 passed.
- `make lint-python` -> exit 0; `uv run lint-imports` -> 8 kept, 0 broken;
  `make check-vendored-subagents`, `check-vendored-invariants`,
  `check-large-files` -> exit 0. subagents/ is byte-untouched.
- Wheel audit: `uv build --wheel`, listed contents -> 5 launchers +
  5 manifests under raven/agents, 0 raven/subagents entries, no secrets.
- Fresh-venv install test: wheel installed into an isolated venv discovers
  and materializes all five rows; design/ppt correctly disabled without
  their engine wheels; research launcher runs to its credential verdict.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: fork rows stop appearing on fresh installs; the five
product rows appear instead. Machines with existing config rows are
unaffected (config precedence unchanged). The RPC wire shape is frozen
(vendored/building keys and subagents.build kept), so no client regeneration
is needed. Rollback: revert the squash commit; discovery returns to the
fork trees, which are still on disk in the repo and still embedded in any
wheel built before this change.

## Related Issues

N/A
2026-09-05 05:13:56 +00:00
0xKT
3872bdad6a feat(*): raven.i18n translates by English source text; the onboarding CLI speaks through it
The onboarding wizard carried its two languages inline, _t(en, zh) at 401 call
sites plus bilingual tables (the curated provider labels, the EverOS role
descriptions), and the wizard's language lived as a module global in the shared
CLI state. That kept Chinese literals in seven source files and gave nothing
else in the package a way to speak the user's language. raven.i18n is the seam:
t(text, **arguments) translates by the English source text (gettext style, no
invented keys), the Chinese catalog is one module, and the language is process
state the host sets (the wizard from its first screen, later from
config.language). The seven files now call t() with English only; the tables
hold English; a boundary test lists the modules still carrying CJK text and
lets that list only shrink.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-30 03:56:33 +08:00
Handsome-wzw
9a87470fdf fix(rpc): turn.started is a turn event, and says so on all three sides
## Summary

Found while investigating the first row of Dizhan's `子智能体功能与显示问题` doc (*回注内容过程中未正常显示,刷新后才会正常显示*). This is not that bug's fix, but it is a real defect on the same wire, and it is the one that explains why "前后端接线没有约定好" was a fair thing to say.

**`turn.started` is not in the `TurnEvent` union — on either side.** It has a schema component, it has a Pydantic model, and neither is reachable from the union that `tests/test_rpc_schema_match.py` walks. The 290-test contract guard has never looked at this event.

So it drifted the first time anyone touched it. `05957452` (judge a dag node) taught `announce_dag_exception` to emit:

```python
mark = {"kind": "dag", "label": run_id, "status": "exception", "run_id": run_id, "node_id": node_id}
```

while `TurnStartedEvent.payload.delegated` still declared `status: enum ["ok","error"]` with `additionalProperties: false`, and `TurnStartedPayload` had **no `delegated` field at all** under `extra="forbid"`. Verified with `jsonschema` against the published contract before changing anything:

```
Additional properties are not allowed ('node_id' was unexpected)
```

That commit did carry `exception` into `TranscriptDelegated` and `SubagentDeliveredEvent` — two of the three places this status lives. The third is the one no test covers.

**It does not crash today**, and I want to be exact about that: `SubscriptionEmitter.emit` pushes a raw dict with no validation, and the page reads only the fields it knows. But the contract test's own docstring says what the cost is — *"both sides declare `additionalProperties: false` / `extra="forbid"`, so a consumer validating against the schema rejects the whole event, not just the unknown field"*. Losing this event loses the boundary that advances the workspace turn counter and draws the delegated row.

## What this does

- `TurnStartedEvent` joins the `TurnEvent` union, in `openrpc.json` and in `raven/rpc/models.py`, so the existing sweep covers it from now on.
- The schema's `delegated.status` gains `exception`; `node_id` is declared.
- `TurnStartedPayload` gains `delegated` as a typed model (`TurnStartedDelegated`), three-valued status included.
- Both generated TypeScript clients regenerated.
- A new test drives **the exact marker `manager.py` builds** through both validators, so the next status or field fails here instead of in a client nobody is watching.

## One thing the union surfaced

`ui-tui` never handled `turn.started`. Its exhaustiveness check said so the moment the variant existed. It draws the delegated row from `subagent.delivered`, which the page deliberately ignores in favour of this event — the two mark different moments: **submitted** versus **started**.

Left as it is, with an explicit `case 'turn.started'` saying why. Moving that surface onto the truer moment changes when the TUI draws the row, which is a product change, not a contract fix.

## Verification

- New test red before the fix, by construction: adding the variant to the union alone reds `test_schema_match_turn_event_discriminated_union` and `..._payload_fields` (schema/pydantic drift), which is how the gap was proved.
- Mutation checks, each reverting one thing:

| reverted | reds |
|---|---|
| `exception` out of the schema enum | 1 |
| `node_id` out of the schema | 1 |
| `delegated` off the Pydantic payload | 2 |

- `uv run pytest -q` — **12900 passed**, 50 skipped, 5 failed; the same five pre-existing on `origin/main` (four `test_security_network.py` NAT64 cases, one `test_doctor_answers_where_the_memories_are`), reproduced on an untouched merge-base worktree.
- `uv run ruff check` / `ruff format --check` on `raven tests scripts` — both clean (both halves of `make lint-python`).
- `ui`: `gen:check` 147 methods, `type-check` clean, `npx vitest run` 89 files / **1225 passed**.
- `ui-tui`: `type-check` clean, `lint:rpc` in sync, `lint:i18n` up to date, `npm test` 138 files / **1990 passed**.
  - Worth recording: with `ui-tui/node_modules` symlinked from another checkout the suite reports 82 failed / 59 failed — and reports **the same numbers on an untouched `origin/main`**. It is the symlink, not the branch. Ran `npm ci` in the worktree to get a trustworthy number.
- `scripts/commit_lint` — no errors.
- `git ls-files | grep -c node_modules` — 0.

## Type

- [x] Fix

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear

Widening a contract to match what the runtime already emits: every payload valid before is valid now. No emit site changes, no behaviour changes. The one code change outside the schema and the models is an explicit no-op branch in `ui-tui`. Rollback is a revert.
2026-08-28 12:51:10 +00:00
forrestjgq
8ecfae3e0c test(tracing): isolate span emission into per-test tmp dirs (#371)
## Summary

Unit tests were leaking tracing spans into the real `~/.raven/traces`
store. Tracing is on by default and `trace.span()` calls are embedded in
library code (agent loop, subagents, TUI RPC), so any test exercising
those paths emitted spans with fabricated session keys (`session-a`,
`weixin:c`, `tui:default`, ...). Those spans then surfaced as phantom
sessions in `raven trajectory`, while the TUI `/sessions` list (which
reads the real session store) stayed clean.

This PR adds an autouse fixture in `tests/conftest.py` that:

- redirects `RAVEN_TRACING_DIR` to a per-test tmp directory, and
- resets the lazy module-level `TraceStore` cache
(`raven.tracing.spans._store`) around each test, since it pins the
directory resolved at first emit.

Key decisions:

- Only the directory is redirected, not the enabled switch, so the suite
keeps exercising the real emission path with zero behavior change; leaks
land in tmp instead.
- This centralizes the setenv-plus-cache-reset pattern that 8
tracing/trajectory test files already perform manually. Those tests keep
working: their own `monkeypatch.setenv` runs after the fixture and wins,
and they already reset the cache themselves.
- Existing phantom sessions come from archived span logs written before
this fix; the fixture only prevents new leaks. One archive file mixes a
real session with test keys, so cleanup needs line-level filtering and
is left out of scope.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Test-suite hygiene (`test` commit type).

## Verification

- `uv run pytest tests/test_tracing_api.py
tests/test_trajectory_store.py tests/test_trajectory_bundle.py
tests/test_trajectory_replay.py tests/test_trajectory_redact.py
tests/test_cli_tracing_commands.py tests/test_cli_trajectory_commands.py
tests/test_cli_trajectory_browse.py -q`: 308 passed, 1 failed
(`test_viewer_health_true_for_our_viewer`, fails identically on
unmodified `main`, environment-specific)
- `uv run pytest tests/test_agent_loop_run_emit.py
tests/test_subagent_manager.py tests/test_tui_rpc_turn_send.py
tests/test_deep_research_tool.py tests/test_agent_loop_approval.py -q`:
109 passed; the real `~/.raven/traces` log did not grow during the run
- Positive check: running `tests/test_agent_loop_run_emit.py` with
`--basetemp` shows the previously leaked session keys (`cli:c`, `tg:c`,
`weixin:c`) now land in per-test tmp dirs
- `uv run pytest tests/ --ignore=tests/integration -q`: 6837 passed, 11
failed, 20 errors; the same failure set reproduces on unmodified `main`
(environment-specific: cron/tz, everos server, theme, config loader,
viewer health), zero regressions from this change
- `uv run ruff check tests/conftest.py` and `uv run ruff format --check
tests/conftest.py`: both pass

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Docs box left blank: test-only change, no user-facing docs affected.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Test-only change; no production code touched. One known interaction:
`RAVEN_TRACING_DIR` takes precedence over `RAVEN_HOME`, so a future test
that fakes `RAVEN_HOME` and expects traces under it must also set
`RAVEN_TRACING_DIR` itself. Rollback: revert the single commit.

## Related Issues

N/A

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-28 19:21:16 +08:00
arelchan
7c33c4133a fix(rpc): let the viewer open the files a turn actually wrote
## Summary

A delivered artifact could not be opened from the page: the file viewer answered
403 and the transcript drew "Raven is confined to the workspace, so this file
cannot be opened here" over a delivery card it had just rendered.

The viewer's state-directory fence exempts the workspace subtree, on the stated
grounds that the default workspace lives inside raven's home and denying it
would refuse every file the agent itself wrote. Since agent home and the working
directory were split apart, a turn no longer writes there: it writes in
`~/.raven/tmp/<channel>`, a sibling of the workspace, inside the same state root
and outside the carve-out. So the fence denied exactly what the carve-out exists
to allow, one directory over, for every artifact produced since that split.

This exempts the working-directory root alongside the workspace, using the same
containment test in the same direction, so a working-directory root that happens
to sit above raven's home earns no exemption. `config.json`, `oauth/` and
`serve.json` are siblings of both and stay refused, which is what the fence is
for.

Worth noting for a reviewer: the bytes were already reachable. `/files/download`
serves a delivered file by token with no path fence, so this was never a
security boundary for deliveries -- only a viewer that refused to render what
the same page could already download.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_rpc_transport.py tests/test_rpc_files.py tests/test_rpc_knowledge.py tests/test_rpc_console.py -q` -> 123 passed
- `uv run ruff check` and `uv run ruff format --check` on both changed files -> clean
- Both new assertions were mutation-checked, each against its own mutation:
  - narrowing the carve-out back to the workspace alone reds
    `test_the_turn_working_directory_is_exempt_from_the_state_dir_fence` only (1 failed, 3 passed)
  - dropping the containment-direction guard reds
    `test_a_working_directory_root_that_contains_the_state_root_earns_no_exemption`
    and the pre-existing state-dir test (2 failed, 2 passed)
- Against the real reporting case: the two files that were answering 403
  (`~/.raven/tmp/tui/*.md`, `*.pptx`) now resolve, while `~/.raven/config.json`
  and `~/.raven/serve.json` still raise PermissionError.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- Widens what the file viewer will render by exactly one subtree: the directory
  turns work in. Everything the fence was written to protect (provider keys in
  `config.json`, the OAuth token store, `serve.json`) sits beside that subtree,
  not inside it, and stays refused -- pinned by the tests above.
- `fs.list` / `fs.reveal` share the function but pass the session's working
  directory as the root, which their own containment check already bounds, so
  their behaviour is unchanged.
- Rollback is a one-line revert of the exemption tuple.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-28 04:57:29 +00:00
LivXue
4e35a5d627 fix(tui): show the copy-on-select confirmation as a transient notice (#366)
## Summary

PR #360 reports every copy-on-select write as a permanent transcript
line, so a
reading session that drags through the transcript piles up "copied N
characters" rows under the conversation. The confirmation splits by what
the
line is for: the unbounded terse repeats become a transient notice
rendered
above the input box -- one line that replaces itself on every new copy
and
clears after 3 seconds -- while the first copy of a session keeps its
path
caveat in the transcript. The caveat is the answer a user comes looking
for
after a paste comes up empty minutes later, and it cannot pile up
because it
is once per session by construction.

The reporter still owns the per-session caveat tally and now returns
`{ text, firstOfSession }` so the caller can route the caveat to the
transcript and the repeats to the notice. A new `copyNoticeStore` holds
the
single notice; `showCopyNotice` keeps one dismiss timer, clearing any
pending
one before arming the next, so an earlier deadline can never clear a
newer
notice. `/copy` is deliberately untouched: an explicit command's answer
still
belongs in the transcript, and it cannot stack on its own.

## Type

- [x] Fix

## Verification

Run from `ui-tui/` at branch head, in a fresh worktree of `github/main`.
Dependencies came from `npm ci` in `ui-tui/` and in the hermes package;
the
hermes package's own `node_modules` holds only esbuild, so the ink
render
tests run against one react copy (two copies break every hook call).

```
npx vitest run --no-file-parallelism
  Test Files  93 passed (93)
  Tests  1084 passed | 13 skipped (1097)

npm run type-check
  clean, exit 0

npm run lint
  22 problems (0 errors, 22 warnings)

npx prettier --check src/app/copyNoticeStore.ts src/__tests__/copyNotice.test.tsx
  src/__tests__/clipboard.test.ts src/app/useMainApp.ts
  src/components/appLayout.tsx src/lib/clipboard.ts
  All matched files use Prettier code style!

python3 -m scripts.check_commit_messages github/main..HEAD
  exit 0
```

The 22 lint warnings are the baseline, not new: 22 before the branch and
22
after, none in a file this branch touches. The suite is run with
`--no-file-parallelism` deliberately: the ink render tests fail under
default
worker parallelism at this suite size, and no CI job runs them.

Suites that exercise the changed code directly:
`src/__tests__/copyNotice.test.tsx`
(7 cases), `src/__tests__/clipboard.test.ts` (30 cases, the 3 reporter
cases
updated to the new return shape).

New tests were proved load-bearing rather than assumed:

- Removing the notice render line in `appLayout.tsx` makes the render
test
  fail (content and placement above the input box).
- Removing the pending-timer clear in `showCopyNotice` makes the
re-shown-notice test fail: the earlier 3s deadline then clears the newer
  notice.
- The dismissed-notice case now shows a notice first, and removing
  `dismissCopyNotice`'s atom clear turns it red (review fix).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No doc change. `ui-tui/CONTEXT.md` has no clipboard or selection entry,
and
"copy notice" continues the existing "notice" vocabulary of
`copyResultNotice`
/ `copyOnSelectNotice`, so nothing new is coined.

## Risk

Behaviour change: the per-drag copy confirmation is now transient and
above
the input box instead of a permanent transcript line, and the first copy
of a
session lands one transcript line, exactly as on `main`. Reverting the
branch
restores the #360 behaviour; no data or on-disk format is involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (deepseek-v4-pro) <noreply@anthropic.com>
2026-08-25 14:27:24 +08:00
LivXue
22eaf7d370 feat(cli): prune shadowing subagent rows in onboard
## Summary

The onboard sub-agent step stopped registering agents: vendored discovery materializes a row per folder on every table build, and a stored row an older install.py wrote outranks that discovered row. After a `git pull` updates a folder's manifest, the stale row still wins, so the update never reaches the roster - this is exactly what broke raven-research when its transport switched from cli to acp.

Running `raven onboard` (interactive) now prunes: any config row whose name matches a vendored folder manifest is listed, a disabled row is flagged (removing it would re-enable the agent), the step asks for confirmation, and only then deletes the rows after backing the full list up. A decline keeps everything. Non-interactive runs still skip the whole step.

Deliberate decisions:

- Deletion is by name only, whether or not the row's content still matches the manifest: a matching row outranks the discovered one either way. This mirrors `subagents/install.sh --prune-stale`, as does the backup convention (`subagents-backup-<timestamp>.json` beside the tree, one file per prune run, a few KB of the subagent list).
- A same-name config row is also how a user edits a vendored agent, so nothing is deleted unasked: the confirm gate is the consent, mirroring how `install.sh` gates the identical deletion behind its `--prune-stale` flag.
- The backup is private from the first byte (mode 600, truncated on a same-second name collision) because the list can hold an openai row's api key, and its path is announced before any deletion, because a removal that fails partway is exactly when the reader needs it.
- A backup that cannot be written stops the prune with a warning; a removal that fails because a surviving row violates the schema is reported as a warning instead of crashing the wizard; an unreadable config is a warning too (`ConfigReadError`).
- The prune does not hot-apply to a running raven: a live process keeps the roster it read at startup, so a restart is still required after onboard (same as after install.sh). Hot-apply was considered and left out of scope.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_cli_subagent_setup.py` - 44 passed
- `uv run pytest tests/test_cli_subagent_setup.py tests/test_cli_onboard_commands.py tests/test_subagent_vendored_agents.py` - 346 passed, 1 skipped (chmod 000 does not block root)
- `uv run ruff check raven/cli/subagent_setup.py tests/test_cli_subagent_setup.py` - all checks passed
- `uv run ruff format --check raven/cli/subagent_setup.py tests/test_cli_subagent_setup.py` - clean
- `PYTHONPATH=. python3 scripts/check_commit_messages.py origin/main..HEAD` - clean
- Revert-the-fix check: with `raven/cli/subagent_setup.py` restored to origin/main, all 10 prune-related tests fail; with the branch, all pass
- Smoke test on a copy of a live config that was already pruned: helper is a no-op (0 removed, no backup written, config byte-identical)

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed (no UI change; the module docstring is updated)

## Risk

- [x] Security impact considered (backup may contain an api key; written 0600 from creation, never printed except its path)
- [x] Backward compatibility considered (deletion happens only on an explicit interactive onboard run, after a confirm that lists the rows; the previous list is recoverable from the backup)
- [x] Rollback path is clear for risky changes (the backup file holds the pre-prune list; a running raven keeps its in-memory roster until restart, so the prune cannot disturb a live process)

## Related Issues

N/A
2026-08-26 12:26:11 +00:00
gloryfromca
df9a7d7f37 chore(*): merge upstream v0.1.13 into the gitlab trunk
Absorbs the GitHub release into the downstream trunk. A merge rather than a
rebase so v0.1.13 becomes a real ancestor: the next sync starts from v0.1.13
instead of v0.1.12, and the eventual contribution back to GitHub can still tell
our commits from the upstream ones it already has.

Cost this round: 51 conflict hunks over 27 files, against 54 over 31 for
v0.1.12. The hunks were the cheap half. Upstream's #284 moves the model and
provider from process-wide state to a per-session binding, and the trunk had 230
commits standing on the old shape.

What git reported as a clean merge and was not
---------------------------------------------

Fourteen upstream-changed paths do not exist here, because we renamed or moved
them since v0.1.12. Rename detection found none of them, so their changes were
merged into paths nothing reads, silently:

* raven/tui_rpc/* -> raven/rpc/* (renamed in ce526ad5). Nine files, including
  question_broker.py, whose whole #354 rewrite was lost, and methods/config.py,
  methods/model.py, methods/session.py, models.py and errors.py. Each was
  replayed by three-way merge against dd82f52c. The three test modules were
  rebased function by function onto upstream's version instead, after a
  three-way merge spliced assertions into the wrong tests.
* raven/agent/tools/mcp.py -> raven/mcp/client.py. Carried the release-blocking
  fix, and is handled in the commit that follows this one.
* ui-tui/rpc-schema/openrpc.json -> rpc-schema/openrpc.json.

Decisions
---------

Follow upstream wherever the two disagree about provider and model:

* providers/pin.py is gone, so a bare model id is refused instead of derived.
  A model id does not name whose credential serves it, and the derivation
  walked a list -- with two vendors configured, which one pays came down to
  their order in it. rpc/methods/config.py now requires the provider.
* ModelSwitchInTurnError (-32009) is gone with it. A session switch lands on
  that session's next turn, so there is nothing to refuse mid-turn.
* AgentLoop.refresh_context_window and the LazyProvider on_built wiring are
  gone: the window rides on the binding now.
* The skill gate no longer forwards an unpaired pin.

Keep ours where the trunk is ahead, and graft upstream's behaviour into it:

* raven/agent/subagent/backends/raven_loop.py is the live sub-agent path, so
  manager.py's inline loop stays retired and the web_search gate from #312
  applies there.
* skill_hub/client.py keeps the public zip limits the skillhub RPC path imports,
  and takes upstream's query-string cap.
* ask_user keeps our JSON-string normalizers and feeds them into upstream's
  batch contract, so a batch validated by _prepare is one _normalize_questions
  has already coerced.
* The chat view composes both filters: hideIntroAfterFirstTurn drops the cover,
  visibleRows still hands the view to a direct chat.

Tests removed, not repaired
---------------------------

Eight tests covered mechanisms this merge deletes, and one of them hung rather
than failed:

* seven in test_agent_loop_model_switch.py over AgentLoop._pending_provider,
  the switch-parking mechanism #284 replaced with per-session bindings. Its
  replacement arrives with upstream as test_agent_loop_session_model.py.
* test_a_refusal_during_a_turn_keeps_its_own_code in test_acp_methods.py, whose
  error class no longer exists.

Baseline: the pipeline for 01d39e58, this merge's base, is green, so nothing
here can be charged to a pre-existing failure.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 16:12:43 +08:00
Handsome-wzw
a1d5b24de3 feat(agent): withhold web_search without a key, and make tool capabilities visible (#312)
## Summary

A WeChat user asked for a recommendation and the bot answered "the web
search
tool is not configured here". That sentence was not the model's own
phrasing --
it was the tool's error text, relayed outward. `web_search` needs a
Serper key
and that deployment has never had one, but the tool was registered
unconditionally, so on every search-shaped question the model saw it,
reached
for it, and passed the setup error on to whoever was in the chat.

Withholding the tool fixes that and creates a second problem: an
unregistered
tool is invisible. Nothing in a running Raven then says the capability
exists at
all -- the model is never offered it, no document lists it, and `raven
doctor`
reports on providers and memory but has never mentioned tools. The only
way to
learn that web search is one account and one edit away was to read the
source.

This does both halves, because either alone leaves a real gap: the
deployer is
told what this install can and cannot do, and the model is still not
offered a
tool it cannot run.

### Withholding the unusable tool

`web_search` is gated on a resolved key, in the main loop and in the
sub-agent
surface (`subagent/manager.py`) -- a sub-agent that reaches for a search
it
cannot run reports the failure to its caller, and that text lands in the
parent
turn, the same leak one level down.

The gate asks the tool, not the config. `WebSearchTool.api_key` resolves
at call
time from the constructor value *or* `SERPER_API_KEY`, so reading
`tools.web.search.apiKey` alone would withdraw a working tool from any
deploy
that exports the variable and configures nothing.

The tool's error message also hard-coded `~/.raven/config.json` while
the
gateway runs with `--config` elsewhere, so following it meant editing a
file the
process never reads. It names the path actually in force now. That text
is
reachable only if the key disappears after registration, which is
exactly why it
should be right: it is the message for the case the gate cannot cover.

### Three rules, and nowhere to read them

Registration is decided per family, each a different shape:

| family | rule |
|---|---|
| `web_search` | a resolved key, asked of the built tool |
| `web_fetch` | nothing -- always registered, a key only improves
extraction |
| media x3 | an `api_key` *or* a `model`, either counting as configured
|

For media those are two questions, not one. A section naming only a
model is
registered, because a model alone counts as asking for the tool, and
then every
call returns a missing-key error. Whether a capability is *offered* and
whether
it *works* come apart there, and a report treating them as one fact
ticks a
capability that cannot run.

Each rule is defensible where it sits. What is missing is anywhere to
read them.
Providers had the same sprawl once and answered it with
`providers.auth`: a
declarative table plus `credential_status` as the single authority, with
an AST
invariant enforcing that authority, because six surfaces had answered
the same
question six ways and each looked reasonable alone. Tools never got the
equivalent.

### What this adds

**Each tool answers for itself,** twice where the questions differ.
`WebSearchTool.is_configured` reads the config value *or*
`SERPER_API_KEY`,
because those are two sources and only the tool consults both. The media
base
answers `is_configured` on a model *or* a key -- which is what stops an
OpenRouter credential set for chat from silently switching on three
tools that
bill per call -- and answers `has_key` separately on the chain it
actually
resolves at call time: its own section, the borrowed provider key, then
`OPENROUTER_API_KEY`. Both rules live with the credential they read.

**`capabilities.py` describes the five** for a human deciding what to
set up:
what each does in one line, how much work it is (nothing / reuse a
credential
you already have / obtain an account), where the key goes, where to get
one, and
what it costs. It rules on nothing -- `is_configured` and
`has_credential` both
ask the tools -- so it cannot become a second opinion.

**`raven doctor` grows a section**, listing every capability configured
or not,
ordered by how much the deployer has to do:

```
(markers render as a green check and a yellow bang; spelled [ok] and [!] here to
keep this description ASCII, since it becomes the squash commit body)

Tool capabilities
  web_fetch:       [ok] Read a web page the agent already has the URL for
  image_generate:  [ok] Generate an image  (borrowed: providers.openrouter.apiKey)
  text_to_speech:  [!] Generate speech from text
                   no key resolves; calls will fail
                   set: tools.media.speech.apiKey
                   or env: OPENROUTER_API_KEY
  video_generate:  -  Generate a video
                   switch on: tools.media.video.model
                   key: reusing providers.openrouter.apiKey
                   Billed per call; needs prepaid OpenRouter credit.
  web_search:      -  Search the web
                   set: tools.web.search.apiKey
                   or env: SERPER_API_KEY
                   key from: https://serper.dev
  2 capability(s) available but not set up; the agent is not offered them.
```

Three deliberate details.

Naming the credential is load-bearing in both directions. A row that
cannot
distinguish a reused credential from a missing one sends someone to
create an
account they already have -- and a row that claims a reuse with nothing
to reuse
is worse, because acting on it means setting a model, getting a
registered tool,
and watching every call fail on a credential they were told they had. So
the
reuse line prints only when a key is genuinely there to pick up, and a
capability already in that broken state says so outright instead of
showing a
satisfied tick.

The credential is named at the path that holds it. For the media family
`config_path` names the *model*, so reusing it as the key source pointed
the
deployer at a line with no credential in it.

And each fact is on its own line rather than in a sentence, because the
terminal
wraps a long line mid-path and a config key broken across two rows
cannot be
copied, which is the only thing that row is for.

Nothing here moves the exit code, including the warned row. An install
without
image generation is a choice, not a fault, and a doctor that fails on it
teaches
people to ignore doctor. The half-finished one is arguable -- the memory
section
does exit non-zero for a role the user configured that the server could
not
build -- but that failure is silent where it happens, recall just
returning
nothing, whereas this one returns an error string to the model on every
call.
Say the word and it becomes an exit code instead.

### Not in this change

Registration still lives in `AgentLoop`. Having it read the table is the
point
of this shape and removes the last duplicate reader, but it edits
`agent/loop/main.py`, which is under active change, and it is worth
doing on its
own once that settles. Until then the table is a description, and the
tests
below are what keep it honest.

`deep_research` is deliberately absent: it is moving to the sub-agent
surface
and its tool is going away.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

Mixed on purpose: the first commit is a fix and the rest are the feature
it
made necessary. Splitting them would land a change that hides a
capability
without landing the one that makes it discoverable.

## Verification

```
make lint                                            exit 0
make build                                           exit 0
npm run test --prefix ui-tui                         969 passed, 13 skipped
uv run --all-extras pytest -q                        6186 passed, 35 skipped, 1 failed
uv run pytest tests/test_tool_capabilities.py \
              tests/test_cli_doctor_commands.py \
              tests/test_agent_loop_web_tools.py     64 passed
uv run --extra dev ruff check raven tests scripts    All checks passed
uv run --extra dev ruff format --check               828 files already formatted
npx commitlint --from origin/main --to HEAD          exit 0
scripts/check_commit_messages.py origin/main..HEAD   exit 0
```

The one failure is
`test_read_file_image.py::test_an_attachment_that_cannot_be_
read_costs_a_note_not_the_turn`, and it is not from this branch: it
fails the
same way on `main` at `1cb604a` with these commits absent. The case
makes a file
unreadable with `chmod 000`, which does not block a root user, so it
fails for
anyone running the suite as root and passes in CI. This branch does not
touch
that file or the code under it.

The capability tests drive a real `AgentLoop` and compare what it
registered
against what the table predicts, rather than asserting the table against
itself.
The gated set is derived -- tools present once credentials are supplied,
absent
without -- so a sixth gated tool whose author forgets the table fails
here rather
than going unnoticed.

That last claim was false when first written, and the last commit is
what makes
it true. The fixture listing the media tools by hand never switched a
fourth one
on, so a new one never joined the gated set and the assertion held over
an
already-incomplete table. #305, which adds a MiniMax voice-clone tool,
is that
case: merged against this branch the assertion passed. Read from
`MediaGenConfig` instead, it fails and names the tool -- `gated but
undeclared:
['voice_clone']`.

Which means whichever of the two lands second turns this red,
deliberately. The
fix is one table entry, and it needs the rule that PR settles:
`voice_clone`
counts as configured on `api_key or api_base or model` in
`effective_media_config` while registration still gates on `api_key or
model`,
so an `apiBase`-only install with no MiniMax key is configured by one
rule and
withheld by the other. That is the divergence this table exists to make
visible,
and it is worth resolving there rather than papering over here.

Fifteen mutations, each caught:

```
the web_search gate removed from the main loop    3 failed
the same gate removed from the sub-agent surface  1 failed
media rule reads only the key                     7 failed
web_search rule reads only the config             1 failed
a tool removed from the table                     6 failed
the doctor section not rendered                   6 failed
doctor lists only configured tools                4 failed
unconfigured treated as a failure                17 failed
the reuse line is printed unconditionally         1 failed
the key source falls back to the model path       3 failed
a keyless registered capability is not flagged    1 failed
has_key always answers yes                        1 failed
has_credential collapses into is_configured       1 failed
the credential chain drops OPENROUTER_API_KEY     3 failed
the reuse check ignores the environment           3 failed
```

Four of those tests exist only because a mutation pass found the earlier
versions insufficient, and the last rounds are why the shape changed. A
media
case that sets both a model and an OpenRouter key proves nothing about
the "or
model" half: the borrow fills the key in, so a rule reading only the key
still
answers correctly. The case that pins it has nothing to borrow.

The sub-agent case is the same argument applied to the second call site:
with
the gate present only in the main loop, every test above still passes,
because
nothing was watching what the sub-agent surface registers.

More usefully, mutating `has_key` to always answer yes changed no test
at all in
the first version, which said the ruling was not load-bearing: the
doctor was
inferring "no credential" from an *empty source string* rather than from
the
tool's answer, so a fourth credential source would have been reported as
a
missing one. `has_credential` is now its own fact, asked of the tools,
and that
mutation fails.

Separately, autouse fixtures clear `SERPER_API_KEY` and
`OPENROUTER_API_KEY`,
without which several of these pass for the wrong reason on any machine
where a
developer exported one.

Rendering was checked against a real config in six states -- nothing
set, a
provider key present, a model with nothing to borrow, a tool with its
own key, a
model plus a borrowable key, and an exported variable as the only source
--
rather than only asserted on.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

One behaviour change: `web_search` is no longer offered to the model
when no key
resolves. That is the fix. A deployment that has a key, in config or in
the
environment, is unaffected; one that has none was getting an error
string in
place of an answer.

The rest is additive. No other registration logic changed, so which
tools an
agent is offered is otherwise exactly what it was; `is_configured` moved
the
existing predicates onto the tools without altering them, and the
mutation
results above are what pins that. `has_key` and `_resolve_key` are new
names for
the chain `api_key` already resolved -- the property calls the extracted
one, so
callers see the same answers.

`raven doctor` gains a section and no new exit code. Its zero-network
guarantee
holds: the table reads config and environment only.

Two allowlist entries were added to
`test_only_the_auth_module_decides_configuredness_from_a_key`, argued in
place:
`subagent/manager.py`, which asks the built tool whether a key resolved
so an
unusable search is withheld, and `capabilities.py`, which reads keys to
*report*
-- which source supplied one, and whether one is there to reuse. Both
ruling
halves are delegated to the tools, whose files were already listed.

Rollback is a revert. Nothing is written and no configuration is read
differently.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 10:05:58 +08:00
arelchan
82eea1dd20 fix(ui): show Raven logo as page favicon
## Summary

- Declare the existing Raven application logo as the served page favicon.
- Copy the logo into the packaged page assets so installed builds render the same branding.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

- `npm test --prefix ui` - 32 test files and 528 tests passed.
- `npm run type-check --prefix ui` - passed.
- `npm run build --prefix ui` - passed.
- `python3 ui/build.py` - built the page and copied the Raven logo asset.
- `node ui/scripts/check-page.mjs` - passed.
- `make check-large-files` - passed.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Low risk. The change only adds favicon metadata and packages an existing application asset. Reverting the two source changes restores the prior browser-default icon.

## Related Issues

N/A
2026-08-23 11:11:06 +00:00
gloryfromca
56858acbce fix(*): report an invalid memory identity as the config error it is (#350)
## Summary

An identity EverOS cannot accept becomes a directory segment on its
write path, so `EverosBackend.start()` rejects it. The rejection carried
the one message a user can act on -- it names the key and the accepted
pattern -- and no surface delivered it as one. Three of the four call
sites fold the raise into `logger.exception`; under the TUI loguru is
file-only, so there an invalid identity turned memory off with nothing
on screen. The fourth, in `import_commands._build_and_run`, is not
wrapped at all, so the `ValueError` escaped as a bare traceback. Where
it was visible it was a traceback, which is not a thing a user reads as
"edit this key".

The state machine then told the user the opposite of the truth.
`start()` died before assigning a state, so `_state` stayed `UNKNOWN`,
which is not in `_NEVER_HAD_MEMORY`; every `store()` after it counted a
dropped write, and the session ended with:

```
1 turn(s) were not written to long-term memory because the memory service was
unavailable.
```

The service was fine. That sends the user to debug a server that never
broke.

**The change.** Catch the `ValueError` in `start()`, give it a
`BAD_IDENTITY` state, and file that state with the other two the user
must fix themselves:

- terminal, so a stray probe cannot promote it to `READY`;
- never-had-memory, so no dropped-write count and no outage wording;
- the message goes to stderr, matching what the unconfigured-LLM and
missing-binary paths beside it already do.

Two things came out of running it rather than reading it:

1. **The message needs escaping.** It quotes the accepted-character
class, and rich reads `[a-zA-Z0-9_.@+-]` as a markup tag and eats it,
leaving the user the pattern `^+$` to match their id against. The test
now pins the full class.
2. **`raven import` had the same defect on its own gate.** It kept the
outage wording and pointed at the server log, which holds nothing about
a config error. It now names the config keys instead.

Before / after, same config (`memory.userId: "bad/id"`), same command:

```
- 1 turn(s) were not written to long-term memory because the memory service was
- unavailable.

+ Long-term memory is off: memory.userId='bad/id' is not accepted by EverOS: it
+ becomes a directory segment on the write path, so it must match
+ ^[a-zA-Z0-9_.@+-]+$ and must not be '.' or '..'.
+ Fix memory.userId / memory.agentId in your config.json, then start a new session.
```

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Re-run after rebasing onto `d35e398c`, the current `main`.

```
uv run pytest tests/test_everos_backend.py tests/test_cli_import_commands.py \
              tests/test_everos_plugin_discovery.py -q
162 passed

uv run pytest -q
1 failed, 6478 passed, 43 skipped, 13 deselected

uv run ruff check <the four touched files>            All checks passed
uv run ruff format --check <the four touched files>   4 files already formatted
```

The one failure is
`tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare`. It
is pre-existing on this base: checked out `d35e398c` in a clean worktree
and ran `uv run pytest tests/test_cli_theme.py` there, same failure, 1
failed / 44 passed. It passes in isolation and fails when the file runs
as a whole, so it is order-dependent, and it is unrelated to anything
here. That file is untouched by this change.

Manual check, sandbox `RAVEN_HOME`, `memory.userId: "bad/id"`, `raven
agent -m "hi"`: prints the block quoted above, and the session no longer
ends with the dropped-write line.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Terminal-only wording plus one new enum member. No config, schema, or
protocol change; an identity that was accepted before is accepted now,
on the same code path. The only behaviour change for a valid setup is
none.

For an invalid identity the turn behaves as it did -- memory off, turn
completes -- and only what the user is told changes. Rollback is
reverting the single commit.

`ServiceState.BAD_IDENTITY` is new. Every consumer was checked:
`raven/cli/import_commands.py` (handled here) and
`tests/test_everos_plugin_discovery.py` (reads `READY` only).
`_may_spawn` allowlists `UNKNOWN` via `_SPAWNABLE_STATES`, so a new
member is non-spawnable without a change.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #271

Co-authored-by: zhanghui <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-08-20 14:41:45 +08:00
arelchan
bbd5b6f1b5 feat(subagent): discover the vendored raven builds and install them from the page
## Summary

The four raven builds under `subagents/` were reachable only by running each folder's `install.py` by hand, while four rows the package hardcoded -- `research-raven`, `code-raven`, `data-raven`, `content-raven` -- sat on the agent table looking like agents. They were not. The backend factory reads four fields off a built-in row (`model`, `restrict_to_workspace`, `tools`, `skills`) and all four were unset on all four rows, so every one of them dispatched the same provider, the same model, the same tool set and the same system prompt; the sub-agent was never even told which name it ran under. What they had was four descriptions on the roster, which is why the dispatching model wrote `research-raven` on a node -- it was picking off a menu, not from the work. Per-step differentiation was always in `promptTemplate`, and it still is.

So the labels are gone and the real builds take their place, discovered rather than written: a folder shipping a manifest and an installer becomes a `cli` row on every table build, and deleting the folder removes the row. Composition happens inside `AgentRegistry.apply` for the reason `enabled_agents` documents about its own filter -- six paths hand it a config list, and composing at the boundary would be six places to keep in step.

Three failure modes this had to be built around, each of which would have been silent:

- a folder's venv lives inside its own checkout, and a wheel install is replaced wholesale on upgrade, so a tree under site-packages loses every venv each time. The tree is installed out to the raven home under a version stamp; the stamp is also what makes deleting a folder stick, since a same-version start is a no-op.
- the launchers inherit the host raven's whole provider block when a folder has no key of its own, so gating readiness on the folder's own key would have disabled every row on a correctly configured machine.
- `install.py` bakes an absolute path into the row it writes, and a config row otherwise beats a discovered one, so a stored row would keep naming a launcher an upgrade had deleted.

An unready folder is listed and disabled rather than dropped: `venv_ready` already records why a name the model can pick and then fail on is worse than no name, and dropping it would hide the folder from the operations view that exists to show "present, not set up". Onboarding now offers to build one instead of only mentioning that it is unbuilt, and no longer registers anything -- the row `install.py` used to write is the stale-path hazard above.

The beta wheel carries the tree through a force-include added to the same temporary pyproject rewrite that stamps the version, exported from git rather than copied from the working tree: force-include bypasses hatch's exclusion patterns, and a directory reference would ship four venvs and a `.env` holding a live credential. The GitHub release builds from the committed pyproject, so a stable release is unaffected -- no size change and no extra rows for anyone who did not opt into beta.

Eight further defects were found by opening the page rather than by reading the diff, and are the second and third commits:

- `subagents.list` assembled its own view and never asked the discovery layer, so the four folders were absent from the one screen a human reads to answer "what can it dispatch to". On the screen built to say they installed, that reads as "they did not".
- all four then reported "installed, ready" while three could not start: the probe launches nothing and checks the command's first token, which for one of these is the interpreter and always exists. The readiness verdict now overrides the probe and carries the reason.
- the rows landed in the "connect one" section behind a Connect button they cannot honour, having no preset to install from and no row to write.
- adding the flag that fixes that took four edits, because one row shape is declared in four places: the OpenRPC contract, the pydantic model that validates the response with `extra=forbid`, the island's TypeScript type, and the live data source -- which is a whitelist, so a field it does not name is a field the page never sees.
- the switch on a discovered row answered `subagent_not_found`, because `subagents.toggle` creates an override row for a built-in name and refuses anything else.
- `"enabled": false` in a folder's `subagent.json` did nothing: the readiness verdict overwrote it, so the manifest carried a field the code accepted and ignored.

The fourth commit adds the action the section existed to make reachable. A folder that is not built has its code and lacks its dependencies, and the only way to supply them was a terminal -- the card said "run subagents/install.sh raven-ppt" and left, which most readers do not do, so four agents that shipped read as four that are broken. `subagents.build` runs the tree's own installer for one folder, returns as soon as it is under way (one measures 627 MB and several minutes, so holding the request open would time it out and leave the build unobservable), and the row carries `building` until it finishes. Two things are read back rather than trusted: success is `venv_ready` on the filesystem rather than the installer's exit status, and a build in flight outranks the reason it is fixing so the card stops saying "go run this" for the minutes it takes. Delegated to `install.sh` rather than calling `uv sync` here because that script knows each folder's optional-dependency extra -- `raven-ppt` really does resolve as `uv sync --extra ppt`.

Stored playbooks naming one of the retired four are repointed at `raven`, by name and not by the `-raven` suffix. Left alone they would fail validation, and the failure is invisible: the runtime warns and the playbook drops out of the library.

## Type

- [x] Feature
- [ ] Fix
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Ran on the branch rebased onto `origin/main`:

- `uv run --frozen pytest tests/` -- 9557 passed, 9 failed. The nine are environment-specific and fail identically on a clean tree (verified by stashing): `test_cli_update_notice` reaches the release registry, `test_start_webapp` needs bash 4 and this host has 3.2.
- `make coverage && make coverage-diff` -- diff coverage passed the 90% threshold. `make coverage-ratchet` -- line 79.38% -> 80.11% (+0.73pp), branch 69.17% -> 69.95% (+0.79pp).
- `uv run --frozen ruff check` and `ruff format --check` -- clean.
- `make lint-ui`, `make lint-tui` -- clean, including the `openrpc.json` -> `generated.ts` and `messages.json` -> catalogue sync checks (both regenerated in this branch).
- `npm run test --prefix ui` 373 passed, `npm test --prefix ui-tui` 1301 passed.
- `node ui/scripts/check-page.mjs`, `count-shared-globals.mjs` (24 of a ceiling of 24), `check-css.mjs` -- clean.
- `make lint-bridge` was not run: this host has no `tsc` in `bridge/node_modules`. `bridge/` is untouched.
- 20 mutation checks on the new logic (readiness gate, placeholder resolution, version stamp, host-key inheritance, installer-presence test, refresh-preserves-venv, stale-row guard both ways, playbook migration both ways, the page's section split both ways, the manifest's declared `enabled`, the switch variant both ways, build-verdict-from-exit-status, the one-build-at-a-time guard, the stale reason during a build, the failure reason reaching the row, the `building` flag reaching the row). All 20 were caught by the tests.
- End to end on the served page, twice. With one folder's venv built by hand: the row flips to enabled, reaches the dag tool's `subagent` enum and the roster the model reads, tagged `no-progress`; the other three list with the exact command that would build them. Deleting a folder drops its row on the next table build. And through the page's own Install button on a second folder: `uv sync --extra ppt` ran for real, the button held "Installing..." and disabled, the row flipped to ready on its own when the build finished, and the name then appeared in the graph tool's `subagent` enum. An actual dispatch to a vendored agent was not run -- it spends the host's LLM budget -- so verification stops at backend construction plus the launcher, interpreter and venv paths all existing.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes:

- the four label agents are gone from the roster. A dag node or playbook naming one is migrated to `raven` on load; a hand-typed name is refused by the tool's enum, as any unknown name is. Nothing that ran before runs differently, because all five rows dispatched identically.
- the roster the model reads shrinks from ~299 tokens to ~61 on every request that carries the spawn or graph tool.
- an install that has the `subagents/` tree gains up to four rows, disabled until each folder's venv is built. A wheel without the tree gets a byte-identical table to before.
- `subagents.list` grows `vendored` and `building` booleans, and `subagents.build` is a new method. Both fields are optional in the contract, so a client that predates them is unaffected.

Rollback: revert the four commits. The discovery layer writes no config, so there is no stored state to undo. The one thing it does write is a copy of the packaged tree under `~/.raven/subagents/` with a `.raven-version` stamp; deleting that directory is a complete rollback of it.

Not covered here, and deliberately: the beta wheel does not build the venvs (`.venv` is platform-specific and absolute-path bound) and does not ship credentials, so a fresh beta install shows the four rows disabled until something builds them -- either `raven onboard`, which now offers to, or the page's own Install button. Neither happens by itself: one venv measures 543 MB and 320 packages, which is not something to do silently at install time.

## Related Issues

N/A
2026-08-21 02:23:10 +00:00
arelchan
feacba682c refactor(ui): the session rail becomes a react island
## Summary

The session rail (left sidebar) becomes a React island: `ui/src/features/rail/` with types, a plain external store (draw/skeleton/markNew/remove/rename, fold state on the `raven.gui.grpfold` localStorage key), the Row/Group/RailApp components with classes and DOM shape copied byte-for-byte from the legacy renderer, and 13 tests.

Key decisions:
- `SESS`/`cur` stay shared page globals: ten demo/live files write them (including whole-array rebinds), so the island owns no rows. It reads through `DS.sessions.snapshot()`, a closure over the rebindable bindings, re-read on every `drawList()`.
- Island actions go out through 9 new optional bridge verbs (`drawList, setCur, openSession, removeSession, renameTitle, dropDraft, pinPersist, openCron, navState`), each one a guarded one-liner in 155-bridge.js and late-bound so live rebinds keep winning.
- `plainTitle` moved into the bundle and is republished as `window.plainTitle` for its 6 legacy callers (same pattern as `cronExprHuman`).
- Shrink: demo/050-rail.js 222 -> 16 lines; live/010-boot-guard.js raw-DOM skeleton painter became one `RavenIslands.rail.skeleton()` call; live/030-sessions.js untouched (already renderer-free, its helpers are read by other domains).
- Pre-existing bug left as-is: live/080-overrides' rename persist wrapper queries `#title input` after the h1 swap and throws today; the island reproduces the demo DOM dance exactly rather than patching another domain in this MR.

Ratchet: 57 -> 57 (the strands touching this domain remain demo-declared/live-assigned by design; nothing new added).

Note: pipelines are currently red project-wide with `ci_quota_exceeded` (shared-runner minutes exhausted). Local gates below are the same commands CI runs; the pipeline should be retried once quota recovers.

Review round: the fake shell handed `markNew` an empty nav and a folded More group, so both of its branches were dead in every test -- exactly the part of the port that changed shape (`Object.keys(NAV_OF)` to `nav.pages`, `$('#' + p)` to `getElementById`, `MORE_ROWS[i][0]` to `nav.morePages[i]`). Two cases added with the real nav: a module page up marks its own rail button over the new-task row, and an open More group puts the mark on the row whose page is up while the parent gives it up and takes it back when folded. Both fail against a wrong port.

## Type

- [x] Refactor

## Verification

- `npm run --prefix ui type-check` clean
- `npm test --prefix ui` 49/49 (13 new rail tests)
- `npm run --prefix ui build` (223.95 kB IIFE) and `python3 ui/build.py` (1,094,616 bytes)
- `node ui/scripts/check-page.mjs` OK; `node ui/scripts/count-shared-globals.mjs` 57/57
- `uv run pytest tests/test_ui_language_repaint.py` 9/9
- Stub browser pass (`?stub=1`): DOM byte-compared to legacy (.grp/.sess shapes, aria-current moves, fold persistence, search, pin+persist, delete+undo toast, rename via top bar, running mark lifecycle, More flyout marks, zh/en repaint)
- Live pass against a throwaway gateway (temp RAVEN_HOME, dummy keys, nonce auth): boot shows exactly 6 skeleton rows with the legacy width formula while `data-counts="pending"` holds, then the real list swaps in; draft's first send creates a real session; two-session switching moves a single mark; second boot renders with no mock flash
- Not exercised live: a successful provider turn and parked done/err badges (needs a real key); covered by unit tests and the stub replay

## Risk

User-visible behavior is intended to be identical; the rail DOM shape and class names are unchanged. Rollback is reverting the single commit.

## Related Issues

N/A
2026-08-20 06:17:21 +00:00
arelchan
e32f4b6034 fix(*): keep migration notices off stdout, and stop planting defaults (#346)
## Summary

Three follow-ups from #341, which fixed a retired `contextWindowTokens:
65536` capping every upgraded install at 64k. Two are its loose ends;
the third is the reason it happened at all.

**The notice no longer prints on stdout.** #341 drains config-migration
notices in the `run()` wrapper so every command tells the user, which
caught the two whose stdout is a machine-readable document: `raven
doctor --json` and `raven import --json`, both advertised for
automation. Reproduced -- a config still carrying the retired pin makes
`doctor --json` emit JSON with a sentence after it, and `jq` refuses the
result. Moved to stderr, where the rest of this class of message already
goes (the `ConfigReadError` branch a few lines up in the same wrapper,
and the loader's malformed-config warning). That fixes any future
`--json` command without it having to know this exists, which
allow-listing the two would not. The window is narrow -- the migration
fires once per install -- but the run it can spoil is a first run after
upgrading, and a CI job is exactly the caller not watching.

**`raven doctor` gains a Config section and a `--fix`.** The premise
changed while building it: the migrations run at load, so by the time
this command looks, nothing is pending -- they already happened,
silently, one launch ago. What is left for a person to ask about is what
Raven deliberately will not decide for them. So it reports a
`contextWindowTokens` pinned below what the configured model holds, and
names what rides on it: the per-turn history budget, when the Curator
starts paying for a slow path, and when memory consolidation archives.
`--fix` removes the pin.

Reported without `--fix`, never written: that pin is a real
configuration for a local model or an endpoint served smaller than the
catalogue thinks, and the flag is the user saying which case theirs is.
The write carries the file mode across (config.json holds
`providers.*.apiKey`) and uses a per-PID temp name -- the same rules the
loader's migration writer follows.

One check rather than two. A config naming no provider is the other
thing worth asking about, but it reads differently on either side of the
explicit-provider rule in #284, so it belongs to that change; a model
that routes to nothing is already reported above it.

**The bootstrap stops writing every default into the user's config.**
This is the cause of the bug #341 cured. `save_config` dumped the whole
`Config()`, so a new install started life with eight kilobytes of
settings its owner had never heard of and, worse, with that day's
defaults frozen into their file -- after which a default we improve
never reaches anyone who already has one. `contextWindowTokens: 65536`
got onto disk exactly this way.

It now writes only what differs from the defaults. Lossless on reload,
since a value equal to its default reloads as that default. A fresh
config drops from 7,985 bytes to 2, and to 634 once the extension blocks
are seeded -- what is left is the handful of lines someone actually
chose, which is also a config they can read.

That third one is the only change here with a broad blast radius, and it
has one visible consequence, which is an improvement: the onboarding
picker used to lead with the schema's default model, because the
bootstrap had written it into the config and the wizard read it back as
"the current model" -- a value nobody had chosen, offered as though
somebody had. With nothing written, the wizard leads with the provider's
own recommendation instead. The test that pinned the old list now pins
the new one and says why.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
pytest tests/                       6650 passed, 1 failed, 33 skipped
                                    (tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare,
                                    which fails identically on github/main when its file runs
                                    together and passes alone on both)
ruff check raven/ tests/            All checks passed
ruff format                         clean
```

Driven by hand as well as by tests, each against a throwaway home:

```
doctor --json on a fossil config   stdout parses as JSON again; the notice is on stderr
doctor    (pin 65536, model 1M)    reports the pin and what is sized against it
doctor --fix                       pin removed, file still 0600, a second run reports nothing
save_config on a fresh Config()    2 bytes; 634 after the extension blocks are seeded
```

New cases cover the stderr split, both doctor paths including the mode
preservation, and the save round-trip keeping every value the user
chose.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

The bootstrap change alters what every new install's config file looks
like: nearly empty instead of a full dump. Existing configs are
untouched -- nothing rewrites them -- and both shapes load identically,
so the risk is confined to anyone reading `config.json` as documentation
of what can be set. `raven doctor` and the schema remain the answer to
that question.

The onboarding picker's default moves from the schema default to the
provider's recommendation, as described above.

`doctor --fix` writes only when asked, and only removes a key.
Everything else here is output routing.

Rollback: revert the commits. Nothing persists a new format, so a
reverted build reads every config written under this one.

## Related Issues

N/A

---------

Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-18 18:55:29 +08:00
LivXue
ddb4d1129e feat(*): set the sub-agents up during onboarding instead of at install time
## Summary

Setup now offers the sub-agents that ship in a source checkout, and the
installer stops trying to.

**The bug this fixes is that a first install registered nothing.**
`subagents/install.sh` decided whether to register a folder by reading
`~/.raven/config.json` for a provider key the folder could fall back on. On a
fresh machine that file does not exist yet - it is written by the setup wizard,
minutes after the installer has finished - so the read failed, every folder took
the "no LLM to inherit" branch, and the script built three venvs and registered
none of them. The other half of the same gap: nothing anywhere ever asked which
LLM a sub-agent should run on. There was no step, in either the installer or the
wizard, that set one up.

**Registration moves into the wizard, as its fifth step.** It lists each folder
under `subagents/` and asks, per folder, whether to enable it on this raven's
LLM, on a key of its own, or not at all, then writes the roster entries. Placing
it there is what fixes the ordering: the step runs after the provider step, so
the config and the key it depended on both exist by then. It also needs no
restart, which the installer's path did - the startup gate runs the wizard
before `_build_agent_loop` reads `config.subagents.third_party`, so one process
registers the agents and then serves them.

It replaces the deep_research step. That tool is otherwise untouched and stays
reachable through `raven deep-research enable`; only its wizard step is gone,
and `--skip-deep-research` becomes `--skip-subagents`.

**A folder is offered only once its venv is built**, tested the way the
installer tests it (`[ -x ]`, not mere existence - the two readers disagreeing
on a present-but-unexecutable launcher is a defect this branch found and fixed).
A name in the roster that fails the moment the dispatching model picks it is
worse than an absent one. A wheel install has no `subagents/` tree at all, which
is the whole gate for that case: the release artifacts ship `raven/` alone, so
the step finds nothing and says so.

**What "its own key" means, and does not.** An agent registered without one is
not a broken agent: its launcher already copies the host raven's provider block
at every run. What a key of its own buys is the model the folder is tuned for -
inheritance brings the host's `agents.defaults.model` along with its
credentials, so a keyless folder answers on whatever this raven answers on. That
is why the tuned model leads the menu rather than sitting under the fallback.

The model itself is never asked for. Each folder's `config.json` already pins the
one it is tuned for, and that file is tracked, so writing a chosen model into it
would dirty the checkout and be clobbered by the next pull. A key, by contrast,
goes to the folder's gitignored `.env` at mode 600.

**An OpenRouter key already in this raven is reused rather than retyped.** All
three folders are tuned for models served through OpenRouter, so a user who
configured one in step 1 is asked for nothing; the prompt says so, and the step
writes that key and moves on. Two things narrow it deliberately. The reuse reads
`providers.openrouter` alone - a key parked in `custom` belongs to whichever
private gateway that section names, and spending it against openrouter.ai would
surface as a bad credential rather than as the configuration mistake it is. And
whether a folder is on OpenRouter is decided from the base in its own
`config.json`, not from a provider name: all three spell the section `custom`,
which says nothing about who answers. A folder pointed at another gateway keeps
the ask, and the prompt names the gateway either way so "needs an OpenRouter key"
tells the user which account to look in.

`subagents/install.sh` keeps the machine work - build each checkout's venv,
scaffold its `.env` - and loses the registration, the host-config read that
drove it, and the `--config` flag that fed the write. Its summary now reports
which folders are ready instead of claiming a dry run built them.

The models probe is lifted out of `deep_research_commands` into
`raven/cli/_key_probe.py` so both steps share one implementation.
`_validate_key` keeps its name, signature and behaviour, so nothing that
imports or monkeypatches it changes.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run --frozen --python 3.12 --extra dev ruff check raven tests scripts` and
  `ruff format --check` on the same targets -> "All checks passed", "1041 files
  already formatted".
- `.venv/bin/python -m pytest -q` at the final head -> `9129 passed, 48 skipped,
  13 deselected`, no failures and no errors, in 220s.
  An earlier run on this machine reported two failures in `test_atomic_io` and
  `test_agent_loop_lazy_provider`, at load average 25-50 and four times the usual
  wall time. Both pass alone, both pass in either order alongside every file this
  branch touches, and neither imports a module this branch changes; the clean run
  above is the same tree on the same machine. Called out rather than quietly
  re-run.
- Baseline comparison, because a count alone proves nothing: collecting on the
  merge base gives 9141 tests, collecting on this branch gives 9164 before the
  later cases were added. The difference was 23, which was exactly the new test
  file; nothing was removed, and both sides report no failures.
- Two defects in this branch's own new code were found by the repository's own
  guards rather than by review, and are fixed here:
  `test_only_the_auth_module_decides_configuredness_from_a_key` caught the
  OpenRouter lookup deciding "the host has a key" by reading the key and testing
  it for truth - a seventh opinion on what configured means, which is the exact
  divergence that guard exists to prevent. The verdict now comes from
  `providers.auth` and only then is the value read. And a test helper pointed the
  loader at a temp config with `set_config_path`, a module global that outlives
  the test and then wins over the environment, which turned two
  `test_config_loader.py` cases red in a full run while the file passed alone;
  it uses `RAVEN_HOME` now, which monkeypatch restores.
- `scripts/check_commit_messages.py origin/main..HEAD` and `make check-commits`
  -> pass. `make check-large-files` -> pass. `sh -n install.sh` and
  `bash -n subagents/install.sh` -> pass.
- The new tests are not theatre: reverting the executability check to a plain
  existence check turns `test_a_present_but_unexecutable_launcher_is_not_built`
  red, and restoring it turns it green.
- End to end on the path that was broken, driven against a throwaway
  `RAVEN_HOME` with no config file: before, `config.json` does not exist; the
  step discovers all three folders, registers all three, and afterwards
  `config.json` holds `{"subagents": ...}` and nothing else - no `providers`, no
  `agents.defaults.model`. `_is_config_populated()` still returns False, so the
  provider step still runs rather than being swallowed by the file now existing.
- The venv gate was exercised from both sides in the same run: with no venvs
  built, all three folders are declined with the "not built yet" message and
  none is registered; with the launchers present and executable, all three are
  offered and registered.
- The menu was rendered against two host configs to check the reuse decision
  rather than reason about it. With `providers.openrouter.apiKey` set, every
  folder leads with "Recommended: <model> via OpenRouter (reusing this raven's
  OpenRouter key)" and no key is asked for. With the only key in
  `providers.custom` pointed at a private gateway, the same folders lead with
  "needs an OpenRouter key" and the ask is kept - the host having *a* key is not
  the host having *that* key.
- `subagents/install.sh --dry-run` and `--help` were run against the rewritten
  script.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

**The wizard's fifth step is a different step.** Anyone scripting
`--skip-deep-research` gets an unrecognised-option error; the flag is
`--skip-subagents`. The deep_research tool itself, its config block and
`raven deep-research enable` are unchanged, so nothing configured stops working
- it is only no longer offered during setup.

**`subagents/install.sh --config PATH` is gone.** It existed to aim the
registration write at a config other than the host raven's, and there is no
longer a write to aim. A caller passing it now gets "unknown option".

**Running the installer alone no longer produces a roster.** That is the
intended split - it never reliably did, and on a first install it never could -
but a machine that runs `./install.sh` and never runs `raven` now has three
built venvs and no entries. Both READMEs say so.

Non-interactive and `--skip-subagents` runs skip the step entirely and record a
warning rather than defaulting to enabling everything: an unattended install
should not put agents in a roster nobody asked for.

Reusing this raven's OpenRouter key copies it into the sub-agent's `.env`, so
that folder's spend lands on the same account. Choosing this raven's LLM instead
leaves no copy: the launcher reads the host config at each run.

The key the user types is probed against the base in the folder's own
`config.json`, a value that ships in this repository - not one the caller picks
- so this adds no path from a stored secret to a chosen destination.

Rolling back is reverting the branch: the installer's registration is deleted
here, not disabled, so a partial revert would leave neither side registering.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-20 04:14:04 +00:00