40 Commits

Author SHA1 Message Date
zhanghui
dc3e57a890 fix(*): make the eight reload-only settings apply without a restart (#543)
## Summary

Eight keys on the settings page answered `Saved. Applies after the next
gateway
reload or restart.` They do not any more, and the reload-only set is
gone with
them -- there is nothing left for it to name.

They were all the same shape: the value was copied onto the objects the
loop was
built with, and nothing read the file again. The fix is not a
notification
mechanism. Each value is read from the file where it is used, so a write
from
the page, the CLI or a hand edit is covered by construction rather than
one
writer at a time.

**The two subsystem pins** -- `context.curatorModel` and
`skillForge.llmGateModel`, each with its provider -- are resolved per
call
instead of once in the context-engine factory. Both consumers already
funnelled
through one method (`CuratorSegmentBuilder._curator_binding`,
`LLMGateFilter._binding`) and already read the turn's binding when
unpinned;
only the pin itself was frozen. `permissions.judgeModel` and
`sessionTitle.model` already read their pins this way, which is why
those two
were never reload-only -- so the part that re-reads, binds through the
pool and
reports an unusable pair once is shared as
`providers.pool.live_pin_resolver`.

**`agents.defaults.maxToolIterations`** becomes a property over the live
file,
falling back to what the loop was built with. A turn reads it once at
its start,
so a change lands on the next turn rather than halfway through one.

**`agents.defaults.contextWindowTokens`** was the interesting one. It
had two
copies with different readers: one frozen on the loop that only the
usage report
read, and one on each `ModelBinding` that the budget read. A write
reached
neither until a restart -- and after a model switch the two disagreed,
because
the pool rebuilds its bindings when the number changes while the loop's
copy
never moves. Measured on a real gateway before this change: trimming ran
on the
new number while the page reported the previous model's catalogue size.

There is one source now, on the binding. The turn entry resolves it from
the
file once and holds it for the whole turn tree, the same contract the
provider/model pair already has (`AgentLoop._with_live_window`): the
budget is
read several times while a turn runs -- history trimming, the compaction
check,
the usage report -- and a number that moved between those reads would
leave one
turn disagreeing with itself. Outside a turn the default binding
answers, which
is the rule `context_window_tokens` already followed.

Two consequences a reviewer should weigh:

- **The file is now the source for these values, not the object passed
at
  construction.** In every production path they are the same thing --
`EngineWiring` is filled from the same loader -- but a host that builds
a loop
with a synthesised `ContextConfig` or window and no config file behind
it now
gets the file's answer (or none). Clearing a key therefore takes effect
too,
  which is what made `contextWindowTokens: null` on the settings page do
  anything at all.
- **`set_provider` carries the current default's window forward** rather
than
re-reading, because that entry point exists for a caller holding a
provider of
  its own, which need not be a process this config was loaded for.

Not in this change: `assembler.set_context_window` /
`ContextEngine.set_context_window`
have no caller in `raven/`, so the out-of-turn fallback windows held by
the
curator and the history trimmer have never been updated after
construction.
Pre-existing, untouched, and invisible to a turn (both read the active
binding).

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Real host, one `raven serve` process, one session that never switched
model, no
restart at any point:

```
turn 1                 context_max=0
settings.set 50000     {"applied": true, "previous": null}        <- no warning
turn 2                 context_max=50000
settings.set null      {"applied": true, "previous": 50000}
turn 3                 context_max=0
```

The same host earlier, on the code before this change, answered
`context_max=0`
on turn 2 and carried `Saved. Applies after the next gateway reload or
restart.`
in the reply; a restart on the same RAVEN_HOME was what produced 50000.
The
disagreement described above was measured the same way: after
`config.set model`
in that process the report read 131072 while a pool-built binding
carried 50000.

`agents.defaults.maxToolIterations` was checked the same way on a real
host
before this branch existed: the gateway log goes from `Iteration 1/40`
to
`Iteration 1/3` across a `settings.set`, in one process.

The two pins are covered by tests rather than by a live host: the
curator's slow
path and the skill gate did not run on a short turn, so there was
nothing to
observe. The cases do a real factory build, rewrite the config file
underneath
it and assert the next call follows.

- `uv run pytest tests/ -q`: 23325 passed, 4 failed, 111 skipped. All
four
  failures reproduce on a clean checkout of the base commit in the same
  environment (`test_agent_loop_token_budget`, the two
`test_agents_*_launcher` tool-face cases,
`test_ppt_engine_image_search`);
the launcher pair looks environment-dependent (a `SERPER_API_KEY` in the
  developer environment makes `web_search` available).
- After the rebase onto the current base: the same sweep over
settings/context/skill-forge/loop/pool/usage-sink/onboarding, 2781
passed,
  1 failed (the pre-existing token-budget case).
- `npm run type-check` clean; `npx vitest run` 185 files, 2568 tests
passed.
- `uv run ruff check` and `ruff format --check`: clean.
- Eighteen new or changed cases, each mutation-verified on its own --
among
them: caching the resolver's first answer, warning on every call,
reading one
spelling of a block or a field, a turn rebuilding its window when the
number
has not changed, a running turn following a mid-turn edit,
`set_provider`
dropping a pinned window, and a settings write still claiming a restart
is
  needed.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

User-visible: these eight writes take effect where they are read -- the
next
call for a pin, the next turn for the cap and the window -- instead of
after a
restart, and the settings surface no longer says otherwise. A turn
already
running is never moved.

Contract: `settings.set` still returns `warning`; the only producer left
is an
embedding model change, which the page renders the same way.

Cost: the added reads are a byte comparison on one small file
(`LiveConfig`
re-parses only when the bytes change) plus a pool cache hit for a pin
already
built. Nothing on this path builds a provider that was not going to be
built.

Rollback: revert the commit. The two deleted holders
(`AgentLoop._configured_window`,
`_RELOAD_ONLY_KEYS`) come back with it, and no other branch reads
either.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-22 01:05:18 +08:00
Handsome-wzw
7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00
forrestjgq
f01231fef6 fix(cli): overhaul trajectory browser interaction ux (#379)
## Summary

Overhaul the interactive `raven trajectory` browser end to end, fixing
every
issue from real-terminal testing:

- Navigation: every screen opens with a help area under its title (page
  purpose plus "[Key] function" hotkey hints, rendered in a new readable
help color). Esc walks one level up (action -> attempt -> session); on
the
top-level session screen Esc stays put and quitting is Ctrl+C only,
which
prints a single exit notice. Esc inside a running action cancels just
that
action through a dedicated exception at the action boundary, so the back
  sentinel can never reach a conversion or the data layer; a cancelled
  report keeps its pre-confirm bundle-and-pin side effects (the existing
  declined-report contract). Back/Exit menu entries are gone; answered
  prompts erase themselves and navigation is echoed as breadcrumb lines
  with markup-escaped untrusted text.
- Tables: the session list is TITLE | ATTEMPTS | LAST ACTIVITY and the
  attempt list is # | STARTED | TURNS | SPANS | VERDICT | PIN | MERGED |
PREVIEW with green check cells. Time cells keep the year. The layout is
  budgeted per screen build from the live terminal width (row indent, a
safety margin, and inter-column gaps are all part of the math); PREVIEW
  drops as a whole column when it no longer fits, and below the dynamic
  minimum width the tightest layout is kept while the renderer clips
  overlong lines at the terminal edge. Every dynamic cell passes a
  single-line sanitization gate before width math (verdict sidecars only
  guarantee a non-empty string), cells are measured and truncated by
terminal display width (CJK is 2 cells), and fixed column widths always
  fit their headers.
- Space preview: Space on an attempt row prints that attempt's per-turn
input/output previews, collected in the same snapshot scan and sorted by
a fully-stringified key so ordering never follows log source order (the
  table PREVIEW cell derives from the same sorted collection). Any key,
arrow keys included, returns with the cursor kept on the row; the waiter
  passes the terminal's CPR reply through to the renderer instead of
  treating it as a keypress, and Ctrl+C keeps the browser-wide cancel.
- Merge hotkey: with two or more attempts, m/M opens the multi-select
merge
  (advertised in the help line; the old "Merge attempts" menu entry is
gone). The checkbox screen carries its own help line for toggle,
confirm,
  and cancel. A single attempt neither binds nor advertises the key.
- Theme and feel: new success (green check) and help (readable hint
text)
colors in both palettes, all WCAG AA on their backgrounds; a lone ESC is
flushed after 50ms instead of prompt_toolkit's 0.5s default, removing
the
  laggy Esc feel.

The data layer is untouched: changes are confined to
raven/cli/trajectory_browse.py, raven/cli/_theme.py, and their tests.
The
id-taking subcommands (raven trajectory save/list/...) behave as before.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `COLORTERM=truecolor uv run pytest tests/test_cli_theme.py
tests/test_cli_trajectory_browse.py
tests/test_cli_trajectory_commands.py -q` -> 220 passed
- `COLORTERM=truecolor uv run pytest tests/ --ignore=tests/integration
-q` -> 6917 passed; the 10 failed + 20 errors are the pre-existing
environment-dependent set, identical file-by-file on a clean main
checkout (cron, everos server, config loader, tracing viewer)
- `uv run ruff check` and `uv run ruff format --check` on the four
touched files -> clean
- PTY smoke on a CPR-answering pseudo terminal at 100 and 58 columns:
full flow (preview, merge cancel, action screen, staged Esc backout,
Ctrl+C quit) and the narrow-width degradation contract
- New coverage includes real pipe-input key-binding tests (Esc on all
four prompt kinds, Space/m/M hits, waiter key paths with CPR
pass-through), table budget/clipping boundaries at the computed minimum
width, and corrupt verdict/timestamp corpora

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The interactive browser's look and key protocol change wholesale (that
is
  the point of this PR): Esc no longer quits from the top level (Ctrl+C
  does), Back/Exit entries are gone, and lists render as tables. The
machine-facing subcommands and the trajectory data layer are untouched.
- Rollback: revert the squash commit; the browser returns to the
previous
  interaction wholesale, no data migration involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-05 23:58:20 +08:00
0xKT
2f862d8ca2 feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
hylarucoder
c703db6ffd feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
0xKT
a671fdddcf refactor(agent): the AgentLoop constructor takes five wiring bundles and nothing else
The constructor kept a **legacy shim that folded forty-odd flat keyword names
into the wiring bundles field by field. Production stopped using it when the
entrances moved to bundles; fifty-five test files had not. Every one of those
call sites (and the helpers that forward **kw to the loop) now constructs the
bundle it means, ToolWiring / SubagentWiring / EngineWiring / TurnPolicy /
HostWiring, so a wiring field has one address in the tests as in the code. The
shim, its field-to-bundle map's private name and the docstring that promised
the old names go; resolve_wiring only fills in the default bundles.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-30 03:52:45 +08:00
0xKT
08596d7ba8 refactor(*): shape-only consumers import the provider papers from contracts
raven.providers.base defines the LLMProvider adapters subclass (retry,
sanitizing, classification, tracing) on top of the paper of the same name in
raven.contracts.llm_provider, and re-exports the paper's other shapes. Ninety
files imported from the base; only the adapters and the test fakes that
subclass it need the machinery. The forty-odd modules that merely type against
the shapes (the context engine, the sentinel, the assembly stacks, the
sub-agent backends, the consolidator) now import them from the paper, so the
dependency they declare is the one they have. The base keeps its name and its
re-exports for the modules that extend it.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-30 03:42:59 +08:00
Handsome-wzw
9a87470fdf fix(rpc): turn.started is a turn event, and says so on all three sides
## Summary

Found while investigating the first row of Dizhan's `子智能体功能与显示问题` doc (*回注内容过程中未正常显示,刷新后才会正常显示*). This is not that bug's fix, but it is a real defect on the same wire, and it is the one that explains why "前后端接线没有约定好" was a fair thing to say.

**`turn.started` is not in the `TurnEvent` union — on either side.** It has a schema component, it has a Pydantic model, and neither is reachable from the union that `tests/test_rpc_schema_match.py` walks. The 290-test contract guard has never looked at this event.

So it drifted the first time anyone touched it. `05957452` (judge a dag node) taught `announce_dag_exception` to emit:

```python
mark = {"kind": "dag", "label": run_id, "status": "exception", "run_id": run_id, "node_id": node_id}
```

while `TurnStartedEvent.payload.delegated` still declared `status: enum ["ok","error"]` with `additionalProperties: false`, and `TurnStartedPayload` had **no `delegated` field at all** under `extra="forbid"`. Verified with `jsonschema` against the published contract before changing anything:

```
Additional properties are not allowed ('node_id' was unexpected)
```

That commit did carry `exception` into `TranscriptDelegated` and `SubagentDeliveredEvent` — two of the three places this status lives. The third is the one no test covers.

**It does not crash today**, and I want to be exact about that: `SubscriptionEmitter.emit` pushes a raw dict with no validation, and the page reads only the fields it knows. But the contract test's own docstring says what the cost is — *"both sides declare `additionalProperties: false` / `extra="forbid"`, so a consumer validating against the schema rejects the whole event, not just the unknown field"*. Losing this event loses the boundary that advances the workspace turn counter and draws the delegated row.

## What this does

- `TurnStartedEvent` joins the `TurnEvent` union, in `openrpc.json` and in `raven/rpc/models.py`, so the existing sweep covers it from now on.
- The schema's `delegated.status` gains `exception`; `node_id` is declared.
- `TurnStartedPayload` gains `delegated` as a typed model (`TurnStartedDelegated`), three-valued status included.
- Both generated TypeScript clients regenerated.
- A new test drives **the exact marker `manager.py` builds** through both validators, so the next status or field fails here instead of in a client nobody is watching.

## One thing the union surfaced

`ui-tui` never handled `turn.started`. Its exhaustiveness check said so the moment the variant existed. It draws the delegated row from `subagent.delivered`, which the page deliberately ignores in favour of this event — the two mark different moments: **submitted** versus **started**.

Left as it is, with an explicit `case 'turn.started'` saying why. Moving that surface onto the truer moment changes when the TUI draws the row, which is a product change, not a contract fix.

## Verification

- New test red before the fix, by construction: adding the variant to the union alone reds `test_schema_match_turn_event_discriminated_union` and `..._payload_fields` (schema/pydantic drift), which is how the gap was proved.
- Mutation checks, each reverting one thing:

| reverted | reds |
|---|---|
| `exception` out of the schema enum | 1 |
| `node_id` out of the schema | 1 |
| `delegated` off the Pydantic payload | 2 |

- `uv run pytest -q` — **12900 passed**, 50 skipped, 5 failed; the same five pre-existing on `origin/main` (four `test_security_network.py` NAT64 cases, one `test_doctor_answers_where_the_memories_are`), reproduced on an untouched merge-base worktree.
- `uv run ruff check` / `ruff format --check` on `raven tests scripts` — both clean (both halves of `make lint-python`).
- `ui`: `gen:check` 147 methods, `type-check` clean, `npx vitest run` 89 files / **1225 passed**.
- `ui-tui`: `type-check` clean, `lint:rpc` in sync, `lint:i18n` up to date, `npm test` 138 files / **1990 passed**.
  - Worth recording: with `ui-tui/node_modules` symlinked from another checkout the suite reports 82 failed / 59 failed — and reports **the same numbers on an untouched `origin/main`**. It is the symlink, not the branch. Ran `npm ci` in the worktree to get a trustworthy number.
- `scripts/commit_lint` — no errors.
- `git ls-files | grep -c node_modules` — 0.

## Type

- [x] Fix

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear

Widening a contract to match what the runtime already emits: every payload valid before is valid now. No emit site changes, no behaviour changes. The one code change outside the schema and the models is an explicit no-op branch in `ui-tui`. Rollback is a revert.
2026-08-28 12:51:10 +00:00
forrestjgq
8ecfae3e0c test(tracing): isolate span emission into per-test tmp dirs (#371)
## Summary

Unit tests were leaking tracing spans into the real `~/.raven/traces`
store. Tracing is on by default and `trace.span()` calls are embedded in
library code (agent loop, subagents, TUI RPC), so any test exercising
those paths emitted spans with fabricated session keys (`session-a`,
`weixin:c`, `tui:default`, ...). Those spans then surfaced as phantom
sessions in `raven trajectory`, while the TUI `/sessions` list (which
reads the real session store) stayed clean.

This PR adds an autouse fixture in `tests/conftest.py` that:

- redirects `RAVEN_TRACING_DIR` to a per-test tmp directory, and
- resets the lazy module-level `TraceStore` cache
(`raven.tracing.spans._store`) around each test, since it pins the
directory resolved at first emit.

Key decisions:

- Only the directory is redirected, not the enabled switch, so the suite
keeps exercising the real emission path with zero behavior change; leaks
land in tmp instead.
- This centralizes the setenv-plus-cache-reset pattern that 8
tracing/trajectory test files already perform manually. Those tests keep
working: their own `monkeypatch.setenv` runs after the fixture and wins,
and they already reset the cache themselves.
- Existing phantom sessions come from archived span logs written before
this fix; the fixture only prevents new leaks. One archive file mixes a
real session with test keys, so cleanup needs line-level filtering and
is left out of scope.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Test-suite hygiene (`test` commit type).

## Verification

- `uv run pytest tests/test_tracing_api.py
tests/test_trajectory_store.py tests/test_trajectory_bundle.py
tests/test_trajectory_replay.py tests/test_trajectory_redact.py
tests/test_cli_tracing_commands.py tests/test_cli_trajectory_commands.py
tests/test_cli_trajectory_browse.py -q`: 308 passed, 1 failed
(`test_viewer_health_true_for_our_viewer`, fails identically on
unmodified `main`, environment-specific)
- `uv run pytest tests/test_agent_loop_run_emit.py
tests/test_subagent_manager.py tests/test_tui_rpc_turn_send.py
tests/test_deep_research_tool.py tests/test_agent_loop_approval.py -q`:
109 passed; the real `~/.raven/traces` log did not grow during the run
- Positive check: running `tests/test_agent_loop_run_emit.py` with
`--basetemp` shows the previously leaked session keys (`cli:c`, `tg:c`,
`weixin:c`) now land in per-test tmp dirs
- `uv run pytest tests/ --ignore=tests/integration -q`: 6837 passed, 11
failed, 20 errors; the same failure set reproduces on unmodified `main`
(environment-specific: cron/tz, everos server, theme, config loader,
viewer health), zero regressions from this change
- `uv run ruff check tests/conftest.py` and `uv run ruff format --check
tests/conftest.py`: both pass

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Docs box left blank: test-only change, no user-facing docs affected.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Test-only change; no production code touched. One known interaction:
`RAVEN_TRACING_DIR` takes precedence over `RAVEN_HOME`, so a future test
that fakes `RAVEN_HOME` and expects traces under it must also set
`RAVEN_TRACING_DIR` itself. Rollback: revert the single commit.

## Related Issues

N/A

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-28 19:21:16 +08:00
arelchan
7c33c4133a fix(rpc): let the viewer open the files a turn actually wrote
## Summary

A delivered artifact could not be opened from the page: the file viewer answered
403 and the transcript drew "Raven is confined to the workspace, so this file
cannot be opened here" over a delivery card it had just rendered.

The viewer's state-directory fence exempts the workspace subtree, on the stated
grounds that the default workspace lives inside raven's home and denying it
would refuse every file the agent itself wrote. Since agent home and the working
directory were split apart, a turn no longer writes there: it writes in
`~/.raven/tmp/<channel>`, a sibling of the workspace, inside the same state root
and outside the carve-out. So the fence denied exactly what the carve-out exists
to allow, one directory over, for every artifact produced since that split.

This exempts the working-directory root alongside the workspace, using the same
containment test in the same direction, so a working-directory root that happens
to sit above raven's home earns no exemption. `config.json`, `oauth/` and
`serve.json` are siblings of both and stay refused, which is what the fence is
for.

Worth noting for a reviewer: the bytes were already reachable. `/files/download`
serves a delivered file by token with no path fence, so this was never a
security boundary for deliveries -- only a viewer that refused to render what
the same page could already download.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_rpc_transport.py tests/test_rpc_files.py tests/test_rpc_knowledge.py tests/test_rpc_console.py -q` -> 123 passed
- `uv run ruff check` and `uv run ruff format --check` on both changed files -> clean
- Both new assertions were mutation-checked, each against its own mutation:
  - narrowing the carve-out back to the workspace alone reds
    `test_the_turn_working_directory_is_exempt_from_the_state_dir_fence` only (1 failed, 3 passed)
  - dropping the containment-direction guard reds
    `test_a_working_directory_root_that_contains_the_state_root_earns_no_exemption`
    and the pre-existing state-dir test (2 failed, 2 passed)
- Against the real reporting case: the two files that were answering 403
  (`~/.raven/tmp/tui/*.md`, `*.pptx`) now resolve, while `~/.raven/config.json`
  and `~/.raven/serve.json` still raise PermissionError.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- Widens what the file viewer will render by exactly one subtree: the directory
  turns work in. Everything the fence was written to protect (provider keys in
  `config.json`, the OAuth token store, `serve.json`) sits beside that subtree,
  not inside it, and stays refused -- pinned by the tests above.
- `fs.list` / `fs.reveal` share the function but pass the session's working
  directory as the root, which their own containment check already bounds, so
  their behaviour is unchanged.
- Rollback is a one-line revert of the exemption tuple.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-28 04:57:29 +00:00
LivXue
4e35a5d627 fix(tui): show the copy-on-select confirmation as a transient notice (#366)
## Summary

PR #360 reports every copy-on-select write as a permanent transcript
line, so a
reading session that drags through the transcript piles up "copied N
characters" rows under the conversation. The confirmation splits by what
the
line is for: the unbounded terse repeats become a transient notice
rendered
above the input box -- one line that replaces itself on every new copy
and
clears after 3 seconds -- while the first copy of a session keeps its
path
caveat in the transcript. The caveat is the answer a user comes looking
for
after a paste comes up empty minutes later, and it cannot pile up
because it
is once per session by construction.

The reporter still owns the per-session caveat tally and now returns
`{ text, firstOfSession }` so the caller can route the caveat to the
transcript and the repeats to the notice. A new `copyNoticeStore` holds
the
single notice; `showCopyNotice` keeps one dismiss timer, clearing any
pending
one before arming the next, so an earlier deadline can never clear a
newer
notice. `/copy` is deliberately untouched: an explicit command's answer
still
belongs in the transcript, and it cannot stack on its own.

## Type

- [x] Fix

## Verification

Run from `ui-tui/` at branch head, in a fresh worktree of `github/main`.
Dependencies came from `npm ci` in `ui-tui/` and in the hermes package;
the
hermes package's own `node_modules` holds only esbuild, so the ink
render
tests run against one react copy (two copies break every hook call).

```
npx vitest run --no-file-parallelism
  Test Files  93 passed (93)
  Tests  1084 passed | 13 skipped (1097)

npm run type-check
  clean, exit 0

npm run lint
  22 problems (0 errors, 22 warnings)

npx prettier --check src/app/copyNoticeStore.ts src/__tests__/copyNotice.test.tsx
  src/__tests__/clipboard.test.ts src/app/useMainApp.ts
  src/components/appLayout.tsx src/lib/clipboard.ts
  All matched files use Prettier code style!

python3 -m scripts.check_commit_messages github/main..HEAD
  exit 0
```

The 22 lint warnings are the baseline, not new: 22 before the branch and
22
after, none in a file this branch touches. The suite is run with
`--no-file-parallelism` deliberately: the ink render tests fail under
default
worker parallelism at this suite size, and no CI job runs them.

Suites that exercise the changed code directly:
`src/__tests__/copyNotice.test.tsx`
(7 cases), `src/__tests__/clipboard.test.ts` (30 cases, the 3 reporter
cases
updated to the new return shape).

New tests were proved load-bearing rather than assumed:

- Removing the notice render line in `appLayout.tsx` makes the render
test
  fail (content and placement above the input box).
- Removing the pending-timer clear in `showCopyNotice` makes the
re-shown-notice test fail: the earlier 3s deadline then clears the newer
  notice.
- The dismissed-notice case now shows a notice first, and removing
  `dismissCopyNotice`'s atom clear turns it red (review fix).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

No doc change. `ui-tui/CONTEXT.md` has no clipboard or selection entry,
and
"copy notice" continues the existing "notice" vocabulary of
`copyResultNotice`
/ `copyOnSelectNotice`, so nothing new is coined.

## Risk

Behaviour change: the per-drag copy confirmation is now transient and
above
the input box instead of a permanent transcript line, and the first copy
of a
session lands one transcript line, exactly as on `main`. Reverting the
branch
restores the #360 behaviour; no data or on-disk format is involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (deepseek-v4-pro) <noreply@anthropic.com>
2026-08-25 14:27:24 +08:00
arelchan
172b05a064 fix(ui): give the delivery card one shape, and unbreak four panels
## Summary

Six defects a review pass over the served page turned up, plus the registry read
underneath one of them.

**The delivery card was measured rather than stated.** A width-derived capacity
hook decided how many tiles fit and whether a description was drawn, so one card
was four different cards across six widths. The cap is a stated count now, the
description always shows, and a tile keeps one preview ratio with a max-height
instead of a growing box. Measured after: a single card is 142px at every width
with a description and 134px without, and a grid preview is a constant 198x124.
The preview itself lost its inset white sheet -- content fills the tile.

**A desk pane re-opened itself.** Clicking a row whose pane was already up
replaced the pane object, re-tiled the grid and re-revealed the workspace to
arrive at exactly the screen that was already there, and `revealWorkspace` ran a
full workspace draw even when the split was already open. Both are no-ops now;
workspace draws on a re-click went from one to zero, and none of it ever
repainted the pane's content -- that was measured before changing anything.

**A dag node header ran past its panel.** The node id is unbounded -- a playbook
names one `daily-ai-digest-36e275-scan-news` -- and with nothing able to shrink,
the flex row pushed the "open run" link off the right edge, where it read as the
card having been cut in half. Measured before: a 320px panel overflowed by 18px
and a 280px one by 58px. The name gives way now, and only the name.

**The file view's action row was `.acts`**, which the transcript already uses for
an answer's icon strip -- 27x27 square buttons. Two text buttons were squeezed
into those squares and their labels overlapped, which reads as mojibake, not as a
layout bug. Renamed to `.binacts`: the buttons measure 186x30 and 107x30 with no
overlap. Second class collision found in this stylesheet, after `.mini`.

**The floating desk covered the settings dialog.** It sits in its own band below
the takeover layer now, verified with `elementFromPoint` at the palette centre
returning the dialog's own panel.

**The instance registry was read once per process.** `_load` cached its records
for the life of the gateway, while sub-agents register from their own `cli`
processes into the same shared file -- so every instance created after startup
was invisible to the panel until a restart. The decisive test was appending a row
from outside a running server and getting zero back. It reloads on a change in
`(mtime_ns, size)`, stamped from the stat taken *before* the read so a writer
landing between the two is seen on the next call rather than stamped as
already-loaded and skipped forever; `_flush` restamps after its own write, so
writing does not cost a reparse. This also closes a silent data loss: `_flush`
writes the whole cached map back, so flushing from a stale cache dropped every
record another process had added in the meantime.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run at this head, on `da0f0124`:

- `uv run --frozen --all-extras pytest -q` -> `11214 passed, 33 skipped, 13 deselected, 2 warnings in 715.40s`, exit 0
- `npm test --prefix ui` -> `58 passed (58) / 686 passed (686)`
- `make lint-ui` -> `generated.ts matches the contract (132 methods)`, then `tsc --noEmit` with no output
- `uv run --frozen --extra dev ruff check` + `ruff format --check` on the two python files -> `All checks passed!`, `2 files already formatted`
- `PYTHONPATH=. python3 scripts/check_commit_messages.py origin/main..HEAD` -> no output, exit 0

The five registry tests were mutation-checked: disabling the reload reds three of
them, and removing the restamp in `_flush` reds the one that pins "our own write
is not a reason to reparse". The two desk-store tests were checked the same way.

`make lint-python` over the whole repo is not clean at this head, and was not
before this branch: the errors are in `raven/agent/tools/deliver.py`,
`raven/cli/tui_commands.py`, `tests/test_channels_outlet.py` and
`tests/test_mcp_client.py`, none of which appears in this diff.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible behaviour changes, all of them on the served page:

- a delivery card is one height per state instead of one per width, and always
  carries its description;
- clicking a desk row whose pane is open no longer redraws anything;
- a dag node's name truncates instead of pushing its run link out of view;
- the desk no longer paints over the settings dialog;
- an instance registered by another process appears without a restart.

Rollback: revert the two commits. Nothing is persisted that a rollback would
strand -- the registry change is read-side only, and the css and tsx changes hold
no state.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Known gap, deliberately not fixed here: a spawn's answer is missing from three
instance logs on my own machine (the spawn's prompt is logged as a `user` row
with no `assistant` row beside it), so opening those instances shows a question
that was never answered. Reproduced on disk, not yet diagnosed, and unrelated to
anything in this diff.

## Related Issues

N/A
2026-08-25 13:04:45 +00:00
arelchan
35abacb489 fix(subagent): re-read the instance registry when another process wrote it
`_load` cached the records for the life of the process, so the gateway held
whichever rows existed when it started. Sub-agents register from their own
`cli` processes into the same shared file, which made every instance created
after startup invisible to the panel until a restart -- the decisive test was
appending a row from outside a running server and getting zero back.

Reloaded on a change in (mtime_ns, size), stamped from the stat taken before
the read so a writer landing between the two is seen on the next call rather
than stamped as already-loaded and skipped forever. `_flush` restamps after
its own write, so writing does not cost a reparse on the following call.

This also closes a silent data loss: `_flush` writes the whole cached map
back, so flushing from a stale cache dropped every record another process had
added in the meantime.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 20:41:38 +08:00
LivXue
56b0010a10 feat(agent): add schema-hidden cancel_dag and dag_status control tools
## Summary

Two model-facing control tools for runs the model submitted via `run_subagent_dag`:

- `cancel_dag` -- stops the run (running nodes finish their current step, pending nodes are skipped, nothing is announced) and returns the same per-node view as of the cancellation.
- `dag_status` -- without a run_id lists this conversation's in-flight runs and points at the per-run form; with a run_id renders the run's tallies plus, per node, its summary, subagent, inputs, prompt template (head 10 lines plus the full prompt's path when cut), instance, output file, memory file, and raw timestamps. Missing fields render `(none)`.

Both are registered on the loop but hidden from the provider's tool schema (`ToolRegistry.hide_from_schema`) and from the tool_search catalog. The DAG tool's backgrounded acceptance text is their only advertisement, and only when a call path exists: the fold threshold lives on the `ToolSearchController`, and a single `tool_call_available()` predicate -- read by both the strategy's per-turn assembly and the acceptance text -- decides whether `tool_call` actually ships this turn. No path, no advertisement.

Both tools are conversation-scoped: the listing intersects the loop-wide live set with this conversation's session index, and `cancel_dag` verifies the run resolves under this conversation's run history before signalling the cancel, so one chat's model can neither see nor stop another chat's run (which that chat would never be told about, a cancelled run being silent by design). Reads by id were already session-scoped.

The read path (`_reader.read_run`) now carries `task_summary`, `node_summary`, and `memory_file` for every consumer, the `dag.get` RPC surfaces included. The summary fields come from graph.json and are forward-compatible: runs recorded before the dispatch-task-summary merge read back as `(none)`.

Deliberate decision, kept rather than fixed:

- The tool's description and the orchestration guide skill still say the result "comes to you" and not to poll; the acceptance text is the only place the control tools are advertised, per instruction. Both sentences are left as they are.

## Type

- [x] Feature

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally

Exact commands and results, at the pushed head:

- `uv run pytest tests/` (via the project venv): 11204 passed, 41 skipped, 13 deselected, 0 failed. The suite was also green at each earlier commit of the branch.
- `ruff check` over the changed files: clean.
- `make lint-python`: the 4 remaining errors are pre-existing on origin/main in files this branch does not touch.
- `scripts/check_commit_messages.py origin/main..HEAD`: clean.
- Functional probe (not committed): a real `SubAgentDagTool` with a real CLI backend ran a two-node graph while the probe asserted the acceptance hint, the in-flight listing, mid-run status with live statuses, cancel with the node block, the post-cancel cancelled/skipped state, the no-in-flight text afterwards, and the refusal of a foreign run id.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security impact: the new fields are read-only additions and the path joins stay behind the existing run id / node id validation; the control tools refuse anything that does not resolve under the caller's own conversation. Backward compatible: the hidden tools are additive to the registry, the reader fields are additive to the payload, older run dirs show `(none)` for the summary fields, and a deploy without progressive disclosure simply never sees the advertisement. Rollback is reverting the branch; the acceptance text reverts with it.

## Related Issues

N/A
2026-08-25 13:04:32 +00:00
LivXue
c72b3f25bb feat(agent): add schema-hidden cancel_dag and dag_status control tools
## Summary

Two model-facing control tools for runs the model submitted via `run_subagent_dag`:

- `cancel_dag` -- stops the run (running nodes finish their current step, pending nodes are skipped, nothing is announced) and returns the same per-node view as of the cancellation.
- `dag_status` -- without a run_id lists this conversation's in-flight runs and points at the per-run form; with a run_id renders the run's tallies plus, per node, its summary, subagent, inputs, prompt template (head 10 lines plus the full prompt's path when cut), instance, output file, memory file, and raw timestamps. Missing fields render `(none)`.

Both are registered on the loop but hidden from the provider's tool schema (`ToolRegistry.hide_from_schema`) and from the tool_search catalog. The DAG tool's backgrounded acceptance text is their only advertisement, and only when a call path exists: the fold threshold lives on the `ToolSearchController`, and a single `tool_call_available()` predicate -- read by both the strategy's per-turn assembly and the acceptance text -- decides whether `tool_call` actually ships this turn. No path, no advertisement.

Both tools are conversation-scoped: the listing intersects the loop-wide live set with this conversation's session index, and `cancel_dag` verifies the run resolves under this conversation's run history before signalling the cancel, so one chat's model can neither see nor stop another chat's run (which that chat would never be told about, a cancelled run being silent by design). Reads by id were already session-scoped.

The read path (`_reader.read_run`) now carries `task_summary`, `node_summary`, and `memory_file` for every consumer, the `dag.get` RPC surfaces included. The summary fields come from graph.json and are forward-compatible: runs recorded before the dispatch-task-summary merge read back as `(none)`.

Deliberate decision, kept rather than fixed:

- The tool's description and the orchestration guide skill still say the result "comes to you" and not to poll; the acceptance text is the only place the control tools are advertised, per instruction. Both sentences are left as they are.

## Type

- [x] Feature

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally

Exact commands and results, at the pushed head:

- `uv run pytest tests/` (via the project venv): 11204 passed, 41 skipped, 13 deselected, 0 failed. The suite was also green at each earlier commit of the branch.
- `ruff check` over the changed files: clean.
- `make lint-python`: the 4 remaining errors are pre-existing on origin/main in files this branch does not touch.
- `scripts/check_commit_messages.py origin/main..HEAD`: clean.
- Functional probe (not committed): a real `SubAgentDagTool` with a real CLI backend ran a two-node graph while the probe asserted the acceptance hint, the in-flight listing, mid-run status with live statuses, cancel with the node block, the post-cancel cancelled/skipped state, the no-in-flight text afterwards, and the refusal of a foreign run id.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No security impact: the new fields are read-only additions and the path joins stay behind the existing run id / node id validation; the control tools refuse anything that does not resolve under the caller's own conversation. Backward compatible: the hidden tools are additive to the registry, the reader fields are additive to the payload, older run dirs show `(none)` for the summary fields, and a deploy without progressive disclosure simply never sees the advertisement. Rollback is reverting the branch; the acceptance text reverts with it.

## Related Issues

N/A
2026-08-25 13:04:32 +00:00
gloryfromca
f02fac0551 chore(*): merge the trunk into the v0.1.13 sync branch
Picks up fd54c282 so the branch is a descendant of main again. Fast-forward
merge needs that, and the rebase GitLab offers instead would flatten the five
merge commits this branch exists to carry -- v0.1.13 stops being an ancestor the
moment they go, which is the whole point of the sync.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 20:39:46 +08:00
arelchan
aa42b88b2c feat(tracing): read the span logs per shard, not all at once
## Summary

!267 stopped the tracing panel rebuilding what had not changed. It did not make a
rebuild cheap, and one happens on every append, which with any raven running is
continuously. This makes the reader read shards instead.

The store was already sharded and had been all along: rotation only ever writes
`archive/<date>/`, and the store measured here is 38 files. The reader
concatenated all of them into one document and shipped it. So nothing here
deletes anything or changes what is written -- it changes what is read.

Measured on a real 3.2 GB store, 270,335 span records, 187 sessions:

| | before | after |
|---|---|---|
| opening the panel | 3.4s, 171 MB | 0.045s, 57 KB |
| opening one session | included above | 0.13s, 102 KB |
| API view, last 24h | included above | 0.048s, 774 KB |
| resolving a trace id | scan of every session's traces | 0.017s |

### The constraint that shaped it

A span cannot be attributed to a session by looking at the span. On this store
152,059 of 270,344 records carry no `session.id` at all, and none of the ones
that do is in uuid form; attribution runs through an election over the whole
corpus (`buildSessionIdentityIndex`). A per-file index that recorded "this file
holds session X" would therefore be wrong rather than merely partial: a later
rotation can change the election an earlier attribution depended on.

What the election consumes is not spans, it is counts, and counts aggregate. So a
sidecar records the raw (session.id, session.key) pairs it saw plus local
tallies, and the election runs over the merged tallies. `server.js` builds the
identical tallies from spans in memory and calls the same elector, so the two
readers cannot drift into electing differently. `assembleSession` is shared the
same way, so a session cannot come out different depending on which reader asked.

Tallies are deduplicated by span id per file, matching what `dedupeSpans` does to
the whole corpus -- a re-emitted span carries a later endTime and counting both
skews the election. Per file is exact here because a span id does not straddle a
rotation: measured, all 22,998 duplicates sat inside one file.

### What moved off the whole payload

Three views read across every session and each needed somewhere else to go:

- the sidebar filter matched trace ids out of `session.traces`. Trace ids are
  resolved through the index now, which is exact and instant.
- the API view aggregates `llm.call` spans across everything, and a quarter of
  all spans is one. `/api/llm-calls` windows it, and applies the window to whole
  files first so a file that predates it is never opened.
- jumping between a subagent run and its parent turn scanned every session's
  traces. It scans the loaded ones, then asks `/api/trace-owner`.

### Three things the plan got wrong

Recorded in the design doc rather than quietly fixed, because each cost real time
to find:

1. **A list row cannot carry a trace count.** `buildTraceGroups` groups by
   parent-child linkage, not by trace id, so the count is the number of spans
   whose parent is absent from the session -- not decidable one file at a time. A
   test with a parent and child split across two files caught the index saying 2
   where the reader said 1. Exactness would mean holding every span id and edge,
   about 18 MB of sidecar for this store, parsed per list request. The row
   carries an exact span count instead, and the card reads "N spans" until the
   session's own response brings the trace count.
2. **Changing the indexing logic needs the schema bumped.** A sidecar is
   validated against its log's size and mtime, which say nothing about the code
   that wrote it. The first dedupe appeared to do nothing because every sidecar
   on disk had been built by the previous version.
3. **A NUL byte reached a source file.** `pairKey` joined ids on a literal NUL,
   which made `file` report the source as data and made this shell's `grep`
   return silent false negatives on it. Replaced with a serialized key.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest -q tests/integration/test_tracing_viewer_e2e.py -m integration`
  -- 14 passed. Six are new and compare the sharded reader against `/api/data`
  rather than asserting a shape, since a disagreement is silent and the sharded
  answer is the wrong one.
- `uv run pytest -q tests/test_cli_tracing_commands.py tests/test_tracing_api.py
  tests/test_tracing_compact.py tests/test_no_otel_tracing.py` -- 86 passed.
- `ruff check` and `ruff format --check` on the test file -- clean.
  `make check-large-files` -- clean.
- `/api/data`'s payload was hashed against a frozen copy of the real store before
  and after each refactor of shared code, and is byte-identical throughout.
- Eight mutations, each failing exactly one test: reading one file per session,
  gutting the election, trusting stale sidecars, keeping unattributed llm calls,
  answering 200 for an unknown trace, answering 200 for an unknown session, and
  the two from !267 that still apply. Two of these tests initially survived their
  mutation and were rewritten -- the election one because every fixture span
  carried a `session.id` and resolution short-circuits, the sidecar one because
  the fixture only corrupted sidecars, which an earlier check rejects, leaving
  the size and mtime path untested.
- Driven in a browser against the real store, because nothing under
  `raven/tracing/viewer/` is covered by a lint or type gate: 187 session cards,
  a click draws skeletons and then 40 span nodes nested to depth 2 with no
  missing fields, the API view returns 291 rows for 24 hours and 7,440 for 7 days
  as the window changes, a pasted trace id surfaces exactly its session, and the
  console is clean.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- `/api/data` is unchanged and still served, so a rollback of the page alone is
  viable. The tests depend on it, deliberately.
- A list row no longer carries `traceCount`, and the session card reads "N spans"
  until a session is opened. This is the one visible change to what the panel
  says.
- Sidecars are new files under `logs/index/`. A missing, corrupt, stale or
  future-schema sidecar rebuilds rather than being skipped, so the worst case is
  slowness, not a session vanishing; there is a test for each of those four
  cases. Deleting `logs/index/` is always safe.
- A refresh drops loaded sessions and re-reads only the open one, so a long-lived
  tab no longer accumulates every session it visited.

## Related Issues

Found while testing this, pre-existing and NOT addressed: `/api/search` is
synchronous and its result cap only stops a scan that reaches the cap, so a term
too rare to match 200 spans reads every artifact in the store -- 106s for a trace
id against 4s for a common word, on merged main, during which nothing else the
viewer serves can be answered. It wants the search index this work makes
possible, and it is why trace ids are resolved through the index here.

N/A
2026-08-25 12:27:24 +00:00
gloryfromca
5a89ecca95 chore(*): merge the trunk into the v0.1.13 sync branch
Picks up raven-ppt's second workflow-refresh sync. Named rather than left as
git's default subject, which the push rule rejects.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 19:14:30 +08:00
LivXue
1a6ac3f639 feat(subagents): sync raven-ppt with the feat/ppt workflow refresh
## Summary

Import `subagents/raven-ppt` from `origin/feat/ppt`, and carry the fixes that this
MR's five review rounds produced. Six commits on the branch: the original import, then
one re-sync per review round.

The routing is the key decision. This change is an import, not authored code, so its
correctness proof is byte-equality with the upstream branch -- `git diff
origin/feat/ppt -- subagents/raven-ppt` prints nothing, before and after every fix
round. Patching the vendored copy would have destroyed that proof and forked the tree,
so every finding was fixed upstream on `feat/ppt` first and came back through a re-sync.

The import itself carries three upstream workflow commits `main` never received:
`b2ce11a4` (Route A authoring, rendered-page feedback, template defaults, image search,
figure captions, visual layout checks; removal of the fact gate, design pass now
opt-in), `5e2c33ed` (one rendered page per build call, ten bundled light templates),
`fe12f0ac` (prototype-first content routing, prototype clearing on replan, resuming a
prepared project at `ppt_build`).

`main` added nothing to `subagents/raven-ppt` since the fork point, and nothing in this
range sits outside `subagents/raven-ppt/Raven-PPT/`. The launcher layer is untouched:
`run.py`, `install.py`, `config.json`, `subagent.json` and `README.md` keep the host
key inheritance added earlier.

### What the review rounds changed

Six blocking findings in the first round, three in the second, one in the third, one in
the fourth, one in the fifth -- all reproduced independently before being accepted, and
two further defects found while fixing them. One commit per finding on `feat/ppt`:

| Finding | Fix |
|---|---|
| `section` dropped on load | `39d5f74d` |
| CairoSVG native-library failure | `ef5a88d2` |
| prepare resume path | `1f8a2144` |
| outline checks judged the outline that got persisted | `0d32aed2` |
| redirect SSRF | `8bf46922` |
| revised request resumed past | `f5f10538` |
| fact anchoring rule changed (see Risk) | `357e81ba` |
| contracts still advertised the fact index | `6e7fd826` |
| ppt_outline / ppt_build still promised number enforcement | `cedcc244` |
| author prompts promised it, and the verifier required it | `708d1294` |
| existing workspaces kept it, and sync fills only missing files | `bd22b211` |
| generate_image's own key unargued in the auth guard's allowlist | `a970918e` |

Two findings beyond the filed ones, worth recording for what they reveal:

1. `_resume_payload` read `deck_state` without importing it, so resuming a prepared
   deck raised `NameError` rather than returning stale data. The pristine upstream
   branch's own `ruff check` already reported it (`F821` at `prepare.py:152`); no test
   covered the function. Fixed in `1f8a2144`.
2. The skill-claims verifier carried `ok("check_text" in outline_tool, "the skill says
   the outline's numbers are checked and nothing checks them")` -- an assertion written
   for precisely the failure it was meant to guard -- and a sibling assertion still
   *required* the false wording, so the suite went green by enforcing the lie. The
   assertion is inverted rather than deleted in `708d1294`, so re-adding the promise
   now fails the verifier (verified: `150 claims checked, 1 failed`).

The same withdrawn guarantee survived five different sweeps by failing each on a
different axis: the spec is written in Chinese (wrong language), the code said `fact
index` where the search said `fact gate` (wrong vocabulary), the behaviour spelled out
where the mechanism name was searched (wrong abstraction level), tool descriptions
enumerated but skills and profile templates omitted (wrong surface), and existing
workspaces never re-read the corrected template at all (wrong time -- backward
compatibility). Each axis is now a check in our review catalog.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run with the vendored checkout's own venv (`uv sync --frozen --extra ppt`), from inside
the checkout so `raven` resolves to the tree under test, confirmed with
`python -c "import raven; print(raven.__file__)"`.

- `python -m pytest tests/ppt tests/test_read_file_image.py tests/test_security_untrusted_context.py tests/test_web_image_harvest.py tests/test_workspace_migration_attention_behaviors.py -q`
  -> **1147 passed, 2 skipped** at the pushed head, re-run at each new head (after every
  rebase) rather than carried forward.
- Baseline, `origin/main`'s copy of the same tree extracted with `git archive` and run
  the same way -> **1099 passed, 2 skipped**. Zero failures on both sides, so **0
  introduced failures**; +48 tests, of which 8 are new regression tests from the review
  rounds.
- `python tests/ppt/_skill_claims.py` -> `150 claims checked, 0 failed`; restoring the
  stale promise produces `150 claims checked, 1 failed`.
- The new tests are not theatre: each was written first and watched fail for its own
  reason before the fix existed -- the redirect test with both URLs in the request log;
  the `section` test with `'' != 'section divider'`; the libcairo test with the
  `OSError` escaping `_sniff`; the resume test with the `NameError`; the ordering test
  with the checks judging `(8, 9)` while `(None, None)` was persisted; the verifier with
  `1 failed` when the promise was reinstated; the workspace-migration test with the
  stale sentence still in the file and the prompt.
- `scripts/check_commit_messages.py origin/main..HEAD` -> exit 0. Commit messages have
  0 non-ASCII lines.
- **Full vendored suite, both trees, same interpreter** (the method that surfaced the
  sixth-round finding): at the pushed head `pytest tests/ -q` gives `95 failed, 7348
  passed, 48 skipped`; on `origin/main`'s subtree copy extracted with `git archive` and
  run through the same venv it gives `95 failed, 7305 passed, 48 skipped`. The two
  failing-ID sets are **identical** (95 = 95), so 0 failures are introduced by this
  range; all 95 are environment-dependent on this box and include the
  `test_provider_auth_method` guard, which now passes 42/42 at the head. The reviewer's
  environment shows the same picture with fewer environment failures (25, of which 24
  reproduced on his baseline).
- Every rebase was proved a pure replay: `git range-diff` shows each commit as `=`, and
  the two file sets (`main's` new commits vs the MR range) do not intersect at all.
  `git merge-tree --write-tree` is unavailable (needs git 2.38, this box has 2.34.1),
  so the overlap check replaces it.
- Runtime wiring: `raven.ppt.tools` exposes 13 modules including the new
  `generate_image` and `inspect`; `DEFAULT_TEMPLATES` loads 10 entries and every `path`
  resolves on disk.
- Launcher smoke: `run.py --help` parses, the vendored CLI reports `Raven v0.1.12`, and
  a live job initialised its workspace, auto-selected a bundled template, rendered
  template previews through `soffice` and wrote `prepared.pptx`.

- [x] Relevant tests pass locally
- [ ] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Lint is left unchecked, with a measured reason rather than a claim. The host gate does
not cover this tree (`PYTHON_LINT_TARGETS` is `raven tests scripts`). The tree's own
nested spec mandates ruff, and the pristine upstream branch does not pass it: `ruff
format --check` reports 11 files and `ruff check` reports 4 errors on `origin/feat/ppt`
untouched. This range keeps format at the same 11 (each round formatted only the files
it dirtied; upstream's 11 are left alone rather than reformatted into the diff) and
takes `ruff check` from 4 to 3 by removing the `F821`.

## Risk

Behaviour changes for anyone invoking the raven-ppt subagent: the fact gate no longer
runs, the design pass is opt-in, `ppt_build` returns one rendered page per call instead
of the whole deck, and content pages prefer an editable template prototype.

**Fact anchoring is no longer enforced in code, by decision.** The nested
`raven/ppt/AGENTS.md` listed it among the guarantees that live in the tool execution
path, and the citation gate's own contract explained its narrow scope by asserting that
another gate had already made every number on the page real -- false once the gate was
deleted, and false in a load-bearing place. The maintainer chose to change the rule
rather than restore the gate. `357e81ba` lifts it out of the hard invariants and
drops the property-test requirement that named it; the correction then propagates
through every surface that promised it -- the tool descriptions (`6e7fd826`,
`cedcc244`), the always-loaded authoring skill, the bundled agent profile template and
its verifier (`708d1294`), and finally existing workspaces (`bd22b211`), whose
`agent_memory/profile/agent.md` is migrated sentence-by-sentence so a user's own notes
are never touched. The gap is recorded as an accepted decision with the measured reason
the gate was removed (all-caps ordinary words refused; `A100,480p` read as grouped
thousands, refusing three pages of a live deck). Consequence to be explicit about: a
number that appears in no source material is no longer refused, and the review
demonstrated that shape (material stating `42 million`, outline claiming `999 million`).

**Repo weight.** The ten bundled templates total 50.5 MiB and nine exceed the 1 MiB
limit in AGENTS.md section 7; `scripts/check_large_files.py` reports exactly those nine
and zero blocked-extension hits. Before this commit no tracked file under `subagents/`
exceeded 1 MiB, so there is no precedent. The maintainer approved them before the
commit. There is no reclaim policy; moving template assets out of git is not attempted
here.

**Security.** The redirect SSRF is fixed rather than disclosed: `_extract_images` walks
redirects by hand and puts every `Location` through the same validator as the submitted
URL, reporting a refused hop instead of following it. `_reader_get` was left alone on
purpose -- it only ever requests the fixed `r.jina.ai` host, and HTTPX drops
`Authorization` across origins -- rather than widening the diff into a path nobody
reproduced.

**Known gap, unlabelled: the live end-to-end run has not been repeated.** Before the
review, one full live job timed out at 2400s with no deck, having spent most of its
budget probing the BLOCKING `house_page` gate across six scratch projects. The ordering
defect that plausibly caused that loop is `0d32aed2`, and the nested spec's own
invariant 7 (a tool must not refuse into a dead end) describes the failure precisely.
That fix is unit-verified only; the live job has not been re-run, so this is stated as
unresolved rather than fixed.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Rollback is one revert: the range touches nothing outside
`subagents/raven-ppt/Raven-PPT/`.

## Related Issues

N/A
2026-08-25 11:06:19 +00:00
gloryfromca
94ac30e2a7 chore(*): merge the trunk again, and keep the picker's scope through it
Four commits landed on main while the review was being answered. Three ui-tui
files conflicted, all on the same feature: main had independently implemented
what upstream's #361 did, as `withoutSpentIntro`, and its version is the better
one -- it counts a slash echo as the user having acted and filters every intro
row rather than only row 0. Taken wholesale.

Taking it wholesale dropped something it had no reason to carry, though, and the
suite caught it: upstream also exports `modelSelectCommand` from that file, which
is what keeps `--default` on a selection made through the picker.
`interfaces.ts` still holds the widened `modelPicker: boolean | 'default'` from
the merge, so without it `/model --default` opened the picker and came back with
a session-scoped switch that read as having changed the default. Grafted onto
main's callback.

One behavioural difference the choice leaves, recorded rather than hidden:
upstream dropped the cover for a transcript opening with an assistant row, and
`userHasActed` does not count one. A resumed session normally carries the user's
own rows too, so it needs a transcript whose user rows are gone -- after a
compaction -- to differ. Main's rule is deliberate about which rows count, so it
stands; the owner of that code should know the case exists.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 19:02:53 +08:00
arelchan
a825c4aeeb test(ui): pin the two seam calls that bypass ds()
## Summary

Measure the gap `!272` disclosed, and close the two real holes it turned out to hide.

`!272` found undefended wiring by making `shell()` and `ds()` return a proxy that no-ops one named member. It also said, in its own description, that the `ds()` half could not see call sites reading `window.DS` directly, and named the four. This measures those four by cutting each call instead. Two were undefended.

**`browser` frame subscription.** `hook()` puts the island's frame handler on the source. Every other test in that file calls `store.onFrame(...)` directly, which is what a pushed frame ends up doing -- but only once the handler is there. Cutting `src.onFrame = onFrame` left every test green while the island went deaf to the live layer: frames arrive and nothing repaints. The new test pins the registration, and the `embedded` guard beside it, since a links-shaped source must not be handed the callback.

**`transcript` path shortening.** An edit's detail header names the file through the workspace source's shortener. This one hid for a subtler reason than a missing assertion: the fixture answered `shortPath` with the identity function, so calling it and skipping it produce the same string, and no assertion could tell them apart. The test gives it something that actually shortens. `edit_file` appears nowhere else in that file, so this is also the first coverage of the diff detail header.

The other two -- `model`'s `installed()` and `subagents`' watch registration -- are defended: cutting them fails 18 and 3 tests respectively.

So the tally across both branches, corrected: **9 real holes, all now closed** -- 3 shell verbs, 4 source members reachable through `ds()`, and these 2 that are not.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other

Tests only; no production file is touched.

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

On `origin/main` at `076d44e0`:

- `npm run --prefix ui type-check`
- `npm test --prefix ui` (58 files, 684 tests)
- `node ui/scripts/count-shared-globals.mjs` (0 / 0 / 18)

Each test was checked by cutting the call it defends:

| mutation | result |
|---|---|
| `browser/store.ts` `src.onFrame = onFrame` cut | fails |
| `browser/store.ts` guard widened to `if (src)` | fails |
| `TranscriptPage.tsx` `shortOr` bypassed to return the raw path | fails |

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Test-only. Rollback is dropping the branch.

## Related Issues

N/A
2026-08-25 09:16:35 +00:00
gloryfromca
de56f91323 chore(*): merge the trunk again before the v0.1.13 sync lands
Two commits landed on main while this branch was being verified.
`SubagentManager.spawn` renamed `label` to `task_summary`, and the dispatch
label and the snapshotted session binding both belong in the call it made.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 17:02:47 +08:00
LivXue
7e4171a2cf feat(agent): make every dispatch say what it is dispatching
## Summary

Every model-authored dispatch now states, in one user-facing line, what it is dispatching.

- `spawn` gains a required `task_summary` and loses its optional `label` from the tool face.
- `run_subagent_dag` gains a required graph-level `task_summary`, and every node a required `node_summary`.
- `PlaybookSpec` gains its own `task_summary`, coexisting with the `description` it already had. They answer different questions: `description` is matched against to decide whether to run the playbook at all, `task_summary` says what running it dispatches.

Key decisions, all of which shaped the diff:

**The obligation lives on the pydantic models**, not only on the hand-written tool schemas, so one declaration binds the DAG tool, playbook files and playbook generation at once. That is why the diff is wide: `DagNodeSpec` is also a playbook's node model, so ~150 raw node dicts and ~24 model constructions across the test suites gained the new field.

**Validation is deliberately asymmetric.** A node's `node_summary` is `str = ""`, blank-tolerant at parse and refused by `validate_and_order` through `_REQUIRED_NON_BLANK`; the graph-level fields use `min_length=1`. The reason is an existing invariant this repo documents at `_graph.py:24-28`: a playbook may ship a node with a field deliberately left for the model to fill, and `load_playbook` can only report that gap if the file loads. The playbook fill workflow was extended to match, so a blank `nodeSummary` is reported as a gap and closed with `fills` rather than dying at dispatch.

**`label` is renamed at three layers and kept at a fourth.** The tool parameter is deleted; the manager API and the on-disk record meta key become `task_summary`; the wire contract keeps `label`, because `SubagentCall.label` is a required field in `rpc-schema/openrpc.json` and `subagent.label` is a span attribute in `raven/tracing/semconv.py`. Only what fills them changed. Spawn records already on disk are read through a three-level fallback (`task_summary` -> legacy `label` -> first line of the prompt), so no existing row degrades.

**Summaries are display text and never enter a sub-agent's prompt.** `manager.py` still dispatches `prompt=task`, and a node still gets its rendered template.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run at the head after the review-fix round, on the same base (`6fdf1d7b`):

- `uv run pytest -q` -> `10839 passed, 51 skipped, 13 deselected, 1 warning in 329.24s`, exit 0
- `cd ui-tui && npx vitest run src/__tests__/dagStatus.test.ts src/__tests__/dagPanel.test.tsx src/__tests__/dagRun.test.ts` -> `3 passed (3) / 84 passed (84)`
- `cd ui-tui && npx tsc --noEmit -p tsconfig.json` -> exit 0, no output
- `cd ui-tui && npm run lint:rpc` -> `OK: generated.ts in sync`
- `cd ui && npm run gen:check` -> `generated.ts matches the contract (132 methods)`
- `PYTHONPATH=. python3 scripts/check_commit_messages.py origin/main..HEAD` -> no output, exit 0 (all 23 subjects and trailers pass)

The single warning is not this branch's: reproduced at this head with
`uv run pytest tests/test_config_raven_loader.py -q`, it is a `DeprecationWarning` raised deliberately by `raven/config/raven.py:1399` about the retired `skill_forge.mass_library_db` field.

This worktree symlinks `ui-tui/node_modules` to the main checkout, so the type-check was proved to have read this worktree's own files rather than another copy:
`npx tsc --noEmit -p tsconfig.json --listFiles | grep -E "dagStatus|dagPanel|dagRun"` returns three paths, all under `.claude/worktrees/feat+dispatch_task_summary/`.

The branch was rebased three times as `main` moved under it. The first rebase (onto `7d239509`) was verified as a pure replay with `git range-diff`: 14 of 15 commits report `=`, and the one marked `!` (the TUI commit) differs only in context -- its own changed line is byte-identical on both sides. The second rebase (onto `6fdf1d7b`) resolved two modify/delete conflicts by letting the deletion win: `main` deleted the packaged builtin playbooks, which this branch had edited to add the new fields, so those edits were dropped. Its `git range-diff` reports 14 of 16 `=` and the same two commits `!` for exactly that reason. The third rebase (onto `01d39e58`, a `ui/`-only frontend fix) has zero file overlap and replays all 24 commits as pure `=`.

One more change landed in the review round: `docs/specs/playbook-spec.md` was translated from Chinese to English (`77ea8fde`) under the project ruling that no file or code may carry Chinese. Structure is 1:1 (430 lines in, 430 out) and ASCII-verified; the semantic hotspots were re-checked after translation.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

`make lint-python` fails at this head with 3 `I001`/unused-import errors, in `raven/agent/tools/deliver.py`, `tests/test_channels_outlet.py` and `tests/test_mcp_client.py`. None of those files appears in `git diff --name-only origin/main...HEAD`; the first two predate this branch and the last was introduced by `3d3c8713`, a commit on an earlier base. They are inherited, not this branch's (one further inherited error was fixed by `6fdf1d7b` on `main` between the two rebases). Its own files are clean under `ruff check` and `ruff format --check`.

## Risk

User-visible behaviour changes:

- The `spawn` tool no longer accepts `label`; it requires `task_summary`. A caller passing the old name is rejected by the schema.
- A stored playbook that predates `taskSummary` loads with its summary backfilled from its `description` in the legacy migration (`store.py`); the field stays required everywhere new content is produced (the generation schema, the executor, the docs). A dag-mode node without `nodeSummary` loads and reports the blank as a fillable gap rather than dying at dispatch.
- Sub-agent rows in the TUI and the panel's dag rows are now named by the summary rather than by a guess taken from the first line of the prompt template. The heuristic remains as the fallback for runs that predate the field.

Rollback: revert the range. Nothing is persisted that a rollback would strand -- `graph.json` and the spawn record meta gain a key that older readers ignore, and both wire additions are optional fields absent from every `required` list.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Known gaps, deliberately not fixed here:

- The graph-level `task_summary` is write-only today. `SubAgentDagSpec.task_summary` is persisted into `graph.json` and read by nobody, and `PlaybookSpec.task_summary` dead-ends into it; only spawn's summary reaches handles, rows, records and announcements. The model is therefore obliged to write a line on every graph dispatch that no surface shows yet. Giving it a display is a separate change; `CONTEXT.md`'s entry states the current reality rather than the intent.
- Fixture summaries in `tests/test_subagent_dag_runner.py` restate the node id ("node a") for 34 of ~146 nodes rather than describing the node. Fixtures only, never asserted on.
- No regression test pins `SpawnTool.parameters` itself -- its required list, property order, `minLength` and description text. The python signature is pinned twelve ways in `tests/test_subagent_third_party.py`; only the JSON schema is unpinned.
- `_reader.py`'s emit of `node_summary` has no unit test. It is covered end to end by the RPC dag-row tests, which read a real `graph.json`.
- `docs/specs/playbook-spec.md` section 11's field index listed neither summary field and still wrote `agent` for `subagent`, and section 9's example YAML carried the same stale key and neither required field. All fixed here; the example is now a valid complete playbook.

Two rounds of base movement falsified claims in this branch's own design spec, and both were corrected the same way (keep the design intent, correct the current-state claim, re-verify the cited line numbers):

- `7d239509` changed `dagNodeToggleKey` so a dag row is expandable when it has a template **or** has started, where the spec said it still keyed off the template alone. All five `file:line` references in that section had shifted and were updated.
- `6fdf1d7b` deleted the packaged builtin playbooks. Three passages describing them as current were rewritten, and two further `file:line` references corrected (`executor.py:493`, `types.py:155`).
- `6fdf1d7b` also added `test_a_user_playbook_in_the_library_registers_the_loader` with an inline playbook fixture written against the pre-change contract. The required `taskSummary` made it fail at the rebased head (`taskSummary Field required`, reproduced); the fixture was updated to carry the field. A repo sweep of the same pattern found every other inline playbook block already carrying `taskSummary` or on an error path that never reaches the field gate.

## Related Issues

N/A
2026-08-25 08:25:09 +00:00
gloryfromca
64cb7e434f fix(*): land the upstream changes the sync could not deliver, and adapt the trunk
Follow-up to the v0.1.13 merge. Everything here is a consequence of that merge
that a clean three-way could not produce, split out so the merge commit stays
readable.

Upstream changes replayed onto renamed paths
--------------------------------------------

Nine paths upstream changed do not exist here, so their changes were merged into
files nothing reads. Each was replayed against the merge base:

* raven/rpc/{errors,models}.py, rpc/methods/{__init__,config,model,session}.py
  from raven/tui_rpc/*. question_broker.py came with the merge commit.
* rpc-schema/openrpc.json from ui-tui/rpc-schema/. `npm run lint:rpc` confirms
  the generated types still match.
* tests/test_rpc_{config,model,session}.py were rebased function by function
  onto upstream's version, keeping the 41 tests this trunk added. A three-way
  merge had spliced assertions into the wrong tests, which is why the seam is
  drawn at function boundaries here.

raven/agent/loop/streaming.py is the tenth, by a different route: it is this
tree's extraction of the loop's inline stream call, so upstream's change to stop
fabricating a terminal finish_reason had no counterpart to merge against. A
stream that dies without one now reports "unknown" instead of impersonating a
normal finish.

Two runtime bugs the merge introduced silently
----------------------------------------------

* web_rpc refused every per-session model. Its guard read
  agents.defaults.provider != "auto" as "pinned", and "auto" was the pin
  sentinel that went with providers/pin.py -- an unset provider is "" now, which
  the guard read as a pin to nothing.
* SubagentManager.provider and .model are properties over the fallback binding
  upstream introduced, and this tree's __init__ still assigned them. Every
  construction raised, taking 502 tests with it.

AgentLoop.refresh_context_window is gone with the wiring that called it: the
window rides on the binding, and LazyProvider.on_built has no production setter
upstream any more.

Tests adapted to this tree, not to make them pass
-------------------------------------------------

Each of these is a premise that the merge made false, with the behaviour the
test protects asserted where it still holds:

* test_tool_capabilities: upstream compares is_offered against registry
  membership, because there a disabled tool is unregistered at startup. This
  loop keeps it registered and withholds it per request, so what the agent is
  offered is "registered and not withheld" -- a helper says so once and both
  tests read it.
* test_agent_loop_session_model: the skills segment and its gate only exist
  under push discovery here, and this tree's build_executor takes an argument
  upstream's stub does not pass, which killed the spawn inside its own handler.
* test_trajectory_replay: upstream's tool stub predates channels and
  metadata_owner, which this registry and loop read on every call.
* test_ask_user_tool: a one-option question is refused now, so the fixture for
  the JSON-string coercion carries a real choice, and the lone-option-string
  case is asserted on the normalizer where it can still be observed.
* test_rpc_session: the monkeypatched guard is is_session_busy here, but one
  test exists to tell that apart from is_turn_active and keeps the latter.
* test_rpc_model: the premise that five providers carry no shortlist is false in
  this trunk, so the catalogue lookup it protected is asked directly.

Not from this merge: the eight test_start_webapp failures reproduce on
origin/main with byte-identical files, and its pipeline is green -- the local
bash rejects a construct CI's accepts.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 17:02:47 +08:00
gloryfromca
0e324a14d3 chore(*): merge the trunk into the v0.1.13 sync branch
Picks up the three commits that landed on main while the upstream merge was
being resolved. Named rather than left as git's default subject, which the push
rule rejects.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 17:02:47 +08:00
arelchan
a95302d88f perf(tracing): stop the viewer replaying all history per request
## Summary

Opening the tracing dashboard was unusable on a state dir with a few months of
history: the viewer rebuilt its entire snapshot on every request, and the page
re-asks every 5 seconds, so it was never idle. Measured on a 3.2 GB tree
(324k files, 102k spans, raven actively writing), a poll cost 7.6 s and the
response was 325 MB.

Four costs on the read path, each fixed here:

1. **The log reader recursed all of `logs/`** to find a few dozen `.log` files,
   walking the 324k-entry `audit-artifacts/` payload store on the way. Logs only
   ever land in the active file or `archive/<date>/` -- the only place either
   writer renames one to -- so the search stays inside those (600 ms -> 4 ms).
   Candidates are also stat'ed once each rather than inside a sort comparator.
2. **The deposit index walked the everos root per request**, which contains the
   engine's own virtualenv, and re-read every deposit. Dotted directories that
   hold no deposits are skipped and the index is reused while the tree is
   unchanged (433 ms -> 13 ms). The 54 `.md` files no longer visited are all
   pip-installed package readmes, which the entry parser already rejected.
3. **Nothing cached the snapshot.** The serialized body is now held, keyed on the
   inputs it was built from, and refreshed *behind* the response instead of
   making the reader wait -- a running raven appends spans continuously, so a
   poll that waits for the rebuild waits every single time. Only the body is
   retained, not the object graph it came from; retaining the graph measured at
   2.4 GB resident and grows with history, while the body is bounded by the
   payload. The refresh is deferred to after the response leaves the socket,
   because the rebuild is synchronous and would otherwise stall the write it was
   meant to skip.
4. **The tree nested a copy of every span** beside the spans array, so the
   payload carried the whole history twice. It now carries ids, depth and
   nesting, resolved against `trace.spans` in the page.

Search shared the disk cost and added its own: the result cap used an unlabelled
`break`, which caps nothing past the first trace. A broad term therefore matched
13k spans and read an artifact off disk for each -- 138k reads returning 802 MB
of text for one query. The cap now ends the scan, and artifact text is cached per
path under a byte ceiling.

Results, same tree and the UI's own polling cadence:

| | before | after |
|---|---|---|
| `/api/data`, first request | 7.6 s | 3.3 s |
| `/api/data`, subsequent polls | 4.8-7.6 s | 0.27 s |
| payload size | 325 MB | 171 MB |
| `/api/search` (`tool` / `memory` / `store`) | 19 s / 22 s / 72 s | 3.0 s / 2.6 s / 2.9 s |

Not addressed, and deliberately left out: the archives have no retention policy
at all, so the reader still reads the whole retained history on a rebuild. A time
window would cut this further but removes older sessions from the panel, which is
a product decision rather than a performance one. The remaining 2.6 s of a search
is the rebuild it triggers; removing that needs an index that survives rebuilds,
and two in-memory attempts were rejected here on measurement -- retaining the
searchable text of every span exhausted memory, and re-parsing the held body per
query was slower than rebuilding.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other (performance)

## Verification

Commands run, and what they reported:

- `uv run pytest -q` -- 11040 passed, 33 skipped, 2 failed. Both failures are in
  `tests/test_start_webapp.py` and reproduce unchanged on the pristine base
  commit, so they are not from this branch.
- `uv run pytest -q tests/integration/test_tracing_viewer_e2e.py -m integration`
  -- 6 passed. New file; spawns the real Node viewer against a synthetic state
  dir and drives it over HTTP.
- `uv run pytest -q tests/test_cli_tracing_commands.py tests/test_tracing_api.py
  tests/test_tracing_compact.py tests/test_no_otel_tracing.py` -- 86 passed.
- `uv run ruff check` and `ruff format --check` on the new test -- both clean.
- `make check-large-files` -- clean.
- Each of the six new assertions was mutation-checked: undoing the log scoping,
  dropping the archive subtree, removing the snapshot cache, pinning the
  fingerprint constant, nesting whole spans in the tree, flattening the tree,
  reversing the search sort, and removing the result cap each fail exactly the
  intended test and nothing else.
- The page itself was driven in a browser against the real tree: 181 session
  cards render, switching sessions redraws 267 span nodes nested to depth 3 with
  no missing fields, the content search returns 50 highlighted hits, and the
  console is clean. This matters because the tree change moves work into the
  page, which no automated gate covers -- `raven/tracing/viewer/*.js` sits
  outside every lint and type-check target in the Makefile.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Behaviour changes a reader can see:

- Data can be one refresh interval behind while raven is writing, by design. The
  page already polls, so it converges on the next tick; the e2e test pins that an
  appended span reaches a later poll.
- A log file placed under `logs/` outside the active file and `archive/` is no
  longer read. Neither writer produces one.
- A search for a broad term now returns the newest 50 of the first 200 matches
  scanned rather than of every match. The scan runs newest-first, so the result
  set is nearly identical, and the 200 cap was the existing intent.
- The viewer holds the serialized payload for as long as it runs -- roughly
  650 MB resident when idle on the tree measured here, against 60 MB before.
  `TRACE_ARTIFACT_CACHE_MAX_BYTES` bounds the artifact cache separately.

Rollback is a revert of this commit; the changes are confined to the viewer's
read path and add no state, schema or file format.

## Related Issues

N/A
2026-08-25 06:01:58 +00:00
gloryfromca
df9a7d7f37 chore(*): merge upstream v0.1.13 into the gitlab trunk
Absorbs the GitHub release into the downstream trunk. A merge rather than a
rebase so v0.1.13 becomes a real ancestor: the next sync starts from v0.1.13
instead of v0.1.12, and the eventual contribution back to GitHub can still tell
our commits from the upstream ones it already has.

Cost this round: 51 conflict hunks over 27 files, against 54 over 31 for
v0.1.12. The hunks were the cheap half. Upstream's #284 moves the model and
provider from process-wide state to a per-session binding, and the trunk had 230
commits standing on the old shape.

What git reported as a clean merge and was not
---------------------------------------------

Fourteen upstream-changed paths do not exist here, because we renamed or moved
them since v0.1.12. Rename detection found none of them, so their changes were
merged into paths nothing reads, silently:

* raven/tui_rpc/* -> raven/rpc/* (renamed in ce526ad5). Nine files, including
  question_broker.py, whose whole #354 rewrite was lost, and methods/config.py,
  methods/model.py, methods/session.py, models.py and errors.py. Each was
  replayed by three-way merge against dd82f52c. The three test modules were
  rebased function by function onto upstream's version instead, after a
  three-way merge spliced assertions into the wrong tests.
* raven/agent/tools/mcp.py -> raven/mcp/client.py. Carried the release-blocking
  fix, and is handled in the commit that follows this one.
* ui-tui/rpc-schema/openrpc.json -> rpc-schema/openrpc.json.

Decisions
---------

Follow upstream wherever the two disagree about provider and model:

* providers/pin.py is gone, so a bare model id is refused instead of derived.
  A model id does not name whose credential serves it, and the derivation
  walked a list -- with two vendors configured, which one pays came down to
  their order in it. rpc/methods/config.py now requires the provider.
* ModelSwitchInTurnError (-32009) is gone with it. A session switch lands on
  that session's next turn, so there is nothing to refuse mid-turn.
* AgentLoop.refresh_context_window and the LazyProvider on_built wiring are
  gone: the window rides on the binding now.
* The skill gate no longer forwards an unpaired pin.

Keep ours where the trunk is ahead, and graft upstream's behaviour into it:

* raven/agent/subagent/backends/raven_loop.py is the live sub-agent path, so
  manager.py's inline loop stays retired and the web_search gate from #312
  applies there.
* skill_hub/client.py keeps the public zip limits the skillhub RPC path imports,
  and takes upstream's query-string cap.
* ask_user keeps our JSON-string normalizers and feeds them into upstream's
  batch contract, so a batch validated by _prepare is one _normalize_questions
  has already coerced.
* The chat view composes both filters: hideIntroAfterFirstTurn drops the cover,
  visibleRows still hands the view to a direct chat.

Tests removed, not repaired
---------------------------

Eight tests covered mechanisms this merge deletes, and one of them hung rather
than failed:

* seven in test_agent_loop_model_switch.py over AgentLoop._pending_provider,
  the switch-parking mechanism #284 replaced with per-session bindings. Its
  replacement arrives with upstream as test_agent_loop_session_model.py.
* test_a_refusal_during_a_turn_keeps_its_own_code in test_acp_methods.py, whose
  error class no longer exists.

Baseline: the pipeline for 01d39e58, this merge's base, is green, so nothing
here can be charged to a pre-existing failure.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 16:12:43 +08:00
Handsome-wzw
a1d5b24de3 feat(agent): withhold web_search without a key, and make tool capabilities visible (#312)
## Summary

A WeChat user asked for a recommendation and the bot answered "the web
search
tool is not configured here". That sentence was not the model's own
phrasing --
it was the tool's error text, relayed outward. `web_search` needs a
Serper key
and that deployment has never had one, but the tool was registered
unconditionally, so on every search-shaped question the model saw it,
reached
for it, and passed the setup error on to whoever was in the chat.

Withholding the tool fixes that and creates a second problem: an
unregistered
tool is invisible. Nothing in a running Raven then says the capability
exists at
all -- the model is never offered it, no document lists it, and `raven
doctor`
reports on providers and memory but has never mentioned tools. The only
way to
learn that web search is one account and one edit away was to read the
source.

This does both halves, because either alone leaves a real gap: the
deployer is
told what this install can and cannot do, and the model is still not
offered a
tool it cannot run.

### Withholding the unusable tool

`web_search` is gated on a resolved key, in the main loop and in the
sub-agent
surface (`subagent/manager.py`) -- a sub-agent that reaches for a search
it
cannot run reports the failure to its caller, and that text lands in the
parent
turn, the same leak one level down.

The gate asks the tool, not the config. `WebSearchTool.api_key` resolves
at call
time from the constructor value *or* `SERPER_API_KEY`, so reading
`tools.web.search.apiKey` alone would withdraw a working tool from any
deploy
that exports the variable and configures nothing.

The tool's error message also hard-coded `~/.raven/config.json` while
the
gateway runs with `--config` elsewhere, so following it meant editing a
file the
process never reads. It names the path actually in force now. That text
is
reachable only if the key disappears after registration, which is
exactly why it
should be right: it is the message for the case the gate cannot cover.

### Three rules, and nowhere to read them

Registration is decided per family, each a different shape:

| family | rule |
|---|---|
| `web_search` | a resolved key, asked of the built tool |
| `web_fetch` | nothing -- always registered, a key only improves
extraction |
| media x3 | an `api_key` *or* a `model`, either counting as configured
|

For media those are two questions, not one. A section naming only a
model is
registered, because a model alone counts as asking for the tool, and
then every
call returns a missing-key error. Whether a capability is *offered* and
whether
it *works* come apart there, and a report treating them as one fact
ticks a
capability that cannot run.

Each rule is defensible where it sits. What is missing is anywhere to
read them.
Providers had the same sprawl once and answered it with
`providers.auth`: a
declarative table plus `credential_status` as the single authority, with
an AST
invariant enforcing that authority, because six surfaces had answered
the same
question six ways and each looked reasonable alone. Tools never got the
equivalent.

### What this adds

**Each tool answers for itself,** twice where the questions differ.
`WebSearchTool.is_configured` reads the config value *or*
`SERPER_API_KEY`,
because those are two sources and only the tool consults both. The media
base
answers `is_configured` on a model *or* a key -- which is what stops an
OpenRouter credential set for chat from silently switching on three
tools that
bill per call -- and answers `has_key` separately on the chain it
actually
resolves at call time: its own section, the borrowed provider key, then
`OPENROUTER_API_KEY`. Both rules live with the credential they read.

**`capabilities.py` describes the five** for a human deciding what to
set up:
what each does in one line, how much work it is (nothing / reuse a
credential
you already have / obtain an account), where the key goes, where to get
one, and
what it costs. It rules on nothing -- `is_configured` and
`has_credential` both
ask the tools -- so it cannot become a second opinion.

**`raven doctor` grows a section**, listing every capability configured
or not,
ordered by how much the deployer has to do:

```
(markers render as a green check and a yellow bang; spelled [ok] and [!] here to
keep this description ASCII, since it becomes the squash commit body)

Tool capabilities
  web_fetch:       [ok] Read a web page the agent already has the URL for
  image_generate:  [ok] Generate an image  (borrowed: providers.openrouter.apiKey)
  text_to_speech:  [!] Generate speech from text
                   no key resolves; calls will fail
                   set: tools.media.speech.apiKey
                   or env: OPENROUTER_API_KEY
  video_generate:  -  Generate a video
                   switch on: tools.media.video.model
                   key: reusing providers.openrouter.apiKey
                   Billed per call; needs prepaid OpenRouter credit.
  web_search:      -  Search the web
                   set: tools.web.search.apiKey
                   or env: SERPER_API_KEY
                   key from: https://serper.dev
  2 capability(s) available but not set up; the agent is not offered them.
```

Three deliberate details.

Naming the credential is load-bearing in both directions. A row that
cannot
distinguish a reused credential from a missing one sends someone to
create an
account they already have -- and a row that claims a reuse with nothing
to reuse
is worse, because acting on it means setting a model, getting a
registered tool,
and watching every call fail on a credential they were told they had. So
the
reuse line prints only when a key is genuinely there to pick up, and a
capability already in that broken state says so outright instead of
showing a
satisfied tick.

The credential is named at the path that holds it. For the media family
`config_path` names the *model*, so reusing it as the key source pointed
the
deployer at a line with no credential in it.

And each fact is on its own line rather than in a sentence, because the
terminal
wraps a long line mid-path and a config key broken across two rows
cannot be
copied, which is the only thing that row is for.

Nothing here moves the exit code, including the warned row. An install
without
image generation is a choice, not a fault, and a doctor that fails on it
teaches
people to ignore doctor. The half-finished one is arguable -- the memory
section
does exit non-zero for a role the user configured that the server could
not
build -- but that failure is silent where it happens, recall just
returning
nothing, whereas this one returns an error string to the model on every
call.
Say the word and it becomes an exit code instead.

### Not in this change

Registration still lives in `AgentLoop`. Having it read the table is the
point
of this shape and removes the last duplicate reader, but it edits
`agent/loop/main.py`, which is under active change, and it is worth
doing on its
own once that settles. Until then the table is a description, and the
tests
below are what keep it honest.

`deep_research` is deliberately absent: it is moving to the sub-agent
surface
and its tool is going away.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

Mixed on purpose: the first commit is a fix and the rest are the feature
it
made necessary. Splitting them would land a change that hides a
capability
without landing the one that makes it discoverable.

## Verification

```
make lint                                            exit 0
make build                                           exit 0
npm run test --prefix ui-tui                         969 passed, 13 skipped
uv run --all-extras pytest -q                        6186 passed, 35 skipped, 1 failed
uv run pytest tests/test_tool_capabilities.py \
              tests/test_cli_doctor_commands.py \
              tests/test_agent_loop_web_tools.py     64 passed
uv run --extra dev ruff check raven tests scripts    All checks passed
uv run --extra dev ruff format --check               828 files already formatted
npx commitlint --from origin/main --to HEAD          exit 0
scripts/check_commit_messages.py origin/main..HEAD   exit 0
```

The one failure is
`test_read_file_image.py::test_an_attachment_that_cannot_be_
read_costs_a_note_not_the_turn`, and it is not from this branch: it
fails the
same way on `main` at `1cb604a` with these commits absent. The case
makes a file
unreadable with `chmod 000`, which does not block a root user, so it
fails for
anyone running the suite as root and passes in CI. This branch does not
touch
that file or the code under it.

The capability tests drive a real `AgentLoop` and compare what it
registered
against what the table predicts, rather than asserting the table against
itself.
The gated set is derived -- tools present once credentials are supplied,
absent
without -- so a sixth gated tool whose author forgets the table fails
here rather
than going unnoticed.

That last claim was false when first written, and the last commit is
what makes
it true. The fixture listing the media tools by hand never switched a
fourth one
on, so a new one never joined the gated set and the assertion held over
an
already-incomplete table. #305, which adds a MiniMax voice-clone tool,
is that
case: merged against this branch the assertion passed. Read from
`MediaGenConfig` instead, it fails and names the tool -- `gated but
undeclared:
['voice_clone']`.

Which means whichever of the two lands second turns this red,
deliberately. The
fix is one table entry, and it needs the rule that PR settles:
`voice_clone`
counts as configured on `api_key or api_base or model` in
`effective_media_config` while registration still gates on `api_key or
model`,
so an `apiBase`-only install with no MiniMax key is configured by one
rule and
withheld by the other. That is the divergence this table exists to make
visible,
and it is worth resolving there rather than papering over here.

Fifteen mutations, each caught:

```
the web_search gate removed from the main loop    3 failed
the same gate removed from the sub-agent surface  1 failed
media rule reads only the key                     7 failed
web_search rule reads only the config             1 failed
a tool removed from the table                     6 failed
the doctor section not rendered                   6 failed
doctor lists only configured tools                4 failed
unconfigured treated as a failure                17 failed
the reuse line is printed unconditionally         1 failed
the key source falls back to the model path       3 failed
a keyless registered capability is not flagged    1 failed
has_key always answers yes                        1 failed
has_credential collapses into is_configured       1 failed
the credential chain drops OPENROUTER_API_KEY     3 failed
the reuse check ignores the environment           3 failed
```

Four of those tests exist only because a mutation pass found the earlier
versions insufficient, and the last rounds are why the shape changed. A
media
case that sets both a model and an OpenRouter key proves nothing about
the "or
model" half: the borrow fills the key in, so a rule reading only the key
still
answers correctly. The case that pins it has nothing to borrow.

The sub-agent case is the same argument applied to the second call site:
with
the gate present only in the main loop, every test above still passes,
because
nothing was watching what the sub-agent surface registers.

More usefully, mutating `has_key` to always answer yes changed no test
at all in
the first version, which said the ruling was not load-bearing: the
doctor was
inferring "no credential" from an *empty source string* rather than from
the
tool's answer, so a fourth credential source would have been reported as
a
missing one. `has_credential` is now its own fact, asked of the tools,
and that
mutation fails.

Separately, autouse fixtures clear `SERPER_API_KEY` and
`OPENROUTER_API_KEY`,
without which several of these pass for the wrong reason on any machine
where a
developer exported one.

Rendering was checked against a real config in six states -- nothing
set, a
provider key present, a model with nothing to borrow, a tool with its
own key, a
model plus a borrowable key, and an exported variable as the only source
--
rather than only asserted on.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

One behaviour change: `web_search` is no longer offered to the model
when no key
resolves. That is the fix. A deployment that has a key, in config or in
the
environment, is unaffected; one that has none was getting an error
string in
place of an answer.

The rest is additive. No other registration logic changed, so which
tools an
agent is offered is otherwise exactly what it was; `is_configured` moved
the
existing predicates onto the tools without altering them, and the
mutation
results above are what pins that. `has_key` and `_resolve_key` are new
names for
the chain `api_key` already resolved -- the property calls the extracted
one, so
callers see the same answers.

`raven doctor` gains a section and no new exit code. Its zero-network
guarantee
holds: the table reads config and environment only.

Two allowlist entries were added to
`test_only_the_auth_module_decides_configuredness_from_a_key`, argued in
place:
`subagent/manager.py`, which asks the built tool whether a key resolved
so an
unusable search is withheld, and `capabilities.py`, which reads keys to
*report*
-- which source supplied one, and whether one is there to reuse. Both
ruling
halves are delegated to the tools, whose files were already listed.

Rollback is a revert. Nothing is written and no configuration is read
differently.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-25 10:05:58 +08:00
arelchan
82eea1dd20 fix(ui): show Raven logo as page favicon
## Summary

- Declare the existing Raven application logo as the served page favicon.
- Copy the logo into the packaged page assets so installed builds render the same branding.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

- `npm test --prefix ui` - 32 test files and 528 tests passed.
- `npm run type-check --prefix ui` - passed.
- `npm run build --prefix ui` - passed.
- `python3 ui/build.py` - built the page and copied the Raven logo asset.
- `node ui/scripts/check-page.mjs` - passed.
- `make check-large-files` - passed.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Low risk. The change only adds favicon metadata and packages an existing application asset. Reverting the two source changes restores the prior browser-default icon.

## Related Issues

N/A
2026-08-23 11:11:06 +00:00
gloryfromca
56858acbce fix(*): report an invalid memory identity as the config error it is (#350)
## Summary

An identity EverOS cannot accept becomes a directory segment on its
write path, so `EverosBackend.start()` rejects it. The rejection carried
the one message a user can act on -- it names the key and the accepted
pattern -- and no surface delivered it as one. Three of the four call
sites fold the raise into `logger.exception`; under the TUI loguru is
file-only, so there an invalid identity turned memory off with nothing
on screen. The fourth, in `import_commands._build_and_run`, is not
wrapped at all, so the `ValueError` escaped as a bare traceback. Where
it was visible it was a traceback, which is not a thing a user reads as
"edit this key".

The state machine then told the user the opposite of the truth.
`start()` died before assigning a state, so `_state` stayed `UNKNOWN`,
which is not in `_NEVER_HAD_MEMORY`; every `store()` after it counted a
dropped write, and the session ended with:

```
1 turn(s) were not written to long-term memory because the memory service was
unavailable.
```

The service was fine. That sends the user to debug a server that never
broke.

**The change.** Catch the `ValueError` in `start()`, give it a
`BAD_IDENTITY` state, and file that state with the other two the user
must fix themselves:

- terminal, so a stray probe cannot promote it to `READY`;
- never-had-memory, so no dropped-write count and no outage wording;
- the message goes to stderr, matching what the unconfigured-LLM and
missing-binary paths beside it already do.

Two things came out of running it rather than reading it:

1. **The message needs escaping.** It quotes the accepted-character
class, and rich reads `[a-zA-Z0-9_.@+-]` as a markup tag and eats it,
leaving the user the pattern `^+$` to match their id against. The test
now pins the full class.
2. **`raven import` had the same defect on its own gate.** It kept the
outage wording and pointed at the server log, which holds nothing about
a config error. It now names the config keys instead.

Before / after, same config (`memory.userId: "bad/id"`), same command:

```
- 1 turn(s) were not written to long-term memory because the memory service was
- unavailable.

+ Long-term memory is off: memory.userId='bad/id' is not accepted by EverOS: it
+ becomes a directory segment on the write path, so it must match
+ ^[a-zA-Z0-9_.@+-]+$ and must not be '.' or '..'.
+ Fix memory.userId / memory.agentId in your config.json, then start a new session.
```

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Re-run after rebasing onto `d35e398c`, the current `main`.

```
uv run pytest tests/test_everos_backend.py tests/test_cli_import_commands.py \
              tests/test_everos_plugin_discovery.py -q
162 passed

uv run pytest -q
1 failed, 6478 passed, 43 skipped, 13 deselected

uv run ruff check <the four touched files>            All checks passed
uv run ruff format --check <the four touched files>   4 files already formatted
```

The one failure is
`tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare`. It
is pre-existing on this base: checked out `d35e398c` in a clean worktree
and ran `uv run pytest tests/test_cli_theme.py` there, same failure, 1
failed / 44 passed. It passes in isolation and fails when the file runs
as a whole, so it is order-dependent, and it is unrelated to anything
here. That file is untouched by this change.

Manual check, sandbox `RAVEN_HOME`, `memory.userId: "bad/id"`, `raven
agent -m "hi"`: prints the block quoted above, and the session no longer
ends with the dropped-write line.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Terminal-only wording plus one new enum member. No config, schema, or
protocol change; an identity that was accepted before is accepted now,
on the same code path. The only behaviour change for a valid setup is
none.

For an invalid identity the turn behaves as it did -- memory off, turn
completes -- and only what the user is told changes. Rollback is
reverting the single commit.

`ServiceState.BAD_IDENTITY` is new. Every consumer was checked:
`raven/cli/import_commands.py` (handled here) and
`tests/test_everos_plugin_discovery.py` (reads `READY` only).
`_may_spawn` allowlists `UNKNOWN` via `_SPAWNABLE_STATES`, so a new
member is non-spawnable without a change.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

Fixes #271

Co-authored-by: zhanghui <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-08-20 14:41:45 +08:00
ypflll
5de5754458 feat(context_engine): pull-mode skill discovery with a scent menu and find_skill
## Summary

Retrieved skills stop being pushed into the system prefix. Under the new
default (`skillForge.discovery: "pull"`), discovery works in tiers, each
cheaper and more frequent than the next:

- **Scent menu** (per fat turn, zero LLM): a rule classifier (length,
  function-word residue, character-bigram novelty against the recent user
  window; an anaphora opener needs near-total novelty to count as a new
  task) decides whether the turn warrants a lookup. One `router.select`
  over the same three sources renders a few `qualified_id: description`
  lines into the turn's user envelope -- wrapped untrusted, annotated with
  the query they matched, capped at 2s so a slow remote source costs a
  missing menu rather than a stalled turn. Lean turns and zero-hit
  lookups render nothing; the previous menu stays in the append-only
  history.
- **find_skill** (new tool): searches the library on the model's own
  terms through the same router. Its description carries the library
  size, the 16 SkillCorpus categories and the when-to-search moments, so
  the awareness that a large library exists is byte-stable prompt
  material instead of per-turn injection.
- **read_skill** (unchanged) fetches bodies by qualified id; the
  tool-failure-loop nudge now suggests one find_skill call when the tool
  is registered.

Why: the push pipeline spent two LLM calls per turn (rewriter + gate)
before the turn started, injected full bodies into the system prefix
(bytes that changed every turn, defeating prefix caching at the earliest
segments), and fixed the skill set at turn start although skill needs
arise per step. Pull removes the per-turn LLM calls and the
system-prefix churn, and lets a mid-task step fetch what it needs.

The push pipeline is kept intact behind `skillForge.discovery: "push"`
for deployments whose main model under-uses tools; tests that assert its
internals now pin it explicitly.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/` (excluding tests/integration and three
  pre-existing environment-dependent files) passed: 9052 passed, 0 failed
- `tests/test_context_scent.py` (new, 18 tests): classifier shapes, query
  construction, menu rendering/truncation/silence/untrusted wrapping,
  2s timeout degradation, find_skill result and error paths, nudge flag
- Real-engine smoke: factory-built assembler with a stub provider that
  raises on any LLM call -- a fat turn produced a menu from an on-disk
  skill via BM25, the envelope carried it, find_skill resolved through
  the exposed router, a lean turn stayed silent, and no LLM call happened
- Live end-to-end (OpenRouter, deepseek-v3.1 as the main model): with a
  planted skill whose body carries marker instructions, a task query led
  the model to call read_skill off the menu and follow the markers in its
  answer; a small-talk control turn made no skill tool calls

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Default-flip: upgraded deployments move from push to pull unless they set
`skillForge.discovery: "push"`. Behaviour change is fewer injected skill
bytes and no rewriter/gate calls; the known trade-off is that pulling
depends on the model's tool discipline (a menu line can be ignored where
an injected body could not), mitigated by the menu, the tool description
and the failure nudge, and verified live with a mid-tier model. Menu
content is retrieval-derived text and rides the envelope wrapped in the
untrusted markers. Rollback is one config line or reverting the commit.
A systematic skill-usage-rate evaluation (push vs pull over tasks with
known matching skills) is listed as follow-up work.

## Related Issues

N/A
2026-08-21 08:15:45 +00:00
arelchan
feacba682c refactor(ui): the session rail becomes a react island
## Summary

The session rail (left sidebar) becomes a React island: `ui/src/features/rail/` with types, a plain external store (draw/skeleton/markNew/remove/rename, fold state on the `raven.gui.grpfold` localStorage key), the Row/Group/RailApp components with classes and DOM shape copied byte-for-byte from the legacy renderer, and 13 tests.

Key decisions:
- `SESS`/`cur` stay shared page globals: ten demo/live files write them (including whole-array rebinds), so the island owns no rows. It reads through `DS.sessions.snapshot()`, a closure over the rebindable bindings, re-read on every `drawList()`.
- Island actions go out through 9 new optional bridge verbs (`drawList, setCur, openSession, removeSession, renameTitle, dropDraft, pinPersist, openCron, navState`), each one a guarded one-liner in 155-bridge.js and late-bound so live rebinds keep winning.
- `plainTitle` moved into the bundle and is republished as `window.plainTitle` for its 6 legacy callers (same pattern as `cronExprHuman`).
- Shrink: demo/050-rail.js 222 -> 16 lines; live/010-boot-guard.js raw-DOM skeleton painter became one `RavenIslands.rail.skeleton()` call; live/030-sessions.js untouched (already renderer-free, its helpers are read by other domains).
- Pre-existing bug left as-is: live/080-overrides' rename persist wrapper queries `#title input` after the h1 swap and throws today; the island reproduces the demo DOM dance exactly rather than patching another domain in this MR.

Ratchet: 57 -> 57 (the strands touching this domain remain demo-declared/live-assigned by design; nothing new added).

Note: pipelines are currently red project-wide with `ci_quota_exceeded` (shared-runner minutes exhausted). Local gates below are the same commands CI runs; the pipeline should be retried once quota recovers.

Review round: the fake shell handed `markNew` an empty nav and a folded More group, so both of its branches were dead in every test -- exactly the part of the port that changed shape (`Object.keys(NAV_OF)` to `nav.pages`, `$('#' + p)` to `getElementById`, `MORE_ROWS[i][0]` to `nav.morePages[i]`). Two cases added with the real nav: a module page up marks its own rail button over the new-task row, and an open More group puts the mark on the row whose page is up while the parent gives it up and takes it back when folded. Both fail against a wrong port.

## Type

- [x] Refactor

## Verification

- `npm run --prefix ui type-check` clean
- `npm test --prefix ui` 49/49 (13 new rail tests)
- `npm run --prefix ui build` (223.95 kB IIFE) and `python3 ui/build.py` (1,094,616 bytes)
- `node ui/scripts/check-page.mjs` OK; `node ui/scripts/count-shared-globals.mjs` 57/57
- `uv run pytest tests/test_ui_language_repaint.py` 9/9
- Stub browser pass (`?stub=1`): DOM byte-compared to legacy (.grp/.sess shapes, aria-current moves, fold persistence, search, pin+persist, delete+undo toast, rename via top bar, running mark lifecycle, More flyout marks, zh/en repaint)
- Live pass against a throwaway gateway (temp RAVEN_HOME, dummy keys, nonce auth): boot shows exactly 6 skeleton rows with the legacy width formula while `data-counts="pending"` holds, then the real list swaps in; draft's first send creates a real session; two-session switching moves a single mark; second boot renders with no mock flash
- Not exercised live: a successful provider turn and parked done/err badges (needs a real key); covered by unit tests and the stub replay

## Risk

User-visible behavior is intended to be identical; the rail DOM shape and class names are unchanged. Rollback is reverting the single commit.

## Related Issues

N/A
2026-08-20 06:17:21 +00:00
arelchan
e26e0e1b12 feat(agent): bind the model to the conversation, not the process (#284)
## Summary

The model was two attributes on a process-wide `AgentLoop`, so there was
one answer for everyone: two sessions could not run different models, a
switch in one moved all of them, and the "default" was whatever the last
switch happened to write to `agents.defaults`. This makes the model a
property of the conversation, and -- following the same thread down --
makes the credential that serves it something the user says rather than
something Raven infers.

A `ModelBinding` is a model id, the credential that serves it, and how
much that model can hold, as one value. `ProviderPool` is the single
place deciding which credential a model id pairs with, cached per
(vendor, model). `run_turn` resolves the binding for the turn's session
and holds it in a `ContextVar` for the whole turn tree; the loop's
`provider` / `model` / `context_window_tokens`, the context engine's
LLM-backed segments, the skill gate and rewriter, and the consolidator
all read that instead of a reference of their own.

| Rule | How it lands |
|---|---|
| Different sessions, different models | A dict of overrides, read once
at turn entry |
| A switch moves only the session that asked | Nothing else reads that
session's entry |
| A new session starts on the configured default | A session-scoped
switch does not write `agents.defaults` |
| A configured subsystem uses its own model+credentials; otherwise it
follows the conversation | The factory resolves each pin through the
pool, so a holder has either a complete pair or nothing |
| A switch mid-turn takes effect next turn | Free -- the turn holds the
binding it entered on |

Detached work inherits the context copy `asyncio` makes at task
creation, so a subagent finishes on the model it was spawned under.

### The window belongs to the binding

Sizing came to a head during the rebase. `refresh_context_window` and
its cascade assumed the loop has one current model -- which this PR
makes false, since two sessions can be on a 200k and a 1M model at once,
and one int on the loop cannot answer for both. That is the same
mis-sizing #341 removed, arriving through another door.

So the window moved onto `ModelBinding`, beside the credential, and
every holder reads the binding of the turn it is running under.
`refresh_context_window` is gone: a turn now enters on a binding that
already knows its own size, so there is nothing to refresh.

It resolves on first read rather than at construction, because building
a provider is what imports LiteLLM -- a lazily built one has not done it
yet, so an eager window is the catalogue-miss default for every model,
which is the defect `on_built` existed to paper over. A miss is
deliberately not cached, so a cold read cannot make the process wrong
for its lifetime. An explicit `contextWindowTokens` travels on every
binding, so a pinned number survives whatever model a session switches
to.

### The provider is a word the user says

A model id does not name whose credential serves it. `openrouter`
serving `anthropic/claude-haiku-4-5` and `anthropic` serving
`claude-haiku-4-5` are both real, name different keys, bill different
accounts, and look identical on the wire. Everything that used to fill
in the blank was guessing which account pays.

`agents.defaults.provider: "auto"` was the same guess at the config
layer, and it did not detect anything: a prefixed id was answered by the
provider it names, but a bare id fell to keyword matching in `PROVIDERS`
order and took the first configured claimant -- so with anthropic and
openrouter both keyed, `gpt-4.1` went to openrouter over openai because
openrouter sits third in that list and openai eighth.

- `/model <id>` is refused, and the refusal names the two ways out:
`/model` for the picker, or `/model <provider> <id>`. A prefixed id is
refused too -- a prefix is LiteLLM routing syntax, not evidence about a
credential.
- `config.set model` refuses without a provider, which is the boundary
where the rule can be enforced rather than asked politely.
- `raven provider use` requires `--provider`, and names it alongside
`raven provider list` in the error.
- `--default` is not an exception: changing what new sessions start on
is the same choice about the same thing.
- `agents.defaults.provider` carries no default, and a one-shot
migration resolves each pre-rule config through the old derivation and
writes the answer into it. Behaviour is unchanged by construction -- the
value written is what that config was already getting -- but it stops
being inferred. It reuses the sidecar watermark from #341, with a floor
per migration rather than one shared mark, so raising the generation for
this one does not re-open the previous one. A config the derivation
cannot answer for is left blank rather than filled with a vendor chosen
to have something there.

The picker already asked in this order -- provider, then model, with a
free-text row for an id no catalogue lists -- so it needed nothing. Both
help strings now carry the usage, because a rule the user meets first as
an error is a rule we chose not to tell them.

With nothing left deriving, `providers/pin.py` is deleted: its whole
subject was "which provider to pin when the user changed the model
without naming one", and its last caller was the onboarding wizard,
which asks for the provider before the model and therefore always had
one in hand.

### Config defaults

No subsystem ships a vendor default. `context.curator_model`,
`token_wise.tool_result_lifecycle.summary_model` and
`skill_forge.detect_model` all hardcoded the same Gemini id, and
`token_wise.smart_routing.tiers` shipped six models across three vendors
-- for users who may hold no key for any of them. All are unset now,
which is what "not configured" has to mean for the subsystem rule to be
expressible.

`context.curator_model` and `skill_forge.llm_gate_model` took a model id
and nothing else, so the pool had to guess which credential served it.
Each pin now takes a provider alongside the model (`curator_provider`,
`llm_gate_provider`). With the provider named, a vendor whose
credentials are unusable is logged and dropped rather than silently
borrowing the conversation's key. With it unset the pin still binds: a
configured gateway takes it, because a gateway serves whatever id it is
handed under its own credential, and only without one is the vendor
derived from the id. That branch is unchanged from main and is
documented as it behaves rather than as the rule would prefer -- an
earlier revision of this description and of `CONTEXT.md` claimed the
drop applied there too, which was wrong.

**Release note.** With `context.curator_model` unset, the Curator's slow
path runs on the conversation's model instead of failing on a Gemini id
nobody had a key for and dropping to the deterministic plan. That is the
rule working as asked, but on a long conversation it is up to 12
tool-calling requests per turn of context housekeeping that previously
cost nothing. Set `context.curator_model` (and `curator_provider`) to a
small model to keep it cheap.

### Review findings

**Round 1 (pre-rebase), five findings.** A pin naming a vendor with no
spec crashed the agent at startup -- fixed on main, which now passes the
provider name separately instead of reading it off a spec that may be
None; verified with the reporter's own repro. `/model` before the first
message no longer manufactures a zero-message session record
(`session.create` is lazy, and `session.title` guards the identical case
the same way). Every rpc call in the TUI's session commands has a
`.catch` -- without it, `/model` typed before the first `session.create`
resolves wrote a raw stderr line over the Ink render while the
transcript showed nothing. The remaining two are the picker's provider
row and the gateway pin; see below.

**Round 2, three before-merge findings, all reproduced here before being
fixed.**

- The migration watermark was one high-water mark shared by every
stamped migration, so raising it to 2 for the provider migration re-ran
the context-window one on every config already stamped at 1 -- deleting
a `contextWindowTokens: 65536` a user had put back by hand after our own
notice invited them to, and printing that invitation again. Each
migration now has its own floor. The test that should have caught it
built its precondition from `CURRENT_CONFIG_VERSION`, so it moved with
the bump; it now pins the literal a shipped build wrote, and two tests
cover the upgrade-from-an-older-generation path in both directions.
- Deleting `_adopt_provider` took its `_image_tool_result_ok.clear()`
with it. Both capability caches key on a model id but are computed from
the provider, so an `apiBase` repointed at a box with different
capabilities, or a re-authenticated provider, kept the old endpoint's
verdict for the life of the process -- images silently dropped from tool
results with nothing in the log. Both binding setters now clear them.
- `--default` had no working path through the TUI. The picker footer
taught `/model <name> --default`, which is the bare-id spelling the
parser refuses; the refusal dropped the flag from the suggestion it told
the user to follow; and `/model --default` opened the picker, whose
selection sent a session-scoped switch that looked like it had changed
the default. The flag now rides into the overlay state and back out
through the picker's callback, the refusal keeps it, and the footer
writes the spelling that works.

**The two carried from round 1 are fixed here rather than dissolved.**
An earlier revision of this description said the work had dissolved
them; it had not, and both were still reproducible.

- `model.options` re-derived the provider from the id. For a vendor with
a spec that now reads the prefix `stored_model_id` writes, but
`find_by_model` answers None for a passthrough vendor (mistral, xai), so
the picker starred `agents.defaults.provider` for a session running on
someone else's key -- and the starred row is exactly what a user reads
to answer whose key is paying. It now reads the head of the stored id,
and stars nothing when that names no configured provider.
- The gateway pin is documented as it behaves; see the Config defaults
section above.

**Round 3, after the rebase onto a main that gained #346.** The conflict
was additive and is
described in the thread; two follow-on commits closed the two
nonblocking notes that survived it.
`modelPicker.test.tsx` was rendering `<ModelPicker>` without the `scope`
prop this branch made
required, so the `scope === 'default'` footer had no coverage -- and
`tsconfig.json` excludes
`src/__tests__`, which is why `tsc --noEmit` was clean and silent about
it. And `onModelSelect`, the
callback that carries `--default` back out of the picker, had no test;
the rule it applies is now its
own function, `modelSelectCommand`, because what a selection becomes on
the command line has to
satisfy three separate constraints in `parseModelArg` at once.

**Also taken, each a defect in what this PR itself added or removed:**
`clear_session_binding` was gated on `SessionManager.delete` returning
True, which it does not for a session that switched model before its
first save, so the entry leaked for the life of the process;
`_migrate_auto_provider` validated its probe against a strict `Config`
before the shims that relocate legacy top-level blocks, so a config
still carrying `skillRouter` failed the probe, was skipped silently, and
was stamped anyway -- never retried, on exactly the oldest configs most
likely to still say `auto`; `providers/lazy.py` and
`AgentLoop.set_provider` carried docstrings naming methods and callers
this PR deletes; onboarding still wrote `provider="auto"` when clearing
a removed provider's default; and `ui-tui/package-lock.json` is reverted
to main, its only change being npm-version drift in `libc` metadata.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
pytest tests/                     6700 passed, 33 skipped, 13 deselected   (TERM=dumb)
ruff check / format --check       clean, 832 files formatted
npx tsc --noEmit (ui-tui)         clean
npm test (ui-tui, vitest run)     998 passed (86 files)
npm run lint:rpc                  generated.ts in sync
commitlint + check_commit_messages.py   clean over github/main..HEAD
```

`tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare` is
`TERM`-dependent, not
ordering-dependent: it fails under an interactive `TERM` and passes
under `TERM=dumb`, which is what
the `coverage` target pins and what CI runs. That file is not touched by
this PR.

Every fix in rounds 2 and 3 was mutation-checked rather than assumed:
reverting the migration floor,
the cache clear, the flag pass-through, the passthrough-vendor fallback,
the picker's `scope` prop
and the `--default` command-building each turns the corresponding new
test red.

Driven by hand in an isolated home against the built TUI bundle, not
only by tests: `/model
claude-opus-4-5` refused with the two ways out; `/model
openrouter/anthropic/claude-opus-4-5` refused
as well; `/model openrouter claude-opus-4-5` applied; `/model` opening
the picker at the provider
level. A real config carrying `provider: auto` and a bare
`claude-opus-4-5` migrated to `provider:
openrouter` -- the vendor it was already resolving to -- with the notice
printed and the watermark
written to the sidecar.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

User-visible behaviour changes, all deliberate:

`/model <id>`, `config.set model` without a provider, and `raven
provider use` without `--provider` now fail where they used to guess.
That is the point, but it is a workflow change for anyone who typed a
bare id -- the errors name the fix, and the picker needs nothing new
learned.

`agents.defaults.provider` is written into every config that did not
have one, on the first launch after this lands. The value is what that
config was already resolving to, so no request changes vendor; what
changes is that it is now visible and editable.

With `curator_model` unset out of the box, the Curator's slow path
starts actually running (see the release note above). Cost, not
correctness.

One contract tightening is not backward compatible on its own:
`config.set key="model"` now rejects a call with no `provider`, and the
`-32009 model_switch_in_turn` code is removed from the error table. The
in-repo TUI moves with it, but any client outside this repo that sends a
bare model id starts failing on the first request after this lands.

Rollback: revert the commits. The config keeps an explicit `provider`,
which every published build accepts and treats exactly as it treats a
hand-written one. The `config.migrations.json` sidecar is left behind
and every build since #341 does read it, so a reverted build sees
`{"version": 2}` and treats the config as fully migrated -- which is
correct, since it is. Delete the sidecar to force the migrations to be
reconsidered.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-19 11:16:26 +08:00
arelchan
e32f4b6034 fix(*): keep migration notices off stdout, and stop planting defaults (#346)
## Summary

Three follow-ups from #341, which fixed a retired `contextWindowTokens:
65536` capping every upgraded install at 64k. Two are its loose ends;
the third is the reason it happened at all.

**The notice no longer prints on stdout.** #341 drains config-migration
notices in the `run()` wrapper so every command tells the user, which
caught the two whose stdout is a machine-readable document: `raven
doctor --json` and `raven import --json`, both advertised for
automation. Reproduced -- a config still carrying the retired pin makes
`doctor --json` emit JSON with a sentence after it, and `jq` refuses the
result. Moved to stderr, where the rest of this class of message already
goes (the `ConfigReadError` branch a few lines up in the same wrapper,
and the loader's malformed-config warning). That fixes any future
`--json` command without it having to know this exists, which
allow-listing the two would not. The window is narrow -- the migration
fires once per install -- but the run it can spoil is a first run after
upgrading, and a CI job is exactly the caller not watching.

**`raven doctor` gains a Config section and a `--fix`.** The premise
changed while building it: the migrations run at load, so by the time
this command looks, nothing is pending -- they already happened,
silently, one launch ago. What is left for a person to ask about is what
Raven deliberately will not decide for them. So it reports a
`contextWindowTokens` pinned below what the configured model holds, and
names what rides on it: the per-turn history budget, when the Curator
starts paying for a slow path, and when memory consolidation archives.
`--fix` removes the pin.

Reported without `--fix`, never written: that pin is a real
configuration for a local model or an endpoint served smaller than the
catalogue thinks, and the flag is the user saying which case theirs is.
The write carries the file mode across (config.json holds
`providers.*.apiKey`) and uses a per-PID temp name -- the same rules the
loader's migration writer follows.

One check rather than two. A config naming no provider is the other
thing worth asking about, but it reads differently on either side of the
explicit-provider rule in #284, so it belongs to that change; a model
that routes to nothing is already reported above it.

**The bootstrap stops writing every default into the user's config.**
This is the cause of the bug #341 cured. `save_config` dumped the whole
`Config()`, so a new install started life with eight kilobytes of
settings its owner had never heard of and, worse, with that day's
defaults frozen into their file -- after which a default we improve
never reaches anyone who already has one. `contextWindowTokens: 65536`
got onto disk exactly this way.

It now writes only what differs from the defaults. Lossless on reload,
since a value equal to its default reloads as that default. A fresh
config drops from 7,985 bytes to 2, and to 634 once the extension blocks
are seeded -- what is left is the handful of lines someone actually
chose, which is also a config they can read.

That third one is the only change here with a broad blast radius, and it
has one visible consequence, which is an improvement: the onboarding
picker used to lead with the schema's default model, because the
bootstrap had written it into the config and the wizard read it back as
"the current model" -- a value nobody had chosen, offered as though
somebody had. With nothing written, the wizard leads with the provider's
own recommendation instead. The test that pinned the old list now pins
the new one and says why.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
pytest tests/                       6650 passed, 1 failed, 33 skipped
                                    (tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare,
                                    which fails identically on github/main when its file runs
                                    together and passes alone on both)
ruff check raven/ tests/            All checks passed
ruff format                         clean
```

Driven by hand as well as by tests, each against a throwaway home:

```
doctor --json on a fossil config   stdout parses as JSON again; the notice is on stderr
doctor    (pin 65536, model 1M)    reports the pin and what is sized against it
doctor --fix                       pin removed, file still 0600, a second run reports nothing
save_config on a fresh Config()    2 bytes; 634 after the extension blocks are seeded
```

New cases cover the stderr split, both doctor paths including the mode
preservation, and the save round-trip keeping every value the user
chose.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

The bootstrap change alters what every new install's config file looks
like: nearly empty instead of a full dump. Existing configs are
untouched -- nothing rewrites them -- and both shapes load identically,
so the risk is confined to anyone reading `config.json` as documentation
of what can be set. `raven doctor` and the schema remain the answer to
that question.

The onboarding picker's default moves from the schema default to the
provider's recommendation, as described above.

`doctor --fix` writes only when asked, and only removes a key.
Everything else here is output routing.

Rollback: revert the commits. Nothing persists a new format, so a
reverted build reads every config written under this one.

## Related Issues

N/A

---------

Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-18 18:55:29 +08:00
LivXue
32f8044e86 feat(*): direct chat with a sub-agent instance, in the TUI and the web UI
## Summary

Adds direct chat with a sub-agent instance: talking to one of the instances a
session has spawned, without that exchange entering the main agent's context.
The TUI gets the feature end to end; the web UI then gets the same thing on the
same runtime.

**The runtime.** A direct turn runs on its own lane (`session#agent/handle`)
rather than the session's, so several instances answer at once while the main
agent keeps its own serial lane, and direct turns draw on a separate
concurrency pool so a user is never queued behind them. Direct turns are not
cancellable: `turn.cancel` keeps meaning the main agent's turn on both
surfaces. Each turn writes a record (`prompt.md` / `out.md` / `meta.json`), and
the main agent learns only that the exchange happened, through a handoff block.

**Sub-agent replies over ACP.** Three separate causes were losing most of a
reply. Raven answered every agent-initiated request with `method not found`,
including `session/request_permission`, and the codex adapter turns any error
from that request into a cancel -- so a turn died as soon as the agent asked to
run something. Raven now approves, choosing among the options the agent offered
by their protocol kind. A stop reason other than `end_turn` was returned as a
finished answer, so a truncated reply read as a complete one; it now carries a
notice. And several messages in one turn were joined with no boundary, so a
preamble, its tool calls and the answer ran together as one paragraph.

**Instance visibility.** A failed resume deleted the whole registry row, which
took the instance off the strip for the length of the turn recovering it, and
permanently if that turn failed; it now drops only the stale binding. A
capability snapshot whose fingerprint no longer matched was discarded, which
defaulted the agent to *stateless* -- editing an agent's `env` silently cost it
resume, its chip, and the `instance` parameter in the spawn schema. Its
capabilities are now kept and flagged, while its verdict is not, so the
`/subagents` row asks for a re-test.

**Working directory.** A direct chat was the one sub-agent dispatch that
supplied none, so it fell back to agent home and ran its commands against
raven's own memory instead of the user's checkout.

**The web UI.** Two wiring gaps came first, both silent: the browser's
dispatcher registered neither instance method, and the web channel never built
the map that tags an event with its addressee, so nothing a client could
demultiplex on was ever sent. The service then routes on that tag at the push
rather than at the read, because the main reply and a direct turn share one
subscription and two consumers on one queue would each swallow the other's
events -- including a `message.complete`, which would end the wrong turn. A
direct turn's events are published out of band; feeding them into the reply
stream is not an option, since that stream is one reply per turn.

Decisions and their measurements are recorded in `docs/specs/`. Notably, the
vendored AgentScope tree is *not* edited, which is a decision rather than an
omission: making a direct chat a real reply would mean re-keying a blocking
per-session lock that guards message storage and tool-call state as well as the
reply loop, and rendering reuse was obtained without it.

## Type

- [x] Feature

## Verification

- `uv run pytest -q` -- 8021 passed, 36 skipped. The one deselected failure,
  `tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare`, fails the
  same way on `origin/main` in a clean worktree, so this introduces none.
- `uv run ruff check raven/ tests/ ui-webui/service/` -- clean;
  `uv run ruff format --check` -- 957 files already formatted.
- Web service: `python -m pytest ui-webui/service/tests -q` -- 13 passed.
- Frontend: `pnpm lint` -- 0 errors, 18 warnings, each one also present before
  this change; `pnpm build` -- succeeds; `tsc --noEmit` -- clean.
- Every fix here was checked by reverting it and watching its test go red. Two
  tests that passed either way were rewritten after that check exposed them.
- End to end against a running gateway: a direct chat sent through the new web
  route completed twice, each leaving a record whose `out.md` held the exact
  reply asked for, and the history route returned those turns afterwards.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The last box is unchecked: the terms are defined in `CONTEXT.md` and the designs
in `docs/specs/`, but no end-user documentation or screenshots were produced.

## Risk

**Approving sub-agent permission requests is a widened trust boundary and the
main thing to review here.** An ACP sub-agent that asks to run something now
gets a yes, and the codex preset is configured for its most permissive mode --
wider than the `-a never` the cli transport used. It is unattended by
construction: a dispatch has no operator and no surface that could render a
prompt, so the alternatives were approving or refusing, and refusing leaves the
agent unable to do the work. Rolling back is per-agent config, or reverting
`raven/agent/acp/permissions.py` to restore the refusal.

Two known gaps, neither reachable today, both stated rather than fixed:

- The web UI's event consumer has no `target` guard of its own. The two
  surfaces use disjoint session keys and the browser never sends a target, so a
  direct turn cannot reach it; a caller that sent one on a `web:` session would
  see a sub-agent's reply rendered as the main agent's. Fixing it would widen
  this into unrelated files.
- Direct-chat record directories have no TTL and no count cap, like the spawn
  and DAG trees beside them. Reclaiming them would also start deleting audit
  trails that intentionally outlive session deletion, which is its own change.

Behaviour changes worth naming: a direct chat under a sandbox whose volumes do
not cover the working directory now fails where it used to run quietly in agent
home, matching what the main path already does; and a partial reply now carries
a `[raven]` notice, so anything parsing a sub-agent's reply verbatim sees it.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-17 02:52:57 +00:00
Elliot Chen
9bc959d7c5 docs: redesign README around benchmark proof (#280)
## Summary

- Reframe the README around benchmark evidence, onboarding, Deep
Research, and Tracing.
- Restore the main branch's opening product capability overview before
the benchmark evidence.
- Keep the table of contents removed and place the native Markdown
benchmark table directly after the introduction.
- Remove outbound links from the three benchmark row labels while
preserving the reported results and supporting context.
- Synchronize the English and Chinese READMEs around the same structure,
spacing, and Back to top section boundaries.
- Place the GitHub User Content onboarding recording directly below the
benchmark table in both READMEs, using the same bare attachment URL.
- Reduce duplicate capability, gateway, status, and ecosystem copy while
preserving concise architecture and command references.
- Document current runtime behavior for the six-step onboarding flow,
MiroThinker-backed research delivery, and local trace storage.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [ ] Relevant tests pass locally
- [ ] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Commands and results:

- `.venv/bin/pre-commit run --files README.md README.zh-CN.md` - passed
all applicable hooks.
- `git diff --check origin/main...HEAD` - passed.
- `make check-large-files` - passed.
- `git merge-tree --write-tree HEAD origin/main` - passed with no
conflicts.
- README structure assertions - passed: no table of contents, no
benchmark-label links, and four matching Back to top boundaries in each
language.
- Confirmed both READMEs use the same 33378-byte GitHub User Content
hero banner; benchmark evidence is native Markdown.
- Confirmed the 1179517-byte onboarding MP4 round-trips from GitHub User
Content with SHA-256
124024e6d64c23b3e314aa6136d9e0d922f57ded97a830a5c97cd1323cb2c57f and
renders with controls below Benchmarks in both languages.
- `uv run pytest --tb=short -q` - 6130 passed, 33 skipped, and one local
Python 3.14 ANSI color assertion failed; the GitHub Python 3.12 unit job
passed.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

This is a low-risk documentation-only change. Rollback requires
reverting the README commits; externally hosted user-content assets can
remain unreferenced without affecting repository size.

## Related Issues

N/A
2026-08-12 21:02:13 +08:00
0xKT
8700656155 chore(*): merge upstream v0.1.11 into the gitlab trunk
Absorbs the GitHub release into the downstream trunk. A merge rather than a
rebase so v0.1.11 becomes a real ancestor: the next sync then starts from
v0.1.11 instead of the July fork point, and the eventual contribution back to
GitHub can tell our commits from the upstream ones it already has.

Notable conflict resolutions, 19 hunks over 14 files:

* providers/litellm_provider.py takes upstream's wire_model delegation. Both
  sides had fixed the same gateway-prefix bug; upstream moved the logic into
  providers/wire.py and deleted _canonicalize_explicit_prefix, so keeping our
  call would reference a method that no longer exists.
* cli/onboard_commands.py takes upstream's three-way file split. Our only
  change to that file was _NON_CREDENTIAL_CHANNEL_FIELDS, ported to its new
  home in cli/onboard_channels.py.
* agent/loop/main.py keeps our stream reconnect loop and grafts upstream's
  terminal error-delta handling into it; takes upstream's _route_result_images
  helper, which also covers the no-vision case our inline version did not, and
  keeps our read_skill/use_skill reporting; takes upstream's ordering that
  routes before assembly, since assembly decides whether an attachment is
  inlined as a picture, while keeping our per-session model override.
* agent/subagent/manager.py keeps our backend dispatch and spawn history, and
  threads the provider/model pair spawn snapshots into backend.run so a live
  /model switch reaches subagents. The SubagentBackend protocol gained the
  pair as optional keywords, travelling per call for the same reason workspace
  already does.

Three breakages git merged cleanly and never flagged:

* web_rpc/methods_config.py imported six EverOS symbols from onboard_commands,
  which upstream moved to onboard_everos.
* tui_rpc/methods/turn.py passed a single directory to _resolve_path, whose
  third parameter we had changed to a tuple. Left as it was, an attachment was
  dropped in silence whenever restrict_to_workspace is on.
* config/schema.py carried a second configuredness rule next to the one
  upstream routes through providers.auth, so routing bypassed the
  endpoints-first check and read a flat key that no request would use. It now
  delegates, with include_external for parity with list_providers.

Verification: uv run ruff check passes; uv run pytest reports 6823 passed and
4 failed. All four fail on origin/main as well (one theme assertion, three
channel QR cases) and are left for a separate fix. Upstream's own 53 changed
test files pass, and so do our 85, which is what pins both sides' semantics
through these resolutions.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-12 23:59:07 +08:00
LivXue
2a6f46c1eb feat(*): bring the ravenx core work onto the shared trunk
Land the accumulated core development from the ravenx workspace as a
single change on top of origin/main. It covers the gateway web-app RPC
channel, pluggable sub-agent execution backends with third-party CLI
and OpenAI-API presets, the sub-agent DAG subsystem and its TUI
surface, vendor-aware provider resolution, the deliverable token store
and the deliver_files tool, and the skill tool gating rework.

The web UI itself is deliberately excluded. ui-webui/ and
start_webapp.sh land separately, together with the lint ignores and the
third-party notice that reference them, because the repository asset
policy for that tree has to be settled first.

Condensed from 68 commits whose history predates this repository's
linear-trunk rule; the three-way merge against origin/main applied
without conflicts.

A follow-up commit fixes a stale assertion that arrived with the import:
read_local_body grew hub-shaped metadata and its test still described
the older four-key return.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-06 09:44:24 +00:00
0xKT
5719cbe537 feat(providers): endpoint failover and provider path close-out (#287)
## Summary

Closes out the provider path in one PR: the provider module redesign,
every
open issue and backlog item that lives on the identity / connection /
routing / capability axes, and multi-endpoint failover as a new feature.
78 commits, one problem per commit, rebased onto current main.

Redesign (base): four provider decisions were implemented outside
`raven/providers/` and had drifted copies -- wire form, credential
grammar,
pin resolution, price/window ladder, cache dialect. Each now has one
owning
module the surfaces call.

Fixes on that base, most visible first:

- The context window walks one ladder (explicit config > the model's
real
  window > documented fallback). Previously the config default 65536 fed
trimming and budgets raw, so a 200k-window model lost two thirds of its
context every turn; an unresolvable window now renders the gauge's empty
  state instead of a number that is nobody's. Construction and /model
switches resolve without touching the network, and a switch parked
behind
a running turn re-resolves the window at adoption, not at the RPC call.
- A provider without real streaming (azure, codex) no longer renders
  upstream errors as normal assistant text: the terminal stream delta
  carries the classification and joins the same recovery path the
non-streaming call uses. Both classify their non-200 from the live
status
  code (shared ProviderHTTPError) instead of regex-guessing the rendered
text, which also removes the bare "404" substring match that misfiled a
  400 whose body happened to embed one.
- Codex SSE failures keep their structured code, so an overloaded
backend
  is retried instead of classified unknown.
- Orphan `</think>` recovery (backends launched without their reasoning
  parser) is gated to the backend shapes that produce it, so ordinary
  content mentioning the tag is never cut.
- Fallback hops are vetoed when both identities are certain and disagree
(previously a cross-vendor hop went out under the wrong key, or silently
  to the wrong backend on a shared model name); the knn path dispatches
  each hop to its own endpoint and inherits model_overrides through the
  rotor.
- User model_overrides win over shipped extra_body defaults, and the
merge
  no longer drops user keys behind a gateway.
- The credential gate, the reader and the builder answer one way: a
spec's
shipped default address satisfies the gate exactly when the reader would
  serve it (custom runs on a bare key again, azure still demands its
  address), endpoint entries inherit the flat api_base/extra_headers per
  field, and every display face -- provider list, endpoint list, the TUI
picker -- reports the same resolved view, secrets redacted
(extra_headers
  values included, on the flat section field too).
- The vision probe stops joining operator-chosen names against the
vendor
  catalog: it reads through the TUI's lazy proxy to see the Azure
  transport, treats an explicit-selection gateway (custom) as
  caller-chosen, and the background catalog warm is no longer suppressed
  for the life of the process by a stale on-disk table.
- Skill forge rewriter/gate follow the configured agent model (the only
  auxiliary LLM calls that did not).
- The onboarding wizard refuses the six litellm vendors a bare API key
cannot configure, with the actual requirement named, instead of writing
  a section that 401s forever.
- The OAuth handoff replays a signal swallowed mid-login, so a SIGHUP no
  longer leaves a headless TUI.

New feature -- multi-endpoint failover (several accounts on one vendor):

- `providers.<name>.endpoints` (label / apiKey / apiBase / extraHeaders)
  with `endpointStrategy: sticky | round_robin`; the three credential
  spellings (explicit list, Gemini api_key_list, flat fields) resolve
  through one reader with strict precedence and no key merging.
- `EndpointRotorProvider` rotates and fails over with per-endpoint
cooldown
  (30s doubling to 300s, process-local state); auth failures rotate --
  another account's key is exactly what a dead key needs -- while
  endpoint-agnostic failures return immediately. Streams rotate only
  before the first delta, so tokens are never replayed.
- Managed from `raven provider endpoint add|remove|list` and the TUI
model
picker; the session footer names the active endpoint. Write faces refuse
a keyless endpoint for key-credential providers (local deployments keep
their legitimate keyless shape); invalid sections fail loudly instead of
  being read as empty and overwritten.
- Verified live twice: the rotor directly, and the full stack from a
  config file through make_provider (a dead key 401s, cools, fails over,
  answers). The first live run caught the auth-rotation gap the mocks
  could not.

Also in this PR: the onboarding wizard split (5069 -> 3000 lines, pure
moves with every migrated monkeypatch target mutation-checked),
CONTEXT.md
terms (Provider Endpoint, window ladder), benchmark alignment
(pinchbench
now prices through the shared ladder instead of a private drifted copy),
and the model picker reads the config twice per open instead of twice
per
provider row.

Reviewed adversarially across three external rounds and four internal
panel rounds on two model families: 24 before-merge findings raised in
total, every one either fixed with a mutation-verified test or refuted
with executed evidence (one reviewer finding was withdrawn after a
945-case main-parity sweep); final verdicts RATIFY. The rebase also
adopted the review notes left on #282/#285 that landed on this code:
image-capability verdicts are invalidated on a provider switch, and a
parked switch logs its park and its adoption.

Known follow-ups, named in the review thread (issues to follow): the
fallback routing loop contradicts the registry's explicit-selection note
for `custom` (pre-existing on main), the remaining bare status
substrings
in classify_error (429/5xx, pre-existing), per-hop identity rebuilding
for
fallback chains, and the pre-existing SessionInfo shape mismatch.

## Type

- [x] Feature
- [ ] Fix
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/ -q` -- 6128 passed, 33 skipped, run after the
  rebase onto current main; the provider-path test files additionally
  re-run under an empty HOME with identical results.
- `make lint-python` clean; commit messages pass commitlint and
  scripts/check_commit_messages.py across all 78 commits.
- `cd ui-tui && npm run lint && npx tsc --noEmit && npx vitest run` --
86
  files, 982 tests passed; `npm run gen:rpc -- --check` in sync.
- Live probes (OpenRouter, tiny max_tokens): rotor failover and
full-stack
  assembly failover, both passing; logs kept locally.
- Fixes are pinned by deletion mutations that turn a named test red;
  review rounds ran 22 such mutations and the two survivors were
  themselves fixed (one dead guard deleted, one vacuous test replaced).

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible behavior changes: `contextWindowTokens` unset now resolves
to
the model's real window (explicit values are fully respected and no
longer
overridden); the context gauge shows an empty state when the window is
unknown; azure/codex upstream errors surface as errors instead of
assistant text; the wizard refuses six key-only-unconfigurable vendors
with the real requirement named; `custom` with only an apiKey starts
again
(as on main) and the picker no longer demands an address the gate does
not; key-credential providers refuse keyless endpoints at write time and
`endpoint add --api-key` becomes optional for local deployments;
endpoint
keys and extra_headers values are redacted in `provider get`/`list` and
over RPC; a model switch logs when it parks behind a running turn and
when
it adopts. Rollback is a straight revert of the squash commit; no data
migration is involved (config additions are opt-in fields).

## Related Issues

Fixes #124, fixes #234, fixes #155, fixes #152, fixes #254, fixes #151,
fixes #143, fixes #144, fixes #197. References #281 (not reproducible on
current or reported code; the api_key forwarding it suspected is now
pinned
by regression tests), #119 (already fixed by #116; remaining item is the
installer redirect, not provider code).

---------

Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
2026-08-10 23:42:16 +08:00
arelchan
a756a72f6d fix(*): carry a live model switch to every provider holder (#282)
## Summary

A live `/model` switch rebuilt the provider but only reassigned
`loop.provider` / `loop.model`. `AgentLoop.__init__` had already handed
that provider to the subagent manager, the context engine's LLM-backed
segments and the memory consolidator, and each kept its own reference.
Switching away from a dead credential fixed the main loop while
subagents and the skill rewriter/gate went on authenticating against the
endpoint the user had just abandoned. Cron was a fourth victim: its runs
failed with the same 401 and its history rendered the failures as blank
rows.

`AgentLoop.set_provider` now fans the pair out to every holder it built,
and the context engine walks its builders duck-typed so a text-only
segment is skipped rather than raising.

### In-flight work

Every LLM call site reads the provider off `self` at call time, so an
unconditional swap relays one conversation across two vendors. How that
surfaces depends on the path: the `chat_with_retry` sites turn a
rejected request into `finish_reason="error"` content, so the turn
reports a failure with no sign that its endpoint moved, while
`_llm_call_stream` -- the path a TUI turn takes -- catches only
`TimeoutError` and lets the rejection propagate. Neither is a diagnosis
the user can act on.

Two mechanisms, because the two lifetimes differ. Both are superseded by
#284, which makes the
model a property of the conversation and gets the same guarantee from
the context copy that
`asyncio` makes at task creation -- if the two land together, the park
described here exists only
between the two merges.

- **The loop parks.** A switch arriving while any turn runs is held and
adopted at the next `run_turn` entry. One boundary covers eight
`self.provider` reads in `loop/main.py` plus the context engine and
consolidator underneath them; a snapshot would have to be threaded
through each. The park is a depth counter, not a flag: `OriginPools`
gates USER and system origins on independent semaphores with no global
cap, and the TUI defaults to one slot each, so a user turn and a cron
turn overlap on one loop. Both ends gate on zero, and the last turn out
adopts so a park cannot outlive the turns it waited on.
- **Subagents snapshot.** A spawn is a detached task that outlives the
turn, so the park cannot reach it. `spawn` captures the pair it was
asked for and passes it down; capturing later would miss the window
where a spawn waits on the concurrency gate and a sandbox boot.

This is the second line of defence, not the first.
`tui_rpc.methods.config` already rejects a switch outright when the
caller's own session has a turn in flight; the park covers what that
guard cannot see -- a caller that passes no `session_id`, and proactive
turns running in their own lanes. Note the RPC still answers `applied:
True` and the config file is already written, so a parked switch is
applied on disk while the loop reports the old model until the last turn
drains.

### Also here

- `curator_model` is re-derived on a switch with the constructor's own
expression, so the same config cannot mean one thing at build time and
another after. The default is non-empty, so in practice it is a pin; an
explicitly empty `context.curator_model` follows the agent model, and
now follows it in both places.
- The concrete no-op `set_provider` on the context-engine ABC is
concrete so a future implementation with no LLM-backed segment is not
forced to write an empty override. `ContextAssembler` is the only one
today and does override it.

### Scope

`AgentLoop` and the subsystems it builds. `HeartbeatService` and the
Sentinel stack take the same provider but are siblings on the gateway
side, which registers no tui_rpc methods, so `loop.set_provider` cannot
and does not reach them. Not reachable today; worth an issue if the two
sides ever converge. `MemoryConsolidator` is re-pointed but its detached
consolidation tasks are not snapshotted -- a single call rather than a
multi-turn conversation, so the split-conversation argument does not
apply, but it is the same shape.

## Type

- [x] Fix

## Verification

```
uv run pytest tests/ -q
5302 passed, 1 failed

uv run ruff check raven/ tests/     # All checks passed
uv run ruff format raven/ tests/    # unchanged
```

That failure is not this branch.
`tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare`
fails the same way on an unmodified `main` at `53aeb0c` when the whole
file runs (verified in a
detached worktree) and passes when the single test runs alone; it is a
`COLORTERM` artifact and CI
is green on it.
`tests/test_default_context_engine.py::TestTwoTrackConcurrency::test_skill_and_memory_run_concurrently`
also failed in some runs of this branch and of unrelated ones -- a
timing assertion that flakes
under full-suite load, passing alone and with its own file. A clean
re-run at this head has only the
theme failure.

A later review round found one of these mutations still surviving --
moving the spawn snapshot into
`_run_subagent_inner` -- because neither existing test could see the
window it exists for: one
stubbed `_run_subagent` wholesale (proving `spawn` passes a pair, not
when the pair is read) and the
other called `_run_subagent_inner` directly, bypassing `spawn`, the
concurrency gate and the sandbox
boot. There is now a test that holds the gate shut, switches the
provider while the task sits in
that window, releases it, and asserts which provider actually served the
call. The same round found
two docstrings scoped wider than the code: the mid-turn split does not
raise on the
`chat_with_retry` sites but does on `_llm_call_stream`, which is the
path a TUI turn takes; and the
context-engine ABC's concrete no-op was justified by a reason an
abstract method would satisfy
equally. Both corrected.

The tests here were rebuilt after a review found the previous set only
exercised the dispatcher -- replacing any receiver's `set_provider` with
`pass` left it green. They now build a real `AgentLoop` and assert the
gate, rewriter, curator, curator assembler, history trimmer, subagent
manager and consolidator all moved; guard the attribute names the
fan-out walks against a rename; and drive the real `run_turn` and the
real `_run_subagent_inner`. Verified by mutation -- each of these turns
something red:

| Mutation | Result |
|---|---|
| `SubagentManager.set_provider` -> `pass` | 3 failed |
| `CuratorSegmentBuilder.set_provider` -> `pass` | 1 failed |
| rename the `subagents` attribute the fan-out reaches | 2 failed |
| delete the `finally` that releases the turn slot | 3 failed |
| adopt on `run_turn` entry unconditionally | 1 failed |
| never park | 2 failed |
| spawn without the snapshot | 1 failed |
| curator `set_provider` drops the re-derive | 1 failed |
| move the spawn snapshot into `_run_subagent_inner` | 1 failed |

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

`_run_subagent` / `_run_subagent_inner` take the provider and model as
parameters now; the two suites that stub them are updated. No public API
changes. Rollback is a revert -- the previous behaviour is the 401.

## Related Issues

N/A

---------

Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-10 21:36:00 +08:00