19 Commits

Author SHA1 Message Date
0xKT
a27559e331 fix(*): the final audit closes consent, tty truth, and the cache lies
An adversarial last pass over the whole branch found what the review
rounds had not. Consent first: the sh decline arm matched only bare n/N,
so a typed "no" installed LibreOffice through sudo; the arm now takes
the word too, and a tripwire pins the default-yes prompt in both
installers so the review-round decision cannot silently flip back. The
tty gate probed existence where openability is the truth -- the node
exists in CI, cron and docker -t without -i -- so the gate now opens it,
and a consoleless run skips instead of defaulting yes or hanging.
install.ps1 finally carries the everos memory wheel through both of its
paths with sh's own rung discipline, so the Windows summary's memory row
can be green at all. The two cache resolvers follow playwright's real
lookups (XDG_CACHE_HOME, LOCALAPPDATA, chrome-win64 layouts), a
non-object registry degrades instead of raising, and the four mouths
that name a fix now say the same runnable thing on every platform.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-12 00:50:35 +08:00
0xKT
d08a416de4 build(*): the three playwright pins read as one specifier
The same library was pinned three ways: the host browser extra said
>=1.49,<2.0 while the design render extra and the ppt-engine carrier
both said >=1.58,<2. One environment resolves one version, so the host
floor rises to match through uv (lockfile re-resolved in the same
change; the resolved version does not move). A contract test now holds
the three declarations byte-identical and each table to exactly one
playwright entry, so a lone bump -- or a cleanup that reads the
unimported carrier as stale -- turns red here before it breaks a
rendering install. dev.md gains the one line naming how a source
checkout gets the chromium binary itself.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-11 23:48:28 +08:00
Tchen
5b4cce3f43 feat(ppt-engine): deliver where the user named, one route onto a page, and the ten templates
## Summary

This branch folds four merge requests and one unpushed commit into one, so that `main`
takes one review, one approval and one merge instead of four, and so the eight-file
overlap between them is resolved once. It carries, in this order:

- **!602** `feat/ppt_deliver_to_named_path` -- `ppt_build` takes `deliver_to`; every
  publish writes the deck under `out/` and copies the same measured bytes to the path
  the user named, recording it with `role: "delivery"`. The PDF preview rides along only
  where its own `.pdf` name is free: a file already at that stem is the user's and is
  left alone, and the reply says the preview stayed under `out/`.
- **!628** `feat/ppt_copy_route_and_page_isolation` -- `adapt` is removed, so cloning a
  template page and writing into it with `replace_text` is the only route; a page whose
  block raises loses only itself and the pages after it still build; a draft counts
  against the tier's build cap and one is released at it.
- **!632** `feat/ship_templates_and_soffice_dep` -- the ten bundled deck templates are
  tracked in git with per-file sha256 pins, so a fresh clone can build a deck; the
  LibreOffice dependency is declared and resolved through one resolver.
- **!642** `fix/ppt_copy_route_and_clone_typography` -- a copy-route lookup that finds
  nothing names the near miss before the inventory; a text frame with no paragraph at
  all can be written into; a cloned page's title size is said; `row_type_drift` compares
  the same structural slot across a run's units.
- **D57's other-page clause, rewired.** D57 gave five refusals a "did you mean" hint,
  and one of them also answered "the string you named is on page N of this template".
  That clause needs the file a page was cloned out of, and the only refusal holding it
  was `adapt`'s, which !628 removed. It is restored by reading `PPT_TEMPLATE_SOURCE`,
  which the runner already sets in the author's own process and which this module
  already reads for `bundled()`; the two helpers behind the clause are unchanged.
- **D60** `read the ground under each word, not across the whole box` -- the `unreadable`
  contrast gate also measures the render's own word boxes, as extra regions through the
  same guards, so a two-line block crossed by a narrow bright ridge can no longer elect
  the dark ground and read as legible.

Decisions the union forced, none of which drops a behaviour:

- The publish record is built !602's way, with its delivery `role`, and written the way
  D49 requires -- read back off the file, atomic, and refused rather than silent when
  the record cannot be written.
- `clear_region`, `shapes_in`, `page_box` and the refusal of a bare four-number region
  survive `adapt`'s removal; so do `_heading_rows` and its helpers, which the row
  reading loads even though the call they were written for is gone.
- The author-facing guidance names !628's single route and keeps D59's rule that a
  region must declare which reading it is.
**The decision log carries a numbering collision, on purpose.** It is a set union with
no wording changed, and the numbers are left exactly as each branch wrote them, so a
reviewer meets this rather than discovers it:

- !602 and !628 each wrote a **D37** and a **D38**, about different decisions.
- !628's **D45** is not `main`'s D45; they are unrelated entries under one number.
- !628's **D42** is `main`'s **D45** word for word, so the merged log carries one
  decision under two numbers.

Renumbering touches every cross-reference in the log, and doing that inside a
seventy-commit fold immediately before a merge is how a silent break gets in. The
numbering is being fixed on a separate branch, which carries a script and a test that
fails when a citation points at nothing.

## Type

- [x] Feature
- [x] Fix
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on an interpreter carrying `python-pptx`, with `PYTHONPATH` at this worktree and its
`plugins-dist/ppt-engine`, confirmed from a neutral working directory that `raven` and
`raven_ppt` resolve out of the worktree and not the host repo.

- `make test-python` (the whole suite, as CI runs it) -- 20826 passed, 49 skipped, and
  26 failures this machine produces on any branch: it has no `/usr/share/zoneinfo`, so
  every cron timezone case raises `ValueError: unknown timezone 'Asia/Shanghai'`. Checked
  out plain `origin/main` in a clean worktree and ran the same two files: the same 6
  failed and 20 errors, the ids identical character for character. Nothing on this branch
  touches cron or timezones. The stale `test_trunks_executor_would_have_dropped_that_output`
  that failed on the previous revision is retired on the new base and its replacement,
  `test_trunks_executor_preserves_output_on_timeout`, was run on its own and passes.
  The 6 failures and 20 errors are all cron timezone tests failing identically on plain
  `main` in a clean worktree, on a host with no `/usr/share/zoneinfo`.
- `make check-commits` -- exit 0.
- `make check-source-language` -- exit 0.
- `make check-large-files` -- exit 0.
- `make lint-imports` -- exit 0, 9 contracts kept, 0 broken.
- `ruff check` and `ruff format --check` over the 80 changed Python files -- exit 0.
- `pytest tests/test_ppt_engine_templates.py tests/test_utils_office.py
  tests/test_cli_doctor_commands.py` -- 97 passed; the ten templates are tracked and
  satisfy their pins, and the soffice resolver finds the local binary.
- The delivery contract, the record read-back, the page isolation and the near miss were
  each run as a measurement rather than asserted; the `unreadable` gate was run through
  the real LibreOffice render chain on a delivered 12-page deck and reports BLOCKING on
  page 12 under one word at 1.6:1.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes: `ppt_build` gains `deliver_to` and delivers outside `out/`;
`adapt` is gone from the author-facing template module, so a program calling it fails to
import -- `title=` and `subtitle=` have no replacement and the heading is named by the
words the template put there; a draft now counts against the tier's build cap; three
gates that nothing measured against being wrong no longer refuse; and the ten templates
are now bytes in the repository rather than a private registry fetch.

Two pieces here are about not destroying something the user has. The delivery
refuses to write its `.pdf` where a file the user already owns sits at the same
stem, rather than replacing it; and the large-file roster moved out of the manifest
inside the tree being checked and into the gate's own source, so a change can no
longer grant itself the exception the gate exists to withhold.

Rollback is per piece: each of the five sources is a contiguous run of commits on this
branch, in the order listed above, so any one of them can be reverted as a range without
touching the others.

- [x] Backward compatibility considered
- [x] Security impact considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 05:52:36 +00:00
Handsome-wzw
7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00
forrestjgq
f01231fef6 fix(cli): overhaul trajectory browser interaction ux (#379)
## Summary

Overhaul the interactive `raven trajectory` browser end to end, fixing
every
issue from real-terminal testing:

- Navigation: every screen opens with a help area under its title (page
  purpose plus "[Key] function" hotkey hints, rendered in a new readable
help color). Esc walks one level up (action -> attempt -> session); on
the
top-level session screen Esc stays put and quitting is Ctrl+C only,
which
prints a single exit notice. Esc inside a running action cancels just
that
action through a dedicated exception at the action boundary, so the back
  sentinel can never reach a conversion or the data layer; a cancelled
  report keeps its pre-confirm bundle-and-pin side effects (the existing
  declined-report contract). Back/Exit menu entries are gone; answered
  prompts erase themselves and navigation is echoed as breadcrumb lines
  with markup-escaped untrusted text.
- Tables: the session list is TITLE | ATTEMPTS | LAST ACTIVITY and the
  attempt list is # | STARTED | TURNS | SPANS | VERDICT | PIN | MERGED |
PREVIEW with green check cells. Time cells keep the year. The layout is
  budgeted per screen build from the live terminal width (row indent, a
safety margin, and inter-column gaps are all part of the math); PREVIEW
  drops as a whole column when it no longer fits, and below the dynamic
  minimum width the tightest layout is kept while the renderer clips
  overlong lines at the terminal edge. Every dynamic cell passes a
  single-line sanitization gate before width math (verdict sidecars only
  guarantee a non-empty string), cells are measured and truncated by
terminal display width (CJK is 2 cells), and fixed column widths always
  fit their headers.
- Space preview: Space on an attempt row prints that attempt's per-turn
input/output previews, collected in the same snapshot scan and sorted by
a fully-stringified key so ordering never follows log source order (the
  table PREVIEW cell derives from the same sorted collection). Any key,
arrow keys included, returns with the cursor kept on the row; the waiter
  passes the terminal's CPR reply through to the renderer instead of
  treating it as a keypress, and Ctrl+C keeps the browser-wide cancel.
- Merge hotkey: with two or more attempts, m/M opens the multi-select
merge
  (advertised in the help line; the old "Merge attempts" menu entry is
gone). The checkbox screen carries its own help line for toggle,
confirm,
  and cancel. A single attempt neither binds nor advertises the key.
- Theme and feel: new success (green check) and help (readable hint
text)
colors in both palettes, all WCAG AA on their backgrounds; a lone ESC is
flushed after 50ms instead of prompt_toolkit's 0.5s default, removing
the
  laggy Esc feel.

The data layer is untouched: changes are confined to
raven/cli/trajectory_browse.py, raven/cli/_theme.py, and their tests.
The
id-taking subcommands (raven trajectory save/list/...) behave as before.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `COLORTERM=truecolor uv run pytest tests/test_cli_theme.py
tests/test_cli_trajectory_browse.py
tests/test_cli_trajectory_commands.py -q` -> 220 passed
- `COLORTERM=truecolor uv run pytest tests/ --ignore=tests/integration
-q` -> 6917 passed; the 10 failed + 20 errors are the pre-existing
environment-dependent set, identical file-by-file on a clean main
checkout (cron, everos server, config loader, tracing viewer)
- `uv run ruff check` and `uv run ruff format --check` on the four
touched files -> clean
- PTY smoke on a CPR-answering pseudo terminal at 100 and 58 columns:
full flow (preview, merge cancel, action screen, staged Esc backout,
Ctrl+C quit) and the narrow-width degradation contract
- New coverage includes real pipe-input key-binding tests (Esc on all
four prompt kinds, Space/m/M hits, waiter key paths with CPR
pass-through), table budget/clipping boundaries at the computed minimum
width, and corrupt verdict/timestamp corpora

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The interactive browser's look and key protocol change wholesale (that
is
  the point of this PR): Esc no longer quits from the top level (Ctrl+C
  does), Back/Exit entries are gone, and lists render as tables. The
machine-facing subcommands and the trajectory data layer are untouched.
- Rollback: revert the squash commit; the browser returns to the
previous
  interaction wholesale, no data migration involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-05 23:58:20 +08:00
0xKT
3a7cc17aeb fix(*): raven-research takes the host web vendor and key, and the ask_user gate closes
## Summary

Three strands of raven-research work, held together because they all touch the vendored fork and so share one `subagents/TREE_HASHES` line.

**Web search and fetch route through the vendor the deployment picked.** Both tools in the research-flow plugin were pinned to Serper and Jina, so a host on Tavily or Brave still needed a Serper key for the pair. They now carry the kernel's vendor tables (seven search backends, five fetch) with per-vendor request encoding and response normalisation. Serper and Jina wire traffic is byte-identical to before. A vendor that publishes no result offset declares it up front (`SearchProviderSpec.paginates`), so the saturation rule never escalates to a page rung the vendor cannot answer.

The launcher resolves the vendor first and mirrors the SELECTED vendor's key into the plugin slice, never a key by fixed path: a mismatched pair sends one vendor's credential to another's endpoint and every call comes back unauthorized. Every vendor gets an env slot at the same path the host holds it, inherited from the host config when unset, so a deployment configures a vendor once, centrally. The two legacy vendors are also read from the vendor slot, where a host configured after the vendor table landed writes them. `require_search` follows the selection rather than the default.

**The research modes take the host's effort vocabulary**: `deep` becomes `high`, `ultra` becomes `max`, and `medium` is the baseline. This is what lets the companion MR's host tier reach this agent by name.

**The ask_user gate closes after a hook rollback**, and host questions take the conversation lock like every other route, including the `direct_ask` grant. The digest follows its session's mode, a typo in a mode overlay refuses to launch rather than running the base value under the overlay's label, and the reviewer records which model judged.

Three defects found while building it, each with the test that pins it:

- **A credential reached disk.** The retry rows of the search and fetch ledger recorded `str(exc)` for a status error, and httpx puts the whole request URL in that text. SerpApi authenticates by query parameter, so a 429 or 503 wrote the key into the JSONL file. The redaction the search and fetch rows already had is now one function, `_error_text`, used by both writers, in the plugin and in the vendored copy. The regression test drives a retryable status and reads the ledger file off disk; the previous test used 401, which is not retryable and so never reached the leaking write. The key never reached the model or the answer: the appendix partitions on `search` / `fetch` / `verify` / `force_finalize` and reads nothing from a retry row.
- **A keyless fetch backend refused every call.** The kernel's `web_fetch` degrades a keyed backend with no resolvable key to Jina, out loud. The plugin's replacement did not, and it REPLACES the kernel's, so `fetch.provider = tavily` with no Tavily key made a research run the single place that config could not read a page, while a plain raven on it read the page fine. The launcher itself produces that state: it writes the provider unconditionally and the key only when one resolves. Fixed at the build site, where the kernel resolves it, and a test asserts both implementations answer identically for every vendor.

  Reported by chandler.zhang and completed in `ac7d364c`: the degraded backend was still handed the key of the vendor it replaced, which is absent by definition, so a deployment holding a Jina key read pages authenticated through raven's built-in fetch and anonymously through the plugin -- losing the configured quota on the very fallback the rule exists to match. The kernel looks the key up a second time under the EFFECTIVE provider; the slice carries one key per role and no vendor table, so the launcher now mirrors the default reader's key beside the selected vendor's and the build picks by destination, a lookup keyed on the provider the tool will actually call. It is mirrored whenever the selection is not already the default rather than only when the fallback will fire, because the rule that decides that lives in `effective_provider` and a launcher second-guessing it would be the copy that drifts.
- **A typo in a vendor name refused to launch for the wrong reason.** The plugin degrades an unknown vendor to its default, which is right for a slice nothing validates. Carried through the launcher, the same typo had `require_search` demand a key for a vendor that does not exist while the working Serper key sat in the config -- sending the operator to set a credential they already had. The launcher now refuses naming the name and the valid list.

`.env.example` and the plugin docstrings were still describing the fixed Serper and Jina pair; both now cover the provider switch and the per-vendor key slots. The plugin MANIFEST was missed in that pass and is corrected in `18ce29df`, also on chandler.zhang's report: it still called `search.apiKey` a Serper key and `fetch.apiKey` a Jina reader key, and said web_search declines "when no Serper key resolves". A direct plugin user reaching it without the launcher could set `search.provider = tavily` and supply a Serper credential, which this code then sends to Tavily's endpoint.

`subagents/TREE_HASHES` is recorded once, in the final commit, rather than per commit: the hash covers every tracked byte under the fork, so an intermediate value goes stale the moment the next edit or a rebase lands.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch at its current head:

- `uv run pytest` -- 5 failed, 18545 passed, 69 skipped. The five are this host's pre-existing baseline, identical on `origin/main`: `test_cli_doctor_commands.py::test_doctor_answers_where_the_memories_are` and four `test_security_network.py` NAT64 parameters. 0 introduced.
- `make lint-python` -- passed, 1787 files already formatted.
- `uv run lint-imports` -- 8 contracts kept, 0 broken.
- `make check-vendored-subagents` -- exit 0, hash recorded.
- `make check-vendored-invariants`, `make check-large-files`, `make lint-ui`, `make lint-tui`, `scripts/check_commit_messages.py origin/main..HEAD` -- all exit 0.
- The vendored fork's own suite, run with the fork on `PYTHONPATH` so `raven` resolves to its copy: 19 failed, 6223 passed, 4 errors. Reverting only this branch's change to the fork gives the same 19 failures and 4 errors, all in three files that never import the changed module, so 0 introduced. Those failures are an artifact of running the fork against the trunk's venv, not the fork's own.

Reverting each fix and confirming the matching test fails: the ledger redaction (the 429 case red, the 401 case correctly still green), the fetch degradation (4 keyed vendors red, Jina correctly unaffected), the vendor-name refusal (2 red), and the slice key following the selected vendor (2 red).

Three more for the review fix: the build passing the selected vendor's key to the reader that replaced it (1 red), the launcher not mirroring the fallback key (1 red), and the launcher mirroring it for the default vendor too (1 red, the case where a second copy of the same key is one more place for the pair to disagree).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Security: a credential stopped being written to disk. Any research-flow ledger file written before this change by a deployment on SerpApi may contain the key in a `search_retry` or `fetch_retry` row. Those files live under the plugin's state root, one per turn, and are unlinked at the end of the turn that wrote them; a process killed mid-turn leaves one behind. Treat a SerpApi key used on an affected build as exposed and rotate it, then check for leftover files.

Other user-visible changes:

- A research deployment that selected a keyed fetch vendor without its key now reads pages through Jina instead of failing every fetch, and does so on the host's configured Jina key rather than anonymously. Behaviour improves; the log says which vendor is serving.
- That fallback key now travels in the rendered plugin slice as `fetch.fallbackApiKey`, beside the selected vendor's key, wherever the rendered config already holds credentials. It is only ever handed to the default reader: the build looks the key up by destination, so a provider neither key belongs to gets none.
- An unknown vendor name in `search.provider` or `fetch.provider` now refuses to launch. A deployment running on a typo today would have been failing its search calls anyway.
- The research mode ids change: `deep` becomes `high` and `ultra` becomes `max`. A caller that names `deep` by id gets the agent's default instead.
- Ledger `*_retry` rows record `HTTPStatusError: HTTP <status>` where they recorded the full httpx message. The one offline reader, the fork's `bench/rollout.py`, buckets by status either way and already read the search and fetch rows in that shape.

Rollback: revert the commits on this branch. `TREE_HASHES` is recorded in its own final commit, so reverting the set restores the recorded hash with the tree.

## Related Issues

N/A

<!-- The repo guide says to leave this trailer to the commits, on the premise that
     the platform collects it into the squash commit. This GitLab does not, and the
     squash body is taken from this description, so it is repeated here to survive. -->

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-06 04:12:09 +00:00
userName20260323
fec055779d fix(*): raven-research takes the host web vendor and key, and the ask_user gate closes
## Summary

Three strands of raven-research work, held together because they all touch the vendored fork and so share one `subagents/TREE_HASHES` line.

**Web search and fetch route through the vendor the deployment picked.** Both tools in the research-flow plugin were pinned to Serper and Jina, so a host on Tavily or Brave still needed a Serper key for the pair. They now carry the kernel's vendor tables (seven search backends, five fetch) with per-vendor request encoding and response normalisation. Serper and Jina wire traffic is byte-identical to before. A vendor that publishes no result offset declares it up front (`SearchProviderSpec.paginates`), so the saturation rule never escalates to a page rung the vendor cannot answer.

The launcher resolves the vendor first and mirrors the SELECTED vendor's key into the plugin slice, never a key by fixed path: a mismatched pair sends one vendor's credential to another's endpoint and every call comes back unauthorized. Every vendor gets an env slot at the same path the host holds it, inherited from the host config when unset, so a deployment configures a vendor once, centrally. The two legacy vendors are also read from the vendor slot, where a host configured after the vendor table landed writes them. `require_search` follows the selection rather than the default.

**The research modes take the host's effort vocabulary**: `deep` becomes `high`, `ultra` becomes `max`, and `medium` is the baseline. This is what lets the companion MR's host tier reach this agent by name.

**The ask_user gate closes after a hook rollback**, and host questions take the conversation lock like every other route, including the `direct_ask` grant. The digest follows its session's mode, a typo in a mode overlay refuses to launch rather than running the base value under the overlay's label, and the reviewer records which model judged.

Three defects found while building it, each with the test that pins it:

- **A credential reached disk.** The retry rows of the search and fetch ledger recorded `str(exc)` for a status error, and httpx puts the whole request URL in that text. SerpApi authenticates by query parameter, so a 429 or 503 wrote the key into the JSONL file. The redaction the search and fetch rows already had is now one function, `_error_text`, used by both writers, in the plugin and in the vendored copy. The regression test drives a retryable status and reads the ledger file off disk; the previous test used 401, which is not retryable and so never reached the leaking write. The key never reached the model or the answer: the appendix partitions on `search` / `fetch` / `verify` / `force_finalize` and reads nothing from a retry row.
- **A keyless fetch backend refused every call.** The kernel's `web_fetch` degrades a keyed backend with no resolvable key to Jina, out loud. The plugin's replacement did not, and it REPLACES the kernel's, so `fetch.provider = tavily` with no Tavily key made a research run the single place that config could not read a page, while a plain raven on it read the page fine. The launcher itself produces that state: it writes the provider unconditionally and the key only when one resolves. Fixed at the build site, where the kernel resolves it, and a test asserts both implementations answer identically for every vendor.

  Reported by chandler.zhang and completed in `ac7d364c`: the degraded backend was still handed the key of the vendor it replaced, which is absent by definition, so a deployment holding a Jina key read pages authenticated through raven's built-in fetch and anonymously through the plugin -- losing the configured quota on the very fallback the rule exists to match. The kernel looks the key up a second time under the EFFECTIVE provider; the slice carries one key per role and no vendor table, so the launcher now mirrors the default reader's key beside the selected vendor's and the build picks by destination, a lookup keyed on the provider the tool will actually call. It is mirrored whenever the selection is not already the default rather than only when the fallback will fire, because the rule that decides that lives in `effective_provider` and a launcher second-guessing it would be the copy that drifts.
- **A typo in a vendor name refused to launch for the wrong reason.** The plugin degrades an unknown vendor to its default, which is right for a slice nothing validates. Carried through the launcher, the same typo had `require_search` demand a key for a vendor that does not exist while the working Serper key sat in the config -- sending the operator to set a credential they already had. The launcher now refuses naming the name and the valid list.

`.env.example` and the plugin docstrings were still describing the fixed Serper and Jina pair; both now cover the provider switch and the per-vendor key slots. The plugin MANIFEST was missed in that pass and is corrected in `18ce29df`, also on chandler.zhang's report: it still called `search.apiKey` a Serper key and `fetch.apiKey` a Jina reader key, and said web_search declines "when no Serper key resolves". A direct plugin user reaching it without the launcher could set `search.provider = tavily` and supply a Serper credential, which this code then sends to Tavily's endpoint.

`subagents/TREE_HASHES` is recorded once, in the final commit, rather than per commit: the hash covers every tracked byte under the fork, so an intermediate value goes stale the moment the next edit or a rebase lands.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch at its current head:

- `uv run pytest` -- 5 failed, 18545 passed, 69 skipped. The five are this host's pre-existing baseline, identical on `origin/main`: `test_cli_doctor_commands.py::test_doctor_answers_where_the_memories_are` and four `test_security_network.py` NAT64 parameters. 0 introduced.
- `make lint-python` -- passed, 1787 files already formatted.
- `uv run lint-imports` -- 8 contracts kept, 0 broken.
- `make check-vendored-subagents` -- exit 0, hash recorded.
- `make check-vendored-invariants`, `make check-large-files`, `make lint-ui`, `make lint-tui`, `scripts/check_commit_messages.py origin/main..HEAD` -- all exit 0.
- The vendored fork's own suite, run with the fork on `PYTHONPATH` so `raven` resolves to its copy: 19 failed, 6223 passed, 4 errors. Reverting only this branch's change to the fork gives the same 19 failures and 4 errors, all in three files that never import the changed module, so 0 introduced. Those failures are an artifact of running the fork against the trunk's venv, not the fork's own.

Reverting each fix and confirming the matching test fails: the ledger redaction (the 429 case red, the 401 case correctly still green), the fetch degradation (4 keyed vendors red, Jina correctly unaffected), the vendor-name refusal (2 red), and the slice key following the selected vendor (2 red).

Three more for the review fix: the build passing the selected vendor's key to the reader that replaced it (1 red), the launcher not mirroring the fallback key (1 red), and the launcher mirroring it for the default vendor too (1 red, the case where a second copy of the same key is one more place for the pair to disagree).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Security: a credential stopped being written to disk. Any research-flow ledger file written before this change by a deployment on SerpApi may contain the key in a `search_retry` or `fetch_retry` row. Those files live under the plugin's state root, one per turn, and are unlinked at the end of the turn that wrote them; a process killed mid-turn leaves one behind. Treat a SerpApi key used on an affected build as exposed and rotate it, then check for leftover files.

Other user-visible changes:

- A research deployment that selected a keyed fetch vendor without its key now reads pages through Jina instead of failing every fetch, and does so on the host's configured Jina key rather than anonymously. Behaviour improves; the log says which vendor is serving.
- That fallback key now travels in the rendered plugin slice as `fetch.fallbackApiKey`, beside the selected vendor's key, wherever the rendered config already holds credentials. It is only ever handed to the default reader: the build looks the key up by destination, so a provider neither key belongs to gets none.
- An unknown vendor name in `search.provider` or `fetch.provider` now refuses to launch. A deployment running on a typo today would have been failing its search calls anyway.
- The research mode ids change: `deep` becomes `high` and `ultra` becomes `max`. A caller that names `deep` by id gets the agent's default instead.
- Ledger `*_retry` rows record `HTTPStatusError: HTTP <status>` where they recorded the full httpx message. The one offline reader, the fork's `bench/rollout.py`, buckets by status either way and already read the search and fetch rows in that shape.

Rollback: revert the commits on this branch. `TREE_HASHES` is recorded in its own final commit, so reverting the set restores the recorded hash with the tree.

## Related Issues

N/A

<!-- The repo guide says to leave this trailer to the commits, on the premise that
     the platform collects it into the squash commit. This GitLab does not, and the
     squash body is taken from this description, so it is repeated here to survive. -->

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-06 04:12:09 +00:00
0xKT
2f862d8ca2 feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
hylarucoder
c703db6ffd feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
0xKT
009d0474e6 feat(*): subagent discovery switches rails from the forks to agents/
## Summary

Subagent discovery switches rails from the subagents/ fork trees to the
agents/ product tree. This makes the five migrated products (code, oncall,
ppt, research, design) the factory default: a fresh install discovers them
without running any per-product installer, and the fork trees stop being
scanned entirely (they remain on disk, untouched, until their separately
scheduled retirement).

Key decisions:

- Discovery resolves an agents root through the same three levels the fork
  scan used: raven home copy, beside the package, wheel-embedded. The copy
  to raven home is kept folders-only so a product's .env survives upgrades.
- Readiness has new semantics: the launcher's absolute-path command tokens
  must exist, and a product-declared engine distribution must be importable
  (importlib find_spec). A product whose engine wheel is missing is listed
  but disabled, naming the wheel to install. Credential checks leave the
  discovery phase: the launcher itself refuses loudly, which live handshake
  smoke verified for all five products.
- Config rows keep whole-row precedence over discovered rows; the merge
  mechanics are unchanged (verified byte-equivalent modulo renames by an
  independent adversarial review).
- The main wheel now ships the five product launchers under raven/agents
  and no longer embeds raven/subagents (118 files / 1.3 MB, verified by
  building and listing the wheel; zero pyproject changes, the inclusion
  lives in the hatch build hook).
- The design/ppt manifests gain one machine-readable field,
  `engine: {package, wheel}`, consumed by the readiness probe; launch
  behavior is untouched and older readers ignore the field (verified).
- A discovered row whose readiness is clean but whose runtime probe misses
  now renders as attention instead of a dead Install button (fix from the
  adversarial review).

An independent adversarial review (verdict: land after fixes) preceded this
MR; its one HIGH (net test-coverage loss on host_can_lend_a_key edge
branches) and one MED (the Install dead-end above) are fixed in the commit,
with companion tests. Fork-era behaviors it confirmed as bit-identical
inheritances (home-first shadowing, path-with-spaces refusal) are recorded,
not changed.

## Type

- [x] Feature

## Verification

- Full canonical suite on the branch: `uv run pytest tests -q` under
  `uv sync --all-extras` -> 18494 passed, 40 skipped, exit 0 (rehearsal
  tree); 18472 passed, 62 skipped on the drafting stack. Net def-level
  movement vs baseline accounted name-for-name in the wave report.
- Targeted families at landing: tests/test_agents_ppt_launcher.py,
  tests/test_cli_subagent_setup.py, tests/test_hatch_build.py,
  tests/test_rpc_subagents.py, tests/test_subagent_dag_machines.py,
  tests/test_subagent_vendored_agents.py -> 242 passed.
- `make lint-python` -> exit 0; `uv run lint-imports` -> 8 kept, 0 broken;
  `make check-vendored-subagents`, `check-vendored-invariants`,
  `check-large-files` -> exit 0. subagents/ is byte-untouched.
- Wheel audit: `uv build --wheel`, listed contents -> 5 launchers +
  5 manifests under raven/agents, 0 raven/subagents entries, no secrets.
- Fresh-venv install test: wheel installed into an isolated venv discovers
  and materializes all five rows; design/ppt correctly disabled without
  their engine wheels; research launcher runs to its credential verdict.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: fork rows stop appearing on fresh installs; the five
product rows appear instead. Machines with existing config rows are
unaffected (config precedence unchanged). The RPC wire shape is frozen
(vendored/building keys and subagents.build kept), so no client regeneration
is needed. Rollback: revert the squash commit; discovery returns to the
fork trees, which are still on disk in the repo and still embedded in any
wheel built before this change.

## Related Issues

N/A
2026-09-05 05:13:56 +00:00
0xKT
9a98c3107f feat(*): subagent discovery switches rails from the forks to agents/
## Summary

Subagent discovery switches rails from the subagents/ fork trees to the
agents/ product tree. This makes the five migrated products (code, oncall,
ppt, research, design) the factory default: a fresh install discovers them
without running any per-product installer, and the fork trees stop being
scanned entirely (they remain on disk, untouched, until their separately
scheduled retirement).

Key decisions:

- Discovery resolves an agents root through the same three levels the fork
  scan used: raven home copy, beside the package, wheel-embedded. The copy
  to raven home is kept folders-only so a product's .env survives upgrades.
- Readiness has new semantics: the launcher's absolute-path command tokens
  must exist, and a product-declared engine distribution must be importable
  (importlib find_spec). A product whose engine wheel is missing is listed
  but disabled, naming the wheel to install. Credential checks leave the
  discovery phase: the launcher itself refuses loudly, which live handshake
  smoke verified for all five products.
- Config rows keep whole-row precedence over discovered rows; the merge
  mechanics are unchanged (verified byte-equivalent modulo renames by an
  independent adversarial review).
- The main wheel now ships the five product launchers under raven/agents
  and no longer embeds raven/subagents (118 files / 1.3 MB, verified by
  building and listing the wheel; zero pyproject changes, the inclusion
  lives in the hatch build hook).
- The design/ppt manifests gain one machine-readable field,
  `engine: {package, wheel}`, consumed by the readiness probe; launch
  behavior is untouched and older readers ignore the field (verified).
- A discovered row whose readiness is clean but whose runtime probe misses
  now renders as attention instead of a dead Install button (fix from the
  adversarial review).

An independent adversarial review (verdict: land after fixes) preceded this
MR; its one HIGH (net test-coverage loss on host_can_lend_a_key edge
branches) and one MED (the Install dead-end above) are fixed in the commit,
with companion tests. Fork-era behaviors it confirmed as bit-identical
inheritances (home-first shadowing, path-with-spaces refusal) are recorded,
not changed.

## Type

- [x] Feature

## Verification

- Full canonical suite on the branch: `uv run pytest tests -q` under
  `uv sync --all-extras` -> 18494 passed, 40 skipped, exit 0 (rehearsal
  tree); 18472 passed, 62 skipped on the drafting stack. Net def-level
  movement vs baseline accounted name-for-name in the wave report.
- Targeted families at landing: tests/test_agents_ppt_launcher.py,
  tests/test_cli_subagent_setup.py, tests/test_hatch_build.py,
  tests/test_rpc_subagents.py, tests/test_subagent_dag_machines.py,
  tests/test_subagent_vendored_agents.py -> 242 passed.
- `make lint-python` -> exit 0; `uv run lint-imports` -> 8 kept, 0 broken;
  `make check-vendored-subagents`, `check-vendored-invariants`,
  `check-large-files` -> exit 0. subagents/ is byte-untouched.
- Wheel audit: `uv build --wheel`, listed contents -> 5 launchers +
  5 manifests under raven/agents, 0 raven/subagents entries, no secrets.
- Fresh-venv install test: wheel installed into an isolated venv discovers
  and materializes all five rows; design/ppt correctly disabled without
  their engine wheels; research launcher runs to its credential verdict.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: fork rows stop appearing on fresh installs; the five
product rows appear instead. Machines with existing config rows are
unaffected (config precedence unchanged). The RPC wire shape is frozen
(vendored/building keys and subagents.build kept), so no client regeneration
is needed. Rollback: revert the squash commit; discovery returns to the
fork trees, which are still on disk in the repo and still embedded in any
wheel built before this change.

## Related Issues

N/A
2026-09-05 05:13:56 +00:00
0xKT
8f6ef4c0ce ci(*): the governance batch brings the rulebook current and makes it enforced
## Summary

Governance hardening batch: the collaboration rulebook catches up with the
post-layering tree, the repository's comments and records are brought up to
its own standards, and the standards themselves become enforced gates.

Five commits, in series:

1. docs: the rulebook catches up -- AGENTS.md gains the source-language rule
   with the owner-signed CJK zones, a portable PR check replaces the
   GitHub-only wording, README's repo-layout table becomes the canonical
   scope list, CONTEXT-MAP gains its third route, and CONTEXT.md records the
   migration vocabulary.
2. docs: 133 surgical comment fixes across 83 files -- comments now say why,
   and only why; edit markers and restated-what comments are gone.
3. docs: stale orphans leave -- 5 dead documents deleted, 4 dated records
   stamped as historical, the zh README shrinks to a translated landing
   page, and the living-docs guard widens its glob.
4. ci: the honor system grows teeth -- scripts/check_source_language.py
   (diff-based, three passes), a commitlint scope enum computed from the
   tree, tests/test_readme_scope_canon.py, and the CI/make wiring for all
   three.
5. fix: a second pass brings the batch current with what landed on main
   after it was drafted -- the design engine's two owner-signed zones enter
   the gate and the rulebook, the scope enum learns the fifth product and
   the wheel distributions, CONTEXT.md records the third distributed member
   and the Visual Domain Selector, and one zh fixture added by a follow-on
   wave is respelled to English with its placeholder semantics preserved.

The batch is docs and CI only: no runtime behavior changes. The one source
edit outside docs/CI is the fixture respell in commit 5, verified by its own
test family.

## Type

- [x] CI / tooling

## Verification

- Full suite on the batch tree: `uv run --frozen --all-extras pytest tests -q`
  -> 18513 passed, 40 skipped, exit 0.
- `make lint-python` -> All checks passed; `uv run lint-imports` -> 0 broken.
- Guard families after landing commit 5: `uv run pytest
  tests/test_playbook_generator.py tests/test_commit_lint.py
  tests/test_living_docs.py tests/test_readme_scope_canon.py
  tests/test_source_language_check.py -q` -> 55 passed.
- Language-gate self-run over the batch's own range and over main's history
  since the batch base: exit 0 (one genuine pre-existing violation found and
  fixed in commit 5).
- Commitlint over all five batch subjects: 5/5 pass; `make check-large-files`
  -> exit 0.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No runtime code paths change. The new CI gates can reject future commits by
design; if a gate misfires, reverting the squash commit removes the gates
and the rulebook text together. Existing CJK carriers in main's history are
exempt: the gate is diff-based and only judges new changes.

## Related Issues

N/A
2026-09-05 05:13:15 +00:00
gloryfromca
0809794b66 chore(*): merge the trunk again before the v0.1.12 sync lands
Three more commits landed on the trunk during the review round, so the branch
stopped fast-forwarding again. Merged rather than rebased for the same reason as
before: a rebase flattens the v0.1.12 merge commit and costs the ancestry this
whole branch exists to establish.

One conflict, additive: tests/test_cli_gateway_commands.py had a _risk_banner
group added on this branch and a gateway-shutdown ordering test added on the
trunk. Both kept.

ui-tui/src/rpc/generated.ts merged automatically. Since it is a generated file,
its result was checked rather than assumed: `npm run lint:rpc` reports it in sync
with rpc-schema/openrpc.json, which merged alongside it.

Verification:

    uv run pytest                9018 passed, 33 skipped, 13 deselected
    uv run ruff check            all checks passed
    uv run ruff format --check   1022 files already formatted
    npx vitest run               99 files, 1265 tests passed
    npx tsc --noEmit             clean
    npm run lint:rpc             schema and codegen in sync

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-18 17:30:49 +08:00
LivXue
ede8e7ed3c refactor(*): install the shipped sub-agents without a running raven
## Summary

Installing one of the sub-agents that ship under `subagents/` required a running
raven: each folder's `install.py` registered its entry through
`PUT /raven/subagents`, so both the web service and the gateway had to be up. On a
machine that has just cloned the repository, neither is.

The installer now writes `subagents.thirdParty` in the host raven's config
directly, through `raven.config.update_subagents` - the supported write path,
which validates the entry against the schema, replaces the entry sharing its name,
refuses a list that would hold duplicates, and replaces the file atomically. Hand-editing the JSON gets none of that, and a
half-written config is one raven will not start on. That module is importable only
from the host raven's environment, which is rarely the interpreter running the
installer, so it is reached in a subprocess resolved from the shebang of `raven`
on `PATH` (or `RAVEN_PYTHON`, or `--raven-python`). The installer therefore stays
standard-library only and runs under a bare `python3`.

What the file write costs is hot-apply: a running raven holds the roster it read
at startup, so the installer prints a restart instruction and the guide says the
same. `--endpoint` is gone; `--config` and `--raven-python` are new.

Two additions on top:

- `subagents/install.sh` does the whole install for every folder, or the ones
  named: builds each checkout's venv, scaffolds `.env` from its template, then
  registers. It refuses to register a folder whose api key is still blank, so a
  half-configured agent never reaches the roster, and exits non-zero while
  anything still needs attention. It discovers the checkout by its lone
  `pyproject.toml` and derives the key prefix from the folder name, so a new
  folder needs no edit in the script.
- `subagents/README.md` is the directory-level guide: what a folder is made of,
  the install steps, what the installer does and does not do, how to remove an
  agent, what a new folder has to satisfy, and what never gets committed.

`subagents/*` stays excluded by default. Only the folders that were already
tracked are whitelisted, plus those two new files by name; a folder holding a
docker image tar or a private checkout is unaffected. The change also drops a
sentence that pointed a published README at an unpublished sibling folder.

The pre-install backup is written before the config is touched, holds the list as
it sits on disk rather than a schema round-trip of it (loading coerces an `openai`
entry's `readsLocalFiles` away, so a validated copy would restore something the
user never wrote), and is at mode 600 because another entry among the others can
carry its own api key. A backup that cannot be written stops the install with the
config untouched, rather than going missing after a config that already changed.

`install.sh` reads the api key the way the launcher does, first non-empty match
wins, so the two cannot disagree about a `.env` that sets it twice. It expands its
argument array through `${args[@]+...}`, since an empty array under `set -u` is an
unbound-variable error before bash 4.4 - the stock `/bin/bash` on macOS. A relative
`--config` is made absolute at parse time, because `install.py` runs with each
folder as its cwd and would otherwise resolve the path once per folder.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [x] Refactor
- [ ] Other

## Verification

The branch touches 12 files: `.gitignore` and 11 under `subagents/`. Nothing under
`raven/` or `tests/`, and `testpaths = ["tests"]`, so no unit test collects any of
it.

```
uv run pytest -q          8237 passed, 46 skipped, 13 deselected in 170s
uv run ruff check  --no-force-exclude subagents/raven-code/install.py   All checks passed
uv run ruff format --check --no-force-exclude subagents/raven-code/install.py   1 file already formatted
bash -n subagents/install.sh                              clean
python3 scripts/check_commit_messages.py origin/main..HEAD exit 0
python3 scripts/check_large_files.py origin/main..HEAD     exit 0
```

`subagents/` is excluded in `[tool.ruff] extend-exclude`, so the two ruff commands
above need `--no-force-exclude` to reach the file at all.

End-to-end on a freshly provisioned host with no `raven`, no `uv`, and Python 3.10:
cloned the repository, installed `uv`, built the venvs, ran the installer twice and
spawned one agent for real.

- Installer on a clean clone with no keys: three checkout venvs built, three `.env`
  files scaffolded at mode 600, stopped at the key gate for all three, exit 1. The
  host config's mtime was unchanged and no backup file was written, so nothing
  half-configured reached the roster.
- Same host with keys filled in: all three entries `replaced` in a config that
  already held 8 third-party sub-agents, and all 8 survived. Each backup was mode
  600 and held the full 8-entry list. `load_config()` then listed all 8, so the
  written entry is valid to the runtime and not only to the writer.
- The resolved `command` of each entry pointed at the clone's own path, confirming
  the placeholders resolve against the installer's location rather than the machine
  that built the folder.
- One real spawn answered correctly in 22s.
- Before that, on a machine with no `raven` on `PATH`, the installer reported
  `no raven on PATH; pass --raven-python ...` and recorded the folder as failed.

Probes for the fixed defects: a config seeded with `readsLocalFiles: true`
on disk produced a backup recording `false` before the fix and `true` after; a
`.env` setting the key twice with an empty second line made the shell installer
refuse an agent the launcher would run, and both now agree; an interpreter stub
printing non-JSON now yields `cannot read the writer's output` instead of a
traceback; a `--config` whose parent is a regular file makes the config write fail
with the backup already on disk, which is what shows the backup precedes it; and a
run with no flags at all registers against a scratch `HOME`, which is the path the
empty-array expansion used to break on bash before 4.4.

Proving the backup ordering took two attempts. Making the folder unwritable with
`chmod 555` proved nothing, because this runs as root and root ignores mode bits -
a trap the repository's own skipped test records. The failing-config probe above is
the one that holds.

- [ ] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

The first box is deliberately unchecked: there is no unit test covering this tree,
because pytest does not collect it. The end-to-end run above is the evidence
offered in its place. The unit suite was run anyway, on the rebased tree, and
reported no failures at all - so there are no failing ids to diff against the
target branch.

## Risk

**Behaviour change.** An install no longer takes effect immediately. Anyone
following the old flow will see the agent appear only after raven or the gateway
restarts; the installer's last line and the guide both say so. `--endpoint` no
longer exists, so a caller passing it gets an argparse error rather than a silent
fallback.

**Not reproduced, environment-dependent.** On the test host, the provider's model
listing answered 200 while inference answered
`403 This model is not available in your region.`, which the launcher reports as a
run that exits 0 having committed no answer - it reads like a launcher bug and is
not one. Exporting the site proxy fixed it, and the guide now records both that and
the fact that a missing EverOS degrades rather than fails.

**Security.** The pre-write backup can contain another entry's api key, so it is
written at mode 600 and is covered by the existing ignore rules. Keys stay in each
folder's `.env` at mode 600; the tracked `config.json` files hold none. The
installer no longer sends the entry list over a loopback HTTP service.

**Rollback.** Revert the commit: the previous installer talked to the RPC and needs
nothing else restored. Entries already written stay valid either way, since the
schema did not change.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-18 09:07:22 +00:00
gloryfromca
aaf993896e chore(*): merge upstream v0.1.12 into the gitlab trunk
Absorbs the GitHub release into the downstream trunk. A merge rather than a
rebase so v0.1.12 becomes a real ancestor: the next sync starts from v0.1.12
instead of v0.1.11, and the eventual contribution back to GitHub can still tell
our commits from the upstream ones it already has.

Cost this round: 54 conflict hunks over 31 files, against 19 over 14 for
v0.1.11. The growth is ours, not upstream's -- 33 commits landed on the trunk in
the six days since the last sync, and they concentrate in the three files the
sync notes already name as hot (agent/loop/main.py, cli/tui_commands.py,
config/schema.py).

Notable conflict resolutions:

* agent/loop/streaming.py keeps our extracted stream_llm_call and grafts
  upstream's one real addition into it: the truncation verdict
  (upstream_finish_reason, flag_truncation, and the two LLMResponse fields).
  Comparing function bodies against the merge base was what showed aclosing()
  and the structured error classification were already there, not new.
* agent/loop/main.py splits an 87-line block that upstream left empty: two
  helpers moved into upstream's new loop/failure_streak.py and had to go, four
  with live call sites outside the block had to stay. Taking the block whole in
  either direction was wrong.
* _dispatch_backend_store takes upstream's detached-task architecture and keeps
  our memory-failure tracking inside the task body. The shared tail after the
  hunk reads `e`, which upstream's `except Exception:` does not bind, and
  drain_backend_stores already had three merged-in callers.
* context_engine/segments/render.py merges two dimensions into identity_text:
  our work_dir split and upstream's routed model id. The shared body already
  reads names from both sides.
* agent/tools/registry.py keeps both unparseable-argument mechanisms. They sit
  at different layers on purpose -- RunMeta comes from a provider on the
  non-streaming path, RAW_ARGUMENTS_KEY from the loop on the streaming one.

Nine breakages git merged cleanly and never flagged. This is the shape the
change fails in, and it is worth recording in full:

* The three vendored builds under subagents/ carry a copy of the pre-rename
  raven/tui_rpc/ tree, so rename detection sent upstream's 50 lines of
  tui_rpc changes into a dead copy instead of raven/rpc/. Only one of the four
  files conflicted; the other three were silent.
* subagent/backends/raven_loop.py holds the tool loop we extracted from the
  subagent manager, so upstream's run_meta argument never reached it.
* AgentDefaults.max_tokens: upstream dropped it for a dynamic
  send_max_tokens(). Both make_provider sites merged onto the new scheme;
  providers/resolving_provider.py did not, because upstream has no such file.
* ResolvingProvider never forwarded wire_model_id. Only upstream's own
  invariant test catches that class, and the cost is a silently mis-sized
  request (16384 against 4096 for one gpt-4o).
* CronService.add_job kept our body but took upstream's signature, losing
  `deliver` from both the parameter list and CronPayload while two RPC methods
  still pass it, and leaving a call to the _arm_timer scheme upstream replaced
  with a wake Event.
* rpc/bootstrap.py still called make_on_cron_job positionally after the
  signature narrowed.
* update_everos lost _EVEROS_CONFIG for a root-lookup family, and gained an
  ownership gate over its write primitives.
* SessionInitInfo never declared config_notices, in the model or the schema,
  while methods/session.py returns it and our own TUI already reads it through
  a hand-written type intersection. Both are strict, so a config migration
  would have made session.create fail validation in production.
* tests/test_skill_hub_tools.py keyed a fixture under the "local" router
  namespace rather than a real registry layer, which upstream's new assertion
  exists to refuse.

Also fixed while here: the rich Style cache poisoning that the ci branch
documents as a known crutch (one pinned color_system), and the coverage
baseline's 29 stale raven/tui_rpc/ paths. The baseline totals are still
upstream's own numbers and should be regenerated separately.

Verification, on macOS with a non-root runner:

    uv run pytest                8888 passed, 33 skipped, 13 deselected
    uv run ruff check            all checks passed
    uv run ruff format --check   1017 files already formatted
    npx vitest run               99 files, 1265 tests passed
    npx tsc --noEmit             clean, TurnEvent exhaustiveness included
    npm run lint:rpc             schema and codegen in sync
    make check-large-files       exit 0

rpc-schema/openrpc.json is the source of truth here, so ui-tui/src/rpc/
generated.ts was regenerated rather than hand-resolved. Nothing under
subagents/ changed.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-08-18 15:50:04 +08:00
LivXue
32f8044e86 feat(*): direct chat with a sub-agent instance, in the TUI and the web UI
## Summary

Adds direct chat with a sub-agent instance: talking to one of the instances a
session has spawned, without that exchange entering the main agent's context.
The TUI gets the feature end to end; the web UI then gets the same thing on the
same runtime.

**The runtime.** A direct turn runs on its own lane (`session#agent/handle`)
rather than the session's, so several instances answer at once while the main
agent keeps its own serial lane, and direct turns draw on a separate
concurrency pool so a user is never queued behind them. Direct turns are not
cancellable: `turn.cancel` keeps meaning the main agent's turn on both
surfaces. Each turn writes a record (`prompt.md` / `out.md` / `meta.json`), and
the main agent learns only that the exchange happened, through a handoff block.

**Sub-agent replies over ACP.** Three separate causes were losing most of a
reply. Raven answered every agent-initiated request with `method not found`,
including `session/request_permission`, and the codex adapter turns any error
from that request into a cancel -- so a turn died as soon as the agent asked to
run something. Raven now approves, choosing among the options the agent offered
by their protocol kind. A stop reason other than `end_turn` was returned as a
finished answer, so a truncated reply read as a complete one; it now carries a
notice. And several messages in one turn were joined with no boundary, so a
preamble, its tool calls and the answer ran together as one paragraph.

**Instance visibility.** A failed resume deleted the whole registry row, which
took the instance off the strip for the length of the turn recovering it, and
permanently if that turn failed; it now drops only the stale binding. A
capability snapshot whose fingerprint no longer matched was discarded, which
defaulted the agent to *stateless* -- editing an agent's `env` silently cost it
resume, its chip, and the `instance` parameter in the spawn schema. Its
capabilities are now kept and flagged, while its verdict is not, so the
`/subagents` row asks for a re-test.

**Working directory.** A direct chat was the one sub-agent dispatch that
supplied none, so it fell back to agent home and ran its commands against
raven's own memory instead of the user's checkout.

**The web UI.** Two wiring gaps came first, both silent: the browser's
dispatcher registered neither instance method, and the web channel never built
the map that tags an event with its addressee, so nothing a client could
demultiplex on was ever sent. The service then routes on that tag at the push
rather than at the read, because the main reply and a direct turn share one
subscription and two consumers on one queue would each swallow the other's
events -- including a `message.complete`, which would end the wrong turn. A
direct turn's events are published out of band; feeding them into the reply
stream is not an option, since that stream is one reply per turn.

Decisions and their measurements are recorded in `docs/specs/`. Notably, the
vendored AgentScope tree is *not* edited, which is a decision rather than an
omission: making a direct chat a real reply would mean re-keying a blocking
per-session lock that guards message storage and tool-call state as well as the
reply loop, and rendering reuse was obtained without it.

## Type

- [x] Feature

## Verification

- `uv run pytest -q` -- 8021 passed, 36 skipped. The one deselected failure,
  `tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare`, fails the
  same way on `origin/main` in a clean worktree, so this introduces none.
- `uv run ruff check raven/ tests/ ui-webui/service/` -- clean;
  `uv run ruff format --check` -- 957 files already formatted.
- Web service: `python -m pytest ui-webui/service/tests -q` -- 13 passed.
- Frontend: `pnpm lint` -- 0 errors, 18 warnings, each one also present before
  this change; `pnpm build` -- succeeds; `tsc --noEmit` -- clean.
- Every fix here was checked by reverting it and watching its test go red. Two
  tests that passed either way were rewritten after that check exposed them.
- End to end against a running gateway: a direct chat sent through the new web
  route completed twice, each leaving a record whose `out.md` held the exact
  reply asked for, and the history route returned those turns afterwards.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The last box is unchecked: the terms are defined in `CONTEXT.md` and the designs
in `docs/specs/`, but no end-user documentation or screenshots were produced.

## Risk

**Approving sub-agent permission requests is a widened trust boundary and the
main thing to review here.** An ACP sub-agent that asks to run something now
gets a yes, and the codex preset is configured for its most permissive mode --
wider than the `-a never` the cli transport used. It is unattended by
construction: a dispatch has no operator and no surface that could render a
prompt, so the alternatives were approving or refusing, and refusing leaves the
agent unable to do the work. Rolling back is per-agent config, or reverting
`raven/agent/acp/permissions.py` to restore the refusal.

Two known gaps, neither reachable today, both stated rather than fixed:

- The web UI's event consumer has no `target` guard of its own. The two
  surfaces use disjoint session keys and the browser never sends a target, so a
  direct turn cannot reach it; a caller that sent one on a `web:` session would
  see a sub-agent's reply rendered as the main agent's. Fixing it would widen
  this into unrelated files.
- Direct-chat record directories have no TTL and no count cap, like the spawn
  and DAG trees beside them. Reclaiming them would also start deleting audit
  trails that intentionally outlive session deletion, which is its own change.

Behaviour changes worth naming: a direct chat under a sandbox whose volumes do
not cover the working directory now fails where it used to run quietly in agent
home, matching what the main path already does; and a partial reply now carries
a `[raven]` notice, so anything parsing a sub-agent's reply verbatim sees it.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-08-17 02:52:57 +00:00
0xKT
796ae8f38c refactor(cli): retire the interactive repl and the cli cron channel (#329)
## Summary

Consolidate the interactive surface on the TUI. Bare `raven agent` no
longer hosts a REPL: it prints a pointer to `raven tui` and exits
non-zero, while `raven agent -m "..."` keeps working for scripted
one-shot turns. The REPL-only plumbing goes with it (the interactive
loop, slash-command handling, the exit-time pending-reminder notice);
CliOutlet and build_repl stay because the one-shot path renders through
them.

Cron follows the same consolidation (this builds on the fire-at-origin
ownership rule): the one-shot path no longer registers the cron tool -
with no REPL there is no runner left for a cli-bound reminder, so
creating one from a one-shot turn would mint a job that never fires.
The "cli" channel value retires: `cron add` validates `--channel`
against `tui` plus the enabled IM channels, and jobs stored with
`channel="cli"` by older builds migrate to `"tui"` at load time, one
log line per migrated job (old jobs.json files keep loading unchanged
otherwise).

## Type

- [x] Refactor

## Verification

- `uv run pytest tests/ --ignore=tests/integration` on this branch:
  5991 passed / 2 failed, both failures pre-existing on main and on the
  base branch run the same way (test_read_file_image, plus the
  order-dependent test_cli_theme flake that fails whenever its file
  runs as a whole) - zero failures introduced.
- Affected-area suites (onboard, agent CLI, cron CLI, repl spine,
  migration): 308 passed.
- `uv run ruff format --check` and `uv run ruff check`: clean.
- Real-machine checks (real CLI subprocesses under an isolated HOME):
  bare `agent` exits non-zero with the tui pointer; `cron add
  --channel cli` is rejected listing the valid channels; a legacy
  jobs.json with a cli-bound job shows the job migrated to tui on read.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Behavior change by design: the interactive REPL entry point is removed
(pointer + non-zero exit); scripts using `raven agent -m` are
unaffected. Stored cli-bound cron jobs migrate to tui automatically;
no data is dropped. Rollback = revert the squash commit.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-08-16 20:17:23 -07:00
Jelly Su
a5fa199377 feat(providers): add MiniMax Global and CN OAuth (#209)
## Summary

Add first-class MiniMax Token Plan OAuth support for both Global and CN
regions.

This change adds:

- `minimax_global` and `minimax_cn` provider entries with curated
MiniMax-M3 and M2.7 models.
- OAuth 2.0 device authorization with PKCE S256 and strict state
validation.
- The shared MiniMax public client ID `coding-plan-cli` with the `openid
profile coding_plan` scope.
- Region-specific OAuth and Anthropic-compatible API endpoints.
- Proactive access-token refresh with a five-minute buffer,
cross-process locking, transient retry handling, and atomic
refresh-token persistence.
- Separate Global and CN credential files, isolated from existing Codex
OAuth storage.
- Runtime token refresh and MiniMax authentication headers for both
normal and streaming requests.
- CLI login commands, onboarding, provider status/test/reset support,
TUI setup detection, and model-picker entries.
- English and Chinese documentation updates.

Public model references remain under `minimax-global/` and
`minimax-cn/`. The runtime translates them to the Anthropic-compatible
LiteLLM route only when sending a request, so provider selection
continues to use the correct MiniMax OAuth credential.

The implementation accepts MiniMax expiry values expressed as relative
seconds, Unix epoch seconds, or Unix epoch milliseconds. OAuth-returned
verification and resource URLs are restricted to the expected HTTPS
hosts before use.

## Type

- [ ] Fix
- [x] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests/test_minimax_oauth.py
tests/test_cli_provider_commands.py tests/test_provider_catalog.py
tests/test_config_update_providers.py tests/test_tui_rpc_model.py
tests/test_tui_rpc_setup.py tests/test_cli_status_commands.py -q -x` -
143 passed.
- Focused MiniMax onboarding tests - 3 passed.
- `uv run ruff check raven tests` - passed.
- `uv run ruff format --check raven tests` - 761 files already
formatted.
- `make check-large-files` - passed.
- `make check-commits` - passed.
- Two independent read-only reviews completed; the final review found no
blocking issues.
- The full test collection was not run because the local environment
does not include the optional `dingtalk_stream` channel dependency. All
provider, OAuth, onboarding, TUI setup, status, and configuration tests
relevant to this change passed.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- OAuth credentials are stored in region-specific files with restrictive
permissions and atomic replacement.
- Refresh operations use a cross-process lock and persist rotated
refresh tokens before exposing them to the runtime.
- OAuth verification and API resource URLs are validated against
region-specific host allowlists.
- Existing `minimax` API-key configuration remains unchanged; the new
OAuth providers use separate slugs and storage.
- Rollback consists of reverting this change. Existing API-key provider
configuration and credentials are not migrated or modified.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

- Live OAuth smoke test: MiniMax Global MiniMax-M3 returned "MiniMax
OAuth works" successfully after normalizing the Anthropic-compatible
resource URL.

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-07-28 11:14:54 +08:00
0xKT
dc0fb2822c raven init 2026-06-29 00:23:07 +08:00