15 Commits

Author SHA1 Message Date
Handsome-wzw
7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00
forrestjgq
f01231fef6 fix(cli): overhaul trajectory browser interaction ux (#379)
## Summary

Overhaul the interactive `raven trajectory` browser end to end, fixing
every
issue from real-terminal testing:

- Navigation: every screen opens with a help area under its title (page
  purpose plus "[Key] function" hotkey hints, rendered in a new readable
help color). Esc walks one level up (action -> attempt -> session); on
the
top-level session screen Esc stays put and quitting is Ctrl+C only,
which
prints a single exit notice. Esc inside a running action cancels just
that
action through a dedicated exception at the action boundary, so the back
  sentinel can never reach a conversion or the data layer; a cancelled
  report keeps its pre-confirm bundle-and-pin side effects (the existing
  declined-report contract). Back/Exit menu entries are gone; answered
  prompts erase themselves and navigation is echoed as breadcrumb lines
  with markup-escaped untrusted text.
- Tables: the session list is TITLE | ATTEMPTS | LAST ACTIVITY and the
  attempt list is # | STARTED | TURNS | SPANS | VERDICT | PIN | MERGED |
PREVIEW with green check cells. Time cells keep the year. The layout is
  budgeted per screen build from the live terminal width (row indent, a
safety margin, and inter-column gaps are all part of the math); PREVIEW
  drops as a whole column when it no longer fits, and below the dynamic
  minimum width the tightest layout is kept while the renderer clips
  overlong lines at the terminal edge. Every dynamic cell passes a
  single-line sanitization gate before width math (verdict sidecars only
  guarantee a non-empty string), cells are measured and truncated by
terminal display width (CJK is 2 cells), and fixed column widths always
  fit their headers.
- Space preview: Space on an attempt row prints that attempt's per-turn
input/output previews, collected in the same snapshot scan and sorted by
a fully-stringified key so ordering never follows log source order (the
  table PREVIEW cell derives from the same sorted collection). Any key,
arrow keys included, returns with the cursor kept on the row; the waiter
  passes the terminal's CPR reply through to the renderer instead of
  treating it as a keypress, and Ctrl+C keeps the browser-wide cancel.
- Merge hotkey: with two or more attempts, m/M opens the multi-select
merge
  (advertised in the help line; the old "Merge attempts" menu entry is
gone). The checkbox screen carries its own help line for toggle,
confirm,
  and cancel. A single attempt neither binds nor advertises the key.
- Theme and feel: new success (green check) and help (readable hint
text)
colors in both palettes, all WCAG AA on their backgrounds; a lone ESC is
flushed after 50ms instead of prompt_toolkit's 0.5s default, removing
the
  laggy Esc feel.

The data layer is untouched: changes are confined to
raven/cli/trajectory_browse.py, raven/cli/_theme.py, and their tests.
The
id-taking subcommands (raven trajectory save/list/...) behave as before.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `COLORTERM=truecolor uv run pytest tests/test_cli_theme.py
tests/test_cli_trajectory_browse.py
tests/test_cli_trajectory_commands.py -q` -> 220 passed
- `COLORTERM=truecolor uv run pytest tests/ --ignore=tests/integration
-q` -> 6917 passed; the 10 failed + 20 errors are the pre-existing
environment-dependent set, identical file-by-file on a clean main
checkout (cron, everos server, config loader, tracing viewer)
- `uv run ruff check` and `uv run ruff format --check` on the four
touched files -> clean
- PTY smoke on a CPR-answering pseudo terminal at 100 and 58 columns:
full flow (preview, merge cancel, action screen, staged Esc backout,
Ctrl+C quit) and the narrow-width degradation contract
- New coverage includes real pipe-input key-binding tests (Esc on all
four prompt kinds, Space/m/M hits, waiter key paths with CPR
pass-through), table budget/clipping boundaries at the computed minimum
width, and corrupt verdict/timestamp corpora

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- The interactive browser's look and key protocol change wholesale (that
is
  the point of this PR): Esc no longer quits from the top level (Ctrl+C
  does), Back/Exit entries are gone, and lists render as tables. The
machine-facing subcommands and the trajectory data layer are untouched.
- Rollback: revert the squash commit; the browser returns to the
previous
  interaction wholesale, no data migration involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>
2026-09-05 23:58:20 +08:00
0xKT
3a7cc17aeb fix(*): raven-research takes the host web vendor and key, and the ask_user gate closes
## Summary

Three strands of raven-research work, held together because they all touch the vendored fork and so share one `subagents/TREE_HASHES` line.

**Web search and fetch route through the vendor the deployment picked.** Both tools in the research-flow plugin were pinned to Serper and Jina, so a host on Tavily or Brave still needed a Serper key for the pair. They now carry the kernel's vendor tables (seven search backends, five fetch) with per-vendor request encoding and response normalisation. Serper and Jina wire traffic is byte-identical to before. A vendor that publishes no result offset declares it up front (`SearchProviderSpec.paginates`), so the saturation rule never escalates to a page rung the vendor cannot answer.

The launcher resolves the vendor first and mirrors the SELECTED vendor's key into the plugin slice, never a key by fixed path: a mismatched pair sends one vendor's credential to another's endpoint and every call comes back unauthorized. Every vendor gets an env slot at the same path the host holds it, inherited from the host config when unset, so a deployment configures a vendor once, centrally. The two legacy vendors are also read from the vendor slot, where a host configured after the vendor table landed writes them. `require_search` follows the selection rather than the default.

**The research modes take the host's effort vocabulary**: `deep` becomes `high`, `ultra` becomes `max`, and `medium` is the baseline. This is what lets the companion MR's host tier reach this agent by name.

**The ask_user gate closes after a hook rollback**, and host questions take the conversation lock like every other route, including the `direct_ask` grant. The digest follows its session's mode, a typo in a mode overlay refuses to launch rather than running the base value under the overlay's label, and the reviewer records which model judged.

Three defects found while building it, each with the test that pins it:

- **A credential reached disk.** The retry rows of the search and fetch ledger recorded `str(exc)` for a status error, and httpx puts the whole request URL in that text. SerpApi authenticates by query parameter, so a 429 or 503 wrote the key into the JSONL file. The redaction the search and fetch rows already had is now one function, `_error_text`, used by both writers, in the plugin and in the vendored copy. The regression test drives a retryable status and reads the ledger file off disk; the previous test used 401, which is not retryable and so never reached the leaking write. The key never reached the model or the answer: the appendix partitions on `search` / `fetch` / `verify` / `force_finalize` and reads nothing from a retry row.
- **A keyless fetch backend refused every call.** The kernel's `web_fetch` degrades a keyed backend with no resolvable key to Jina, out loud. The plugin's replacement did not, and it REPLACES the kernel's, so `fetch.provider = tavily` with no Tavily key made a research run the single place that config could not read a page, while a plain raven on it read the page fine. The launcher itself produces that state: it writes the provider unconditionally and the key only when one resolves. Fixed at the build site, where the kernel resolves it, and a test asserts both implementations answer identically for every vendor.

  Reported by chandler.zhang and completed in `ac7d364c`: the degraded backend was still handed the key of the vendor it replaced, which is absent by definition, so a deployment holding a Jina key read pages authenticated through raven's built-in fetch and anonymously through the plugin -- losing the configured quota on the very fallback the rule exists to match. The kernel looks the key up a second time under the EFFECTIVE provider; the slice carries one key per role and no vendor table, so the launcher now mirrors the default reader's key beside the selected vendor's and the build picks by destination, a lookup keyed on the provider the tool will actually call. It is mirrored whenever the selection is not already the default rather than only when the fallback will fire, because the rule that decides that lives in `effective_provider` and a launcher second-guessing it would be the copy that drifts.
- **A typo in a vendor name refused to launch for the wrong reason.** The plugin degrades an unknown vendor to its default, which is right for a slice nothing validates. Carried through the launcher, the same typo had `require_search` demand a key for a vendor that does not exist while the working Serper key sat in the config -- sending the operator to set a credential they already had. The launcher now refuses naming the name and the valid list.

`.env.example` and the plugin docstrings were still describing the fixed Serper and Jina pair; both now cover the provider switch and the per-vendor key slots. The plugin MANIFEST was missed in that pass and is corrected in `18ce29df`, also on chandler.zhang's report: it still called `search.apiKey` a Serper key and `fetch.apiKey` a Jina reader key, and said web_search declines "when no Serper key resolves". A direct plugin user reaching it without the launcher could set `search.provider = tavily` and supply a Serper credential, which this code then sends to Tavily's endpoint.

`subagents/TREE_HASHES` is recorded once, in the final commit, rather than per commit: the hash covers every tracked byte under the fork, so an intermediate value goes stale the moment the next edit or a rebase lands.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch at its current head:

- `uv run pytest` -- 5 failed, 18545 passed, 69 skipped. The five are this host's pre-existing baseline, identical on `origin/main`: `test_cli_doctor_commands.py::test_doctor_answers_where_the_memories_are` and four `test_security_network.py` NAT64 parameters. 0 introduced.
- `make lint-python` -- passed, 1787 files already formatted.
- `uv run lint-imports` -- 8 contracts kept, 0 broken.
- `make check-vendored-subagents` -- exit 0, hash recorded.
- `make check-vendored-invariants`, `make check-large-files`, `make lint-ui`, `make lint-tui`, `scripts/check_commit_messages.py origin/main..HEAD` -- all exit 0.
- The vendored fork's own suite, run with the fork on `PYTHONPATH` so `raven` resolves to its copy: 19 failed, 6223 passed, 4 errors. Reverting only this branch's change to the fork gives the same 19 failures and 4 errors, all in three files that never import the changed module, so 0 introduced. Those failures are an artifact of running the fork against the trunk's venv, not the fork's own.

Reverting each fix and confirming the matching test fails: the ledger redaction (the 429 case red, the 401 case correctly still green), the fetch degradation (4 keyed vendors red, Jina correctly unaffected), the vendor-name refusal (2 red), and the slice key following the selected vendor (2 red).

Three more for the review fix: the build passing the selected vendor's key to the reader that replaced it (1 red), the launcher not mirroring the fallback key (1 red), and the launcher mirroring it for the default vendor too (1 red, the case where a second copy of the same key is one more place for the pair to disagree).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Security: a credential stopped being written to disk. Any research-flow ledger file written before this change by a deployment on SerpApi may contain the key in a `search_retry` or `fetch_retry` row. Those files live under the plugin's state root, one per turn, and are unlinked at the end of the turn that wrote them; a process killed mid-turn leaves one behind. Treat a SerpApi key used on an affected build as exposed and rotate it, then check for leftover files.

Other user-visible changes:

- A research deployment that selected a keyed fetch vendor without its key now reads pages through Jina instead of failing every fetch, and does so on the host's configured Jina key rather than anonymously. Behaviour improves; the log says which vendor is serving.
- That fallback key now travels in the rendered plugin slice as `fetch.fallbackApiKey`, beside the selected vendor's key, wherever the rendered config already holds credentials. It is only ever handed to the default reader: the build looks the key up by destination, so a provider neither key belongs to gets none.
- An unknown vendor name in `search.provider` or `fetch.provider` now refuses to launch. A deployment running on a typo today would have been failing its search calls anyway.
- The research mode ids change: `deep` becomes `high` and `ultra` becomes `max`. A caller that names `deep` by id gets the agent's default instead.
- Ledger `*_retry` rows record `HTTPStatusError: HTTP <status>` where they recorded the full httpx message. The one offline reader, the fork's `bench/rollout.py`, buckets by status either way and already read the search and fetch rows in that shape.

Rollback: revert the commits on this branch. `TREE_HASHES` is recorded in its own final commit, so reverting the set restores the recorded hash with the tree.

## Related Issues

N/A

<!-- The repo guide says to leave this trailer to the commits, on the premise that
     the platform collects it into the squash commit. This GitLab does not, and the
     squash body is taken from this description, so it is repeated here to survive. -->

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-06 04:12:09 +00:00
userName20260323
fec055779d fix(*): raven-research takes the host web vendor and key, and the ask_user gate closes
## Summary

Three strands of raven-research work, held together because they all touch the vendored fork and so share one `subagents/TREE_HASHES` line.

**Web search and fetch route through the vendor the deployment picked.** Both tools in the research-flow plugin were pinned to Serper and Jina, so a host on Tavily or Brave still needed a Serper key for the pair. They now carry the kernel's vendor tables (seven search backends, five fetch) with per-vendor request encoding and response normalisation. Serper and Jina wire traffic is byte-identical to before. A vendor that publishes no result offset declares it up front (`SearchProviderSpec.paginates`), so the saturation rule never escalates to a page rung the vendor cannot answer.

The launcher resolves the vendor first and mirrors the SELECTED vendor's key into the plugin slice, never a key by fixed path: a mismatched pair sends one vendor's credential to another's endpoint and every call comes back unauthorized. Every vendor gets an env slot at the same path the host holds it, inherited from the host config when unset, so a deployment configures a vendor once, centrally. The two legacy vendors are also read from the vendor slot, where a host configured after the vendor table landed writes them. `require_search` follows the selection rather than the default.

**The research modes take the host's effort vocabulary**: `deep` becomes `high`, `ultra` becomes `max`, and `medium` is the baseline. This is what lets the companion MR's host tier reach this agent by name.

**The ask_user gate closes after a hook rollback**, and host questions take the conversation lock like every other route, including the `direct_ask` grant. The digest follows its session's mode, a typo in a mode overlay refuses to launch rather than running the base value under the overlay's label, and the reviewer records which model judged.

Three defects found while building it, each with the test that pins it:

- **A credential reached disk.** The retry rows of the search and fetch ledger recorded `str(exc)` for a status error, and httpx puts the whole request URL in that text. SerpApi authenticates by query parameter, so a 429 or 503 wrote the key into the JSONL file. The redaction the search and fetch rows already had is now one function, `_error_text`, used by both writers, in the plugin and in the vendored copy. The regression test drives a retryable status and reads the ledger file off disk; the previous test used 401, which is not retryable and so never reached the leaking write. The key never reached the model or the answer: the appendix partitions on `search` / `fetch` / `verify` / `force_finalize` and reads nothing from a retry row.
- **A keyless fetch backend refused every call.** The kernel's `web_fetch` degrades a keyed backend with no resolvable key to Jina, out loud. The plugin's replacement did not, and it REPLACES the kernel's, so `fetch.provider = tavily` with no Tavily key made a research run the single place that config could not read a page, while a plain raven on it read the page fine. The launcher itself produces that state: it writes the provider unconditionally and the key only when one resolves. Fixed at the build site, where the kernel resolves it, and a test asserts both implementations answer identically for every vendor.

  Reported by chandler.zhang and completed in `ac7d364c`: the degraded backend was still handed the key of the vendor it replaced, which is absent by definition, so a deployment holding a Jina key read pages authenticated through raven's built-in fetch and anonymously through the plugin -- losing the configured quota on the very fallback the rule exists to match. The kernel looks the key up a second time under the EFFECTIVE provider; the slice carries one key per role and no vendor table, so the launcher now mirrors the default reader's key beside the selected vendor's and the build picks by destination, a lookup keyed on the provider the tool will actually call. It is mirrored whenever the selection is not already the default rather than only when the fallback will fire, because the rule that decides that lives in `effective_provider` and a launcher second-guessing it would be the copy that drifts.
- **A typo in a vendor name refused to launch for the wrong reason.** The plugin degrades an unknown vendor to its default, which is right for a slice nothing validates. Carried through the launcher, the same typo had `require_search` demand a key for a vendor that does not exist while the working Serper key sat in the config -- sending the operator to set a credential they already had. The launcher now refuses naming the name and the valid list.

`.env.example` and the plugin docstrings were still describing the fixed Serper and Jina pair; both now cover the provider switch and the per-vendor key slots. The plugin MANIFEST was missed in that pass and is corrected in `18ce29df`, also on chandler.zhang's report: it still called `search.apiKey` a Serper key and `fetch.apiKey` a Jina reader key, and said web_search declines "when no Serper key resolves". A direct plugin user reaching it without the launcher could set `search.provider = tavily` and supply a Serper credential, which this code then sends to Tavily's endpoint.

`subagents/TREE_HASHES` is recorded once, in the final commit, rather than per commit: the hash covers every tracked byte under the fork, so an intermediate value goes stale the moment the next edit or a rebase lands.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch at its current head:

- `uv run pytest` -- 5 failed, 18545 passed, 69 skipped. The five are this host's pre-existing baseline, identical on `origin/main`: `test_cli_doctor_commands.py::test_doctor_answers_where_the_memories_are` and four `test_security_network.py` NAT64 parameters. 0 introduced.
- `make lint-python` -- passed, 1787 files already formatted.
- `uv run lint-imports` -- 8 contracts kept, 0 broken.
- `make check-vendored-subagents` -- exit 0, hash recorded.
- `make check-vendored-invariants`, `make check-large-files`, `make lint-ui`, `make lint-tui`, `scripts/check_commit_messages.py origin/main..HEAD` -- all exit 0.
- The vendored fork's own suite, run with the fork on `PYTHONPATH` so `raven` resolves to its copy: 19 failed, 6223 passed, 4 errors. Reverting only this branch's change to the fork gives the same 19 failures and 4 errors, all in three files that never import the changed module, so 0 introduced. Those failures are an artifact of running the fork against the trunk's venv, not the fork's own.

Reverting each fix and confirming the matching test fails: the ledger redaction (the 429 case red, the 401 case correctly still green), the fetch degradation (4 keyed vendors red, Jina correctly unaffected), the vendor-name refusal (2 red), and the slice key following the selected vendor (2 red).

Three more for the review fix: the build passing the selected vendor's key to the reader that replaced it (1 red), the launcher not mirroring the fallback key (1 red), and the launcher mirroring it for the default vendor too (1 red, the case where a second copy of the same key is one more place for the pair to disagree).

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Security: a credential stopped being written to disk. Any research-flow ledger file written before this change by a deployment on SerpApi may contain the key in a `search_retry` or `fetch_retry` row. Those files live under the plugin's state root, one per turn, and are unlinked at the end of the turn that wrote them; a process killed mid-turn leaves one behind. Treat a SerpApi key used on an affected build as exposed and rotate it, then check for leftover files.

Other user-visible changes:

- A research deployment that selected a keyed fetch vendor without its key now reads pages through Jina instead of failing every fetch, and does so on the host's configured Jina key rather than anonymously. Behaviour improves; the log says which vendor is serving.
- That fallback key now travels in the rendered plugin slice as `fetch.fallbackApiKey`, beside the selected vendor's key, wherever the rendered config already holds credentials. It is only ever handed to the default reader: the build looks the key up by destination, so a provider neither key belongs to gets none.
- An unknown vendor name in `search.provider` or `fetch.provider` now refuses to launch. A deployment running on a typo today would have been failing its search calls anyway.
- The research mode ids change: `deep` becomes `high` and `ultra` becomes `max`. A caller that names `deep` by id gets the agent's default instead.
- Ledger `*_retry` rows record `HTTPStatusError: HTTP <status>` where they recorded the full httpx message. The one offline reader, the fork's `bench/rollout.py`, buckets by status either way and already read the search and fetch rows in that shape.

Rollback: revert the commits on this branch. `TREE_HASHES` is recorded in its own final commit, so reverting the set restores the recorded hash with the tree.

## Related Issues

N/A

<!-- The repo guide says to leave this trailer to the commits, on the premise that
     the platform collects it into the squash commit. This GitLab does not, and the
     squash body is taken from this description, so it is repeated here to survive. -->

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-06 04:12:09 +00:00
0xKT
2f862d8ca2 feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
hylarucoder
c703db6ffd feat(build): RAVEN_WHEEL_SLIM switch for a headless cloud wheel
## What

Adds a `RAVEN_WHEEL_SLIM=1` build switch to `hatch_build.py`: when set, the wheel build skips the vendored sub-agent tree and the prebuilt TUI/web bundle inclusion. Default builds are byte-identical to before.

## Why

The release wheel is ~106 MiB compressed; ~94% is the `subagents/` snapshots (raven-ppt alone carries 74 MiB of `.pptx` templates and fonts) plus UI bundles. A cloud provisioner that only ever runs `raven acp` inside a sandbox uploads all of that per instance for assets nothing there opens. The slim wheel is ~3.0 MiB with the full python package intact, cutting sandbox provisioning upload from 15-23s to under a second.

## Behavior on slim installs

- Sub-agents: absent by design; onboarding step 5 reports no tree (headless installs never reach it). Their own distribution channel is future work.
- `raven tui` / `raven serve`: fall back to the source-checkout resolvers exactly as a dev build without prebuilt bundles does today; headless installs never call them.

## Verified

- `RAVEN_WHEEL_SLIM=1 uv build --wheel` -> 3.0 MiB wheel, 734 files, `raven/acp` complete, no `raven/subagents` or UI dist entries.
- The slim wheel provisioned a real E2B-compatible sandbox and served a full `raven acp` session end to end (delegated task with shell execution, ask_user elicitation round trip, multi-turn session continuity).
- Default build without the variable is unchanged.
2026-09-05 17:09:13 +00:00
0xKT
009d0474e6 feat(*): subagent discovery switches rails from the forks to agents/
## Summary

Subagent discovery switches rails from the subagents/ fork trees to the
agents/ product tree. This makes the five migrated products (code, oncall,
ppt, research, design) the factory default: a fresh install discovers them
without running any per-product installer, and the fork trees stop being
scanned entirely (they remain on disk, untouched, until their separately
scheduled retirement).

Key decisions:

- Discovery resolves an agents root through the same three levels the fork
  scan used: raven home copy, beside the package, wheel-embedded. The copy
  to raven home is kept folders-only so a product's .env survives upgrades.
- Readiness has new semantics: the launcher's absolute-path command tokens
  must exist, and a product-declared engine distribution must be importable
  (importlib find_spec). A product whose engine wheel is missing is listed
  but disabled, naming the wheel to install. Credential checks leave the
  discovery phase: the launcher itself refuses loudly, which live handshake
  smoke verified for all five products.
- Config rows keep whole-row precedence over discovered rows; the merge
  mechanics are unchanged (verified byte-equivalent modulo renames by an
  independent adversarial review).
- The main wheel now ships the five product launchers under raven/agents
  and no longer embeds raven/subagents (118 files / 1.3 MB, verified by
  building and listing the wheel; zero pyproject changes, the inclusion
  lives in the hatch build hook).
- The design/ppt manifests gain one machine-readable field,
  `engine: {package, wheel}`, consumed by the readiness probe; launch
  behavior is untouched and older readers ignore the field (verified).
- A discovered row whose readiness is clean but whose runtime probe misses
  now renders as attention instead of a dead Install button (fix from the
  adversarial review).

An independent adversarial review (verdict: land after fixes) preceded this
MR; its one HIGH (net test-coverage loss on host_can_lend_a_key edge
branches) and one MED (the Install dead-end above) are fixed in the commit,
with companion tests. Fork-era behaviors it confirmed as bit-identical
inheritances (home-first shadowing, path-with-spaces refusal) are recorded,
not changed.

## Type

- [x] Feature

## Verification

- Full canonical suite on the branch: `uv run pytest tests -q` under
  `uv sync --all-extras` -> 18494 passed, 40 skipped, exit 0 (rehearsal
  tree); 18472 passed, 62 skipped on the drafting stack. Net def-level
  movement vs baseline accounted name-for-name in the wave report.
- Targeted families at landing: tests/test_agents_ppt_launcher.py,
  tests/test_cli_subagent_setup.py, tests/test_hatch_build.py,
  tests/test_rpc_subagents.py, tests/test_subagent_dag_machines.py,
  tests/test_subagent_vendored_agents.py -> 242 passed.
- `make lint-python` -> exit 0; `uv run lint-imports` -> 8 kept, 0 broken;
  `make check-vendored-subagents`, `check-vendored-invariants`,
  `check-large-files` -> exit 0. subagents/ is byte-untouched.
- Wheel audit: `uv build --wheel`, listed contents -> 5 launchers +
  5 manifests under raven/agents, 0 raven/subagents entries, no secrets.
- Fresh-venv install test: wheel installed into an isolated venv discovers
  and materializes all five rows; design/ppt correctly disabled without
  their engine wheels; research launcher runs to its credential verdict.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: fork rows stop appearing on fresh installs; the five
product rows appear instead. Machines with existing config rows are
unaffected (config precedence unchanged). The RPC wire shape is frozen
(vendored/building keys and subagents.build kept), so no client regeneration
is needed. Rollback: revert the squash commit; discovery returns to the
fork trees, which are still on disk in the repo and still embedded in any
wheel built before this change.

## Related Issues

N/A
2026-09-05 05:13:56 +00:00
0xKT
9a98c3107f feat(*): subagent discovery switches rails from the forks to agents/
## Summary

Subagent discovery switches rails from the subagents/ fork trees to the
agents/ product tree. This makes the five migrated products (code, oncall,
ppt, research, design) the factory default: a fresh install discovers them
without running any per-product installer, and the fork trees stop being
scanned entirely (they remain on disk, untouched, until their separately
scheduled retirement).

Key decisions:

- Discovery resolves an agents root through the same three levels the fork
  scan used: raven home copy, beside the package, wheel-embedded. The copy
  to raven home is kept folders-only so a product's .env survives upgrades.
- Readiness has new semantics: the launcher's absolute-path command tokens
  must exist, and a product-declared engine distribution must be importable
  (importlib find_spec). A product whose engine wheel is missing is listed
  but disabled, naming the wheel to install. Credential checks leave the
  discovery phase: the launcher itself refuses loudly, which live handshake
  smoke verified for all five products.
- Config rows keep whole-row precedence over discovered rows; the merge
  mechanics are unchanged (verified byte-equivalent modulo renames by an
  independent adversarial review).
- The main wheel now ships the five product launchers under raven/agents
  and no longer embeds raven/subagents (118 files / 1.3 MB, verified by
  building and listing the wheel; zero pyproject changes, the inclusion
  lives in the hatch build hook).
- The design/ppt manifests gain one machine-readable field,
  `engine: {package, wheel}`, consumed by the readiness probe; launch
  behavior is untouched and older readers ignore the field (verified).
- A discovered row whose readiness is clean but whose runtime probe misses
  now renders as attention instead of a dead Install button (fix from the
  adversarial review).

An independent adversarial review (verdict: land after fixes) preceded this
MR; its one HIGH (net test-coverage loss on host_can_lend_a_key edge
branches) and one MED (the Install dead-end above) are fixed in the commit,
with companion tests. Fork-era behaviors it confirmed as bit-identical
inheritances (home-first shadowing, path-with-spaces refusal) are recorded,
not changed.

## Type

- [x] Feature

## Verification

- Full canonical suite on the branch: `uv run pytest tests -q` under
  `uv sync --all-extras` -> 18494 passed, 40 skipped, exit 0 (rehearsal
  tree); 18472 passed, 62 skipped on the drafting stack. Net def-level
  movement vs baseline accounted name-for-name in the wave report.
- Targeted families at landing: tests/test_agents_ppt_launcher.py,
  tests/test_cli_subagent_setup.py, tests/test_hatch_build.py,
  tests/test_rpc_subagents.py, tests/test_subagent_dag_machines.py,
  tests/test_subagent_vendored_agents.py -> 242 passed.
- `make lint-python` -> exit 0; `uv run lint-imports` -> 8 kept, 0 broken;
  `make check-vendored-subagents`, `check-vendored-invariants`,
  `check-large-files` -> exit 0. subagents/ is byte-untouched.
- Wheel audit: `uv build --wheel`, listed contents -> 5 launchers +
  5 manifests under raven/agents, 0 raven/subagents entries, no secrets.
- Fresh-venv install test: wheel installed into an isolated venv discovers
  and materializes all five rows; design/ppt correctly disabled without
  their engine wheels; research launcher runs to its credential verdict.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible change: fork rows stop appearing on fresh installs; the five
product rows appear instead. Machines with existing config rows are
unaffected (config precedence unchanged). The RPC wire shape is frozen
(vendored/building keys and subagents.build kept), so no client regeneration
is needed. Rollback: revert the squash commit; discovery returns to the
fork trees, which are still on disk in the repo and still embedded in any
wheel built before this change.

## Related Issues

N/A
2026-09-05 05:13:56 +00:00
0xKT
8f6ef4c0ce ci(*): the governance batch brings the rulebook current and makes it enforced
## Summary

Governance hardening batch: the collaboration rulebook catches up with the
post-layering tree, the repository's comments and records are brought up to
its own standards, and the standards themselves become enforced gates.

Five commits, in series:

1. docs: the rulebook catches up -- AGENTS.md gains the source-language rule
   with the owner-signed CJK zones, a portable PR check replaces the
   GitHub-only wording, README's repo-layout table becomes the canonical
   scope list, CONTEXT-MAP gains its third route, and CONTEXT.md records the
   migration vocabulary.
2. docs: 133 surgical comment fixes across 83 files -- comments now say why,
   and only why; edit markers and restated-what comments are gone.
3. docs: stale orphans leave -- 5 dead documents deleted, 4 dated records
   stamped as historical, the zh README shrinks to a translated landing
   page, and the living-docs guard widens its glob.
4. ci: the honor system grows teeth -- scripts/check_source_language.py
   (diff-based, three passes), a commitlint scope enum computed from the
   tree, tests/test_readme_scope_canon.py, and the CI/make wiring for all
   three.
5. fix: a second pass brings the batch current with what landed on main
   after it was drafted -- the design engine's two owner-signed zones enter
   the gate and the rulebook, the scope enum learns the fifth product and
   the wheel distributions, CONTEXT.md records the third distributed member
   and the Visual Domain Selector, and one zh fixture added by a follow-on
   wave is respelled to English with its placeholder semantics preserved.

The batch is docs and CI only: no runtime behavior changes. The one source
edit outside docs/CI is the fixture respell in commit 5, verified by its own
test family.

## Type

- [x] CI / tooling

## Verification

- Full suite on the batch tree: `uv run --frozen --all-extras pytest tests -q`
  -> 18513 passed, 40 skipped, exit 0.
- `make lint-python` -> All checks passed; `uv run lint-imports` -> 0 broken.
- Guard families after landing commit 5: `uv run pytest
  tests/test_playbook_generator.py tests/test_commit_lint.py
  tests/test_living_docs.py tests/test_readme_scope_canon.py
  tests/test_source_language_check.py -q` -> 55 passed.
- Language-gate self-run over the batch's own range and over main's history
  since the batch base: exit 0 (one genuine pre-existing violation found and
  fixed in commit 5).
- Commitlint over all five batch subjects: 5/5 pass; `make check-large-files`
  -> exit 0.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No runtime code paths change. The new CI gates can reject future commits by
design; if a gate misfires, reverting the squash commit removes the gates
and the rulebook text together. Existing CJK carriers in main's history are
exempt: the gate is diff-based and only judges new changes.

## Related Issues

N/A
2026-09-05 05:13:15 +00:00
0xKT
3820ec8f4f fix(commitlint): unify commit/PR length to header-max-length (#196)
## Change description

Follow-up to #187 (which fixed the subject-case dimension). The length
dimension
had the same "passes the PR, fails after merge" flaw:
`scripts/commit_lint.py`
capped the subject at 72 (commits) / 90 (PR titles) while commitlint
capped the
whole header at 100, so a PR title with a 73-90 char subject passed the
pre-merge
PR-title check but failed commit-lint on the squash commit after merge.

This makes commitlint `header-max-length` (100) the single length
authority (the
config-conventional standard) and removes the bespoke Python 72/90 caps.
CI now
validates the PR title as the prospective squash header `<title> (#NN)`
(in the
commit-messages job, which already has Node), so a title that passes
cannot fail
after merge; the `edited` trigger re-checks title edits. Deliberately
NOT
`extends @commitlint/config-conventional`, to avoid pulling in
body/footer
line-length rules.

Verified locally: proper-noun titles pass, a 79-char subject (header
102) is now
rejected at PR stage, header exactly 100 passes and 101 fails, Title
Case still
rejected, `tests/test_commit_lint.py` green.

## Type of change
- [x] Bug fix
- [ ] New feature
- [ ] Document
- [ ] Others

## Related issues (if there is)

Closes #195

## Checklists

### Development

- [x] Lint rules pass locally
- [x] Application changes have been tested thoroughly
- [x] Automated tests covering modified code pass

### Security

- [x] Security impact of change has been considered
- [x] Code follows security best practices and guidelines

### Code review

- [x] Pull request has a descriptive title and context useful to a
reviewer. Screenshots or screencasts are attached as necessary

Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
2026-07-23 16:25:22 +08:00
0xKT
ee9f583b30 ci: enforce ascii-only pr body and widen the self-check grep (#98)
## Summary

A non-ASCII PR body slipped onto main via #97: the repo squash-merges
with
the PR description as the commit body, but nothing checked that body
before
merge, so an em-dash only tripped the post-merge commit lint on main
(too
late to fix cleanly). Close the gap at both the guidance and the
enforcement
layer.

- Enforcement (CI): add `check_pr_body` (ASCII-only, reusing
`commit_lint._is_ascii`) and a "Validate PR body" step in the
pull_request
job, so a non-ASCII PR body fails the PR check before merge and points
at
  the offending line.
- Guidance (AGENTS.md 3.7): the preview grep now uses `[^\x00-\x7F]`
(any
  non-ASCII), not a CJK-only pattern that gave a false pass.
- Guidance (AGENTS.md 3.1.1): state the punctuation rule is ASCII-only,
not
  merely no-full-width (calls out em-dash / curly quotes / ellipsis).

## Type

- [x] CI / tooling
- [x] Docs

## Verification

- [x] Relevant tests pass locally

```
uv run pytest tests/test_commit_lint.py -q                    # 12 passed
PR_BODY="line with a non-ascii dash" python3 scripts/check_pr_body.py  # exit 1, flags line
python3 scripts/check_commit_messages.py "origin/main..HEAD"  # PASS
grep -nP "[^\x00-\x7F]" <this body>                           # 0 matches
```

## Risk

- [x] Backward compatibility considered

The new check runs in the existing required pull_request job, so it is
enforced immediately without a new required-check name. ASCII-only PR
bodies
are already implied by the squash-to-main constraint, so this only moves
the
check earlier (pre-merge instead of post-merge).

## Related Issues

N/A (follow-up to the commit-lint failure on #97)

---------

Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
2026-07-07 21:20:25 +08:00
JasonJarvan
256b78b23b build: whole-repo lint + pre-commit convention (config only) (#59)
## Change description

> Whole-repo lint + pre-commit convention, CONFIG ONLY (no source
reformat in this PR). Python (ruff) and TypeScript (eslint + prettier)
share a 120-col margin via the committed pre-commit gate. This upgrades
main's existing basic ruff setup and adds the TS lint stack (none
existed on main).
>
> What this PR adds:
> - ruff: line-length 120; security rules
S101/S105/S106/S107/S506/S605/S607; policy per-file-ignores (tests +
accepted CLI deferred-imports + product asserts + N818, each commented);
ruff pinned <0.16.
> - eslint/prettier: shared base (`eslint.base.mjs` +
`prettier.config.mjs`); ui-tui extends it with React rules, bridge with
a Node-only config; hermes-ink vendor and generated RPC types excluded.
> - pre-commit: keeps main's hooks (conventional-commit,
detect-private-key, etc.), adds a top-level exclude and 4 local
eslint/prettier hooks; bumps the pre-commit dev dep.
> - 2 lint-surfaced fixes: PlannerContext import (F821) in the sentinel
runner; lower-cased file-local cron-heartbeat constants (N806).
(raven/config/raven.py's F821 was already fixed on main.)
>
> Sequencing: config only. `ruff check` is intentionally red on ~307
auto-fixables (I001/F401/F541/E701/E702); a SEPARATE reformat branch
runs `ruff --fix` + `ruff format` + eslint/prettier --fix to clear them
(100% auto-resolvable, zero manual). Enforcement (pre-commit install /
CI) turns on only after the reformat lands, so do not expect a green
tree from this PR alone.
>
> CLI-owned files (the reformat and several per-file-ignores touch
`raven/cli/*`) are under review by the CLI team. Judgment items (TS
no-explicit-any, no-unused-expressions, S101 convert-vs-keep, N818
rename, C90, mypy) are deferred to v2.

## Type of change
- [ ] Bug fix
- [x] New feature
- [ ] Document
- [x] Others

## Related issues (if there is)

> none

## Checklists

### Development
- [ ] Lint rules pass locally
- [ ] Application changes have been tested thoroughly
- [ ] Automated tests covering modified code pass

> config-only: `ruff check` is red on auto-fixables by design (the
reformat branch clears them); eslint/prettier configs load in both
ui-tui and bridge.

### Security
- [x] Security impact of change has been considered
- [ ] Code follows company security practices and guidelines

> Only dependency change is the dev-only pre-commit tool plus the ruff
pin; the detect-private-key hook from main is retained.

### Code review
- [x] Pull request has a descriptive title and context useful to a
reviewer
- [ ] Pull request linked to JIRA ticket when applicable

---------

Co-authored-by: Sheng Zhao <sheng.zhao@evermind.ai>
Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
2026-07-01 12:09:42 +08:00
Elliot Chen
0b0ecbebf9 Merge pull request #10 from EverMind-AI/codex/open_source_readme_ci
chore: prepare open source launch
2026-06-30 12:00:30 +08:00
libin.zhang
35444df3bb refactor(memory): scope EverOS home under ~/.everos/raven
EverOS defaulted both its user-level config toml and its data root
(sqlite / lancedb / .index / ome.db) to a bare ~/.everos. Scope raven's
instance under ~/.everos/raven so config and data stay isolated from any
other EverOS consumer on the machine.

Add update_everos.configure_everos_env(): it sets EVEROS_CONFIG_FILE and
EVEROS_MEMORY__ROOT via setdefault (so an explicit operator override
still wins) before EverOS's @cache-d load_settings() first runs. The
everos backend factory and the understand_media tool factory call it as
the earliest raven-controlled EverOS entry points; _EVEROS_CONFIG now
derives from a single _EVEROS_BASE constant.

No data migration: existing ~/.everos data is left untouched and a fresh
~/.everos/raven is created lazily on first write/use.

Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
2026-06-30 09:37:11 +08:00
Elliot Chen
aa6fe8a7ed chore: prepare open source launch 2026-06-30 08:15:34 +08:00