<img width="2318" height="1448" alt="image"
src="https://github.com/user-attachments/assets/bca71d69-4972-420c-b14e-c8cf74a17de9"
/>
## Summary
Everything a knowledge base could not read, and the pieces it cut them
into
saying what they are.
**Reading a PDF.** A PDF records glyphs at coordinates; whether they are
a
heading, a table cell or a running footer is inferred from how they
look. There
are now two readers. The geometric one infers structure from type size
and
position and needs nothing but the file. The other asks deepdoc's
models, ported
into `raven/knowledge/parser/deepdoc`: a layout model that says what
each region
is, a detector and recognizer that read a page carrying no text at all,
and a
table-structure model that rebuilds a grid from a picture of one.
Detecting boxes around text the file already spells
out exactly costs an inference pass per page to learn less, and
recognising it
costs accuracy.
The layout graph answers with a shape its own postprocessing
raises on, and its classes are DocLayout-YOLO's order rather than the
list
Both were established by running the model, and the label order is
pinned by a test.
**Tables go in as tables.** `<table>` is the only shape that keeps what
a grid
states -- which column a cell stood under, which cells are merged, which
row is
the header -- and it is the shape a model reads a table in, which is
what these
chunks are for. PDF, Word and PowerPoint all state their merges exactly,
and
they now survive as `colspan` and `rowspan`. The flat form this replaces
had to
repeat a header standing over two columns across both of them, because
there was
nowhere to record a span; it had no callers left afterwards and is gone.
Two
deviations from upstream, both about what the string is for: every
cell's text
is escaped, where upstream interpolates it raw and a cell holding a `<`
produces
markup that is no longer a table, and the span attributes are written
only where
there is a span, where upstream emits `<td >` for every ordinary cell.
**A picture of where each chunk came from.** A chunk is a few hundred
words with
a page number on it, which is enough to cite and not enough to answer
whether
the parser read the page correctly. So every PDF chunk that knows where
it sits
carries one image: the regions it covers, cropped out of the rendered
pages and
stacked. here they are already structured in the parser's element spans,
so
the markers would only put text nobody wrote into the thing being
embedded.
The panel draws a chunk's tables as tables, parsed and
rebuilt as elements rather than injected: a person can rewrite any piece
from
that same panel, so what comes back from the store is user input however
it got
there.
**The page that reads them.** Upstream retired the knowledge page while
this
branch was out; it is rebuilt against the new architecture, and the half
that
never existed is the one a reader needs -- what a document actually
became once
it was cut and embedded. Documents sit in one-level folders and arrive
by four
routes: a file, a folder, a URL, a written note. Adding and indexing are
two
calls on the contract and nothing drains that queue on its own, so a
page that
stopped after the first left every upload saying Queued for good.
Opening a file takes the whole page: the document on the left, the
pieces the
index holds of it on the right. The left half is the file drawn as its
own
pages rather than framed by the browser's PDF viewer, which reads where
to open
once and answers to nothing afterwards -- and which nothing can be drawn
on.
Pages are rendered once and kept, and the region a piece was cut from is
washed
over the page it came from, placed in fractions of the page so the
picture's
own scale never enters the arithmetic. Clicking a piece takes the file
there;
double-clicking opens it for rewriting.
The pieces can be searched, filtered by state, turned on and off,
deleted,
rewritten and added to, one at a time or in a batch. Every one of those
was
already on the contract with no reader for it. A base's settings are
back and
fuller: how many pieces a search returns, how documents are cut, and how
much
prose comes with a table or a figure. The embedding model is a picker
while the
base holds no pieces and a stated fact once it does -- moving it then is
the
collection made again at the new width and every document re-cut, which
is not
a thing a panel does by being saved.
**An agent can read them.** The retrieval engine was there and nothing
could
reach it: no agent had a tool that opened a library. A reader now points
a
conversation at bases from the composer, the way they pin a working
directory,
and the selection rides in the same place -- the session's metadata,
bound
around the turn.
The tool takes a query and never a base. The scope is the reader's
choice, so a
model cannot reach material nobody offered it, and it is withheld where
nothing
was picked rather than offered and refused. A hit says which document
and page
it came from, because a hit a reader cannot place is a hit they cannot
check --
and the passages are pressable in the transcript, walking into the base,
the
document, the folder it is filed under and then the piece. A citation a
reader
cannot follow is a claim they have to take on trust, and the whole point
of
searching their own material is that they can go and look.
**The word segmenter and the resources.** `raven.core.Tokenizer` is
Infinity's
`rag_tokenizer`, ported: a forward and backward maximum match over a
dictionary
trie, with stemming and lemmatisation on the latin path. XGBoost is not
carried
over. None of the data lives in git -- the dictionary, the NLTK corpora
and the
ONNX graphs run to about a hundred megabytes, so `make fetch-resources`
on an
install and an optional Docker layer behind `RAVEN_RESOURCES` both point
at one
script. A checkout that could not reach the network still works, and
every
feature that reads this data says so when asked.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Run on macOS against Python 3.12, with the deepdoc weights fetched (the
model
tests skip without them):
```
uv run pytest 27450 passed, 111 skipped, 3 failed
npm run test --prefix ui-web 3276 passed
npm run test --prefix ui-tui 2106 passed
npm run type-check --prefix ui-web clean (tsc)
npm run lint --prefix ui-web clean (4 pre-existing hook warnings)
make lint-python clean (ruff check + format)
make lint-types clean (ty)
make lint-imports 10 contracts kept, 0 broken
make check-source-language clean
make check-large-files clean
make build-ui built, both boot snapshots match
node ui-web/scripts/check-css.mjs clean
node ui-web/scripts/check-class-namespace.mjs clean
```
The three Python failures are not this branch's and were each confirmed
on the
base alone: one asks a host without LibreOffice to refuse a render and
this host
has LibreOffice installed, and two simulation cases fail on an
unmodified
checkout of the base commit.
The RPC contract was regenerated for both consumers and both match it
(`npm run gen --prefix ui-web`, `npm run gen:rpc --prefix ui-tui`, then
`gen:check`). The page's boot snapshots and region goldens were
refreshed where
the composer bar gained a button, and each records that button and
nothing else.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
A PDF's tables, and a Word or PowerPoint file's, are now indexed as HTML
rather
than as header-prefixed lines. Chunk text changes shape for those
documents, so
a base indexed before this and searched after it holds both forms until
its
documents are reindexed; retrieval keeps working either way, because
both forms
are text. The same applies to how much prose a table or a figure carries
with
it: the setting now reaches as far as it says, where before it stopped
at the
nearest paragraph, so a chunk cut before this keeps the narrower context
until
its document is rebuilt. Rolling back is reverting the branch and
reindexing.
The deepdoc models are optional. An install that never ran `make
fetch-resources` reads PDFs geometrically, exactly as before, and says
so when
the models are asked for by name. Nothing here reaches the network at
request
time.
Chunk crops and rendered pages are written at index time and on first
view, both
under the state directory rather than in the repository, and both are
deleted
with their document. A page is immutable for the life of its document
id, so
the route serving them caches; the document route does not. All three
routes
take ids and no path, and the two image routes answer under the
strictest
sandbox in the tree.
What an agent may search is the reader's choice and nothing else. The
tool takes
no base argument, so a model cannot name one, and it is absent from the
schema
entirely for a conversation that named none -- a lane nobody pointed at
a base
is unchanged by this branch. The bases a session carries are stored
beside its
working directory and travel no further.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
## Summary
- Align English and Chinese documentation for Docker/Compose, sandbox
execution, and the in-tree tracing API with verified runtime behavior.
- Separate proactivity usage guidance from developer design and
implementation details, with configuration, troubleshooting, cost, and
safety boundaries.
- Update navigation and remove redundant section separators. Keep page
URLs unchanged.
- Preserve published proactivity section and title anchors in both
languages. Use native aliases for same-page content and visible
migration links for moved sections, with regression tests.
- Clarify Sentinel delivery fallback and menu-selection requirements,
tracing-on/off argument handling, headless CLI menu output, and viewer
descriptor lookup. Add focused regression coverage without changing
runtime behavior.
- Clarify the initial frozen baseline and improve Evolver wording in the
bilingual site and repository mapping. Keep its structure, navigation,
implementation terms, and technical claims unchanged; broader Evolver
documentation work is excluded.
- No runtime changes.
## Type
- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Passed on the rebased revision:
```bash
uv run --frozen --no-sync pytest tests/test_docs_site.py tests/test_tracing_api.py tests/test_sandbox_unit.py tests/test_sentinel_runner.py tests/test_sentinel_planner.py tests/test_sentinel_fast_path.py tests/test_nudge_policy.py tests/test_proactive_spawn.py tests/test_core_sentinel_stack.py tests/test_core_cron_stack_ledger.py tests/test_cli_sentinel_commands.py tests/test_decision_router.py -q -n 0 -x
# 395 passed
uv run --frozen --no-sync --with 'mkdocs-material>=9.5' --with 'mkdocs-static-i18n>=1.0' --with 'mkdocs-git-revision-date-localized-plugin>=1.6.0' --with 'jieba>=0.42.1' mkdocs build -f docs-site/mkdocs.yml --strict --site-dir /tmp/raven-pr551-push.9ZfOEi/site
# Strict English and Chinese builds passed
UV_NO_SYNC=1 make lint-python PYTHON_LINT_TARGETS='tests/test_docs_site.py tests/test_tracing_api.py tests/test_decision_router.py tests/test_cli_sentinel_commands.py'
UV_NO_SYNC=1 make check-large-files check-source-language COMMIT_RANGE=origin/main..HEAD
PYTHONPATH=. uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
npm exec --yes --package=@commitlint/cli@21.1.0 -- commitlint --from origin/main --to HEAD --config commitlint.config.cjs
git diff --check origin/main...HEAD
# All passed
```
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
Documentation checks also covered bilingual examples and the proactivity
JSON configuration schema. A built-HTML audit verified all 110
historical proactivity section fragments and four title fragments,
including same-language migration destinations, and found zero broken
internal fragment links across 29 HTML pages (672 links). Replaying
pre-fix content made all four page compatibility cases fail as expected.
The new descriptor-directory test failed on all three documents before
their stale paths were corrected. Added coverage exercises tracing
on/off, bare-number versus explicit menu selection with incomplete
classifier configuration, and discovery-menu dispatch through the real
CLI stack's headless sink. Runtime type checks and the full repository
test suite were not run; no runtime code changes. No browser click test,
live LLM, real-VM, or container end-to-end run was performed.
## Risk
Documentation-only, with regression tests. Proactivity navigation
changes, but page URLs and historical heading fragments are preserved.
An old link to content moved between pages lands on a visible migration
entry and requires one click to reach the new section; no JavaScript
redirect is needed. Existing queued-nudge policy and external-agent
isolation limitations are described, not fixed. Roll back by reverting
the documentation and associated test changes; no data migration is
involved.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: zhao.wang <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
## Summary
<img width="2518" height="1798" alt="image"
src="https://github.com/user-attachments/assets/9bb2dbb6-fe29-42ff-8dba-2e035557b254"
/>
Turns the knowledge page into something a reader can actually run a
knowledge base from: put material into it four ways, look at what is in
it, test what it answers with, and configure it.
**Data sources.** The panel had one button that uploaded one file. It
now offers file, folder, note and URL behind one menu, plus a drop
target over the whole panel. A picked folder is walked by the browser; a
dropped one is walked here, because a drop hands over a directory entry
rather than files. Either way the files are filtered to what this build
can index, capped, and uploaded one at a time: each carries its bytes
base64 in a single websocket frame under a 25 MB ceiling, so N at once
is N of those in memory. One file failing no longer takes the rest of
the folder with it.
A note and a fetched page are stored as the markdown they were captured
as, in the same blob store as every upload, so nothing downstream learns
a second shape. Reading a page follows Cherry Studio's own capture: Jina
Reader, asked for JSON so the page title comes back as a field rather
than as a preamble to parse out of the text, and through
`validate_url_target` first, because the URL arrives from a browser and
without that check the gateway is a request forger.
**Recall Test.** Asks a base a question and shows what comes back: the
chunk, where in its document it sat, its score beside its rank, and what
the search cost. An explicit button rather than search-as-you-type,
because every keystroke used to spend a billed embedding call. Past
questions are kept in localStorage and nowhere else.
**Settings.** Each base carries its own Top K and chunking, with a
sentence on every field saying what moving it costs. The embedding model
is shown and not offered: the collection is sized to its vector width at
creation, so changing it is a rebuild of every vector rather than a
setting, which is what the stale-base check exists to detect. Letting a
reader change it is a separate piece of work.
**Two defects found while measuring the above.**
SiliconFlow refuses an entire request when one input is over the model's
token limit, naming neither which input nor what was wrong with it. A
document of eighteen chunks where seven were long indexed none of the
other eleven, and four documents in a real base were failing this way.
The cause is the chunker's token estimate -- utf-8 bytes over four, an
English-prose rule that reads a CJK character as three quarters of a
token when it is one, and a URL as a quarter of what it costs.
Binary-searching the longest accepted prefix of real documentation pages
against the endpoint put the true figure at 2.1 to 2.3 characters per
token. A vendor client now cuts each input to fit before sending; the
four documents embed whole.
Search reported one number for its cost. It now reports two, because
they answer different questions: embedding is a round trip to whatever
endpoint is configured and runs to hundreds of milliseconds, while the
store query is the index doing its job. A surface reporting one figure
as the search time wants the second, or it is describing the provider.
**Deliberately not wired yet.** File processing lists the processors it
will offer with every row disabled and saying why it is unavailable;
smart chunking, separator, chunk size and overlap are stored on the base
but not yet read by the chunker. Both are placeholders with their state
stated rather than controls that would be silently ignored.
## Type
- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Run on this branch rebased onto `main` at `4f5d7dac`:
- `uv run pytest -q` -- 22151 passed, 45 skipped
- `npx vitest run` in `ui-web` -- 1738 passed (103 files), of which 92
cover this page
- `npx vitest run` in `ui-tui` -- 2072 passed
- `npx tsc --noEmit` in `ui-web` and `ui-tui` -- clean
- `uv run ruff check raven/ tests/` and `ruff format --check` -- clean
- `node scripts/check-page.mjs`, `check-css.mjs`,
`check-class-namespace.mjs`, `count-shared-globals.mjs` -- all OK
- `make check-large-files` -- OK
- The SiliconFlow client was measured against the live endpoint, not
mocked: batch size is not the limit (64 short inputs pass), 500 CJK
characters pass and 512 do not, and all four previously failing
documents in a real base now embed whole.
- Every panel was checked in the browser in both light and dark themes.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
User-visible behaviour changes, all additive:
- A search that names no `top_k` now answers with the base's own setting
instead of a hardcoded 5. New bases default to 6.
- Documents carry an `origin` and bases carry six settings. Every field
is defaulted, so a registry written before them loads with the behaviour
it already had; an older gateway answering without them is read as those
same defaults. A field from a build *after* this one is ignored rather
than losing the row, which is the same protection in the other
direction.
- The SiliconFlow client truncates an over-long input rather than
failing the request. A cut chunk's vector speaks for its beginning; the
chunk is still stored and shown in full when retrieved. The proper fix
is chunking to the model's real limit, which this does not do.
- Four brand-logo SVGs are added under `ui-web/src/assets/providers/`
(16 KB total) and spared the dark-mode inversion that monochrome vendor
marks get.
- Previewing an office file retains a suffixed copy of its bytes in the
render cache, because the converter picks its input filter from the
extension and a blob is stored under a bare id. Deleting the document or
its base now removes that copy, and the cache sweep reaches it as a
backstop for one whose owner never got the chance.
Rollback is a `git revert` of the squash commit; nothing migrates on
disk.
A registry written by this branch stays readable by the previous build,
and that took work: its loader hands every stored key to a dataclass
constructor and drops the row when one is a keyword that constructor
does not take, so a new field written beside the record would make every
base and document vanish from a rolled-back binary, with the records
still on disk and nothing listing them. The settings and the document
origins are therefore written under two sibling keys of `bases` and
`documents`, which a loader that does not know them does not read; a
record that uses none of them is written exactly as before, down to its
keys. What a rollback loses is the settings, which that build has no use
for. What it keeps is every base and every document. A test pins the
stored shape against the field set the shipped loader accepts, rather
than against our own reader.
Security: the URL source takes an arbitrary address from a browser and
fetches it server-side, so it goes through the same
`validate_url_target` guard the web tools use, and there is a test
asserting a refused target never reaches the network.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
---------
Co-authored-by: admin <admin@SH-HuKai.local>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
## Summary
A session on DeepSeek's own API (`deepseek/deepseek-flash`) became unrecoverable on 2026-09-11: every request was refused with HTTP 400 `No tool call found for tool output` (or the reverse, `No tool output found for tool call`), whatever the user typed. Three defects lined up:
1. **The Curator's budget trimming broke tool pairing.** `HistoryTrimmer.trim` closed the plan's selection over tool-call adjacency, then dropped one id at a time without re-closing. Dropping the assistant that declared three parallel calls left its three results; dropping one result left its call unanswered. The fallback path never ran `structural_errors`, so the trace recorded `ok: True` for a request a strict backend refused. `trim` now drops a call and its results as one group (`tool_group`), re-closes and re-anchors after every drop, and sweeps a selection whose parent the session no longer holds; `CuratorAssembler.build` runs the structural check on every path and carries the verdict in `validation`.
2. **The context window fell back to 65,536 silently.** `deepseek/deepseek-flash` is in no catalogue, so `effective_context_window` returned the default -- one sixteenth of the model's 1,048,576 -- and only LiteLLM's DEBUG noise said so. With 49,152 reserved for output, 4,151 tokens were left for history, which is what drove the trimming above. The fallback now warns once per model (only when the catalogues were actually asked), `raven doctor`'s `Context win` line says whether the number is pinned, from a catalogue, or the default, and the config health check reports the unpinned-and-unknown case.
3. **The id was not in LiteLLM's table.** `import_litellm` now registers `deepseek/deepseek-flash` (and the bare `deepseek-flash`) into the live table: window 1,048,576, output ceiling 384,000, DeepSeek's peak-hour prices. This is the dict the window resolver, the output-ceiling resolver and the cost recorder all read, so it also stops `cost_usd` being recorded as 0. A row a newer LiteLLM already carries is left alone; OpenRouter-routed ids are deliberately not registered.
Evidence is in the Curator's per-turn traces: `window=65536 out=49152 tools=10594 sys=1639 avail_hist=4151`, fallback path, `dropped message 32` (the assistant) with `included_message_ids: [35]` (one of its results). The session file itself was intact (34 calls, 34 results); after pinning the window the same session resumed on the fast path.
Not in this MR: the Curator's own calls are capped at `max_tokens=2048`, and on a model that reasons before answering two of six curator calls in one turn were cut with the last tool call unparsed, sending four turns in a row to the deterministic fallback. Left as it is -- it affects every product's every turn (up to 12 calls) and there is no comparison data; if it is raised it should be a config knob in its own MR.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `pytest tests/test_history_trimmer.py tests/test_curator_context_engine.py tests/test_provider_rates.py tests/test_litellm_setup.py tests/test_cli_doctor_window_line.py`: new cases -- a group drop removes the parent with all its results and a result with its parent and siblings; the history re-anchors on a user message after a drop; a result whose parent the session lost is not shipped; the fallback path records `errors` and ships no orphan; the default-window fallback warns once, not for a pin, not on the cheap tier; the registered row is present, reaches both resolvers, and does not override an existing row; the doctor line's three branches.
- Reverse check: with the three source files restored to `origin/main`, the new tests fail (trimmer 4, curator 1, rates 3, litellm_setup 3, doctor 1 import error); with the fix they pass.
- Wider run: `tests/test_agent_loop*.py tests/test_context*.py tests/test_curator*.py tests/test_memory*.py tests/test_litellm*.py tests/test_provider*.py tests/test_acp_*.py` + the above: 2799 passed, 38 skipped, 0 failed.
- `ruff check` / `ruff format --check` clean on every changed file.
- `tests/test_cli_doctor_commands.py` could not run in this environment (its autouse fixture needs `raven_everos`, 74 pre-existing errors on `main` too); the doctor line is covered by a separate file that needs only the resolver.
- End to end: the failing session (`~/.raven-ar6-ds`, 54 messages, one assistant with three parallel calls) resumed after the window was pinned; every request since carried the full history on the fast path.
## Risk
- Trimming semantics: a drop now removes a whole tool group where it removed one message; a budget that previously kept a partial group now keeps either all of it or none. The set of ids that survive can differ from before, but never in a way a provider rejects. Every existing test in the touched suites passes unchanged.
- `validation` gains an `errors` key on every path (previously only in `validate_candidate`); `ok` is now false when pairing is broken. Nothing in the tree read `ok` as budget-only.
- Registered prices are peak-hour; off-peak is half, and a flat table cannot say so, so recorded cost is an upper bound. Users on OpenRouter are unaffected (their ids are not registered).
- The new warning fires once per model per process; `allow_fetch=False` callers do not trigger it.
- Rollback: revert the three commits; no data or config involved.
## Related Issues
N/A
## Summary
A session on DeepSeek's own API (`deepseek/deepseek-flash`) became unrecoverable on 2026-09-11: every request was refused with HTTP 400 `No tool call found for tool output` (or the reverse, `No tool output found for tool call`), whatever the user typed. Three defects lined up:
1. **The Curator's budget trimming broke tool pairing.** `HistoryTrimmer.trim` closed the plan's selection over tool-call adjacency, then dropped one id at a time without re-closing. Dropping the assistant that declared three parallel calls left its three results; dropping one result left its call unanswered. The fallback path never ran `structural_errors`, so the trace recorded `ok: True` for a request a strict backend refused. `trim` now drops a call and its results as one group (`tool_group`), re-closes and re-anchors after every drop, and sweeps a selection whose parent the session no longer holds; `CuratorAssembler.build` runs the structural check on every path and carries the verdict in `validation`.
2. **The context window fell back to 65,536 silently.** `deepseek/deepseek-flash` is in no catalogue, so `effective_context_window` returned the default -- one sixteenth of the model's 1,048,576 -- and only LiteLLM's DEBUG noise said so. With 49,152 reserved for output, 4,151 tokens were left for history, which is what drove the trimming above. The fallback now warns once per model (only when the catalogues were actually asked), `raven doctor`'s `Context win` line says whether the number is pinned, from a catalogue, or the default, and the config health check reports the unpinned-and-unknown case.
3. **The id was not in LiteLLM's table.** `import_litellm` now registers `deepseek/deepseek-flash` (and the bare `deepseek-flash`) into the live table: window 1,048,576, output ceiling 384,000, DeepSeek's peak-hour prices. This is the dict the window resolver, the output-ceiling resolver and the cost recorder all read, so it also stops `cost_usd` being recorded as 0. A row a newer LiteLLM already carries is left alone; OpenRouter-routed ids are deliberately not registered.
Evidence is in the Curator's per-turn traces: `window=65536 out=49152 tools=10594 sys=1639 avail_hist=4151`, fallback path, `dropped message 32` (the assistant) with `included_message_ids: [35]` (one of its results). The session file itself was intact (34 calls, 34 results); after pinning the window the same session resumed on the fast path.
Not in this MR: the Curator's own calls are capped at `max_tokens=2048`, and on a model that reasons before answering two of six curator calls in one turn were cut with the last tool call unparsed, sending four turns in a row to the deterministic fallback. Left as it is -- it affects every product's every turn (up to 12 calls) and there is no comparison data; if it is raised it should be a config knob in its own MR.
## Type
- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `pytest tests/test_history_trimmer.py tests/test_curator_context_engine.py tests/test_provider_rates.py tests/test_litellm_setup.py tests/test_cli_doctor_window_line.py`: new cases -- a group drop removes the parent with all its results and a result with its parent and siblings; the history re-anchors on a user message after a drop; a result whose parent the session lost is not shipped; the fallback path records `errors` and ships no orphan; the default-window fallback warns once, not for a pin, not on the cheap tier; the registered row is present, reaches both resolvers, and does not override an existing row; the doctor line's three branches.
- Reverse check: with the three source files restored to `origin/main`, the new tests fail (trimmer 4, curator 1, rates 3, litellm_setup 3, doctor 1 import error); with the fix they pass.
- Wider run: `tests/test_agent_loop*.py tests/test_context*.py tests/test_curator*.py tests/test_memory*.py tests/test_litellm*.py tests/test_provider*.py tests/test_acp_*.py` + the above: 2799 passed, 38 skipped, 0 failed.
- `ruff check` / `ruff format --check` clean on every changed file.
- `tests/test_cli_doctor_commands.py` could not run in this environment (its autouse fixture needs `raven_everos`, 74 pre-existing errors on `main` too); the doctor line is covered by a separate file that needs only the resolver.
- End to end: the failing session (`~/.raven-ar6-ds`, 54 messages, one assistant with three parallel calls) resumed after the window was pinned; every request since carried the full history on the fast path.
## Risk
- Trimming semantics: a drop now removes a whole tool group where it removed one message; a budget that previously kept a partial group now keeps either all of it or none. The set of ids that survive can differ from before, but never in a way a provider rejects. Every existing test in the touched suites passes unchanged.
- `validation` gains an `errors` key on every path (previously only in `validate_candidate`); `ok` is now false when pairing is broken. Nothing in the tree read `ok` as budget-only.
- Registered prices are peak-hour; off-peak is half, and a flat table cannot say so, so recorded cost is an upper bound. Users on OpenRouter are unaffected (their ids are not registered).
- The new warning fires once per model per process; `allow_fetch=False` callers do not trigger it.
- Rollback: revert the three commits; no data or config involved.
## Related Issues
N/A
## Summary
`docker compose up --build` rebuilt the whole image on every start. For a reader
who only wants to try raven that is several minutes of Python and Node build to
arrive at what the registry already holds, so the quickstarts now say
`docker compose up`. Changed in README.md, README.zh-CN.md and docker/README.md.
`RAVEN_IMAGE` points at `evermindshanda/raven:latest`. The compose fallback is
corrected to the same account: it read `everminbd/raven:dev`, a transposition
that named nothing and would only have surfaced for someone running without a
.env file.
Two things a reviewer should decide rather than take from me:
- The fallback in docker-compose.yml is `:dev` while .env pins `:latest`. That
is only reachable with no .env present, so it may be deliberate; it is worth
a look either way.
- The service still carries a `build:` section and no `pull_policy`. Dropping
`--build` stops forcing a rebuild, but with a build section present Compose
builds a missing image rather than pulling it. If the intent is that a reader
never builds, that likely wants `pull_policy: always` or the build section
moved behind a profile. I have not changed it, because which of those is right
depends on whether contributors are still expected to build from this file.
## Type
- [x] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
- `uv run pytest tests -q -k "docker or readme"` -- 34 passed, 8 skipped
- `make check-source-language`, `make check-large-files` -- clean
- `npx commitlint` and `scripts/check_commit_messages.py` over the branch range -- clean
Not verified: no Docker daemon was available, so the published image was not
pulled and `docker compose up` was not run end to end. The claim that the image
exists at that name is the author's, not something this branch proves.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
If `evermindshanda/raven:latest` is absent, private, or stale, `docker compose up`
now builds from source instead (the build section is still there), so the failure
mode is a slow start rather than a broken one. Rollback is by revert; nothing
here changes runtime behaviour, only which image Compose reaches for.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A
## Summary
Two things arrive together because the second is what made the first usable.
**Run raven from a container.** A Dockerfile builds the page, the Python
environment and the engine into one image; `cd docker && docker compose up`
serves it. nginx serves the built page and proxies the RPC socket to the engine
over the loopback they share -- the engine binds 127.0.0.1 only, so a proxy in a
second container would have no address to dial. Compose publishes on 127.0.0.1,
and the entrypoint prints a one-time sign-in link on boot.
**A first run is no longer a dead end.** The gateway used to refuse to start
until a provider was configured, while being the thing that serves the page
where a provider gets configured. It now starts, says what is missing, and picks
up a key saved in Settings on the next turn without a restart. The onboarding
overlay no longer takes the page either: New chat, the model chip and Send send
the reader to Settings > Models instead, and an empty model list is marked on
the control that fills it.
Also carried: the provider catalogue registry (capabilities, modalities and
context windows per model), the reworked Settings > Models pane, and four fixes
found in review of that work -- a sibling catalogue answering 200 with a
non-catalogue body no longer crashes the whole fetch; adding a tag no longer
erases the label and description already stored for that model; `authenticated`
no longer demands a bearer token from a local provider reached by address; and
registry rows are keyed by the spelling every lookup produces, which recovers
593 rows that no reader could reach.
## Type
- [ ] Fix
- [x] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other
## Verification
Commands run on the branch head, with their results:
- `uv run pytest tests -q -k "provider or registry or catalog or model or rpc or gateway or docker or i18n"` -- 4389 passed, 46 skipped
- `npx vitest run` in `ui-web` -- 1573 passed (101 files)
- `npx vitest run` in `ui-tui` -- 2037 passed (142 files)
- `uv run ruff check raven scripts tests` and `ruff format --check` -- clean
- `npx tsc --noEmit` in `ui-web` and `npm run type-check` in `ui-tui` -- clean
- `npm run lint:rpc` in `ui-tui` -- generated.ts in sync
- `make check-source-language`, `make check-large-files` -- clean
Each of the four review fixes was checked by reverting it and watching a named
test go red, then restoring it.
The container image itself has not been built by CI or by me -- no Docker daemon
was available on the machine this was written on. It has been built and run by
the author of the branch.
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed
## Risk
Two deliberate security trade-offs in the container deployment, both documented
in `docker/README.md`:
- nginx rewrites the `Origin` header, because the engine accepts only an Origin
naming its own loopback port and no browser reaching a published port can send
one. This drops a defence-in-depth layer. The session cookie is
SameSite=Strict, so an unauthenticated socket is still refused -- but SameSite
ignores ports, so another page on localhost counts as same-site.
- `RAVEN_AUTO_LOGIN=1` is the default: nginx attaches the session cookie to
every page load, so anyone who can reach the published port is signed in. It
is contained by Compose publishing on 127.0.0.1 only. Set it to 0 for any
deployment reachable from elsewhere. These are two settings that must agree,
and nothing warns when they do not.
Behaviour changes outside the container: the gateway now starts without a
configured provider rather than exiting 1, and the first-run onboarding overlay
no longer opens by default (`?onboard=1` still forces it).
Rollback is by revert; nothing here migrates state or changes an on-disk format.
- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes
## Related Issues
N/A