9 Commits

Author SHA1 Message Date
Dizhan Xue
24433cb7a6
fix(rpc): sign in a page reached through a port forward (#939)
## Summary

A page reached through a port forward could not sign in. The gateway
accepted a browser `Origin` only when it named the port the gateway
bound, and VS Code and `ssh -L` pick another local port whenever the
remote one is taken on the browser's machine, so every
`POST /auth/exchange` answered `403 bad origin` and every `/rpc`
upgrade was refused with it. The sign-in page reported an expired link
over a refusal it cannot see, and re-running `raven web` minted one
more link into the same refusal. Measured on 2026-10-13: 13 refusals
in six minutes, each from a page at `127.0.0.1:18793` against a
gateway on 18792, all within a minute of their nonce being minted. The
same check had already forced the container's nginx to rewrite `Origin`
for every request it proxies.

`_origin_ok` now also accepts an `Origin` that asks the request's own
origin (`Origin` equals `Host`) on a loopback name. That is what a
browser sends through a forward, since a forwarder relays `Host`
unchanged. It still refuses a page another local server shows, because
cookies and SameSite ignore the port and that origin names a port this
page does not ask; and it still refuses DNS rebinding, where a name
resolving to 127.0.0.1 asks its own origin and the two agree. The
bound-port allowlist stays for the container's nginx, which forwards
`Host` without a port.

Three smaller changes ride with it:

- The session cookie is named for the port the browser reached rather
  than the port bound, so two gateways that both bound 18792 no longer
  share one jar entry when one browser reaches them through different
  forwards. A `Host` without a port keeps the bound port, which is what
  the container's auto-login sets.
- `handle_auth_exchange` answers with the reason it refused, and the
  sign-in page shows that text rather than calling every failure an
  expired link.
- `_open` prints the sign-in link every time, and before handing it to
  a browser. `webbrowser.open` returns True whenever it ran something:
  VS Code's `browser.sh` exits 0 whatever happened, and a machine with
  no display hands the URL to a console browser that has no JavaScript
  and never redeems the nonce. The old fallback was unreachable, so the
  link was never printed at all.

The standalone serve test now uses a real aiohttp `Application` while
replacing only `AppRunner` and `TCPSite`. It selects temporary UI
locations for both an absent page and a built page, and checks that the
response revalidation hook is registered only for the built page. This
removes dependence on whether the checkout contains a UI build.

This PR also retains the reviewed Windows test guards and separate
`RAVEN_HOME` directories for real-upgrade fixtures.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Linux fixture follow-up (Python 3.12.13; the three-file suite and diff
coverage were rechecked after rebasing onto the current main):

- Before the fix, parameterizing `TestTheSignInLink` over an absent page
and a built page produced 4 passed / 1 failed. Only the built-page case
failed, with the same missing `on_response_prepare` as CI.
- `uv run --no-sync pytest -q -n 0
tests/test_cli_serve_commands.py::TestTheSignInLink`: 5 passed after the
fix.
- `uv run --no-sync pytest -q tests/test_rpc_transport.py
tests/test_cli_serve_commands.py tests/test_docker_runtime.py
--cov=raven.rpc.transports.ws --cov=raven.cli.serve_commands
--cov-report=json:<coverage-json>`: 234 passed. `<coverage-json>`
denotes the local coverage output file.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main`: 100.00% (32/32 executable
changed lines), passing the 90% threshold. This measures diff coverage,
not the full-suite ratchet.
- A temporary test plugin removed the response hook without changing the
checkout. The built-page case failed its hook assertion and the
absent-page control passed (1 failed / 1 passed).
- `uv run --no-sync ruff check tests/test_cli_serve_commands.py
raven/cli/serve_commands.py`, `uv run --no-sync ruff format --check
tests/test_cli_serve_commands.py raven/cli/serve_commands.py`, and `uv
run --no-sync ty check raven/cli/serve_commands.py
tests/test_cli_serve_commands.py`: passed.
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`:
passed after rebase (2268 Python files formatted; 10 import contracts
kept; no dependency issues; type checks passed). Both commit-message
validators and the repository source-language and large-file gates also
passed.
- The earlier description incorrectly called this an unrelated CI
failure. It was introduced by this PR's standalone-serve test, and the
CI build made that incomplete fixture fail. The fixture follow-up was
validated on Linux; the native Windows and full-suite measurements below
are earlier verification. CI must validate the pushed revision.

Native Windows follow-up (build 10.0.26300):

- Parallel real-upgrade cases now get separate `RAVEN_HOME` directories,
isolating both upgrade markers and downloaded wheels. A scratch barrier
forced both helpers to finish writing the same partial-wheel name before
rename: the original fixture produced 1 failed / 1 passed with
`Distribution not found` in the shared cache; the identical probe with
isolated homes produced 2 passed on Python 3.13.12. The ordinary run had
passed before the forced overlap, so this result identifies a race
rather than a consistently failing upgrade.
- Captured upgrade diagnostics tolerate localized Windows error bytes
and include stderr on a missing success message. Version and
companion-package assertions remain strict.
- Three reported unit-test constraints were reproduced. Symlink setup
now skips only Windows privilege error 1314; other errors propagate and
confinement still runs when a link can be created. Owner-only mode bits
and two SIGTERM-handler tests are explicitly POSIX-only. A native
subprocess probe confirmed that Windows Popen.terminate does not execute
the Python SIGTERM handler.
- `PYTHONUTF8=1`; `uv run --frozen --no-sync pytest
tests/test_rpc_transport.py tests/test_cli_serve_commands.py
tests/test_docker_runtime.py::test_container_bootstraps_the_browser_after_the_engine_is_ready
-n 0 -x --tb=short`: 195 passed, 30 skipped on Python 3.13.12.
Twenty-six skips were already POSIX-specific; four now identify the
reported mode/signal/symlink preconditions. The skipped checks are not
Windows equivalents.
- `PYTHONUTF8=1`, `UV_PYTHON=3.12`, and an isolated caller `RAVEN_HOME`;
`uv run --no-project --python 3.12 --with 'pytest>=8,<10' --with
pytest-asyncio --with pytest-xdist pytest --noconftest
tests/integration/test_cli_upgrade_real_uv.py -q --tb=short -n 4`: 3
passed, covering two real uv upgrade variants and the workflow contract.
- Whole-PR pre-commit, full Python-target Ruff check/format, all ten
import contracts, changed-production type checks, commitlint and
repository commit/asset/source-language gates passed.

Earlier author verification before this Windows follow-up:
- `uv run --frozen --python 3.12 --all-extras pytest -q
tests/test_rpc_transport.py tests/test_cli_serve_commands.py
tests/test_docker_runtime.py` -> `233 passed`.
- The same three files plus every `tests/test_rpc_*.py` and
`tests/test_cli_smoke.py` -> `3121 passed` on the first revision, before
the fixture follow-up.
- The earlier full suite, under `--cov=raven --cov-branch` -> `8 failed,
29791 passed, 158 skipped`, then `scripts/coverage_gate.py diff
--base-ref origin/main --threshold 90` -> `100.00% (32/32 executable
changed lines)`, passed. All eight failures are local baselines: five
proxy tests (`tests/test_config_update_providers.py`) that fail whenever
the shell carries this box's proxy;
`test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`
(node/npm resolution on this box);
`test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing`
(fails as root); and
`test_knowledge_parser_excel.py::test_a_legacy_workbook_is_converted_and_read`.
The last three were replayed in an unmodified baseline worktree and fail
there too (`3 failed`).
- The tests were written before the code and watched fail on it, each
for its own reason: the forwarded origin as `403: bad origin`, the
port-forward cookie case as `two sign-ins wrote one jar entry:
['raven_session_18792']`, the launcher cases on empty output. Against
`main`'s two source files the three test files now fail 18 tests.
- Eleven mutants, one construct each, all killed, with both files
restored and compared byte for byte: the forwarded rule off, Origin
equal to Host dropped, the loopback test dropped, the `localhost` branch
removed, either parse left unguarded, the cookie keyed to the bound
port, the two refusals given one text, the link not printed, the link
printed after the browser, and `raven serve --open` bypassing `_open`.
- `ruff check` and `ruff format --check` clean; `lint-imports` reports
`10 kept, 0 broken`.
- `scripts/check_source_language.py origin/main...HEAD` and
`scripts/check_large_files.py origin/main...HEAD` both exit 0.
- Real browser, end to end, against an isolated `raven serve` on its own
`RAVEN_HOME` reached through a TCP forwarder: the tab signed in and
landed on `/`, the cookie it kept was `raven_session_<forwarded port>`,
and an RPC call over a socket opened from that page was answered (server
log: `serve: ws client connected`, `system.hello`). A direct tab on the
bound port signed in too. With `fetch` stubbed, the page rendered each
of the four answers it can receive (refused address, used or expired
link, other status, no answer) with the matching text.

## Risk

- Security impact considered. The change widens one check: an `Origin`
that equals the request's `Host` on a loopback name. The cross-origin
local-page and DNS-rebinding cases remain refused, and both are covered
by tests. What it accepts that the old rule did not is a page whose
origin is exactly the address it asked for -- which only something
serving that address can show, and a file the agent produced arrives
with the CSP `sandbox` directive and so has an opaque origin.
- Backward compatibility: a tab signed in directly on the bound port
keeps its cookie name and stays signed in. A tab reaching the gateway
through a forward had no working session before, so there is nothing to
migrate. The container's auto-login cookie is unchanged, and a test
derives the entrypoint's name from `cookie_name` so the two cannot
drift.
- Rollback: revert the commit. Nothing is written outside the process
except the cookie already in the browser, and no config or format
changed.
- A `Host` that does not parse (`127.0.0.1:99999`, `127.0.0.1:abc`,
`[::1`) is answered 401 on `/rpc`, as on `main`, and an Origin equal to
such a Host is answered 403. Without the guard both would be a 500,
since the cookie's name is read before any credential is checked.
- Not fixed here: the page's own "this tab's sign-in expired" notice
(main page, i18n `gui.auth.stale`) still names the remedy correctly but
not the cause, so a tab that reaches the gateway by a non-loopback
address still reports a sign-in failure it cannot act on. Also
unmeasured: a real VS Code forward and a real container (both request
shapes were reproduced instead).

## Related Issues

N/A
2026-10-14 11:05:26 +08:00
Kevin Hu
6e7926a8e5
fix(knowledge): seven defects in the document reader, six of them docker-only (#938)
## Summary

Seven fixes to the knowledge reader, all found by running the Docker
image
against real documents. Six of the seven only reproduce there or only
reproduce in a browser, which is why they reached a release.

**The image could not draw the reader at all.** The document viewer
serves a
page of the file as a picture (`/knowledge/page`), the region a chunk
was cut
from (`/knowledge/crop`), and the file itself (`/knowledge/file`). All
three
are engine routes and none was in the nginx server block, so they fell
to
`location /` and were answered with index.html -- a 200 of text/html
where the
page put an `<img>`, which a browser draws as a broken image and reports
nowhere. Measured before: `200 text/html`. After, with a session:
`200 image/webp`, 126060 bytes, byte for byte what
`knowledge_pages.image_for`
writes. Source installs proxy nothing, so the reader has always worked
there.

**A document could be stranded at `indexing` for good.**
`index_document` is
awaited inside the RPC that asks for it, on the page's websocket, so a
reload
or a lost socket cancels it. `asyncio.CancelledError` is a
`BaseException`:
the five-type clause there and the `except Exception` in the RPC method
both
missed it, and the record was left reading `indexing` with nothing
indexing
it, no error beside it and nothing to retry it. It is set back to
`pending`
and re-raised now.

**Nothing drained that queue.** `_requeue_interrupted` puts back what a
restart interrupted and the above puts back what a cancel interrupted,
both
into a queue whose only other caller was a model switch on one base. The
row
read `Queued` and stayed there -- the same complaint
`_requeue_interrupted`'s own docstring makes about the state it rescues
documents from, one status further along. The gateway drains it at boot
now,
after a delay and one document at a time.

**Every .docx, .pptx and .xlsx failed to parse in the image.**
`mimetypes.guess_type` answers from the host, and the image carries none
of
the six files `mimetypes.knownfiles` names, so Python's built-in table
is all
it knows -- and that table predates OOXML. It knows `.doc` and `.ppt`
and
answers None for `.docx`, `.pptx`, `.xlsx` and `.rtf`. The manager falls
back
to `text/plain` for an unknown type, so a Word document went to the text
parser, which decoded the zip container as UTF-8 and died on the first
byte
that is not a character: "'utf-8' codec can't decode byte 0x91 in
position
15", against a file nothing is wrong with. The same table reversed is
what
`supported_extensions` builds the upload dialog's accept list from, so
those
formats were missing there too. The parsers state what they read now
instead
of asking the machine they run on.

**Indexing silenced every other control on the page.** `busy` is one
flag for
the whole store and both the upload and the rebuild held it across the
entire
parse-and-embed, tens of seconds on a long PDF. For all of that time
delete,
rename, star, pin, move, folder create and delete and every chunk edit
returned at their guard and did nothing -- no toast, nothing in the
console.
The confirm had already armed, so a reader answered "Delete, really" to
a
question that was then thrown away. Indexing has its own flag now; it
still
refuses to start a second one, which was the part that mattered.

That makes deleting a document while it indexes reachable, so the
manager
checks the record still exists before inserting. The delete already took
that
document's vectors, and inserting the ones that run computed would put
pieces
of it back with no row to own them, answering searches out of a deleted
file.

**A freshly uploaded file read `Queued` until the page was reloaded.**
The add
answers with the row as the server first wrote it, and the next line
starts
the call that indexes it. The row was left at what the add said for all
of it
while the server had already moved to `indexing` -- so reloading showed
the
truth and waiting did not. The rebuild path has never had this problem
because
it marks its own row, and the comment above that line is this bug
described in
advance: "a row that does not move until it is over reads as a press
that did
nothing". Both call one helper now.

**Picking a folder uploaded everything in it.** No browser applies
`accept` to
a directory, so a chosen folder arrived as every file under it --
.DS_Store,
lock files, images -- and each became an upload, a parse and a row
reading
Failed for a file nobody meant to add. The single-file picker set no
`accept`
at all. `knowledge.status` has answered with the readable extensions
since the
reader was written, and the comment above it asks for exactly this: "a
surface
that walks a folder has to filter by today's answer". Nothing on the
page had
ever called it. Both pickers carry that list now and what comes back is
filtered before anything is uploaded, because `accept` is advisory twice
over.
What is left out is reported rather than dropped quietly, and an install
whose
list could not be read filters nothing.

## Type

- [x] Fix

## Verification

- `uv run --frozen pytest tests/` -- 29694 passed, 121 skipped, 2
xfailed, 3
failed. The three are environmental and fail identically on this
branch's
base: macOS resolves soffice under /Applications so blanking PATH does
not
hide it, a case-insensitive filesystem, and process-descendant
enumeration.
- `npx vitest run` (ui-web) -- 238 files, 3724 passed.
- `npx tsc --noEmit` and `npx eslint src/features/knowledge/` -- clean.
- `uv run --frozen ruff check .` -- clean.
- `uv run --frozen --all-extras deptry raven` -- no dependency issues.
- `make check-source-language` and `make check-large-files` -- clean.
- `npm run --prefix ui-tui lint:i18n` -- clean after regenerating the
  catalogue for the one added message.
- In a running container: `/knowledge/page` answers 200 image/webp where
it
answered 200 text/html; a document deleted during a rebuild goes in
about
three seconds with the PDF still reading Indexing; opening a base sends
knowledge.status and both pickers are built carrying all 27 extensions.
- Every new test was checked against the unfixed code and fails there.

Not verified by hand: the upload picker is an `<input>` built detached
and
clicked, so it cannot be driven from a page script. The freshly-uploaded
status and the folder filtering rest on unit tests and on the element
the
picker builds, which was inspected live.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

User-visible behaviour changes, each of them a surface that did nothing
before:

- the knowledge reader's pictures load in Docker;
- a `.docx`, `.pptx` or `.xlsx` indexes in Docker instead of failing on
its
  zip header;
- an interrupted index is retried instead of sitting at `indexing`;
- the gateway indexes what is queued when it starts, which is embedding
work
at boot that did not happen before. It waits, runs one document at a
time,
  never raises, and does nothing when the queue is empty;
- structural controls work while a document indexes;
- a file picked from a folder that this install cannot read is left out
with a
  count rather than uploaded and failed.

Rollback is per commit: each of the seven stands alone and reverting any
one
restores the behaviour it changed. The nginx block, the mime
registration and
the cancel clause are additive and revert cleanly. The queue drain is
one call
at boot and removing that call disables it entirely.

Security: the mime registration adds only IANA mappings for formats the
parsers already accept, and routes no new input anywhere. The nginx
block
proxies three engine routes that enforce their own session check --
verified
by the 401 an unauthenticated request still gets.

Backward compatibility: no stored shape changes. The one record
transition
added, `indexing` to `pending` on cancel, is what a restart already did.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-14 10:43:57 +08:00
Kevin Hu
82c76a82a9
feat(knowledge): read every format, show what the index holds, and let an agent search it (#489)
<img width="2318" height="1448" alt="image"
src="https://github.com/user-attachments/assets/bca71d69-4972-420c-b14e-c8cf74a17de9"
/>

## Summary

Everything a knowledge base could not read, and the pieces it cut them
into
saying what they are.

**Reading a PDF.** A PDF records glyphs at coordinates; whether they are
a
heading, a table cell or a running footer is inferred from how they
look. There
are now two readers. The geometric one infers structure from type size
and
position and needs nothing but the file. The other asks deepdoc's
models, ported
into `raven/knowledge/parser/deepdoc`: a layout model that says what
each region
is, a detector and recognizer that read a page carrying no text at all,
and a
table-structure model that rebuilds a grid from a picture of one. 
 Detecting boxes around text the file already spells
out exactly costs an inference pass per page to learn less, and
recognising it
costs accuracy.

The layout graph answers with a shape its own postprocessing
raises on, and its classes are DocLayout-YOLO's order rather than the
list
Both were established by running the model, and the label order is
pinned by a test.

**Tables go in as tables.** `<table>` is the only shape that keeps what
a grid
states -- which column a cell stood under, which cells are merged, which
row is
the header -- and it is the shape a model reads a table in, which is
what these
chunks are for. PDF, Word and PowerPoint all state their merges exactly,
and
they now survive as `colspan` and `rowspan`. The flat form this replaces
had to
repeat a header standing over two columns across both of them, because
there was
nowhere to record a span; it had no callers left afterwards and is gone.
Two
deviations from upstream, both about what the string is for: every
cell's text
is escaped, where upstream interpolates it raw and a cell holding a `<`
produces
markup that is no longer a table, and the span attributes are written
only where
there is a span, where upstream emits `<td  >` for every ordinary cell.

**A picture of where each chunk came from.** A chunk is a few hundred
words with
a page number on it, which is enough to cite and not enough to answer
whether
the parser read the page correctly. So every PDF chunk that knows where
it sits
carries one image: the regions it covers, cropped out of the rendered
pages and
stacked. here they are already structured in the parser's element spans,
so
the markers would only put text nobody wrote into the thing being
embedded.
The panel draws a chunk's tables as tables, parsed and
rebuilt as elements rather than injected: a person can rewrite any piece
from
that same panel, so what comes back from the store is user input however
it got
there.

**The page that reads them.** Upstream retired the knowledge page while
this
branch was out; it is rebuilt against the new architecture, and the half
that
never existed is the one a reader needs -- what a document actually
became once
it was cut and embedded. Documents sit in one-level folders and arrive
by four
routes: a file, a folder, a URL, a written note. Adding and indexing are
two
calls on the contract and nothing drains that queue on its own, so a
page that
stopped after the first left every upload saying Queued for good.

Opening a file takes the whole page: the document on the left, the
pieces the
index holds of it on the right. The left half is the file drawn as its
own
pages rather than framed by the browser's PDF viewer, which reads where
to open
once and answers to nothing afterwards -- and which nothing can be drawn
on.
Pages are rendered once and kept, and the region a piece was cut from is
washed
over the page it came from, placed in fractions of the page so the
picture's
own scale never enters the arithmetic. Clicking a piece takes the file
there;
double-clicking opens it for rewriting.

The pieces can be searched, filtered by state, turned on and off,
deleted,
rewritten and added to, one at a time or in a batch. Every one of those
was
already on the contract with no reader for it. A base's settings are
back and
fuller: how many pieces a search returns, how documents are cut, and how
much
prose comes with a table or a figure. The embedding model is a picker
while the
base holds no pieces and a stated fact once it does -- moving it then is
the
collection made again at the new width and every document re-cut, which
is not
a thing a panel does by being saved.

**An agent can read them.** The retrieval engine was there and nothing
could
reach it: no agent had a tool that opened a library. A reader now points
a
conversation at bases from the composer, the way they pin a working
directory,
and the selection rides in the same place -- the session's metadata,
bound
around the turn.

The tool takes a query and never a base. The scope is the reader's
choice, so a
model cannot reach material nobody offered it, and it is withheld where
nothing
was picked rather than offered and refused. A hit says which document
and page
it came from, because a hit a reader cannot place is a hit they cannot
check --
and the passages are pressable in the transcript, walking into the base,
the
document, the folder it is filed under and then the piece. A citation a
reader
cannot follow is a claim they have to take on trust, and the whole point
of
searching their own material is that they can go and look.

**The word segmenter and the resources.** `raven.core.Tokenizer` is
Infinity's
`rag_tokenizer`, ported: a forward and backward maximum match over a
dictionary
trie, with stemming and lemmatisation on the latin path. XGBoost is not
carried
over. None of the data lives in git -- the dictionary, the NLTK corpora
and the
ONNX graphs run to about a hundred megabytes, so `make fetch-resources`
on an
install and an optional Docker layer behind `RAVEN_RESOURCES` both point
at one
script. A checkout that could not reach the network still works, and
every
feature that reads this data says so when asked.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on macOS against Python 3.12, with the deepdoc weights fetched (the
model
tests skip without them):

```
uv run pytest                        27450 passed, 111 skipped, 3 failed
npm run test --prefix ui-web          3276 passed
npm run test --prefix ui-tui          2106 passed
npm run type-check --prefix ui-web    clean (tsc)
npm run lint --prefix ui-web          clean (4 pre-existing hook warnings)
make lint-python                      clean (ruff check + format)
make lint-types                       clean (ty)
make lint-imports                     10 contracts kept, 0 broken
make check-source-language            clean
make check-large-files                clean
make build-ui                         built, both boot snapshots match
node ui-web/scripts/check-css.mjs     clean
node ui-web/scripts/check-class-namespace.mjs  clean
```

The three Python failures are not this branch's and were each confirmed
on the
base alone: one asks a host without LibreOffice to refuse a render and
this host
has LibreOffice installed, and two simulation cases fail on an
unmodified
checkout of the base commit.

The RPC contract was regenerated for both consumers and both match it
(`npm run gen --prefix ui-web`, `npm run gen:rpc --prefix ui-tui`, then
`gen:check`). The page's boot snapshots and region goldens were
refreshed where
the composer bar gained a button, and each records that button and
nothing else.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

A PDF's tables, and a Word or PowerPoint file's, are now indexed as HTML
rather
than as header-prefixed lines. Chunk text changes shape for those
documents, so
a base indexed before this and searched after it holds both forms until
its
documents are reindexed; retrieval keeps working either way, because
both forms
are text. The same applies to how much prose a table or a figure carries
with
it: the setting now reaches as far as it says, where before it stopped
at the
nearest paragraph, so a chunk cut before this keeps the narrower context
until
its document is rebuilt. Rolling back is reverting the branch and
reindexing.

The deepdoc models are optional. An install that never ran `make
fetch-resources` reads PDFs geometrically, exactly as before, and says
so when
the models are asked for by name. Nothing here reaches the network at
request
time.

Chunk crops and rendered pages are written at index time and on first
view, both
under the state directory rather than in the repository, and both are
deleted
with their document. A page is immutable for the life of its document
id, so
the route serving them caches; the document route does not. All three
routes
take ids and no path, and the two image routes answer under the
strictest
sandbox in the tree.

What an agent may search is the reader's choice and nothing else. The
tool takes
no base argument, so a model cannot name one, and it is absent from the
schema
entirely for a conversation that named none -- a lane nobody pointed at
a base
is unchanged by this branch. The bases a session carries are stored
beside its
working directory and travel no further.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-12 19:05:16 +08:00
userName20260323
e708f34a91
docs: align bilingual documentation with runtime behavior (#551)
## Summary

- Align English and Chinese documentation for Docker/Compose, sandbox
execution, and the in-tree tracing API with verified runtime behavior.
- Separate proactivity usage guidance from developer design and
implementation details, with configuration, troubleshooting, cost, and
safety boundaries.
- Update navigation and remove redundant section separators. Keep page
URLs unchanged.
- Preserve published proactivity section and title anchors in both
languages. Use native aliases for same-page content and visible
migration links for moved sections, with regression tests.
- Clarify Sentinel delivery fallback and menu-selection requirements,
tracing-on/off argument handling, headless CLI menu output, and viewer
descriptor lookup. Add focused regression coverage without changing
runtime behavior.
- Clarify the initial frozen baseline and improve Evolver wording in the
bilingual site and repository mapping. Keep its structure, navigation,
implementation terms, and technical claims unchanged; broader Evolver
documentation work is excluded.
- No runtime changes.

## Type

- [ ] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Passed on the rebased revision:

```bash
uv run --frozen --no-sync pytest tests/test_docs_site.py tests/test_tracing_api.py tests/test_sandbox_unit.py tests/test_sentinel_runner.py tests/test_sentinel_planner.py tests/test_sentinel_fast_path.py tests/test_nudge_policy.py tests/test_proactive_spawn.py tests/test_core_sentinel_stack.py tests/test_core_cron_stack_ledger.py tests/test_cli_sentinel_commands.py tests/test_decision_router.py -q -n 0 -x
# 395 passed

uv run --frozen --no-sync --with 'mkdocs-material>=9.5' --with 'mkdocs-static-i18n>=1.0' --with 'mkdocs-git-revision-date-localized-plugin>=1.6.0' --with 'jieba>=0.42.1' mkdocs build -f docs-site/mkdocs.yml --strict --site-dir /tmp/raven-pr551-push.9ZfOEi/site
# Strict English and Chinese builds passed

UV_NO_SYNC=1 make lint-python PYTHON_LINT_TARGETS='tests/test_docs_site.py tests/test_tracing_api.py tests/test_decision_router.py tests/test_cli_sentinel_commands.py'
UV_NO_SYNC=1 make check-large-files check-source-language COMMIT_RANGE=origin/main..HEAD
PYTHONPATH=. uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD
npm exec --yes --package=@commitlint/cli@21.1.0 -- commitlint --from origin/main --to HEAD --config commitlint.config.cjs
git diff --check origin/main...HEAD
# All passed
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Documentation checks also covered bilingual examples and the proactivity
JSON configuration schema. A built-HTML audit verified all 110
historical proactivity section fragments and four title fragments,
including same-language migration destinations, and found zero broken
internal fragment links across 29 HTML pages (672 links). Replaying
pre-fix content made all four page compatibility cases fail as expected.
The new descriptor-directory test failed on all three documents before
their stale paths were corrected. Added coverage exercises tracing
on/off, bare-number versus explicit menu selection with incomplete
classifier configuration, and discovery-menu dispatch through the real
CLI stack's headless sink. Runtime type checks and the full repository
test suite were not run; no runtime code changes. No browser click test,
live LLM, real-VM, or container end-to-end run was performed.

## Risk

Documentation-only, with regression tests. Proactivity navigation
changes, but page URLs and historical heading fragments are preserved.
An old link to content moved between pages lands on a visible migration
entry and requires one click to reach the new section; no JavaScript
redirect is needed. Existing queued-nudge policy and external-agent
isolation limitations are described, not fixed. Roll back by reverting
the documentation and associated test changes; no data migration is
involved.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: zhao.wang <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-20 19:20:27 +08:00
Kevin Hu
7beffb9d0f
feat(knowledge): four data sources for a knowledge base, recall test, and per-base settings (#426)
## Summary

<img width="2518" height="1798" alt="image"
src="https://github.com/user-attachments/assets/9bb2dbb6-fe29-42ff-8dba-2e035557b254"
/>


Turns the knowledge page into something a reader can actually run a
knowledge base from: put material into it four ways, look at what is in
it, test what it answers with, and configure it.

**Data sources.** The panel had one button that uploaded one file. It
now offers file, folder, note and URL behind one menu, plus a drop
target over the whole panel. A picked folder is walked by the browser; a
dropped one is walked here, because a drop hands over a directory entry
rather than files. Either way the files are filtered to what this build
can index, capped, and uploaded one at a time: each carries its bytes
base64 in a single websocket frame under a 25 MB ceiling, so N at once
is N of those in memory. One file failing no longer takes the rest of
the folder with it.

A note and a fetched page are stored as the markdown they were captured
as, in the same blob store as every upload, so nothing downstream learns
a second shape. Reading a page follows Cherry Studio's own capture: Jina
Reader, asked for JSON so the page title comes back as a field rather
than as a preamble to parse out of the text, and through
`validate_url_target` first, because the URL arrives from a browser and
without that check the gateway is a request forger.

**Recall Test.** Asks a base a question and shows what comes back: the
chunk, where in its document it sat, its score beside its rank, and what
the search cost. An explicit button rather than search-as-you-type,
because every keystroke used to spend a billed embedding call. Past
questions are kept in localStorage and nowhere else.

**Settings.** Each base carries its own Top K and chunking, with a
sentence on every field saying what moving it costs. The embedding model
is shown and not offered: the collection is sized to its vector width at
creation, so changing it is a rebuild of every vector rather than a
setting, which is what the stale-base check exists to detect. Letting a
reader change it is a separate piece of work.

**Two defects found while measuring the above.**

SiliconFlow refuses an entire request when one input is over the model's
token limit, naming neither which input nor what was wrong with it. A
document of eighteen chunks where seven were long indexed none of the
other eleven, and four documents in a real base were failing this way.
The cause is the chunker's token estimate -- utf-8 bytes over four, an
English-prose rule that reads a CJK character as three quarters of a
token when it is one, and a URL as a quarter of what it costs.
Binary-searching the longest accepted prefix of real documentation pages
against the endpoint put the true figure at 2.1 to 2.3 characters per
token. A vendor client now cuts each input to fit before sending; the
four documents embed whole.

Search reported one number for its cost. It now reports two, because
they answer different questions: embedding is a round trip to whatever
endpoint is configured and runs to hundreds of milliseconds, while the
store query is the index doing its job. A surface reporting one figure
as the search time wants the second, or it is describing the provider.

**Deliberately not wired yet.** File processing lists the processors it
will offer with every row disabled and saying why it is unavailable;
smart chunking, separator, chunk size and overlap are stored on the base
but not yet read by the chunker. Both are placeholders with their state
stated rather than controls that would be silently ignored.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Run on this branch rebased onto `main` at `4f5d7dac`:

- `uv run pytest -q` -- 22151 passed, 45 skipped
- `npx vitest run` in `ui-web` -- 1738 passed (103 files), of which 92
cover this page
- `npx vitest run` in `ui-tui` -- 2072 passed
- `npx tsc --noEmit` in `ui-web` and `ui-tui` -- clean
- `uv run ruff check raven/ tests/` and `ruff format --check` -- clean
- `node scripts/check-page.mjs`, `check-css.mjs`,
`check-class-namespace.mjs`, `count-shared-globals.mjs` -- all OK
- `make check-large-files` -- OK
- The SiliconFlow client was measured against the live endpoint, not
mocked: batch size is not the limit (64 short inputs pass), 500 CJK
characters pass and 512 do not, and all four previously failing
documents in a real base now embed whole.
- Every panel was checked in the browser in both light and dark themes.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible behaviour changes, all additive:

- A search that names no `top_k` now answers with the base's own setting
instead of a hardcoded 5. New bases default to 6.
- Documents carry an `origin` and bases carry six settings. Every field
is defaulted, so a registry written before them loads with the behaviour
it already had; an older gateway answering without them is read as those
same defaults. A field from a build *after* this one is ignored rather
than losing the row, which is the same protection in the other
direction.
- The SiliconFlow client truncates an over-long input rather than
failing the request. A cut chunk's vector speaks for its beginning; the
chunk is still stored and shown in full when retrieved. The proper fix
is chunking to the model's real limit, which this does not do.
- Four brand-logo SVGs are added under `ui-web/src/assets/providers/`
(16 KB total) and spared the dark-mode inversion that monochrome vendor
marks get.
- Previewing an office file retains a suffixed copy of its bytes in the
render cache, because the converter picks its input filter from the
extension and a blob is stored under a bare id. Deleting the document or
its base now removes that copy, and the cache sweep reaches it as a
backstop for one whose owner never got the chance.

Rollback is a `git revert` of the squash commit; nothing migrates on
disk.

A registry written by this branch stays readable by the previous build,
and that took work: its loader hands every stored key to a dataclass
constructor and drops the row when one is a keyword that constructor
does not take, so a new field written beside the record would make every
base and document vanish from a rolled-back binary, with the records
still on disk and nothing listing them. The settings and the document
origins are therefore written under two sibling keys of `bases` and
`documents`, which a loader that does not know them does not read; a
record that uses none of them is written exactly as before, down to its
keys. What a rollback loses is the settings, which that build has no use
for. What it keeps is every base and every document. A test pins the
stored shape against the field set the shipped loader accepts, rather
than against our own reader.

Security: the URL source takes an arbitrary address from a browser and
fetches it server-side, so it goes through the same
`validate_url_target` guard the web tools use, and there is a test
asserting a refused target never reaches the network.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: admin <admin@SH-HuKai.local>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-09-16 03:02:44 -07:00
0xKT
93cf7ff74e fix(context_engine): keep a tool call with its results when trimming, and catalogue deepseek-flash
## Summary

A session on DeepSeek's own API (`deepseek/deepseek-flash`) became unrecoverable on 2026-09-11: every request was refused with HTTP 400 `No tool call found for tool output` (or the reverse, `No tool output found for tool call`), whatever the user typed. Three defects lined up:

1. **The Curator's budget trimming broke tool pairing.** `HistoryTrimmer.trim` closed the plan's selection over tool-call adjacency, then dropped one id at a time without re-closing. Dropping the assistant that declared three parallel calls left its three results; dropping one result left its call unanswered. The fallback path never ran `structural_errors`, so the trace recorded `ok: True` for a request a strict backend refused. `trim` now drops a call and its results as one group (`tool_group`), re-closes and re-anchors after every drop, and sweeps a selection whose parent the session no longer holds; `CuratorAssembler.build` runs the structural check on every path and carries the verdict in `validation`.
2. **The context window fell back to 65,536 silently.** `deepseek/deepseek-flash` is in no catalogue, so `effective_context_window` returned the default -- one sixteenth of the model's 1,048,576 -- and only LiteLLM's DEBUG noise said so. With 49,152 reserved for output, 4,151 tokens were left for history, which is what drove the trimming above. The fallback now warns once per model (only when the catalogues were actually asked), `raven doctor`'s `Context win` line says whether the number is pinned, from a catalogue, or the default, and the config health check reports the unpinned-and-unknown case.
3. **The id was not in LiteLLM's table.** `import_litellm` now registers `deepseek/deepseek-flash` (and the bare `deepseek-flash`) into the live table: window 1,048,576, output ceiling 384,000, DeepSeek's peak-hour prices. This is the dict the window resolver, the output-ceiling resolver and the cost recorder all read, so it also stops `cost_usd` being recorded as 0. A row a newer LiteLLM already carries is left alone; OpenRouter-routed ids are deliberately not registered.

Evidence is in the Curator's per-turn traces: `window=65536 out=49152 tools=10594 sys=1639 avail_hist=4151`, fallback path, `dropped message 32` (the assistant) with `included_message_ids: [35]` (one of its results). The session file itself was intact (34 calls, 34 results); after pinning the window the same session resumed on the fast path.

Not in this MR: the Curator's own calls are capped at `max_tokens=2048`, and on a model that reasons before answering two of six curator calls in one turn were cut with the last tool call unparsed, sending four turns in a row to the deterministic fallback. Left as it is -- it affects every product's every turn (up to 12 calls) and there is no comparison data; if it is raised it should be a config knob in its own MR.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest tests/test_history_trimmer.py tests/test_curator_context_engine.py tests/test_provider_rates.py tests/test_litellm_setup.py tests/test_cli_doctor_window_line.py`: new cases -- a group drop removes the parent with all its results and a result with its parent and siblings; the history re-anchors on a user message after a drop; a result whose parent the session lost is not shipped; the fallback path records `errors` and ships no orphan; the default-window fallback warns once, not for a pin, not on the cheap tier; the registered row is present, reaches both resolvers, and does not override an existing row; the doctor line's three branches.
- Reverse check: with the three source files restored to `origin/main`, the new tests fail (trimmer 4, curator 1, rates 3, litellm_setup 3, doctor 1 import error); with the fix they pass.
- Wider run: `tests/test_agent_loop*.py tests/test_context*.py tests/test_curator*.py tests/test_memory*.py tests/test_litellm*.py tests/test_provider*.py tests/test_acp_*.py` + the above: 2799 passed, 38 skipped, 0 failed.
- `ruff check` / `ruff format --check` clean on every changed file.
- `tests/test_cli_doctor_commands.py` could not run in this environment (its autouse fixture needs `raven_everos`, 74 pre-existing errors on `main` too); the doctor line is covered by a separate file that needs only the resolver.
- End to end: the failing session (`~/.raven-ar6-ds`, 54 messages, one assistant with three parallel calls) resumed after the window was pinned; every request since carried the full history on the fast path.

## Risk

- Trimming semantics: a drop now removes a whole tool group where it removed one message; a budget that previously kept a partial group now keeps either all of it or none. The set of ids that survive can differ from before, but never in a way a provider rejects. Every existing test in the touched suites passes unchanged.
- `validation` gains an `errors` key on every path (previously only in `validate_candidate`); `ok` is now false when pairing is broken. Nothing in the tree read `ok` as budget-only.
- Registered prices are peak-hour; off-peak is half, and a flat table cannot say so, so recorded cost is an upper bound. Users on OpenRouter are unaffected (their ids are not registered).
- The new warning fires once per model per process; `allow_fetch=False` callers do not trigger it.
- Rollback: revert the three commits; no data or config involved.

## Related Issues

N/A
2026-09-11 08:47:49 +00:00
silverLXT
70c29d1b93 fix(context_engine): keep a tool call with its results when trimming, and catalogue deepseek-flash
## Summary

A session on DeepSeek's own API (`deepseek/deepseek-flash`) became unrecoverable on 2026-09-11: every request was refused with HTTP 400 `No tool call found for tool output` (or the reverse, `No tool output found for tool call`), whatever the user typed. Three defects lined up:

1. **The Curator's budget trimming broke tool pairing.** `HistoryTrimmer.trim` closed the plan's selection over tool-call adjacency, then dropped one id at a time without re-closing. Dropping the assistant that declared three parallel calls left its three results; dropping one result left its call unanswered. The fallback path never ran `structural_errors`, so the trace recorded `ok: True` for a request a strict backend refused. `trim` now drops a call and its results as one group (`tool_group`), re-closes and re-anchors after every drop, and sweeps a selection whose parent the session no longer holds; `CuratorAssembler.build` runs the structural check on every path and carries the verdict in `validation`.
2. **The context window fell back to 65,536 silently.** `deepseek/deepseek-flash` is in no catalogue, so `effective_context_window` returned the default -- one sixteenth of the model's 1,048,576 -- and only LiteLLM's DEBUG noise said so. With 49,152 reserved for output, 4,151 tokens were left for history, which is what drove the trimming above. The fallback now warns once per model (only when the catalogues were actually asked), `raven doctor`'s `Context win` line says whether the number is pinned, from a catalogue, or the default, and the config health check reports the unpinned-and-unknown case.
3. **The id was not in LiteLLM's table.** `import_litellm` now registers `deepseek/deepseek-flash` (and the bare `deepseek-flash`) into the live table: window 1,048,576, output ceiling 384,000, DeepSeek's peak-hour prices. This is the dict the window resolver, the output-ceiling resolver and the cost recorder all read, so it also stops `cost_usd` being recorded as 0. A row a newer LiteLLM already carries is left alone; OpenRouter-routed ids are deliberately not registered.

Evidence is in the Curator's per-turn traces: `window=65536 out=49152 tools=10594 sys=1639 avail_hist=4151`, fallback path, `dropped message 32` (the assistant) with `included_message_ids: [35]` (one of its results). The session file itself was intact (34 calls, 34 results); after pinning the window the same session resumed on the fast path.

Not in this MR: the Curator's own calls are capped at `max_tokens=2048`, and on a model that reasons before answering two of six curator calls in one turn were cut with the last tool call unparsed, sending four turns in a row to the deterministic fallback. Left as it is -- it affects every product's every turn (up to 12 calls) and there is no comparison data; if it is raised it should be a config knob in its own MR.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `pytest tests/test_history_trimmer.py tests/test_curator_context_engine.py tests/test_provider_rates.py tests/test_litellm_setup.py tests/test_cli_doctor_window_line.py`: new cases -- a group drop removes the parent with all its results and a result with its parent and siblings; the history re-anchors on a user message after a drop; a result whose parent the session lost is not shipped; the fallback path records `errors` and ships no orphan; the default-window fallback warns once, not for a pin, not on the cheap tier; the registered row is present, reaches both resolvers, and does not override an existing row; the doctor line's three branches.
- Reverse check: with the three source files restored to `origin/main`, the new tests fail (trimmer 4, curator 1, rates 3, litellm_setup 3, doctor 1 import error); with the fix they pass.
- Wider run: `tests/test_agent_loop*.py tests/test_context*.py tests/test_curator*.py tests/test_memory*.py tests/test_litellm*.py tests/test_provider*.py tests/test_acp_*.py` + the above: 2799 passed, 38 skipped, 0 failed.
- `ruff check` / `ruff format --check` clean on every changed file.
- `tests/test_cli_doctor_commands.py` could not run in this environment (its autouse fixture needs `raven_everos`, 74 pre-existing errors on `main` too); the doctor line is covered by a separate file that needs only the resolver.
- End to end: the failing session (`~/.raven-ar6-ds`, 54 messages, one assistant with three parallel calls) resumed after the window was pinned; every request since carried the full history on the fast path.

## Risk

- Trimming semantics: a drop now removes a whole tool group where it removed one message; a budget that previously kept a partial group now keeps either all of it or none. The set of ids that survive can differ from before, but never in a way a provider rejects. Every existing test in the touched suites passes unchanged.
- `validation` gains an `errors` key on every path (previously only in `validate_candidate`); `ok` is now false when pairing is broken. Nothing in the tree read `ok` as budget-only.
- Registered prices are peak-hour; off-peak is half, and a flat table cannot say so, so recorded cost is an upper bound. Users on OpenRouter are unaffected (their ids are not registered).
- The new warning fires once per model per process; `allow_fetch=False` callers do not trigger it.
- Rollback: revert the three commits; no data or config involved.

## Related Issues

N/A
2026-09-11 08:47:49 +00:00
KevinHuSh
f60bab3183 fix(docker): run the container from the published image instead of building it
## Summary

`docker compose up --build` rebuilt the whole image on every start. For a reader
who only wants to try raven that is several minutes of Python and Node build to
arrive at what the registry already holds, so the quickstarts now say
`docker compose up`. Changed in README.md, README.zh-CN.md and docker/README.md.

`RAVEN_IMAGE` points at `evermindshanda/raven:latest`. The compose fallback is
corrected to the same account: it read `everminbd/raven:dev`, a transposition
that named nothing and would only have surfaced for someone running without a
.env file.

Two things a reviewer should decide rather than take from me:

- The fallback in docker-compose.yml is `:dev` while .env pins `:latest`. That
  is only reachable with no .env present, so it may be deliberate; it is worth
  a look either way.
- The service still carries a `build:` section and no `pull_policy`. Dropping
  `--build` stops forcing a rebuild, but with a build section present Compose
  builds a missing image rather than pulling it. If the intent is that a reader
  never builds, that likely wants `pull_policy: always` or the build section
  moved behind a profile. I have not changed it, because which of those is right
  depends on whether contributors are still expected to build from this file.

## Type

- [x] Fix
- [ ] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- `uv run pytest tests -q -k "docker or readme"` -- 34 passed, 8 skipped
- `make check-source-language`, `make check-large-files` -- clean
- `npx commitlint` and `scripts/check_commit_messages.py` over the branch range -- clean

Not verified: no Docker daemon was available, so the published image was not
pulled and `docker compose up` was not run end to end. The claim that the image
exists at that name is the author's, not something this branch proves.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

If `evermindshanda/raven:latest` is absent, private, or stale, `docker compose up`
now builds from source instead (the build section is still there), so the failure
mode is a slow start rather than a broken one. Rollback is by revert; nothing
here changes runtime behaviour, only which image Compose reaches for.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 07:44:46 +00:00
KevinHuSh
4a4a3fe15b feat(docker): run raven in a docker container and make a first run configurable from the page
## Summary

Two things arrive together because the second is what made the first usable.

**Run raven from a container.** A Dockerfile builds the page, the Python
environment and the engine into one image; `cd docker && docker compose up`
serves it. nginx serves the built page and proxies the RPC socket to the engine
over the loopback they share -- the engine binds 127.0.0.1 only, so a proxy in a
second container would have no address to dial. Compose publishes on 127.0.0.1,
and the entrypoint prints a one-time sign-in link on boot.

**A first run is no longer a dead end.** The gateway used to refuse to start
until a provider was configured, while being the thing that serves the page
where a provider gets configured. It now starts, says what is missing, and picks
up a key saved in Settings on the next turn without a restart. The onboarding
overlay no longer takes the page either: New chat, the model chip and Send send
the reader to Settings > Models instead, and an empty model list is marked on
the control that fills it.

Also carried: the provider catalogue registry (capabilities, modalities and
context windows per model), the reworked Settings > Models pane, and four fixes
found in review of that work -- a sibling catalogue answering 200 with a
non-catalogue body no longer crashes the whole fetch; adding a tag no longer
erases the label and description already stored for that model; `authenticated`
no longer demands a bearer token from a local provider reached by address; and
registry rows are keyed by the spelling every lookup produces, which recovers
593 rows that no reader could reach.

## Type

- [ ] Fix
- [x] Feature
- [x] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Commands run on the branch head, with their results:

- `uv run pytest tests -q -k "provider or registry or catalog or model or rpc or gateway or docker or i18n"` -- 4389 passed, 46 skipped
- `npx vitest run` in `ui-web` -- 1573 passed (101 files)
- `npx vitest run` in `ui-tui` -- 2037 passed (142 files)
- `uv run ruff check raven scripts tests` and `ruff format --check` -- clean
- `npx tsc --noEmit` in `ui-web` and `npm run type-check` in `ui-tui` -- clean
- `npm run lint:rpc` in `ui-tui` -- generated.ts in sync
- `make check-source-language`, `make check-large-files` -- clean

Each of the four review fixes was checked by reverting it and watching a named
test go red, then restoring it.

The container image itself has not been built by CI or by me -- no Docker daemon
was available on the machine this was written on. It has been built and run by
the author of the branch.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

Two deliberate security trade-offs in the container deployment, both documented
in `docker/README.md`:

- nginx rewrites the `Origin` header, because the engine accepts only an Origin
  naming its own loopback port and no browser reaching a published port can send
  one. This drops a defence-in-depth layer. The session cookie is
  SameSite=Strict, so an unauthenticated socket is still refused -- but SameSite
  ignores ports, so another page on localhost counts as same-site.
- `RAVEN_AUTO_LOGIN=1` is the default: nginx attaches the session cookie to
  every page load, so anyone who can reach the published port is signed in. It
  is contained by Compose publishing on 127.0.0.1 only. Set it to 0 for any
  deployment reachable from elsewhere. These are two settings that must agree,
  and nothing warns when they do not.

Behaviour changes outside the container: the gateway now starts without a
configured provider rather than exiting 1, and the first-run onboarding overlay
no longer opens by default (`?onboard=1` still forces it).

Rollback is by revert; nothing here migrates state or changes an on-disk format.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A
2026-09-11 03:27:52 +00:00