Dizhan Xue 24433cb7a6
fix(rpc): sign in a page reached through a port forward (#939)
## Summary

A page reached through a port forward could not sign in. The gateway
accepted a browser `Origin` only when it named the port the gateway
bound, and VS Code and `ssh -L` pick another local port whenever the
remote one is taken on the browser's machine, so every
`POST /auth/exchange` answered `403 bad origin` and every `/rpc`
upgrade was refused with it. The sign-in page reported an expired link
over a refusal it cannot see, and re-running `raven web` minted one
more link into the same refusal. Measured on 2026-10-13: 13 refusals
in six minutes, each from a page at `127.0.0.1:18793` against a
gateway on 18792, all within a minute of their nonce being minted. The
same check had already forced the container's nginx to rewrite `Origin`
for every request it proxies.

`_origin_ok` now also accepts an `Origin` that asks the request's own
origin (`Origin` equals `Host`) on a loopback name. That is what a
browser sends through a forward, since a forwarder relays `Host`
unchanged. It still refuses a page another local server shows, because
cookies and SameSite ignore the port and that origin names a port this
page does not ask; and it still refuses DNS rebinding, where a name
resolving to 127.0.0.1 asks its own origin and the two agree. The
bound-port allowlist stays for the container's nginx, which forwards
`Host` without a port.

Three smaller changes ride with it:

- The session cookie is named for the port the browser reached rather
  than the port bound, so two gateways that both bound 18792 no longer
  share one jar entry when one browser reaches them through different
  forwards. A `Host` without a port keeps the bound port, which is what
  the container's auto-login sets.
- `handle_auth_exchange` answers with the reason it refused, and the
  sign-in page shows that text rather than calling every failure an
  expired link.
- `_open` prints the sign-in link every time, and before handing it to
  a browser. `webbrowser.open` returns True whenever it ran something:
  VS Code's `browser.sh` exits 0 whatever happened, and a machine with
  no display hands the URL to a console browser that has no JavaScript
  and never redeems the nonce. The old fallback was unreachable, so the
  link was never printed at all.

The standalone serve test now uses a real aiohttp `Application` while
replacing only `AppRunner` and `TCPSite`. It selects temporary UI
locations for both an absent page and a built page, and checks that the
response revalidation hook is registered only for the built page. This
removes dependence on whether the checkout contains a UI build.

This PR also retains the reviewed Windows test guards and separate
`RAVEN_HOME` directories for real-upgrade fixtures.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Linux fixture follow-up (Python 3.12.13; the three-file suite and diff
coverage were rechecked after rebasing onto the current main):

- Before the fix, parameterizing `TestTheSignInLink` over an absent page
and a built page produced 4 passed / 1 failed. Only the built-page case
failed, with the same missing `on_response_prepare` as CI.
- `uv run --no-sync pytest -q -n 0
tests/test_cli_serve_commands.py::TestTheSignInLink`: 5 passed after the
fix.
- `uv run --no-sync pytest -q tests/test_rpc_transport.py
tests/test_cli_serve_commands.py tests/test_docker_runtime.py
--cov=raven.rpc.transports.ws --cov=raven.cli.serve_commands
--cov-report=json:<coverage-json>`: 234 passed. `<coverage-json>`
denotes the local coverage output file.
- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
<coverage-json> diff --base-ref origin/main`: 100.00% (32/32 executable
changed lines), passing the 90% threshold. This measures diff coverage,
not the full-suite ratchet.
- A temporary test plugin removed the response hook without changing the
checkout. The built-page case failed its hook assertion and the
absent-page control passed (1 failed / 1 passed).
- `uv run --no-sync ruff check tests/test_cli_serve_commands.py
raven/cli/serve_commands.py`, `uv run --no-sync ruff format --check
tests/test_cli_serve_commands.py raven/cli/serve_commands.py`, and `uv
run --no-sync ty check raven/cli/serve_commands.py
tests/test_cli_serve_commands.py`: passed.
- `UV_NO_SYNC=1 make lint-python lint-imports lint-deps lint-types`:
passed after rebase (2268 Python files formatted; 10 import contracts
kept; no dependency issues; type checks passed). Both commit-message
validators and the repository source-language and large-file gates also
passed.
- The earlier description incorrectly called this an unrelated CI
failure. It was introduced by this PR's standalone-serve test, and the
CI build made that incomplete fixture fail. The fixture follow-up was
validated on Linux; the native Windows and full-suite measurements below
are earlier verification. CI must validate the pushed revision.

Native Windows follow-up (build 10.0.26300):

- Parallel real-upgrade cases now get separate `RAVEN_HOME` directories,
isolating both upgrade markers and downloaded wheels. A scratch barrier
forced both helpers to finish writing the same partial-wheel name before
rename: the original fixture produced 1 failed / 1 passed with
`Distribution not found` in the shared cache; the identical probe with
isolated homes produced 2 passed on Python 3.13.12. The ordinary run had
passed before the forced overlap, so this result identifies a race
rather than a consistently failing upgrade.
- Captured upgrade diagnostics tolerate localized Windows error bytes
and include stderr on a missing success message. Version and
companion-package assertions remain strict.
- Three reported unit-test constraints were reproduced. Symlink setup
now skips only Windows privilege error 1314; other errors propagate and
confinement still runs when a link can be created. Owner-only mode bits
and two SIGTERM-handler tests are explicitly POSIX-only. A native
subprocess probe confirmed that Windows Popen.terminate does not execute
the Python SIGTERM handler.
- `PYTHONUTF8=1`; `uv run --frozen --no-sync pytest
tests/test_rpc_transport.py tests/test_cli_serve_commands.py
tests/test_docker_runtime.py::test_container_bootstraps_the_browser_after_the_engine_is_ready
-n 0 -x --tb=short`: 195 passed, 30 skipped on Python 3.13.12.
Twenty-six skips were already POSIX-specific; four now identify the
reported mode/signal/symlink preconditions. The skipped checks are not
Windows equivalents.
- `PYTHONUTF8=1`, `UV_PYTHON=3.12`, and an isolated caller `RAVEN_HOME`;
`uv run --no-project --python 3.12 --with 'pytest>=8,<10' --with
pytest-asyncio --with pytest-xdist pytest --noconftest
tests/integration/test_cli_upgrade_real_uv.py -q --tb=short -n 4`: 3
passed, covering two real uv upgrade variants and the workflow contract.
- Whole-PR pre-commit, full Python-target Ruff check/format, all ten
import contracts, changed-production type checks, commitlint and
repository commit/asset/source-language gates passed.

Earlier author verification before this Windows follow-up:
- `uv run --frozen --python 3.12 --all-extras pytest -q
tests/test_rpc_transport.py tests/test_cli_serve_commands.py
tests/test_docker_runtime.py` -> `233 passed`.
- The same three files plus every `tests/test_rpc_*.py` and
`tests/test_cli_smoke.py` -> `3121 passed` on the first revision, before
the fixture follow-up.
- The earlier full suite, under `--cov=raven --cov-branch` -> `8 failed,
29791 passed, 158 skipped`, then `scripts/coverage_gate.py diff
--base-ref origin/main --threshold 90` -> `100.00% (32/32 executable
changed lines)`, passed. All eight failures are local baselines: five
proxy tests (`tests/test_config_update_providers.py`) that fail whenever
the shell carries this box's proxy;
`test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`
(node/npm resolution on this box);
`test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing`
(fails as root); and
`test_knowledge_parser_excel.py::test_a_legacy_workbook_is_converted_and_read`.
The last three were replayed in an unmodified baseline worktree and fail
there too (`3 failed`).
- The tests were written before the code and watched fail on it, each
for its own reason: the forwarded origin as `403: bad origin`, the
port-forward cookie case as `two sign-ins wrote one jar entry:
['raven_session_18792']`, the launcher cases on empty output. Against
`main`'s two source files the three test files now fail 18 tests.
- Eleven mutants, one construct each, all killed, with both files
restored and compared byte for byte: the forwarded rule off, Origin
equal to Host dropped, the loopback test dropped, the `localhost` branch
removed, either parse left unguarded, the cookie keyed to the bound
port, the two refusals given one text, the link not printed, the link
printed after the browser, and `raven serve --open` bypassing `_open`.
- `ruff check` and `ruff format --check` clean; `lint-imports` reports
`10 kept, 0 broken`.
- `scripts/check_source_language.py origin/main...HEAD` and
`scripts/check_large_files.py origin/main...HEAD` both exit 0.
- Real browser, end to end, against an isolated `raven serve` on its own
`RAVEN_HOME` reached through a TCP forwarder: the tab signed in and
landed on `/`, the cookie it kept was `raven_session_<forwarded port>`,
and an RPC call over a socket opened from that page was answered (server
log: `serve: ws client connected`, `system.hello`). A direct tab on the
bound port signed in too. With `fetch` stubbed, the page rendered each
of the four answers it can receive (refused address, used or expired
link, other status, no answer) with the matching text.

## Risk

- Security impact considered. The change widens one check: an `Origin`
that equals the request's `Host` on a loopback name. The cross-origin
local-page and DNS-rebinding cases remain refused, and both are covered
by tests. What it accepts that the old rule did not is a page whose
origin is exactly the address it asked for -- which only something
serving that address can show, and a file the agent produced arrives
with the CSP `sandbox` directive and so has an opaque origin.
- Backward compatibility: a tab signed in directly on the bound port
keeps its cookie name and stays signed in. A tab reaching the gateway
through a forward had no working session before, so there is nothing to
migrate. The container's auto-login cookie is unchanged, and a test
derives the entrypoint's name from `cookie_name` so the two cannot
drift.
- Rollback: revert the commit. Nothing is written outside the process
except the cookie already in the browser, and no config or format
changed.
- A `Host` that does not parse (`127.0.0.1:99999`, `127.0.0.1:abc`,
`[::1`) is answered 401 on `/rpc`, as on `main`, and an Origin equal to
such a Host is answered 403. Without the guard both would be a 500,
since the cookie's name is read before any credential is checked.
- Not fixed here: the page's own "this tab's sign-in expired" notice
(main page, i18n `gui.auth.stale`) still names the remedy correctly but
not the cause, so a tab that reaches the gateway by a non-loopback
address still reports a sign-in failure it cannot act on. Also
unmeasured: a real VS Code forward and a real container (both request
shapes were reproduced instead).

## Related Issues

N/A
2026-10-14 11:05:26 +08:00
..

Raven in Docker

Build and run the current checkout:

cd docker
docker compose up --build

To run the configured image without rebuilding, use docker compose up --no-build. Plain docker compose up may reuse or pull an image rather than rebuild local changes.

Then open http://localhost:18793. The local Compose default signs that browser in automatically so Settings > Models is immediately available.

What comes up

One container, two processes:

process what it does
nginx (:80) serves the built page from /app/ui-web/dist, proxies /rpc, /auth*, /health, /file, /files/, /oauth/callback to the engine
the Raven engine the RPC gateway on 127.0.0.1:18793 inside the container

They share a container because the engine binds loopback only (web.TCPSite(runner, "127.0.0.1", port)), so a proxy in a second container would have nothing to dial. The image builds the page in a Node stage and the Python environment with uv sync --frozen, so both halves come from the same commit.

Signing in

The engine mints a session credential per boot and the page needs a cookie for it. Compose defaults RAVEN_AUTO_LOGIN=1, so nginx gives a browser loading the page that credential automatically. This is intended for a locally published port.

For a remotely exposed deployment, put RAVEN_AUTO_LOGIN=0 in .env.local. The entrypoint prints a one-time link on startup:

raven-docker: sign in once with this one-time link (replace localhost if you are remote):

    http://localhost:18793/auth#<nonce>

The cookie it sets lasts 30 days, and survives a restart -- the engine reloads the same cookie from /data rather than minting a new one.

The page's raven web advice is for a host install. In a container, mint another one-time link with:

docker compose exec raven docker-entrypoint.sh signin

Configuring a provider

The engine starts with nothing configured. It says so on the way up:

! no provider is configured yet -- run `raven onboard` for guided setup
  Starting anyway. Add a provider in Settings > Models on the page.

Add one in the page's Settings > Models and the next turn uses it -- the provider reads its credentials per call, so nothing needs restarting.

To have a deployment come up already configured, declare one in .env.local instead:

RAVEN_PROVIDER=anthropic
RAVEN_API_KEY=sk-ant-...

The entrypoint writes that into config on every boot, so where it is set it is the source of truth.

Local providers can be seeded by address without an API key:

RAVEN_PROVIDER=ollama-chat
RAVEN_API_KEY=
RAVEN_API_BASE=http://host.docker.internal:11434

The container always runs raven gateway, including the page, RPC, IM channels, cron, and the proactivity stack. Its resolving provider re-reads credentials on every call, which is what makes a provider added in Settings work on the next turn.

Settings

.env is committed and holds defaults. The service loads the optional, git-ignored .env.local second for container runtime variables such as provider credentials and RAVEN_AUTO_LOGIN. Put every credential in .env.local.

Compose interpolation is separate: RAVEN_IMAGE, RAVEN_WEB_PORT, RAVEN_EXTRAS, RAVEN_PLUGINS, and RAVEN_OFFICE are resolved before the container starts. Setting them only in the service's .env.local does not override the image, published port, or build arguments. From this directory, use shell variables, for example:

RAVEN_IMAGE=raven:local RAVEN_WEB_PORT=18893 docker compose up --build

Or, after creating .env.local, load it explicitly for interpolation too:

docker compose --env-file .env --env-file .env.local up --build

The later file wins; shell environment values override both files.

Data

Everything durable is under /data on the raven-data volume: config.json, the workspace, sessions, logs and the memory store. Removing the container keeps it; docker compose down -v does not.

Running a CLI command

Any argument that is not run or signin is handed to the raven CLI:

docker compose exec raven docker-entrypoint.sh provider test anthropic
docker compose run --rm raven status

Notes and limits

  • Exposure. The default Compose page automatically grants a Raven browser session to anyone who can reach its port, so the published port binds 127.0.0.1 only. A remote deployment must choose its own bind or proxy and set RAVEN_AUTO_LOGIN=0. nginx also rewrites the Origin header so the engine accepts a browser that reached it through a published port. That drops a defence-in-depth layer; the session cookie is SameSite=Strict, so an unauthenticated socket is still refused. Publish this port only where you would have published the engine's own.

  • HTTPS. Not configured here. Terminate TLS in front of this container; the page picks wss:// from location.protocol on its own.

  • The TUI is not in this image. raven tui needs the Node bundle from ui-tui/, which the image does not build.

  • sandbox. The extra is off by default: inside a container the container is the boundary. Turn it on with RAVEN_EXTRAS if you want boxlite as well.

  • LibreOffice is in the image. The page cannot draw a .docx, .xlsx or .pptx, so the gateway converts those to PDF to show them; without it every office file in a knowledge base, and every deck, answers the viewer with "LibreOffice is not installed" -- a hosted image telling its user to install something on it. It is the largest thing here, roughly 500 MB with the CJK fonts it needs to render anything but Latin text. Build with --build-arg RAVEN_OFFICE=0, or RAVEN_OFFICE=0 in the environment Compose reads, for a deployment that will never preview one.

  • The tokenizer dictionary and the deepdoc models are in the image, about 160 MB. They are fetched at build time rather than at first use so the container needs no egress to read a PDF or to segment Chinese -- a deployment without it would otherwise fail the first time somebody uploaded one, long after the build that could have said so. Build with --build-arg RAVEN_RESOURCES=0, or RAVEN_RESOURCES=0 in the environment Compose reads, for a smaller image that does neither; both features report their own absence rather than failing obscurely.

    Before you push this image anywhere, read the "deepdoc models" section of NOTICES.md. One of the four, layout.onnx, stamps AGPL-3.0 into its own metadata under author: Ultralytics, and its metadata contradicts itself about that. Building the image here is your own fetch happening inside a container; pushing it to a registry, or shipping an appliance built from it, is redistribution, and that is where the question becomes yours to answer. RAVEN_RESOURCES=0 is the build that carries none of them.