## Summary Two things arrive together because the second is what made the first usable. **Run raven from a container.** A Dockerfile builds the page, the Python environment and the engine into one image; `cd docker && docker compose up` serves it. nginx serves the built page and proxies the RPC socket to the engine over the loopback they share -- the engine binds 127.0.0.1 only, so a proxy in a second container would have no address to dial. Compose publishes on 127.0.0.1, and the entrypoint prints a one-time sign-in link on boot. **A first run is no longer a dead end.** The gateway used to refuse to start until a provider was configured, while being the thing that serves the page where a provider gets configured. It now starts, says what is missing, and picks up a key saved in Settings on the next turn without a restart. The onboarding overlay no longer takes the page either: New chat, the model chip and Send send the reader to Settings > Models instead, and an empty model list is marked on the control that fills it. Also carried: the provider catalogue registry (capabilities, modalities and context windows per model), the reworked Settings > Models pane, and four fixes found in review of that work -- a sibling catalogue answering 200 with a non-catalogue body no longer crashes the whole fetch; adding a tag no longer erases the label and description already stored for that model; `authenticated` no longer demands a bearer token from a local provider reached by address; and registry rows are keyed by the spelling every lookup produces, which recovers 593 rows that no reader could reach. ## Type - [ ] Fix - [x] Feature - [x] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Commands run on the branch head, with their results: - `uv run pytest tests -q -k "provider or registry or catalog or model or rpc or gateway or docker or i18n"` -- 4389 passed, 46 skipped - `npx vitest run` in `ui-web` -- 1573 passed (101 files) - `npx vitest run` in `ui-tui` -- 2037 passed (142 files) - `uv run ruff check raven scripts tests` and `ruff format --check` -- clean - `npx tsc --noEmit` in `ui-web` and `npm run type-check` in `ui-tui` -- clean - `npm run lint:rpc` in `ui-tui` -- generated.ts in sync - `make check-source-language`, `make check-large-files` -- clean Each of the four review fixes was checked by reverting it and watching a named test go red, then restoring it. The container image itself has not been built by CI or by me -- no Docker daemon was available on the machine this was written on. It has been built and run by the author of the branch. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Two deliberate security trade-offs in the container deployment, both documented in `docker/README.md`: - nginx rewrites the `Origin` header, because the engine accepts only an Origin naming its own loopback port and no browser reaching a published port can send one. This drops a defence-in-depth layer. The session cookie is SameSite=Strict, so an unauthenticated socket is still refused -- but SameSite ignores ports, so another page on localhost counts as same-site. - `RAVEN_AUTO_LOGIN=1` is the default: nginx attaches the session cookie to every page load, so anyone who can reach the published port is signed in. It is contained by Compose publishing on 127.0.0.1 only. Set it to 0 for any deployment reachable from elsewhere. These are two settings that must agree, and nothing warns when they do not. Behaviour changes outside the container: the gateway now starts without a configured provider rather than exiting 1, and the first-run onboarding overlay no longer opens by default (`?onboard=1` still forces it). Rollback is by revert; nothing here migrates state or changes an on-disk format. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A
200 lines
6.1 KiB
Python
200 lines
6.1 KiB
Python
"""Socket conformance suites — the S2 pattern, prototyped on the two hardest
|
|
sockets (provider, executor) with built-in bite-tests.
|
|
|
|
Ruling basis (2026-08-26): replacement clears three gates = the paper
|
|
(shape) + the socket conformance test (behavior) + the baseline diff.
|
|
This file is gate two's exemplar: a reusable checker per socket that ANY answer
|
|
(factory, shelf, third-party) must pass before it may occupy the socket. The
|
|
provider methods are the MEASURED surface (campaign A), not a designed one.
|
|
The response members are read off the paper's dataclass, so the suite checks
|
|
every member the paper declares the day it declares it; the seven the campaign
|
|
measured (campaign A plus the three fields every seven-swap run died on) stay
|
|
pinned as the floor the paper may not drop below.
|
|
"""
|
|
|
|
import dataclasses
|
|
import inspect
|
|
|
|
import pytest
|
|
|
|
from raven.contracts.llm_provider import LLMResponse
|
|
|
|
PROVIDER_METHODS = ("chat", "chat_with_retry", "chat_stream")
|
|
# has_tool_calls is a property, not a field, and gets its own shape check below.
|
|
RESPONSE_MEMBERS = tuple(f.name for f in dataclasses.fields(LLMResponse))
|
|
MEASURED_MEMBERS = (
|
|
"content",
|
|
"tool_calls",
|
|
"reasoning_content",
|
|
"thinking_blocks",
|
|
"usage",
|
|
"finish_reason",
|
|
"error_classification",
|
|
)
|
|
|
|
|
|
def check_provider(provider) -> list[str]:
|
|
"""Shape half of the provider socket's entry ticket."""
|
|
problems = []
|
|
for m in PROVIDER_METHODS:
|
|
fn = getattr(provider, m, None)
|
|
if fn is None:
|
|
problems.append(f"missing method {m}")
|
|
elif not inspect.iscoroutinefunction(fn):
|
|
problems.append(f"{m} must be async")
|
|
return problems
|
|
|
|
|
|
def check_provider_response(resp) -> list[str]:
|
|
problems = [f"response missing {m}" for m in RESPONSE_MEMBERS if not hasattr(resp, m)]
|
|
_MISSING = object()
|
|
val = getattr(resp, "has_tool_calls", _MISSING)
|
|
if val is _MISSING:
|
|
problems.append("response missing has_tool_calls")
|
|
elif callable(val):
|
|
# The loop reads this as an attribute (main.py:3214). A method-shaped
|
|
# stand-in yields a bound method, which is always truthy -- a latent
|
|
# bug, so it is refused here.
|
|
problems.append("has_tool_calls must be a property/attribute, not a method")
|
|
return problems
|
|
|
|
|
|
def check_executor(executor) -> list[str]:
|
|
"""Entry ticket for the sandbox socket (the conformance-test half of the
|
|
hardened admission)."""
|
|
problems = []
|
|
if not isinstance(getattr(type(executor), "is_sandboxed", None), property) and not hasattr(
|
|
executor, "is_sandboxed"
|
|
):
|
|
problems.append("missing is_sandboxed")
|
|
if not hasattr(executor, "supports_process_spawning"):
|
|
problems.append("missing supports_process_spawning")
|
|
for dunder in ("__aenter__", "__aexit__"):
|
|
if not hasattr(type(executor), dunder):
|
|
problems.append(f"{dunder} must live on the type (async CM protocol)")
|
|
return problems
|
|
|
|
|
|
# -- compliant doubles: same shapes the seven-swap runs used; live samples --
|
|
|
|
|
|
class _Resp:
|
|
content = "ok"
|
|
tool_calls: list = []
|
|
reasoning_content = None
|
|
thinking_blocks = None
|
|
usage: dict = {}
|
|
finish_reason = "stop"
|
|
error_classification = None
|
|
truncated = False
|
|
max_tokens = None
|
|
reasoning_ms = None
|
|
call_record = None
|
|
|
|
has_tool_calls = False
|
|
|
|
|
|
class _GoodProvider:
|
|
async def chat(self, messages, **kw):
|
|
return _Resp()
|
|
|
|
async def chat_with_retry(self, messages, **kw):
|
|
return await self.chat(messages, **kw)
|
|
|
|
async def chat_stream(self, messages, **kw):
|
|
return _Resp()
|
|
|
|
|
|
class _GoodExecutor:
|
|
is_sandboxed = False
|
|
supports_process_spawning = True
|
|
|
|
async def __aenter__(self):
|
|
return self
|
|
|
|
async def __aexit__(self, *exc):
|
|
return False
|
|
|
|
|
|
def test_compliant_provider_passes():
|
|
assert check_provider(_GoodProvider()) == []
|
|
|
|
|
|
def test_compliant_response_passes():
|
|
assert check_provider_response(_Resp()) == []
|
|
|
|
|
|
def test_compliant_executor_passes():
|
|
assert check_executor(_GoodExecutor()) == []
|
|
|
|
|
|
def test_factory_response_shape_is_compliant():
|
|
resp = LLMResponse(content="x")
|
|
assert check_provider_response(resp) == []
|
|
|
|
|
|
def test_the_paper_still_declares_every_measured_member():
|
|
"""Deriving the list from the paper must not let the paper quietly drop a
|
|
member the campaign found lethal; the seven are the floor."""
|
|
dropped = sorted(set(MEASURED_MEMBERS) - set(RESPONSE_MEMBERS))
|
|
assert dropped == [], f"the provider paper no longer declares {dropped}"
|
|
|
|
|
|
# -- bite tests: built-in mutation audit; every declared response member --
|
|
# -- and the async context manager must each draw blood --
|
|
|
|
|
|
def test_sync_chat_is_rejected():
|
|
class Bad(_GoodProvider):
|
|
def chat(self, messages, **kw): # not async
|
|
return _Resp()
|
|
|
|
assert any("must be async" in p for p in check_provider(Bad()))
|
|
|
|
|
|
def test_method_form_has_tool_calls_is_rejected():
|
|
class Sneaky(_Resp):
|
|
def has_tool_calls(self): # method-shaped: bound method is always truthy
|
|
return False
|
|
|
|
assert any("not a method" in p for p in check_provider_response(Sneaky()))
|
|
|
|
|
|
def test_missing_retry_method_is_rejected():
|
|
class Bad:
|
|
async def chat(self, messages, **kw):
|
|
return _Resp()
|
|
|
|
async def chat_stream(self, messages, **kw):
|
|
return _Resp()
|
|
|
|
assert any("chat_with_retry" in p for p in check_provider(Bad()))
|
|
|
|
|
|
@pytest.mark.parametrize("missing", sorted(RESPONSE_MEMBERS))
|
|
def test_response_missing_declared_member_is_rejected(missing):
|
|
resp = _Resp()
|
|
|
|
class Stripped:
|
|
pass
|
|
|
|
for m in RESPONSE_MEMBERS:
|
|
if m != missing:
|
|
setattr(Stripped, m, getattr(resp, m))
|
|
Stripped.has_tool_calls = False
|
|
assert any(missing in p for p in check_provider_response(Stripped()))
|
|
|
|
|
|
def test_executor_without_type_level_aenter_is_rejected():
|
|
class Bad:
|
|
is_sandboxed = False
|
|
supports_process_spawning = True
|
|
|
|
def __getattr__(self, name): # instance-level fake, must NOT count
|
|
async def _f(*a):
|
|
return self
|
|
|
|
return _f
|
|
|
|
assert any("__aenter__" in p for p in check_executor(Bad()))
|