## Summary Two things arrive together because the second is what made the first usable. **Run raven from a container.** A Dockerfile builds the page, the Python environment and the engine into one image; `cd docker && docker compose up` serves it. nginx serves the built page and proxies the RPC socket to the engine over the loopback they share -- the engine binds 127.0.0.1 only, so a proxy in a second container would have no address to dial. Compose publishes on 127.0.0.1, and the entrypoint prints a one-time sign-in link on boot. **A first run is no longer a dead end.** The gateway used to refuse to start until a provider was configured, while being the thing that serves the page where a provider gets configured. It now starts, says what is missing, and picks up a key saved in Settings on the next turn without a restart. The onboarding overlay no longer takes the page either: New chat, the model chip and Send send the reader to Settings > Models instead, and an empty model list is marked on the control that fills it. Also carried: the provider catalogue registry (capabilities, modalities and context windows per model), the reworked Settings > Models pane, and four fixes found in review of that work -- a sibling catalogue answering 200 with a non-catalogue body no longer crashes the whole fetch; adding a tag no longer erases the label and description already stored for that model; `authenticated` no longer demands a bearer token from a local provider reached by address; and registry rows are keyed by the spelling every lookup produces, which recovers 593 rows that no reader could reach. ## Type - [ ] Fix - [x] Feature - [x] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Commands run on the branch head, with their results: - `uv run pytest tests -q -k "provider or registry or catalog or model or rpc or gateway or docker or i18n"` -- 4389 passed, 46 skipped - `npx vitest run` in `ui-web` -- 1573 passed (101 files) - `npx vitest run` in `ui-tui` -- 2037 passed (142 files) - `uv run ruff check raven scripts tests` and `ruff format --check` -- clean - `npx tsc --noEmit` in `ui-web` and `npm run type-check` in `ui-tui` -- clean - `npm run lint:rpc` in `ui-tui` -- generated.ts in sync - `make check-source-language`, `make check-large-files` -- clean Each of the four review fixes was checked by reverting it and watching a named test go red, then restoring it. The container image itself has not been built by CI or by me -- no Docker daemon was available on the machine this was written on. It has been built and run by the author of the branch. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Two deliberate security trade-offs in the container deployment, both documented in `docker/README.md`: - nginx rewrites the `Origin` header, because the engine accepts only an Origin naming its own loopback port and no browser reaching a published port can send one. This drops a defence-in-depth layer. The session cookie is SameSite=Strict, so an unauthenticated socket is still refused -- but SameSite ignores ports, so another page on localhost counts as same-site. - `RAVEN_AUTO_LOGIN=1` is the default: nginx attaches the session cookie to every page load, so anyone who can reach the published port is signed in. It is contained by Compose publishing on 127.0.0.1 only. Set it to 0 for any deployment reachable from elsewhere. These are two settings that must agree, and nothing warns when they do not. Behaviour changes outside the container: the gateway now starts without a configured provider rather than exiting 1, and the first-run onboarding overlay no longer opens by default (`?onboard=1` still forces it). Rollback is by revert; nothing here migrates state or changes an on-disk format. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A
246 lines
8.6 KiB
Python
246 lines
8.6 KiB
Python
"""The loop's own pre-request awaits are bounded, and a stall there is reported.
|
|
|
|
Every timeout a provider holds can only fire once the call has been entered.
|
|
The deck stall of 2026-09-10 never got that far: ``Iteration 1/600`` opened,
|
|
``memory.recall`` and ``skill.inject`` finished, and then fifteen minutes passed
|
|
with no ``llm.call`` span, no ``llm.input`` artefact, no retry line and no
|
|
established outbound socket -- the turn never issued its first request, so
|
|
nothing at the provider could have caught it. A retry of the identical turn was
|
|
healthy.
|
|
|
|
Covers:
|
|
- a hook that never returns does not hold the turn: the stage is abandoned at
|
|
``llmFirstByteTimeout`` and the model is asked anyway
|
|
- a strategy that never returns leaves the request un-hooked rather than unsent
|
|
- a ``TimeoutError`` the stage raised itself is not this guard's deadline and
|
|
propagates, so a bad pre-process still cannot send an unprocessed request
|
|
- the record names the stage, the bound and how long it waited
|
|
- a budget of 0 switches the guard off, and a stage that answers in time is
|
|
untouched
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import tempfile
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
import pytest
|
|
|
|
from raven.agent.loop import AgentLoop
|
|
from raven.agent.loop.bundles import EngineWiring, HostWiring, ToolWiring, TurnPolicy
|
|
from raven.agent.loop.first_call import FirstCallGuard
|
|
from raven.contracts.llm_provider import GenerationSettings
|
|
from raven.contracts.loop_hooks import AgentHook, AgentHookContext, HookDecision
|
|
from raven.contracts.token_strategy import TokenStrategy
|
|
from raven.providers.base import LLMProvider, LLMResponse
|
|
from raven.spine.message import ChatType, Source
|
|
from raven.spine.turn import Origin, TurnRequest
|
|
from raven.token_wise.registry import StrategyRegistry
|
|
|
|
|
|
@pytest.fixture
|
|
def workspace():
|
|
with tempfile.TemporaryDirectory() as td:
|
|
yield Path(td)
|
|
|
|
|
|
class _Answers(LLMProvider):
|
|
"""Answers at once, and records what it was asked with."""
|
|
|
|
def __init__(self, first_byte_timeout: float = 0.05):
|
|
super().__init__(api_key="test")
|
|
self.generation = GenerationSettings(timeout=600.0, first_byte_timeout=first_byte_timeout)
|
|
self.calls = 0
|
|
self.asked_tools: list[Any] = []
|
|
|
|
def get_default_model(self) -> str:
|
|
return "stub"
|
|
|
|
_CHAT_RETRY_DELAYS = ()
|
|
|
|
async def chat(self, messages, tools=None, model=None, max_tokens=4096, temperature=0.7, **_):
|
|
self.calls += 1
|
|
self.asked_tools.append(tools)
|
|
return LLMResponse(content="real answer", finish_reason="stop")
|
|
|
|
async def chat_stream(self, *args, **kwargs): # pragma: no cover - non-stream path under test
|
|
raise NotImplementedError
|
|
|
|
|
|
class _StallingHook(AgentHook):
|
|
"""A ``before_iteration`` that never comes back."""
|
|
|
|
def __init__(self) -> None:
|
|
self.entered = 0
|
|
|
|
async def before_iteration(self, ctx: AgentHookContext) -> HookDecision:
|
|
self.entered += 1
|
|
await asyncio.sleep(30)
|
|
return HookDecision(short_circuit_result="the hook answered") # pragma: no cover
|
|
|
|
|
|
class _StallingStrategy(TokenStrategy):
|
|
"""A ``before_llm_call`` that never comes back."""
|
|
|
|
name = "stalls"
|
|
|
|
def __init__(self) -> None:
|
|
self.entered = 0
|
|
|
|
async def before_llm_call(self, messages, tools, model):
|
|
self.entered += 1
|
|
await asyncio.sleep(30)
|
|
return [], None, model # pragma: no cover
|
|
|
|
|
|
def _agent(workspace: Path, provider: LLMProvider, *, hooks=None, strategies=None) -> AgentLoop:
|
|
return AgentLoop(
|
|
provider=provider,
|
|
workspace=workspace,
|
|
model="stub",
|
|
policy=TurnPolicy(max_iterations=4),
|
|
tools=ToolWiring(restrict_to_workspace=True),
|
|
host=HostWiring(hooks=hooks or []),
|
|
engine=EngineWiring(strategies=StrategyRegistry(strategies or [])),
|
|
)
|
|
|
|
|
|
async def _turn(agent: AgentLoop):
|
|
return await agent._process_message(
|
|
TurnRequest(
|
|
origin=Origin.USER,
|
|
source=Source(channel="test", chat_id="c1", sender_id="user", chat_type=ChatType.DM),
|
|
text="build the deck",
|
|
),
|
|
session_key="s1",
|
|
)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_hook_that_never_returns_does_not_hold_the_turn(workspace):
|
|
hook = _StallingHook()
|
|
provider = _Answers()
|
|
agent = _agent(workspace, provider, hooks=[hook])
|
|
|
|
out = await asyncio.wait_for(_turn(agent), 20)
|
|
|
|
assert hook.entered >= 1, "the stalling hook really was entered"
|
|
assert provider.calls >= 1, "the model was asked even though the hook never answered"
|
|
assert out is not None and out[0] == "real answer"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_stalled_strategy_leaves_the_request_unhooked(workspace):
|
|
strategy = _StallingStrategy()
|
|
provider = _Answers()
|
|
agent = _agent(workspace, provider, strategies=[strategy])
|
|
|
|
out = await asyncio.wait_for(_turn(agent), 20)
|
|
|
|
assert strategy.entered >= 1
|
|
assert provider.calls >= 1
|
|
assert provider.asked_tools[0], "asked with the loop's own tool schemas, not the strategy's empty result"
|
|
assert out is not None and out[0] == "real answer"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_stages_own_timeout_error_is_not_this_guards_deadline():
|
|
"""``asyncio.wait_for`` re-raises a child's ``TimeoutError`` as its own.
|
|
|
|
So an ``except TimeoutError`` around it read a strategy's immediate failure
|
|
as a 0.0-second first-call stall and answered the fallback -- which around
|
|
``StrategyRegistry.before_llm_call`` means sending the original messages,
|
|
tools and model. The TokenStrategy contract has before-call errors propagate
|
|
exactly so a bad pre-process cannot send an unprocessed request.
|
|
"""
|
|
raised: list[str] = []
|
|
|
|
async def refuses():
|
|
raised.append("ran")
|
|
raise TimeoutError("the strategy's own bound, not the guard's")
|
|
|
|
guard = FirstCallGuard(30.0, model="stub")
|
|
with pytest.raises(TimeoutError, match="the strategy's own bound"):
|
|
await guard.stage("the before_llm_call strategies", refuses(), iteration=1, fallback=("fallback",))
|
|
assert raised == ["ran"]
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_stages_own_timeout_error_is_not_recorded_as_a_stall():
|
|
"""And it leaves no first-call-guard record: nothing stalled."""
|
|
from loguru import logger
|
|
|
|
lines: list[str] = []
|
|
sink = logger.add(lambda m: lines.append(m), level="ERROR")
|
|
|
|
async def refuses():
|
|
raise TimeoutError("mine, not yours")
|
|
|
|
try:
|
|
guard = FirstCallGuard(30.0, model="stub")
|
|
with pytest.raises(TimeoutError):
|
|
await guard.stage("the before_iteration hooks", refuses(), iteration=1, fallback=None)
|
|
finally:
|
|
logger.remove(sink)
|
|
|
|
assert not [ln for ln in lines if "First call guard" in ln]
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_the_record_names_the_stage_and_the_wait(workspace):
|
|
"""Today's failure left nothing; a stall must say which bound was hit and
|
|
how long it waited."""
|
|
from loguru import logger
|
|
|
|
lines: list[str] = []
|
|
sink = logger.add(lambda m: lines.append(m), level="ERROR")
|
|
try:
|
|
agent = _agent(workspace, _Answers(), hooks=[_StallingHook()])
|
|
await asyncio.wait_for(_turn(agent), 20)
|
|
finally:
|
|
logger.remove(sink)
|
|
|
|
stalls = [ln for ln in lines if "First call guard" in ln]
|
|
assert stalls, f"no first-call-guard record in {lines}"
|
|
record = stalls[0]
|
|
assert "the before_iteration hooks" in record
|
|
assert "llmFirstByteTimeout" in record
|
|
assert "quiet for 0.1s" in record or "quiet for 0.0s" in record
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_stage_that_answers_in_time_is_untouched():
|
|
guard = FirstCallGuard(5.0, model="stub")
|
|
assert await guard.stage("x", asyncio.sleep(0, result="value"), iteration=1, fallback="fallback") == "value"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_zero_budget_switches_the_guard_off():
|
|
"""0 restores what the loop did before: no bound at all on these awaits."""
|
|
|
|
async def slow():
|
|
await asyncio.sleep(0.15)
|
|
return "late but delivered"
|
|
|
|
guard = FirstCallGuard(0, model="stub")
|
|
assert guard.budget == 0.0
|
|
assert await guard.stage("x", slow(), iteration=1, fallback="fallback") == "late but delivered"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_the_stalled_awaitable_is_cancelled_not_left_running():
|
|
"""A guard that abandoned a stage must not leave its coroutine alive to
|
|
finish later and mutate the turn behind the loop's back."""
|
|
finished: list[str] = []
|
|
|
|
async def slow():
|
|
await asyncio.sleep(0.3)
|
|
finished.append("ran on") # pragma: no cover
|
|
|
|
guard = FirstCallGuard(0.05, model="stub")
|
|
assert await guard.stage("x", slow(), iteration=1, fallback=None) is None
|
|
await asyncio.sleep(0.4)
|
|
assert finished == []
|