Raven-1/tests/test_context_engine_factory.py
zhanghui efe0f46aed
feat(agent): annotate the conversation tail into the episodic log (#924)
## Summary

`user_memory/episodic/episodes.md` had no writer on any surface people
use.
Its only one, the annotator, runs inside the token-pressure compaction
loop,
and all three calls into that loop are gated on `owns_compaction`, which
the
single shipped context engine hardcodes to `True`. The annotator
survived as
live code with one caller: `/new` typed as message text in a chat
channel,
which the web and TUI "new session" buttons never send. Six readers read
the
empty file: the prompt line, two attention producers and three predictor
modules.

This gives the log a writer that does not depend on compaction. After a
user
turn, once `memory.annotateEveryTurns` user turns have accumulated since
the
last annotation, the unannotated tail goes to the same annotator, in the
background, under the consolidator's per-session lock so it cannot
interleave
with `/new`.

Key decisions, each with the alternative it rejected:

- **Its own cursor, not `last_consolidated`.** The `owns_compaction`
gate
  exists to keep a second writer off the Curator's pointer, and
  `undo_last_turn` reads that pointer as a floor. The cursor is
  `session.metadata["annotated_through"]`.
- **Patched, not saved.** A detached task that calls `sessions.save`
with its
  own copy of the metadata drops keys the next turn wrote;
  `append_metadata_patch` exists for exactly this writer.
- **Seeded at the length before the first seen turn.** The old loop fed
the
annotator chunks bounded by a token budget; this step has no chunking,
so an
unseeded long session would go to the provider in one call. Seeding also
covers forks, which copy messages but not metadata. Seeding at the
*current*
  length instead would skip the first turn of every new session.
- **Truncation moves the cursor where it happens.** A lazy clamp at the
next
turn leaves the stale index on disk and swallows every turn written
after an
undo or a clear until the count climbs past it. `Session.clear` drops
the
cursor and `Session.undo_last_turn` pulls it back to the cut, beside the
same
  methods' handling of `last_consolidated`.
- **Only `Origin.USER` turns.** Two of the file's readers are Sentinel
producers; writing Sentinel's own turns into it would feed the layer its
own
  output.

`memory.annotateEveryTurns` ships at 0, which turns the step off: the
log has
had no writer for the life of the project, so no install is waiting on
one,
and nobody is billed for a side provider call to fill a file they may
never
read. Set it to 5 (or any positive cadence) to turn it on.

### The second commit, and why it is here

Driving this on a live gateway showed the annotator could not write at
all on
the default model. It sends `tool_choice="required"`, and DeepSeek's
thinking
mode answers `HTTP 400: Thinking mode does not support this
tool_choice`. The
Responses adapter returned that as the call's result, so the annotator
logged
"LLM did not call annotate_conversation" and gave up -- correct
behaviour, over
a feature that could never work.

Nine call sites force a tool choice today: two in the memory
consolidator,
three in the proactive layer, four in the playbook compiler. Each
already
handles a model that declines under `auto`. So the adapter now does what
its
Anthropic sibling already did: a 400 that names `tool_choice` while the
body
forces one relaxes it to `auto` and retries once. Measured against the
live
vendor three ways -- `required` answered 400, `auto` and omitted both
produced
the tool call.

The two commits are separable: the provider repair stands on its own and
fixes
eight other latent call sites; without it the first commit is inert on
this
model.

## Type

- [x] Feature

## Verification

Unit, on this branch:

- `uv run --frozen --python 3.12 pytest
tests/test_agent_loop_episodic_annotate.py
tests/test_providers_forced_tool_choice.py tests/test_session_manager.py
tests/test_segments.py tests/test_agent_loop_backend_dispatch.py
tests/test_context_engine_factory.py tests/test_providers*.py -q` -> 344
passed
- Broader sweep over the loop, session, provider, context-engine, memory
and
  playbook suites -> 2919 passed, 103 skipped, 1 failed:

`test_agent_loop_token_budget.py::test_a_ceiling_as_large_as_the_window_still_leaves_room_for_history`
asserts `reserved_tools < 7_500` and measures 7557. Pre-existing: a
clean
`github/main` worktree at `829eb28e1` fails it with the same 7557. This
  change touches no tool schema.
- `make lint-types` -> All checks passed
- `uv run --frozen --all-extras lint-imports` -> 10 contracts kept, 0
broken
- `ruff check` / `ruff format --check` on every touched file -> clean
- `scripts/check_source_language.py github/main...HEAD` -> exit 0
- `scripts/check_large_files.py github/main` -> exit 0
- commitlint per sha, with the repo config -> both ok

Every new guard was mutation-verified: deleted, the suite run, the
matching
test confirmed red on its own symptom, restored, md5-compared. Nine
guards,
nine reds -- the knob clause, the origin check, the session lock, the
cursor-on-decline return, the seed, the undo pull-back, the clear drop,
and
the provider repair's two halves.

Live gateway, isolated `RAVEN_HOME`, real provider, `memory.backend =
null`:

| What | Result |
|---|---|
| 12-message pre-existing session, one turn | cursor seeded at 12,
`episodes.md` 0 bytes |
| two turns past the cursor | no annotator call, cursor unmoved |
| three substantive turns | `annotate done for 6 messages -> 3
episode(s)`; three lines carrying a token minted at run time |
| those three lines through `parse_episode_line` | all three parse,
tagged `#project-<token>` the way `project_rhythm` ranks by |
| `annotateEveryTurns: 0` | file unchanged, no annotator call, cursor
not even seeded |
| `session.undo` of two turns | `removed: 4`, cursor pulled 26 -> 22 in
the record the undo wrote |
| kill by pid, restart | cursor still 26 |
| trivia-only turns | `annotate done for 8 messages -> 0 episode(s)`,
cursor advanced -- process-only episodes are dropped by design |
| identity prompt, both branches | no-backend unchanged; with a backend
the log is named read-only and the profile path stays out |

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

User-visible changes:

- Nothing changes for an install that leaves `annotateEveryTurns` at its
  default of 0: the step short-circuits before the cursor is seeded. An
install that sets a positive cadence runs one annotator LLM call per
that
  many user turns, plus a profile refresh when a tag crosses the hot
threshold; with a memory backend wired that means two extractors over
the
same conversation. The knob is read at boot, so a change needs a gateway
  reload.
- With a backend wired the identity prompt gains one read-only line
naming the
  episodic log.
- A forced tool choice that a vendor refuses now costs one extra request
instead of failing the call. Callers that relied on the error are none:
all
  nine already handle a declining model.

Rollback: back to `memory.annotateEveryTurns: 0` disables the first
commit's
behaviour without reverting it. The provider repair only changes a path
that previously
ended in an error response.

Not covered: a crash between the file write and the metadata patch
re-annotates
one tail, duplicating lines. Harmless to grep and over-counts a tag by
one
batch at most; making it atomic means writing the cursor into the same
file
under the same lock.

## Related Issues

#122

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-13 20:10:59 +08:00

718 lines
31 KiB
Python

"""ContextAssembler — factory wiring + SkillForgeRouter assembly.
The factory no longer dispatches on ``context.engine`` — it always
builds the single :class:`ContextAssembler` from a flat SegmentBuilder
list, assembling the ``SkillsSegmentBuilder``'s SkillForgeRouter from:
- Local (always),
- Mass (when ``mass.endpoint`` is set),
- the memory backend (when one is supplied).
With no backend the engine still constructs (recall lane yields [],
router runs Local-only). AgentLoop always delegates skill selection to
the engine (``_uses_default_engine`` is always True), and
``_collect_injected_skill_ids`` prefers the assembled-metadata stash.
"""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from raven.agent.context import ContextBuilder
from raven.agent.loop import AgentLoop
from raven.agent.loop.bundles import EngineWiring, SubagentWiring, ToolWiring, TurnPolicy
from raven.agent.subagent.builtin_agents import GENERIC_AGENT
from raven.config.raven import (
ContextConfig,
HubSourceConfig,
MemoryConfig,
SkillForgeConfig,
SkillForgeRouterConfig,
)
from raven.config.schema import SubagentsConfig
from raven.context_engine import ContextAssembler
from raven.context_engine.factory import _build_router, build_context_engine
from raven.context_engine.segments import (
IdentitySegmentBuilder,
MemorySegmentBuilder,
SkillsSegmentBuilder,
)
from raven.context_engine.segments.curator import CuratorSegmentBuilder
from raven.contracts.assembled import TokenBudget
from raven.contracts.context import AssemblyContext
from raven.memory_engine.skill_forge import (
BackendSkillSource,
HubSkillSource,
LocalSkillSource,
)
from raven.providers.binding import ModelBinding, use_binding
# ---------------------------------------------------------------------------
# Test doubles
# ---------------------------------------------------------------------------
class _FakeBackend:
async def start(self):
pass
async def stop(self):
pass
async def feedback(self, signals):
pass
async def store(self, session_id, messages, *, metadata=None):
pass
async def recall(self, query, *, user_id=None, agent_id=None, top_k):
return []
class _StubProvider:
api_key = "test"
def get_default_model(self) -> str:
return "stub"
async def chat(self, *args, **kwargs):
raise NotImplementedError
async def chat_with_retry(self, *args, **kwargs):
raise NotImplementedError
def _stub_get_defs() -> list[dict]:
return []
class _StubPool:
"""Pairs any pin it is asked for, so a test can assert which one was asked."""
def bind_pin(self, model, provider_name=None):
from raven.providers.binding import ModelBinding
return ModelBinding(MagicMock(name=provider_name or "stub-provider"), model) if model else None
def _point_live_config(tmp_path: Path, monkeypatch, payload: dict) -> Path:
"""Point ``LiveConfig``'s default path at a file this test owns."""
cfg = tmp_path / "config.json"
cfg.write_text(json.dumps(payload), encoding="utf-8")
monkeypatch.setattr("raven.config.loader.get_config_path", lambda: cfg)
return cfg
def _build_engine(
tmp_path: Path,
*,
backend=None,
hub_endpoint: str | None = None,
memory_config: MemoryConfig | None = None,
model: str = "stub",
skill_forge_config: SkillForgeConfig | None = None,
rrf_k: int | None = None,
provider_pool=None,
) -> ContextAssembler:
builder = ContextBuilder(workspace=tmp_path)
engine = build_context_engine(
provider_pool=provider_pool,
workspace=tmp_path,
config=ContextConfig(),
builder=builder,
provider=_StubProvider(),
model=model,
context_window_tokens=8192,
get_tool_definitions=_stub_get_defs,
backend=backend,
memory_config=memory_config or MemoryConfig(),
skill_forge_router_config=SkillForgeRouterConfig(
hub=HubSourceConfig(endpoint=hub_endpoint),
**({} if rrf_k is None else {"rrf_k": rrf_k}),
),
# Most assertions in this file exercise the push pipeline's wiring;
# pull-mode wiring has its own tests below.
skill_forge_config=skill_forge_config or SkillForgeConfig(discovery="push"),
)
assert isinstance(engine, ContextAssembler)
return engine
def _router_sources(engine: ContextAssembler):
skills = next(b for b in engine._builders if isinstance(b, SkillsSegmentBuilder))
return [type(s) for s in skills._router._sources], skills._router._sources
def _memory_builder(engine: ContextAssembler) -> MemorySegmentBuilder:
return next(b for b in engine._builders if isinstance(b, MemorySegmentBuilder))
def _curator_builder(engine: ContextAssembler) -> CuratorSegmentBuilder:
return next(b for b in engine._builders if isinstance(b, CuratorSegmentBuilder))
# ---------------------------------------------------------------------------
# Factory — always builds the assembler
# ---------------------------------------------------------------------------
class TestFactory:
def test_returns_assembler_with_backend(self, tmp_path: Path) -> None:
assert isinstance(_build_engine(tmp_path, backend=_FakeBackend()), ContextAssembler)
def test_returns_assembler_without_backend(self, tmp_path: Path) -> None:
engine = _build_engine(tmp_path, backend=None)
assert isinstance(engine, ContextAssembler)
assert _memory_builder(engine)._backend is None
# ---------------------------------------------------------------------------
# SF6: set_context_window cascades down to the Curator's trimmer -- a
# /model switch must not leave it budgeting against the pre-switch window.
# ---------------------------------------------------------------------------
class TestSetContextWindow:
def test_engine_cascades_into_the_curator_trimmer(self, tmp_path: Path) -> None:
engine = _build_engine(tmp_path)
curator = _curator_builder(engine)
assert curator.context_window_tokens == 8192
assert curator.assembler.context_window_tokens == 8192
assert curator.assembler.trimmer.context_window_tokens == 8192
engine.set_context_window(4096)
assert curator.context_window_tokens == 4096
assert curator.assembler.context_window_tokens == 4096
assert curator.assembler.trimmer.context_window_tokens == 4096
def test_engine_cascade_does_not_raise_for_builders_without_the_hook(self, tmp_path: Path) -> None:
"""seg1-5 carry no budget and have no ``set_context_window`` -- the
cascade must skip them rather than assume every builder has it."""
engine = _build_engine(tmp_path)
non_curator = [b for b in engine._builders if not isinstance(b, CuratorSegmentBuilder)]
assert non_curator, "fixture should include builders other than the Curator"
engine.set_context_window(4096) # must not raise
# ---------------------------------------------------------------------------
# SkillForgeRouter assembly — which sources are present
# ---------------------------------------------------------------------------
class TestSkillForgeRouterAssembly:
def test_local_source_always_present(self, tmp_path: Path) -> None:
types, _ = _router_sources(_build_engine(tmp_path, backend=_FakeBackend()))
assert LocalSkillSource in types
def test_backend_source_present_when_backend(self, tmp_path: Path) -> None:
types, _ = _router_sources(_build_engine(tmp_path, backend=_FakeBackend()))
assert BackendSkillSource in types
def test_backend_source_absent_without_backend(self, tmp_path: Path) -> None:
types, _ = _router_sources(_build_engine(tmp_path, backend=None))
assert BackendSkillSource not in types
def test_hub_source_omitted_when_endpoint_unset(self, tmp_path: Path) -> None:
types, _ = _router_sources(_build_engine(tmp_path, backend=_FakeBackend(), hub_endpoint=None))
assert HubSkillSource not in types
def test_hub_source_present_when_endpoint_set(self, tmp_path: Path) -> None:
types, _ = _router_sources(_build_engine(tmp_path, backend=_FakeBackend(), hub_endpoint="http://hub.test"))
assert HubSkillSource in types
def test_rrf_k_forwarded_from_config(self, tmp_path: Path) -> None:
engine = _build_engine(tmp_path, backend=_FakeBackend(), rrf_k=25)
skills = next(b for b in engine._builders if isinstance(b, SkillsSegmentBuilder))
assert skills._router._rrf_k == 25
def test_rrf_k_defaults_to_config_default(self, tmp_path: Path) -> None:
engine = _build_engine(tmp_path, backend=_FakeBackend())
skills = next(b for b in engine._builders if isinstance(b, SkillsSegmentBuilder))
assert skills._router._rrf_k == SkillForgeRouterConfig().rrf_k
def test_track_ids_from_memory_config(self, tmp_path: Path) -> None:
engine = _build_engine(
tmp_path,
backend=_FakeBackend(),
memory_config=MemoryConfig(
user_id="alice",
agent_id="robo",
),
)
assert _memory_builder(engine)._user_id == "alice"
_, sources = _router_sources(engine)
everos = next(s for s in sources if isinstance(s, BackendSkillSource))
assert everos._agent_id == "robo"
def test_router_names_the_backend_source_after_memory_backend(self, tmp_path: Path) -> None:
router = _build_router(
builder=ContextBuilder(workspace=tmp_path),
backend=_FakeBackend(),
memory_config=MemoryConfig(backend="acme", agent_id="a"),
skill_forge_router_config=SkillForgeRouterConfig(weights={"local": 1.0, "acme": 0.7}),
)
names = {s.name: s.weight for s in router._sources}
assert names["acme"] == 0.7
# ---------------------------------------------------------------------------
# Rewriter / gate model wiring — both must follow the agent's main model
# unless the gate has its own dedicated override.
# ---------------------------------------------------------------------------
class TestRewriterGateModelWiring:
def test_the_rewriter_carries_no_model_of_its_own(self, tmp_path: Path) -> None:
"""It follows the conversation, so it must not be pinned to the model
that happened to build the engine -- a session on another model would
otherwise have its query rewritten by the first session's model."""
engine = _build_engine(
tmp_path,
model="main-model",
skill_forge_config=SkillForgeConfig(discovery="push", rewrite_enabled=True, llm_gate_enabled=False),
)
skills = next(b for b in engine._builders if isinstance(b, SkillsSegmentBuilder))
assert not hasattr(skills._rewriter, "_model")
def test_an_unset_gate_model_stays_unset(self, tmp_path: Path) -> None:
"""Same rule one subsystem over: unset means follow the turn, not
"inherit whatever build_context_engine was called with"."""
engine = _build_engine(
tmp_path,
model="main-model",
skill_forge_config=SkillForgeConfig(
discovery="push", rewrite_enabled=False, llm_gate_enabled=True, llm_gate_model=None
),
)
skills = next(b for b in engine._builders if isinstance(b, SkillsSegmentBuilder))
assert skills._gate._pin_resolver() is None
def test_gate_prefers_dedicated_llm_gate_model(self, tmp_path: Path, monkeypatch) -> None:
cfg = _point_live_config(tmp_path, monkeypatch, {"skillForge": {"llmGateModel": "gate-only-model"}})
engine = _build_engine(
tmp_path,
model="main-model",
skill_forge_config=SkillForgeConfig(
discovery="push",
rewrite_enabled=False,
llm_gate_enabled=True,
llm_gate_model="gate-only-model",
),
provider_pool=_StubPool(),
)
skills = next(b for b in engine._builders if isinstance(b, SkillsSegmentBuilder))
assert skills._gate._binding()[1] == "gate-only-model"
assert cfg.exists()
def test_the_gate_pin_follows_the_file_rather_than_the_build_argument(self, tmp_path: Path, monkeypatch) -> None:
"""A gate model repointed on the settings page reaches the next filter.
The factory argument is the config as it stood when the engine was
built; the pin is read from the file when the gate runs, so the two can
disagree and the file is what counts. Before this the pair was resolved
once here, and the settings surface answered every write to it with
"applies after the next gateway reload or restart".
"""
cfg = _point_live_config(tmp_path, monkeypatch, {"skillForge": {"llmGateModel": "gate-from-file"}})
engine = _build_engine(
tmp_path,
skill_forge_config=SkillForgeConfig(
discovery="push",
rewrite_enabled=False,
llm_gate_enabled=True,
llm_gate_model="gate-at-build",
),
provider_pool=_StubPool(),
)
gate = next(b for b in engine._builders if isinstance(b, SkillsSegmentBuilder))._gate
assert gate._binding()[1] == "gate-from-file"
cfg.write_text(json.dumps({"skillForge": {"llmGateModel": "gate-after-edit"}}), encoding="utf-8")
assert gate._binding()[1] == "gate-after-edit"
def test_the_curator_pin_follows_the_file_too(self, tmp_path: Path, monkeypatch) -> None:
"""Same seam, the other subsystem pin. Both blocks are written by the
same settings surface and both used to owe a reload."""
cfg = _point_live_config(tmp_path, monkeypatch, {"context": {"curatorModel": "curator-from-file"}})
engine = _build_engine(tmp_path, provider_pool=_StubPool())
curator = next(b for b in engine._builders if isinstance(b, CuratorSegmentBuilder))
assert curator.curator_model == "curator-from-file"
cfg.write_text(json.dumps({"context": {"curatorModel": "curator-after-edit"}}), encoding="utf-8")
assert curator.curator_model == "curator-after-edit"
# ---------------------------------------------------------------------------
# AgentLoop helpers
# ---------------------------------------------------------------------------
def _make_loop(tmp_path: Path, *, backend=None, agents=None, skill_forge_config=None) -> AgentLoop:
return AgentLoop(
provider=_StubProvider(),
workspace=tmp_path,
model="stub",
policy=TurnPolicy(max_iterations=2),
tools=ToolWiring(restrict_to_workspace=True),
engine=EngineWiring(
backend=backend,
context_config=ContextConfig(),
memory_config=MemoryConfig(),
skill_forge_router_config=SkillForgeRouterConfig(),
skill_forge_config=skill_forge_config,
),
subagents=SubagentWiring(agents=agents),
)
class TestSubagentRosterReachesTheGate:
"""The join the segment-builder stubs cannot make: a real agent table, read
through the real builder. The roster is collected lazily because the loop
builds its context engine before its ``SubagentManager`` exists, so a
snapshot taken at wiring time would be empty every run.
Push discovery, explicitly: pull is the default and builds no skills
segment at all, so there is no gate there to hand a roster to."""
@staticmethod
def _push(tmp_path: Path, agents) -> AgentLoop:
return _make_loop(tmp_path, agents=agents, skill_forge_config=SkillForgeConfig(discovery="push"))
def test_a_configured_specialist_reaches_the_gate(self, tmp_path: Path) -> None:
agents = SubagentsConfig(
agents=[
{
"name": "Scribe",
"kind": "cli",
"command": "echo hi",
"description": "turns a source document into a deck",
}
]
).agents
agent = self._push(tmp_path, agents)
skills = next(b for b in agent.context_engine._builders if isinstance(b, SkillsSegmentBuilder))
roster = skills._collect_subagent_roster()
assert roster is not None
assert "Scribe" in roster
assert "turns a source document into a deck" in roster
def test_the_generic_agent_is_not_offered_as_an_overlap(self, tmp_path: Path) -> None:
agents = SubagentsConfig(agents=[{"name": "Scribe", "kind": "cli", "command": "echo hi"}]).agents
agent = self._push(tmp_path, agents)
skills = next(b for b in agent.context_engine._builders if isinstance(b, SkillsSegmentBuilder))
roster = skills._collect_subagent_roster() or ""
assert "Scribe" in roster
assert "no capability bias" not in roster
class TestAgentLoopEngineDetection:
def test_uses_default_engine_always_true(self, tmp_path: Path) -> None:
assert _make_loop(tmp_path, backend=_FakeBackend())._uses_default_engine() is True
def test_uses_default_engine_true_without_backend(self, tmp_path: Path) -> None:
assert _make_loop(tmp_path, backend=None)._uses_default_engine() is True
class TestSelectSkillsGating:
async def test_skill_selection_short_circuits_to_none(self, tmp_path: Path) -> None:
agent = _make_loop(tmp_path, backend=_FakeBackend())
assert await agent._select_skills_for_turn("hi", []) is None
# ---------------------------------------------------------------------------
# Metadata-stash path for _collect_injected_skill_ids
# ---------------------------------------------------------------------------
class TestInjectedIdsFromMetadata:
def test_returns_qualified_ids_when_stash_populated(self, tmp_path: Path) -> None:
agent = _make_loop(tmp_path, backend=_FakeBackend())
agent._last_injected_skill_ids = ["local/x", "everos/y"]
ids = agent._collect_injected_skill_ids(None)
assert "local/x" in ids
assert "everos/y" in ids
def test_falls_back_to_legacy_when_stash_none(self, tmp_path: Path) -> None:
agent = _make_loop(tmp_path, backend=None)
agent._last_injected_skill_ids = None
fake_meta = MagicMock(spec_set=["source", "id"])
fake_meta.source = "local"
fake_meta.id = "git-resolver"
ids = agent._collect_injected_skill_ids([fake_meta])
assert "local/git-resolver" in ids
# ---------------------------------------------------------------------------
# SF6: the window every holder reads must be the bound model's, not a copy
# taken when the holder was built.
# ---------------------------------------------------------------------------
class TestTheWindowFollowsTheTurnsBinding:
"""The window is a fact about the model, so every holder has to read the
one the running turn is bound to. Held as a copy taken at construction, the
trimmer and the consolidator would size a 1M session against whatever the
session that built them happened to run on."""
def test_every_holder_reads_the_window_of_the_bound_model(self, tmp_path: Path, monkeypatch) -> None:
from raven.providers import rates
windows = {"stub": 8192, "other-model": 4096}
monkeypatch.setattr(rates, "resolve_context_window", lambda model, **kw: windows.get(model))
agent = _make_loop(tmp_path, backend=None)
curator = _curator_builder(agent.context_engine)
assert agent.context_window_tokens == 8192
assert curator.context_window_tokens == 8192
assert curator.assembler.trimmer.context_window_tokens == 8192
assert agent.memory_consolidator.context_window_tokens == 8192
with use_binding(ModelBinding(_StubProvider(), "other-model")):
assert agent.context_window_tokens == 4096
assert curator.context_window_tokens == 4096
assert curator.assembler.context_window_tokens == 4096
assert curator.assembler.trimmer.context_window_tokens == 4096
assert agent.memory_consolidator.context_window_tokens == 4096
# And back: leaving the turn leaves nothing behind on the holders.
assert agent.context_window_tokens == 8192
assert curator.assembler.trimmer.context_window_tokens == 8192
def test_a_pinned_window_answers_for_every_model(self, tmp_path: Path, monkeypatch) -> None:
"""An explicit ``context_window_tokens`` is a deliberate override -- the
model a session switched to does not get to discard it."""
from raven.providers import rates
monkeypatch.setattr(
rates,
"resolve_context_window",
lambda *a, **k: (_ for _ in ()).throw(AssertionError("must not be asked when pinned")),
)
agent = AgentLoop(
provider=_StubProvider(),
workspace=tmp_path,
model="stub",
policy=TurnPolicy(max_iterations=2),
tools=ToolWiring(restrict_to_workspace=True),
engine=EngineWiring(
context_window_tokens=8192,
context_config=ContextConfig(),
memory_config=MemoryConfig(),
skill_forge_router_config=SkillForgeRouterConfig(),
),
)
curator = _curator_builder(agent.context_engine)
with use_binding(ModelBinding(_StubProvider(), "other-model", agent.default_binding.configured_window)):
assert agent.context_window_tokens == 8192
assert curator.context_window_tokens == 8192
assert curator.assembler.trimmer.context_window_tokens == 8192
assert agent.memory_consolidator.context_window_tokens == 8192
# ---------------------------------------------------------------------------
# Pull discovery (the default): no skills segment, scent + router exposed
# ---------------------------------------------------------------------------
def test_pull_default_drops_the_skills_segment_and_wires_scent(tmp_path):
engine = _build_engine(tmp_path, skill_forge_config=SkillForgeConfig())
assert not any(isinstance(b, SkillsSegmentBuilder) for b in engine._builders)
assert engine._scent is not None
assert engine.skills_router is not None
def test_push_config_restores_the_legacy_pipeline(tmp_path):
engine = _build_engine(tmp_path, skill_forge_config=SkillForgeConfig(discovery="push"))
assert any(isinstance(b, SkillsSegmentBuilder) for b in engine._builders)
assert engine._scent is None
assert engine.skills_router is not None
# ---------------------------------------------------------------------------
# Ownership reaching the identity prompt, and the paths it depends on
# ---------------------------------------------------------------------------
class TestTheEngineTellsItsIdentityWhoOwnsMemory:
"""The turn's half of the flag, held where the engine decides it.
Handing ``IdentitySegmentBuilder`` the flag directly pins how it renders,
not who told it. This builds the engine the way the loop does and reads
the builder it chose, so deleting the argument at the factory's call site
reddens here -- the suite was otherwise green with the request path naming
both memory files again.
"""
@staticmethod
async def _identity_text(engine) -> str:
builder = next(b for b in engine._builders if isinstance(b, IdentitySegmentBuilder))
return (await builder.build(_assembly_ctx())).text
async def test_without_a_backend_the_turn_still_names_the_memory_files(self, tmp_path: Path) -> None:
text = await self._identity_text(_build_engine(tmp_path))
assert f"{tmp_path}/user_memory/profile/user.md" in text
async def test_with_a_backend_the_turn_names_no_memory_file(self, tmp_path: Path) -> None:
text = await self._identity_text(_build_engine(tmp_path, backend=_FakeBackend()))
assert "profile/user.md" not in text
assert "# Memory" in text
def _assembly_ctx() -> AssemblyContext:
return AssemblyContext(
session_key="s",
current_message="hi",
media=None,
channel=None,
chat_id=None,
session_messages=[],
budget=TokenBudget(100_000, 4_000, 2_000, 1_000, 93_000),
)
class TestOwnershipReachesTheIdentityPrompt:
"""The join no stub can make: a real config row, through the real registry,
into the assembled prompt -- and a real tool table deciding whether the
prohibition it carries is one the model can act on."""
OWNS = "owns decks. Do not build the deck yourself."
@pytest.fixture(autouse=True)
def _isolated_switches(self, tmp_path: Path, monkeypatch) -> None:
"""Point the live off-switch read at a file of our own.
Every case here goes through the real tool table, which reads
``tools.disabledTools`` from whatever config file is current -- so without
this they consult the developer's own ``~/.raven/config.json``, and a
locally disabled ``spawn`` makes them red on one machine and green in CI.
"""
self._switches = tmp_path / "config.json"
self._disable()
monkeypatch.setattr("raven.home._current_config_path", self._switches)
def _disable(self, *names: str) -> None:
self._switches.write_text(json.dumps({"tools": {"disabledTools": list(names)}}), encoding="utf-8")
@staticmethod
def _identity(agent: AgentLoop) -> IdentitySegmentBuilder:
return next(b for b in agent.context_engine._builders if isinstance(b, IdentitySegmentBuilder))
async def _text(self, agent: AgentLoop) -> str:
return (await self._identity(agent).build(_assembly_ctx())).text
def _scribe(self, kind: str = "cli") -> list:
row = {"name": "Scribe", "kind": kind, "owns": self.OWNS}
if kind == "cli":
row["command"] = "echo hi"
return SubagentsConfig(agents=[row]).agents
async def test_a_custom_builtin_specialist_gets_its_line(self, tmp_path: Path) -> None:
"""A built-in row is on the same dispatchable table as an external one, so
narrowing one into a specialist has to be declarable the same way."""
assert f"`Scribe` {self.OWNS}" in await self._text(_make_loop(tmp_path, agents=self._scribe("builtin")))
async def test_an_external_specialist_gets_its_line(self, tmp_path: Path) -> None:
"""Regression guard for the path that already worked, not a proof of the
fix: this one is green on either side of it."""
assert f"`Scribe` {self.OWNS}" in await self._text(_make_loop(tmp_path, agents=self._scribe()))
async def test_ownership_on_the_generic_row_claims_nothing(self, tmp_path: Path) -> None:
"""Paired with a real specialist so the absence is selective: asserting
only that the claim is missing would also pass on an install where the
whole section failed to render."""
agents = SubagentsConfig(
agents=[
{"name": GENERIC_AGENT, "kind": "builtin", "owns": "owns everything. Do not do anything yourself."},
{"name": "Scribe", "kind": "cli", "command": "echo hi", "owns": self.OWNS},
]
).agents
text = await self._text(_make_loop(tmp_path, agents=agents))
assert f"`Scribe` {self.OWNS}" in text
assert "owns everything" not in text
async def test_withholding_every_dispatch_path_retires_the_prohibition(self, tmp_path: Path) -> None:
"""Read from the file per turn, so a switch flipped on the settings page
takes the section away and putting it back brings it back -- without which
the prompt forbids work the turn has no way to hand off."""
agent = _make_loop(tmp_path, agents=self._scribe())
offered = {d["function"]["name"] for d in agent.tools.get_definitions()}
assert {"spawn", "run_subagent_dag"} <= offered
assert "## Delegation" in await self._text(agent)
self._disable("spawn", "run_subagent_dag")
assert "## Delegation" not in await self._text(agent)
self._disable()
assert "## Delegation" in await self._text(agent)
async def test_withholding_the_whole_tool_table_retires_it_too(self, tmp_path: Path) -> None:
"""Naming the two dispatch tools is not the only way to reach zero live
paths. Withholding every registered tool reaches it through a successful
but empty lookup, which the gate first shipped reporting as both paths
live -- so the prohibition survived on a turn offering no tools at all.
"""
agent = _make_loop(tmp_path, agents=self._scribe())
every = sorted({d["function"]["name"] for d in agent.tools.get_definitions()})
assert {"spawn", "run_subagent_dag"} <= set(every)
assert "## Delegation" in await self._text(agent)
self._disable(*every)
# Not empty any more: raven reserves the tool-search meta-pair from the
# off switch, so what an operator can withhold is everything else. Zero
# live delegation paths is what retires the section, and that is reached.
from raven.agent.tools.tool_search import META_TOOL_NAMES
assert {d["function"]["name"] for d in agent.tools.get_definitions()} == set(META_TOOL_NAMES)
assert "## Delegation" not in await self._text(agent)
async def test_withholding_one_path_keeps_the_other_named(self, tmp_path: Path) -> None:
self._disable("spawn")
text = await self._text(_make_loop(tmp_path, agents=self._scribe()))
assert f"`Scribe` {self.OWNS}" in text
assert "`run_subagent_dag`" in text
assert "`spawn`" not in text
# ---------------------------------------------------------------------------
# context.dropSegments
# ---------------------------------------------------------------------------
def _engine_with(tmp_path: Path, config: ContextConfig) -> ContextAssembler:
return build_context_engine(
workspace=tmp_path,
config=config,
builder=ContextBuilder(workspace=tmp_path),
provider=_StubProvider(),
model="stub",
context_window_tokens=8192,
get_tool_definitions=_stub_get_defs,
memory_config=MemoryConfig(),
skill_forge_router_config=SkillForgeRouterConfig(hub=HubSourceConfig(endpoint=None)),
skill_forge_config=SkillForgeConfig(discovery="push"),
)
def test_a_product_drops_the_host_segments_it_names(tmp_path: Path) -> None:
full = [b.name for b in _engine_with(tmp_path, ContextConfig())._builders]
assert full[:2] == ["identity", "bootstrap"] and "memory" in full and "curator" in full
slim = [b.name for b in _engine_with(tmp_path, ContextConfig(drop_segments=["identity", "memory"]))._builders]
assert "identity" not in slim and "memory" not in slim
assert slim == [n for n in full if n not in ("identity", "memory")], "order and the rest untouched"
def test_an_unknown_segment_name_drops_nothing(tmp_path: Path) -> None:
full = [b.name for b in _engine_with(tmp_path, ContextConfig())._builders]
assert [b.name for b in _engine_with(tmp_path, ContextConfig(drop_segments=["nope"]))._builders] == full