Raven-1/tests/test_curator_context_engine.py
Dizhan Xue ab003ca298
fix: preserve active config aliases when updating settings (#923)
## Summary

A config containing `skillForge: null` beside a populated `skill_forge`
loaded successfully, but saving a setting could fill the null alias and
leave two active blocks. The next settings read or runtime startup then
failed validation. The same problem affected session-title and
skill-gate role pins.

Use one key-spelling selector for `settings.set`, raw self-configuration
reads and writes, and skill block/unblock. It prefers an existing
non-null value, then preserves an existing null spelling, and creates a
section only when appropriate. Invalid sections such as `false`, `0`,
arrays and strings still reach validation and leave the original file
unchanged on refusal. Both dotted writes and whole-block pin merges
return the actual previous value.

This consolidates the settings writer change from #925 and retains the
block/unblock guards and same-turn Manifest grouping regressions
following #913 and #914. No settings keys or RPC fields are added.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

Native Windows, Python 3.12.13, with an isolated user profile and Raven
home. Imports were checked to resolve from the reviewed checkout. The
real user's configuration was hash-checked before and after each
command.

```text
uv run --frozen --no-sync pytest tests/test_rpc_settings.py tests/test_config_update.py tests/test_config_self_surface.py -k 'non_null_alias or two_null_aliases or invalid_alias_beside_null' -q -n 0 --tb=short
```

35 passed. Before the implementation change, the initial regression
selection had 19 failures and 8 passing controls. Replacing the non-null
predicate with a truthiness check produces 15 failures and 5 passing
controls; the mutation ran in a separate process without modifying
source files.

The broader test selection was:

```text
uv run --frozen --no-sync pytest tests/test_rpc_settings.py tests/test_rpc_console.py tests/test_config_update.py tests/test_config_self_surface.py tests/test_raven_config_tool.py tests/test_rpc_contract_shapes.py tests/test_cli_skill_commands.py tests/test_curator_context_engine.py tests/test_history_trimmer.py -q -n 0 --tb=short
```

The final run additionally emitted JUnit and coverage XML outside the
checkout and measured `--cov=raven.config.keys`: 740 passed and 8
failed; the shared selector has 100% line and branch coverage. The same
selection on unmodified current main yielded 673 passed and the
identical 8 failing test IDs, with zero introduced failures. Those
failures concern native Windows path spelling, state-home isolation,
CRLF output, four POSIX file-mode assertions, and session usage. The
full repository suite was not run locally.

```text
uv run --frozen --no-sync pre-commit run --files raven/config/keys.py raven/config/self_surface.py raven/config/update.py raven/rpc/methods/console.py tests/test_config_self_surface.py tests/test_config_update.py tests/test_rpc_settings.py tests/test_curator_context_engine.py
uv run --frozen --no-sync ty check raven/config/keys.py raven/config/self_surface.py raven/config/update.py raven/rpc/methods/console.py
uv run --frozen --no-sync lint-imports
uv run --frozen --no-sync python scripts/check_source_language.py origin/main
uv run --frozen --no-sync python scripts/check_large_files.py origin/main
git diff --check origin/main
```

Applicable hooks, type checks, all ten import contracts, source-language
and repository-file gates passed.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

The focused regressions pass; the broader native Windows baseline
failures are disclosed above. The writer's code contract is updated. No
user-facing controls require documentation changes.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

When one spelling is null and another holds a value, writes and raw
reads now follow the non-null spelling. Invalid non-object sections
remain invalid, including when a null alias is present. Existing
spelling, unrelated fields, the settings allowlist, validation and
locked atomic writes are preserved. A null base section such as `tools`
can become loadable when the requested setting initializes it. No
migration is required; revert this PR to roll back.

## Related Issues

#913, #914. Supersedes #925.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-13 14:42:42 +08:00

656 lines
26 KiB
Python

"""The curator context engine under AgentLoop: archiving, traces, truncation, and fallback."""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from raven.agent.loop import AgentLoop
from raven.agent.loop.bundles import EngineWiring
from raven.config import ContextConfig
from raven.context_engine import ContextAssembler, TurnContext
from raven.context_engine.segments.curator import CuratorSegmentBuilder
from raven.contracts.assembled import TokenBudget
from raven.providers.base import LLMProvider, LLMResponse, ToolCallRequest
from raven.spine.message import ChatType, Source
from raven.spine.turn import Origin, TurnRequest
class CuratorScriptProvider(LLMProvider):
def __init__(self, *, curator_mode: str = "slow"):
super().__init__(api_key="test")
self.curator_mode = curator_mode
self.curator_calls = 0
self.main_calls = 0
async def chat(
self,
messages,
tools=None,
model=None,
max_tokens=4096,
temperature=0.7,
reasoning_effort=None,
tool_choice=None,
):
tool_names = {tool.get("function", {}).get("name") for tool in (tools or []) if isinstance(tool, dict)}
if "curator_build_context" in tool_names:
self.curator_calls += 1
if self.curator_mode == "fallback":
return LLMResponse(content="I cannot decide.", tool_calls=[])
if self.curator_calls == 1:
return LLMResponse(
content=None,
tool_calls=[
ToolCallRequest(
id="curator_archive_1",
name="curator_archive_messages",
arguments={
"message_ids": [0, 1],
"reason": "old context",
"tags": ["old"],
"summary": "old setup",
},
)
],
finish_reason="length" if self.curator_mode == "truncated" else "tool_calls",
)
return LLMResponse(
content=None,
tool_calls=[
ToolCallRequest(
id="curator_build_1",
name="curator_build_context",
arguments={
"include_message_ids": [0, 1, 2, 3],
"working_state_injection": "Keep project setup and latest request.",
"notes": "test plan",
},
)
],
)
self.main_calls += 1
return LLMResponse(content="main done")
def get_default_model(self) -> str:
return "fake-main"
def _session_messages() -> list[dict]:
return [
{"role": "user", "content": "Initial project rule: preserve exact config.", "timestamp": "2026-05-12T10:00:00"},
{"role": "assistant", "content": "Noted the config preservation rule.", "timestamp": "2026-05-12T10:01:00"},
{"role": "user", "content": "Now design curator context management.", "timestamp": "2026-05-12T11:00:00"},
{
"role": "assistant",
"content": "We should use manifest plus selective retrieval.",
"timestamp": "2026-05-12T11:01:00",
},
]
def _budget() -> TokenBudget:
return TokenBudget(
context_length=4096,
reserved_output=512,
reserved_tools=100,
reserved_system=500,
available_history=2984,
)
def test_agentloop_uses_curator_and_keeps_internal_tools_private(tmp_path: Path):
loop = AgentLoop(
provider=CuratorScriptProvider(),
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
assert loop.context_engine.name == "context_assembler"
assert loop.context_engine.owns_compaction is True
assert isinstance(loop.context_engine, ContextAssembler)
assert not any(name.startswith("curator_") for name in loop.tools.tool_names)
@pytest.mark.asyncio
async def test_curator_slow_path_archives_and_writes_trace(tmp_path: Path):
provider = CuratorScriptProvider(curator_mode="slow")
loop = AgentLoop(
provider=provider,
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
assembled = await loop.context_engine.assemble(
"cli:curator-test",
_session_messages(),
_budget(),
turn=TurnContext(current_message="Please continue the curator design.", channel="cli", chat_id="curator-test"),
)
assert assembled.metadata["path"] == "slow"
assert provider.curator_calls == 2
assert "Curator Working State" in assembled.messages[0]["content"]
trace_path = Path(assembled.metadata["trace_path"])
assert trace_path.exists()
trace_text = trace_path.read_text(encoding="utf-8")
assert "curator_archive_messages" in trace_text
assert "slow_path_accepted" in trace_text
manifest = json.loads((tmp_path / "memory/.curator/manifest/cli_curator-test.json").read_text(encoding="utf-8"))
archived = [item for item in manifest["items"] if item["archived"]]
assert [item["id"] for item in archived] == [0, 1]
assert list((tmp_path / "memory/.curator/archive").glob("**/*.jsonl"))
@pytest.mark.asyncio
async def test_curator_does_not_dispatch_a_truncated_call(tmp_path: Path):
"""The curator's loop is a second agent loop, and archiving is destructive.
A cut-off `curator_archive_messages` has an incomplete `message_ids` list
and nothing about it says so -- the arguments that did arrive validate.
`chat_with_retry` reaches the same verdict here as it does for the main
loop; the dispatch has to honour it.
"""
provider = CuratorScriptProvider(curator_mode="truncated")
loop = AgentLoop(
provider=provider,
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
assembled = await loop.context_engine.assemble(
"cli:curator-cut",
_session_messages(),
_budget(),
turn=TurnContext(current_message="Please continue the curator design.", channel="cli", chat_id="curator-cut"),
)
assert assembled.metadata["path"] == "slow"
manifest = json.loads((tmp_path / "memory/.curator/manifest/cli_curator-cut.json").read_text(encoding="utf-8"))
assert [item for item in manifest["items"] if item["archived"]] == []
trace_text = Path(assembled.metadata["trace_path"]).read_text(encoding="utf-8")
assert "[truncated]" in trace_text
@pytest.mark.asyncio
async def test_curator_fallback_when_internal_agent_does_not_finish(tmp_path: Path):
provider = CuratorScriptProvider(curator_mode="fallback")
loop = AgentLoop(
provider=provider,
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
assembled = await loop.context_engine.assemble(
"cli:fallback-test",
_session_messages(),
_budget(),
turn=TurnContext(current_message="Continue.", channel="cli", chat_id="fallback-test"),
)
assert assembled.metadata["path"] == "fallback"
assert assembled.messages[0]["role"] == "system"
assert assembled.messages[-1]["role"] == "user"
assert "Continue." in assembled.messages[-1]["content"]
@pytest.mark.asyncio
async def test_process_message_records_main_and_curator_trajectories(tmp_path: Path):
provider = CuratorScriptProvider(curator_mode="slow")
loop = AgentLoop(
provider=provider,
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
session = loop.sessions.get_or_create("cli:trace-test")
session.messages.extend(_session_messages())
loop.sessions.save(session)
response = await loop._process_message(
TurnRequest(
origin=Origin.USER,
source=Source(
channel="cli",
chat_id="trace-test",
sender_id="user",
chat_type=ChatType.DM,
),
text="Use the curator trajectory and answer.",
)
)
assert response is not None
assert response[0] == "main done"
traces = list((tmp_path / "memory/.curator/traces/cli_trace-test").glob("*.jsonl"))
assert len(traces) == 1
trace_text = traces[0].read_text(encoding="utf-8")
assert "curator_llm_request" in trace_text
assert "main_agent_result" in trace_text
def test_history_from_messages_preserves_reasoning_fields():
messages = [
{"role": "user", "content": "hi"},
{
"role": "assistant",
"content": "answer",
"reasoning_content": "chain of thought",
"thinking_blocks": [{"thinking": "block"}],
},
]
history = CuratorSegmentBuilder._history_from_messages(messages)
assert history[1]["reasoning_content"] == "chain of thought"
assert history[1]["thinking_blocks"] == [{"thinking": "block"}]
# --- pinned skill bodies -------------------------------------------------
_GUIDE_MARKER = "PINNED-GUIDE-BODY-MARKER"
def _use_skill_exchange(call_id: str, skill_id: str, body: str) -> list[dict]:
"""The two messages a `use_skill` fetch leaves in the session log."""
return [
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": call_id,
"type": "function",
"function": {"name": "use_skill", "arguments": json.dumps({"skill_id": skill_id})},
}
],
},
{"role": "tool", "tool_call_id": call_id, "name": "use_skill", "content": body},
]
def _session_with_guide_fetch() -> list[dict]:
return [
{"role": "user", "content": "Set up the project."},
{"role": "assistant", "content": "Done."},
{"role": "user", "content": "Now orchestrate three sub-agents."},
*_use_skill_exchange("call_guide", "local/subagent-dag-orchestration", _GUIDE_MARKER),
{"role": "assistant", "content": "Read the guide."},
]
class _PlanOmittingPinsProvider(CuratorScriptProvider):
"""A curator that builds a plan naming only the first exchange — the shape
that silently loses a mid-session skill body."""
async def chat(self, messages, tools=None, **kwargs):
tool_names = {tool.get("function", {}).get("name") for tool in (tools or []) if isinstance(tool, dict)}
if "curator_build_context" in tool_names:
self.curator_calls += 1
return LLMResponse(
content=None,
tool_calls=[
ToolCallRequest(
id="build_1",
name="curator_build_context",
arguments={"include_message_ids": [0, 1], "notes": "recent only"},
)
],
)
self.main_calls += 1
return LLMResponse(content="main done")
def test_pinned_scan_takes_the_whole_exchange_and_supersedes_earlier_fetches() -> None:
"""Half a tool exchange is a dangling tool_call, and a re-read must move the
pin rather than add a second copy of the same body."""
from raven.context_engine.curator import pinned_message_ids
messages = [
{"role": "user", "content": "go"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "c1",
"type": "function",
"function": {"name": "use_skill", "arguments": '{"skill_id": "local/guide"}'},
},
{"id": "c2", "type": "function", "function": {"name": "grep", "arguments": "{}"}},
],
},
{"role": "tool", "tool_call_id": "c1", "name": "use_skill", "content": "body"},
{"role": "tool", "tool_call_id": "c2", "name": "grep", "content": "hits"},
{"role": "user", "content": "again"},
*_use_skill_exchange("c3", "local/guide", "body v2"),
]
pinned = pinned_message_ids(messages, ["local/guide"])
# Only the latest fetch (5, 6) — and the sibling grep result of the earlier
# turn is not pinned because that whole turn was superseded.
assert pinned == {5, 6}
def test_pinned_scan_covers_sibling_results_of_the_pinned_turn(tmp_path: Path) -> None:
from raven.context_engine.curator import CuratorArchiveStore, pinned_message_ids
messages = [
{"role": "user", "content": "go"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "c1",
"type": "function",
"function": {"name": "use_skill", "arguments": '{"skill_id": "local/guide"}'},
},
{"id": "c2", "type": "function", "function": {"name": "grep", "arguments": "{}"}},
],
},
{"role": "tool", "tool_call_id": "c1", "name": "use_skill", "content": "body"},
{"role": "tool", "tool_call_id": "c2", "name": "grep", "content": "hits"},
]
assert pinned_message_ids(messages, ["local/guide"]) == {1, 2, 3}
store = CuratorArchiveStore(tmp_path, ContextConfig(pinned_skill_ids=["local/guide"]))
manifest = store.build_manifest("cli:pin-siblings", messages)
assert manifest[0].group_id is None
assert manifest[1].group_id is not None
assert manifest[1].group_id == manifest[2].group_id == manifest[3].group_id
@pytest.mark.parametrize("later_id", ["first", "different"])
@pytest.mark.parametrize("reread", [False, True])
def test_pinned_scan_pairs_reused_ids_by_position(tmp_path, later_id, reread):
from raven.context_engine.curator import CuratorArchiveStore, pinned_message_ids
from raven.context_engine.history_trimmer import HistoryTrimmer
messages = [
{"role": "user", "content": "read the guide"},
*_use_skill_exchange("first", "local/guide", "original guide"),
{"role": "user", "content": "continue"},
*_use_skill_exchange(later_id, "local/guide" if reread else "local/unrelated", "later body"),
]
expected = {4, 5} if reread else {1, 2}
pinned = pinned_message_ids(messages, ["local/guide"])
assert pinned == expected
assert HistoryTrimmer.canonical_ids(messages, [0, *pinned]) == [0, *sorted(expected)]
store = CuratorArchiveStore(tmp_path, ContextConfig(pinned_skill_ids=["local/guide"]))
manifest = store.build_manifest("cli:pin-reused", messages)
assert {item.id for item in manifest if item.pinned} == expected
assert manifest[1].group_id == manifest[2].group_id
assert manifest[4].group_id == manifest[5].group_id
assert manifest[1].group_id != manifest[4].group_id
saved = json.loads(store.manifest_path("cli:pin-reused").read_text(encoding="utf-8"))
assert [item["group_id"] for item in saved["items"]] == [item.group_id for item in manifest]
@pytest.mark.parametrize("later_id", ["first", "different"])
def test_manifest_keeps_two_exchanges_of_one_turn_apart(tmp_path: Path, later_id: str) -> None:
from raven.context_engine.curator import CuratorArchiveStore
messages = [
{"role": "user", "content": "read the guide, then continue"},
*_use_skill_exchange("first", "local/guide", "guide body"),
*_use_skill_exchange(later_id, "local/unrelated", "other body"),
]
store = CuratorArchiveStore(tmp_path, ContextConfig(pinned_skill_ids=["local/guide"]))
manifest = store.build_manifest("cli:one-turn", messages)
assert manifest[1].turn_id == manifest[3].turn_id
assert manifest[1].group_id is not None
assert manifest[3].group_id is not None
assert manifest[1].group_id == manifest[2].group_id
assert manifest[3].group_id == manifest[4].group_id
assert manifest[1].group_id != manifest[3].group_id
assert {item.id for item in manifest if item.pinned} == {1, 2}
def test_manifest_leaves_unpaired_messages_without_a_tool_group(tmp_path: Path) -> None:
from raven.context_engine.curator import CuratorArchiveStore
messages = [
{"role": "tool", "tool_call_id": "unknown", "content": "orphan"},
{"role": "assistant", "tool_calls": [{"id": "pending"}]},
{"role": "assistant", "tool_calls": ["malformed", {}]},
{"role": "tool", "content": "no call id"},
]
store = CuratorArchiveStore(tmp_path, ContextConfig())
assert all(item.group_id is None for item in store.build_manifest("cli:unpaired", messages))
@pytest.mark.parametrize(
"tool_calls",
[
[{"id": "c", "type": "function", "function": {"name": "use_skill", "arguments": "{not json"}}],
[
{
"id": "c",
"type": "function",
"function": {"name": "use_skill", "arguments": '{"skill_id": "local/other"}'},
}
],
[
{
"id": "c",
"type": "function",
"function": {"name": "web_fetch", "arguments": '{"skill_id": "local/guide"}'},
}
],
["not-a-dict"],
],
)
def test_pinned_scan_pins_nothing_it_cannot_positively_identify(tool_calls) -> None:
"""Manifest building must never raise on a malformed stored tool_call, and a
fetch of some other skill is not this skill's pin."""
from raven.context_engine.curator import pinned_message_ids
messages = [
{"role": "user", "content": "go"},
{"role": "assistant", "content": None, "tool_calls": tool_calls},
{"role": "tool", "tool_call_id": "c", "name": "use_skill", "content": "body"},
]
assert pinned_message_ids(messages, ["local/guide"]) == set()
def test_pinned_scan_is_off_when_no_ids_are_configured() -> None:
from raven.context_engine.curator import pinned_message_ids
messages = _session_with_guide_fetch()
assert pinned_message_ids(messages, []) == set()
assert pinned_message_ids(messages, None) == set()
@pytest.mark.asyncio
async def test_pinned_guide_body_survives_a_plan_that_omitted_it(tmp_path: Path):
"""The point of the pin: the curator's plan names only ids 0-1, yet the
fetched body still reaches the main agent."""
loop = AgentLoop(
provider=_PlanOmittingPinsProvider(),
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
assembled = await loop.context_engine.assemble(
"cli:pin-test",
_session_with_guide_fetch(),
_budget(),
turn=TurnContext(current_message="Build the DAG now.", channel="cli", chat_id="pin-test"),
)
assert assembled.metadata["path"] == "slow"
rendered = json.dumps(assembled.messages, ensure_ascii=False)
assert _GUIDE_MARKER in rendered
# And the pin is what did it: the same plan drops the body with pinning off.
off = AgentLoop(
provider=_PlanOmittingPinsProvider(),
workspace=tmp_path / "off",
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0, pinned_skill_ids=[])),
)
assembled_off = await off.context_engine.assemble(
"cli:pin-test",
_session_with_guide_fetch(),
_budget(),
turn=TurnContext(current_message="Build the DAG now.", channel="cli", chat_id="pin-test"),
)
assert _GUIDE_MARKER not in json.dumps(assembled_off.messages, ensure_ascii=False)
@pytest.mark.asyncio
async def test_pinned_ids_are_recorded_in_the_turn_trace(tmp_path: Path):
"""The trace is the only way to explain after the fact why an id stayed."""
loop = AgentLoop(
provider=_PlanOmittingPinsProvider(),
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
assembled = await loop.context_engine.assemble(
"cli:pin-trace",
_session_with_guide_fetch(),
_budget(),
turn=TurnContext(current_message="Build the DAG now.", channel="cli", chat_id="pin-trace"),
)
trace = Path(assembled.metadata["trace_path"]).read_text(encoding="utf-8")
assert '"pinned_message_ids": [3, 4]' in trace
def test_archiving_a_pinned_id_is_refused(tmp_path: Path) -> None:
"""Archiving marks a message archived and halves its relevance, which is the
slow path's cue to stop selecting it — a pin has to survive that too."""
from raven.context_engine.curator import CuratorArchiveStore
config = ContextConfig()
store = CuratorArchiveStore(tmp_path, config)
messages = _session_with_guide_fetch()
manifest = store.build_manifest("cli:pin-archive", messages)
assert [item.id for item in manifest if item.pinned] == [3, 4]
result = store.archive_messages("cli:pin-archive", manifest, messages, [3, 4, 5], reason="old")
assert result["refused_pinned_ids"] == [3, 4]
assert result["archived_message_ids"] == [5]
assert [item.id for item in manifest if item.archived] == [5]
def test_pinned_items_carry_a_relevance_floor(tmp_path: Path) -> None:
"""So the slow path's own ranking never argues against the pin."""
from raven.context_engine.curator import CuratorArchiveStore
store = CuratorArchiveStore(tmp_path, ContextConfig())
manifest = store.build_manifest("cli:pin-rel", _session_with_guide_fetch())
assert all(item.relevance >= 0.9 for item in manifest if item.pinned)
def test_the_default_pinned_skill_is_the_id_the_dag_tool_advertises() -> None:
"""The config default and the tool's pointer are the same string in two
layers; this is the seam that catches them drifting apart."""
from raven.agent.subagent.dag_tool import GUIDE_SKILL_ID
assert ContextConfig().pinned_skill_ids == [GUIDE_SKILL_ID]
@pytest.mark.asyncio
async def test_pinned_ids_are_also_protected_from_budget_trimming(tmp_path: Path, monkeypatch):
"""Inclusion alone is not enough: the trimmer drops the lowest-priority
non-protected ids until the prompt fits, so a pinned id that were merely
included would still be the first thing to go under pressure."""
from raven.context_engine.history_trimmer import HistoryTrimmer
seen: dict[str, object] = {}
original = HistoryTrimmer.trim
def _spy(self, **kwargs):
seen.update(kwargs)
return original(self, **kwargs)
monkeypatch.setattr(HistoryTrimmer, "trim", _spy)
loop = AgentLoop(
provider=_PlanOmittingPinsProvider(),
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
await loop.context_engine.assemble(
"cli:pin-protect",
_session_with_guide_fetch(),
_budget(),
turn=TurnContext(current_message="Build the DAG now.", channel="cli", chat_id="pin-protect"),
)
assert {3, 4} <= seen["protected_ids"]
assert {3, 4} <= set(seen["ids"])
def _parallel_tool_session() -> list[dict]:
"""The shape that failed on 2026-09-11: one assistant, three parallel calls."""
return [
{"role": "user", "content": "Open the task and read the notes.", "timestamp": "2026-09-11T11:46:00"},
{
"role": "assistant",
"content": "Reading.",
"timestamp": "2026-09-11T11:46:10",
"tool_calls": [
{"id": f"call_{i}", "type": "function", "function": {"name": "read_skill", "arguments": "{}"}}
for i in range(3)
],
},
*[
{
"role": "tool",
"tool_call_id": f"call_{i}",
"content": "notes " * 1200,
"timestamp": "2026-09-11T11:46:20",
}
for i in range(3)
],
{"role": "assistant", "content": "Here is the plan.", "timestamp": "2026-09-11T11:47:00"},
{"role": "user", "content": "Go on.", "timestamp": "2026-09-11T11:48:00"},
]
@pytest.mark.asyncio
async def test_curator_fallback_never_ships_an_orphan_tool_result(tmp_path: Path):
"""The fallback path trims by budget and used to skip the pairing check, so a
plan that fit the budget could still carry a tool result whose assistant had
been dropped -- and the trace said ``ok: True`` while the backend refused it."""
provider = CuratorScriptProvider(curator_mode="fallback")
loop = AgentLoop(
provider=provider,
workspace=tmp_path,
engine=EngineWiring(context_config=ContextConfig(fast_path_threshold=0.0)),
)
# Room for one or two of the three results, not all: the budget forces the
# drop loop to stop inside the tool group.
assembled = await loop.context_engine.assemble(
"cli:fallback-pairing",
_parallel_tool_session(),
_budget(),
turn=TurnContext(current_message="Go on.", channel="cli", chat_id="fallback-pairing"),
)
assert assembled.metadata["path"] == "fallback"
calls = {tc["id"] for m in assembled.messages if m.get("role") == "assistant" for tc in (m.get("tool_calls") or [])}
results = {m["tool_call_id"] for m in assembled.messages if m.get("role") == "tool"}
assert results <= calls, f"orphan tool results reached the provider: {results - calls}"
assert calls <= results, f"tool calls without results reached the provider: {calls - results}"
# The verdict the fallback path records is the same one a candidate gets.
trace = [
json.loads(line)
for line in Path(assembled.metadata["trace_path"]).read_text(encoding="utf-8").splitlines()
if line.strip()
]
fallback = next(entry for entry in trace if entry.get("event") == "fallback")
validation = fallback["payload"]["validation"]
assert validation["errors"] == []
assert validation["ok"] is True