Raven-1/tests/test_trajectory_bundle.py
江国庆/Forrest 6b938a4267
feat(*): export conversations and report problems from the web page (#948)
## Summary

The web page gains two things a conversation could only get from the
terminal before: exporting the selected turns as a download, and
reporting a problem through the bug report pipeline. The trajectory view
is now on for everyone.

- **Trajectory view without `--dev` (R0).** `raven`, `raven web`, `raven
serve` and `raven gateway` no longer take `--dev`; the trajectory policy
starts enabled. `raven tui --dev` (running the TUI from source) is
unchanged.
- **Turn selection.** From this version each turn's opening user message
records its trace id (the Turn Trace Mark, never sent to the model).
`raven/trajectory/selection.py` lists a session's main-line turns,
closes a selection over the sub-agent and delegated traces it
dispatched, collects their spans, and cuts the session record down to
the selected turns; older records are matched conservatively and an
unmatched turn stays in the trajectory only.
- **Export.** The Trajectory Export package v1 (`schema:
raven_trajectory_export`, `format_version: 1`) is the input contract of
the coming `web-traj-import`; it is defined in `CONTEXT.md` and pinned
by tests. It holds the raw spans, artifacts and v2 message blobs of the
selected traces plus the cut session record, unredacted. Exports run as
background jobs (at most two) and the browser downloads them from the
same-origin `/trajectory/export/download` route with a one-time token.
New RPCs: `trajectory.turns`, `trajectory.export`,
`trajectory.export_status`, `trajectory.export_cancel`.
- **Bug reports.** `freeze_export` is split into staging and finishing
so a report can be prepared from selected turns; the CLI's behaviour is
unchanged. A Report Session holds a web report in the gateway between
page requests: at most two at once, all blocking I/O on worker threads,
a cancel checked after every step (a report cancelled while packaging is
never sent), a failed or uploaded report releases its slot and keeps its
last status for ten minutes, and an idle one is cancelled after thirty
minutes. Seven `bugreport.*` RPCs, the `bugreport-v1` capability
(announced only while the receiver URL is valid), and error codes -32026
/ -32027 / -32028. Every decision made on the web page, defaults and
bulk buttons included, is remembered in the redaction history with `web`
as its source, unlike the CLI's bulk flags.
- **Web page.** A three-dot conversation menu in the header's corner
(the desk launcher moves one slot in) opens the new `export` wizard.
Export: pick turns, name the file under an "unredacted" warning, follow
the job, download once automatically with a link to download again.
Report a problem: pick turns, describe the problem, decide the uncertain
values in two columns (hits highlighted at code points), upload, and get
a receipt. Closing before the upload starts discards the report being
prepared; once the upload starts the report is saved on the machine, a
failure marked retryable offers a retry, and the saved report can be
sent later from the command line by its id. Every answer is checked
against the wizard's epoch and the job or report it was asked about, and
a write step whose answer is lost is settled from the report's status
instead of being sent again. Layouts were measured at 360 and 390 px in
English and Chinese.
- **Docs and tests.** The webui and trajectory-debugging pages (English
and Chinese) describe both flows. Two integration tests drive them end
to end: the export through the real download route, and the bug report
against a real receiver process, including a stopped receiver, a retry,
the redaction history's replay and a cancel at each step.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- The branch merges the current `main` (one merge commit, no conflicts);
everything below ran on the merged tree.
- `uv run pytest tests/ -q` (full suite): 57 failed / 31491 passed / 177
skipped. Every one of the 57 also fails on the current `main` on this
machine (environment-specific, pre-existing); none is introduced by this
branch.
- `uv run pytest -m integration
tests/integration/test_web_traj_export_e2e.py
tests/integration/test_web_bugreport_e2e.py
tests/integration/test_bug_report_upload_e2e.py -q`: 10 passed (three
consecutive runs of the two new files).
- `uv run ruff check`, `uv run ruff format --check`, `uv run ty check`
on the changed Python: pass.
- ui-web: `npm run gen:check` (239 methods), `npm run lint`, `npm run
type-check`, `npx vitest run` (246 files / 3868 passed), `npm run build`
and `python3 ui-web/build.py` (both boot snapshots match),
`check-page.mjs`, `check-css.mjs`, `check-class-namespace.mjs`: pass.
- ui-tui: `npm run lint:rpc`, `npm run lint:i18n`: pass.
- `uv run --group docs mkdocs build -f docs-site/mkdocs.yml --strict`:
pass.
- `make check-source-language check-large-files check-commits`: pass.
- Headless Chromium on the built page (offline fixtures): both flows
walked to the download and to the receipt; the header corner measured in
five layouts (desk panel open and closed, workspace panel open, shell
mode, narrow split) and the wizard at 360 and 390 px in both languages.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- A process started with the old `--dev` flag that runs `system.upgrade`
relaunches itself with `--dev`, which the new version refuses; the user
runs `raven web` once by hand. This was accepted as the cost of dropping
the flag.
- The trajectory view is on by default: each open page calls the light,
in-memory `trajectory.state` every five seconds, and the index is built
only when the view is opened. No switch to turn it off is added here;
the policy keeps a replaceable source for a settings entry later.
- The export is the raw trajectory and may contain secrets. The page
says so in a warning; the download is served only on the same origin,
behind the session guard, by an unguessable one-time token, and no path
ever reaches the page.
- Rollback: revert the squash commit. Nothing is migrated; the
`trace_id` field already written into session records is ignored by
older versions.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-10-14 21:34:46 +08:00

592 lines
24 KiB
Python

"""Tests for the trajectory bundle collector (`raven.trajectory.bundle`)."""
from __future__ import annotations
import json
import shutil
import pytest
from raven.session.manager import SessionManager
from raven.trajectory import bundle as tbundle
from raven.trajectory import store as tstore
from raven.trajectory import verdict as tverdict
def _write_log(path, spans):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("".join(json.dumps(s) + "\n" for s in spans), encoding="utf-8")
def _span(trace_id, *, attempt_id=None, session_key=None, name="session.turn", start=None, end=None, attrs=None):
"""Span in the current format; pass ``attempt_id`` for a legacy-format span."""
attributes = {"session.key": session_key}
if attempt_id is not None:
attributes["attempt.id"] = attempt_id
if attrs:
attributes.update(attrs)
return {
"schemaVersion": "audit.span.v1",
"traceId": trace_id,
"spanId": f"span-{trace_id}-{name}",
"name": name,
"startTime": start or "2026-08-20T10:00:00+00:00",
"endTime": end or "2026-08-20T10:00:01+00:00",
"attributes": attributes,
}
@pytest.fixture
def state(tmp_path):
return tmp_path / "traces"
@pytest.fixture
def workspace(tmp_path):
ws = tmp_path / "ws"
ws.mkdir()
return ws
def _make_artifact(state, name, content):
path = state / "logs" / "audit-artifacts" / "tool.output" / "2026-08-20" / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
return path
def _read_lines(path):
return [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines() if line.strip()]
def test_bundle_roundtrip(state, workspace):
artifact = _make_artifact(state, "output.json", '{"result": 42}')
manager = SessionManager(workspace)
session = manager.get_or_create("cli:chat1")
session.add_message("user", "hello")
manager.save(session)
_write_log(
state / "logs" / "audit-spans.log",
[
_span("trace-1", session_key="cli:chat1", start="2026-08-20T10:00:00+00:00"),
_span(
"trace-1",
session_key="cli:chat1",
name="tool.call",
end="2026-08-20T10:00:05+00:00",
attrs={"tool.output.artifact_path": str(artifact)},
),
],
)
tverdict.record_verdict("trace-1", "fail", source="user", why="wrong answer", state_dir=state)
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
assert bundle_dir == (state / "bundles" / "trace-1").resolve()
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["format_version"] == tbundle.BUNDLE_FORMAT_VERSION
assert manifest["attempt_id"] == "trace-1"
assert manifest["session_key"] == "cli:chat1"
assert manifest["time_range"] == {"start": "2026-08-20T10:00:00+00:00", "end": "2026-08-20T10:00:05+00:00"}
assert manifest["span_count"] == 2
assert manifest["artifact_count"] == 1
assert manifest["verdict_count"] == 1
assert manifest["session_included"] is True
assert manifest["missing_artifacts"] == []
assert manifest["raven_version"]
spans = _read_lines(bundle_dir / "spans.jsonl")
assert [s["name"] for s in spans] == ["session.turn", "tool.call"]
verdicts = _read_lines(bundle_dir / "verdicts.jsonl")
assert verdicts[0]["status"] == "fail" and verdicts[0]["why"] == "wrong answer"
source_session = manager.session_path("cli:chat1")
assert (bundle_dir / "session.jsonl").read_text(encoding="utf-8") == source_session.read_text(encoding="utf-8")
def test_artifact_paths_rewritten_offline(state, workspace, tmp_path):
artifact = _make_artifact(state, "big-output.json", '{"tokens": [1, 2, 3]}')
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", name="tool.call", attrs={"tool.output.artifact_path": str(artifact)})],
)
bundle_dir = tbundle.collect_bundle("trace-1", out_dir=tmp_path / "out", state_dir=state, workspace=workspace)
span = _read_lines(bundle_dir / "spans.jsonl")[0]
rel = span["attributes"]["tool.output.artifact_path"]
assert rel == "artifacts/big-output.json"
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["rewritten_artifact_paths"] == 1
# The bundle must read on its own after the live store is gone.
moved = tmp_path / "elsewhere"
shutil.move(str(bundle_dir), str(moved))
shutil.rmtree(state)
assert (moved / rel).read_text(encoding="utf-8") == '{"tokens": [1, 2, 3]}'
def test_missing_artifact_not_fatal(state, workspace):
gone = str(state / "logs" / "audit-artifacts" / "purged.json")
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", name="tool.call", attrs={"tool.output.artifact_path": gone})],
)
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["missing_artifacts"] == [gone]
assert manifest["artifact_count"] == 0
span = _read_lines(bundle_dir / "spans.jsonl")[0]
assert span["attributes"]["tool.output.artifact_path"] == gone
def test_artifact_basename_collision_disambiguated(state, workspace):
a = _make_artifact(state, "output.json", "first")
b = state / "other" / "output.json"
b.parent.mkdir(parents=True)
b.write_text("second", encoding="utf-8")
_write_log(
state / "logs" / "audit-spans.log",
[
_span("trace-1", name="tool.call", attrs={"tool.output.artifact_path": str(a)}),
_span("trace-1", name="llm.call", attrs={"llm.output.artifact_path": str(b)}),
],
)
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
spans = _read_lines(bundle_dir / "spans.jsonl")
rel_a = spans[0]["attributes"]["tool.output.artifact_path"]
rel_b = spans[1]["attributes"]["llm.output.artifact_path"]
assert rel_a != rel_b
assert (bundle_dir / rel_a).read_text(encoding="utf-8") == "first"
assert (bundle_dir / rel_b).read_text(encoding="utf-8") == "second"
def test_bundle_spans_rotation(state, workspace):
archive = state / "logs" / "archive"
_write_log(
archive / "2026-08-18" / "audit-spans-2026-08-18-010101.log",
[_span("trace-1", attempt_id="att-x", start="2026-08-18T09:00:00+00:00")],
)
_write_log(
state / "logs" / "audit-spans.log",
[
_span("trace-2", attempt_id="att-x", end="2026-08-20T11:00:00+00:00"),
_span("trace-9"),
],
)
bundle_dir = tbundle.collect_bundle("att-x", state_dir=state, workspace=workspace)
spans = _read_lines(bundle_dir / "spans.jsonl")
assert [s["traceId"] for s in spans] == ["trace-1", "trace-2"]
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["time_range"] == {"start": "2026-08-18T09:00:00+00:00", "end": "2026-08-20T11:00:00+00:00"}
def test_save_auto_pins(state, workspace):
_write_log(state / "logs" / "audit-spans.log", [_span("trace-1")])
tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
registry = tstore.pins(state)
assert registry["trace-1"]["reason"] == "bundled"
def test_missing_session_recorded(state, workspace):
_write_log(state / "logs" / "audit-spans.log", [_span("trace-1", session_key="cli:never-saved")])
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["session_included"] is False
assert not (bundle_dir / "session.jsonl").exists()
def test_unknown_id_raises(state, workspace):
_write_log(state / "logs" / "audit-spans.log", [_span("trace-1")])
with pytest.raises(LookupError):
tbundle.collect_bundle("no-such-id", state_dir=state, workspace=workspace)
with pytest.raises(ValueError):
tbundle.collect_bundle("", state_dir=state, workspace=workspace)
def test_trace_id_resolves_to_full_attempt(state, workspace):
"""One turn's trace id must pack the whole multi-turn attempt it belongs to."""
_write_log(
state / "logs" / "audit-spans.log",
[
_span("trace-1", attempt_id="att-x", session_key="cli:a"),
_span("trace-2", attempt_id="att-x", session_key="cli:a", name="tool.call"),
],
)
tverdict.record_verdict("att-x", "fail", source="user", state_dir=state)
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
assert bundle_dir.name == "att-x"
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["attempt_id"] == "att-x"
assert manifest["span_count"] == 2
assert manifest["verdict_count"] == 1
assert [s["traceId"] for s in _read_lines(bundle_dir / "spans.jsonl")] == ["trace-1", "trace-2"]
registry = tstore.pins(state)
assert "att-x" in registry and "trace-1" not in registry
def test_unsafe_id_rejected(state, workspace, tmp_path):
"""An attempt id (user-mintable) must not escape the output root as a path."""
for evil in ("../escaped", str(tmp_path / "abs-escaped"), "a/b"):
_write_log(state / "logs" / "audit-spans.log", [_span("trace-1", attempt_id=evil)])
with pytest.raises(ValueError):
tbundle.collect_bundle(evil, state_dir=state, workspace=workspace)
assert not (tmp_path / "escaped").exists()
assert not (tmp_path / "abs-escaped").exists()
assert not (state / "escaped").exists()
def test_resave_drops_stale_artifacts(state, workspace):
"""A re-pack must not keep artifacts whose source was purged since last pack."""
artifact = _make_artifact(state, "secret.json", '{"api_key": "sk-live"}')
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", name="tool.call", attrs={"tool.output.artifact_path": str(artifact)})],
)
first = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
assert (first / "artifacts" / "secret.json").exists()
artifact.unlink()
second = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
assert not (second / "artifacts" / "secret.json").exists()
manifest = json.loads((second / "manifest.json").read_text(encoding="utf-8"))
assert manifest["artifact_count"] == 0
assert manifest["missing_artifacts"] == [str(artifact)]
leftovers = [p.name for p in second.parent.iterdir() if p.name.startswith(".")]
assert leftovers == []
def test_default_workspace_reads_config(state, tmp_path, monkeypatch):
"""Without an explicit workspace, the configured agent workspace is used."""
import types
ws = tmp_path / "custom-ws"
ws.mkdir()
manager = SessionManager(ws)
session = manager.get_or_create("cli:chat1")
session.add_message("user", "hello")
manager.save(session)
monkeypatch.setattr(
"raven.config.loader.load_config",
lambda config_path=None: types.SimpleNamespace(workspace_path=ws),
)
_write_log(state / "logs" / "audit-spans.log", [_span("trace-1", session_key="cli:chat1")])
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state)
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["session_included"] is True
assert (bundle_dir / "session.jsonl").exists()
def test_end_to_end_with_real_tracer(tmp_path, monkeypatch):
"""A trajectory recorded by the live tracer bundles into an offline dir."""
from raven.tracing import spans as _spans
from raven.tracing import trace
state = tmp_path / "traces"
workspace = tmp_path / "ws"
workspace.mkdir()
monkeypatch.setenv("RAVEN_TRACING", "1")
monkeypatch.setenv("RAVEN_TRACING_DIR", str(state))
_spans._store = None
try:
# A single turn is addressed by its trace id (attempt id = trace id).
with trace.span("session.turn", session_key="cli:e2e") as root:
with trace.span("tool.call") as s:
s.artifact("tool.output", {"result": "ok", "items": list(range(50))})
aid = root.trace_id
bundle_dir = tbundle.collect_bundle(aid, state_dir=state, workspace=workspace)
spans = _read_lines(bundle_dir / "spans.jsonl")
tool_span = next(s for s in spans if s["name"] == "tool.call")
rel = tool_span["attributes"]["tool.output.artifact_path"]
assert rel.startswith("artifacts/")
payload = json.loads((bundle_dir / rel).read_text(encoding="utf-8"))
assert payload["result"] == "ok" and len(payload["items"]) == 50
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["attempt_id"] == aid
assert manifest["artifact_count"] == 1
assert tstore.pins(state)[aid]["reason"] == "bundled"
finally:
_spans._store = None
def _v2_artifact(state, messages, name="llm-input.json"):
"""A v2 shell on disk with its message blobs, as the writer leaves it."""
from raven.tracing import artifact_v2 as v2
artifacts = state / "logs" / "audit-artifacts"
refs = []
for message in messages:
sha1 = v2.message_sha1(message)
blob = v2.message_path(artifacts, sha1)
blob.parent.mkdir(parents=True, exist_ok=True)
blob.write_text(v2.message_text(message), encoding="utf-8")
refs.append(v2.make_ref(sha1))
shell = {
"artifactFormat": v2.ARTIFACT_FORMAT,
"model": "openrouter/x",
"systemPrompt": refs[0],
"prompt": refs[-1],
"messages": refs,
"tools": None,
}
path = artifacts / "llm.input" / "2026-08-20" / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(shell, ensure_ascii=False, indent=2), encoding="utf-8")
return path
_V2_MESSAGES = [
{"role": "system", "content": "sys"},
{"role": "user", "content": "hi"},
]
def test_a_bundled_v2_artifact_is_written_out_resolved(state, workspace, tmp_path):
source = _v2_artifact(state, _V2_MESSAGES)
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", name="llm.call", attrs={"llm.input.artifact_path": str(source)})],
)
bundle_dir = tbundle.collect_bundle("trace-1", out_dir=tmp_path / "out", state_dir=state, workspace=workspace)
packed = json.loads((bundle_dir / "artifacts" / source.name).read_text(encoding="utf-8"))
assert "artifactFormat" not in packed
assert packed["messages"] == _V2_MESSAGES
assert packed["systemPrompt"] == "sys"
assert packed["prompt"] == "hi"
def test_a_bundled_v2_artifact_reads_after_the_live_store_is_gone(state, workspace, tmp_path):
"""The bundle's whole promise: it survives being copied to another machine."""
source = _v2_artifact(state, _V2_MESSAGES)
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", name="llm.call", attrs={"llm.input.artifact_path": str(source)})],
)
bundle_dir = tbundle.collect_bundle("trace-1", out_dir=tmp_path / "out", state_dir=state, workspace=workspace)
moved = tmp_path / "elsewhere"
shutil.move(str(bundle_dir), str(moved))
shutil.rmtree(state)
packed = json.loads((moved / "artifacts" / source.name).read_text(encoding="utf-8"))
assert packed["messages"] == _V2_MESSAGES
def test_a_missing_message_blob_becomes_a_placeholder_and_reaches_the_manifest(state, workspace, tmp_path):
from raven.tracing import artifact_v2 as v2
source = _v2_artifact(state, _V2_MESSAGES)
gone = v2.message_sha1(_V2_MESSAGES[1])
v2.message_path(state / "logs" / "audit-artifacts", gone).unlink()
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", name="llm.call", attrs={"llm.input.artifact_path": str(source)})],
)
bundle_dir = tbundle.collect_bundle("trace-1", out_dir=tmp_path / "out", state_dir=state, workspace=workspace)
packed = json.loads((bundle_dir / "artifacts" / source.name).read_text(encoding="utf-8"))
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert len(packed["messages"]) == 2, "position is preserved; a message is never dropped"
assert packed["messages"][0] == _V2_MESSAGES[0]
assert gone in packed["messages"][1]["content"]
assert manifest["missing_messages"] == [gone]
def test_a_v1_artifact_is_still_copied_byte_for_byte(state, workspace, tmp_path):
raw = '{"messages": [{"role": "user", "content": "hi"}]}'
source = _make_artifact(state, "v1.json", raw)
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", name="tool.call", attrs={"tool.output.artifact_path": str(source)})],
)
bundle_dir = tbundle.collect_bundle("trace-1", out_dir=tmp_path / "out", state_dir=state, workspace=workspace)
assert (bundle_dir / "artifacts" / "v1.json").read_text(encoding="utf-8") == raw
def test_bundle_tolerates_non_string_legacy_id(state, workspace):
"""save by trace id still bundles when the span carries a list attempt.id
(unvalidated history must degrade, not crash the collector)."""
span = _span("trace-1", session_key="cli:chat1")
span["attributes"]["attempt.id"] = ["not", "a", "string"]
_write_log(state / "logs" / "audit-spans.log", [span])
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state, workspace=workspace)
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["attempt_id"] == "trace-1"
assert manifest["span_count"] == 1
assert "trace-1" in tstore.pins(state)
def test_bundle_of_merged_attempt_survives_bad_neighbor_record(state):
"""A list-traceId record elsewhere in the log must not break bundling a
merged attempt (the member filter is a set lookup)."""
bad = {"schemaVersion": "audit.span.v1", "traceId": ["x"], "attributes": {}}
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", session_key="cli:chat1"), bad, _span("trace-2", session_key="cli:chat1")],
)
aid = tstore.merge_attempts(["trace-1", "trace-2"], state_dir=state)
bundle_dir = tbundle.collect_bundle("trace-1", state_dir=state)
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["attempt_id"] == aid
assert manifest["span_count"] == 2
def test_bundle_of_merged_attempt_tolerates_malformed_verdict_lines(state):
"""The alias-set verdict filter must skip a list attempt_id line instead
of failing the whole collection."""
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1", session_key="cli:chat1"), _span("trace-2", session_key="cli:chat1")],
)
tverdict.record_verdict("trace-1", "fail", source="user", state_dir=state)
aid = tstore.merge_attempts(["trace-1", "trace-2"], state_dir=state)
with (state / "verdicts.jsonl").open("a", encoding="utf-8") as f:
f.write(json.dumps({"attempt_id": ["bad"], "status": "fail", "source": "user", "ts": "now"}) + "\n")
bundle_dir = tbundle.collect_bundle(aid, state_dir=state)
verdicts = _read_lines(bundle_dir / "verdicts.jsonl")
assert {v["attempt_id"] for v in verdicts} == {"trace-1"}
def test_bundle_of_merged_attempt_collects_all_member_traces(state):
_write_log(
state / "logs" / "audit-spans.log",
[
_span("trace-1", session_key="cli:chat1", start="2026-08-20T10:00:00+00:00"),
_span("trace-2", session_key="cli:chat1", start="2026-08-20T11:00:00+00:00"),
],
)
tverdict.record_verdict("trace-1", "fail", source="user", state_dir=state)
aid = tstore.merge_attempts(["trace-1", "trace-2"], state_dir=state)
tverdict.record_verdict(aid, "pass", source="user", state_dir=state)
bundle_dir = tbundle.collect_bundle("trace-2", state_dir=state)
assert bundle_dir == (state / "bundles" / aid).resolve()
spans = _read_lines(bundle_dir / "spans.jsonl")
assert [s["traceId"] for s in spans] == ["trace-1", "trace-2"]
manifest = json.loads((bundle_dir / "manifest.json").read_text(encoding="utf-8"))
assert manifest["attempt_id"] == aid
assert manifest["span_count"] == 2
verdicts = _read_lines(bundle_dir / "verdicts.jsonl")
assert {v["attempt_id"] for v in verdicts} == {"trace-1", aid}
def test_bundle_of_definition_with_purged_members_raises_lookup_error(state):
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1"), _span("trace-2")],
)
aid = tstore.merge_attempts(["trace-1", "trace-2"], state_dir=state)
(state / "logs" / "audit-spans.log").unlink()
with pytest.raises(LookupError, match="no spans found for attempt"):
tbundle.collect_bundle(aid, state_dir=state)
def test_bundle_pin_falls_back_to_member_traces_when_attempt_vanishes(state, monkeypatch):
_write_log(
state / "logs" / "audit-spans.log",
[_span("trace-1"), _span("trace-2")],
)
aid = tstore.merge_attempts(["trace-1", "trace-2"], state_dir=state)
def vanished(id_, *, reason="", state_dir=None):
raise LookupError(f"no spans found for id {id_!r}")
monkeypatch.setattr(tbundle, "pin_attempt", vanished)
tbundle.collect_bundle(aid, state_dir=state)
registry = tstore.pins(state)
assert registry["trace-1"]["reason"] == "bundled"
assert registry["trace-2"]["reason"] == "bundled"
def test_write_bundle_tree_takes_the_session_key_it_is_given(state, tmp_path):
from raven.trajectory.bundle import write_bundle_tree
dest = tmp_path / "tree"
dest.mkdir()
seen: list[str | None] = []
def no_session(target, key):
seen.append(key)
return False
spans = [_span("trace-sub", session_key="subagent:x"), _span("trace-a", session_key="cli:main")]
manifest = write_bundle_tree(
dest,
spans,
attempt_id="sel-000000000000",
state_dir=state,
write_session=no_session,
verdicts=[],
session_key="cli:main",
)
assert manifest["session_key"] == "cli:main" and seen == ["cli:main"]
assert manifest["session_included"] is False and not (dest / "session.jsonl").exists()
derived = tmp_path / "derived"
derived.mkdir()
assert (
write_bundle_tree(
derived, spans, attempt_id="sel-000000000000", state_dir=state, write_session=no_session, verdicts=[]
)["session_key"]
== "subagent:x"
)
def test_write_bundle_tree_confined_to_the_store_leaves_outside_files_missing(state, tmp_path):
from raven.trajectory.bundle import write_bundle_tree
outside = tmp_path / "outside.txt"
outside.write_text("not from the store", encoding="utf-8")
inside = _make_artifact(state, "inside.json", '{"content": "from the store"}')
spans = [
_span(
"trace-a", session_key="cli:main", attrs={"a.artifact_path": str(outside), "b.artifact_path": str(inside)}
)
]
store = (state / "logs" / "audit-artifacts").resolve()
def none(_target, _key):
return False
confined = tmp_path / "confined"
confined.mkdir()
manifest = write_bundle_tree(
confined, spans, attempt_id="sel-000000000000", state_dir=state, write_session=none, verdicts=[], confine=store
)
assert manifest["missing_artifacts"] == [str(outside)]
assert sorted(p.name for p in (confined / "artifacts").iterdir()) == ["inside.json"]
open_tree = tmp_path / "open"
open_tree.mkdir()
manifest = write_bundle_tree(
open_tree, spans, attempt_id="sel-000000000000", state_dir=state, write_session=none, verdicts=[]
)
assert manifest["missing_artifacts"] == [], "without confine the attempt bundle keeps its old behavior"