## Summary A third playbook mode. A `mode: stint` playbook declares roles instead of nodes, and the driver compiles one sub-agent graph per round, so a stint is not a sibling of a `dag` run but a layer over it: one fan-out is one graph, and many rounds are one graph each. Thirty rounds is thirty graphs on the shared dispatch path, which is what makes the validation, the dispatch budget, the approval gate and the detached-run bookkeeping the ones already there rather than a second set. The stint is a file. Nothing depends on the process that started it still being alive: a gateway restart, a machine restart, or a `raven playbook stints resume` all pick it up, and the resume is node-granular, so a round that died with two roles done re-runs the third. **One noun, singular and plural.** The mode is `mode: stint`; one run of such a playbook is a stint, with its own id, checkout and record; the plural is the command family and the RPC namespace. `round` keeps its own meaning as the unit inside -- one pass of every role -- and `stage` as one role's turn in it, so "the stint is on round 7 of 30" says three different things with three different words. The whole surface sits under `raven playbook`: `stints` answers what has run on this machine, `stint` acts on one project's layout and the backlog its roles share. **Handover is files, not a conversation.** Every role opens a fresh conversation each round, which is a decision rather than a limitation: which instance serves which role changes, processes restart, transcripts get compacted. What a role learned this round reaches the next one only by being written down. That makes the journal the contract -- an append-only file, windowed by recent rounds and by characters, because a forty-round journal would otherwise fill the prompt it exists to inform. **The boundary is three layers.** `owns` and `appends` are declared in the playbook, rendered into the role's prompt from the same declaration the enforcement reads, measured by the role's own checks, and undone afterwards with the file the role wrote kept under `violations/`. Declaring without telling is a trap; telling without undoing is a suggestion; and the undo reads the stage's own commits as well as the worktree, because a role that commits a stray write disappears from the working tree and a boundary any role can step around by committing is not one. Nothing refuses a write before it lands: a charter narrows a role's tools only where the playbook declares a `playbook:` block, and `owns` is not derived into one, so the shipped playbook's roles have no tool gate and the undo is the boundary rather than its backstop. Two shapes the boundary cannot hold are refused where they are written rather than left to surface as violations in round four: - **Two roles that can run at once where either is enforced.** They share one checkout, so the first judged is graded against the difference the whole tree shows and quarantines the other's work. Observed before it was refused: two roles with disjoint `owns`, each writing only what it owned, and the first judged came back as `dev-a wrote 1 path(s) it may not write: src/b/work.py`, with that file gone. The message names both ways out, because which one is right is the author's call. What would make it work is a checkout per concurrent role, which is a merge problem, not a validation rule. - **`enforce.read: hard`.** It reaches the guard text, which then tells the role that reads outside its paths are refused at the tool gate, and no gate refuses one. The wording stays in `rounds_prompt` for whoever wires the gate; until then the grade is not offered. **Checks come before any model's opinion.** `verify[]` runs real commands, and a failure goes back to the role that caused it with the failure text, up to `maxHandbacks`. When that budget runs out the round records the failure and carries on rather than failing the node: a failed node skips every role downstream, and a failed build is exactly what the reviewer downstream has to see. **One approval, and it names what it is approving.** The whole stint is confirmed once, at the start, and the text states the round budget and every command the stint may run -- a playbook is a distribution unit, so a file somebody handed you does not get to run shell thirty times on one unexamined yes. Every round is charged against the hourly dispatch budget like any graph; a round the budget turns down pauses the stint rather than failing it, and `stints resume` takes it up once the allowance recovers. From the CLI the same approval is put to the terminal, and with no terminal to ask at the run is refused rather than approved on nobody's behalf. A stint works a checkout of its own, cut from the project's head, so hours of its commits do not collide with the conversation that started it. `stop.maxRounds` defaults to 10 and is capped at 99. `stop.until` lets a role end a stint early by reporting a marker on a line of its own -- a whole line, so a role quoting the marker mid-sentence does not end the plan -- and only a terminal role is told the marker, because a role that knows how to stop the stint has a reason to reach for it. **Surfaces.** `raven playbook stints list/get/extend/pause/stop/resume/sweep/answer`, where `resume` and `extend` are relayed to the raven serving the page when one is up, so a round runs in the process that owns the conversation; `playbooks.stints.*` RPC methods, the read-only ones reading the record off disk so opening the page does not cost what a dispatch costs; a Stint tab on the playbook page showing the roles, what each owns, the checks with their full commands, and what carries between rounds; and a Runs tab listing every stint with its standing, how far into its budget it is, each round's checks and what was undone, with pause, stop and resume. `raven playbook stint` lays a project out for one. The shipped playbook is `rounds`: a planner, a builder and a verifier, on any project that has a specification. `init` writes `.stint/`: a guard file per role, the shared prose, and a link to the project's own specification, chosen by two sieves -- the file name first, then how much of the document says something is required, so an empty `SPEC.md` stub does not send thirty rounds off a blank page. `task`, `ask` and `confirm` work the backlog the roles share. New package `raven/stint/` carries the parts a stint needs that are not the playbook: the record, its checkout, the ownership grades, the journal window and the check runner. `CONTEXT.md`, `CHANGELOG.md` and the two `docs-site` repo-layout tables gain their rows; `README.md` is not touched. Three unrelated changes ride along. Two are the same shape -- a web change that does not show: - `index.html` went out with no caching header, so browsers fell back to heuristic freshness and served a rebuilt page from cache without asking, which reads exactly like a server that never picked the change up. `no-cache` keeps the entry and revalidates; the ETag still saves the transfer. - `make ui` is one target for the two-step page build, because running only vite leaves the served one-file page unchanged with nothing to say so. The third is `raven agent` (`raven/cli/agent_commands.py`): `--message-file` reads the prompt from a file, `--permission-mode` picks the tier for the run, and `--no-markdown` prints a soft-wrapped `Text` rather than a markdown render; 92 lines of tests in `tests/test_cli_agent_commands.py`. Designed and deliberately not built, so a reviewer knows where the edges are: a checkout per concurrent role with each role merging itself in when it is judged (`docs/plans/2026-09-18-stint-acceptance.md`), and the backlog-writing round zero. `raven playbook stint confirm` records a person's word on the plan and nothing in the rounds path reads the flag yet, which that document also says. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` uv run ruff check --no-cache raven evolver agents plugins-dist tests scripts uv run ruff format --check --no-cache raven evolver agents plugins-dist tests scripts uv run lint-imports make lint-types make lint-ui make lint-tui uv run pytest -q uv run python scripts/check_commit_messages.py main..HEAD uv run python scripts/check_source_language.py main..HEAD uv run python scripts/check_large_files.py main..HEAD ``` Ruff clean over 2,004 files. Import contracts: 10 kept, 0 broken. `ty` clean, which caught one real defect on the way in: `StintStore.list` shadowed the builtin for every annotation after it in the class body, so `-> list[StintRecord]` on it and on `live()` resolved to the method. The UI gates include the generated-file drift checks (`gen:check`, `lint:rpc`, `lint:i18n`) and both type-checks; eslint reports 23 warnings and 0 errors, none of them in a file this branch touches. `make lint-bridge` was not run: this checkout has no `bridge/node_modules`, and nothing here touches `bridge/`. Suite: the whole `uv run pytest -q` runs in CI as four shards, and every shard is green at this head, with the import-cycle budget (44 modules, the same as `main`), the coverage gate and the three content gates over the commit range. Locally the suites this branch touches are run per change; the full run is CI's. Behaviour, driven rather than asserted: - A stray write by a concurrent role was reproduced end to end before the validation refused the shape: `dev-b's own file still there after dev-a was judged: False`, `violations recorded against dev-a: ['dev-a wrote 1 path(s) it may not write: src/b/work.py']`. - The check runner's timeout kills the process group, not just the direct child. Proved as a regression guard: red with `process.kill()`, green with the group teardown. - The round budget refuses rather than wrapping: extending a real stint past the ceiling answered `that would give it a budget of 104, and 99 is the most any stint may have`, and the file was not written. - The spec picker's two sieves, over a real project: `KEPT docs/PRD.md | weight 12`, `dropped SPEC.md | len 12 | weight 0`, `dropped notes.md | len 868 | weight 0`. - Only a terminal role is told the stop marker: `planner=False builder=False verifier=True` on the shipped example. - The shipped example is discoverable with no user layer: `listed: ['rounds']`, `origin: builtin`. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk Additive. `mode: stint` is a new value; `dag` and `prompt` playbooks load, validate, dispatch and report exactly as before, and the sections the new mode reads are withheld from the generator's emit schema, so no generated playbook can reach them. That withholding is pinned by a test that fails when a section is added to the spec without being added to the refusal. Nothing outside the new mode changes behaviour: `spawn` and the DAG path keep the per-session in-memory dispatch window they have on `main`. A plan runs shell from `verify[]`. It is gated on the approval, the approval text lists every command, and a playbook carrying `verify` cannot be generated -- only written by hand or shipped. The builtin layer ships exactly one playbook, `rounds`, pinned by name in a test, for that reason: withheld from the generator and not shipped either, the mode would be reachable only by transcribing YAML out of prose. Rollback is a revert of the merge commit. A plan's on-disk record is under the sub-agent history root and is read by nothing else; nothing else's on-disk shape changes. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Ethan Wang <ethan.wang@evermind.ai> Co-authored-by: Claude (claude-opus-5[1m]) <noreply@anthropic.com>
281 lines
12 KiB
Python
281 lines
12 KiB
Python
"""Turning a project into one a run can work in: the scaffolding, and the plan's shape."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from raven.stint import bootstrap
|
|
from raven.stint import roster as roster_mod
|
|
from raven.stint.bootstrap import InitError
|
|
|
|
PRD = """# Arena
|
|
|
|
## 7. Acceptance gates
|
|
|
|
| # | name | criterion |
|
|
|---|------|-----------|
|
|
| 7.1 | clean_boot | it boots with an empty stderr |
|
|
| 7.2 | feel_log | every event carries a timestamp |
|
|
| 7.3 | input | the shot leaves within 60 ms |
|
|
"""
|
|
|
|
|
|
def _project(tmp_path: Path, *, docs: dict[str, str] | None = None) -> Path:
|
|
project = tmp_path / "game"
|
|
(project / "docs").mkdir(parents=True)
|
|
for name, body in (docs or {"PRD-arena.md": PRD}).items():
|
|
(project / "docs" / name).write_text(body, encoding="utf-8")
|
|
(project / "README.md").write_text("# readme\n", encoding="utf-8")
|
|
return project
|
|
|
|
|
|
def test_init_lays_out_the_directory_and_links_the_specification(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
result = bootstrap.init(project)
|
|
|
|
home = project / ".stint"
|
|
assert (home / "SPEC.md").is_symlink()
|
|
assert (home / "SPEC.md").resolve() == (project / "docs" / "PRD-arena.md").resolve()
|
|
for name in ("HUMAN_DECISIONS.md", "AGENT_DECISIONS.md", "FIXLOG.md", "PLAYBOOK.md"):
|
|
assert (home / name).is_file(), name
|
|
for role in ("planner", "builder", "verifier"):
|
|
assert (home / f"{role}.md").is_file(), role
|
|
assert (project / "reports").is_dir()
|
|
assert result.spec.endswith("PRD-arena.md")
|
|
|
|
|
|
def test_a_link_rather_than_a_copy_so_a_revision_is_one_edit(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
(project / "docs" / "PRD-arena.md").write_text(PRD + "\n| 7.4 | new | added later |\n", encoding="utf-8")
|
|
assert "7.4" in (project / ".stint" / "SPEC.md").read_text(encoding="utf-8")
|
|
|
|
|
|
def test_the_guard_files_it_writes_are_a_roster_the_runtime_can_read(tmp_path: Path) -> None:
|
|
"""The point of the exercise: init's output is the runtime's input, with nothing in between."""
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
|
|
roster = roster_mod.load(project)
|
|
assert [role.name for role in roster.in_round()] == ["planner", "builder", "verifier"]
|
|
assert roster.get("builder").session == roster_mod.CONTINUE
|
|
assert roster.get("planner").session == roster_mod.CONTINUE, "the Planner holds the whole project across rounds"
|
|
assert roster.conflicts() == []
|
|
assert roster.grade("planner", "reports/brief_01.md", round_index=1) == roster_mod.OWNS
|
|
assert roster.grade("verifier", "reports/brief_01.md", round_index=1) == roster_mod.NEVER
|
|
|
|
|
|
def test_no_slot_is_left_unfilled_in_a_generated_guard(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
for role in ("planner", "builder", "verifier"):
|
|
assert "{{" not in (project / ".stint" / f"{role}.md").read_text(encoding="utf-8"), role
|
|
|
|
|
|
def test_the_enforcement_grade_is_the_operator_s_to_choose(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project, enforce_read="hard", enforce_write="soft")
|
|
role = roster_mod.load(project).get("verifier")
|
|
assert role.enforce_read == roster_mod.HARD
|
|
assert role.enforce_write == roster_mod.SOFT
|
|
|
|
|
|
def test_running_init_twice_keeps_what_a_person_edited(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
edited = project / ".stint" / "planner.md"
|
|
mine = edited.read_text(encoding="utf-8") + "\n## My own rule\n\nAlways ask me first.\n"
|
|
edited.write_text(mine, encoding="utf-8")
|
|
|
|
again = bootstrap.init(project)
|
|
|
|
assert edited.read_text(encoding="utf-8") == mine
|
|
assert ".stint/planner.md" in again.kept
|
|
assert bootstrap.init(project, force=True) and "My own rule" not in edited.read_text(encoding="utf-8")
|
|
|
|
|
|
def test_several_candidate_specifications_are_reported_rather_than_guessed(tmp_path: Path) -> None:
|
|
project = _project(tmp_path, docs={"PRD-a.md": PRD, "spec-b.md": PRD, "design-c.md": PRD})
|
|
result = bootstrap.init(project)
|
|
assert any("more than one document" in note for note in result.notes)
|
|
|
|
|
|
def test_a_project_with_no_specification_says_so(tmp_path: Path) -> None:
|
|
project = tmp_path / "bare"
|
|
project.mkdir()
|
|
result = bootstrap.init(project)
|
|
assert any("no specification document" in note for note in result.notes)
|
|
assert not (project / ".stint" / "SPEC.md").exists()
|
|
|
|
|
|
def test_a_named_specification_that_is_not_there_is_refused(tmp_path: Path) -> None:
|
|
with pytest.raises(InitError):
|
|
bootstrap.init(_project(tmp_path), spec=tmp_path / "nope.md")
|
|
|
|
|
|
def test_the_facts_section_is_drafted_for_a_person_to_check(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
text = (project / ".stint" / "HUMAN_DECISIONS.md").read_text(encoding="utf-8")
|
|
assert "## Facts about this project" in text
|
|
assert "## Awaiting a person" in text
|
|
assert "- Machine:" in text and "- Display:" in text
|
|
# Not the game engine: the layout is what any multi-round project gets,
|
|
# and a Godot line in every project's facts was H* showing through.
|
|
assert "Godot" not in text
|
|
|
|
|
|
def test_a_freshly_initialised_project_is_one_step_from_ready(tmp_path: Path) -> None:
|
|
"""Everything but the plan, which is the step that needs a model and a person."""
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
reasons = roster_mod.preflight(project)
|
|
assert len(reasons) == 1
|
|
assert "raven playbook stint task add" in reasons[0]
|
|
|
|
|
|
# ------------------------------------------------------------------------- plan
|
|
|
|
|
|
def test_the_plan_prompt_names_the_three_files_it_works_between(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
prompt = bootstrap.plan_prompt(project)
|
|
assert str(project / ".stint" / "SPEC.md") in prompt
|
|
assert str(project / ".stint" / "HUMAN_DECISIONS.md") in prompt
|
|
assert str(project / ".stint" / "backlog.json") in prompt
|
|
assert bootstrap.PLAN_SENTINEL in prompt
|
|
assert "depends_on" in prompt and "never a round number" in prompt
|
|
|
|
|
|
def test_gates_are_read_off_the_specification_only_to_check_coverage(tmp_path: Path) -> None:
|
|
assert bootstrap.gates_in(PRD) == ("7.1", "7.2", "7.3")
|
|
|
|
|
|
def test_review_reports_the_gates_no_task_covers(tmp_path: Path) -> None:
|
|
"""A gate nobody planned for is a requirement nobody planned for."""
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
(project / ".stint" / "backlog.json").write_text(
|
|
json.dumps(
|
|
{
|
|
"meta": {},
|
|
"tasks": [
|
|
{"id": 1, "title": "boot", "gates": ["7.1"], "state": "open"},
|
|
{"id": 2, "title": "log", "gates": ["7.2"], "state": "open"},
|
|
],
|
|
}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
review = bootstrap.review(project)
|
|
|
|
assert review.tasks == 2
|
|
assert review.gates == ("7.1", "7.2")
|
|
assert review.uncovered == ("7.3",)
|
|
assert "not covered: 7.3" in review.render()
|
|
|
|
|
|
def test_the_documents_a_project_carried_are_found_at_any_depth_under_docs(tmp_path: Path) -> None:
|
|
"""A half-built project keeps what it learned in the role folders it grew up with."""
|
|
project = _project(tmp_path, docs={"PRD-arena.md": PRD, "DECISIONS.md": "- settled: one boss\n"})
|
|
(project / "docs" / "verifier").mkdir()
|
|
(project / "docs" / "verifier" / "Verifier.md").write_text("# Verifier\n", encoding="utf-8")
|
|
(project / "CLAUDE.md").write_text("# instructions\n", encoding="utf-8")
|
|
(project / "AGENTS.md").symlink_to("CLAUDE.md")
|
|
(project / "reports").mkdir()
|
|
(project / "reports" / "verify_01.md").write_text("# not a document\n", encoding="utf-8")
|
|
bootstrap.init(project)
|
|
|
|
found = bootstrap.project_documents(project, spec=project / ".stint" / "SPEC.md")
|
|
|
|
assert found == ["CLAUDE.md", "docs/DECISIONS.md", "docs/verifier/Verifier.md"], (
|
|
"the spec, README, .stint/ and reports/ are not documents to absorb"
|
|
)
|
|
|
|
|
|
def test_the_plan_prompt_asks_for_the_carry_over_and_the_ledger(tmp_path: Path) -> None:
|
|
project = _project(tmp_path, docs={"PRD-arena.md": PRD, "DECISIONS.md": "- one boss\n"})
|
|
bootstrap.init(project)
|
|
|
|
prompt = bootstrap.plan_prompt(project)
|
|
|
|
assert "- `docs/DECISIONS.md`" in prompt
|
|
assert str(project / ".stint" / "SOURCES.md") in prompt
|
|
assert str(project / ".stint" / "PLAYBOOK.md") in prompt and "## Settled" in prompt
|
|
assert "do not edit or delete them" in prompt
|
|
assert "the reading map" in prompt, "reference material is mapped, not copied"
|
|
assert "is **not**\n copied" in prompt
|
|
assert "a reference image" in prompt and "open the image itself" in prompt, (
|
|
"a picture gets a row that sends the Builder and Verifier to look at it"
|
|
)
|
|
assert "a reference image" in prompt and "open the image itself" in prompt, (
|
|
"a picture gets a row that sends the Builder and Verifier to look at it"
|
|
)
|
|
|
|
|
|
def test_a_greenfield_project_is_told_there_is_nothing_to_carry(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
(project / "README.md").unlink()
|
|
bootstrap.init(project)
|
|
|
|
assert bootstrap.NO_DOCUMENTS in bootstrap.plan_prompt(project)
|
|
|
|
|
|
def test_review_names_the_documents_the_ledger_left_out(tmp_path: Path) -> None:
|
|
project = _project(tmp_path, docs={"PRD-arena.md": PRD, "DECISIONS.md": "- one boss\n", "LESSONS.md": "- x\n"})
|
|
bootstrap.init(project)
|
|
(project / ".stint" / "backlog.json").write_text(
|
|
json.dumps(
|
|
{"meta": {}, "tasks": [{"id": 1, "title": "boot", "gates": ["7.1", "7.2", "7.3"], "state": "open"}]}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
assert bootstrap.review(project).unabsorbed == ("docs/DECISIONS.md", "docs/LESSONS.md")
|
|
|
|
(project / ".stint" / "SOURCES.md").write_text(
|
|
"| document | went to |\n| docs/DECISIONS.md | HUMAN_DECISIONS.md Settled |\n", encoding="utf-8"
|
|
)
|
|
plan = bootstrap.review(project)
|
|
assert plan.unabsorbed == ("docs/LESSONS.md",)
|
|
assert "not carried into .stint/" in plan.render() and "LESSONS.md" in plan.render()
|
|
|
|
|
|
def test_the_plan_is_told_what_the_tree_already_holds(tmp_path: Path) -> None:
|
|
"""The documents say what was meant; the tree says what is."""
|
|
project = _project(tmp_path)
|
|
(project / "project").mkdir()
|
|
(project / "project" / "project.godot").write_text("[application]\n", encoding="utf-8")
|
|
(project / "reports").mkdir()
|
|
(project / "reports" / "verify_02.md").write_text("# verifier 02\n", encoding="utf-8")
|
|
bootstrap.init(project)
|
|
|
|
prompt = bootstrap.plan_prompt(project)
|
|
|
|
assert "## Then, look at the tree" in prompt
|
|
assert "`project/`" in prompt and "`verify_02.md`" in prompt
|
|
assert "checks detected: `import`" in prompt
|
|
assert "look rather than assume" in prompt, "the layout is the repository's own, so the prompt guides a search"
|
|
assert "Work that already is\n goes in as `in_review`" in prompt
|
|
assert "Nothing is done yet" not in prompt
|
|
|
|
|
|
def test_a_greenfield_tree_is_named_as_such(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
prompt = bootstrap.plan_prompt(project)
|
|
assert "none found -- this may be a greenfield project" in prompt and "Markdown under `reports/`: none" in prompt
|
|
|
|
|
|
def test_the_planner_is_asked_for_a_coherent_piece_not_a_count(tmp_path: Path) -> None:
|
|
project = _project(tmp_path)
|
|
bootstrap.init(project)
|
|
guard = (project / ".stint" / "planner.md").read_text(encoding="utf-8")
|
|
assert "one coherent piece of work" in guard and "the number is not the point" in guard
|
|
assert "At most three priorities" not in guard
|