## Summary raven agents new scaffolds a complete, working agent on the installed raven: eleven fire-tested template files land in the raven home's agents/ tree (discovered with zero registration), a four-segment health card examines the result (atomic write, doctor, a real ACP handshake smoke, next steps), and --engine-wheel grows the same scaffold into the wheel-shaped harness the shipped engines use. The developer guide ships with it, in-repo and editor-checkable. Key decisions: - **Templates are real code, fire-tested before templating.** The eleven files under raven/templates/agents_scaffold/ pass ruff and py_compile like any source, ride the wheel, and their content was proven live before landing: discovery, handshake, and plugin tool/hook injection all green on a hand-assembled probe in a disposable home. A dedicated spike also proved the placement is invisible to the import-linter contracts and needs exactly one coverage-gate exclusion line. - **Home-first landing, registration as an explicit act.** The default target is the raven home's agents/ tree, discovery's first root: a scaffolded folder is on the roster without any registration. --register mirrors the generated install.py through the same pinned write door; --here lands in the checkout for in-repo agent work. - **The scaffold examines its own work.** Doctor re-validates the manifest, re-parses the plugin manifest, compiles every generated file, and really scans discovery; smoke spawns the launcher for one initialize frame. Three smoke outcomes are told apart: GREEN, the launcher's loud no-key refusal (fail-closed design, passed through verbatim), and FAILED with the scene preserved. Display names are fenced so no input can corrupt the generated manifest. - **Schemas derive from the models.** scripts/export_agent_schemas.py renders schemas/subagent.schema.json (a discriminated union on kind) and schemas/raven-plugin.schema.json from the pydantic models; a contract family holds the on-disk files byte-equal to a fresh render and validates every real manifest on the tree against them. Editors get instant validation with two settings lines, documented in the guide. - **The lexicon follows the terminology ruling.** agents/BUILDING.md is the from-zero developer guide (gold standard: a developer who never read the source reaches a green handshake in ten minutes); CONTEXT.md renames the category entry to Discovered agent -- agent the thing, harness the mechanism, subagent the protocol seat -- with product absorbed into Avoid; the charter gains the scaffold entry and the engine-ownership line. - **--engine-wheel lands both shapes.** User mode drops the engine project in the working directory (pip install -e territory); --here lands it on the plugins-dist shelf. The launcher adopts the official ENGINE_PACKAGE convention and refuses loudly pre-exec when the engine is missing; the doctor reports that row as disabled by design with the install cure printed. No plugins/ directory ships beside an engine-shaped agent, matching the shipped engines. - **A pre-push adversarial review hardened the whole surface.** Two hostile lenses (filesystem edges of a command that writes into real raven homes; template quality as the file set every future agent copies) found 3 HIGH / 6 MED / 5 LOW -- among them: --register could pin an enabled row that cannot start, the name fence ignored the discovered tree, one installed engine wheel rode the host's and every sibling's turns through a fixed tool name, and crash residue was discovered as a hidden live agent. All fourteen close in a dedicated hardening commit, each with a regression test built from the review's own reproduction. - **Near-zero kernel touch, and the two touches are the review's.** Nothing under raven/contracts/ or the agent loop changed. The branch carries exactly two narrow runtime fixes exposed by the review: discovery now skips dot-directories (crash residue is never an agent), and the config schema gains an engine round-trip field (declared as a field-level neutral seam; merge and readiness semantics untouched). ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification - Every commit landed through a full-suite rehearsal on a clean bench before touching the branch; tails over the original base's 19953 passed / 137 skipped: 19980 (templates + command), 19993 (doctor + smoke + gold tests), 20010 (schemas), 20010 (docs, count-identical as expected), 20018 (engine wheel) -- exit 0 and 137 skipped throughout, every step a pure addition, +65 tests net. The branch diff is fingerprint-stable across the rebase (git patch-id --stable verified pairwise across two clean rebases); the branch closes at 20094 passed / 137 skipped / exit 0 on the current main base, +80 tests net over whatever base it sits on -- re-measured in full after every rebase. - A real-machine pass on the live raven home closed the loop end to end: one command scaffolds into ~/.raven/agents, the four-segment card runs all green, and a genuine LLM turn has the agent's own model call its own scaffolded hello tool through the permission gate and relay the exact output; the probe was then removed without touching the roster. - Gold tests spawn for real: scaffold into an isolated home, assert the four doctor lines, readiness ready, and a live ACP initialize handshake GREEN (~3s, zero tokens); a keyless twin pins the refusal passthrough and an engine twin pins disabled-as-designed. - Schema contract: mutation audits both directions (tampered model and tampered file each turn the family red); all six subagent manifests and all eight raven-plugin.toml files on the tree validate. - make lint-python, uv run lint-imports (9 contracts kept, 0 broken), make check-large-files, make check-source-language: green at every step; wheel build carries all seventeen template files and none of schemas/. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes New command surface only: no existing agent, roster row, or runtime path changes behavior. The generated .env stays untracked and the wheel builder still refuses secrets. The one gate change is a coverage-gate exclusion declaring templates non-production. Rollback: revert the squash commit; scaffolded agents already created keep working, since they run on the installed raven like any discovered folder. ## Related Issues N/A
423 lines
15 KiB
Python
423 lines
15 KiB
Python
"""CLI plugin-stack helper.
|
|
|
|
Exercises :func:`build_plugin_registry` and
|
|
:func:`maybe_build_memory_backend` against the ``raven_everos`` plugin,
|
|
which the dev environment installs from ``plugins-dist/everos-memory`` and
|
|
which discovery finds through the ``raven.plugins`` entry-point group.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import ast
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
import raven
|
|
from raven.config.raven import (
|
|
MemoryConfig,
|
|
PluginsConfig,
|
|
RavenConfig,
|
|
)
|
|
from raven.config.schema import Config, MediaGenConfig, ToolsConfig, WebToolsConfig
|
|
from raven.contracts.memory import MemoryBackend
|
|
from raven.core.plugin_stack import (
|
|
build_plugin_hooks,
|
|
build_plugin_registry,
|
|
build_plugin_tools,
|
|
everos_plugin_installed,
|
|
everos_plugin_missing_note,
|
|
maybe_build_memory_backend,
|
|
named_plugin_roots,
|
|
)
|
|
from raven.plugins import PluginRegistry
|
|
from tests._everos_presence import everos_plugin_absent, everos_plugin_broken
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _config(
|
|
*,
|
|
memory_backend: str | None = "everos",
|
|
disabled: list[str] | None = None,
|
|
plugin_config: dict | None = None,
|
|
dirs: list[str] | None = None,
|
|
tools: ToolsConfig | None = None,
|
|
) -> RavenConfig:
|
|
return RavenConfig(
|
|
memory=MemoryConfig(backend=memory_backend),
|
|
plugins=PluginsConfig(
|
|
disabled=list(disabled or []),
|
|
config=dict(plugin_config or {}),
|
|
dirs=list(dirs or []),
|
|
),
|
|
base=Config(tools=tools) if tools is not None else Config(),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# build_plugin_registry
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestBuildRegistry:
|
|
def test_returns_registry_with_everos_activated(self) -> None:
|
|
reg = build_plugin_registry(_config())
|
|
assert isinstance(reg, PluginRegistry)
|
|
assert "everos-memory" in reg.activated_ids()
|
|
assert "everos" in reg.memory_backend_names()
|
|
|
|
def test_disabled_plugin_id_skipped(self) -> None:
|
|
reg = build_plugin_registry(
|
|
_config(disabled=["everos-memory"]),
|
|
)
|
|
assert "everos-memory" not in reg.activated_ids()
|
|
assert "everos" not in reg.memory_backend_names()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# maybe_build_memory_backend
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestMaybeBuildBackend:
|
|
def test_default_config_builds_everos(self, tmp_path: Path) -> None:
|
|
backend = maybe_build_memory_backend(tmp_path, _config())
|
|
assert backend is not None
|
|
assert isinstance(backend, MemoryBackend)
|
|
|
|
def test_memory_backend_none_returns_none(
|
|
self,
|
|
tmp_path: Path,
|
|
) -> None:
|
|
backend = maybe_build_memory_backend(
|
|
tmp_path,
|
|
_config(memory_backend=None),
|
|
)
|
|
assert backend is None
|
|
|
|
def test_unknown_backend_returns_none_no_raise(
|
|
self,
|
|
tmp_path: Path,
|
|
) -> None:
|
|
"""A user-config typo / missing plugin must NOT crash boot —
|
|
the helper logs + degrades, AgentLoop falls back to legacy."""
|
|
backend = maybe_build_memory_backend(
|
|
tmp_path,
|
|
_config(memory_backend="nonexistent"),
|
|
)
|
|
assert backend is None
|
|
|
|
def test_disabled_backend_returns_none(self, tmp_path: Path) -> None:
|
|
backend = maybe_build_memory_backend(
|
|
tmp_path,
|
|
_config(
|
|
memory_backend="everos",
|
|
disabled=["everos-memory"],
|
|
),
|
|
)
|
|
assert backend is None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Per-plugin config slice resolution
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestConfigSliceResolution:
|
|
def test_config_by_plugin_id(self, tmp_path: Path) -> None:
|
|
backend = maybe_build_memory_backend(
|
|
tmp_path,
|
|
_config(
|
|
plugin_config={
|
|
"everos-memory": {"mode": "embedded", "base_url": "http://x"},
|
|
}
|
|
),
|
|
)
|
|
# Backend received the config slice keyed by plugin id.
|
|
assert backend._config["mode"] == "embedded"
|
|
assert backend._config["base_url"] == "http://x"
|
|
|
|
def test_config_by_backend_name_fallback(
|
|
self,
|
|
tmp_path: Path,
|
|
) -> None:
|
|
"""When the user uses the shorter key (backend name), the
|
|
helper still finds it. Useful for handwritten configs."""
|
|
backend = maybe_build_memory_backend(
|
|
tmp_path,
|
|
_config(
|
|
plugin_config={
|
|
"everos": {"mode": "http", "base_url": "http://y"},
|
|
}
|
|
),
|
|
)
|
|
assert backend._config["mode"] == "http"
|
|
assert backend._config["base_url"] == "http://y"
|
|
|
|
def test_plugin_id_takes_precedence_over_backend_name(
|
|
self,
|
|
tmp_path: Path,
|
|
) -> None:
|
|
"""If both keys are present, the canonical (plugin id) wins."""
|
|
backend = maybe_build_memory_backend(
|
|
tmp_path,
|
|
_config(
|
|
plugin_config={
|
|
"everos-memory": {"marker": "canonical"},
|
|
"everos": {"marker": "fallback"},
|
|
}
|
|
),
|
|
)
|
|
assert backend._config["marker"] == "canonical"
|
|
|
|
def test_no_config_slice_yields_empty_dict(
|
|
self,
|
|
tmp_path: Path,
|
|
) -> None:
|
|
backend = maybe_build_memory_backend(tmp_path, _config())
|
|
assert backend._config == {}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Registry injection — caller can pass a pre-built registry
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestRegistryInjection:
|
|
def test_caller_supplied_registry_used(self, tmp_path: Path) -> None:
|
|
reg = build_plugin_registry(_config())
|
|
backend = maybe_build_memory_backend(
|
|
tmp_path,
|
|
_config(),
|
|
registry=reg,
|
|
)
|
|
# Backend constructed via the explicit registry.
|
|
assert backend is not None
|
|
|
|
def test_default_construction_creates_internal_registry(
|
|
self,
|
|
tmp_path: Path,
|
|
) -> None:
|
|
# Sanity: with no registry passed, the helper still works.
|
|
backend = maybe_build_memory_backend(tmp_path, _config())
|
|
assert backend is not None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Workspace plumbing through ServiceLocator
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestServiceLocatorPlumbing:
|
|
def test_workspace_reaches_backend(self, tmp_path: Path) -> None:
|
|
backend = maybe_build_memory_backend(tmp_path, _config())
|
|
# EverosBackend stores ctx.services on construction.
|
|
assert backend._services.workspace == tmp_path
|
|
|
|
def test_the_two_identities_do_not_arrive_swapped(self, tmp_path: Path) -> None:
|
|
"""The whole point of moving identity into ServiceLocator is that a
|
|
mismatch makes every written memory unrecallable with no warning. Until
|
|
this assertion existed, swapping the two arguments at the only
|
|
production wiring point left the entire suite green."""
|
|
config = _config()
|
|
config.memory.user_id = "u-distinct"
|
|
config.memory.agent_id = "a-distinct"
|
|
backend = maybe_build_memory_backend(tmp_path, config)
|
|
assert backend._services.user_id == "u-distinct"
|
|
assert backend._services.agent_id == "a-distinct"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# plugins.dirs
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestConfiguredDirs:
|
|
def test_named_roots_resolve_from_config(self, tmp_path: Path) -> None:
|
|
assert named_plugin_roots(_config(dirs=[str(tmp_path)])) == (tmp_path,)
|
|
assert named_plugin_roots(_config()) == ()
|
|
assert named_plugin_roots(None) == ()
|
|
|
|
def test_a_plugin_under_a_named_root_activates(self, tmp_path: Path) -> None:
|
|
root = tmp_path / "plugins"
|
|
(root / "shelf").mkdir(parents=True)
|
|
(root / "shelf" / "raven-plugin.toml").write_text(
|
|
'[plugin]\nid = "shelf"\nversion = "0.1.0"\nenabled_by_default = true\n',
|
|
encoding="utf-8",
|
|
)
|
|
assert "shelf" in build_plugin_registry(_config(dirs=[str(root)])).activated_ids()
|
|
assert "shelf" not in build_plugin_registry(_config()).activated_ids()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The provider grant
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class _GrantWatch:
|
|
"""A registry stand-in that records what locator each factory was handed."""
|
|
|
|
def __init__(self, seen: list) -> None:
|
|
self._seen = seen
|
|
|
|
def hook_names(self):
|
|
return ["h"]
|
|
|
|
def tool_names(self):
|
|
return ["t"]
|
|
|
|
def hook_plugin_id(self, name):
|
|
return "p"
|
|
|
|
def tool_plugin_id(self, name):
|
|
return "p"
|
|
|
|
def build_hook(self, name, *, config, services):
|
|
self._seen.append(("hook", services))
|
|
return object()
|
|
|
|
def build_tool(self, name, *, config, services):
|
|
self._seen.append(("tool", services))
|
|
return object()
|
|
|
|
|
|
class TestProviderGrant:
|
|
def test_the_connections_provider_reaches_hooks_and_tools(self, tmp_path: Path) -> None:
|
|
seen: list = []
|
|
lent = object()
|
|
build_plugin_hooks(tmp_path, _config(), registry=_GrantWatch(seen), provider=lent)
|
|
build_plugin_tools(tmp_path, _config(), registry=_GrantWatch(seen), provider=lent)
|
|
assert [(kind, services.provider) for kind, services in seen] == [("hook", lent), ("tool", lent)]
|
|
|
|
def test_no_provider_lends_none(self, tmp_path: Path) -> None:
|
|
seen: list = []
|
|
build_plugin_hooks(tmp_path, _config(), registry=_GrantWatch(seen))
|
|
assert [(kind, services.provider) for kind, services in seen] == [("hook", None)]
|
|
|
|
|
|
class TestHostToolsGrants:
|
|
"""The host side of ``media_proxy``, ``web_config`` and ``media_config``.
|
|
|
|
The plugin side is tested with a hand-built locator; this is the link nothing
|
|
else pins, and the one that was broken: every entrance hands the stack its
|
|
``RavenConfig``, whose ``tools`` lives under ``base``, so reading ``tools`` off
|
|
the extension object lent None to every deployment.
|
|
"""
|
|
|
|
def test_the_hosts_media_proxy_and_web_section_reach_tools(self, tmp_path: Path) -> None:
|
|
seen: list = []
|
|
tools = ToolsConfig(
|
|
media=MediaGenConfig(proxy="http://127.0.0.1:7890"),
|
|
web=WebToolsConfig(proxy="socks5://127.0.0.1:1080"),
|
|
)
|
|
config = _config(tools=tools)
|
|
build_plugin_tools(tmp_path, config, registry=_GrantWatch(seen))
|
|
|
|
[(kind, services)] = seen
|
|
assert kind == "tool"
|
|
assert services.media_proxy == "http://127.0.0.1:7890"
|
|
assert services.web_config is not None
|
|
assert services.web_config() is config.base.tools.web
|
|
assert services.web_config().proxy == "socks5://127.0.0.1:1080"
|
|
assert callable(services.media_config)
|
|
|
|
def test_a_host_without_a_proxy_lends_none_and_its_web_section(self, tmp_path: Path) -> None:
|
|
seen: list = []
|
|
config = _config()
|
|
build_plugin_tools(tmp_path, config, registry=_GrantWatch(seen))
|
|
|
|
[(_, services)] = seen
|
|
assert services.media_proxy is None
|
|
assert services.web_config is not None
|
|
assert services.web_config() is config.base.tools.web
|
|
|
|
def test_a_base_config_is_read_as_itself(self, tmp_path: Path) -> None:
|
|
"""A caller holding the base ``Config`` rather than the extension is read
|
|
directly; the two lanes name the same section."""
|
|
from raven.core.plugin_stack import _host_config
|
|
|
|
base = Config(tools=ToolsConfig(media=MediaGenConfig(proxy="http://10.0.0.1:3128")))
|
|
assert _host_config(base) is base
|
|
assert _host_config(_config(tools=base.tools)).tools.media.proxy == "http://10.0.0.1:3128"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# everos plugin presence
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestEverosPresence:
|
|
def test_the_dev_environment_carries_the_plugin(self) -> None:
|
|
assert everos_plugin_installed() is True
|
|
|
|
def test_an_uninstalled_distribution_reads_as_absent(self) -> None:
|
|
with everos_plugin_absent():
|
|
assert everos_plugin_installed() is False
|
|
|
|
def test_a_plugin_that_is_broken_inside_still_reads_as_installed(self) -> None:
|
|
"""The two failures a host must not confuse: gone, and here but broken.
|
|
|
|
A package whose own imports fail is present -- the guard says so, and the
|
|
import the caller then makes raises the plugin's own error instead of
|
|
being reported as an absence nobody can act on.
|
|
"""
|
|
with everos_plugin_broken():
|
|
assert everos_plugin_installed() is True
|
|
with pytest.raises(ImportError):
|
|
from raven_everos.health import capability_available # noqa: F401
|
|
|
|
def test_no_host_module_imports_the_plugin_at_module_level(self) -> None:
|
|
"""Every one of these imports has to stay inside a function.
|
|
|
|
A module-level one fails at ``import raven.cli``, before any guard can
|
|
run, and no amount of degrading downstream can recover from that.
|
|
"""
|
|
root = Path(raven.__file__).parent
|
|
offenders = [
|
|
f"{path.relative_to(root)}:{node.lineno}"
|
|
for path in sorted(root.rglob("*.py"))
|
|
for node in ast.parse(path.read_text(encoding="utf-8")).body
|
|
if isinstance(node, (ast.Import, ast.ImportFrom))
|
|
for name in ([node.module or ""] if isinstance(node, ast.ImportFrom) else [a.name for a in node.names])
|
|
if name.startswith("raven_everos")
|
|
]
|
|
|
|
assert offenders == []
|
|
|
|
def test_the_note_names_the_distribution_and_the_default(self) -> None:
|
|
note = everos_plugin_missing_note()
|
|
assert "everos-memory" in note
|
|
assert "memory.backend" in note
|
|
|
|
|
|
def test_build_plugin_tools_stamps_the_contributing_plugin(tmp_path):
|
|
"""[seam-1] The wake grant's namespace comes from this stamp; a plugin
|
|
that could name its own namespace could steal another's partition."""
|
|
from types import SimpleNamespace
|
|
|
|
from raven.core.plugin_stack import build_plugin_tools
|
|
|
|
class _Tool:
|
|
name = "t1"
|
|
|
|
class _Reg:
|
|
def tool_names(self):
|
|
return ["t1"]
|
|
|
|
def tool_plugin_id(self, name):
|
|
return "plug-a"
|
|
|
|
def build_tool(self, name, config, services):
|
|
return _Tool()
|
|
|
|
cfg = SimpleNamespace(
|
|
memory=SimpleNamespace(user_id="u", agent_id="a"),
|
|
plugins=SimpleNamespace(config={}),
|
|
)
|
|
tools = build_plugin_tools(tmp_path, cfg, registry=_Reg(), provider=None)
|
|
assert len(tools) == 1
|
|
assert tools[0].contributed_by == "plug-a"
|