Raven-1/tests/test_plugin_hooks.py
Zuyi Zhou 34ed138cc7
fix(plugins): a plugin that fails to load no longer takes every other plugin with it (#703)
## Summary

One plugin that failed to activate dropped every plugin.
`PluginRegistry.activate` raised on the first plugin whose factory would
not import (or whose contribution name collided), and
`build_plugin_registry` (`raven/core/plugin_stack.py`) answered by
returning an empty registry. A single broken user plugin therefore
silently removed research-flow, ppt-engine, design-engine, the memory
backend and the bundled playbook tools, with one stderr warning as the
only trace. Reproduced with two mock plugins under a scratch
`RAVEN_HOME`: `aa-broken` (factory raises `ImportError`) beside
`zz-good`; before this change the registry came back with `activated:
[]`, after it with `design-engine, everos-memory, playbook, ppt-engine,
zz-good` activated and `aa-broken` named as failed.

Changes:

- **Per-plugin activation** (`raven/plugins/registry.py`). Each admitted
plugin activates on its own. One that fails is rolled back whole (no
partial contribution in any table, and the `sys.path` entry
`_ensure_importable` appended for it is withdrawn), recorded as a
`PluginActivationFailure` (`activation_failures()`), and the plugins
after it still activate. In a name conflict the plugin activated first
(discovery order, by id) keeps the name and the other is recorded as
failed. A module `__getattr__` that raises something other than
`AttributeError` is now a recorded failure rather than an uncaught
crash.
- **Said to the user** (`plugin_stack.build_plugin_registry`,
`runtime.build_runtime`). Each failure is one notice naming the plugin,
its cause and the `plugins.disabled` escape, through the host's notifier
(`HostWiring.notify`), else stderr - the same channel the
missing-memory-backend notice already uses. When the configured memory
backend belongs to a plugin that failed, the memory notice names that
plugin and its cause instead of "no installed plugin provides it".
- **`raven plugins`** shows a `failed` status and the cause under the
table (the command called `activate` directly and previously crashed on
such a plugin).
- **Agents smoke check** (`raven/cli/agents_commands.py`) matched the
old log line "plugin activation failed", which no longer exists; it now
reads the shared `PLUGIN_FAILURE_MARKER`, and its test prints the notice
the real producer writes, so the two cannot drift.
- **Sub-agents inherit the host's opt-outs**
(`product_render.write_rendered`, `inherit_plugin_opt_outs`). A product
engine inherits `RAVEN_HOME`, so it scans the host's `<home>/plugins`
and the shared interpreter's entry points, but it read
`plugins.disabled` only from its own rendered config; a plugin disabled
on the host came back in every product. Every render now merges the
host's list after the product's own. The product's own engine plugin
never travels (`own_plugins=`, passed by all five launchers and the
scaffold): the PPT agent is `ppt-engine`, and a host turning it off for
its own agent is not a request to run the PPT agent without it. A
contract test reads every launcher's `write_rendered` call and requires
it to name its `*_PLUGIN_ID`.
- `CONTEXT.md` (Plugin Registry) and
`docs-site/docs/building-plugin{,.zh}.md` describe the new behaviour.

Not in this PR: the Web UI plugin list (`ext.list`) is discovery-only
and does not show a failed plugin; adding it is a wire change
(`ExtListResult`, openrpc, generated TS, the page) and is left for a
follow-up. The TUI, gateway and `raven agent` hosts show the notice
through their notifiers; the rpc stack (serve/acp) prints it to stderr,
which is also what the agents smoke check reads.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

**Rebased**: head 7cd000dbc on origin/main 8ee5ae0d8, without conflicts.
`git range-diff` shows every commit replayed unchanged. The branch's own
test files rerun at this head: 395 passed. Numbers elsewhere in this
description are from earlier heads.

Head b4d51aee8, rebased onto origin/main 6551f8259 after #698 and #708
merged. #698 conflicted in `raven/config/product_render.py` and
`tests/test_config_product_render.py`: both PRs added a function at the
same spot (`inherit_host_denials` there, `inherit_plugin_opt_outs` here)
and a call in `write_rendered`. The resolution keeps both functions and
all their tests; `write_rendered` now reads `host_config()` once and
passes it to both. #708 touches none of these files. At this head:

```
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q -p no:cacheprovider
  25948 passed, 109 skipped
uv run --frozen --python 3.12 --all-extras pytest -q tests/test_config_product_render.py
  44 passed
ruff check / ruff format --check / make lint-imports lint-deps lint-types / git diff --check origin/main...HEAD
check_source_language.py origin/main...HEAD / make check-commits / check-large-files
  all clean
cd ui-tui && npm run lint:rpc && npm run type-check && npm test
  144 files, 2106 passed
```

The earlier `TUI checks` failure (run 35834019105, head 5fc374035) was
`newInstancePicker.test.tsx > quick-selects by the number it printed`.
This PR changes nothing under `ui-tui/`; the test passed 5 of 5 runs
locally at this head and in the full ui-tui run above.

The numbers below are from the earlier heads and are kept for the
record.

```
uv run --frozen --python 3.12 --extra dev ruff check raven evolver agents plugins-dist tests scripts
  All checks passed
uv run --frozen --python 3.12 --extra dev ruff format --check raven evolver agents plugins-dist tests scripts
  2083 files already formatted
make lint-imports && make lint-deps && make lint-types
  Contracts: 10 kept, 0 broken / No dependency issues found / All checks passed
git diff --check origin/main...HEAD
  clean
COLORTERM=truecolor make coverage        (full suite, head before rebasing onto 7fdf296f4)
  25847 passed, 109 skipped
COVERAGE_BASE_REF=origin/main make coverage-diff
  Diff coverage: 100.00% (71/71 executable changed lines)
PYTHONPATH=. uv run --frozen --python 3.12 --extra dev python scripts/check_source_language.py origin/main...HEAD
  exit 0
make check-commits
  pass
git merge-tree --write-tree HEAD origin/main
  clean
```

After rebasing onto 7fdf296f4 (its two new commits touch none of these
files), the touched suites at this head:

```
COLORTERM=truecolor uv run --frozen --python 3.12 --all-extras pytest -q tests/test_plugin_*.py tests/test_core_plugin_stack.py tests/test_cli_plugin_commands.py tests/test_cli_agents_commands.py tests/test_cli_tui_commands.py tests/test_config_product_render.py tests/test_agents_*launcher*.py tests/test_everos_plugin_discovery.py tests/test_release_plugin_list.py tests/test_cli_gateway_commands.py
  727 passed
uv run --frozen --python 3.12 --all-extras pytest -q tests/test_subagent_vendored_agents.py tests/test_rpc_subagents.py tests/test_subagent_acp.py
  452 passed
```

Revert-to-red, one production file restored to origin/main at a time
with the tests kept:

```
raven/plugins/registry.py        -> 2 errors (plugin_stack imports PluginActivationFailure)
raven/core/plugin_stack.py       -> 6 failed
raven/core/runtime.py            -> 1 failed (build_runtime hands the host's notifier to the registry)
raven/config/product_render.py   -> 42 failed
raven/cli/agents_commands.py     -> 1 failed (smoke reds a handshake whose plugin did not load)
raven/cli/plugin_commands.py     -> 1 failed (failed row and escaped cause)
agents/raven-ppt/run.py          -> 2 failed (engine exemption, launcher contract)
```

Ten existing tests pinned "activation raises" (`test_plugin_registry`,
`test_plugin_bootstrap`, and the conflict tests for tools, hooks, tool
gates and session observers). They now assert the new contract instead:
the later plugin is recorded with the same reason text, the earlier one
keeps the name, and nothing of the loser is registered. Two stubs in
`test_cli_tui_commands.py` took `build_plugin_registry(cfg)` and now
accept the `notify` keyword.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

## Risk

- A plugin conflict used to disable all plugins; it now keeps the plugin
that activates first and skips the other with a notice. Two plugins
contributing the same name therefore run with one of them instead of
none.
- A product sub-agent now starts with the host's `plugins.disabled`
merged into its config (its own engine excepted). A plugin that was
disabled on the host but relied on inside a product stops loading there;
the remedy is to enable it on the host.
- `PluginRegistry.activate` no longer raises `PluginConflictError` /
`PluginFactoryImportError`; the only in-tree callers were
`assemble_plugin_registry` and `raven plugins`. The exception classes
stay exported.
- Security: the change only narrows what loads. A failed plugin's
directory leaves sys.path, and the inherited opt-outs can only remove
plugins from a product, never add one.
- Rollback: revert the squash commit.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-opus-5-5) <noreply@anthropic.com>
2026-09-23 22:41:21 +08:00

250 lines
8.3 KiB
Python

"""Plugins contribute hooks: manifest, registry, stack, and the loop's chain.
The loop fires six phases through one chain; without a plugin contribution
kind for hooks, product code launched as `raven acp --config` could not reach
them. Pinned here: the manifest entry, registry activation with cross-plugin
conflict detection, the lenient stack builder, and the end-to-end boarding --
a plugin directory's hook ends up in the built loop's chain.
"""
from __future__ import annotations
import sys
import textwrap
import types
from pathlib import Path
import pytest
from pydantic import ValidationError
from raven.plugins import (
Contributes,
DiscoveredPlugin,
HookContribution,
ManifestOrigin,
PluginManifest,
PluginRegistry,
)
_INJECTED: set[str] = set()
def _install_module(name: str, attrs: dict[str, object]) -> None:
mod = types.ModuleType(name)
for k, v in attrs.items():
setattr(mod, k, v)
sys.modules[name] = mod
_INJECTED.add(name)
@pytest.fixture(autouse=True)
def _cleanup_modules():
yield
for name in _INJECTED:
sys.modules.pop(name, None)
_INJECTED.clear()
def _discovered(plugin_id: str, hooks: list[tuple[str, str]]) -> DiscoveredPlugin:
mf = PluginManifest(
id=plugin_id,
version="0.1.0",
contributes=Contributes(hooks=[HookContribution(name=n, factory=f) for n, f in hooks]),
)
return DiscoveredPlugin(manifest=mf, source=ManifestOrigin.USER, location=None)
class TestManifestHooks:
def test_parses_a_hooks_contribution(self) -> None:
mf = PluginManifest.from_toml_str(
textwrap.dedent(
"""
[plugin]
id = "research-flow"
version = "0.1.0"
[[plugin.contributes.hooks]]
name = "budget_note"
factory = "research_flow.hooks:make_budget_note"
"""
)
)
assert [h.name for h in mf.contributes.hooks] == ["budget_note"]
def test_default_hooks_empty(self) -> None:
assert PluginManifest(id="p", version="0.1.0").contributes.hooks == []
def test_bad_factory_ref_rejected(self) -> None:
with pytest.raises(ValidationError):
HookContribution(name="h", factory="not-a-ref")
def test_duplicate_hook_names_rejected(self) -> None:
with pytest.raises(ValidationError, match="duplicate hook name"):
PluginManifest(
id="p",
version="0.1.0",
contributes=Contributes(
hooks=[HookContribution(name="dup", factory="m:a"), HookContribution(name="dup", factory="m:b")]
),
)
class TestRegistryHooks:
def test_activates_and_builds(self) -> None:
seen = {}
def make(ctx):
seen["config"] = ctx.config
return "hook-instance"
_install_module("_ph_a", {"make": make})
reg = PluginRegistry()
reg.activate([_discovered("alpha", [("budget_note", "_ph_a:make")])])
assert reg.hook_names() == ["budget_note"]
assert reg.hook_plugin_id("budget_note") == "alpha"
from raven.plugins import ServiceLocator
built = reg.build_hook(
"budget_note",
config={"warn": 0.8},
services=ServiceLocator(workspace=Path("/w"), user_id="u", agent_id="a"),
)
assert built == "hook-instance"
assert seen["config"] == {"warn": 0.8}
def test_cross_plugin_name_conflict_skips_the_later_plugin(self) -> None:
_install_module("_ph_b", {"make": lambda ctx: None})
reg = PluginRegistry()
reg.activate([_discovered("one", [("same", "_ph_b:make")]), _discovered("two", [("same", "_ph_b:make")])])
assert reg.activated_ids() == ["one"]
assert reg.hook_plugin_id("same") == "one"
[failure] = reg.activation_failures()
assert failure.plugin_id == "two"
assert "hook 'same' contributed by both 'one' and 'two'" in failure.reason
class TestBuildPluginHooks:
def _config(self, plugin_config: dict | None = None):
from raven.config.raven import PluginsConfig, RavenConfig
return RavenConfig(plugins=PluginsConfig(config=dict(plugin_config or {})))
def test_builds_hooks_with_the_plugins_slice(self, tmp_path: Path) -> None:
from raven.core.plugin_stack import build_plugin_hooks
def make(ctx):
return f"hook::{ctx.config.get('flag')}"
_install_module("_ph_c", {"make": make})
reg = PluginRegistry()
reg.activate([_discovered("myplugin", [("h1", "_ph_c:make")])])
assert build_plugin_hooks(tmp_path, self._config({"myplugin": {"flag": "on"}}), registry=reg) == ["hook::on"]
def test_failing_and_declining_factories_are_skipped(self, tmp_path: Path) -> None:
from raven.core.plugin_stack import build_plugin_hooks
def boom(ctx):
raise RuntimeError("nope")
_install_module("_ph_d", {"boom": boom, "decline": lambda ctx: None})
reg = PluginRegistry()
reg.activate([_discovered("p", [("a", "_ph_d:boom"), ("b", "_ph_d:decline")])])
assert build_plugin_hooks(tmp_path, self._config(), registry=reg) == []
def test_the_user_plugin_dir_follows_the_home(tmp_path: Path, monkeypatch) -> None:
from raven.core.plugin_stack import plugin_discovery_sources
monkeypatch.setenv("RAVEN_HOME", str(tmp_path / "home"))
assert plugin_discovery_sources()["user_dir"] == tmp_path / "home" / "plugins"
DEMO_HOOK = """
from raven.agent.hook import AgentHook, HookDecision
class NoteHook(AgentHook):
fired: list = []
@property
def name(self) -> str:
return "demo-note"
async def before_iteration(self, ctx):
NoteHook.fired.append(ctx.iteration)
return HookDecision()
def make(ctx):
return NoteHook()
"""
@pytest.mark.asyncio
async def test_a_plugin_hook_boards_the_built_loops_chain(tmp_path: Path, monkeypatch) -> None:
"""End to end: a plugin directory's hook is in the runtime's chain and
fires on the loop's iteration phase."""
from raven.config.raven import RavenConfig
from raven.config.schema import Config
from raven.contracts.llm_provider import LLMResponse
from raven.core import plugin_stack, runtime
from raven.providers.base import LLMProvider
from raven.spine import ChatType, Origin, Source, TurnRequest
plug = tmp_path / "plugins" / "demohook"
plug.mkdir(parents=True)
plug.joinpath("raven-plugin.toml").write_text(
'[plugin]\nid = "demohook"\nversion = "1.0"\n'
"[[plugin.contributes.hooks]]\n"
'name = "demo-note"\nfactory = "demohook_mod:make"\n'
)
plug.joinpath("demohook_mod.py").write_text(DEMO_HOOK)
_INJECTED.add("demohook_mod")
monkeypatch.setattr(
plugin_stack,
"plugin_discovery_sources",
lambda: {
"bundled_dir": tmp_path / "none",
"user_dir": tmp_path / "plugins",
"project_dir": tmp_path / "none",
"entry_points_group": None,
},
)
monkeypatch.setattr(runtime.token_wise_stack, "install_from_config", lambda *a, **k: None)
monkeypatch.setattr(runtime.token_wise_stack, "caching_probe", lambda *a, **k: False)
class _Provider(LLMProvider):
def __init__(self) -> None:
super().__init__(api_key="test")
async def chat(
self,
messages,
tools=None,
model=None,
max_tokens=4096,
temperature=0.7,
reasoning_effort=None,
tool_choice=None,
):
return LLMResponse(content="ok", finish_reason="stop")
def get_default_model(self) -> str:
return "fake/default"
config = Config()
config.agents.defaults.workspace = str(tmp_path / "ws")
rt = runtime.build_runtime(config, RavenConfig(), provider=_Provider())
try:
names = [h.name for h in rt.loop.hooks]
assert "demo-note" in names
await rt.loop._process_message(
TurnRequest(
origin=Origin.USER,
source=Source(channel="test", chat_id="c1", sender_id="user", chat_type=ChatType.DM),
text="hello",
),
session_key="test:c1",
)
hook = next(h for h in rt.loop.hooks if h.name == "demo-note")
assert type(hook).fired == [1]
finally:
rt.discard()