## Summary EverOS 1.4.2 shipped today. It fixes what a memory server that stays up for days runs into: a LanceDB index reader leaked file descriptors until searches took seconds, an interactive search queued behind the cascade's background embedding (vector search p50 46 s under three concurrent writers, 0.60 s now), and a cascade pass sent one embedding request per entry instead of one per file. The memory plugin's exact pin moves from `everos[multimodal]==1.4.1` to `==1.4.2`, and the plugin's own version from `1.4.0` to `1.4.2` -- the same number as the EverOS it ships. Both EverOS releases declare the same 36 requirements, so nothing else in the lock follows everos down. **No adapter change.** Every internal everos symbol the adapter reaches is present in 1.4.2 with the same signature, the package layout churn underneath included: `infra.persistence.lancedb.tables` became `infra.persistence.records` and `infra.persistence.backends` became `lancedb.LanceIndexBackend`, but the adapter imports `episode_repo` and `agent_skill_repo` from the `lancedb` package, which re-exports both the way it did before. **The derived index repairs itself, once.** 1.4.2 makes a table's primary key `(app_id, project_id, id)` rather than `id` alone, and the first full cascade pass after the upgrade re-reconciles every indexed markdown file against the index, re-embedding only the rows that are missing and recording that it ran in `.index/.index_reconcile_version`. A healthy root pays one read pass and no embedding; markdown is the source either way and was never affected by the bug this fixes. The two roots EverOS says need `everos cascade rebuild` by hand are a Milvus derived index and a root where reflection already ran across projects; raven writes one project into a LanceDB root and reaches neither, though a user who pointed raven at a root of their own might. **The plugin version now equals the EverOS it pins.** The only precedent was plugin `1.2.0` pinning EverOS `1.2.3` and plugin `1.4.0` pinning EverOS `1.4.1`: the plugin tracked EverOS major.minor and kept its patch for itself, and nothing said so anywhere. The result reads as a version one release behind the thing it ships, so step 5 of the upgrade SOP now says to hold the two equal, and this upgrade skips `1.4.1` to close the gap. That step also named a test that no longer exists for the literal assertion (`test_bundled_shadows_lower_priority_source`); it is `TestPackageSurface::test_imports_clean`, and the step now lists all three files that carry the version rather than the manifest alone. Recorded in `docs/memory-plugin-architecture.md` section 7.5, which the upgrade SOP in section 7.2 asks for. ## Type - [ ] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [x] Other (external dependency upgrade) ## Verification ``` uv run --frozen --python 3.12 --all-extras pytest -q \ tests/test_everos_plugin_discovery.py tests/test_everos_backend.py \ tests/test_everos_backend_contract.py tests/test_everos_http_adapter.py \ tests/test_everos_config.py tests/test_everos_server.py \ tests/test_everos_discover.py tests/test_memory_backend_protocol.py \ tests/test_memory_backend_contract.py tests/test_cli_plugin_commands.py \ tests/test_config_update_everos.py tests/test_dependency_constraints.py \ tests/test_release_plugin_list.py tests/test_skill_router_everos_source.py \ tests/test_agent_loop_everos_pipeline.py -> 567 passed, 1 skipped (the skip is Linux-only inotify) uv run --frozen --python 3.12 --all-extras pytest -q \ tests/test_everos_plugin_discovery.py tests/test_cli_plugin_commands.py \ tests/test_release_plugin_list.py tests/test_plugin_manifest.py \ tests/test_plugin_discover.py tests/test_everos_discover.py -> 91 passed (re-run after the plugin version moved to 1.4.2) make lint-types -> all checks passed make lint-python lint-imports lint-deps -> clean, 10 import contracts kept make check-source-language check-large-files COMMIT_RANGE=github/main..HEAD -> clean ``` Against a copy of a real 55 MB store (568 episodes, seven LanceDB tables) written under 1.2.x and last served by 1.4.1, booted on a scratch root: - 1.4.2 opens it with no schema complaint, `/health` answers `version 1.4.2` with `cascade.healthy true`, and the reconcile pass writes `.index/.index_reconcile_version` with no error in 441 log lines. - The same keyword search (`POST /api/v2/memory/search`, the body the adapter sends) returns the same five episode ids in the same order under 1.4.1 and under 1.4.2; BM25 scores drift in the third decimal, as they did on the 1.2.3 to 1.4.1 move. - Raven's own `installed_everos_version()` reads 1.4.2, and every internal symbol the adapter imports resolves with the signature it had. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [x] User-facing docs or screenshots are updated when needed ## Risk User-visible change: memory searches stop queueing behind background writes, and a long-lived server stops leaking file descriptors. An existing index pays one read pass on the first start after the upgrade. Rollback is `git revert`. The reconcile pass only re-embeds rows the old primary key had lost, and `lancedb` does not move, so 1.4.1 reads an index 1.4.2 has written. The new `.index/.index_reconcile_version` file is ignored by 1.4.1; a re-upgrade later would skip the pass it records, which is the intent. Not exercised here: a Milvus derived index (raven configures LanceDB), a multi-project root, and a real embedding or LLM round trip (the scratch root ran without provider keys, so the checks above are the keyword leg and the cascade's own reconcile). - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com> Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
336 lines
12 KiB
Python
336 lines
12 KiB
Python
"""Tests for ``raven plugins`` and ``raven plugin auth``.
|
|
|
|
``plugins`` is read-only: it renders the activated plugins -- here the
|
|
registered entry-points one, raven_everos -- and resolves
|
|
``config.memory.backend`` against the live registry without invoking any
|
|
plugin runtime (no ``MemoryBackend.start`` is awaited). These tests pin the
|
|
table's contents and the three branches of the backend-selection block:
|
|
present-active, present-unknown, and explicitly-disabled.
|
|
|
|
``plugin auth`` is a chain of refusals in front of one network call: an
|
|
unknown server, a disabled one, and one not configured for OAuth are each
|
|
rejected before anything connects, and a connect that comes back in any state
|
|
other than ``connected`` exits non-zero. Only the last of those needs a
|
|
manager, and it gets a fake one -- the real flow opens a browser.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import os
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
import pytest
|
|
from typer.testing import CliRunner
|
|
|
|
from raven.cli.commands import app
|
|
from raven.config.loader import set_config_path
|
|
|
|
runner = CliRunner()
|
|
|
|
# Through the root app rather than `plugin_app` directly: a Typer group with a
|
|
# single command collapses into that command, so invoking the sub-app with
|
|
# ["auth", name] parses "auth" as the server argument.
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ``raven plugin auth`` -- refusals in front of the one network call
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.fixture
|
|
def config_with(tmp_path: Path):
|
|
"""Write a config carrying the given mcp_servers block, and point the
|
|
loader at it. The command reads the loader's own path, not an argument."""
|
|
|
|
def _write(servers: dict) -> Path:
|
|
path = tmp_path / "config.json"
|
|
path.write_text(json.dumps({"tools": {"mcpServers": servers}}))
|
|
set_config_path(path)
|
|
return path
|
|
|
|
yield _write
|
|
set_config_path(None) # type: ignore[arg-type]
|
|
|
|
|
|
def test_an_unknown_server_is_refused_before_anything_connects(config_with) -> None:
|
|
config_with({})
|
|
|
|
r = runner.invoke(app, ["plugin", "auth", "ghost"])
|
|
|
|
assert r.exit_code == 1
|
|
assert "no MCP server named" in r.stdout
|
|
|
|
|
|
def test_a_disabled_server_is_refused(config_with) -> None:
|
|
config_with({"svc": {"url": "https://svc.example/mcp", "auth": "oauth", "enabled": False}})
|
|
|
|
r = runner.invoke(app, ["plugin", "auth", "svc"])
|
|
|
|
assert r.exit_code == 1
|
|
assert "disabled" in r.stdout
|
|
|
|
|
|
def test_a_server_not_configured_for_oauth_is_refused(config_with) -> None:
|
|
"""The message has to name the current mode, or the fix is a guess."""
|
|
config_with({"svc": {"url": "https://svc.example/mcp", "auth": "apikey", "enabled": True}})
|
|
|
|
r = runner.invoke(app, ["plugin", "auth", "svc"])
|
|
|
|
assert r.exit_code == 1
|
|
assert "apikey" in r.stdout
|
|
|
|
|
|
class _FakeManager:
|
|
"""Stands in for MCPConnectionManager: the real `connect` opens a browser."""
|
|
|
|
def __init__(self, snapshot: dict):
|
|
self._snapshot = snapshot
|
|
self.closed = False
|
|
|
|
async def connect(self, name: str, cfg) -> dict:
|
|
return self._snapshot
|
|
|
|
async def aclose(self) -> None:
|
|
self.closed = True
|
|
|
|
|
|
def _patch_manager(monkeypatch: pytest.MonkeyPatch, snapshot: dict) -> list:
|
|
made: list = []
|
|
import raven.mcp.manager as mgr_mod
|
|
|
|
def _factory(registry):
|
|
m = _FakeManager(snapshot)
|
|
made.append(m)
|
|
return m
|
|
|
|
monkeypatch.setattr(mgr_mod, "MCPConnectionManager", _factory)
|
|
return made
|
|
|
|
|
|
def test_a_connected_snapshot_reports_the_tool_count(config_with, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
config_with({"svc": {"url": "https://svc.example/mcp", "auth": "oauth", "enabled": True}})
|
|
made = _patch_manager(monkeypatch, {"state": "connected", "tool_count": 3, "error": None})
|
|
|
|
r = runner.invoke(app, ["plugin", "auth", "svc"])
|
|
|
|
assert r.exit_code == 0, r.stdout
|
|
assert "authorized" in r.stdout
|
|
assert "3 tools" in r.stdout
|
|
assert made[0].closed, "the manager must be closed even on the happy path"
|
|
|
|
|
|
def test_a_failed_authorization_exits_non_zero_and_names_the_reason(
|
|
config_with, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
"""`auth_required` back from connect means the flow did not complete; the
|
|
command must not report success just because the call returned."""
|
|
config_with({"svc": {"url": "https://svc.example/mcp", "auth": "oauth", "enabled": True}})
|
|
made = _patch_manager(monkeypatch, {"state": "auth_required", "tool_count": 0, "error": "denied by user"})
|
|
|
|
r = runner.invoke(app, ["plugin", "auth", "svc"])
|
|
|
|
assert r.exit_code == 1
|
|
assert "authorization failed" in r.stdout
|
|
assert "denied by user" in r.stdout
|
|
assert made[0].closed, "the manager must be closed on the failure path too"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# ``raven plugins`` -- the listing table and the backend-selection block
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _make_runner_args(tmp_path: Path, config: dict[str, Any]) -> list[str]:
|
|
"""Write a config file + return the typer args to point at it."""
|
|
config_path = tmp_path / "config.json"
|
|
config_path.write_text(json.dumps(config), encoding="utf-8")
|
|
# ``raven plugins -c <path>``
|
|
return ["plugins", "-c", str(config_path)]
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _wide_console(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""Give every command in this file a console wide enough to print a
|
|
factory reference.
|
|
|
|
``COLUMNS`` in the invocation's environment is not enough on its own. The
|
|
command renders through ``plugin_commands.console``, a module-level
|
|
``Console()`` whose width is fixed when the module is first imported, so
|
|
the variable only lands if this file is what imported it -- which held
|
|
until the suite started running in parallel, where another worker's test
|
|
gets there first and the width is Rich's default 80. The factory column
|
|
then truncates to ``...`` and the assertions read as a missing string
|
|
rather than a narrow terminal.
|
|
|
|
Reproduce by importing the CLI at 80 columns before the test runs: the
|
|
failure is `assert 'raven_everos.backend:make_backend' in`
|
|
a string that ends `...___/`.
|
|
"""
|
|
from rich.console import Console
|
|
|
|
from raven.cli import plugin_commands
|
|
|
|
monkeypatch.setattr(plugin_commands, "console", Console(width=200))
|
|
|
|
|
|
def _invoke(args: list[str], tmp_path: Path):
|
|
"""Run the CLI with a sandboxed HOME so user-level plugin discovery
|
|
doesn't surface unrelated plugins on the developer's machine.
|
|
|
|
``COLUMNS=200`` is kept for the child's own sake -- anything the command
|
|
renders through a console it builds itself reads it -- but the table this
|
|
file asserts on comes from the module-level console that ``_wide_console``
|
|
replaces.
|
|
"""
|
|
fake_home = tmp_path / "home"
|
|
fake_home.mkdir(parents=True, exist_ok=True)
|
|
env = {**os.environ, "HOME": str(fake_home), "COLUMNS": "200"}
|
|
return runner.invoke(app, args, env=env)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Default config -- backend selected and available
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestActiveBackend:
|
|
def test_lists_everos_memory(self, tmp_path: Path) -> None:
|
|
result = _invoke(
|
|
_make_runner_args(tmp_path, {}),
|
|
tmp_path,
|
|
)
|
|
assert result.exit_code == 0, result.stdout
|
|
assert "everos-memory" in result.stdout
|
|
assert "1.4.2" in result.stdout
|
|
assert "entry_points" in result.stdout
|
|
|
|
def test_shows_active_backend_with_track_ids(
|
|
self,
|
|
tmp_path: Path,
|
|
) -> None:
|
|
result = _invoke(
|
|
_make_runner_args(
|
|
tmp_path,
|
|
{
|
|
"memory": {
|
|
"backend": "everos",
|
|
"userId": "alice",
|
|
"agentId": "robo",
|
|
},
|
|
},
|
|
),
|
|
tmp_path,
|
|
)
|
|
assert result.exit_code == 0
|
|
assert "Active memory backend" in result.stdout
|
|
assert "everos" in result.stdout
|
|
assert "from plugin: everos-memory" in result.stdout
|
|
assert "User id:" in result.stdout
|
|
assert "alice" in result.stdout
|
|
assert "Agent id:" in result.stdout
|
|
assert "robo" in result.stdout
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Backend explicitly disabled
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestBackendDisabled:
|
|
def test_null_backend_reports_none(self, tmp_path: Path) -> None:
|
|
result = _invoke(
|
|
_make_runner_args(tmp_path, {"memory": {"backend": None}}),
|
|
tmp_path,
|
|
)
|
|
assert result.exit_code == 0
|
|
assert "Active memory backend" in result.stdout
|
|
assert "none" in result.stdout
|
|
# The legacy-fallback hint surfaces.
|
|
assert "legacy" in result.stdout
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Backend name set but no contribution matches
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestBackendUnavailable:
|
|
def test_unknown_backend_flagged(self, tmp_path: Path) -> None:
|
|
result = _invoke(
|
|
_make_runner_args(
|
|
tmp_path,
|
|
{
|
|
"memory": {"backend": "nonexistent"},
|
|
},
|
|
),
|
|
tmp_path,
|
|
)
|
|
assert result.exit_code == 0
|
|
assert "nonexistent" in result.stdout
|
|
assert "not available" in result.stdout
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Plugin disabled list shows in table
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestDisabledList:
|
|
def test_disabled_plugin_status(self, tmp_path: Path) -> None:
|
|
result = _invoke(
|
|
_make_runner_args(
|
|
tmp_path,
|
|
{
|
|
"plugins": {"disabled": ["everos-memory"]},
|
|
},
|
|
),
|
|
tmp_path,
|
|
)
|
|
assert result.exit_code == 0
|
|
assert "everos-memory" in result.stdout
|
|
# The status column shows "disabled" for the row.
|
|
assert "disabled" in result.stdout
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# A plugin that fails to load gets its own row, not the whole table
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestFailedPlugin:
|
|
def test_a_broken_plugin_reads_failed_with_its_reason(self, tmp_path: Path) -> None:
|
|
package = "_cli_wreck_pkg"
|
|
plug = tmp_path / "shelf" / "wreck"
|
|
(plug / package).mkdir(parents=True)
|
|
(plug / package / "__init__.py").write_text("raise ImportError('boom [red]')\n", encoding="utf-8")
|
|
(plug / "raven-plugin.toml").write_text(
|
|
'[plugin]\nid = "wreck"\nversion = "0.1.0"\n'
|
|
f'[[plugin.contributes.tools]]\nname = "wreck_tool"\nfactory = "{package}:make"\n',
|
|
encoding="utf-8",
|
|
)
|
|
|
|
result = _invoke(_make_runner_args(tmp_path, {"plugins": {"dirs": [str(tmp_path / "shelf")]}}), tmp_path)
|
|
|
|
assert result.exit_code == 0, result.stdout
|
|
rows = {line.split()[1]: line for line in result.stdout.splitlines() if line.startswith("│")}
|
|
assert "failed" in rows["wreck"]
|
|
assert "activated" in rows["everos-memory"]
|
|
assert "wreck failed to load: plugin 'wreck': importing '_cli_wreck_pkg' failed: boom [red]" in result.stdout
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Verbose flag shows factory references
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class TestVerboseFlag:
|
|
def test_verbose_shows_factory(self, tmp_path: Path) -> None:
|
|
args = _make_runner_args(tmp_path, {})
|
|
args.append("--verbose")
|
|
result = _invoke(args, tmp_path)
|
|
assert result.exit_code == 0
|
|
# The factory reference is the canonical ``module:callable`` form.
|
|
assert "raven_everos.backend:make_backend" in result.stdout
|