Raven-1/tests/test_plugin_boundary.py
zhanghui d106a5c6ef
feat(plugins): add the everos-cloud-memory plugin and its host wiring (#883)
## Summary

Adds `everos-cloud-memory`, a separately distributed memory plugin for
EverOS Cloud, and wires the host so a second memory plugin is a
first-class choice.

The plugin (`plugins-dist/everos-cloud-memory`) is a thin REST client
over httpx: one API key (the config slice
`plugins.config.everos-cloud-memory.api_key`, or
`EVEROS_CLOUD_API_KEY`), owner ids from the host's `ServiceLocator`
(never from the slice), every add in `agent` mode, and a flush only when
a caller marks the conversation over (`metadata.flush` /
`metadata.is_final`: the sub-agent handoff and the importer's final
batch) and once at stop. It ships its own onboard screen (env or on-file
key used without a prompt; 401 / 403 / unreachable get their own
sentences with Re-enter or Skip), `delete` answers False (the cloud
deletes by scope, nothing calls it), and `health()` classifies seven
answers for `raven doctor` and the settings card.

Host changes, each guarded so `memory.backend` `everos` and `null`
behave exactly as before:

- `raven onboard` step 4 asks "Which memory backend?" when more than one
plugin contributes a screen (preselects the current backend, `Off`
clears it, cancel exits 1 and writes nothing); with one plugin the step
is byte-identical to today.
- The memory page reads whichever backend `memory.backend` names: the
cloud slice's key goes out as a Bearer header and the `/health` probe is
skipped for the cloud; a missing distribution is named in the page's
note.
- `settings.everos` reports `reason: "other_backend"` for any non-EverOS
backend, so the Model page hides the three EverOS-only roles (embedding
stays: knowledge bases use that pin); `settings.everosSet` refuses those
three sections for another backend; `settings.set` admits
`plugins.config.<id>.<field>` against the plugin's manifest through the
same admission door channels use; new `settings.everosCloud` feeds the
card (key source, endpoint, health).
- Provider-key and main-model saves no longer restart the local EverOS
when another backend is active.
- ui-web: an EverOS Cloud card (status chip, key row, endpoint) and the
`settings.everosCloud` data behind it. No page mounts the card: it drew
on Model settings until the owner ruled that a memory backend's endpoint
and credential answer neither question that page asks, and where it
belongs is the Memory section's to say once that section has a design.
Until then the key is entered through `raven onboard` or
`EVEROS_CLOUD_API_KEY`. The fixture gate pins the four optional fields
the offline library never sends.
- Gates: the plugin boundary test scans both memory plugins, coverage
and the release workflow build the second wheel, deptry's ignore list
names the plugin beside `raven_everos`, and a conftest fixture keeps the
second plugin out of entry-point discovery for the existing onboarding
tests (they describe a one-plugin install).

A second topic rides along, because it is what makes the hosted backend
reachable from an agent product at all. Every product under `agents/`
pinned its memory at the stock EverOS address and only `raven-design`
read the host's, so the other four addressed whatever answered on 18791
-- the wizard moves a managed server when the port is taken, and a
server the user runs themselves is wherever they put it.
`inherit_memory_backend` in `raven/config/product_render.py` now carries
`memory.backend`, the endpoint, and (only when the product holds none
itself) the credential from the host into every render, beside the host
denials and plugin opt-outs that already travel there; `raven-design`'s
private copy is deleted and the five product configs stop naming an
address. Identity never travels, which is what keeps five products apart
inside one store. `memory.backend: null` travels like any other value,
because that null is how an operator turns long-term memory off and a
product that kept writing would be the one surface ignoring them; the
product's own default stands only when the host carries no key at all.
Which store memories live in is not a product's choice either: the host
reads a sub-agent's memories back through its own backend, so a product
writing anywhere else left that record empty with nothing reporting it.
`owned` is forced off whenever the local backend is in force, because a
product infers ownership from a slice naming no root -- the shape all
five ship -- and so answered a host that was merely not serving by
starting a second server against a root of its own, stranding every
memory written so far.

Decisions worth a reviewer's eye: the fake cloud
(`tests/_everos_cloud_fake.py`) honours or refuses every field the
backend can send and serves both an in-process transport and an HTTP
face for the real-host acceptance; the live tests
(`tests/integration/test_everos_cloud_real_cloud.py`) skip without
`EVEROS_CLOUD_API_KEY`; run with a real key on 2026-10-09 all six pass
against api.evermind.ai (round trip, get by session, empty flush, 401
wording, probe with and without a key, and extraction without a flush
within two minutes), so the documentation and the service agree on
everything the plugin sends. Design, acceptance cases and plan are under
`docs/specs/2026-10-08-everos-cloud-memory-*.md` and `docs/plans/`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Rebased onto main `182dacac0` on 2026-10-13. What that rebase and the
four commits on top were verified with:

- Full suite, `uv run --frozen --python 3.12 --extra dev pytest -q` ->
**29748 passed, 8 failed** in 4m10s. All eight fail identically with
these four commits' sources reverted in-tree at the same base, so none
of them is this branch's: the two `test_the_products_tool_face_*` cases
in the code and oncall launchers (an extra `web_search` in the visible
tool set), `test_agent_loop_token_budget`,
`test_rpc_files::test_a_host_without_libreoffice_says_so`,
`test_ppt_engine_image_search`, `test_shell_background`, and the two
`test_simulation_*` cases.
- Real-host before and after for the inheritance, driving the launcher
the host spawns (`agents/raven-research/run.py`) against an isolated
`RAVEN_HOME` whose EverOS address is `19701`, not the stock port.
Before: the product rendered `base_url: http://localhost:18791` with no
`owned` key, ignored the host, and started an EverOS of its own -- an
orphaned `everos server start --root <its own state root>/everos`
holding 18791, which is the stranding this change exists to stop. After:
it rendered `{"base_url": "http://localhost:19701", "owned": false}` and
said so itself, "the EverOS you manage is not running at
http://localhost:19701 ... Raven does not start or stop it", with `ps`
and `lsof` confirming nothing was started. On the cloud the product
rendered `memory.backend: everos-cloud` with the host's key and left the
out-of-force local slice untouched, and `raven doctor` over that
rendered config answered `everos-cloud: ok https://api.evermind.ai`, so
the credential reached the service. Identity stayed `raven-research-ng`
throughout, and a fingerprint of the operator's own EverOS root was
identical before and after.
- The first CI round on this head found one that the local gates did
not: `tests/test_import_cycle_budget.py`. `inherit_memory_backend` read
the backend-to-distribution pairing from `raven.core.plugin_stack`, and
that edge from `raven.config` into `raven.core` -- which imports
`raven.config` back -- took mutual pairs from 13 to 14 and packages
inside a cycle from 18 to 22. `make lint-imports` keeps ten contracts
and none of them covers that pair, so it was green throughout. The
pairing now lives in `raven/config/raven.py` beside the two fields it
joins (`MemoryConfig.backend` names a contribution,
`PluginsConfig.config` is keyed by the distribution), which `raven.core`
may import and which imports nothing back; `plugin_stack` re-exports the
two names its own callers read from it. An intermediate commit put it in
`raven.contracts` instead, which broke the cycle but took the papers
ceiling from 3700 to 3719 lines -- `tests/test_kernel_budget.py` says a
new paper is a reviewed change to that ceiling, so it moved again rather
than raising it.
- The review round on this head raised one non-blocking finding, fixed
rather than carried forward: the inheritance read `memory.backend` with
`.get()`, which collapses an explicit `null` into an absent key, so a
host with long-term memory off left every packaged agent on its own
shipped backend and still writing. Reproduced by calling the function
directly before the change and after, now read on presence, and pinned
by a case whose mutation (restoring the `.get()`) turns it and nothing
else red.
- One ordering problem found while checking the above, pre-existing and
not fixed here: `tests/test_rpc_settings.py` run before
`tests/test_cli_onboard_commands.py` leaves
`raven.home._current_config_path` pointing at a dead tmp config, so the
onboard cases write somewhere else. With main's own copies of both files
and main's conftest in the tree it is 57 failures; with this branch's,
19. Which cases turn red changes run to run as xdist distributes them
differently.
- `cd ui-web && npx vitest run` -> 876 files, 3728 passed, 0 failed;
`npm run lint` -> 0 errors (4 warnings, all in files this change does
not touch); `npm run gen:check` -> in sync (228 methods); `npm run
type-check` -> clean; `cd ui-tui && npm run lint:i18n && npm run
lint:rpc` -> in sync
- `make lint-python` / `lint-imports` (10 contracts kept) / `lint-types`
/ `lint-deps` -> all pass; `uv lock --check` -> exit 0 with both sides'
new packages present; `scripts/check_source_language.py` and
`scripts/check_large_files.py` on `github/main..HEAD` -> exit 0;
commitlint over every commit -> ok
- Semantic-conflict audit of the rebase, beyond what the suite answers:
21 of the branch's 66 files are also in main's 499, and every one of
main's changes in them is additive and elsewhere in the file. The one
that could have broken silently is main's new `vision.model` /
`vision.provider` simple keys meeting this branch's new
`plugins.config.<id>.<field>` branch in `settings_set`; nothing in the
suite exercises `vision.*` through that method, so it was run directly
-> `{'applied': True}` and `{"model": "..."}` on disk. The subtraction
decisions were re-checked against the new base: no new caller of a
memory backend's `delete`, main's new onboard cases are about TLS and
not the memory step, `inherit_everos_address` exists nowhere, and
`DEFAULT_EVEROS_BASE_URL` is still defined.
- Mutation runs on the new code, restored afterwards: dropping `owned:
false`, letting the host's credential overwrite the product's own, not
writing the host's backend name, and carrying every memory slice instead
of the one in force each turn their own cases red; making the cloud card
render nothing turns 4 of its 5 new cases red and leaves the fifth
("draws nothing when not selected") green.

From the earlier round, on base `6d509a8c5`:

- `uv run --frozen --python 3.12 --extra dev pytest -p no:cacheprovider
-q tests/test_everos_cloud_fake.py tests/test_everos_cloud_backend.py
tests/test_everos_cloud_backend_contract.py
tests/test_everos_cloud_onboard.py tests/test_everos_cloud_discover.py
tests/test_cli_onboard_commands.py tests/test_rpc_memory.py
tests/test_rpc_settings.py tests/test_plugin_boundary.py
tests/test_rpc_schema_match.py tests/test_rpc_registration.py
tests/test_release_plugin_list.py
tests/integration/test_everos_cloud_real_cloud.py` -> 1001 passed, 6
skipped (the six live tests, no key)
- `cd ui-web && npx vitest run` -> 214 files, 3211 passed; `npm run
lint` -> 0 errors (4 warnings, all in files this change does not touch);
`npm run gen:check` -> in sync (207 methods); `cd ui-tui && npm run
lint:rpc && npm run lint:i18n` -> in sync
- `make lint` (ruff, ruff format, ui tsc, deptry) -> ok; `make
lint-types` -> All checks passed; `make build-ui` -> boot snapshots
match; `scripts/check_large_files.py` and
`scripts/check_source_language.py` on `6d509a8c5..HEAD` -> exit 0;
commitlint and `scripts/check_commit_messages.py` on each commit -> ok
- Mutation runs per change (restored afterwards, `git diff` clean): a
host import of the plugin turns the boundary test red; `import
raven_everos` in the plugin turns the import-isolation test red;
removing the chooser's cancel exit, the Bearer header, the restart
guard, the slice-admission guard, the Roles filter and the fake's two
fixes each turn their own test red
- Real-host acceptance (64 cases,
`docs/specs/2026-10-08-everos-cloud-memory-acceptance.md`) run against a
fake EverOS Cloud: the wizard in tmux (chooser, key typed / from env /
on file, 401 / 403 / unreachable wording, back, Off, Ctrl-C,
non-interactive, base-tree transcript byte-identical with one plugin),
`raven agent` turns (user and agent-track recall, agent-mode adds, 4 s
recall bound, a 300 KB message dropped with a warning, a sub-agent trace
stored under its own ids, flushed and read back, one exit flush per
unflushed session), `raven import run` (3 adds then one flush; a hung
flush retried 4x and the exit sweep defeated), `raven doctor` under
seven cloud conditions (seven sentences, exit 2 only for the five
faults) and with the distribution hidden, and the web page through
playwright-cli (memory page counts and the fake's episode, empty and
refused stores, the Model page hiding the three EverOS-only roles while
the embedding row stays, the cloud card rotating a key and showing the
401 wording, three refused settings.set frames over the gateway
WebSocket, the local everos / null / mem0 backends drawing the page as
before, and a provider-key save that restarts the local EverOS on the
base tree and does not on this branch). Evidence and the per-case
verdicts are kept outside the repo. The live service was reached with a
real key after the acceptance run: six tests, six passes, the service's
replies archived with the evidence.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible changes: the wizard's step 4 chooser (only when two memory
plugins are installed); Model settings hides the three EverOS-only role
rows when `memory.backend` is not `everos`/`null` and draws no cloud
card; the memory page sends a Bearer header for the cloud backend.

Two behaviour changes for agent products, both on the `agents/` lane
only. An agent product now follows the host's `memory.backend` and
endpoint instead of its own, which is a fix for the four products that
never followed the host and a change for `raven-design`, which followed
the endpoint but not the backend. And a product no longer starts an
EverOS server of its own: where it used to answer a host that was not
serving by starting one against an empty root of its own, it now reports
long-term memory as unavailable, which is what the host does. Rollback
for this part is the one commit; the product configs it edits are
shipped files, not user config. The plugin is not installed by default
(its wheel is built by the release workflow; nothing changes for an
install without it). Rollback: revert the squash commit; the config
schema gains no new key (the slice is a plugin slice), so an older raven
reads a config written by this one.

Credentials: the cloud key is written only by `set_plugin_config_fields`
into the plugin's slice; the settings page's key row goes through
`settings.set` with a manifest-declared field, and
`settings.everosCloud` reports `api_key_set` / `key_source`, never the
value.

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 11:08:11 +08:00

88 lines
3.7 KiB
Python

"""Dependency direction between the host and the bundled everos plugin.
The host may know the plugin only through the plugin contract
(``raven.contracts.memory`` and ``raven.plugins``); it must not import the
``raven_everos`` package. The plugin may use the host's public helpers --
``raven.config.update`` included, since the ``plugins.config`` slice it writes
there is its own data -- but must not reach into host modules that exist for
the plugin's sake. The list below holds the violations still standing; a task
that removes one deletes its entry here, and the final task asserts it is
empty.
"""
from __future__ import annotations
import re
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
HOST_DIR = REPO_ROOT / "raven"
# Every distribution beside the host wheel, by its package directory: a scan
# that knew one plugin by name would let any other import whatever it liked.
PLUGIN_DIRS = sorted(d for d in (REPO_ROOT / "plugins-dist").glob("*/raven_*") if d.is_dir())
_PLUGIN_IMPORT = re.compile(r"^\s*(from|import)\s+raven_everos(_cloud)?\b", re.M)
_HOST_PRIVATE = re.compile(
r"^\s*(?:from\s+raven\.(cli|config\.loader|config\.raven)\b|import\s+raven\.(cli|config\.loader|config\.raven)\b)",
re.M,
)
# The two GUI surfaces still shaped around EverOS's own store: the memory
# browser reads its four kinds over EverOS's HTTP API, and the settings page
# edits the model roles behind EverOS's extraction. Both need the wire contract
# generalised before a second backend could answer them, which is a frontend
# change; everything else the host does now goes through MemoryBackend.
HOST_WIRE_PROTOCOL_SURFACES: frozenset[str] = frozenset(
{
"raven/rpc/methods/console.py",
"raven/rpc/methods/memory.py",
}
)
def _rel(p: Path) -> str:
return str(p.relative_to(REPO_ROOT))
def _host_files() -> list[Path]:
return list(HOST_DIR.rglob("*.py"))
def _plugin_files() -> list[Path]:
return [f for d in PLUGIN_DIRS for f in d.rglob("*.py")]
def test_scan_roots_exist() -> None:
assert _host_files(), f"no host sources under {HOST_DIR}: the scan would pass vacuously"
assert {"raven_everos", "raven_everos_cloud"} <= {d.name for d in PLUGIN_DIRS}, PLUGIN_DIRS
for d in PLUGIN_DIRS:
assert any(d.rglob("*.py")), f"no plugin sources under {d}: the scan would pass vacuously"
def test_host_does_not_import_plugin_internals() -> None:
offenders = {_rel(p) for p in _host_files() if _PLUGIN_IMPORT.search(p.read_text(encoding="utf-8"))}
assert offenders == set(HOST_WIRE_PROTOCOL_SURFACES), (
f"new host->plugin imports: {sorted(offenders - HOST_WIRE_PROTOCOL_SURFACES)}; "
f"stale entries: {sorted(HOST_WIRE_PROTOCOL_SURFACES - offenders)}"
)
def test_plugin_does_not_import_host_private_modules() -> None:
offenders = {_rel(p) for p in _plugin_files() if _HOST_PRIVATE.search(p.read_text(encoding="utf-8"))}
assert offenders == set(), f"new plugin->host imports: {sorted(offenders)}"
def test_nothing_under_raven_decides_ownership() -> None:
"""Who may write an EverOS root is the plugin's judgement, never the host's.
A20's only acceptance evidence. The host asking `everos_owned` for itself is
how the guard came to live in two places with one of them out of date -- the
write primitives are where it belongs, so a new caller cannot opt out of it.
A grep, because the property is about absence: nothing under `raven/` may
name it, and a test that only checked the callers it knows about would pass
the moment somebody adds one it does not.
"""
offenders = {_rel(p) for p in _host_files() if re.search(r"\beveros_owned\b", p.read_text(encoding="utf-8"))}
assert offenders == set(), f"the host is deciding EverOS ownership in: {sorted(offenders)}"