Raven-1/tests/_everos_env.py
Handsome-wzw fa3cf3821e
test(tests): put the EverOS role binding back after every test (#964)
## Summary

`bind_roles_here` writes the four EverOS roles into `os.environ`
directly, records them in `raven_everos.config._BOUND_HERE` and
publishes that record as `RAVEN_EVEROS_BOUND`. `monkeypatch` undoes none
of it: it records only what it changed itself, and `delenv(...,
raising=False)` on a name that is not there records nothing. The everos
backend tests have put the `EVEROS_*` variables back since #489, but not
the record or the set, and the binding is not theirs alone: every
runtime assembled with the memory plugin on binds through the plugin's
start. A run of the whole suite with a watcher on these three things
(2026-10-15) found fourteen files binding -- the design and ppt
launchers, eval hooks, plugin tools, remote file tools, direct chat,
onboarding, the TUI command tests. So one test's binding was the next
test's environment, and which test paid depended on how the suite was
dealt into shards: ten `tests/test_rpc_settings.py` save cases refused
with "decided by the `EVEROS_LLM__*` environment variables" in shard 4/4
of #920 (whose tree predated #489), and nowhere else.

What this does, tests only:

- `tests/_everos_env.py`: `confined_everos_binding()`, a block that puts
the variables (`EVEROS_ROOT` included), the provenance variable and the
contents of the provenance set back when it ends, exception or not; and
`forget_everos_binding(monkeypatch)`, which starts a reader with none of
them and swaps the set rather than clearing it.
- `tests/conftest.py`: one autouse fixture, `_no_leaked_everos_binding`,
around every test, next to the other process-global restores
(`_unbind_the_permission_turn`, `_no_leaked_usage_sink`). It replaces
the everos backend file's own fixture, which covered the variables
alone.
- `tests/test_rpc_settings.py` and `tests/test_config_update_everos.py`:
the two readers of `role_is_env_managed` start with no role of any kind
exported, not only no embedding, through the helper.
- `tests/test_everos_env_confinement.py`: the helper's contract (a
binding made inside the block is undone; what was there before comes
back; so on an exception; the forget half), and one slow test per
representative file that, in a child interpreter started clean and
running under the suite's own conftest, checks that the file leaves no
variable, no record and an empty set behind. Red on the tree before this
change for all three targets (the backend file left the record; the
launcher case left twelve empty role variables, `EVEROS_ROOT` under a
`tmp_path` that is gone, and the record).

The board's own reproduction (the named everos case, then
`tests/test_rpc_settings.py`, one process) does not fail on main any
more because of #489's fixture; it was kept as a check and passes.

Third commit, after review: the block restored the contents of whichever
set the attribute held when it ended. That is the wrong object for a
test that binds and then swaps the attribute with `monkeypatch`, because
`monkeypatch`'s undo runs after an autouse fixture's teardown here
(`_isolate_tracing_state_dir` asks for it and is set up earlier; autouse
fixtures of one conftest are set up in name order): the swapped-in set
got cleaned and the module's own came back with everything bound into it
-- a probe handed the next test twelve keys. No test in the suite binds
before it swaps, so nothing failed, but the helper's docstring claimed
the reverse order. The block now keeps the object it began with and
restores that object's contents and the attribute's binding to it,
whatever the attribute holds by then; the docstring says why, and a test
that binds, swaps with a MonkeyPatch undone only after the block, and
reads a clean set is red before the change.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [x] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- A throwaway pytest plugin recording, per test, every change to
`EVEROS_*`, `RAVEN_EVEROS_BOUND` and `_BOUND_HERE`, over the whole suite
(`-n 6`): before this change 63 tests in 16 files changed them (14
binding, 2 clearing); after it, the watcher wrote nothing: no test
changed them (31719 passed, same run).
- `pytest tests/test_everos_env_confinement.py -n 0`: 8 passed, with and
without `CI=1`; the three child-interpreter cases fail on the tree
before this change with the leftovers named above. (The first push had
them print the report after `pytest.main`, which on CI never returns --
the suite's conftest hard-exits the interpreter in `pytest_unconfigure`
when `CI` is set -- so shard 1/4 failed on "the child did not report"
while the targets had passed; `CI=1` reproduces that locally. The child
now writes the report from a `pytest_sessionfinish` plugin into a file
the parent reads.)
- `tests/test_everos_backend.py` 191 passed,
`tests/test_agents_ppt_launcher.py` 61 passed,
`tests/test_rpc_settings.py` 153 passed,
`tests/test_config_update_everos.py` 50 passed, each on its own; the
board's reproduction order 154 passed; the fourteen binding files
followed by every reader in one process, 1261 passed.
- Full suite on the merged tree (`-n 6`): after the third commit, 31720
passed, 173 skipped, 2 xfailed, 3 failed -- the same three that fail on
main on this machine (a LibreOffice case and two simulation cases); the
related group (`test_everos_backend`, `test_rpc_settings`,
`test_config_update_everos`, `test_everos_config`, `test_everos_server`,
the two launcher/eval-hook files) 651 passed.
- `ruff check`/`ruff format`, `scripts/check_source_language.py`,
`scripts/check_large_files.py`, `scripts/check_commit_messages.py`:
pass. No production code changes, so the diff coverage gate has no
changed executable lines to measure.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

- [ ] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

Tests only. A test that asserted on a binding left by an earlier test
would now see it gone; the full-suite run says none does. Rollback is
reverting the commit.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-16 18:19:20 +08:00

82 lines
3.7 KiB
Python

"""Run a block that binds the EverOS roles into this process, and put the process back after it.
``bind_roles_here`` (:mod:`raven_everos.config`) is reached by
``EverosBackend.start()`` and by the plugin's tool factory
(:mod:`raven_everos.tools`), so by any runtime built with the memory plugin
on. It writes the four roles into ``os.environ`` directly, records them in
``_BOUND_HERE`` and publishes that record as ``RAVEN_EVEROS_BOUND`` so it
survives ``os.execv``; ``configure_everos_env`` sets ``EVEROS_ROOT`` beside
them. ``monkeypatch`` undoes none
of it: it records only what it changed itself, and ``delenv(..., raising=False)``
on a name that is not there records nothing. So a binding made in one test was
still in the process when the next began, and which test paid depended on how
the suite was dealt into shards -- a settings test reading a saved endpoint was
told the environment outranks it (``role_is_env_managed``) and refused the
write, in one shard and not in another.
Three things go back together, because the readers read them together: the
``EVEROS_*`` variables (the binding, and ``EVEROS_ROOT``), the provenance
variable, and the provenance set -- restored as the object the block began
with, its contents and the attribute's binding to it. Not whichever object the
attribute holds when the block ends: a test may bind into the module's own set
and then swap the attribute with ``monkeypatch``, and ``monkeypatch``'s undo
runs after an autouse fixture's teardown when a fixture set up earlier asked
for it (the root conftest's ``_isolate_tracing_state_dir`` does; autouse
fixtures of one conftest are set up in name order). Restoring only what the
attribute held then would clean the swapped-in set, about to be dropped, and
leave the module's own to come back with everything bound into it (measured
2026-10-16: the next test saw twelve keys).
"""
from __future__ import annotations
import contextlib
import os
from collections.abc import Iterator
import pytest
from raven_everos import config as cf
def is_binding(name: str) -> bool:
"""Whether the environment variable ``name`` is part of what a binding writes."""
return name.startswith("EVEROS_") or name == cf.PROVENANCE_ENV
@contextlib.contextmanager
def confined_everos_binding() -> Iterator[None]:
"""Everything a binding changes, as it was when the block began, once the block ends -- exception or not.
The provenance set goes back as the object the block began with, whatever
the attribute holds by then (see the module docstring): that object's
contents, and the attribute bound to it again.
"""
before = {key: value for key, value in os.environ.items() if is_binding(key)}
original = cf._BOUND_HERE
bound = set(original)
try:
yield
finally:
for key in [key for key in os.environ if is_binding(key) and key not in before]:
del os.environ[key]
os.environ.update(before)
original.clear()
original.update(bound)
cf._BOUND_HERE = original
def forget_everos_binding(monkeypatch: pytest.MonkeyPatch) -> None:
"""Start a test with no role exported and no provenance remembered, and have it put back when the test ends.
For a reader of ``role_is_env_managed``: an operator's export in the
developer's own shell, or a binding a test elsewhere left behind, would
turn a case that assumes nothing is exported into a different case --
silently, and only on that machine or in that shard. The set is swapped
rather than cleared, so what the test's own binding puts in it goes with
the swap.
"""
for key in [key for key in os.environ if is_binding(key)]:
monkeypatch.delenv(key)
monkeypatch.setattr(cf, "_BOUND_HERE", set())