gloryfromca ce2713e9a5 fix(*): ship the cloud memory plugin with raven, not beside the release
The release builds everos-cloud-memory's wheel and attaches it, but no
installer and no upgrade named it, so the only way in was `uv tool install
--with`, which the next `raven upgrade` undoes: uv replaces the tool's
requirement set with what the command names, and the upgrade names the
release's plugin list alone. A user who picked the cloud backend was told to
install a plugin that had no install path.

The live-service gate that kept it out is met: the six key-gated tests in
tests/integration/test_everos_cloud_real_cloud.py ran against api.evermind.ai
on 2026-10-09. So the distribution joins the list a complete install is made
of -- release.yml, publish_beta.py for the beta channel, both installers'
local-clone lists, and the Docker image's three plugin defaults. Installed is
not selected: memory.backend still decides, doctor and the gateway build only
the backend it names, and the one visible change is that `raven onboard` asks
which memory backend to use rather than taking the only one installed.

The rung that drops the product engines kept only everos-memory. That rung
exists so long-term memory outlives a native build a platform refuses, so on
such a machine every upgrade uninstalled the cloud backend from under a host
configured to use it. install.sh, install.ps1 and the upgrade helper now keep
both memory distributions, and the contract test reads which those are off the
backend/distribution pairing rather than naming one by hand.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-10-20 21:54:35 +08:00
..

Raven Test Suite

The test suite covers the CLI, provider routing, context assembly, memory, channels, sandbox behavior, TUI RPC contracts, and proactive engine flows.

Running Tests

Run the default suite from the repository root:

uv run pytest

For focused work, target the relevant file or marker:

uv run pytest tests/test_cli_doctor_commands.py
uv run pytest -m "not real_llm"

The suite runs on four workers by default (-n 4 in addopts), which takes it from about ten minutes to two. Two things that costs you, both undone by -n0 on the command line:

uv run pytest -n0 --pdb tests/test_cli_doctor_commands.py
  • --pdb refuses to start under the workers: --pdb is incompatible with distributing tests. There is no debugger to attach to, since the test runs in a child process.
  • Output is no longer in file order. Reading a run top to bottom, or comparing two runs line by line, wants -n0 as well.

Tests should avoid live network calls by default. When a test needs external services or a real model, guard it behind an explicit marker or environment variable so CI and local contributors get deterministic results.

Coverage Governance

Raven uses three related checks:

  • Coverage measures line and branch execution across the default Python suite. It is the broad health signal for the production raven/ package.
  • Diff coverage measures executable lines added or modified relative to the PR target branch. Comments, deleted lines, and non-executable lines are not part of its denominator. The initial threshold is 90%.
  • The ratchet compares total line and branch coverage with the audited baseline in .github/coverage-baseline.json. Neither metric may decline by more than 0.05 percentage points. This protects the established level without pretending the current project is already near 100%.

Run the same coverage command used by CI:

make coverage

PYTHON_VERSION defaults to the CI version (3.12) and is shared by all Python Make targets. It can be overridden for exploratory compatibility runs, for example make coverage PYTHON_VERSION=3.13; only the configured CI matrix version may produce an authoritative baseline.

The command prints missing lines and creates coverage.xml, coverage.json, and htmlcov/. Open htmlcov/index.html in a browser for line-by-line detail. Generated reports are ignored by git.

After running coverage, use the local gates:

make coverage-summary
make coverage-diff COVERAGE_BASE_REF=origin/main
make coverage-ratchet
make coverage-baseline-check COVERAGE_BASE_REF=origin/main

The summary prints separate total line and branch percentages plus the ten lowest-covered production files. A diff failure lists every uncovered changed line as path:line. A ratchet failure prints the current value, baseline, change in percentage points, and the files with the largest declines.

The CI unit job runs the default suite once and reuses its data for all gates. It uploads XML, JSON, and HTML reports even when tests or a gate fail. Diff coverage runs only for pull requests because a push has no PR target branch; push builds still enforce the overall ratchet.

Baseline updates

make coverage-baseline-candidate creates coverage-baseline-candidate.json from the current report. The tracked bootstrap baseline is accepted only after the introducing PR reproduces it in Python 3.12 Ubuntu CI within the ratchet tolerance. After bootstrap, the baseline must only be replaced with a candidate downloaded from a successful main CI run. Review the reference commit and totals in the candidate before replacement. Never edit the percentages by hand or lower the baseline to make a change pass. PR CI compares a proposed baseline with the target branch and rejects any decrease in either line or branch coverage.

Scope and exclusions

The default coverage suite excludes tests marked integration or e2e, plus tests/integration/, because they require real services, credentials, subprocess stacks, VMs, or network access and are not deterministic unit-CI inputs. Provider cases that require live credentials remain skipped.

All production Python under raven/ is measured except:

  • raven/__main__.py and evolver/__main__.py, which only forward their module entry points.

Type-checking-only blocks and if __name__ == "__main__" launcher blocks are excluded as non-runtime paths. No low-coverage feature module is omitted.