Raven-1/scripts/coverage_gate.py
zhanghui d106a5c6ef
feat(plugins): add the everos-cloud-memory plugin and its host wiring (#883)
## Summary

Adds `everos-cloud-memory`, a separately distributed memory plugin for
EverOS Cloud, and wires the host so a second memory plugin is a
first-class choice.

The plugin (`plugins-dist/everos-cloud-memory`) is a thin REST client
over httpx: one API key (the config slice
`plugins.config.everos-cloud-memory.api_key`, or
`EVEROS_CLOUD_API_KEY`), owner ids from the host's `ServiceLocator`
(never from the slice), every add in `agent` mode, and a flush only when
a caller marks the conversation over (`metadata.flush` /
`metadata.is_final`: the sub-agent handoff and the importer's final
batch) and once at stop. It ships its own onboard screen (env or on-file
key used without a prompt; 401 / 403 / unreachable get their own
sentences with Re-enter or Skip), `delete` answers False (the cloud
deletes by scope, nothing calls it), and `health()` classifies seven
answers for `raven doctor` and the settings card.

Host changes, each guarded so `memory.backend` `everos` and `null`
behave exactly as before:

- `raven onboard` step 4 asks "Which memory backend?" when more than one
plugin contributes a screen (preselects the current backend, `Off`
clears it, cancel exits 1 and writes nothing); with one plugin the step
is byte-identical to today.
- The memory page reads whichever backend `memory.backend` names: the
cloud slice's key goes out as a Bearer header and the `/health` probe is
skipped for the cloud; a missing distribution is named in the page's
note.
- `settings.everos` reports `reason: "other_backend"` for any non-EverOS
backend, so the Model page hides the three EverOS-only roles (embedding
stays: knowledge bases use that pin); `settings.everosSet` refuses those
three sections for another backend; `settings.set` admits
`plugins.config.<id>.<field>` against the plugin's manifest through the
same admission door channels use; new `settings.everosCloud` feeds the
card (key source, endpoint, health).
- Provider-key and main-model saves no longer restart the local EverOS
when another backend is active.
- ui-web: an EverOS Cloud card (status chip, key row, endpoint) and the
`settings.everosCloud` data behind it. No page mounts the card: it drew
on Model settings until the owner ruled that a memory backend's endpoint
and credential answer neither question that page asks, and where it
belongs is the Memory section's to say once that section has a design.
Until then the key is entered through `raven onboard` or
`EVEROS_CLOUD_API_KEY`. The fixture gate pins the four optional fields
the offline library never sends.
- Gates: the plugin boundary test scans both memory plugins, coverage
and the release workflow build the second wheel, deptry's ignore list
names the plugin beside `raven_everos`, and a conftest fixture keeps the
second plugin out of entry-point discovery for the existing onboarding
tests (they describe a one-plugin install).

A second topic rides along, because it is what makes the hosted backend
reachable from an agent product at all. Every product under `agents/`
pinned its memory at the stock EverOS address and only `raven-design`
read the host's, so the other four addressed whatever answered on 18791
-- the wizard moves a managed server when the port is taken, and a
server the user runs themselves is wherever they put it.
`inherit_memory_backend` in `raven/config/product_render.py` now carries
`memory.backend`, the endpoint, and (only when the product holds none
itself) the credential from the host into every render, beside the host
denials and plugin opt-outs that already travel there; `raven-design`'s
private copy is deleted and the five product configs stop naming an
address. Identity never travels, which is what keeps five products apart
inside one store. `memory.backend: null` travels like any other value,
because that null is how an operator turns long-term memory off and a
product that kept writing would be the one surface ignoring them; the
product's own default stands only when the host carries no key at all.
Which store memories live in is not a product's choice either: the host
reads a sub-agent's memories back through its own backend, so a product
writing anywhere else left that record empty with nothing reporting it.
`owned` is forced off whenever the local backend is in force, because a
product infers ownership from a slice naming no root -- the shape all
five ship -- and so answered a host that was merely not serving by
starting a second server against a root of its own, stranding every
memory written so far.

Decisions worth a reviewer's eye: the fake cloud
(`tests/_everos_cloud_fake.py`) honours or refuses every field the
backend can send and serves both an in-process transport and an HTTP
face for the real-host acceptance; the live tests
(`tests/integration/test_everos_cloud_real_cloud.py`) skip without
`EVEROS_CLOUD_API_KEY`; run with a real key on 2026-10-09 all six pass
against api.evermind.ai (round trip, get by session, empty flush, 401
wording, probe with and without a key, and extraction without a flush
within two minutes), so the documentation and the service agree on
everything the plugin sends. Design, acceptance cases and plan are under
`docs/specs/2026-10-08-everos-cloud-memory-*.md` and `docs/plans/`.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

Rebased onto main `182dacac0` on 2026-10-13. What that rebase and the
four commits on top were verified with:

- Full suite, `uv run --frozen --python 3.12 --extra dev pytest -q` ->
**29748 passed, 8 failed** in 4m10s. All eight fail identically with
these four commits' sources reverted in-tree at the same base, so none
of them is this branch's: the two `test_the_products_tool_face_*` cases
in the code and oncall launchers (an extra `web_search` in the visible
tool set), `test_agent_loop_token_budget`,
`test_rpc_files::test_a_host_without_libreoffice_says_so`,
`test_ppt_engine_image_search`, `test_shell_background`, and the two
`test_simulation_*` cases.
- Real-host before and after for the inheritance, driving the launcher
the host spawns (`agents/raven-research/run.py`) against an isolated
`RAVEN_HOME` whose EverOS address is `19701`, not the stock port.
Before: the product rendered `base_url: http://localhost:18791` with no
`owned` key, ignored the host, and started an EverOS of its own -- an
orphaned `everos server start --root <its own state root>/everos`
holding 18791, which is the stranding this change exists to stop. After:
it rendered `{"base_url": "http://localhost:19701", "owned": false}` and
said so itself, "the EverOS you manage is not running at
http://localhost:19701 ... Raven does not start or stop it", with `ps`
and `lsof` confirming nothing was started. On the cloud the product
rendered `memory.backend: everos-cloud` with the host's key and left the
out-of-force local slice untouched, and `raven doctor` over that
rendered config answered `everos-cloud: ok https://api.evermind.ai`, so
the credential reached the service. Identity stayed `raven-research-ng`
throughout, and a fingerprint of the operator's own EverOS root was
identical before and after.
- The first CI round on this head found one that the local gates did
not: `tests/test_import_cycle_budget.py`. `inherit_memory_backend` read
the backend-to-distribution pairing from `raven.core.plugin_stack`, and
that edge from `raven.config` into `raven.core` -- which imports
`raven.config` back -- took mutual pairs from 13 to 14 and packages
inside a cycle from 18 to 22. `make lint-imports` keeps ten contracts
and none of them covers that pair, so it was green throughout. The
pairing now lives in `raven/config/raven.py` beside the two fields it
joins (`MemoryConfig.backend` names a contribution,
`PluginsConfig.config` is keyed by the distribution), which `raven.core`
may import and which imports nothing back; `plugin_stack` re-exports the
two names its own callers read from it. An intermediate commit put it in
`raven.contracts` instead, which broke the cycle but took the papers
ceiling from 3700 to 3719 lines -- `tests/test_kernel_budget.py` says a
new paper is a reviewed change to that ceiling, so it moved again rather
than raising it.
- The review round on this head raised one non-blocking finding, fixed
rather than carried forward: the inheritance read `memory.backend` with
`.get()`, which collapses an explicit `null` into an absent key, so a
host with long-term memory off left every packaged agent on its own
shipped backend and still writing. Reproduced by calling the function
directly before the change and after, now read on presence, and pinned
by a case whose mutation (restoring the `.get()`) turns it and nothing
else red.
- One ordering problem found while checking the above, pre-existing and
not fixed here: `tests/test_rpc_settings.py` run before
`tests/test_cli_onboard_commands.py` leaves
`raven.home._current_config_path` pointing at a dead tmp config, so the
onboard cases write somewhere else. With main's own copies of both files
and main's conftest in the tree it is 57 failures; with this branch's,
19. Which cases turn red changes run to run as xdist distributes them
differently.
- `cd ui-web && npx vitest run` -> 876 files, 3728 passed, 0 failed;
`npm run lint` -> 0 errors (4 warnings, all in files this change does
not touch); `npm run gen:check` -> in sync (228 methods); `npm run
type-check` -> clean; `cd ui-tui && npm run lint:i18n && npm run
lint:rpc` -> in sync
- `make lint-python` / `lint-imports` (10 contracts kept) / `lint-types`
/ `lint-deps` -> all pass; `uv lock --check` -> exit 0 with both sides'
new packages present; `scripts/check_source_language.py` and
`scripts/check_large_files.py` on `github/main..HEAD` -> exit 0;
commitlint over every commit -> ok
- Semantic-conflict audit of the rebase, beyond what the suite answers:
21 of the branch's 66 files are also in main's 499, and every one of
main's changes in them is additive and elsewhere in the file. The one
that could have broken silently is main's new `vision.model` /
`vision.provider` simple keys meeting this branch's new
`plugins.config.<id>.<field>` branch in `settings_set`; nothing in the
suite exercises `vision.*` through that method, so it was run directly
-> `{'applied': True}` and `{"model": "..."}` on disk. The subtraction
decisions were re-checked against the new base: no new caller of a
memory backend's `delete`, main's new onboard cases are about TLS and
not the memory step, `inherit_everos_address` exists nowhere, and
`DEFAULT_EVEROS_BASE_URL` is still defined.
- Mutation runs on the new code, restored afterwards: dropping `owned:
false`, letting the host's credential overwrite the product's own, not
writing the host's backend name, and carrying every memory slice instead
of the one in force each turn their own cases red; making the cloud card
render nothing turns 4 of its 5 new cases red and leaves the fifth
("draws nothing when not selected") green.

From the earlier round, on base `6d509a8c5`:

- `uv run --frozen --python 3.12 --extra dev pytest -p no:cacheprovider
-q tests/test_everos_cloud_fake.py tests/test_everos_cloud_backend.py
tests/test_everos_cloud_backend_contract.py
tests/test_everos_cloud_onboard.py tests/test_everos_cloud_discover.py
tests/test_cli_onboard_commands.py tests/test_rpc_memory.py
tests/test_rpc_settings.py tests/test_plugin_boundary.py
tests/test_rpc_schema_match.py tests/test_rpc_registration.py
tests/test_release_plugin_list.py
tests/integration/test_everos_cloud_real_cloud.py` -> 1001 passed, 6
skipped (the six live tests, no key)
- `cd ui-web && npx vitest run` -> 214 files, 3211 passed; `npm run
lint` -> 0 errors (4 warnings, all in files this change does not touch);
`npm run gen:check` -> in sync (207 methods); `cd ui-tui && npm run
lint:rpc && npm run lint:i18n` -> in sync
- `make lint` (ruff, ruff format, ui tsc, deptry) -> ok; `make
lint-types` -> All checks passed; `make build-ui` -> boot snapshots
match; `scripts/check_large_files.py` and
`scripts/check_source_language.py` on `6d509a8c5..HEAD` -> exit 0;
commitlint and `scripts/check_commit_messages.py` on each commit -> ok
- Mutation runs per change (restored afterwards, `git diff` clean): a
host import of the plugin turns the boundary test red; `import
raven_everos` in the plugin turns the import-isolation test red;
removing the chooser's cancel exit, the Bearer header, the restart
guard, the slice-admission guard, the Roles filter and the fake's two
fixes each turn their own test red
- Real-host acceptance (64 cases,
`docs/specs/2026-10-08-everos-cloud-memory-acceptance.md`) run against a
fake EverOS Cloud: the wizard in tmux (chooser, key typed / from env /
on file, 401 / 403 / unreachable wording, back, Off, Ctrl-C,
non-interactive, base-tree transcript byte-identical with one plugin),
`raven agent` turns (user and agent-track recall, agent-mode adds, 4 s
recall bound, a 300 KB message dropped with a warning, a sub-agent trace
stored under its own ids, flushed and read back, one exit flush per
unflushed session), `raven import run` (3 adds then one flush; a hung
flush retried 4x and the exit sweep defeated), `raven doctor` under
seven cloud conditions (seven sentences, exit 2 only for the five
faults) and with the distribution hidden, and the web page through
playwright-cli (memory page counts and the fake's episode, empty and
refused stores, the Model page hiding the three EverOS-only roles while
the embedding row stays, the cloud card rotating a key and showing the
401 wording, three refused settings.set frames over the gateway
WebSocket, the local everos / null / mem0 backends drawing the page as
before, and a provider-key save that restarts the local EverOS on the
base tree and does not on this branch). Evidence and the per-case
verdicts are kept outside the repo. The live service was reached with a
real key after the acceptance run: six tests, six passes, the service's
replies archived with the evidence.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

User-visible changes: the wizard's step 4 chooser (only when two memory
plugins are installed); Model settings hides the three EverOS-only role
rows when `memory.backend` is not `everos`/`null` and draws no cloud
card; the memory page sends a Bearer header for the cloud backend.

Two behaviour changes for agent products, both on the `agents/` lane
only. An agent product now follows the host's `memory.backend` and
endpoint instead of its own, which is a fix for the four products that
never followed the host and a change for `raven-design`, which followed
the endpoint but not the backend. And a product no longer starts an
EverOS server of its own: where it used to answer a host that was not
serving by starting one against an empty root of its own, it now reports
long-term memory as unavailable, which is what the host does. Rollback
for this part is the one commit; the product configs it edits are
shipped files, not user config. The plugin is not installed by default
(its wheel is built by the release workflow; nothing changes for an
install without it). Rollback: revert the squash commit; the config
schema gains no new key (the slice is a plugin slice), so an older raven
reads a config written by this one.

Credentials: the cloud key is written only by `set_plugin_config_fields`
into the plugin's slice; the settings page's key row goes through
`settings.set` with a manifest-declared field, and
`settings.everosCloud` reports `api_key_set` / `key_source`, never the
value.

## Related Issues

N/A

---------

Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
2026-10-14 11:08:11 +08:00

366 lines
15 KiB
Python

"""Coverage reporting and governance gates shared by CI and local development."""
from __future__ import annotations
import argparse
import json
import re
import shutil
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterable
SCHEMA_VERSION = 1
DEFAULT_TOLERANCE = 0.05
# Glob magic so ** matches zero-or-more path components; the default pathspec
# raven/**/*.py drops top-level files like raven/__init__.py because plain * spans
# slashes and the middle separator then has nothing to match.
PRODUCTION_PATHSPECS = (
":(glob)raven/**/*.py",
# The everos memory plugin beside the host wheel is production code too;
# without this a change there dodges the diff gate. Only that plugin: the
# gate can judge what coverage measures (--cov=raven --cov=raven_everos), and
# a changed file in an unmeasured tree would fail every PR that touches it.
":(glob)plugins-dist/everos-memory/**/*.py",
":(glob)plugins-dist/everos-cloud-memory/**/*.py",
# Scaffold templates are copied out by the scaffolder, never imported by
# the runtime; they are data with a .py spelling, not production code.
":(glob,exclude)raven/templates/agents_scaffold/**",
)
DEFAULT_DIFF_THRESHOLD = 90.0
OMITTED_PATHS = {
"raven/__main__.py",
"evolver/__main__.py",
}
@dataclass(frozen=True)
class Metrics:
statements: int
covered_lines: int
branches: int
covered_branches: int
@property
def line_percent(self) -> float:
return _percent(self.covered_lines, self.statements)
@property
def branch_percent(self) -> float:
return _percent(self.covered_branches, self.branches)
@dataclass(frozen=True)
class DiffCoverage:
executable: frozenset[tuple[str, int]]
uncovered: frozenset[tuple[str, int]]
zero_coverage_files: tuple[str, ...]
unmeasured_files: tuple[str, ...]
@property
def percent(self) -> float:
return _percent(len(self.executable) - len(self.uncovered), len(self.executable))
def _percent(covered: int, total: int) -> float:
return 100.0 if total == 0 else covered * 100.0 / total
def _metrics(summary: dict[str, Any]) -> Metrics:
return Metrics(
statements=int(summary["num_statements"]),
covered_lines=int(summary["covered_lines"]),
branches=int(summary["num_branches"]),
covered_branches=int(summary["covered_branches"]),
)
def load_coverage(path: Path) -> dict[str, Any]:
with path.open(encoding="utf-8") as handle:
report = json.load(handle)
if not isinstance(report.get("files"), dict) or not isinstance(report.get("totals"), dict):
raise ValueError(f"{path} is not a coverage.py JSON report")
return report
def print_summary(report: dict[str, Any], limit: int) -> None:
totals = _metrics(report["totals"])
print(f"Line coverage: {totals.line_percent:.2f}% ({totals.covered_lines}/{totals.statements})")
print(f"Branch coverage: {totals.branch_percent:.2f}% ({totals.covered_branches}/{totals.branches})")
print(f"Lowest {limit} production files by combined coverage:")
ranked = sorted(
(
(float(data["summary"]["percent_covered"]), name, _metrics(data["summary"]))
for name, data in report["files"].items()
if int(data["summary"]["num_statements"]) > 0
),
key=lambda item: (item[0], item[1]),
)
for combined, name, metrics in ranked[:limit]:
print(f" {combined:6.2f}% line {metrics.line_percent:6.2f}% branch {metrics.branch_percent:6.2f}% {name}")
def create_baseline(report: dict[str, Any], commit: str) -> dict[str, Any]:
files = {
name: _metrics_payload(_metrics(data["summary"]))
for name, data in sorted(report["files"].items())
if int(data["summary"]["num_statements"]) > 0
}
return {
"schema_version": SCHEMA_VERSION,
"source": "raven",
"python": f"{sys.version_info.major}.{sys.version_info.minor}",
"reference_commit": commit,
"totals": _metrics_payload(_metrics(report["totals"])),
"files": files,
}
def _metrics_payload(metrics: Metrics) -> dict[str, int | float]:
return {
"statements": metrics.statements,
"covered_lines": metrics.covered_lines,
"line_percent": round(metrics.line_percent, 6),
"branches": metrics.branches,
"covered_branches": metrics.covered_branches,
"branch_percent": round(metrics.branch_percent, 6),
}
def write_baseline(report: dict[str, Any], output: Path, commit: str) -> None:
output.write_text(json.dumps(create_baseline(report, commit), indent=2, sort_keys=True) + "\n", encoding="utf-8")
print(f"Wrote coverage baseline candidate to {output}")
def check_baseline_update(current: dict[str, Any], previous: dict[str, Any]) -> bool:
if current.get("schema_version") != SCHEMA_VERSION or previous.get("schema_version") != SCHEMA_VERSION:
raise ValueError("current and target-branch baselines must use the supported schema")
passed = True
for metric in ("line_percent", "branch_percent"):
value = float(current["totals"][metric])
target = float(previous["totals"][metric])
delta = value - target
print(f"Baseline {metric}: proposed={value:.6f}% target={target:.6f}% delta={delta:+.6f}pp")
if value < target:
passed = False
if passed:
print("Coverage baseline update is monotonic.")
else:
print("Coverage baseline files may not lower line or branch coverage.")
return passed
def check_ratchet(report: dict[str, Any], baseline: dict[str, Any], tolerance: float, limit: int) -> bool:
if baseline.get("schema_version") != SCHEMA_VERSION:
raise ValueError(f"unsupported baseline schema: {baseline.get('schema_version')!r}")
current = _metrics(report["totals"])
expected = baseline["totals"]
checks = (
("line", current.line_percent, float(expected["line_percent"])),
("branch", current.branch_percent, float(expected["branch_percent"])),
)
failed = [(name, value, target) for name, value, target in checks if value + tolerance < target]
for name, value, target in checks:
delta = value - target
print(f"Ratchet {name}: current={value:.2f}% baseline={target:.2f}% delta={delta:+.2f}pp")
if not failed:
print(f"Coverage ratchet passed with {tolerance:.2f}pp tolerance.")
return True
print("Largest per-file coverage declines:")
declines = _file_declines(report, baseline)
for decline, name, line_delta, branch_delta in declines[:limit]:
print(f" {decline:6.2f}pp line {line_delta:+7.2f}pp branch {branch_delta:+7.2f}pp {name}")
return False
def _file_declines(report: dict[str, Any], baseline: dict[str, Any]) -> list[tuple[float, str, float, float]]:
declines: list[tuple[float, str, float, float]] = []
current_files = report["files"]
for name, expected in baseline["files"].items():
data = current_files.get(name)
if data is None:
continue
current = _metrics(data["summary"])
line_delta = current.line_percent - float(expected["line_percent"])
branch_delta = current.branch_percent - float(expected["branch_percent"])
decline = max(-line_delta, -branch_delta, 0.0)
if decline > 0:
declines.append((decline, name, line_delta, branch_delta))
return sorted(declines, key=lambda item: (-item[0], item[1]))
_HUNK_RE = re.compile(r"^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@")
def parse_changed_lines(diff: str) -> dict[str, set[int]]:
"""Return new-side hunk lines; deleted lines never enter the denominator."""
changed: dict[str, set[int]] = {}
path: str | None = None
for line in diff.splitlines():
if line.startswith("+++ "):
# Reset on a non-b/ target (e.g. +++ /dev/null) so a deleted file's
# hunks are not misattributed to the previously seen path.
path = line[6:] if line.startswith("+++ b/") else None
if path is not None:
changed.setdefault(path, set())
continue
match = _HUNK_RE.match(line)
if path is None or match is None:
continue
start = int(match.group(1))
count = int(match.group(2) or "1")
changed[path].update(range(start, start + count))
return changed
def calculate_diff_coverage(
report: dict[str, Any], changed: dict[str, set[int]], added_files: Iterable[str] = ()
) -> DiffCoverage:
"""Intersect changed lines with coverage.py statements to ignore comments and blanks."""
executable: set[tuple[str, int]] = set()
uncovered: set[tuple[str, int]] = set()
zero_coverage_files: list[str] = []
unmeasured_files: list[str] = []
added = set(added_files)
for name, lines in changed.items():
data = report["files"].get(name)
if data is None:
if name not in OMITTED_PATHS:
unmeasured_files.append(name)
continue
executed = set(data["executed_lines"])
missing = set(data["missing_lines"])
statements = executed | missing
for line in lines & statements:
item = (name, line)
executable.add(item)
if line in missing:
uncovered.add(item)
if name in added and statements and not executed:
zero_coverage_files.append(name)
return DiffCoverage(
frozenset(executable),
frozenset(uncovered),
tuple(sorted(zero_coverage_files)),
tuple(sorted(unmeasured_files)),
)
def _git(*args: str) -> str:
executable = shutil.which("git")
if executable is None:
raise OSError("git is required for coverage diff checks")
completed = subprocess.run([executable, *args], check=True, capture_output=True, text=True)
return completed.stdout
def load_baseline_from_ref(base_ref: str, path: Path) -> dict[str, Any] | None:
_git("rev-parse", "--verify", f"{base_ref}^{{commit}}")
try:
content = _git("show", f"{base_ref}:{path.as_posix()}")
except subprocess.CalledProcessError:
return None
return json.loads(content)
def git_diff_inputs(base_ref: str, head: str | None) -> tuple[dict[str, set[int]], set[str]]:
"""Resolve the merge base and collect Python changes for CI or the local worktree."""
merge_base = _git("merge-base", base_ref, "HEAD").strip()
end = head or ""
range_args = [merge_base, end] if end else [merge_base]
diff = _git("diff", "--unified=0", "--no-color", "--diff-filter=ACMR", *range_args, "--", *PRODUCTION_PATHSPECS)
names = _git("diff", "--name-only", "--diff-filter=A", *range_args, "--", *PRODUCTION_PATHSPECS)
return parse_changed_lines(diff), {name for name in names.splitlines() if name}
def check_diff(report: dict[str, Any], base_ref: str, head: str | None, threshold: float) -> bool:
changed, added = git_diff_inputs(base_ref, head)
result = calculate_diff_coverage(report, changed, added)
covered = len(result.executable) - len(result.uncovered)
print(f"Diff coverage: {result.percent:.2f}% ({covered}/{len(result.executable)} executable changed lines)")
if result.uncovered:
print("Uncovered changed lines:")
for name, line in sorted(result.uncovered):
print(f" {name}:{line}")
if result.zero_coverage_files:
print("New production files with 0% line coverage:")
for name in result.zero_coverage_files:
print(f" {name}")
if result.unmeasured_files:
print("Changed production files missing from the coverage report:")
for name in result.unmeasured_files:
print(f" {name}")
passed = result.percent + 1e-9 >= threshold and not result.zero_coverage_files and not result.unmeasured_files
if passed:
print(f"Diff coverage passed the {threshold:.2f}% threshold.")
else:
print(f"Diff coverage failed the {threshold:.2f}% threshold.")
return passed
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--coverage-json", type=Path, default=Path("coverage.json"))
subparsers = parser.add_subparsers(dest="command", required=True)
summary = subparsers.add_parser("summary")
summary.add_argument("--limit", type=int, default=10)
baseline = subparsers.add_parser("baseline")
baseline.add_argument("--output", type=Path, default=Path("coverage-baseline-candidate.json"))
baseline.add_argument("--commit", default="HEAD")
baseline_check = subparsers.add_parser("baseline-check")
baseline_check.add_argument("--baseline", type=Path, default=Path(".github/coverage-baseline.json"))
baseline_check.add_argument("--base-ref", required=True)
ratchet = subparsers.add_parser("ratchet")
ratchet.add_argument("--baseline", type=Path, default=Path(".github/coverage-baseline.json"))
ratchet.add_argument("--tolerance", type=float, default=DEFAULT_TOLERANCE)
ratchet.add_argument("--limit", type=int, default=10)
diff = subparsers.add_parser("diff")
diff.add_argument("--base-ref", required=True)
diff.add_argument("--head", default=None)
diff.add_argument("--threshold", type=float, default=DEFAULT_DIFF_THRESHOLD)
return parser
def main(argv: list[str] | None = None) -> int:
args = _parser().parse_args(argv)
try:
if args.command == "baseline-check":
with args.baseline.open(encoding="utf-8") as handle:
current_baseline = json.load(handle)
previous_baseline = load_baseline_from_ref(args.base_ref, args.baseline)
if previous_baseline is None:
print(f"No coverage baseline exists on {args.base_ref}; accepting the initial bootstrap baseline.")
return 0
return 0 if check_baseline_update(current_baseline, previous_baseline) else 1
report = load_coverage(args.coverage_json)
if args.command == "summary":
print_summary(report, args.limit)
return 0
if args.command == "baseline":
commit = _git("rev-parse", args.commit).strip()
write_baseline(report, args.output, commit)
return 0
if args.command == "ratchet":
with args.baseline.open(encoding="utf-8") as handle:
baseline = json.load(handle)
return 0 if check_ratchet(report, baseline, args.tolerance, args.limit) else 1
if args.command == "diff":
return 0 if check_diff(report, args.base_ref, args.head, args.threshold) else 1
except (OSError, ValueError, KeyError, json.JSONDecodeError, subprocess.CalledProcessError) as exc:
print(f"coverage gate error: {exc}", file=sys.stderr)
return 2
raise AssertionError(f"unhandled command: {args.command}")
if __name__ == "__main__":
raise SystemExit(main())