2026-06-19 16:46:11 -05:00
|
|
|
#!/usr/bin/env python3
|
|
|
|
|
"""Classify a PR's changed files into CI work lanes.
|
|
|
|
|
|
|
|
|
|
Reads newline-separated changed paths on stdin and writes ``key=value``
|
|
|
|
|
booleans (one per lane) to ``$GITHUB_OUTPUT`` and stdout. The
|
|
|
|
|
``detect-changes`` composite action consumes them so steps gate on
|
|
|
|
|
``if: steps.changes.outputs.<lane> == 'true'``.
|
|
|
|
|
|
2026-06-23 09:13:19 -04:00
|
|
|
Lanes:
|
|
|
|
|
|
|
|
|
|
* ``python`` — pytest / ruff / ty / footguns.
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
* ``python_prod`` — Python changes OUTSIDE tests/. A tests-only PR keeps
|
|
|
|
|
``python`` (pytest must run) while this stays off.
|
2026-06-23 09:13:19 -04:00
|
|
|
* ``docker_meta`` — Dockerfiles etc.
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
* ``docker`` — the image build: docker meta, the dependency manifests, PM,
|
|
|
|
|
and the image's own tests.
|
|
|
|
|
* ``nix`` — ``nix flake check``: the flake files and the dependency
|
|
|
|
|
manifests.
|
|
|
|
|
* ``e2e``, ``e2e_upgrade``, ``e2e_desktop_core``, ``e2e_desktop_update`` —
|
2026-10-08 19:24:19 -07:00
|
|
|
the end-to-end suites. The classifier still says which suites a diff
|
|
|
|
|
touches (``_E2E_LANES``, the ``run-e2e`` label), but ci.yaml's ``detect``
|
|
|
|
|
gate forces these lanes off on pull requests and pushes to main: the E2E
|
|
|
|
|
suites run only on a release run or a manual dispatch.
|
2026-06-23 09:13:19 -04:00
|
|
|
* ``frontend`` — TS typecheck matrix + desktop build.
|
|
|
|
|
* ``site`` — Docusaurus + generated skill docs.
|
|
|
|
|
* ``scan`` — supply-chain scan (Python files, .pth, setup hooks).
|
|
|
|
|
* ``deps`` — pyproject.toml dependency bounds check.
|
2026-09-11 18:14:43 -04:00
|
|
|
* ``uv_lock`` — ``PM lock check``. Re-resolves the whole graph against
|
2026-08-12 13:30:01 -05:00
|
|
|
PyPI, so a diff that touches neither ``pyproject.toml`` nor ``uv.lock``
|
|
|
|
|
must not run it.
|
2026-07-15 23:18:15 -04:00
|
|
|
* ``npm_lock`` — semantic package-lock.json diff PR comment.
|
feat(pm): unified package manager — pm store foundation
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
2026-08-29 21:23:27 -04:00
|
|
|
* ``bootstrap`` — the bootstrap installer lane: install.sh sandbox install,
|
|
|
|
|
pin-fragment drift check, and shipped version-stamp verification.
|
2026-09-01 21:58:50 -07:00
|
|
|
* ``desktop_updater`` — the Windows desktop-update hand-off script and the
|
|
|
|
|
tests that drive the REAL ``windows.ps1`` (``-SelfTestUi`` / pipe drain /
|
|
|
|
|
retry policy). These are integration tests of a PowerShell process on a
|
|
|
|
|
shared runner; running them on every Python PR made their timing noise
|
|
|
|
|
everyone's problem. They still run on push (fail-open) and whenever the
|
|
|
|
|
script, its siblings, or their tests change.
|
2026-08-20 11:40:49 -05:00
|
|
|
* ``rust`` — ``cargo test`` for the Tauri bootstrap installer. ``.rs``
|
|
|
|
|
lives under ``apps/``, so without this lane a Rust change matched ``frontend``
|
|
|
|
|
and only the TypeScript matrix ran.
|
2026-10-16 04:23:57 -07:00
|
|
|
* ``catalog`` — the plugin-catalog tests (catalog-tests.yml). A diff made
|
|
|
|
|
only of catalog entry data (``plugin-catalog/*.yaml``, ``contributors/emails/``)
|
|
|
|
|
runs this lane INSTEAD of the Python suite: those files are data that a known
|
|
|
|
|
set of tests reads, so a one-entry PR need not run 23k tests on three OSes.
|
|
|
|
|
``plugin-catalog/README.md`` is not entry data (a docs page mirrors it), and
|
|
|
|
|
any other file in the diff brings the full Python suite back.
|
ci: run the SSH and Docker terminal backends against a real sshd and daemon
The SSH backend's real-host tests skipped on every CI run (nothing set
TERMINAL_SSH_HOST), and every Docker backend test mocked subprocess or the
docker CLI, so no CI run had executed either backend end to end.
- remote-backends.yml: starts a throwaway sshd container on 127.0.0.1 with a
per-run keypair (scripts/ci/start_test_sshd.sh) and runs
test_ssh_environment.py plus the new test_docker_backend_real.py against it
and the runner's Docker daemon with a digest-pinned debian:12-slim. A file
that skips fails the job, so the lane cannot go green over zero coverage.
- New classifier lane `remote_backends` (tools/environments/, tools/terminal_tool*,
the real tests, the sshd harness, run_tests.sh). ci.yaml calls the workflow
only when it fires, plus every push to main; required_results makes a selected
lane mandatory, like the update consumers.
- The real tests read HERMES_TEST_SSH_* / HERMES_TEST_DOCKER_IMAGE (forwarded
by run_tests.sh), because the hermetic conftest blanks TERMINAL_SSH_KEY as a
credential before any test can read it.
- New real SSH cases: exit code/stderr, cwd persistence with spaces vs per-call
workdir, quoting across the ssh hop, SendEnv passthrough through a real
AcceptEnv, file tools write/read on the remote.
- New real Docker cases: exec + exit codes, cwd vs workdir, exported env
persistence, docker_env/docker_forward_env reach the container and
nothing else does, quoting, file tools write/read, /root surviving a
container replacement.
- The existing one-shot test is xfail(strict): TERMINAL_SSH_PERSISTENT=false
is read into ssh_config["persistent"] but SSHEnvironment never consumes it,
so every SSH session persists exported env. Its _cleanup also cleaned the
raw task id, not the resolved env key, so one test's env leaked into the next.
2026-10-11 05:26:48 -07:00
|
|
|
* ``remote_backends`` — the SSH and Docker terminal backends against a real
|
|
|
|
|
sshd container and the runner's Docker daemon (remote-backends.yml). Every
|
|
|
|
|
other test of those backends is mocked, so a change to the backends, the
|
|
|
|
|
terminal tool, or the real-backend tests runs it on the PR.
|
2026-06-23 09:13:19 -04:00
|
|
|
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
``docker``, ``nix`` and the E2E lanes take most of the larger-runner minutes.
|
|
|
|
|
An ordinary product change does not start them on a pull request. Every push
|
|
|
|
|
to main runs them all (a push has no diff, so every lane is on), so a
|
|
|
|
|
regression they catch shows up on main after the merge. A stable-release run
|
|
|
|
|
forces every lane on.
|
2026-06-19 16:46:11 -05:00
|
|
|
|
|
|
|
|
Contract — *fail open, never closed*. We may run a lane we didn't need, but
|
|
|
|
|
must never skip one a change could break:
|
|
|
|
|
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
* The slow lanes above are the one deliberate exception. On a pull request
|
|
|
|
|
they skip changes that could still break them, and main is where those
|
|
|
|
|
changes meet the suite. Their path lists decide which PRs run them early.
|
2026-06-19 16:46:11 -05:00
|
|
|
* An empty diff, or any ``.github/`` change, runs everything.
|
|
|
|
|
* ``python`` is a denylist: skipped only when *every* file is provably prose
|
|
|
|
|
or a frontend-only package; an unrecognized path keeps it on.
|
|
|
|
|
* ``skills/`` (incl. ``SKILL.md``) is python-relevant — the skill-doc tests
|
|
|
|
|
read that tree, so a doc-looking edit can still break Python.
|
2026-08-18 18:00:56 -04:00
|
|
|
* ``nix/``, ``flake.nix`` and ``flake.lock`` are the exception the other way:
|
|
|
|
|
only the flake reads them, so they skip the Python lanes and run ``nix``
|
|
|
|
|
alone. ``pyproject.toml`` and ``uv.lock`` are flake inputs too, but the
|
|
|
|
|
packaging tests read them, so they keep every Python lane.
|
2026-08-11 14:17:11 -04:00
|
|
|
* ``website/static/oauth/`` is python-relevant too: it publishes the OAuth
|
|
|
|
|
Client ID Metadata Document that ``tests/tools/test_mcp_cimd.py`` checks
|
|
|
|
|
against the pinned callback ports in ``tools/mcp_oauth.py``.
|
2026-08-19 12:05:41 -05:00
|
|
|
* ``website/docs/`` and ``website/scripts/`` are python-relevant for the same
|
|
|
|
|
reason: the docs tree generates ``llms.txt``, and
|
|
|
|
|
``tests/website/test_generate_llms_txt.py`` asserts every page reaches it.
|
2026-10-04 01:54:46 -07:00
|
|
|
* A cross-language fixture (``_SHARED_FIXTURE_CONSUMERS``) selects the test
|
|
|
|
|
lanes of every consumer that reads it: the update-marker corpus runs pytest,
|
|
|
|
|
cargo, vitest and the Windows hand-off tests, not only ``python``.
|
2026-06-19 16:46:11 -05:00
|
|
|
"""
|
|
|
|
|
|
|
|
|
|
from __future__ import annotations
|
|
|
|
|
|
2026-07-22 19:19:53 -04:00
|
|
|
import json
|
2026-06-19 16:46:11 -05:00
|
|
|
import os
|
2026-08-18 23:37:18 +07:00
|
|
|
import subprocess
|
2026-06-19 16:46:11 -05:00
|
|
|
import sys
|
|
|
|
|
|
|
|
|
|
_FRONTEND = ("ui-tui/", "web/", "apps/") # TS typecheck-matrix packages
|
2026-09-05 11:07:05 -05:00
|
|
|
# Shipped page outside those packages, exercised by the desktop Electron suite.
|
|
|
|
|
_FRONTEND_FILES = {"scripts/desktop-update/ui.html"}
|
2026-06-19 16:46:11 -05:00
|
|
|
_ROOT_NPM = {"package.json", "package-lock.json"} # shifts every package's tree
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
_DOCKER_META = ("docker/", ".hadolint.yaml", "Dockerfile", ".dockerignore") # docker setup
|
2026-08-18 18:00:56 -04:00
|
|
|
_NIX_PATHS = ("nix/",) # nix files
|
|
|
|
|
_NIX_FILES = {"flake.nix", "flake.lock"} # base nix files
|
2026-06-19 16:46:11 -05:00
|
|
|
_SITE = ("website/", "skills/", "optional-skills/") # docs site + skill pages
|
|
|
|
|
# Prose/frontend trees that can't touch Python. skills/ is excluded on purpose.
|
|
|
|
|
_PY_SKIP = ("docs/", "website/") + _FRONTEND
|
2026-08-11 14:17:11 -04:00
|
|
|
# Published artifacts that live under website/ but that Python asserts about.
|
|
|
|
|
# The OAuth Client ID Metadata Document is cross-checked against the pinned
|
|
|
|
|
# callback ports in tools/mcp_oauth.py, so editing it alone must still run the
|
|
|
|
|
# Python lane — otherwise dropping a redirect URI goes green here and breaks
|
|
|
|
|
# every CIMD login on main.
|
2026-08-19 12:05:41 -05:00
|
|
|
# website/docs/ and website/scripts/ are asserted about the same way. The docs
|
|
|
|
|
# tree generates llms.txt — the index every LLM (Hermes included, via the
|
|
|
|
|
# hermes-agent skill) reads to learn what Hermes can do — and
|
|
|
|
|
# tests/website/test_generate_llms_txt.py holds every page to appearing in it.
|
|
|
|
|
# Skipping Python on a docs-only PR is how the index drifted to 53% coverage.
|
|
|
|
|
_PY_RELEVANT_SITE = (
|
|
|
|
|
"website/static/oauth/",
|
|
|
|
|
"website/docs/",
|
|
|
|
|
"website/scripts/",
|
|
|
|
|
)
|
2026-09-12 23:57:42 -07:00
|
|
|
# Cross-language contract files: data committed under a frontend tree that a
|
|
|
|
|
# pytest pins against the Python side (emitter inventory, command registry).
|
|
|
|
|
# Editing only the JSON in an apps/-only PR would otherwise skip the one test
|
|
|
|
|
# that can catch the drift, so these force the Python lane too.
|
|
|
|
|
_PY_RELEVANT_CONTRACT_FILES = {
|
2026-09-14 02:47:51 -07:00
|
|
|
# tests/tui_gateway/contracts/test_generated.py (rendered from tui_gateway/contracts)
|
2026-09-14 01:29:49 -07:00
|
|
|
"apps/shared/src/gateway-contract.generated.ts",
|
|
|
|
|
"apps/shared/src/gateway-contract.openrpc.json",
|
2026-09-12 23:57:42 -07:00
|
|
|
# tests/hermes_cli/test_desktop_slash_registry.py
|
|
|
|
|
"apps/desktop/src/lib/desktop-slash-registry.json",
|
2026-09-29 12:27:26 -04:00
|
|
|
# tests/tui_gateway/test_show_reasoning_display_gate.py (card-tool names vs the gateway lifecycle set)
|
|
|
|
|
"apps/desktop/src/lib/tool-render-class.ts",
|
docs: one complete plugin catalog submission guide, synced with the README rules
Submitters only saw a stale 5-point summary on the catalog page (it even listed
a 'Released' requirement nothing enforces), while reviewers applied ten README
rules plus criteria written nowhere public (credentials, approvals, disclosure,
compatibility metadata, forks of bundled plugins).
- plugin-catalog/README.md: rules 11-15 write down those review criteria; the
rule list sits between admission-rules markers.
- New developer-guide page 'Submitting to the plugin catalog': before you
submit, the PR flow, the full rule block (verbatim mirror), pin updates,
delisting vs removal, asking for a hook.
- The /plugins 'Submit' links and the user-guide section point at it.
- tests/website/test_catalog_rules_mirror.py fails when the two rule blocks
differ; classify_changes makes a README-only edit run it.
- AGENTS.md, plugins/AGENTS.md and CONTRIBUTING.md reference both files.
2026-10-01 17:36:22 -07:00
|
|
|
# tests/website/test_catalog_rules_mirror.py (docs page mirrors the canonical rules)
|
|
|
|
|
"plugin-catalog/README.md",
|
2026-09-12 23:57:42 -07:00
|
|
|
}
|
2026-06-19 16:46:11 -05:00
|
|
|
|
2026-10-16 04:23:57 -07:00
|
|
|
# Catalog entry data: read by the catalog tests and the Plugin Catalog CI
|
|
|
|
|
# admission gate, executed by nothing. Same set that gate allows in an entry PR.
|
|
|
|
|
_CATALOG_DATA = ("plugin-catalog/", "contributors/emails/")
|
|
|
|
|
|
2026-06-23 09:13:19 -04:00
|
|
|
# Supply-chain scan: files that can execute code at install/import time.
|
|
|
|
|
_SCAN_EXTS = (".py", ".pth")
|
|
|
|
|
_SCAN_FILES = {"setup.cfg", "pyproject.toml"}
|
|
|
|
|
|
feat(pm): unified package manager — pm store foundation
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
2026-08-29 21:23:27 -04:00
|
|
|
# Bootstrap installer: the POSIX shell installer, the dev-checkout wrapper
|
|
|
|
|
# that carries the same pin fragment, and the Tauri app's non-Rust sources
|
|
|
|
|
# (the .rs/Cargo files are the ``rust`` lane's job). Changes here get the
|
|
|
|
|
# bootstrap-installer.yml lane — a real sandboxed install + stamp check.
|
|
|
|
|
_BOOTSTRAP_PATHS = ("apps/bootstrap-installer/",)
|
|
|
|
|
_BOOTSTRAP_FILES = {"scripts/install.sh", "setup-hermes.sh"}
|
2026-09-01 21:58:50 -07:00
|
|
|
# Windows desktop-update hand-off (scripts/desktop-update/windows.ps1 + the
|
|
|
|
|
# Electron side that launches it) and the pytest files that spawn it.
|
2026-09-24 14:40:09 -04:00
|
|
|
# tests/_fixtures/ holds the conftest's platform gating, so it re-arms the lane too.
|
|
|
|
|
_DESKTOP_UPDATER_PATHS = ("scripts/desktop-update/", "tests/_fixtures/")
|
test: make tests/ mirror the source tree; drop issue numbers from filenames
`scripts/run_tests.sh tests/<dir>/` is how a change gets its regression
coverage run, so a test filed under the wrong directory is a test nobody
runs when that code changes. Two kinds of drift had accumulated.
Parallel directories for one source package, folded into the mirror:
tests/acp -> tests/acp_adapter (its __init__/conftest move with it)
tests/cli -> tests/hermes_cli (prompt_toolkit fixture merged into
hermes_cli/conftest.py)
tests/run_agent -> tests/agent (backoff fixture becomes
agent/conftest.py)
tests/relay -> tests/gateway/relay
tests/state -> tests/hermes_state
246 loose files at tests/ root, routed by the package they import/patch:
hermes_cli, hermes_state, agent, gateway, tools, plugins, tui_gateway, cron.
Installer and desktop-update script tests go to tests/scripts/{install,
desktop_update}/. 43 tests of root-level modules (batch_runner, utils,
hermes_constants, packaging) stay at the root.
Filenames drop their issue numbers (95 files: test_89315_x.py -> test_x.py);
the number stays in the module docstring where it has context.
Collisions: test_cli_skin_integration.py existed in both tests/ and tests/cli
with different subsets — merged into one (10 tests, all kept);
run_agent/test_pre_compress_memory_context.py -> agent/..._handoff.py;
tests/test_account_usage.py -> agent/test_account_usage_fetch.py;
tests/test_web_server.py -> hermes_cli/test_web_server_ws_ping.py.
Deleted: test_minisweagent_path.py (empty since PR #2804),
test_model_picker_scroll.py (tested a private copy of the logic, imported
nothing), test_process_loop_event_loop_warning.py (asserted asyncio behaviour,
imported nothing from Hermes).
Repo-root path arithmetic (Path(__file__).parents[N], dirname chains) is
bumped for the 202 files that changed depth and verified by evaluating every
such expression against the new location. classify_changes' desktop-updater
lane prefix, tests-os.yml's ignore glob and every in-tree path comment follow
the moves. tests/test_tests_tree_layout.py keeps the tree from drifting back.
2026-09-13 06:27:46 -07:00
|
|
|
_DESKTOP_UPDATER_TEST_PREFIX = "tests/scripts/desktop_update/"
|
2026-09-01 21:58:50 -07:00
|
|
|
_DESKTOP_UPDATER_FILES = {
|
|
|
|
|
"apps/desktop/electron/updater-process.ts",
|
|
|
|
|
"apps/desktop/electron/managed-ssh-update.ts",
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
# The other half of the marker / result contract the script implements.
|
|
|
|
|
"apps/desktop/electron/update-marker.ts",
|
2026-10-04 07:47:03 -07:00
|
|
|
"apps/desktop/electron/update-marker-gate.ts", # the gate's live-marker probe
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
"apps/desktop/electron/handoff-result.ts",
|
2026-10-04 07:47:03 -07:00
|
|
|
# Stops a remote backend for the update the hand-off script then runs.
|
|
|
|
|
"apps/desktop/electron/remote-lifecycle.ts",
|
fix(update): CI routes each update-path file to the suites that exercise it
- hermes_cli/web_routers/status.py (review CI2): /api/health `commit` is what
host-backend-attach.ts compares to the checkout before attaching after an
update, yet an edit ran only the python lane. Now also e2e_desktop_update,
the lane host-backend-attach already starts.
- Tauri updater (review CI3): marker.rs started e2e_upgrade and
e2e_desktop_update, neither of which builds or runs the bootstrap installer
(no workflow of either lane touches apps/bootstrap-installer or cargo),
while its callers update.rs/paths.rs started none. All three now route as
one unit to the lane that compiles and tests them: rust (cargo test, via
.rs) and nothing slow. _TAURI_UPDATE_MARKER is gone.
- remote-update-marker-programs.ts (+ test) (review CI3): the SSH remote's
marker judge/gate ran only frontend; it is the marker.sh/marker.ps1
contract run remotely, and its importers (remote-lifecycle.ts,
managed-ssh-update.ts) are desktop_updater files, so it is one too.
- gateway/status (review CI3): the prefix also matched status_phrases.py
(turn-status wording); the named status_inline_source sibling does not
exist. Tightened to gateway/status.py.
Changed assertion: _R6_CASES pinned marker.rs -> (rust, e2e_upgrade,
e2e_desktop_update); the finding is that the two e2e lanes are the bug (they
never build marker.rs), so it now pins (rust,), and a new test pins the
three Tauri updater files off every slow lane.
2026-10-05 01:43:55 -07:00
|
|
|
# The SSH remote's marker judge/gate programs: marker.sh/marker.ps1's contract, run remotely.
|
|
|
|
|
"apps/desktop/electron/remote-update-marker-programs.ts",
|
|
|
|
|
"apps/desktop/electron/remote-update-marker-programs.test.ts",
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
# Python the script runs: the post-update verify and the staged app swap.
|
|
|
|
|
"hermes_cli/desktop_update_verify.py",
|
|
|
|
|
"hermes_cli/main_desktop.py",
|
2026-09-01 21:58:50 -07:00
|
|
|
"tests/conftest.py",
|
|
|
|
|
"pyproject.toml",
|
|
|
|
|
}
|
feat(pm): unified package manager — pm store foundation
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
2026-08-29 21:23:27 -04:00
|
|
|
|
2026-10-04 01:54:46 -07:00
|
|
|
# Cross-language fixtures: one data file that tests in several languages read as
|
|
|
|
|
# their shared contract. Editing it is editing every consumer, so it selects each
|
|
|
|
|
# lane that runs one (a path-prefix rule would see only ``tests/`` -> python, and
|
|
|
|
|
# the cargo / vitest / PowerShell readers of the same cases would never run).
|
|
|
|
|
# tests/ci/test_update_ci_routing.py finds the consumers in the tree and fails
|
|
|
|
|
# until every one is listed here.
|
|
|
|
|
_SHARED_FIXTURE_CONSUMERS: dict[str, tuple[str, ...]] = {
|
|
|
|
|
# A7 rule 7: the update-marker parse / judge / release corpus.
|
|
|
|
|
"tests/fixtures/update_marker_corpus.json": (
|
|
|
|
|
"tests/hermes_cli/test_update_marker_corpus.py", # Python: update_lock
|
|
|
|
|
"apps/bootstrap-installer/src-tauri/src/marker_tests.rs", # Rust: cargo test
|
|
|
|
|
"apps/desktop/electron/update-marker-corpus.test.ts", # Electron: vitest
|
|
|
|
|
"tests/scripts/desktop_update/test_desktop_update_posix_marker_corpus.py", # marker.sh
|
|
|
|
|
"tests/scripts/desktop_update/test_desktop_update_windows_marker_corpus.py", # marker.ps1
|
2026-10-05 01:07:05 -07:00
|
|
|
"apps/desktop/electron/remote-update-marker-programs.test.ts", # SSH remote judge
|
2026-10-06 00:19:34 -07:00
|
|
|
"apps/desktop/electron/remote-lifecycle-v2-marker.test.ts", # SSH relaunch/spawn gate
|
2026-10-04 01:54:46 -07:00
|
|
|
),
|
|
|
|
|
}
|
|
|
|
|
# What a fixture inherits from its consumers: the lanes that run them as tests.
|
|
|
|
|
# Not the slow suites a consumer's path also matches (an Electron test file under
|
|
|
|
|
# electron/update-* starts the Desktop update E2E, which never reads the corpus).
|
|
|
|
|
_FIXTURE_CONSUMER_LANES = ("python", "rust", "frontend", "desktop_updater")
|
|
|
|
|
|
2026-08-20 11:40:49 -05:00
|
|
|
# Rust crates — currently just the Tauri bootstrap installer (Hermes-Setup).
|
|
|
|
|
# These live under ``apps/``, so before this lane existed a ``.rs`` edit matched
|
|
|
|
|
# ``frontend`` and nothing more: the TypeScript matrix built, cargo never ran,
|
|
|
|
|
# and the crate's unit tests had never executed in CI at all.
|
|
|
|
|
_RUST_PATHS = ("apps/bootstrap-installer/src-tauri/",)
|
|
|
|
|
_RUST_FILENAMES = {"Cargo.toml", "Cargo.lock"}
|
|
|
|
|
|
ci: run the SSH and Docker terminal backends against a real sshd and daemon
The SSH backend's real-host tests skipped on every CI run (nothing set
TERMINAL_SSH_HOST), and every Docker backend test mocked subprocess or the
docker CLI, so no CI run had executed either backend end to end.
- remote-backends.yml: starts a throwaway sshd container on 127.0.0.1 with a
per-run keypair (scripts/ci/start_test_sshd.sh) and runs
test_ssh_environment.py plus the new test_docker_backend_real.py against it
and the runner's Docker daemon with a digest-pinned debian:12-slim. A file
that skips fails the job, so the lane cannot go green over zero coverage.
- New classifier lane `remote_backends` (tools/environments/, tools/terminal_tool*,
the real tests, the sshd harness, run_tests.sh). ci.yaml calls the workflow
only when it fires, plus every push to main; required_results makes a selected
lane mandatory, like the update consumers.
- The real tests read HERMES_TEST_SSH_* / HERMES_TEST_DOCKER_IMAGE (forwarded
by run_tests.sh), because the hermetic conftest blanks TERMINAL_SSH_KEY as a
credential before any test can read it.
- New real SSH cases: exit code/stderr, cwd persistence with spaces vs per-call
workdir, quoting across the ssh hop, SendEnv passthrough through a real
AcceptEnv, file tools write/read on the remote.
- New real Docker cases: exec + exit codes, cwd vs workdir, exported env
persistence, docker_env/docker_forward_env reach the container and
nothing else does, quoting, file tools write/read, /root surviving a
container replacement.
- The existing one-shot test is xfail(strict): TERMINAL_SSH_PERSISTENT=false
is read into ssh_config["persistent"] but SSHEnvironment never consumes it,
so every SSH session persists exported env. Its _cleanup also cleaned the
raw task id, not the resolved env key, so one test's env leaked into the next.
2026-10-11 05:26:48 -07:00
|
|
|
# The real SSH/Docker backend lane: the backends, the terminal tool that drives
|
|
|
|
|
# them, the tests it runs, the sshd it starts, and the runner's env allowlist
|
|
|
|
|
# that carries the sshd/image coordinates into each pytest process.
|
|
|
|
|
_REMOTE_BACKEND_PATHS = (
|
|
|
|
|
"tools/environments/",
|
|
|
|
|
"tools/terminal_tool",
|
|
|
|
|
"tests/tools/test_ssh_environment.py",
|
|
|
|
|
"tests/tools/test_docker_backend_real.py",
|
|
|
|
|
"scripts/ci/start_test_sshd.sh",
|
2026-10-11 11:43:05 -07:00
|
|
|
"scripts/ci/test_sshd_container_setup.sh",
|
ci: run the SSH and Docker terminal backends against a real sshd and daemon
The SSH backend's real-host tests skipped on every CI run (nothing set
TERMINAL_SSH_HOST), and every Docker backend test mocked subprocess or the
docker CLI, so no CI run had executed either backend end to end.
- remote-backends.yml: starts a throwaway sshd container on 127.0.0.1 with a
per-run keypair (scripts/ci/start_test_sshd.sh) and runs
test_ssh_environment.py plus the new test_docker_backend_real.py against it
and the runner's Docker daemon with a digest-pinned debian:12-slim. A file
that skips fails the job, so the lane cannot go green over zero coverage.
- New classifier lane `remote_backends` (tools/environments/, tools/terminal_tool*,
the real tests, the sshd harness, run_tests.sh). ci.yaml calls the workflow
only when it fires, plus every push to main; required_results makes a selected
lane mandatory, like the update consumers.
- The real tests read HERMES_TEST_SSH_* / HERMES_TEST_DOCKER_IMAGE (forwarded
by run_tests.sh), because the hermetic conftest blanks TERMINAL_SSH_KEY as a
credential before any test can read it.
- New real SSH cases: exit code/stderr, cwd persistence with spaces vs per-call
workdir, quoting across the ssh hop, SendEnv passthrough through a real
AcceptEnv, file tools write/read on the remote.
- New real Docker cases: exec + exit codes, cwd vs workdir, exported env
persistence, docker_env/docker_forward_env reach the container and
nothing else does, quoting, file tools write/read, /root surviving a
container replacement.
- The existing one-shot test is xfail(strict): TERMINAL_SSH_PERSISTENT=false
is read into ssh_config["persistent"] but SSHEnvironment never consumes it,
so every SSH session persists exported env. Its _cleanup also cleaned the
raw task id, not the resolved env key, so one test's env leaked into the next.
2026-10-11 05:26:48 -07:00
|
|
|
"scripts/run_tests.sh",
|
|
|
|
|
)
|
|
|
|
|
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
# The slow lanes and the paths that start each one on a pull request. A path
|
|
|
|
|
# belongs here when the lane is the only CI that exercises it for real: the
|
|
|
|
|
# suite itself, its harness, and the product code the suite exists to guard.
|
|
|
|
|
# Everything else reaches these lanes on the next push to main.
|
|
|
|
|
_DEP_MANIFESTS = ("pyproject.toml", "uv.lock", "setup.py")
|
|
|
|
|
_NPM_MANIFESTS = ("package.json", "package-lock.json")
|
|
|
|
|
# Shared pytest harness: a bad edit here can break every Python E2E suite.
|
|
|
|
|
_PY_TEST_HARNESS = (
|
|
|
|
|
"tests/conftest.py",
|
|
|
|
|
"tests/_fixtures/",
|
|
|
|
|
"tests/fakes/",
|
|
|
|
|
"tests/e2e/conftest.py",
|
|
|
|
|
"tests/e2e/core/_",
|
|
|
|
|
"scripts/run_tests.sh",
|
|
|
|
|
"scripts/run_tests_parallel.py",
|
|
|
|
|
)
|
|
|
|
|
# Both Desktop suites package the app and share this harness (the update
|
|
|
|
|
# suite imports the core suite's harness and provider).
|
|
|
|
|
_DESKTOP_E2E_SHARED = (
|
|
|
|
|
"apps/desktop/package.json",
|
|
|
|
|
"apps/desktop/electron-builder.config.cjs",
|
|
|
|
|
"apps/desktop/scripts/",
|
|
|
|
|
"apps/desktop/e2e/core/harness",
|
|
|
|
|
"apps/desktop/e2e/core/provider",
|
|
|
|
|
"apps/desktop/e2e/electron-binary",
|
|
|
|
|
"apps/desktop/e2e/fix-electron-tracing",
|
|
|
|
|
"apps/desktop/e2e/run-tmp",
|
|
|
|
|
)
|
fix(update): CI routes each update-path file to the suites that exercise it
- hermes_cli/web_routers/status.py (review CI2): /api/health `commit` is what
host-backend-attach.ts compares to the checkout before attaching after an
update, yet an edit ran only the python lane. Now also e2e_desktop_update,
the lane host-backend-attach already starts.
- Tauri updater (review CI3): marker.rs started e2e_upgrade and
e2e_desktop_update, neither of which builds or runs the bootstrap installer
(no workflow of either lane touches apps/bootstrap-installer or cargo),
while its callers update.rs/paths.rs started none. All three now route as
one unit to the lane that compiles and tests them: rust (cargo test, via
.rs) and nothing slow. _TAURI_UPDATE_MARKER is gone.
- remote-update-marker-programs.ts (+ test) (review CI3): the SSH remote's
marker judge/gate ran only frontend; it is the marker.sh/marker.ps1
contract run remotely, and its importers (remote-lifecycle.ts,
managed-ssh-update.ts) are desktop_updater files, so it is one too.
- gateway/status (review CI3): the prefix also matched status_phrases.py
(turn-status wording); the named status_inline_source sibling does not
exist. Tightened to gateway/status.py.
Changed assertion: _R6_CASES pinned marker.rs -> (rust, e2e_upgrade,
e2e_desktop_update); the finding is that the two e2e lanes are the bug (they
never build marker.rs), so it now pins (rust,), and a new test pins the
three Tauri updater files off every slow lane.
2026-10-05 01:43:55 -07:00
|
|
|
# The Tauri updater (update.rs, its marker claim marker.rs, the paths.rs both resolve) is
|
|
|
|
|
# compiled and tested by ``cargo test`` alone (the ``rust`` lane, via ``.rs``): no slow lane
|
|
|
|
|
# builds the bootstrap installer, so none of the three starts one.
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
# What `hermes update` runs outside the update_* module family: the steps of the
|
|
|
|
|
# pipeline (entry, lock, early recovery, completion tail, launchers, fleet
|
|
|
|
|
# restart/verify, Windows pause/resume) and the lock / marker / recovery state
|
|
|
|
|
# the update_* modules import. Editing any of these changes what a real update
|
|
|
|
|
# does, so the real-update suites (Linux e2e-upgrade and the Windows
|
|
|
|
|
# install + update journey, including its crash cells) must run on the PR.
|
|
|
|
|
_UPDATE_PIPELINE = (
|
|
|
|
|
"hermes_cli/main.py", # cmd_update: lock, pre-update backup, receipt boundary
|
|
|
|
|
"hermes_cli/main_dashboard.py", # hangup protection + update.log mirror
|
2026-10-05 01:07:05 -07:00
|
|
|
# Prefix: main_desktop.py (staged Desktop swap / rebuild in the tail) and the
|
|
|
|
|
# main_desktop_* siblings it imports (macOS signing identity).
|
|
|
|
|
"hermes_cli/main_desktop",
|
2026-10-06 00:19:34 -07:00
|
|
|
# Prefix: _early_recovery.py and its _early_recovery_* siblings (ZIP swap journal).
|
|
|
|
|
"hermes_cli/_early_recovery", # interrupted pull / shim restore at launch
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
"hermes_cli/venv_sync.py", # completion obligation + launch-time tail
|
|
|
|
|
"hermes_cli/source_", # source_completion/_build/_releases/_check/_stamp
|
|
|
|
|
"hermes_cli/_launchers.py",
|
|
|
|
|
"hermes_cli/release_channels.py",
|
|
|
|
|
"hermes_cli/gitlock.py", # git self-heal + partial-clone fetch
|
2026-10-25 14:05:35 -07:00
|
|
|
"hermes_cli/git_commit_graph.py", # every update starts the install checkout's graph refresh
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
"hermes_cli/process_identity.py", # marker owner liveness
|
|
|
|
|
"hermes_cli/runtime_state.py",
|
|
|
|
|
"hermes_cli/relaunch.py",
|
|
|
|
|
"hermes_cli/managed_uv.py",
|
|
|
|
|
"hermes_cli/npm_engine.py",
|
|
|
|
|
"hermes_cli/_scan_venv_blockers.py",
|
|
|
|
|
"hermes_cli/dashboard_procs.py",
|
|
|
|
|
"hermes_cli/desktop_update", # desktop_update_verify
|
|
|
|
|
"hermes_cli/gateway.py", # fleet restart / verify
|
|
|
|
|
"hermes_cli/gateway_windows", # Windows pause / resume
|
|
|
|
|
"hermes_cli/gateway_launchd.py",
|
|
|
|
|
"hermes_cli/gateway_migrate",
|
|
|
|
|
"hermes_cli/gateway_supervised_restart.py",
|
2026-10-05 03:45:12 -07:00
|
|
|
"hermes_cli/git_pack_tidy.py", # partial-clone pack tidy: every update's pre-fetch runs it
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
"hermes_bootstrap.py", # every launch's prepare_launch
|
|
|
|
|
"hermes_constants.py", # root home = update marker location
|
2026-10-17 04:48:55 -07:00
|
|
|
"hermes_dir_acl.py", # update restores the Windows user ACL on the home after elevation
|
fix(update): CI routes each update-path file to the suites that exercise it
- hermes_cli/web_routers/status.py (review CI2): /api/health `commit` is what
host-backend-attach.ts compares to the checkout before attaching after an
update, yet an edit ran only the python lane. Now also e2e_desktop_update,
the lane host-backend-attach already starts.
- Tauri updater (review CI3): marker.rs started e2e_upgrade and
e2e_desktop_update, neither of which builds or runs the bootstrap installer
(no workflow of either lane touches apps/bootstrap-installer or cargo),
while its callers update.rs/paths.rs started none. All three now route as
one unit to the lane that compiles and tests them: rust (cargo test, via
.rs) and nothing slow. _TAURI_UPDATE_MARKER is gone.
- remote-update-marker-programs.ts (+ test) (review CI3): the SSH remote's
marker judge/gate ran only frontend; it is the marker.sh/marker.ps1
contract run remotely, and its importers (remote-lifecycle.ts,
managed-ssh-update.ts) are desktop_updater files, so it is one too.
- gateway/status (review CI3): the prefix also matched status_phrases.py
(turn-status wording); the named status_inline_source sibling does not
exist. Tightened to gateway/status.py.
Changed assertion: _R6_CASES pinned marker.rs -> (rust, e2e_upgrade,
e2e_desktop_update); the finding is that the two e2e lanes are the bug (they
never build marker.rs), so it now pins (rust,), and a new test pins the
three Tauri updater files off every slow lane.
2026-10-05 01:43:55 -07:00
|
|
|
"gateway/status.py", # code_sha stamp the fleet verify reads
|
2026-10-06 06:48:46 -07:00
|
|
|
"gateway/status_inline_source.py", # the Windows pause identifies inline-bootstrapped gateways
|
2026-10-17 06:48:54 -07:00
|
|
|
"gateway/status_lock_dir.py", # gateway/status.py imports it at module level
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
"gateway/control_socket.py", # pause-for-update verb
|
|
|
|
|
"gateway/code_skew.py",
|
|
|
|
|
"gateway/host_rendezvous.py",
|
|
|
|
|
"gateway/shutdown_forensics.py",
|
|
|
|
|
"gateway/restart.py",
|
|
|
|
|
"scripts/desktop-update/", # the hand-off scripts run `hermes update`
|
|
|
|
|
)
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
# Selectors are owned by the import graph, not remembered. The update
|
|
|
|
|
# transaction's own modules (what `hermes update` and the launch-time completion
|
|
|
|
|
# run between the lock and the receipt) are the entry points;
|
|
|
|
|
# tests/ci/test_update_ci_routing.py reads every repo module they import (AST,
|
|
|
|
|
# module level and lazy) and every build script they run, and fails until each
|
|
|
|
|
# one is routed to the suite below or is a _SHARED_HUBS entry.
|
|
|
|
|
_UPDATE_ENTRY_POINTS = (
|
|
|
|
|
"hermes_cli/update_",
|
|
|
|
|
"hermes_cli/_update_",
|
|
|
|
|
"hermes_cli/source_",
|
|
|
|
|
"hermes_cli/old_updater",
|
|
|
|
|
"hermes_cli/_old_updater",
|
|
|
|
|
"hermes_cli/post_update",
|
|
|
|
|
"hermes_cli/venv_sync.py",
|
2026-10-06 00:19:34 -07:00
|
|
|
"hermes_cli/_early_recovery",
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
"hermes_cli/main_desktop.py",
|
|
|
|
|
"hermes_cli/desktop_update_verify.py",
|
|
|
|
|
"hermes_cli/desktop_build_lock.py",
|
|
|
|
|
"hermes_cli/subcommands/update",
|
|
|
|
|
)
|
|
|
|
|
# The entry points that build or verify the Desktop app inside an update
|
|
|
|
|
# (`hermes desktop --build-only`, the source build/completion that feeds it).
|
|
|
|
|
_DESKTOP_BUILD_ENTRY_POINTS = (
|
|
|
|
|
"hermes_cli/source_build.py",
|
|
|
|
|
"hermes_cli/source_completion.py",
|
|
|
|
|
"hermes_cli/main_desktop.py",
|
|
|
|
|
"hermes_cli/desktop_update_verify.py",
|
|
|
|
|
"hermes_cli/desktop_build_lock.py",
|
|
|
|
|
)
|
|
|
|
|
# What the entry points import, outside the update_* family and the pipeline above.
|
|
|
|
|
_UPDATE_DEPENDENCIES = (
|
|
|
|
|
"hermes_cli/_subprocess_compat.py", # update git env, process-tree kill, PM git exposure
|
2026-10-04 12:57:58 -07:00
|
|
|
"hermes_cli/local_runtime/processes.py", # bounded probes' spawn_server/job custody
|
|
|
|
|
"agent/deadline.py", # bounded probes' process-tree timeout cleanup
|
fix: plugin-install modules no longer start the update journeys they never reach
Replaying the last 131 non-merge main commits (4787e4d56fc) through the
classifier: main selects e2e_upgrade 5 / e2e_desktop_update 3, the CI branch
12 / 10. Four of the seven new hits per lane touched only plugins_cmd.py or
plugins_cmd_install.py. migrate_all_homes reaches them only when a home's
configured memory provider left core (the catalog install); no upgrade or
Desktop update journey configures one, so those suites cannot observe the
change, and the unit lane already runs on every .py edit. Measured on a
fresh home: memory_provider_migration, pm/plugins_state, agent/memory_provider
and plugins/memory run; plugins_cmd_install executes only its module body,
plugins_cmd is never imported.
Kept, because a fresh-home update runs them: memory_provider_migration.py
(migrate_all_homes on every source build), plugins/memory/__init__.py (the
post-update Honcho profile sync), pm/install.py (sealed() /
lazy_installs_allowed() gate every update's default-tool install), and
scripts/build/ (frontend-common.mjs is imported by freshness/web/tui/desktop).
After: 8 / 6. A guard test fails if the journey-home migration starts running
plugin-install code, the signal to route those modules again.
2026-10-05 22:28:10 -07:00
|
|
|
# migrate_all_homes' second-hop provider/profile decisions, run on every update. Its
|
|
|
|
|
# plugin-install branch (plugins_cmd, plugins_cmd_install) runs only for a home whose
|
|
|
|
|
# configured memory provider left core; no update journey's home has one, so those
|
|
|
|
|
# modules stay with the unit lane (tests/ci/test_update_transitive_routing.py).
|
2026-10-04 12:57:58 -07:00
|
|
|
"agent/memory_provider.py",
|
|
|
|
|
"pm/plugins_state.py",
|
fix: plugin-install modules no longer start the update journeys they never reach
Replaying the last 131 non-merge main commits (4787e4d56fc) through the
classifier: main selects e2e_upgrade 5 / e2e_desktop_update 3, the CI branch
12 / 10. Four of the seven new hits per lane touched only plugins_cmd.py or
plugins_cmd_install.py. migrate_all_homes reaches them only when a home's
configured memory provider left core (the catalog install); no upgrade or
Desktop update journey configures one, so those suites cannot observe the
change, and the unit lane already runs on every .py edit. Measured on a
fresh home: memory_provider_migration, pm/plugins_state, agent/memory_provider
and plugins/memory run; plugins_cmd_install executes only its module body,
plugins_cmd is never imported.
Kept, because a fresh-home update runs them: memory_provider_migration.py
(migrate_all_homes on every source build), plugins/memory/__init__.py (the
post-update Honcho profile sync), pm/install.py (sealed() /
lazy_installs_allowed() gate every update's default-tool install), and
scripts/build/ (frontend-common.mjs is imported by freshness/web/tui/desktop).
After: 8 / 6. A guard test fails if the journey-home migration starts running
plugin-install code, the signal to route those modules again.
2026-10-05 22:28:10 -07:00
|
|
|
"pm/install.py", # also sealed()/lazy_installs_allowed(): every update's default-tool install
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
"hermes_cli/desktop_build_lock.py",
|
|
|
|
|
"hermes_cli/memory_provider_migration.py",
|
2026-10-04 00:49:23 -07:00
|
|
|
"hermes_cli/left_core_migration.py", # source_build migrates plugins that left core
|
2026-10-23 01:27:07 -07:00
|
|
|
"hermes_cli/config_version_stamp.py", # post_update converts legacy AGENT_BROWSER_* before stamping
|
2026-10-07 16:27:08 -07:00
|
|
|
"hermes_cli/web_build_limits.py", # source_build caps the dashboard Node build
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
"hermes_cli/desktop_console.py",
|
|
|
|
|
"hermes_cli/bundled_app.py",
|
|
|
|
|
"hermes_cli/gui_uninstall.py",
|
|
|
|
|
"hermes_cli/linux_desktop_entry.py",
|
|
|
|
|
"hermes_cli/github_api.py", # source_check's release lookup
|
|
|
|
|
"hermes_cli/build_info.py",
|
|
|
|
|
"hermes_cli/image_provenance.py",
|
|
|
|
|
"hermes_cli/backup.py", # pre-update backup
|
2026-10-07 11:37:25 -07:00
|
|
|
"hermes_cli/backup_cron_prompts.py", # restore_cron_prompt_fields_if_degraded (update_cmd_config)
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
"hermes_cli/backup_restore.py",
|
2026-10-07 10:06:29 +11:00
|
|
|
"hermes_cli/config_backend.py", # update_channel writes the channel key through the seam
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
"hermes_cli/relay_plugin_migrate.py",
|
|
|
|
|
"hermes_cli/macos_tcc_anchor.py",
|
|
|
|
|
"hermes_cli/model_catalog.py",
|
|
|
|
|
"hermes_cli/sqlite_runtime.py",
|
|
|
|
|
"hermes_cli/sqlite_safe_read.py",
|
|
|
|
|
"hermes_cli/sizefmt.py",
|
|
|
|
|
"hermes_cli/tools_config_cua.py",
|
|
|
|
|
"hermes_cli/_startup_fast.py",
|
|
|
|
|
"hermes_cli/_parser.py",
|
|
|
|
|
"hermes_cli/gateway_multiplex_mode.py",
|
|
|
|
|
"hermes_cli/plugin_catalog.py",
|
|
|
|
|
"hermes_cli/steward.py",
|
|
|
|
|
"hermes_cli/observability/shared_metrics_update.py",
|
|
|
|
|
"hermes_cli/main_install_repair.py",
|
|
|
|
|
"hermes_logging.py",
|
|
|
|
|
"hermes_platform/host/__init__.py",
|
|
|
|
|
"hermes_platform/host/facts.py",
|
2026-10-05 07:36:10 -07:00
|
|
|
"hermes_platform/resolver/__init__.py", # update_cmd_commit's interpreter lookup
|
|
|
|
|
"hermes_platform/resolver/base.py",
|
|
|
|
|
"hermes_platform/resolver/core.py",
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
"agent/curator.py",
|
|
|
|
|
"plugins/memory/__init__.py",
|
|
|
|
|
"tools/checkpoint_maintenance.py",
|
|
|
|
|
"tools/skills_sync.py",
|
|
|
|
|
"tools/environments/local_env_policy.py",
|
|
|
|
|
"pm/progress.py",
|
|
|
|
|
# The compilers source_build / the Desktop build run (freshness, node-deps,
|
|
|
|
|
# tui, web, desktop and their shared frontend-common).
|
|
|
|
|
"scripts/build/",
|
2026-10-19 23:26:58 -04:00
|
|
|
"apps/desktop/scripts/build-webapp.mjs", # source_build's Webapp renderer build
|
2026-10-22 11:37:21 -04:00
|
|
|
"hermes_cli/webapp.py", # source_build rebuilds the Webapp bundle a home serves
|
|
|
|
|
"gateway/webapp_listener.py", # source_build reads which homes serve the Webapp
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
)
|
|
|
|
|
# General-purpose modules an entry point imports but that half the product
|
|
|
|
|
# imports too (>= HUB_MIN_IMPORTERS product modules, checked by the test). The
|
|
|
|
|
# unit lanes cover them on every PR and the update suites on every push to main;
|
|
|
|
|
# routing each config.py / utils.py edit through the update suites would make
|
|
|
|
|
# them run on most PRs. Never an update-specific module: own those above.
|
|
|
|
|
HUB_MIN_IMPORTERS = 25
|
|
|
|
|
_SHARED_HUBS = frozenset({
|
|
|
|
|
"hermes_cli/__init__.py",
|
|
|
|
|
"hermes_cli/config.py",
|
|
|
|
|
"hermes_cli/profiles.py",
|
|
|
|
|
"hermes_cli/version_info.py",
|
|
|
|
|
"utils.py",
|
|
|
|
|
"hermes_state.py",
|
|
|
|
|
"agent/__init__.py",
|
|
|
|
|
"cron/jobs.py",
|
|
|
|
|
"tools/environments/local.py",
|
|
|
|
|
# Desktop lane only (the upgrade lane owns these outright):
|
|
|
|
|
"hermes_constants.py",
|
|
|
|
|
"gateway/status.py",
|
|
|
|
|
"pm/__init__.py",
|
|
|
|
|
"pm/paths.py",
|
|
|
|
|
"pm/environments.py",
|
|
|
|
|
})
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
_E2E_LANES: dict[str, tuple[str, ...]] = {
|
|
|
|
|
"e2e": (
|
|
|
|
|
*_PY_TEST_HARNESS,
|
|
|
|
|
"tests/e2e/",
|
|
|
|
|
# The state.db torture chamber and the compaction/exactly-once
|
|
|
|
|
# suites are the only tests that run real concurrent writers.
|
|
|
|
|
"hermes_state",
|
2026-10-08 17:09:29 -07:00
|
|
|
# Messaging-gated gateway cases: only the e2e job installs the real
|
|
|
|
|
# Discord/Telegram SDKs (tests.yml "Run messaging-gated gateway cases").
|
|
|
|
|
"tests/gateway/test_native_role_reauthorization.py",
|
|
|
|
|
"tests/gateway/test_native_telegram_startup_recovery.py",
|
|
|
|
|
"tests/gateway/test_native_pause_notice.py",
|
2026-10-09 19:43:16 -07:00
|
|
|
"tests/gateway/test_native_reply_crash_recovery.py",
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
),
|
|
|
|
|
"e2e_upgrade": (
|
|
|
|
|
*_PY_TEST_HARNESS,
|
|
|
|
|
*_DEP_MANIFESTS,
|
|
|
|
|
"tests/e2e/core/upgrade/",
|
|
|
|
|
"tests/e2e/core/windows_update/",
|
|
|
|
|
"tests/compat/",
|
|
|
|
|
"pm/",
|
|
|
|
|
"scripts/install.",
|
|
|
|
|
"setup-hermes.sh",
|
|
|
|
|
"hermes_cli/update_",
|
|
|
|
|
"hermes_cli/_update_",
|
|
|
|
|
"hermes_cli/old_updater",
|
2026-10-09 02:54:01 -07:00
|
|
|
# The downgrade refusal (hermes_cli/update_downgrade_guard.py) reads state.db through it.
|
|
|
|
|
"hermes_state_holders.py",
|
2026-10-21 11:12:29 +02:00
|
|
|
# The search-index notice (hermes_cli/update_cmd_maint.py) reads the stored FTS layout through it.
|
|
|
|
|
"hermes_state_fts.py",
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
"hermes_cli/_old_updater",
|
|
|
|
|
"hermes_cli/post_update",
|
|
|
|
|
"hermes_cli/config_migrations",
|
|
|
|
|
"hermes_cli/subcommands/update",
|
|
|
|
|
"hermes_cli/install_",
|
|
|
|
|
"hermes_cli/_install_",
|
|
|
|
|
"hermes_cli/main_install",
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
*_UPDATE_PIPELINE,
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
*_UPDATE_ENTRY_POINTS,
|
|
|
|
|
*_UPDATE_DEPENDENCIES,
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
),
|
|
|
|
|
"e2e_desktop_core": (
|
|
|
|
|
*_DESKTOP_E2E_SHARED,
|
|
|
|
|
"apps/desktop/e2e/",
|
|
|
|
|
"apps/desktop/electron/backend-",
|
|
|
|
|
"apps/shared/src/",
|
|
|
|
|
"apps/desktop/src/store/session",
|
|
|
|
|
"apps/desktop/src/store/transcript",
|
2026-10-05 17:26:36 -07:00
|
|
|
# Every core spec drives open/resume/switch; #132017 changed resume
|
|
|
|
|
# without running them and main sat red on remote-secondary.
|
|
|
|
|
"apps/desktop/src/app/session/",
|
|
|
|
|
"apps/desktop/src/app/open-session",
|
2026-10-03 14:55:27 -05:00
|
|
|
# fleet-condensed-default.spec.ts: the profile rail's doors per gateway.
|
|
|
|
|
"apps/desktop/src/app/chat/sidebar/profile-switcher",
|
|
|
|
|
"apps/desktop/src/app/chat/sidebar/fleet-",
|
2026-10-17 09:41:09 -07:00
|
|
|
# reconnect-turn-state.spec.ts: the lost-turn card and Discard live in the queue panel.
|
|
|
|
|
"apps/desktop/src/app/chat/composer/queue-",
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
),
|
|
|
|
|
"e2e_desktop_update": (
|
|
|
|
|
*_DESKTOP_E2E_SHARED,
|
|
|
|
|
"apps/desktop/e2e/update/",
|
|
|
|
|
"apps/desktop/electron/updater",
|
|
|
|
|
"apps/desktop/electron/update-",
|
|
|
|
|
"apps/desktop/electron/app-updater",
|
|
|
|
|
"apps/desktop/electron/gateway-stop-before-update",
|
|
|
|
|
"apps/desktop/electron/pre-update-",
|
|
|
|
|
"apps/desktop/electron/install-stamp",
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
# The rest of the Electron update path (codemap desktop-update §1):
|
|
|
|
|
# main.ts owns the gate / backend stop / hand-off launch hunks (the
|
|
|
|
|
# classifier sees files, not hunks), the result reader, the install
|
|
|
|
|
# kind, the attach-time version check and the in-place app swap.
|
|
|
|
|
"apps/desktop/electron/main.ts",
|
|
|
|
|
"apps/desktop/electron/handoff-result",
|
|
|
|
|
"apps/desktop/electron/desktop-installation",
|
|
|
|
|
"apps/desktop/electron/backend-discovery",
|
2026-10-07 07:33:07 -07:00
|
|
|
"apps/desktop/electron/local-gateway", # gateway ensure: code_sha vs checkout, restart on skew
|
ci: start the real-update suites for every file on the update path
The e2e_upgrade lane (Linux e2e-upgrade + the Windows install + update
journey) only matched the update_* module family, so a PR touching
cmd_update in hermes_cli/main.py, _early_recovery.py, venv_sync.py,
relaunch.py, the gateway restart/pause surface, hermes_bootstrap.py or
the Desktop hand-off scripts merged without a real `hermes update`. The
e2e_desktop_update / desktop_updater lanes likewise missed main.ts, the
hand-off result reader, the install-kind / attach modules,
desktop_update_verify.py and main_desktop.py.
The added paths are the steps of the update pipeline (updater audit
code maps, step tables) plus the lock / marker / recovery state the
update_* modules import. Two invariant tests pin every such file to its
lane and fail when one is renamed.
(cherry picked from commit b02e2cc124e74033ccb4c6e1c7c939590ef65fac)
(cherry picked from commit 16068789ea98ce55e92eeb148ec9345d6778e781)
2026-10-03 09:59:32 -07:00
|
|
|
"apps/desktop/electron/bundle-swap",
|
|
|
|
|
"apps/desktop/electron/app-installer-file",
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
"scripts/desktop-update/",
|
|
|
|
|
"scripts/install.sh",
|
|
|
|
|
"hermes_cli/update_",
|
2026-10-05 01:07:05 -07:00
|
|
|
"hermes_cli/main_desktop", # the entry point and the main_desktop_* siblings it imports
|
ci: route update owners by the import graph and dispatch every set lane
The classifier missed modules the update entry points import
(_subprocess_compat, desktop_build_lock, memory_provider_migration, the
scripts/build compilers) and never sent source_build/source_completion to
the Desktop update suite. ci.yaml gated tests-os on `python` alone, so a
TypeScript-only change to the Electron marker/result readers set
desktop_updater and still never reached the Windows hand-off tests.
- classify_changes.py: _UPDATE_ENTRY_POINTS / _DESKTOP_BUILD_ENTRY_POINTS,
an owner list of what they import, and an explicit _SHARED_HUBS set.
- tests/ci/test_update_ci_routing.py reads the entry points' real imports
(AST incl. lazy imports; ESM import/step() closure of the build scripts)
and fails until each is routed or is a real hub (>= 25 importers).
- It also replays the REAL classifier's output through the composite
action, ci.yaml detect outputs, job if:/with:, the called workflows' job
if: and the rendered Windows step, for every combination of the gated
lanes (tests/ci/_gha_expr.py models the expression subset and raises on
anything else).
- ci.yaml: tests runs for python|e2e|e2e_upgrade; tests-os for
python|desktop_updater|e2e|e2e_upgrade.
Red on 8cc78279a36: 30 of the new routing tests fail (R15 table cells,
16 lane combinations, ownership/closure tests).
(cherry picked from commit 3b17b5d2ad8081534bbeefcf5d8334cd79a7e6fd)
(cherry picked from commit 594f8b0c4ba32324da922d5289b0932299ffe7a9)
2026-10-03 22:08:20 -07:00
|
|
|
*_DESKTOP_BUILD_ENTRY_POINTS,
|
|
|
|
|
*_UPDATE_DEPENDENCIES,
|
|
|
|
|
# Pipeline modules the Desktop build entry points import directly.
|
|
|
|
|
"hermes_cli/main.py", # `hermes desktop --build-only`
|
|
|
|
|
"hermes_cli/venv_sync.py",
|
|
|
|
|
"hermes_cli/source_stamp.py",
|
|
|
|
|
# Imported by the Desktop build's scripts (scripts/build/desktop.mjs closure).
|
|
|
|
|
"apps/desktop/product-identity.cjs",
|
|
|
|
|
"scripts/msix-shared.mjs",
|
2026-10-04 07:47:03 -07:00
|
|
|
# The launchers the Desktop relaunches through reach the launch-time repair first.
|
|
|
|
|
"hermes_cli/_launchers.py",
|
2026-10-07 07:33:07 -07:00
|
|
|
# The backend's /api/health `commit`, which the Desktop compares to the checkout
|
|
|
|
|
# before attaching to a running backend after an update.
|
fix(update): CI routes each update-path file to the suites that exercise it
- hermes_cli/web_routers/status.py (review CI2): /api/health `commit` is what
host-backend-attach.ts compares to the checkout before attaching after an
update, yet an edit ran only the python lane. Now also e2e_desktop_update,
the lane host-backend-attach already starts.
- Tauri updater (review CI3): marker.rs started e2e_upgrade and
e2e_desktop_update, neither of which builds or runs the bootstrap installer
(no workflow of either lane touches apps/bootstrap-installer or cargo),
while its callers update.rs/paths.rs started none. All three now route as
one unit to the lane that compiles and tests them: rust (cargo test, via
.rs) and nothing slow. _TAURI_UPDATE_MARKER is gone.
- remote-update-marker-programs.ts (+ test) (review CI3): the SSH remote's
marker judge/gate ran only frontend; it is the marker.sh/marker.ps1
contract run remotely, and its importers (remote-lifecycle.ts,
managed-ssh-update.ts) are desktop_updater files, so it is one too.
- gateway/status (review CI3): the prefix also matched status_phrases.py
(turn-status wording); the named status_inline_source sibling does not
exist. Tightened to gateway/status.py.
Changed assertion: _R6_CASES pinned marker.rs -> (rust, e2e_upgrade,
e2e_desktop_update); the finding is that the two e2e lanes are the bug (they
never build marker.rs), so it now pins (rust,), and a new test pins the
three Tauri updater files off every slow lane.
2026-10-05 01:43:55 -07:00
|
|
|
"hermes_cli/web_routers/status.py",
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
),
|
|
|
|
|
}
|
ci: package __init__ files on the update path start the update E2Es
Importing a/b/c.py runs a/__init__.py and a/b/__init__.py first. The
update-lane routing listed hermes_cli/local_runtime/processes.py (the
v2026.9.24 updater imports it after the pull) but not the package init
that import runs. #134116 changed what that init loads, its PR never ran
e2e-upgrade, and main went red until #134242.
classify_changes now routes the package inits above every module the
e2e_upgrade and e2e_desktop_update lanes own, and above each shared hub,
because a hub's inits run whenever the hub is imported. An init that is
itself a hub stays out. The routing guard's import walker counts the
package inits a module's imports run, so the next unrouted one fails it.
This routes eight inits that were missing: local_runtime, gateway, cron,
tools, tools/environments, plugins, hermes_platform,
hermes_cli/observability.
2026-10-07 05:48:02 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
def _with_package_inits(paths: tuple[str, ...]) -> tuple[str, ...]:
|
|
|
|
|
"""Importing a/b/c.py runs a/__init__.py and a/b/__init__.py first, so a routed module
|
|
|
|
|
routes the package inits above it. A shared hub's inits run whenever the hub is imported, so
|
|
|
|
|
they are routed too; only an init that is itself a hub stays out."""
|
|
|
|
|
root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
|
|
|
|
inits = [f"{'/'.join(parts[:i])}/__init__.py"
|
|
|
|
|
for p in (*paths, *sorted(_SHARED_HUBS)) if p.endswith(".py")
|
|
|
|
|
for parts in [p.split("/")[:-1]] for i in range(1, len(parts) + 1)]
|
|
|
|
|
return tuple(dict.fromkeys([*paths, *(i for i in inits if i not in _SHARED_HUBS
|
|
|
|
|
and os.path.isfile(os.path.join(root, i)))]))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
for _lane in ("e2e_upgrade", "e2e_desktop_update"):
|
|
|
|
|
_E2E_LANES[_lane] = _with_package_inits(_E2E_LANES[_lane])
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
# The upgrade journeys are their own lane; editing one does not start ``e2e``.
|
|
|
|
|
# The update suite shares apps/desktop/e2e/ with the core suite but not its specs.
|
|
|
|
|
_E2E_LANE_EXCLUDES = {
|
|
|
|
|
"e2e": ("tests/e2e/core/upgrade/", "tests/e2e/core/windows_update/"),
|
|
|
|
|
"e2e_desktop_core": ("apps/desktop/e2e/update/",),
|
|
|
|
|
}
|
|
|
|
|
_DOCKER_PATHS = (*_DOCKER_META, *_DEP_MANIFESTS, *_NPM_MANIFESTS, "pm/", "tests/docker/")
|
|
|
|
|
_NIX_LANE_PATHS = (*_NIX_PATHS, *_NIX_FILES, *_DEP_MANIFESTS, *_NPM_MANIFESTS)
|
|
|
|
|
|
|
|
|
|
# A pull request with this label runs every slow lane, whatever it touches.
|
|
|
|
|
RUN_E2E_LABEL = "run-e2e"
|
|
|
|
|
|
2026-06-19 16:46:11 -05:00
|
|
|
def _is_docs(p: str) -> bool:
|
|
|
|
|
if p.startswith(("skills/", "optional-skills/")):
|
|
|
|
|
return False
|
|
|
|
|
return p.endswith((".md", ".mdx")) or p.startswith("docs/") or p.startswith("LICENSE")
|
|
|
|
|
|
|
|
|
|
|
2026-08-18 18:00:56 -04:00
|
|
|
def _is_nix(p: str) -> bool:
|
|
|
|
|
return p.startswith(_NIX_PATHS) or p in _NIX_FILES
|
|
|
|
|
|
|
|
|
|
|
2026-06-19 16:46:11 -05:00
|
|
|
def _py_irrelevant(p: str) -> bool:
|
2026-09-12 23:57:42 -07:00
|
|
|
if p.startswith(_PY_RELEVANT_SITE) or p in _PY_RELEVANT_CONTRACT_FILES:
|
2026-08-11 14:17:11 -04:00
|
|
|
return False
|
2026-08-18 18:00:56 -04:00
|
|
|
return (
|
|
|
|
|
_is_docs(p)
|
|
|
|
|
or p in _ROOT_NPM
|
|
|
|
|
or p.startswith(_PY_SKIP)
|
|
|
|
|
or p.startswith(_DOCKER_META)
|
|
|
|
|
or _is_nix(p)
|
|
|
|
|
)
|
2026-06-23 09:13:19 -04:00
|
|
|
|
|
|
|
|
|
2026-10-16 04:23:57 -07:00
|
|
|
def _is_catalog_data(p: str) -> bool:
|
|
|
|
|
return p.startswith(_CATALOG_DATA) and p not in _PY_RELEVANT_CONTRACT_FILES
|
|
|
|
|
|
|
|
|
|
|
2026-07-31 22:32:18 -07:00
|
|
|
def _py_test_only(p: str) -> bool:
|
|
|
|
|
"""Is ``p`` inside the test suite (never shipped / imported by the product)?
|
|
|
|
|
|
|
|
|
|
Product jobs (Desktop E2E's ``hermes serve`` backend, the Docker image)
|
|
|
|
|
run installed code — nothing under ``tests/`` is packaged or importable
|
refactor(tests): host-dispatched runner — per-file isolation on POSIX, xdist on Windows
xdist-as-standard was the wrong shape for Linux: the per-file subprocess
model ran the full suite there in 3m45s with zero cross-file failures,
while the first xdist run took 7m51s and failed 111 tests. The two
costs are structural opposites:
* xdist multiplies full-tree collection by the worker count — every
worker imports the whole suite and pays pytest's per-item
fixture-closure machinery (profiled: ~98M function calls to
collect 42k items against the conftest's autouse fixtures —
_matchfactories alone 1.37M calls, traverse_fixture_closure 783k).
On the 96-core runner that front-loads 106s before any test runs.
* The per-file model pays a spawn+import wall per file: ~15ms on
Linux (nothing), 0.5-1.5s on Windows (~6-min floor across 3400
files — the dominant cost of that lane).
So scripts/run_tests.sh now dispatches on host: run_tests_parallel.py
(restored verbatim from b8696aedea^, with its two self-tests) on POSIX,
pytest-xdist --dist loadfile on Windows. Both paths share one hermetic
env contract (env -i scrub, TZ=UTC, PYTHONHASHSEED=0, venv probing,
bytecode pre-compile). -j/HERMES_TEST_WORKERS feeds either backend.
The 111 xdist co-scheduling failures on Linux were the per-file
model's isolation guarantee surfacing as test bugs — with the POSIX
lane back on per-file, that class disappears where it never existed;
the Windows lane keeps loadfile (with its remaining co-scheduling
hazards as fix-at-the-test work).
Workflow comments, AGENTS.md, CONTRIBUTING.md, conftest comments and
the classify test follow the dual-path truth.
2026-08-31 15:38:07 -04:00
|
|
|
there. scripts/run_tests.sh and scripts/run_tests_parallel.py are deliberately
|
2026-07-31 22:32:18 -07:00
|
|
|
NOT test-only: they are runner infrastructure, and a bad edit there can
|
|
|
|
|
mask real failures, so they stay conservative (python_prod=true).
|
|
|
|
|
"""
|
|
|
|
|
return p.startswith("tests/")
|
|
|
|
|
|
|
|
|
|
|
2026-06-23 09:13:19 -04:00
|
|
|
def _is_scan(p: str) -> bool:
|
|
|
|
|
return p.endswith(_SCAN_EXTS) or p in _SCAN_FILES
|
|
|
|
|
|
|
|
|
|
|
2026-09-01 21:58:50 -07:00
|
|
|
def _is_desktop_updater(p: str) -> bool:
|
|
|
|
|
return (
|
|
|
|
|
p.startswith(_DESKTOP_UPDATER_PATHS)
|
|
|
|
|
or p.startswith(_DESKTOP_UPDATER_TEST_PREFIX)
|
|
|
|
|
or p in _DESKTOP_UPDATER_FILES
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
2026-08-20 11:40:49 -05:00
|
|
|
def _is_rust(p: str) -> bool:
|
|
|
|
|
return (
|
|
|
|
|
p.endswith(".rs")
|
|
|
|
|
or p.startswith(_RUST_PATHS)
|
|
|
|
|
or os.path.basename(p) in _RUST_FILENAMES
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
def _slow_lanes(files: list[str]) -> dict[str, bool]:
|
|
|
|
|
"""The slow lanes a pull request with this diff runs; see ``_E2E_LANES``."""
|
|
|
|
|
lanes = {
|
|
|
|
|
lane: any(
|
|
|
|
|
f.startswith(paths) and not f.startswith(_E2E_LANE_EXCLUDES.get(lane, ()))
|
|
|
|
|
for f in files
|
|
|
|
|
)
|
|
|
|
|
for lane, paths in _E2E_LANES.items()
|
|
|
|
|
}
|
|
|
|
|
lanes["docker"] = any(f.startswith(_DOCKER_PATHS) for f in files)
|
|
|
|
|
lanes["nix"] = any(f.startswith(_NIX_LANE_PATHS) for f in files)
|
|
|
|
|
return lanes
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def classify(files: list[str], run_e2e: bool = False) -> dict[str, bool]:
|
|
|
|
|
"""Map changed paths to ``{lane: should_run}``.
|
|
|
|
|
|
|
|
|
|
``run_e2e`` is the pull request's ``run-e2e`` label: it turns every slow
|
|
|
|
|
lane on.
|
|
|
|
|
"""
|
2026-06-19 16:46:11 -05:00
|
|
|
files = [f.strip() for f in files if f.strip()]
|
2026-10-16 04:23:57 -07:00
|
|
|
catalog = any(_is_catalog_data(f) for f in files)
|
|
|
|
|
code = [f for f in files if not _is_catalog_data(f)]
|
|
|
|
|
python = any(not _py_irrelevant(f) for f in code)
|
|
|
|
|
python_prod = any(not _py_irrelevant(f) and not _py_test_only(f) for f in code)
|
2026-09-05 11:07:05 -05:00
|
|
|
frontend = any(
|
|
|
|
|
f.startswith(_FRONTEND) or f in _ROOT_NPM or f in _FRONTEND_FILES
|
refactor(build): share product recipes across distributions
Build TUI, web, desktop UI and runnable agent products from explicit
prepared inputs. Keep dependency preparation separate from distribution
packaging, with PM and native builds sharing uv environment construction.
Docker copies compiled frontend products instead of build dependencies.
Nix retains uv2nix environments and consumes shared assembly through store
references. Native desktop and Termux use the same launcher and frontend
contracts. Preserve the independent PM runtime and source imports from
arbitrary working directories.
Keep failed frontend builds from replacing the previous product, reject
source/output overlap, and bound dependency-process output draining.
Include hermes_wisdom in the Nix wheel: real CLI smoke tests exposed its
missing package declaration on the base revision too.
Verified focused Python and JavaScript suites, Docker build/runtime checks,
Nix desktop and CLI/ACP checks, standalone TUI and packaged Electron PTY,
and real full-Chromium interaction. Native signed installers, Android device
installation and the full repository suite remain CI verification.
2026-09-11 13:16:55 -04:00
|
|
|
or f.startswith("tests-js/")
|
|
|
|
|
or (f.startswith("scripts/build/") and f.endswith((".mjs", ".js", ".ts")))
|
2026-09-05 11:07:05 -05:00
|
|
|
for f in files
|
|
|
|
|
)
|
2026-08-18 18:00:56 -04:00
|
|
|
deps = any(f == "pyproject.toml" for f in files)
|
|
|
|
|
npm_lock = any(f.split("/")[-1] == "package-lock.json" for f in files)
|
|
|
|
|
docker_meta = any(f.startswith(_DOCKER_META) for f in files)
|
|
|
|
|
|
2026-06-23 12:35:17 -04:00
|
|
|
ret = {
|
2026-08-18 18:00:56 -04:00
|
|
|
"python": python,
|
|
|
|
|
"python_prod": python_prod,
|
|
|
|
|
"docker_meta": docker_meta,
|
|
|
|
|
"frontend": frontend,
|
2026-06-19 16:46:11 -05:00
|
|
|
"site": any(f.startswith(_SITE) for f in files),
|
2026-06-23 09:13:19 -04:00
|
|
|
"scan": any(_is_scan(f) for f in files),
|
2026-08-18 18:00:56 -04:00
|
|
|
"deps": deps,
|
2026-08-12 13:30:01 -05:00
|
|
|
"uv_lock": any(f in ("pyproject.toml", "uv.lock") for f in files),
|
2026-08-18 18:00:56 -04:00
|
|
|
"npm_lock": npm_lock,
|
feat(pm): unified package manager — pm store foundation
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
2026-08-29 21:23:27 -04:00
|
|
|
"bootstrap": any(
|
|
|
|
|
f.startswith(_BOOTSTRAP_PATHS) or f in _BOOTSTRAP_FILES for f in files
|
|
|
|
|
),
|
2026-09-01 21:58:50 -07:00
|
|
|
"desktop_updater": any(_is_desktop_updater(f) for f in files),
|
2026-08-20 11:40:49 -05:00
|
|
|
"rust": any(_is_rust(f) for f in files),
|
ci: run the SSH and Docker terminal backends against a real sshd and daemon
The SSH backend's real-host tests skipped on every CI run (nothing set
TERMINAL_SSH_HOST), and every Docker backend test mocked subprocess or the
docker CLI, so no CI run had executed either backend end to end.
- remote-backends.yml: starts a throwaway sshd container on 127.0.0.1 with a
per-run keypair (scripts/ci/start_test_sshd.sh) and runs
test_ssh_environment.py plus the new test_docker_backend_real.py against it
and the runner's Docker daemon with a digest-pinned debian:12-slim. A file
that skips fails the job, so the lane cannot go green over zero coverage.
- New classifier lane `remote_backends` (tools/environments/, tools/terminal_tool*,
the real tests, the sshd harness, run_tests.sh). ci.yaml calls the workflow
only when it fires, plus every push to main; required_results makes a selected
lane mandatory, like the update consumers.
- The real tests read HERMES_TEST_SSH_* / HERMES_TEST_DOCKER_IMAGE (forwarded
by run_tests.sh), because the hermetic conftest blanks TERMINAL_SSH_KEY as a
credential before any test can read it.
- New real SSH cases: exit code/stderr, cwd persistence with spaces vs per-call
workdir, quoting across the ssh hop, SendEnv passthrough through a real
AcceptEnv, file tools write/read on the remote.
- New real Docker cases: exec + exit codes, cwd vs workdir, exported env
persistence, docker_env/docker_forward_env reach the container and
nothing else does, quoting, file tools write/read, /root surviving a
container replacement.
- The existing one-shot test is xfail(strict): TERMINAL_SSH_PERSISTENT=false
is read into ssh_config["persistent"] but SSHEnvironment never consumes it,
so every SSH session persists exported env. Its _cleanup also cleaned the
raw task id, not the resolved env key, so one test's env leaked into the next.
2026-10-11 05:26:48 -07:00
|
|
|
"remote_backends": any(f.startswith(_REMOTE_BACKEND_PATHS) for f in files),
|
2026-10-16 04:23:57 -07:00
|
|
|
"catalog": catalog,
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
**{lane: run_e2e or on for lane, on in _slow_lanes(files).items()},
|
2026-06-19 16:46:11 -05:00
|
|
|
}
|
2026-10-04 01:54:46 -07:00
|
|
|
consumers = [c for f in files for c in _SHARED_FIXTURE_CONSUMERS.get(f, ())]
|
|
|
|
|
if consumers:
|
|
|
|
|
inherited = classify(consumers)
|
|
|
|
|
for lane in _FIXTURE_CONSUMER_LANES:
|
|
|
|
|
ret[lane] = ret[lane] or inherited[lane]
|
2026-06-23 12:35:17 -04:00
|
|
|
if not files or any(f.startswith(".github/") for f in files):
|
|
|
|
|
ret["python"] = True
|
2026-07-31 22:32:18 -07:00
|
|
|
ret["python_prod"] = True
|
2026-06-23 12:35:17 -04:00
|
|
|
ret["docker_meta"] = True
|
|
|
|
|
ret["frontend"] = True
|
|
|
|
|
ret["site"] = True
|
|
|
|
|
ret["scan"] = True
|
|
|
|
|
ret["deps"] = True
|
2026-08-12 13:30:01 -05:00
|
|
|
ret["uv_lock"] = True
|
2026-07-15 23:18:15 -04:00
|
|
|
ret["npm_lock"] = True
|
feat(pm): unified package manager — pm store foundation
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
2026-08-29 21:23:27 -04:00
|
|
|
ret["bootstrap"] = True
|
2026-09-01 21:58:50 -07:00
|
|
|
ret["desktop_updater"] = True
|
2026-08-20 11:40:49 -05:00
|
|
|
ret["rust"] = True
|
ci: run the SSH and Docker terminal backends against a real sshd and daemon
The SSH backend's real-host tests skipped on every CI run (nothing set
TERMINAL_SSH_HOST), and every Docker backend test mocked subprocess or the
docker CLI, so no CI run had executed either backend end to end.
- remote-backends.yml: starts a throwaway sshd container on 127.0.0.1 with a
per-run keypair (scripts/ci/start_test_sshd.sh) and runs
test_ssh_environment.py plus the new test_docker_backend_real.py against it
and the runner's Docker daemon with a digest-pinned debian:12-slim. A file
that skips fails the job, so the lane cannot go green over zero coverage.
- New classifier lane `remote_backends` (tools/environments/, tools/terminal_tool*,
the real tests, the sshd harness, run_tests.sh). ci.yaml calls the workflow
only when it fires, plus every push to main; required_results makes a selected
lane mandatory, like the update consumers.
- The real tests read HERMES_TEST_SSH_* / HERMES_TEST_DOCKER_IMAGE (forwarded
by run_tests.sh), because the hermetic conftest blanks TERMINAL_SSH_KEY as a
credential before any test can read it.
- New real SSH cases: exit code/stderr, cwd persistence with spaces vs per-call
workdir, quoting across the ssh hop, SendEnv passthrough through a real
AcceptEnv, file tools write/read on the remote.
- New real Docker cases: exec + exit codes, cwd vs workdir, exported env
persistence, docker_env/docker_forward_env reach the container and
nothing else does, quoting, file tools write/read, /root surviving a
container replacement.
- The existing one-shot test is xfail(strict): TERMINAL_SSH_PERSISTENT=false
is read into ssh_config["persistent"] but SSHEnvironment never consumes it,
so every SSH session persists exported env. Its _cleanup also cleaned the
raw task id, not the resolved env key, so one test's env leaked into the next.
2026-10-11 05:26:48 -07:00
|
|
|
ret["remote_backends"] = True
|
2026-10-16 04:23:57 -07:00
|
|
|
ret["catalog"] = True
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
ret.update(dict.fromkeys(_slow_lanes([]), True))
|
2026-06-23 12:35:17 -04:00
|
|
|
return ret
|
|
|
|
|
|
2026-06-19 16:46:11 -05:00
|
|
|
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
def _event_payload() -> dict:
|
|
|
|
|
"""The Actions event payload, or ``{}`` when it is absent or unreadable."""
|
2026-08-18 23:37:18 +07:00
|
|
|
event_path = os.environ.get("GITHUB_EVENT_PATH")
|
|
|
|
|
if not event_path:
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
return {}
|
2026-08-18 23:37:18 +07:00
|
|
|
try:
|
2026-09-04 04:15:10 -04:00
|
|
|
with open(event_path, encoding="utf-8-sig") as fh:
|
2026-08-18 23:37:18 +07:00
|
|
|
payload = json.load(fh)
|
|
|
|
|
except (OSError, json.JSONDecodeError):
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
return {}
|
|
|
|
|
return payload if isinstance(payload, dict) else {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _pull_request_number() -> str | None:
|
|
|
|
|
"""Read the PR number from the Actions event payload, if present."""
|
|
|
|
|
number = (_event_payload().get("pull_request") or {}).get("number")
|
2026-08-18 23:37:18 +07:00
|
|
|
return str(number) if number else None
|
|
|
|
|
|
|
|
|
|
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
def _gh_lines(*args: str) -> list[str] | None:
|
|
|
|
|
"""Run ``gh`` and return its stdout lines, or ``None`` when it fails."""
|
|
|
|
|
try:
|
|
|
|
|
completed = subprocess.run(
|
|
|
|
|
["gh", *args],
|
|
|
|
|
check=False,
|
|
|
|
|
capture_output=True,
|
|
|
|
|
text=True,
|
|
|
|
|
timeout=30,
|
|
|
|
|
)
|
|
|
|
|
except (OSError, subprocess.TimeoutExpired):
|
|
|
|
|
return None
|
|
|
|
|
if completed.returncode != 0:
|
|
|
|
|
return None
|
|
|
|
|
return [line.strip() for line in completed.stdout.splitlines() if line.strip()]
|
|
|
|
|
|
|
|
|
|
|
2026-08-18 23:37:18 +07:00
|
|
|
def pull_request_changed_files() -> list[str]:
|
|
|
|
|
"""Recover the PR file list when the compare API returned nothing.
|
|
|
|
|
|
|
|
|
|
``detect-changes`` calls ``repos/.../compare/base...head`` with raw SHAs.
|
|
|
|
|
A fork force-push can 404 for ~30s until GitHub attaches the new head SHA
|
ci: replace the ci-reviewed label gate with CODEOWNERS
Path-based review (workflows, composite actions, eslint/prettier config,
the MCP catalog, install-hook files) moves to .github/CODEOWNERS. GitHub
requests and enforces that review, so CI no longer classifies those paths,
checks a label, or reruns failed jobs when the label is added.
Removes review-labels.yml, emit_review_status.py, the rerun-review-labels
job, and the ci_review / ci_review_files / mcp_catalog classifier outputs.
The run-e2e rerun job in label-rerun.yml stays.
The supply-chain scan keeps its content patterns (base64 decode + exec,
obfuscated subprocess), which CODEOWNERS cannot express. It fails on its
own again, as it did before #68833 routed it through the label gate. Its
.pth and install-hook path checks are dropped in favour of CODEOWNERS.
2026-10-02 15:22:49 -04:00
|
|
|
to the base repo, so the action fails open with an empty file list and
|
|
|
|
|
runs every lane even when the change is narrow.
|
2026-08-18 23:37:18 +07:00
|
|
|
|
|
|
|
|
The pull-request files endpoint already knows the PR's files (it is how
|
|
|
|
|
this action used to classify), so use it as a fallback on pull_request
|
|
|
|
|
events only. Push/dispatch keep the empty-diff fail-open.
|
|
|
|
|
"""
|
|
|
|
|
if os.environ.get("EVENT_NAME") != "pull_request":
|
|
|
|
|
return []
|
|
|
|
|
repo = os.environ.get("REPO") or os.environ.get("GITHUB_REPOSITORY") or ""
|
|
|
|
|
pr = _pull_request_number()
|
|
|
|
|
if not repo or not pr:
|
|
|
|
|
return []
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
return _gh_lines("api", "--paginate", f"repos/{repo}/pulls/{pr}/files", "--jq", ".[].filename") or []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def pull_request_labels() -> list[str]:
|
|
|
|
|
"""The pull request's labels as they are now, not as the event saw them.
|
|
|
|
|
|
|
|
|
|
A re-run replays the original event payload, so the payload alone would
|
|
|
|
|
miss a ``run-e2e`` label added after the push. The API answer wins; the
|
|
|
|
|
payload is the fallback when the API call fails.
|
|
|
|
|
"""
|
|
|
|
|
if os.environ.get("EVENT_NAME") != "pull_request":
|
2026-08-18 23:37:18 +07:00
|
|
|
return []
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
repo = os.environ.get("REPO") or os.environ.get("GITHUB_REPOSITORY") or ""
|
|
|
|
|
pr = _pull_request_number()
|
|
|
|
|
if repo and pr:
|
|
|
|
|
live = _gh_lines("api", f"repos/{repo}/pulls/{pr}", "--jq", ".labels[].name")
|
|
|
|
|
if live is not None:
|
|
|
|
|
return live
|
|
|
|
|
labels = (_event_payload().get("pull_request") or {}).get("labels") or []
|
|
|
|
|
return [label["name"] for label in labels if isinstance(label, dict) and label.get("name")]
|
2026-08-18 23:37:18 +07:00
|
|
|
|
2026-06-19 16:46:11 -05:00
|
|
|
|
|
|
|
|
def main() -> int:
|
2026-07-22 19:19:53 -04:00
|
|
|
files = sys.stdin.read().splitlines()
|
2026-08-18 23:37:18 +07:00
|
|
|
if not any(f.strip() for f in files):
|
|
|
|
|
recovered = pull_request_changed_files()
|
|
|
|
|
if recovered:
|
|
|
|
|
print(
|
|
|
|
|
f"compare API returned no files; recovered {len(recovered)} "
|
|
|
|
|
"path(s) from the pull request files endpoint",
|
|
|
|
|
file=sys.stderr,
|
|
|
|
|
)
|
|
|
|
|
files = recovered
|
ci: run the slow lanes on a PR only when it touches their paths
The E2E suites, the Windows E2E jobs, the Docker image build and the Nix
flake check took about 120 of the ~140 larger-runner minutes each PR push
spent, and fired on nearly every PR. The 32-core pool backed up behind
them.
On a pull request each slow lane now runs only when the diff touches the
suite, its harness, or the code it guards (classify_changes.py
_E2E_LANES, _DOCKER_PATHS, _NIX_LANE_PATHS), or when the PR carries the
run-e2e label. Every push to main still runs all of them, and a release
run still forces every lane on.
Adding run-e2e reruns the head commit's CI, Docker and Nix runs; detect
reads the live labels because a rerun replays the original event.
Replayed over the last 200 merged PRs, the modelled 32-core cost of one
push drops from about 120 to about 10 runner-minutes.
2026-09-30 11:43:07 -04:00
|
|
|
lanes = classify(files, run_e2e=RUN_E2E_LABEL in pull_request_labels())
|
ci: replace the ci-reviewed label gate with CODEOWNERS
Path-based review (workflows, composite actions, eslint/prettier config,
the MCP catalog, install-hook files) moves to .github/CODEOWNERS. GitHub
requests and enforces that review, so CI no longer classifies those paths,
checks a label, or reruns failed jobs when the label is added.
Removes review-labels.yml, emit_review_status.py, the rerun-review-labels
job, and the ci_review / ci_review_files / mcp_catalog classifier outputs.
The run-e2e rerun job in label-rerun.yml stays.
The supply-chain scan keeps its content patterns (base64 decode + exec,
obfuscated subprocess), which CODEOWNERS cannot express. It fails on its
own again, as it did before #68833 routed it through the label gate. Its
.pth and install-hook path checks are dropped in favour of CODEOWNERS.
2026-10-02 15:22:49 -04:00
|
|
|
out = "\n".join(f"{key}={str(value).lower()}" for key, value in lanes.items())
|
2026-06-19 16:46:11 -05:00
|
|
|
if dest := os.environ.get("GITHUB_OUTPUT"):
|
|
|
|
|
with open(dest, "a", encoding="utf-8") as fh:
|
|
|
|
|
fh.write(out + "\n")
|
|
|
|
|
print(out) # echo for local runs + CI step logs
|
|
|
|
|
return 0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
if __name__ == "__main__":
|
|
|
|
|
raise SystemExit(main())
|