## Summary A gateway, and the supervisor `raven web` leaves running beside it, held memory from startup on for things they never use or may never be asked to use. Two of them are no longer kept, and three are loaded on first use. On a key-less gateway with a warm home and no terminal, as `raven web` runs it, PSS 90 seconds after start goes from 257.2 MiB to 211.8 MiB, and the supervisor's from 77.3 MiB to 35.2 MiB. Nothing on the wire, on disk or in the page changes shape. - **The supervisor no longer imports the command table** (77.3 to 35.2 MiB). It only restarts its gateway, but it reached its loop through typer and the table. `raven/cli/entry.py` now hands the argv that `raven web` and `system.upgrade` start it with, `web --supervise --port N` with N in ASCII digits, to `serve_commands.supervise_from_argv`, which calls `_web` with the arguments the `web` command passes; a refusal ends the process with the code the command would have exited with. Any other argv is parsed by the table as before. Two process-wide settings stay ahead of the fast path in the entry point: the TLS switch, as before, and, moved there from the table's import, the switch of a Windows process's stdout and stderr to UTF-8 with `PYTHONIOENCODING` set for its children. The table's own setup, the saved language and the config migration notices, is skipped with it: the supervisor prints no translated text and never reads the config. - **Two tokenizer copies left by litellm's import are freed** (25.3 MiB). `litellm.utils` keeps the dict it parses from the bundled Anthropic tokenizer beside the string it re-serializes from it, and only the string is read: the Claude tokenizer is built from it. cl100k's registered encoding keeps a Python dict of its ranks beside the copy tiktoken's core encodes with; tiktoken reads the Python one only to pickle an encoding outside the registry, since a registered one pickles by name. `import_litellm` now drops the dict, only while the string is there, and the Python ranks. A failure is logged at debug level and leaves the import standing. - **A process whose terminal cannot be asked for its colours finds the console's host modules on its first console command** (6.2 MiB). Registering the console feature imported every CLI module to find the ones that host console commands. `cli_socket.register_cli` now also accepts a function, run once on the first `console_hosts()`. That covers a process with no terminal, as the gateway `raven web` detaches has none, and Windows, SSH, screen or tmux, and CI, which `_theme._can_probe` rules out. Where the terminal can be asked, as from `raven tui`, the host modules are still found at registration: `raven.cli._styles` asks the terminal for its colours as it is imported, and that question has to come before the Ink screen starts. - **The RPC models build each validator on first use** (6.0 MiB). `_Strict`, the base every model in `raven/rpc/models.py` derives from, sets `defer_build=True`. The reason is a comment rather than the docstring, because pydantic copies the docstring into the JSON schema. - **Loading the deck engine's plugin no longer loads python-pptx** (7.9 MiB). `plugins-dist/ppt-engine/raven_ppt/services/state.py` imported the template service, and with it python-pptx, so a gateway with the plugin installed held it even with the plugin's config slice off, which is the default. The service is now imported where a deck is read. With the slice enabled, python-pptx is still loaded by the check that decks can be made. The upgrade regression tests also use the native launcher filename (`raven.exe` on Windows, `raven` elsewhere), so the complete relaunch-argv and supervisor-handoff assertions run on both platforms. This follow-up changes test expectations only. ## Type - [ ] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [x] Other ## Verification The original author's Linux performance and suite measurements below were recorded before the Windows test-fixture follow-up and rebase. The seven original patches remain equivalent after rebasing; the Windows measurements below are from the final tree. No new memory or live-provider measurements are claimed. - **Memory:** an isolated gateway with a key-less config and `memory.backend` null, started without a terminal on an exported copy of the base commit and of this head, each on its own home warmed by an earlier start, base and head runs alternating; PSS from `/proc/self/smaps_rollup` 90 seconds after start. The base read 257.2 MiB and this head 211.8 MiB, each within 0.1 MiB from run to run. The figure on each change above is what taking that change back out of this head, alone, puts back; the four add up to the difference. The supervisor's figures are its PSS under `raven web` started on each tree. - **Behaviour, base against this head on the same trees:** - Token ids and counts over the repo's docs, the `raven/agent` sources and the message catalogs, through raven's two estimators, cl100k directly and `litellm.token_counter` for Claude, GPT, GLM, DeepSeek and Gemini model names, with decoding, copying and pickling the encoding: identical. - The console: the same host modules, found at registration on the base and on the first lookup here. On a pseudo-terminal the colour query is written during registration on both. - Every model in `raven/rpc/models.py`: validation and serialization schemas, fields, and the outcome of validating the same probe payloads, identical. On the base all but eight were built at import, here none. Sixteen threads using every model for the first time at once: no error, one set of answers. - The deck engine's plugin imports without python-pptx here; on the base it loads it. - `raven web` end to end: it starts and serves the page; after a SIGKILL of the gateway the supervisor brings it back on the same port with `RAVEN_SERVE_PORT_STRICT=1`; `raven web --stop` leaves no process and no `web.json`; `web.log` carries the same lines, and the gateway's environment is the same key by key. - A streamed turn and a tool-calling turn on glm-5.3-flash through a live config's LiteLLMProvider: the same reply, the same tool call, the same prompt usage, and the same Raven estimate. - **New tests, against the base code:** of the 33 new or changed cases, 29 fail there. Four pass there by design: the token counts, a near-miss argv that goes to the table, a process whose terminal can be asked finding the host modules at registration, and the streams left alone off Windows. All 33 pass here. - **Mutation:** each construct was removed or bent one at a time, with the backup outside the tree, and each mutant failed the tests written for it: - the supervisor: no fast path, `isdigit` as the port check, extra arguments accepted, a refusal swallowed, every refusal ending with code 1, the port ignored; - the Windows streams: no switch, the switch after the fast path, no `PYTHONIOENCODING`, the switch off Windows too, strict errors, no guard around the switch; - the tokenizer copies: none freed, the dict dropped without its string, the string dropped too, the ranks kept (in this process and in a fresh one), no guard around the freeing; - the console: the finder run on every lookup, a reset keeping a finder that never ran, lazy everywhere, eager everywhere (in this process and in a fresh one), registration importing one of the finder's modules; - the validators built at import; the template service imported with the state module. - **Directly affected and neighbouring files:** `uv run --frozen --python 3.12 --all-extras pytest -q -p no:cacheprovider --idle-ceiling-strict` over the seven changed test files and `tests/test_ppt_engine_state.py`, `tests/test_ppt_engine_prepare.py`, `tests/test_rpc_console.py`, `tests/test_cli_tui_commands.py`, `tests/test_cli_gateway_commands.py`, `tests/test_rpc_contract_shapes.py`, `tests/test_cli_smoke.py`, `tests/test_updates_upgrade.py` and `tests/test_security_tls.py`. Result at this head: 1522 passed. - **Full suite at this head:** `uv run --frozen --python 3.12 --all-extras pytest -q -p no:cacheprovider` (repo addopts: `-n 4`, integration and e2e deselected; proxy variables unset). Result: 3 failed, 31663 passed, 164 skipped, 2 xfailed. The three are this box's environment and fail the same way on an exported copy of the base commit: `tests/test_install_script.py::test_resolve_node_dir_answers_each_case_it_exists_for`, `tests/test_knowledge_parser_excel.py::test_a_legacy_workbook_is_converted_and_read` (no LibreOffice) and `tests/test_subagent_node_runtime.py::test_what_cannot_be_read_names_nothing` (run as root). - **Exported trees:** the same suite on exported copies of the base commit and of this head, compared case by case: every case they share has the same outcome, apart from two timing-sensitive cases that failed in the base run and pass in this head's run and on repeated runs of both trees: `tests/test_subagent_acp.py::test_an_interrupted_first_turn_still_binds_its_session` and `tests/test_subagent_cli_remote.py::test_a_cancel_ends_the_agent_s_whole_process_tree_on_the_machine`. - **Diff coverage:** `uv run --frozen --python 3.12 python scripts/coverage_gate.py diff --base-ref origin/main --threshold 90` over the neighbouring run's coverage. Result: 100.00%, 50 of 50 executable changed lines. The gate measures `raven/`; `tests/test_ppt_engine_*.py` executes both imports moved in the deck engine. - **Lint and gates:** `make lint-python lint-imports lint-deps lint-types check-large-files check-source-language`, and `commitlint --from origin/main --to HEAD --config commitlint.config.cjs` plus `scripts/check_commit_messages.py origin/main..HEAD`. Result: all exit 0; import contracts 10 kept, 0 broken. **Native Windows follow-up (2026-10-15):** Windows 10.0.26300, Python 3.13.12, `PYTHONUTF8=1`, isolated Raven/pytest homes and a dedicated worktree. Setup: `uv sync --frozen --python 3.13 --all-packages --extra dev`. Imports were verified against the tested worktree. The same frozen sync was repeated after the rebase picked up the memory-plugin dependency update. - Before the fix, `uv run --frozen --no-sync pytest tests/test_rpc_system.py -n 0 --tb=short` returned **3 failed, 40 passed**. The gateway-hosted handoff, standalone handoff and dev-flag regression each expected `raven` where native Windows correctly selected `raven.exe`. - After the fix, `uv run --frozen --no-sync pytest tests/test_rpc_system.py tests/test_cli_serve_commands.py::TestTheSupervisorStartsWithoutTheCommandTable tests/test_cli_system_ca.py -n 0 --idle-ceiling-strict --tb=short` returned **66 passed, 4 skipped**. - On the final rebased tree, the following command returned **631 passed, 4 skipped**, exit 0 with the idle ceiling enforced. The four skips require the Linux certificate store; none are counted as Windows passes. JUnit output paths are omitted from these commands. ```powershell uv run --frozen --no-sync pytest tests/test_rpc_system.py tests/test_cli_serve_commands.py::TestTheSupervisorStartsWithoutTheCommandTable tests/test_cli_system_ca.py tests/test_litellm_setup.py tests/test_ppt_engine_plugin.py::test_loading_the_plugin_leaves_python_pptx_for_the_first_deck tests/test_ppt_engine_state.py tests/test_rpc_cli_dispatch.py tests/test_rpc_schema_match.py -n 0 --idle-ceiling-strict --tb=short ``` - `uv run --frozen --no-sync ruff check raven evolver agents plugins-dist tests scripts docs-site` and the matching `ruff format --check`: passed (2,337 files formatted). - `uv run --frozen --no-sync lint-imports`: 10 contracts kept. `uv run --frozen --no-sync ty check raven/cli/_console_feature.py raven/cli/commands.py raven/cli/entry.py raven/cli/serve_commands.py raven/providers/litellm_setup.py raven/rpc/cli_socket.py raven/rpc/models.py plugins-dist/ppt-engine/raven_ppt/services/state.py`: passed. - `uv run --frozen --no-sync pre-commit run --from-ref origin/main --to-ref HEAD`, `node node_modules/@commitlint/cli/cli.js --from origin/main --to HEAD --config commitlint.config.cjs`, `uv run --frozen --no-sync python scripts/check_commit_messages.py origin/main..HEAD`, `uv run --frozen --no-sync python scripts/check_large_files.py origin/main...HEAD`, `uv run --frozen --no-sync python scripts/check_source_language.py origin/main...HEAD`, and `git diff --check origin/main...HEAD`: passed. The test correction retains the complete executable path, argv, credentials, delayed shutdown and supervisor-handoff assertions, without adding a skip. Upgrade spawning and the supervisor loop are substituted in these tests. No real upgrade or resident restart, live provider call, Windows memory measurement, full Windows suite or new Linux/macOS run was performed by this follow-up. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed No user-facing docs change, because no command, flag, setting or file format does. ## Risk User-visible behaviour changes: - **The first console command in a process whose terminal cannot be asked** imports the finder's modules then, once, on the event loop and under the console's dispatch lock, instead of at startup: about 0.1 to 0.2 seconds on this box. A CLI module that fails to import then fails each console command, where before it failed the process's startup. - **Each RPC model builds its validator the first time it is used**, so a method's first call pays for its own models, and a model whose fields pydantic cannot build a validator for fails there rather than when `raven.rpc.models` is imported. - **An unhandled exception in the supervisor** reaches `web.log` as Python's plain traceback rather than typer's formatted one. - **The deck engine's template service is imported at the first deck read** instead of when the plugin loads. A python-pptx that cannot be imported therefore no longer stops the plugin from loading: with its config slice enabled, the check that decks can be made withholds the deck tools, and the turn hook's note on deck state, which logs an unreadable state at debug level and goes on, meets the import error there. - **Code in the same process that reads the private `_mergeable_ranks` off the registered cl100k encoding**, as the extension example in tiktoken's README does, gets an `AttributeError` once raven has imported litellm. In raven, its plugins and the locked dependencies, only tiktoken's educational module does. Left as today: - **Any other supervisor argv**, for example one typed by hand with `--port=N` or the flags reordered, goes through the command table and keeps today's memory. - **Freeing the tokenizer copies relies on litellm's and tiktoken's internals.** The tests read each copy where it lives, check in a fresh interpreter that litellm's own import registers the encoding whose ranks are freed, and pin the token counts, so a release that moves a copy, starts reading one or stops registering the encoding fails them. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes Rollback is a revert. Nothing written to disk or sent over the wire changes. ## Related Issues N/A
186 lines
6.7 KiB
Python
186 lines
6.7 KiB
Python
"""The CLI verifies TLS against the operating system's certificate store.
|
|
|
|
A company gateway, or a proxy that inspects TLS, presents a certificate signed by
|
|
a root that IT installs in the operating system; no bundle a Python package
|
|
ships carries it. Which store a client trusts is settled before the first client
|
|
is built, so these launch raven the two ways it is launched -- the console
|
|
script, and ``python -m raven`` as the web supervisor starts its gateway -- each
|
|
in a fresh interpreter.
|
|
|
|
OpenSSL's directory store, relocated by ``SSL_CERT_DIR``, stands in for the
|
|
operating system's: it is the store Linux verifies against. The shell also names
|
|
a bundle of public roots in ``SSL_CERT_FILE``, so a client that reads those
|
|
variables itself, as httpx does, cannot pass by taking ``SSL_CERT_DIR`` on its
|
|
own.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
import certifi
|
|
import pytest
|
|
|
|
from raven.config.update_providers import set_provider_fields
|
|
from tests._tls import OpenAIModels, https_endpoint, private_ca
|
|
|
|
linux_only = pytest.mark.skipif(sys.platform != "linux", reason="the stand-in store is the one Linux verifies against")
|
|
|
|
_CONSOLE_SCRIPT = (
|
|
"import sys; from importlib.metadata import entry_points; "
|
|
"sys.argv[0] = 'raven'; sys.exit(entry_points(group='console_scripts')['raven'].load()())"
|
|
)
|
|
LAUNCHES = {
|
|
"console script": [sys.executable, "-c", _CONSOLE_SCRIPT],
|
|
"python -m raven": [sys.executable, "-P", "-m", "raven"],
|
|
}
|
|
|
|
|
|
def _isolated_env(tmp_path: Path) -> dict[str, str]:
|
|
"""The test's own environment, minus anything that would point raven elsewhere."""
|
|
env = {
|
|
key: value
|
|
for key, value in os.environ.items()
|
|
if not key.lower().endswith("_proxy") and not key.startswith(("SSL_CERT_", "RAVEN_"))
|
|
}
|
|
env.update(HOME=str(tmp_path / "user"), RAVEN_HOME=str(tmp_path / "home"), RAVEN_NO_UPDATE_CHECK="1")
|
|
return env
|
|
|
|
|
|
@pytest.fixture
|
|
def corporate(tmp_path):
|
|
"""The environment of a shell whose gateway only the operating system vouches for."""
|
|
ca = private_ca(tmp_path / "pki")
|
|
with https_endpoint(ca.server, OpenAIModels) as origin:
|
|
set_provider_fields(
|
|
"custom", {"api_key": "k", "api_base": f"{origin}/v1"}, config_path=tmp_path / "home" / "config.json"
|
|
)
|
|
yield {**_isolated_env(tmp_path), "SSL_CERT_FILE": certifi.where(), "SSL_CERT_DIR": str(ca.store)}
|
|
|
|
|
|
def _provider_test(launch: list[str], env: dict[str, str], cwd: Path) -> subprocess.CompletedProcess[str]:
|
|
return subprocess.run(
|
|
[*launch, "provider", "test", "custom", "--timeout", "5"],
|
|
env=env,
|
|
cwd=cwd,
|
|
capture_output=True,
|
|
text=True,
|
|
timeout=120,
|
|
)
|
|
|
|
|
|
@pytest.mark.slow
|
|
@linux_only
|
|
@pytest.mark.parametrize("launch", LAUNCHES.values(), ids=LAUNCHES.keys())
|
|
def test_a_root_only_the_system_store_holds_is_trusted(corporate, launch, tmp_path) -> None:
|
|
done = _provider_test(launch, corporate, tmp_path)
|
|
|
|
assert done.returncode == 0, done.stdout + done.stderr
|
|
assert "custom OK" in done.stdout
|
|
|
|
|
|
@pytest.mark.slow
|
|
@linux_only
|
|
@pytest.mark.parametrize("launch", LAUNCHES.values(), ids=LAUNCHES.keys())
|
|
def test_opting_out_leaves_the_probe_on_its_bundled_roots(corporate, launch, tmp_path) -> None:
|
|
done = _provider_test(launch, {**corporate, "RAVEN_NO_SYSTEM_CA": "1"}, tmp_path)
|
|
|
|
assert done.returncode == 1, done.stdout + done.stderr
|
|
assert "certificate_untrusted" in done.stdout
|
|
|
|
|
|
@pytest.mark.slow
|
|
def test_trust_is_settled_before_the_cli_builds_a_context(tmp_path) -> None:
|
|
"""aiohttp makes its default context when it is imported, and importing the
|
|
CLI reaches it, so a switch made after that import would leave every aiohttp
|
|
session on the context built before it. The probe reads that context back.
|
|
"""
|
|
probe = (
|
|
"import sys\n"
|
|
"sys.argv = ['raven', '--help']\n"
|
|
"from raven.cli import entry\n"
|
|
"try:\n"
|
|
" entry.run()\n"
|
|
"except SystemExit:\n"
|
|
" pass\n"
|
|
"import aiohttp.connector, truststore\n"
|
|
"print('built after the switch:', isinstance(aiohttp.connector._SSL_CONTEXT_VERIFIED, truststore.SSLContext))\n"
|
|
)
|
|
done = subprocess.run(
|
|
[sys.executable, "-P", "-c", probe],
|
|
env=_isolated_env(tmp_path),
|
|
cwd=tmp_path,
|
|
capture_output=True,
|
|
text=True,
|
|
timeout=120,
|
|
)
|
|
|
|
assert "built after the switch: True" in done.stdout, done.stdout[-2000:] + done.stderr[-2000:]
|
|
|
|
|
|
def test_the_entry_settles_trust_then_hands_the_process_to_the_cli(monkeypatch) -> None:
|
|
"""The in-process half of the entry's contract: the switch, then the CLI.
|
|
|
|
The subprocess tests above show what that order buys; this one runs the
|
|
entry in the test process, where its own lines can be measured.
|
|
"""
|
|
from raven.cli import commands, entry
|
|
from raven.security import tls
|
|
|
|
calls: list[str] = []
|
|
monkeypatch.setattr(tls, "use_system_ca", lambda: calls.append("trust"))
|
|
monkeypatch.setattr(commands, "run", lambda: calls.append("cli"))
|
|
|
|
entry.run()
|
|
|
|
assert calls == ["trust", "cli"]
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("argv", "taken"),
|
|
[
|
|
(["raven", "web", "--supervise", "--port", "18999"], ["trust", "supervisor"]),
|
|
(["raven", "web", "--supervise", "--port", "x"], ["trust", "cli"]),
|
|
],
|
|
ids=["the supervisor", "a near miss"],
|
|
)
|
|
def test_the_supervisor_skips_the_cli_but_not_the_switch(monkeypatch, argv: list[str], taken: list[str]) -> None:
|
|
"""`raven web` starts its supervisor through the entry too. That launch is taken
|
|
without the command table, after the switch like every other."""
|
|
from raven.cli import commands, entry, serve_commands
|
|
from raven.security import tls
|
|
|
|
calls: list[str] = []
|
|
monkeypatch.setattr(tls, "use_system_ca", lambda: calls.append("trust"))
|
|
monkeypatch.setattr(commands, "run", lambda: calls.append("cli"))
|
|
monkeypatch.setattr(serve_commands, "_web", lambda *_a, **_kw: calls.append("supervisor"))
|
|
monkeypatch.setattr(sys, "argv", argv)
|
|
|
|
entry.run()
|
|
|
|
assert calls == taken
|
|
|
|
|
|
def test_no_cli_module_is_a_third_way_to_start_raven() -> None:
|
|
"""The two launches above are the only ones: a CLI module run as a script skips the switch.
|
|
|
|
Pointing such a guard at ``cli.entry.run`` instead would not help either. By the
|
|
time a module's guard runs, its own imports have built aiohttp's context.
|
|
"""
|
|
import ast
|
|
|
|
import raven.cli
|
|
|
|
package = Path(raven.cli.__file__).parent
|
|
guarded = sorted(
|
|
path.relative_to(package).as_posix()
|
|
for path in package.rglob("*.py")
|
|
for node in ast.parse(path.read_text(encoding="utf-8")).body
|
|
if isinstance(node, ast.If) and ast.unparse(node.test).replace("'", '"') == '__name__ == "__main__"'
|
|
)
|
|
|
|
assert guarded == []
|