Raven-1/tests/test_security_tls.py
Dizhan Xue 39deee89ef
fix(*): stop uv's deprecation warning on upgrades and pin the store hand-over (#896)
## Summary

System-store upgrades passed both certificate-variable names to uv,
causing a deprecation warning on releases that warn even when the
replacement variable is present. Ask the uv used for installation for
its version and add `UV_NATIVE_TLS` only for releases before 0.11 or
when the version cannot be read. Explicit user settings remain intact.

- Keep certificate-store handoff in a fresh private directory, remove it
from the helper's import path after loading, and test both properties.
- Pin ordinary sub-agent connection failures as `network_error`,
distinct from certificate refusals.
- Let `security.tls.system_ca_in_use()` own the query of the actual
injected SSL factory. Upgrade and integration fixtures use the public
switch rather than duplicating its implementation.
- Cover a timed-out `uv --version` in the real helper orchestration. The
existing 30-second timeout falls back to the legacy certificate
variables so installation and relaunch can continue. Removing the
`SubprocessError` catch now fails this test.

This completes review follow-ups to the system-store change. The timeout
follow-up adds coverage without changing the existing fallback or its
duration.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

Linux, Python 3.12, after rebasing onto current main:

```text
uv run --frozen --all-extras pytest tests/test_security_tls.py tests/test_updates_upgrade.py tests/test_updates_install_guard.py tests/test_cli_subagent_setup.py tests/test_cli_system_ca.py tests/test_cli_onboard_commands.py tests/test_cli_provider_commands.py tests/test_config_update_providers.py -q -x --tb=short
uv run --frozen --all-extras pytest tests/integration/test_cli_upgrade_real_uv.py -o addopts= -n 0 -q --tb=short
```

Results: 894 passed and 3 passed, respectively. The real upgrades cover
library roots and handed-over system roots. Two existing idle-ceiling
warnings were reported by the integration tests.

Native Windows, Python 3.13:

```text
uv run --frozen --extra dev pytest tests/test_updates_upgrade.py -k older_than_system_certs -n 0 -q -x --tb=short
```

Result: 6 passed, including `subprocess.TimeoutExpired`. An isolated
mutation removing `subprocess.SubprocessError` from the version probe's
catch failed exactly the timeout case; the production source was then
restored and checked against Git.

Additional checks passed:

```text
uv run --frozen --extra dev pre-commit run --from-ref origin/main --to-ref HEAD
uv run --frozen --all-extras ty check raven/security/tls.py raven/updates/upgrade.py
uv run --frozen --all-extras lint-imports
uv run --frozen python scripts/check_commit_messages.py origin/main..HEAD
commitlint --from origin/main --to HEAD
uv run --frozen python scripts/check_source_language.py origin/main..HEAD
uv run --frozen python scripts/check_large_files.py origin/main..HEAD
git diff --check origin/main...HEAD
```

Import-linter kept all 10 contracts. The full repository suite and
native macOS were not rerun for this follow-up. No user-facing
documentation change is needed for the additional timeout test. Existing
integration fixtures can use the user's upgrade cache, and some helper
tests reach GitHub before exercising a fallback; these predate this
follow-up.

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

An older or unrecognized uv still receives both certificate-variable
names, preserving compatibility behind a TLS-inspecting proxy. An
explicit legacy setting may still produce uv's own warning. The version
query is bounded, and its failure does not prevent an upgrade attempt.
No certificate verification is disabled.

Rollback: revert the squash commit, restoring the previous warning and
unpinned handoff behavior.

## Related Issues

#878, #637

---------

Co-authored-by: Claude (claude-opus-5-5[1m]) <noreply@anthropic.com>
2026-10-12 16:08:26 +08:00

151 lines
4.8 KiB
Python

"""Which certificate store the process verifies TLS against, and telling a refused certificate apart."""
from __future__ import annotations
import http.server
import socket
import ssl
import sys
import threading
import httpx
import pytest
import truststore
from raven.security.tls import is_untrusted_certificate, system_ca_in_use, use_system_ca
from tests._tls import OpenAIModels, https_endpoint, private_ca
@pytest.fixture
def process_ssl():
"""Hand ``ssl`` back as it was: what these change belongs to the whole process."""
yield
truststore.extract_from_ssl()
def test_contexts_built_afterwards_verify_with_the_operating_system(monkeypatch, process_ssl) -> None:
monkeypatch.delenv("RAVEN_NO_SYSTEM_CA", raising=False)
use_system_ca()
assert isinstance(ssl.create_default_context(), truststore.SSLContext)
@pytest.mark.parametrize("value", ["1", "true", "YES", " on "])
def test_opting_out_leaves_ssl_as_it_was(monkeypatch, process_ssl, value: str) -> None:
monkeypatch.setenv("RAVEN_NO_SYSTEM_CA", value)
use_system_ca()
assert not isinstance(ssl.create_default_context(), truststore.SSLContext)
@pytest.mark.parametrize("value", ["", "0", "false", "off"])
def test_a_value_that_does_not_say_yes_is_not_an_opt_out(monkeypatch, process_ssl, value: str) -> None:
monkeypatch.setenv("RAVEN_NO_SYSTEM_CA", value)
use_system_ca()
assert isinstance(ssl.create_default_context(), truststore.SSLContext)
def test_a_process_says_whether_it_switched(monkeypatch, process_ssl) -> None:
monkeypatch.delenv("RAVEN_NO_SYSTEM_CA", raising=False)
truststore.extract_from_ssl()
assert not system_ca_in_use()
use_system_ca()
assert system_ca_in_use()
def test_a_process_that_opted_out_has_not_switched(monkeypatch, process_ssl) -> None:
monkeypatch.setenv("RAVEN_NO_SYSTEM_CA", "1")
truststore.extract_from_ssl()
use_system_ca()
assert not system_ca_in_use()
def test_a_process_without_truststore_is_answered_without_an_import(monkeypatch) -> None:
"""The upgrade asks from processes started without `cli.entry`, some of which have no
truststore to import: the real-uv upgrade test's tool environment is one."""
monkeypatch.setitem(sys.modules, "truststore", None)
assert not system_ca_in_use()
def _failure(url: str) -> httpx.HTTPError:
with httpx.Client(timeout=5, trust_env=False) as client:
try:
client.get(url)
except httpx.HTTPError as exc:
return exc
raise AssertionError(f"{url} answered")
def test_a_refused_certificate_is_told_apart_from_the_other_ways_a_connection_fails(tmp_path) -> None:
"""Each failure here is a real one, raised by a real connection attempt."""
ca = private_ca(tmp_path / "pki")
with https_endpoint(ca.server, OpenAIModels) as origin:
refused_certificate = _failure(f"{origin}/v1/models")
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
closed = sock.getsockname()[1]
refused_connection = _failure(f"https://127.0.0.1:{closed}/v1/models")
plain = http.server.ThreadingHTTPServer(("127.0.0.1", 0), OpenAIModels)
threading.Thread(target=plain.serve_forever, daemon=True).start()
try:
not_tls = _failure(f"https://127.0.0.1:{plain.server_address[1]}/v1/models")
finally:
plain.shutdown()
plain.server_close()
assert is_untrusted_certificate(refused_certificate)
assert not is_untrusted_certificate(refused_connection)
assert not is_untrusted_certificate(not_tls)
def _raised_through_httpx(cause: ssl.SSLError) -> httpx.ConnectError:
try:
try:
raise cause
except ssl.SSLError as exc:
raise httpx.ConnectError(str(exc)) from exc
except httpx.ConnectError as wrapped:
return wrapped
@pytest.mark.parametrize(
("verify_code", "message"),
[
(-67901, '"127.0.0.1" certificate is not standards compliant'),
(
0x800B0109,
"A certificate chain processed, but terminated in a root certificate "
"which is not trusted by the trust provider.",
),
],
ids=["macos", "windows"],
)
def test_a_refusal_in_the_system_verifiers_own_words_is_still_a_refused_certificate(
verify_code: int, message: str
) -> None:
"""Once truststore verifies, the refusal carries the operating system's wording.
These are the shapes truststore raises on macOS and Windows: an
``SSLCertVerificationError`` that never says CERTIFICATE_VERIFY_FAILED. A
classifier reading the message would call every one of them a network error.
"""
refusal = ssl.SSLCertVerificationError(message)
refusal.verify_message = message
refusal.verify_code = verify_code
failure = _raised_through_httpx(refusal)
assert "CERTIFICATE_VERIFY_FAILED" not in str(failure)
assert is_untrusted_certificate(failure)