Raven-1/tests/test_routine_learner_decay.py
0xKT 1a35bc5a4e test: clear the audit debt in the test suite (#230)
## Summary

A read-only audit of all 298 test files found the suite in good shape --
exactly one genuinely fake test -- with the real debt sitting in file
names and a scattering of assertions that passed for the wrong reason.
This clears that, and fixes the bugs found while clearing it.

**Three dead tests dealt with.** Two removed because neither could ever
assert anything:

- `test_chat_e2e.py` was a module-level skip over three
`NotImplementedError` stubs. The wire-up it waited for has shipped
(`tui_commands` instantiates `SubscriptionEmitter`) and the proposal doc
it cited is gone.
- `test_tui_exit_e2e.py` asserted that building the agent loop leaves
lancedb's thread live. It does not: the loop is constructed but never
started, and the memory plugin talks to everos over HTTP, so lancedb is
not even imported. The assertion could not hold in any configuration.

The third, `slashParity.test.ts`, shelled out to python for a
`raven_cli.commands.COMMAND_REGISTRY` that no longer exists, and skipped
all three of its cases when the import failed -- three skips on every
run. The parity it wanted needs no python: `createSlashHandler.ts:45-51`
routes a slash locally when `findSlashCommand` resolves it and otherwise
hands it to `slash.exec`, a CLI subprocess a mutation cannot reach the
live session from. It now asserts that predicate directly.
`NATIVE_MUTATING_COMMANDS` and the three-way `classifyRoute` went with
it: all six names in that set are also in the local registry, so the
native branch could not change any outcome.

**19 files renamed** off phase, ticket and bug codes (AGENTS.md section
5.1), including one reversed CLI name and two integration files moved
onto a legal `<kind>`. Reference updates ride along:
`.pre-commit-config.yaml` names one of these files in a
`detect-private-key` exclude, and three docs pages point at them.

**Assertions that passed for the wrong reason, tightened**: eight
over-broad `pytest.raises(Exception)` narrowed to the type each site
actually raises; an `except` clause that swallowed the `AssertionError`
its own fakes use as a tripwire; a `hasattr` probe replaced by binding a
broker and checking it stuck; and a recall assertion that read as a pass
on an empty result set.

**Dead weight removed**: ten unused locals, three dead code blocks, six
`importorskip` guards on hard dependencies, two hardcoded `/tmp` paths,
and ticket codes left in docstrings after the renames.

**Two coverage gaps the deletions exposed are now closed**:

- `tests/test_cli_exit.py` pins the hard-exit guard, which had no tests
at all.
- The TUI chat e2e gains a multi-turn case. It reads the persisted
session rather than the screen, because a prompt is echoed into the
transcript the moment it is typed -- screen-scraping for a planted word
passes even when history is broken.

**Three timing bugs fixed in the TUI e2e tests.** They keyed off the
banner, which paints about seven seconds before the app accepts input,
and submitted the moment text was typed. Dropped prompts made the
pipeline look dead in one test, and made a negative assertion (`assert
not leak_detected`) pass vacuously in another -- no prompt means no
streaming means no leak.

**Gaps in the two features that landed during review**, small enough to
close here: the `session.py` wiring that fills `update_available` /
`update_command` (removing it left the suite green),
`_upgrade_command_works` (a fixture stubbed it to `True` everywhere), an
`episodeSummary` case with an empty test body, and `clipToWidth` /
`hasMeaningfulReasoning`, which had no coverage at all. The larger items
from that audit are tracked in #229.

**Upstream review attributions dropped from ui-tui comments.** 36
mentions of another project's review thread (#19835 as "Copilot round-N
review"), plus #18994, #19194 and #14045 -- none resolve to anything
here, and AGENTS.md section 1.1 rules out comments pointing at
information only visible elsewhere. The rationale each carried is kept;
four `describe()` titles lose a trailing `(#18994)`. Three went further
and documented a python counterpart that does not exist: `platform.ts`
credited key-spelling normalization to `raven_cli/voice.py` (no such
module -- `raven/tui_rpc/methods/_stubs.py` answers `voice.toggle` with
"voice not supported in Raven v0.1"), and `useConfigSync.ts` plus its
test credited an `interrupt` default to `raven_cli/config.py` and
`tui_gateway/server.py::_load_busy_input_mode` (`busy_input_mode`
appears nowhere in python; neither module exists).

**One flaky test fixed, caught by this PR's own CI run.**
`test_stdio_no_executor_does_not_raise` set `command="true"` and let
`connect_mcp_servers` really spawn it, so the result depended on how a
live child process raced the MCP handshake -- it failed once on CI in
the full-suite run while passing 30 out of 30 standalone runs locally.
It also asserted only that whatever came back was not a
`SandboxInitError`. The transport now raises instead of running (the
idiom its sibling test already uses) and the test asserts the stdio
branch was reached.

Review follow-ups, folded in: the hard-exit test now skips rather than
asserting no lancedb thread exists process-wide, `tests/conftest.py` no
longer credits the pytest segfault to a lancedb thread the suite never
starts (raven imports lancedb nowhere), and the status-bar patterns the
two TUI e2e tests wait on live in one module instead of a hand-mirrored
copy per file.

Every new or changed assertion was checked by mutating the code under
test and confirming the assertion fails.

## Type

- [x] Others

Test files, test configuration, and comments. The only non-test files
touched are ui-tui comments and four `describe()` titles -- no behaviour
change.

## Verification

```
uv run --all-extras pytest -q
# 4654 passed, 25 deselected

cd ui-tui && npx vitest run
# 936 passed, no skips

uv run --all-extras pytest tests/tui/autotest -m e2e -q
# passes locally with tui-use installed and ui-tui built

bash .claude/scripts/preflight_ci.sh
# ALL PREFLIGHT CHECKS PASSED (including the TUI lane)
```

Mutation checks, each confirming the new assertion fails when the code
under test is broken: the `session.py` nudge wiring removed;
`clipToWidth` replaced with identity; `hasMeaningfulReasoning` forced to
`true`; a never-sent prompt added to the multi-turn session assertion.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally

## Risk

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

No behaviour changes: the non-test diff is comments only. The renames
touch git history for 19 test files; #208 is the only open PR affected
and has been notified.

Two known items are deliberately left alone: `test_dogfood_whitelist.py`
is flaky when the whole file runs (tracked in #228, unchanged here), and
the `lancedb_finalization_hazard` gate in `commands.py` never fires
today -- its test docstring says so explicitly, so the coverage does not
read as an argument for keeping it.

## Related Issues

Closes #223

---------

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
2026-07-29 21:26:55 +08:00

198 lines
6.9 KiB
Python

"""Unit tests for RoutineLearner.learn_with_decay (MS2.5)."""
from __future__ import annotations
from datetime import datetime, timedelta
from raven.proactive_engine.sentinel.predictor.routine_learner import (
RoutineLearner,
_decay_factor,
)
_NOW = datetime(2026, 5, 8, 12, 0)
def _history_line(ts: datetime, content: str) -> str:
return f"[{ts.strftime('%Y-%m-%d %H:%M')}] {content}"
def _history_md(*lines: str) -> str:
"""Join HISTORY.md-style entries with the same blank-line separator
MemoryStore.append_history uses."""
return "\n\n".join(lines)
def test_decay_factor_today_is_one():
assert _decay_factor(now=_NOW, ts=_NOW, half_life_days=14) == 1.0
def test_decay_factor_one_half_life_is_half():
ts = _NOW - timedelta(days=14)
factor = _decay_factor(now=_NOW, ts=ts, half_life_days=14)
assert abs(factor - 0.5) < 1e-9
def test_decay_factor_two_half_lives_is_quarter():
ts = _NOW - timedelta(days=28)
factor = _decay_factor(now=_NOW, ts=ts, half_life_days=14)
assert abs(factor - 0.25) < 1e-9
def test_decay_factor_future_clamps_to_one():
"""Defensive: parsing artifact shouldn't make weights blow up."""
ts = _NOW + timedelta(days=5)
assert _decay_factor(now=_NOW, ts=ts, half_life_days=14) == 1.0
def test_decay_factor_zero_half_life_is_one():
"""Defensive against divide-by-zero / config typo."""
ts = _NOW - timedelta(days=10)
assert _decay_factor(now=_NOW, ts=ts, half_life_days=0) == 1.0
def test_learn_with_decay_recent_routine_outweighs_stale_one():
"""Two routines with same occurrence_count: recent fresh > stale old."""
learner = RoutineLearner(
min_occurrences=3,
learning_window_days=60,
now_fn=lambda: _NOW,
)
# Build entries for two clearly-distinct bins:
fresh_entries = []
stale_entries = []
# 4 Tuesdays at 9am, recent (within last 28 days)
for week in range(4):
d = _NOW - timedelta(days=_NOW.weekday() + 7 * week - 1) # Tuesday
d = d.replace(hour=9, minute=0)
fresh_entries.append(_history_line(d, f"meeting standup week {week}"))
# 4 Fridays at 14:00, OLD (35-56 days ago)
for week in range(4):
d = _NOW - timedelta(days=35 + 7 * week)
d = d.replace(hour=14, minute=0)
stale_entries.append(_history_line(d, f"old report week {week}"))
md = _history_md(*(fresh_entries + stale_entries))
routines = learner.learn_with_decay(md)
# Both bins should produce a candidate
assert len(routines) >= 2
# Sort guarantees fresh-and-frequent first
fresh_r = next(r for r in routines if r.day_of_week == 1) # Tuesday
stale_r = next(r for r in routines if r.day_of_week == 4) # Friday
assert fresh_r.weight > stale_r.weight
# Returned order has fresh first
fresh_idx = routines.index(fresh_r)
stale_idx = routines.index(stale_r)
assert fresh_idx < stale_idx
def test_learn_with_decay_skips_below_min_occurrences():
learner = RoutineLearner(
min_occurrences=3,
learning_window_days=60,
now_fn=lambda: _NOW,
)
# Only 2 occurrences in same bin — below threshold
md = _history_md(
_history_line(_NOW.replace(hour=9), "meeting"),
_history_line(_NOW - timedelta(days=7, hours=3), "meeting"),
)
routines = learner.learn_with_decay(md)
assert routines == []
def test_learn_with_decay_handles_empty_history():
learner = RoutineLearner(now_fn=lambda: _NOW)
assert learner.learn_with_decay("") == []
assert learner.learn_with_decay("no parseable lines here") == []
def test_learn_with_decay_outside_learning_window_drops_entries():
learner = RoutineLearner(
min_occurrences=3,
learning_window_days=14, # short window
now_fn=lambda: _NOW,
)
# 4 entries 30 days ago — outside window
old = [_history_line(_NOW - timedelta(days=30 + i), f"old activity {i}") for i in range(4)]
routines = learner.learn_with_decay(_history_md(*old))
assert routines == []
def test_tfidf_keywords_downweights_common_terms():
"""Words appearing in many bins (high DF) get TF-IDF score 0; bin-
specific terms surface."""
learner = RoutineLearner(
min_occurrences=3,
learning_window_days=60,
now_fn=lambda: _NOW,
)
# Three different bins, each with 'work' and one unique term.
# Bin 1 (Mon 9): work + standup
# Bin 2 (Wed 14): work + review
# Bin 3 (Fri 16): work + planning
entries = []
for week in range(4):
# Monday 9am — standup
mon = _NOW - timedelta(days=_NOW.weekday() + 7 * week)
mon = mon.replace(hour=9, minute=0)
entries.append(_history_line(mon, "work standup meeting"))
# Wednesday 2pm — review
wed = _NOW - timedelta(days=(_NOW.weekday() - 2) % 7 + 7 * week)
wed = wed.replace(hour=14, minute=0)
entries.append(_history_line(wed, "work review session"))
# Friday 4pm — planning
fri = _NOW - timedelta(days=(_NOW.weekday() - 4) % 7 + 7 * week)
fri = fri.replace(hour=16, minute=0)
entries.append(_history_line(fri, "work planning sync"))
routines = learner.learn_with_decay(_history_md(*entries))
# Each bin's keywords should NOT be dominated by 'work' (which appears
# in all 3 bins → DF=3, log(3/3)=0 score). It should be bin-specific.
by_dow = {r.day_of_week: r for r in routines}
# 'work' should be dropped or at least not first in any bin's keywords
for r in by_dow.values():
if r.keywords:
assert r.keywords[0] != "work", f"TF-IDF should downweight common 'work' term, got {r.keywords}"
def test_learn_with_decay_sets_weight_field():
learner = RoutineLearner(
min_occurrences=3,
learning_window_days=60,
now_fn=lambda: _NOW,
)
# 3 fresh Tuesday 9am entries
entries = []
for week in range(3):
d = _NOW - timedelta(days=_NOW.weekday() + 7 * week - 1)
d = d.replace(hour=9, minute=0)
entries.append(_history_line(d, "morning meeting"))
routines = learner.learn_with_decay(_history_md(*entries))
assert len(routines) == 1
r = routines[0]
assert r.weight > 0
# 3 entries within ~21 days, half-life 14: roughly 1 + 0.5 + 0.25 = ~1.75
# (loose bound — just check it's plausibly in range)
assert 0.5 < r.weight < 3.0
def test_legacy_learn_still_works():
"""`learn` (without decay) shouldn't have regressed — existing
ContextAssembler paths still call it."""
learner = RoutineLearner(
min_occurrences=3,
learning_window_days=60,
now_fn=lambda: _NOW,
)
entries = []
for week in range(3):
d = _NOW - timedelta(days=_NOW.weekday() + 7 * week - 1)
d = d.replace(hour=9, minute=0)
entries.append(_history_line(d, "morning meeting"))
routines = learner.learn(_history_md(*entries))
assert len(routines) == 1
# weight defaults to 0.0 in legacy path (only set by learn_with_decay)
assert routines[0].weight == 0.0