Handsome-wzw 7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00

263 lines
12 KiB
Python

"""activation_spec — a node's machine-checkable "when do I take effect".
Kinds (one per mechanism class, design section 3):
- ``trajectory_regex`` code class. Counts trajectories containing a line
matching ``pattern`` within ``scope`` records.
(Delta-3's "pure cd" predicate is this kind.)
- ``consecutive_repeat`` hook class, repetition-trigger family. Counts
trajectories with >= ``threshold`` consecutive
identical ``scope`` contents. Empty/whitespace
contents are skipped by default (``ignore_empty: false``
to count them). Other hook trigger families express as
trajectory_regex or get a new kind + evaluator here.
- ``short_content_run`` hook class, reasoning-visibility family. Counts
trajectories with >= ``threshold`` consecutive
tool-call iterations whose visible assistant content
is shorter than ``max_chars`` (default 80). Uses the
shared predicate ``is_short_toolcall_iteration`` from
evolver.activation.predicates, the same function the
ReasoningVisibilityHook calls at runtime: only records
carrying tool_calls count; a record WITHOUT tool_calls
(or a long-content one) resets the run. Content is
measured after stripping ``<think>...</think>`` blocks
and collapsing whitespace. Approximation: the hook
reads the live ``response.content``; over recorded
sessions we read the ``content`` field of the assistant
record (Qwen's chain-of-thought lands in a separate
``reasoning_content`` field the hook does not read).
- ``empty_run`` hook class, response-quality family. Counts
trajectories with >= ``threshold`` consecutive empty
iterations. Uses the shared predicate
``is_empty_response`` (evolver.activation.predicates),
the same the EmptyRunBreakerHook calls: an iteration is
empty iff its content is blank AND it carries no
tool_calls. A blank-content record that still issues a
tool call is NOT empty (round-1 incident C1: the old
content-only predicate predicted 64.8% reachable while
the tool_calls-respecting hook fired 0). Any non-empty
iteration resets the run.
- ``repeated_failure_run`` hook class, robustness family. Counts trajectories
with >= ``threshold`` consecutive failures of the same
command (same head token). Walks raw records: assistant
content sets a pending head token (first whitespace-split
token); tool record resolves it (failed iff exit code
is nonzero); same head + fail grows the run; success
or head-change resets it. Tool records without an
"Exit code:" line count as success (conservative:
under-counts reachability, never over-counts).
``scope`` is ignored (the kind inherently walks
assistant and tool records).
- ``min_iterations`` hook class, budget family. Counts trajectories with
>= ``threshold`` assistant iterations — the wrap-up
nudge trigger is an iteration-count crossing,
structurally drift-free (no semantic predicate).
- ``skill_routing`` skill class. Reachability is answered by a routing
dry-query, not by corpus replay — the chamber
delegates; evaluate_spec rejects it.
- ``presence`` always-on class. Answered by offline render/config
assert in the preflight CLI; evaluate_spec rejects
it likewise.
``evaluate_spec(spec, corpus) -> int`` returns HOW MANY trajectories the
spec is reachable in. 0 = the mechanism would never run = block.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from evolver.activation.predicates import (
command_head,
is_empty_response,
is_short_toolcall_iteration,
)
_CORPUS_KINDS = {
"trajectory_regex",
"consecutive_repeat",
"short_content_run",
"empty_run",
"repeated_failure_run",
"min_iterations",
}
_KNOWN_KINDS = _CORPUS_KINDS | {"skill_routing", "presence"}
_DEFAULT_MAX_CHARS = 80
def _normalize_record(r: dict) -> dict:
"""Coerce a logged session record into the predicate record shape
(content as str, tool_calls as list) so the shared predicate functions
see the same view the hooks build via normalize_response()."""
return {"content": str(r.get("content") or ""), "tool_calls": r.get("tool_calls") or []}
@dataclass
class ActivationSpec:
kind: str
raw: dict = field(default_factory=dict)
@classmethod
def from_dict(cls, d: dict) -> "ActivationSpec":
kind = d.get("kind")
if not kind:
raise ValueError("activation_spec requires a 'kind' field")
if kind not in _KNOWN_KINDS:
raise ValueError(f"unknown activation_spec kind: {kind!r}")
if kind == "trajectory_regex" and "pattern" not in d:
raise ValueError("trajectory_regex requires 'pattern'")
if kind == "trajectory_regex":
try:
re.compile(d["pattern"])
except re.error as exc:
raise ValueError(f"trajectory_regex pattern is not valid regex: {exc}") from exc
if kind == "consecutive_repeat" and "threshold" not in d:
raise ValueError("consecutive_repeat requires 'threshold'")
if kind == "consecutive_repeat":
try:
int(d["threshold"])
except (TypeError, ValueError) as exc:
raise ValueError(f"consecutive_repeat threshold must be an int: {d['threshold']!r}") from exc
if kind == "short_content_run" and "threshold" not in d:
raise ValueError("short_content_run requires 'threshold'")
if kind == "short_content_run":
try:
int(d["threshold"])
except (TypeError, ValueError) as exc:
raise ValueError(f"short_content_run threshold must be an int: {d['threshold']!r}") from exc
if "max_chars" in d:
try:
int(d["max_chars"])
except (TypeError, ValueError) as exc:
raise ValueError(f"short_content_run max_chars must be an int: {d['max_chars']!r}") from exc
if kind == "empty_run" and "threshold" not in d:
raise ValueError("empty_run requires 'threshold'")
if kind == "empty_run":
try:
int(d["threshold"])
except (TypeError, ValueError) as exc:
raise ValueError(f"empty_run threshold must be an int: {d['threshold']!r}") from exc
if kind == "repeated_failure_run" and "threshold" not in d:
raise ValueError("repeated_failure_run requires 'threshold'")
if kind == "repeated_failure_run":
try:
int(d["threshold"])
except (TypeError, ValueError) as exc:
raise ValueError(f"repeated_failure_run threshold must be an int: {d['threshold']!r}") from exc
if kind == "min_iterations" and "threshold" not in d:
raise ValueError("min_iterations requires 'threshold'")
if kind == "min_iterations":
try:
int(d["threshold"])
except (TypeError, ValueError) as exc:
raise ValueError(f"min_iterations threshold must be an int: {d['threshold']!r}") from exc
if kind == "skill_routing" and "skill_name" not in d:
raise ValueError("skill_routing requires 'skill_name'")
if kind == "presence" and "needle" not in d:
raise ValueError("presence requires 'needle'")
raw = {k: v for k, v in d.items() if k != "kind"}
return cls(kind=kind, raw=raw)
@property
def pattern(self) -> str:
return self.raw["pattern"]
@property
def threshold(self) -> int:
return int(self.raw["threshold"])
@property
def max_chars(self) -> int:
return int(self.raw.get("max_chars", _DEFAULT_MAX_CHARS))
def _scope_contents(traj: list[dict], scope: str) -> list[str]:
return [str(r.get("content") or "") for r in traj if scope in ("any", r.get("role"))]
def evaluate_spec(spec: ActivationSpec, corpus: list[list[dict]]) -> int:
if spec.kind not in _CORPUS_KINDS:
raise ValueError(f"{spec.kind} is not corpus-evaluable; use the preflight CLI path")
scope = spec.raw.get("scope", "assistant")
hits = 0
if spec.kind == "trajectory_regex":
pat = re.compile(spec.pattern, re.M)
for traj in corpus:
if any(pat.search(c) for c in _scope_contents(traj, scope)):
hits += 1
elif spec.kind == "consecutive_repeat":
threshold = spec.threshold
ignore_empty = spec.raw.get("ignore_empty", True)
for traj in corpus:
run, prev = 0, object()
for c in _scope_contents(traj, scope):
if ignore_empty and not c.strip():
# Empty responses carry no command; a repetition
# trigger comparing commands never sees them.
continue
run = run + 1 if c == prev else 1
prev = c
if run >= threshold:
hits += 1
break
elif spec.kind == "short_content_run":
threshold = spec.threshold
max_chars = spec.max_chars
for traj in corpus:
run = 0
fired = False
for r in traj:
if scope not in ("any", r.get("role")):
continue
rec = _normalize_record(r)
run = run + 1 if is_short_toolcall_iteration(rec, max_chars) else 0
if run >= threshold:
fired = True
break
if fired:
hits += 1
elif spec.kind == "empty_run":
threshold = spec.threshold
for traj in corpus:
run = 0
for r in traj:
if scope not in ("any", r.get("role")):
continue
rec = _normalize_record(r)
run = run + 1 if is_empty_response(rec) else 0
if run >= threshold:
hits += 1
break
elif spec.kind == "repeated_failure_run":
threshold = spec.threshold
exit_re = re.compile(r"Exit code: (\d+)")
for traj in corpus:
run, prev_head = 0, None
pending_head = None
for r in traj:
role, c = r.get("role"), str(r.get("content") or "")
if role == "assistant" and c.strip():
pending_head = command_head(_normalize_record(r))
elif role == "tool" and pending_head is not None:
m = exit_re.search(c)
failed = bool(m and m.group(1) != "0")
if failed and pending_head == (prev_head or pending_head):
run += 1
prev_head = pending_head
else:
run = 1 if failed else 0
prev_head = pending_head if failed else None
pending_head = None
if run >= threshold:
hits += 1
break
elif spec.kind == "min_iterations":
threshold = spec.threshold
for traj in corpus:
count = sum(1 for r in traj if r.get("role") == "assistant")
if count >= threshold:
hits += 1
return hits