Raven-1/tests/test_ui_css_token_gate.py
Handsome-wzw 7b45527b5b fix(tools): an array sent as its own JSON is still an array
## Summary

Reported as "deliver_files is broken". It is not: **the tool never ran.**

Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:

```
Error: Invalid parameters for tool 'deliver_files': files should be array
```

What the model sent was the array, spelled as a string:

```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```

`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.

## Type

- [x] Fix
- [ ] Feature

## The edge of the forgiving is the design

When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.

**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.

**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.

## Not just deliver_files

Six built-in tools declare an array or object parameter and were all exposed:

| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |

The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.

## Verification

```
make lint-python    ruff check + ruff format --check, 1850 files
make lint-imports   9 contracts kept, 0 broken
pytest tests/       19,699 passed, 7 failed (all pre-existing, see below)
```

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string

**End to end, through the real tool**, with the exact payload that failed five times in production:

```
files sent as:            str -> [{"path": ".../poster.png", "title": "..."}]
returned:                 Delivered 1 file(s): poster.png (72 B).
still "should be array":  False
registered in the store:  True
```

Twelve cases, five mutations, each killing at least one:

| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |

The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.

**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.

## Risk

- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert

## Related Issues

N/A

## Swept

`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.

- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.

Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).

## Deliberately not fixed

The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.

## Moved from !594

Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.

Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
2026-09-09 15:42:36 +00:00

153 lines
7.4 KiB
Python

"""The CSS token gate rejects what it claims to reject.
``ui-web/scripts/check-css.mjs`` is the only thing keeping "which color is this
role" answerable in one place per theme: it fails CI when a color is written
as a literal instead of a token. A gate is only worth its line in the
pipeline if it actually catches the thing it names, and two earlier revisions
of this one passed rules it was written to reject -- one because the check
was positional (a colored rule above the reset was exempt, the identical rule
at the end of the file was not), one because the shadow exemption was shaped
by property rather than by what the literal is (``box-shadow: 0 0 0 2px
#b4402f`` is a role color frozen to one theme).
So each case is pinned here: the stylesheet is copied, one rule is appended,
and the gate is run over the copy. Both directions matter -- a gate that
rejects everything is as useless as one that rejects nothing.
"""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
import pytest
# The gate is a node script, and this suite is the only place that checks what
# it REJECTS: running it over the current stylesheet, which is all the GitHub
# workflow's page job does, proves the gate runs and that the file passes, and
# stays green against a gate that allowlists everything.
#
# So skipping is a real loss, not a formality, and it is here only because the
# pipeline that gates a merge request cannot run it: GitLab's `tests` job (the
# whole of `.gitlab-ci.yml`, which lives on this project's `ci` branch) uses
# `python:3.12-slim` and installs git and nothing else. That branch is not
# reachable from this one. The GitHub workflow's unit job now installs node, so
# this suite runs there; closing the GitLab gap is a change to the `ci` branch.
pytestmark = pytest.mark.skipif(
shutil.which("node") is None,
reason="the gate is a node script and the GitLab test image has no node -- see the note above",
)
_ROOT = Path(__file__).resolve().parents[1]
_SCRIPT = _ROOT / "ui-web" / "scripts" / "check-css.mjs"
_CSS = _ROOT / "ui-web" / "src" / "styles" / "page.css"
# (name, rule appended to the stylesheet, is it supposed to pass)
_CASES = [
# Rejected: a role color pinned to one theme, however it is spelled.
("plain hex background", ".probe { background: #ffd0a0; }", False),
("bare rgba overlay", ".probe { background: rgba(0, 0, 0, .5); }", False),
("opaque hex in a shadow", ".probe { box-shadow: 0 0 0 2px #b4402f; }", False),
(
"opaque stops in a gradient",
".probe { background-image: linear-gradient(180deg, #d96a5b, #b4402f); }",
False,
),
# Accepted: the literal is a ramp over whatever is behind it, or is data.
(
"multi-line shadow with alpha",
".probe { box-shadow:\n 0 1px 2px rgba(0, 0, 0, .06),\n 0 10px 24px -12px rgba(0, 0, 0, .2); }",
True,
),
("eight-digit hex shadow", ".probe { box-shadow: 0 18px 40px -24px #000c; }", True),
(
"gradient with alpha stops",
".probe { background-image: linear-gradient(180deg, rgba(0,0,0,.2), transparent); }",
True,
),
("slash-alpha rgb in a shadow", ".probe { box-shadow: 0 2px 4px rgb(0 0 0 / 20%); }", True),
("mask addressed by luminance", ".probe { mask-image: linear-gradient(180deg, #000, transparent); }", True),
# A data URI carries an unescapable `;`, and cutting the declaration there
# judged its tail as a rule of its own -- on a line that is a token.
(
"data-uri token holding a color",
":root { --probe-ico: url('data:image/svg+xml;utf8,<svg fill=\"rgb(247,242,228)\"></svg>'); }",
True,
),
("a rule written in tokens", ".probe { background: var(--surface); color: var(--text); }", True),
# A verdict must not depend on the declaration BEFORE it. The gate once
# shared one global regex between its "does this contain a literal" test
# and its scan, and `.test()` on a global regex moves `lastIndex`, so a
# declaration was skipped whenever the previous one left an offset past
# its end. 73 of the file's 167 literal-bearing declarations went
# unexamined, and every case above missed it by having only one.
("a token def, then a raw color", ".probe { --x: #ffffff; color: #f00; }", False),
(
"an allowlisted rule, then a raw color",
".probeB { mask-image: linear-gradient(#000, transparent); }\n.probeC { color: #f00; }",
False,
),
("an exempt shadow, then a raw color", ".probe { box-shadow: 0 1px 2px #0001; color: #f00; }", False),
]
def _run_gate_over(tmp_path: Path, extra_rule: str) -> int:
"""Run the gate against a copy of the tree with ``extra_rule`` appended."""
ui = tmp_path / "ui"
(ui / "scripts").mkdir(parents=True)
(ui / "src" / "styles").mkdir(parents=True)
shutil.copy(_SCRIPT, ui / "scripts" / "check-css.mjs")
(ui / "src" / "styles" / "page.css").write_text(
_CSS.read_text(encoding="utf-8") + "\n" + extra_rule + "\n", encoding="utf-8"
)
return subprocess.run(["node", str(ui / "scripts" / "check-css.mjs")], capture_output=True).returncode
def test_the_stylesheet_as_it_stands_passes() -> None:
"""The baseline, so a failure below is about the appended rule."""
assert subprocess.run(["node", str(_SCRIPT)], capture_output=True).returncode == 0
@pytest.mark.parametrize(("name", "rule", "allowed"), _CASES, ids=[c[0] for c in _CASES])
def test_the_gate_judges_a_rule_by_what_it_says(tmp_path: Path, name: str, rule: str, allowed: bool) -> None:
code = _run_gate_over(tmp_path, rule)
if allowed:
assert code == 0, f"the gate rejected a legitimate rule: {name}"
else:
assert code != 0, f"the gate accepted a color literal it exists to reject: {name}"
@pytest.mark.parametrize(("name", "rule", "allowed"), _CASES, ids=[c[0] for c in _CASES])
def test_the_verdict_does_not_depend_on_where_the_rule_sits(
tmp_path: Path, name: str, rule: str, allowed: bool
) -> None:
"""The same rule, prepended instead of appended, gets the same verdict.
The gate once exempted everything above the first ordinary rule, which
made it answer for position rather than for content.
"""
ui = tmp_path / "ui"
(ui / "scripts").mkdir(parents=True)
(ui / "src" / "styles").mkdir(parents=True)
shutil.copy(_SCRIPT, ui / "scripts" / "check-css.mjs")
(ui / "src" / "styles" / "page.css").write_text(rule + "\n" + _CSS.read_text(encoding="utf-8"), encoding="utf-8")
code = subprocess.run(["node", str(ui / "scripts" / "check-css.mjs")], capture_output=True).returncode
assert (code == 0) is allowed, f"verdict changed with position: {name}"
@pytest.mark.parametrize(("name", "rule", "allowed"), _CASES, ids=[c[0] for c in _CASES])
def test_the_verdict_does_not_depend_on_the_declaration_before_it(
tmp_path: Path, name: str, rule: str, allowed: bool
) -> None:
"""The same rule keeps its verdict with an exempt declaration ahead of it.
The position pass above varies where the rule sits in the file. This one
varies what precedes it, which is the axis a reader carrying state across
declarations fails on -- and the one the suite was blind to while the gate
skipped nearly half the file.
"""
ahead = ":root { --probe-ahead: #ffffff; }\n.probe-ahead { mask-image: linear-gradient(#000, transparent); }\n"
code = _run_gate_over(tmp_path, ahead + rule)
assert (code == 0) is allowed, f"verdict changed with an exempt declaration ahead of it: {name}"