## Summary
Reported as "deliver_files is broken". It is not: **the tool never ran.**
Every `deliver_files` call on one install was refused before dispatch -- five in a row in a single session -- and nothing was ever delivered:
```
Error: Invalid parameters for tool 'deliver_files': files should be array
```
What the model sent was the array, spelled as a string:
```json
{"files": "[{\"path\": \"...poster.png\", \"title\": \"...\"}]", "message": "..."}
```
`cast_params` already forgives that spelling for every scalar type -- `"3"` for an integer, `"true"` for a boolean -- and did not for arrays or objects, which are exactly the types a model is most likely to stringify.
## Type
- [x] Fix
- [ ] Feature
## The edge of the forgiving is the design
When the value is a string, try `json.loads`, and take the result **only if it is the type the schema asked for**. A string that is not JSON, or is JSON of the wrong shape, is returned unchanged so the validator still names the real problem -- the point is to widen what is accepted, not to swallow a type error.
**Nothing here can raise.** Turning "your argument has the wrong type" into a traceback would be worse than the refusal it replaces, so the parse is wrapped and failure falls through.
**A `type: "string"` field is never parsed.** JSON is ordinary text in a message body or a file's content, and replacing it would hand the tool something it never asked for. Two independent things prevent it today (the string branch returns before the parse is reached, and the parse only wins on a type match), which is why the case pinning it is a tripwire rather than a load-bearing assertion -- said so in its own docstring rather than left to look like coverage.
## Not just deliver_files
Six built-in tools declare an array or object parameter and were all exposed:
| tool | parameter |
|---|---|
| `deliver_files` | `files` |
| `ask_user` | `questions` |
| `image_generate` | `images` |
| `message` | `media` |
| `create_playbook` | `skills` |
| `video_generate` | `params` |
The same failure was already seen from the other side: `acp_client/acp_agent.py` records `update_task_state` and `operations should be array` from a real dispatch.
## Verification
```
make lint-python ruff check + ruff format --check, 1850 files
make lint-imports 9 contracts kept, 0 broken
pytest tests/ 19,699 passed, 7 failed (all pre-existing, see below)
```
- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots -- no user-facing string
**End to end, through the real tool**, with the exact payload that failed five times in production:
```
files sent as: str -> [{"path": ".../poster.png", "title": "..."}]
returned: Delivered 1 file(s): poster.png (72 B).
still "should be array": False
registered in the store: True
```
Twelve cases, five mutations, each killing at least one:
| mutation | red |
|---|---|
| drop the whole branch (the old behaviour) | 3 |
| raise instead of falling through on a parse failure | 5 |
| accept the parse whatever type it yielded | 1 |
| do not re-cast the parsed value, so items are not cast | 1 |
| widen the tuple to include `string` | **survived -- unreachable** |
The last is worth stating rather than hiding: that mutation cannot change behaviour, because the `type == "string"` branch returns earlier and the type check would refuse the result anyway. I also tried moving the new block above that branch, alone and combined with widening the tuple, and neither reddens anything either. The invariant holds by construction; the case that documents it says so.
**Pre-existing failures, checked against the base.** 7 failed in the full run, all in `tests/test_security_network.py` (`test_a_redirect_to_a_dangerous_address_is_refused` and siblings). Running that file alone on this same tree gives 4 -- the count is order-dependent -- and the change touches only `raven/agent/tools/params.py` plus a new test file, neither of which `raven/security/network.py` imports. The failures are the box's resolver: it returns poisoned answers roughly half the time, which I measured separately (`en.wikipedia.org` came back as `2001::1` on 3 of 6 lookups). Not introduced here, and worth someone's attention on its own.
## Risk
- [x] Security impact considered -- widens what reaches a tool's own validation, not what bypasses it. A parsed value is still validated against the same schema, and a parse that yields the wrong type is refused exactly as before
- [x] Backward compatibility considered -- a call that worked before takes the identical path; the new branch is reached only by a `str` where an array or object is declared, which previously always failed
- [x] Rollback path is clear -- single-commit revert
## Related Issues
N/A
## Swept
`mr-review-patterns` pre-submit sweep, over `git diff origin/main...HEAD`.
- **1 (a write location moved)** -- nothing moved; a branch was added to an existing dispatch on `target_type`. Grepped every caller of `cast_params`: the registry is the only one, and it runs `validate_params` right after, which is the reader this affects.
- **2 (a value with several entry points)** -- the cast is the single funnel every tool argument passes through, which is why the fix belongs here and not in `deliver_files`.
- **3 (a guard)** -- checked both directions: what it wrongly lets through (JSON of the wrong type, non-JSON text -- both refused, with cases) and what it wrongly refuses (a real list, unaffected, with a case).
- **4 (tests)** -- five mutations above, one honestly reported as unreachable. Failing-ID sets diffed against the base rather than counts.
- **6 (stale text)** -- the module docstring enumerated what is coerced ("a string that should be an integer"); it now names arrays and objects too. Grepped the tracked tree for other statements of the rule: none.
- **9 (error handling)** -- the new path adds one way to end and it is caught: `json.loads` raises `ValueError` on bad text and `TypeError` on a non-string, and both fall through to the untouched value.
- **11** -- `git ls-files | grep -c node_modules` is 0; the commit-message gate was run, not eyeballed.
Not applicable, with reasons: **3b, 3c, 3d, 3e** (no tool schema produced, no state moved, one lane, no assertion lost its symbol); **4b, 4c, 4d** (no platform branch, no shell, no rendered number); **5, 5b** (no dependency; `json` is stdlib and already imported across this package); **6b, 6c** (no backup, no generated artifact); **7, 7b** (nothing on disk); **8** (no asset, no secret); **10** (no prompt text).
## Deliberately not fixed
The error text still ends with `[Analyze the error above and try a different approach.]`, and the model plainly did not -- it sent the same shape five times. A message naming the expected spelling (`files must be a JSON array, not a string containing one`) would give it something to act on. That is a change to the registry's error surface, affects every tool, and deserves its own argument.
## Moved from !594
Opened on a compliant source branch. AGENTS.md 2.1 asks for a 3-5 word snake_case description and the previous branch had seven; GitLab cannot change an existing merge request's source branch, so the change moves rather than the name.
Same commits, rebased onto `c2a9768f`. Both code findings from the review on !594 are already in them -- `RecursionError` caught with its regression case, and the implementation comment reduced to the two non-obvious reasons. Nothing else changed, so that review carries.
297 lines
11 KiB
Python
297 lines
11 KiB
Python
"""Unit tests for the session working-directory resolver."""
|
|
|
|
from pathlib import Path
|
|
from types import SimpleNamespace
|
|
|
|
import pytest
|
|
|
|
from raven.agent.workdir import (
|
|
WorkdirPolicy,
|
|
WorkdirResolver,
|
|
bind,
|
|
current,
|
|
validate_override,
|
|
)
|
|
|
|
|
|
class _StubSessions:
|
|
"""Stands in for SessionManager: only ``get_or_create`` is used."""
|
|
|
|
def __init__(self, metadata_by_key: dict[str, dict]) -> None:
|
|
self._metadata_by_key = metadata_by_key
|
|
|
|
def get_or_create(self, key: str):
|
|
return SimpleNamespace(metadata=self._metadata_by_key.get(key, {}))
|
|
|
|
|
|
def test_every_chat_on_a_channel_shares_one_directory(tmp_path: Path) -> None:
|
|
"""The channel is the unit a user configures, so it is the unit of
|
|
isolation; two chats on it are the same kind of work."""
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=tmp_path / "chanwork",
|
|
)
|
|
assert resolver.resolve("web:2fb833e6") == tmp_path / "chanwork" / "web"
|
|
assert resolver.resolve("web:other") == tmp_path / "chanwork" / "web"
|
|
assert resolver.resolve("qq:2fb833e6") == tmp_path / "chanwork" / "qq"
|
|
|
|
|
|
def test_a_configured_channel_workspace_wins_over_the_default_root(tmp_path: Path) -> None:
|
|
pinned = tmp_path / "elsewhere"
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=tmp_path / "chanwork",
|
|
channel_workspaces={"qq": str(pinned), "web": ""},
|
|
)
|
|
assert resolver.resolve("qq:123", create=False) == pinned
|
|
# An empty configured value is not a configured value.
|
|
assert resolver.resolve("web:123", create=False) == tmp_path / "chanwork" / "web"
|
|
|
|
|
|
def test_per_session_creates_the_directory(tmp_path: Path) -> None:
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=tmp_path / "chanwork",
|
|
)
|
|
assert resolver.resolve("web:abc").is_dir()
|
|
|
|
|
|
def test_colonless_key_gets_a_placeholder_chat_id(tmp_path: Path) -> None:
|
|
"""``heartbeat`` is the one session key in the codebase with no colon."""
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=tmp_path / "chanwork",
|
|
)
|
|
assert resolver.resolve("heartbeat") == tmp_path / "chanwork" / "heartbeat"
|
|
|
|
|
|
def test_chat_id_with_separators_is_sanitised(tmp_path: Path) -> None:
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=tmp_path / "chanwork",
|
|
)
|
|
resolved = resolver.resolve("whatsapp:../../etc/passwd")
|
|
assert (tmp_path / "chanwork") in resolved.parents
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"session_key",
|
|
["web:..", "web:.", "..:chat", ".:chat", "..:.."],
|
|
)
|
|
def test_dot_only_key_segments_cannot_escape_the_session_root(tmp_path: Path, session_key: str) -> None:
|
|
"""``safe_filename`` passes ``.`` and ``..`` through untouched.
|
|
|
|
That is harmless for ``sessions/<chat_id>.jsonl`` (the suffix makes an
|
|
ordinary name) but not for a bare directory segment: ``ws/web/..`` resolves
|
|
to ``ws``, so every session with such a key would share one directory and
|
|
one shadow repo -- the exact isolation this layout exists to provide. The
|
|
sibling case with separators is not enough to catch this: ``/`` becomes
|
|
``_``, which leaves one inert segment, while a bare ``..`` never had one.
|
|
"""
|
|
root = tmp_path / "chanwork"
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=root,
|
|
)
|
|
resolved = resolver.resolve(session_key, create=False)
|
|
assert root in resolved.parents
|
|
assert resolved != root
|
|
assert ".." not in resolved.parts
|
|
|
|
|
|
def test_launch_dir_ignores_the_session_key(tmp_path: Path) -> None:
|
|
project = tmp_path / "project"
|
|
project.mkdir()
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.LAUNCH_DIR,
|
|
agent_home=tmp_path / "home",
|
|
launch_dir=project,
|
|
)
|
|
assert resolver.resolve("cli:one") == project
|
|
assert resolver.resolve("cli:two") == project
|
|
|
|
|
|
def test_persisted_override_beats_the_policy_default(tmp_path: Path) -> None:
|
|
pinned = tmp_path / "pinned"
|
|
pinned.mkdir()
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=tmp_path / "chanwork",
|
|
sessions=_StubSessions({"web:abc": {"workdir": str(pinned)}}),
|
|
)
|
|
assert resolver.resolve("web:abc") == pinned
|
|
|
|
|
|
def test_explicit_workdir_beats_the_persisted_override(tmp_path: Path) -> None:
|
|
"""A flag typed for this run wins over anything on disk."""
|
|
pinned = tmp_path / "pinned"
|
|
flagged = tmp_path / "flagged"
|
|
pinned.mkdir()
|
|
flagged.mkdir()
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.LAUNCH_DIR,
|
|
agent_home=tmp_path,
|
|
launch_dir=tmp_path,
|
|
explicit_workdir=flagged,
|
|
sessions=_StubSessions({"cli:abc": {"workdir": str(pinned)}}),
|
|
)
|
|
assert resolver.resolve("cli:abc") == flagged
|
|
|
|
|
|
def test_override_must_be_absolute(tmp_path: Path) -> None:
|
|
with pytest.raises(ValueError, match="absolute"):
|
|
validate_override("relative/path", tmp_path)
|
|
|
|
|
|
@pytest.mark.parametrize("subtree", ["user_memory", "skills", "sessions"])
|
|
def test_override_cannot_point_into_agent_home_internals(tmp_path: Path, subtree: str) -> None:
|
|
with pytest.raises(ValueError, match=subtree):
|
|
validate_override(tmp_path / subtree / "nested", tmp_path)
|
|
|
|
|
|
def test_override_cannot_be_agent_home_itself(tmp_path: Path) -> None:
|
|
"""Working in agent home puts every protected subtree one relative path away."""
|
|
with pytest.raises(ValueError, match="agent home"):
|
|
validate_override(tmp_path, tmp_path)
|
|
|
|
|
|
def test_override_cannot_be_an_ancestor_of_agent_home(tmp_path: Path) -> None:
|
|
"""An ancestor is refused for a stronger reason than agent home itself.
|
|
|
|
The realistic ancestor is `~/.raven`, which is not merely agent home's
|
|
parent but the instance data directory: `config.json` (provider keys),
|
|
`oauth/`, `cron/`, `logs/`. The per-turn checkpoint runs `add -A` over the
|
|
working directory, and the `.raven/` default exclude cannot help when
|
|
`.raven` is itself the work-tree root, so working there commits every
|
|
credential into a shadow repo. `user_memory/` and `skills/` are also back
|
|
within relative reach, one level deeper than from agent home.
|
|
"""
|
|
home = tmp_path / "nested" / "home"
|
|
home.mkdir(parents=True)
|
|
with pytest.raises(ValueError, match="agent home"):
|
|
validate_override(tmp_path, home)
|
|
with pytest.raises(ValueError, match="agent home"):
|
|
validate_override(tmp_path / "nested", home)
|
|
|
|
|
|
def test_override_may_be_a_sibling_of_agent_home(tmp_path: Path) -> None:
|
|
"""Containment is the test, not proximity: a directory next to agent home
|
|
holds none of it, so `add -A` there captures nothing protected."""
|
|
home = tmp_path / "nested" / "home"
|
|
home.mkdir(parents=True)
|
|
sibling = tmp_path / "nested" / "project"
|
|
sibling.mkdir()
|
|
assert validate_override(sibling, home) == sibling
|
|
|
|
|
|
def test_binding_is_scoped(tmp_path: Path) -> None:
|
|
assert current() is None
|
|
with bind(tmp_path):
|
|
assert current() == tmp_path
|
|
assert current() is None
|
|
|
|
|
|
def test_repoint_moves_the_running_turns_binding_and_only_that(tmp_path: Path) -> None:
|
|
"""``repoint`` serves the ``rebind_workdir`` grant: the very next read in
|
|
the same task sees the new root, and the enclosing ``bind``'s finally
|
|
still resets at turn end -- nothing leaks past the turn."""
|
|
from raven.agent.workdir import repoint
|
|
|
|
first = tmp_path / "first"
|
|
moved = tmp_path / "moved"
|
|
assert current() is None
|
|
with bind(first):
|
|
repoint(moved)
|
|
assert current() == moved
|
|
assert current() is None
|
|
|
|
|
|
def test_mount_root_covers_every_per_session_directory(tmp_path: Path) -> None:
|
|
"""One sandbox mount has to contain every directory the process can produce."""
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=tmp_path,
|
|
session_root=tmp_path / "chanwork",
|
|
)
|
|
root = resolver.mount_root()
|
|
assert root == tmp_path / "chanwork"
|
|
assert root in resolver.resolve("web:abc").parents
|
|
|
|
|
|
def test_mount_root_is_the_launch_dir_for_terminal_entrypoints(tmp_path: Path) -> None:
|
|
project = tmp_path / "project"
|
|
project.mkdir()
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.LAUNCH_DIR,
|
|
agent_home=tmp_path / "home",
|
|
launch_dir=project,
|
|
)
|
|
assert resolver.mount_root() == project
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"configured",
|
|
[
|
|
"", # agent home itself, filled in by the test
|
|
"skills/nested",
|
|
"relative/path",
|
|
"..",
|
|
],
|
|
)
|
|
def test_a_bad_channel_workspace_is_dropped_not_obeyed(tmp_path: Path, configured: str, caplog) -> None:
|
|
"""`channels.<name>.workspace` reaches the resolver from a config file and
|
|
from the web UI's channels form, and nothing else validates it. Left
|
|
unchecked, `channels.x.workspace = <agent home>` is accepted and the
|
|
per-turn `add -A` then runs over user_memory/ and sessions/ -- the shape
|
|
every other entry point already rejects.
|
|
"""
|
|
home = tmp_path / "home"
|
|
home.mkdir()
|
|
value = str(home) if not configured else str(home / configured) if configured != "relative/path" else configured
|
|
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=home,
|
|
session_root=tmp_path / "chanwork",
|
|
channel_workspaces={"qq": value},
|
|
)
|
|
|
|
assert resolver.resolve("qq:1", create=False) == tmp_path / "chanwork" / "qq"
|
|
|
|
|
|
def test_a_valid_channel_workspace_is_still_honoured(tmp_path: Path) -> None:
|
|
"""The guard must reject shapes, not the feature."""
|
|
home = tmp_path / "home"
|
|
pinned = tmp_path / "elsewhere"
|
|
home.mkdir()
|
|
pinned.mkdir()
|
|
resolver = WorkdirResolver(
|
|
WorkdirPolicy.PER_CHANNEL,
|
|
agent_home=home,
|
|
session_root=tmp_path / "chanwork",
|
|
channel_workspaces={"qq": str(pinned)},
|
|
)
|
|
|
|
assert resolver.resolve("qq:1", create=False) == pinned
|
|
|
|
|
|
def test_default_channel_root_never_lands_in_the_system_tmp(tmp_path: Path) -> None:
|
|
"""A home directly under the filesystem root is the case a plain sibling
|
|
gets wrong: `--home /workspace` (a common container mount) would put every
|
|
channel in the shared, world-writable, reboot-cleared system /tmp.
|
|
"""
|
|
from raven.agent.workdir import default_channel_root
|
|
|
|
assert default_channel_root(Path("/workspace")) == Path("/workspace-tmp")
|
|
assert default_channel_root(Path("/")) == Path("/raven-tmp")
|
|
# The shape everyone actually runs is unchanged.
|
|
assert default_channel_root(Path("/root/.raven/workspace")) == Path("/root/.raven/tmp")
|
|
assert default_channel_root(tmp_path / "a" / "home") == tmp_path / "a" / "tmp"
|