## Summary The desk builds the main conversation's changed-files rows on the client, from the file tools' own arguments and results. A file a shell command created (`python3 build.py > out.txt`), rewrote (`echo more >> notes.md`) or removed through a glob left no row, because a command returns its output and names no file. #670 closed that gap for every sub-agent lane by listing the working directory on either side of a command; this does the same for the main conversation, which needs a wire field because its rows are drawn on the page rather than read from a run record. - `raven/agent/loop/turn_path.py`: around a call named `exec` and only that, the directory the command runs in is listed before and after the call with `snapshot.take`, off the event loop. The tool says where that is (`ExecTool.listing_root`, resolved the way `execute` resolves its cwd): the turn's working directory unless the call names a `working_dir`, and no listing at all for a command sent to a registered `machine`, whose files are not on this disk. The in-process sub-agent lane asks the same question through `snapshot.root_for`, so a sub-agent's `working_dir` command is recorded too. Created and modified paths the call did not name itself (its `file_change`, its removals) become the new `tool.complete.file_written` list of `{path, created, size, lines}`; a deletion the listing finds joins `file_removed` with no body, since the file was gone before anything read it. Stored on the tool entry under the same name and passed through `session.resume`, so a reloaded page draws what the live one drew. - Wire: `FileWritten` in `raven/rpc/models.py`, on `ToolCompletePayload` and `TranscriptMessage`; `rpc-schema/openrpc.json` and both generated clients regenerated. `lines` is an integer for a created text file of 256 KB or less and `null` otherwise, including every rewrite, whose old contents the listing never held. - Page: a created file is an `A` row with its line count, a rewritten one a bare `M`, neither with a patch; a row a file tool already made this turn is left alone (its arguments say more than a listing can); the reload path replays the stored field. A change row with no patch to show opens the file itself instead of an empty patch pane (a removal keeps its pane; a task file's row, keyed by node rather than path, is untouched). The offline fixture's GTM run gains one such command so the demo desk shows both rows and the fixture-shape gate sees the field sent. Not in this PR: the ACP translator still surfaces no file changes to an ACP client, the same as for `file_removed`. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification Run from the worktree on top of `0133c67c` (main with #656, #657 and #670; the branch was rebased onto it when main's own contract additions collided with the generated client, which was regenerated from the merged schema): - `.venv/bin/ruff check raven tests`; `.venv/bin/ruff format --check raven tests` -- clean - `.venv/bin/python -m pytest tests/test_agent_loop_session_stamps.py tests/test_rpc_spine.py tests/test_rpc_session.py tests/test_rpc_schema_match.py tests/test_turn_wire_shape_conformance.py tests/test_spine_events.py tests/test_contracts_two_tier_ledger.py tests/test_kernel_budget.py tests/test_workdir_snapshot.py tests/test_subagent_activity.py tests/test_shell_file_removals.py tests/test_acp_updates.py -q` -- 952 passed - `make test-python` (`uv run --frozen --all-extras pytest -q`) -- 25798 passed, 109 skipped - `npm test --prefix ui-web` (the whole suite, including the fixture-shape gate) -- 200 files, 2767 tests passed; `npm run lint --prefix ui-web` -- 0 errors (4 pre-existing hook-dependency warnings in files this change does not touch); `npm run type-check --prefix ui-web`, `npm run gen:check --prefix ui-web`, `npm run lint:rpc --prefix ui-tui` -- clean - Mutation checks, each restored afterwards: the listing taken around every tool; a created file reported as a rewrite; the held-out result path dropped; the storage rename disabled; `file_written` emitted when empty; the key dropped from `session.resume`; `file_written=None` on the served `ToolEvent`; a count of 0 on a decode failure; the 256 KB cap dropped; the created row drawn as `M`; the existing-row guard dropped; the ninth argument dropped in the stage; the file door removed, ignoring the kind, ignoring a task key; the fixture no longer sending the field (stored, then live); the row takeover removed and its removal guard dropped; `.raven` back in the walk; the count back on the loop; the shell ignoring its bound directory -- every one turned at least one new test red. - Review round two (reviewer's blocker): a command with a `working_dir` of its own is listed there, in both lanes, and a `machine` command takes no listing -- `tests/test_shell_approval.py` (the root follows the arguments), `tests/test_workdir_snapshot.py` (`root_for`), `tests/test_agent_loop_session_stamps.py` (the real `ExecTool` with `working_dir`; a remote command takes no listing) and `tests/test_subagent_manager.py` (a sub-agent command outside its workspace); each red with the root pinned back to the lane's directory or the machine guard dropped. - Review round: one row per file when a file tool follows a command that already left the file behind (the listing keys by the resolved path, the tool by the path the model typed); the listing skips `.raven`, where the checkpoint keeps a shadow git repo inside the working directory; the line counting moved into the worker beside the walks; a test through the real `ExecTool` (its cwd is the directory the listing walks), the served `ToolEvent` hop, a non-text and an over-cap created file, and the live fixture frame. - Real gateway (`raven serve` from this branch, isolated `RAVEN_HOME`, default model `deepseek/deepseek-v4-flash`): the main agent runs `printf "gen one\ngen two\n" > work/main_generated.txt` and `echo more >> work/main_shell_mod.md` beside its file-tool steps. Before this change neither command left a row. After: the desk diff tab draws `A work/main_generated.txt +2` and `M work/main_shell_mod.md` beside `M work/main_keep.md +1-1` (edit_file), a red `D work/main_old.md -2` (`rm`) and `A work/e2e_main_new.md +2` (write_file); a file one command created and the next removed draws nothing; reopening the page (`session.resume`) draws the identical five rows, and the stored tool entries carry `file_written` with `created: true, lines: 2` and `created: false, lines: null`. The same run repeated on the final tree (after the review round) and its reload draw the same rows. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ## Risk Behaviour change: every main-conversation `exec` now walks the turn's working directory twice, bounded at 20000 entries with the machinery directories (`.git`, `node_modules`, `.venv`, caches) skipped and run off the event loop; a tree past the bound records nothing from listings, and the file tools' own rows stand as before. A created file's line count reads the file (up to 256 KB). Wire: one new optional key on `tool.complete` and on `session.resume` tool entries, absent on every call that is not a command; a client that does not read it sees what it saw before. Rollback is reverting the commits. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
977 lines
43 KiB
Python
977 lines
43 KiB
Python
"""ShellCommandPolicy: safe, hard-denied, and approval-required command decisions."""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from raven.agent.tools.registry import ToolRegistry
|
|
from raven.agent.tools.shell import ExecTool
|
|
from raven.config.schema import PermissionsConfig
|
|
from raven.contracts.permissions import ApprovalChoice, ApprovalOutcome
|
|
from raven.contracts.tool import Continuation
|
|
from raven.permissions.builtin import BuiltinRulings
|
|
from raven.permissions.gate import PermissionGate
|
|
from raven.permissions.shell_policy import CommandDecision, ShellCommandPolicy
|
|
from raven.permissions.turn import start_permission_turn
|
|
from raven.sandbox import ExecResult, SandboxExecutor
|
|
|
|
|
|
@pytest.fixture
|
|
def policy() -> ShellCommandPolicy:
|
|
# Mirrors the shipped default in that `rm` is classified from tokens, not
|
|
# matched here -- a fixture that still pattern-denies it would test a
|
|
# policy no user runs.
|
|
return ShellCommandPolicy(deny_patterns=[r"\b(mkfs|diskpart)\b"])
|
|
|
|
|
|
@pytest.fixture
|
|
def asking_policy() -> ShellCommandPolicy:
|
|
"""The ACP editor's shape: the deletion family declared, so deletes ask.
|
|
|
|
The terminal ships no asking families -- deletes answer to the tiers -- but
|
|
the matcher's reach still matters wherever a surface declares it.
|
|
"""
|
|
from raven.permissions.shell_policy import DELETE_MATCHERS
|
|
|
|
policy = ShellCommandPolicy(deny_patterns=[r"\b(mkfs|diskpart)\b"])
|
|
for name, matcher in DELETE_MATCHERS:
|
|
policy.register_approval_matcher(name, matcher)
|
|
return policy
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"pwd",
|
|
"ls -la",
|
|
"echo hello",
|
|
"find . -name '*.py'",
|
|
"printf 'rm file.txt'",
|
|
"echo 'find . -delete'",
|
|
"git grep -n shutdown",
|
|
"grep -rn reboot /var/log",
|
|
"man shutdown",
|
|
"systemctl show reboot.target",
|
|
"grep -rn 'systemctl poweroff' docs/",
|
|
"bash -lc 'ls'",
|
|
],
|
|
)
|
|
def test_safe_commands_are_allowed(policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"rm -rf /",
|
|
"rm -rf /*",
|
|
"rm -rf ~",
|
|
"rm -r $HOME",
|
|
"echo ready && rm -rf /",
|
|
"mkfs /dev/test",
|
|
"shutdown now",
|
|
"halt",
|
|
"sudo -n reboot",
|
|
'bash -c "poweroff"',
|
|
"systemctl poweroff",
|
|
"systemctl reboot",
|
|
"sudo systemctl reboot",
|
|
"busybox poweroff",
|
|
"loginctl poweroff",
|
|
"systemctl -i poweroff",
|
|
"init 0",
|
|
"init 6",
|
|
"telinit 0",
|
|
"telinit 6",
|
|
],
|
|
)
|
|
def test_hard_denied_commands_cannot_be_approved(policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.HARD_DENY
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"rm file.txt",
|
|
"rm -f file.txt",
|
|
"rm file1 file2",
|
|
"/bin/rm file.txt",
|
|
"rm --force file.txt",
|
|
"sudo rm file.txt",
|
|
"sudo -u root rm file.txt",
|
|
"sudo --user=root unlink file.txt",
|
|
"command unlink file.txt",
|
|
"MODE=test rm file.txt",
|
|
"env MODE=test rm file.txt",
|
|
"env -u MODE rm file.txt",
|
|
"unlink file.txt",
|
|
"find ./tmp -delete",
|
|
"echo ready && rm file.txt",
|
|
"printf done | unlink file.txt",
|
|
"cd /tmp\nrm file.txt",
|
|
"cd /tmp\r\nunlink file.txt",
|
|
"(rm file.txt)",
|
|
"{ rm file.txt; }",
|
|
"echo $(rm file.txt)",
|
|
"echo `rm file.txt`",
|
|
"nohup rm file.txt &",
|
|
'bash -c "rm file.txt"',
|
|
"bash -c'rm file.txt'",
|
|
'bash -c"rm file.txt"',
|
|
'bash --rcfile setup.sh -c "rm file.txt"',
|
|
'sh -lc "find tmp -delete"',
|
|
'find . -name "*.log" -exec rm {} \\;',
|
|
'find . -name "*.log" -execdir unlink {} \\;',
|
|
'find . -exec sh -c "rm \\"$1\\"" _ {} \\;',
|
|
],
|
|
)
|
|
def test_delete_commands_require_approval(asking_policy: ShellCommandPolicy, command: str) -> None:
|
|
assert asking_policy.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
|
|
|
|
def test_ordinary_deletes_answer_to_the_tiers_by_default(policy: ShellCommandPolicy) -> None:
|
|
"""The shipped terminal policy declares no asking family: an ordinary
|
|
delete is a mutation like any other, decided by the permission tiers."""
|
|
assert policy.evaluate("rm file.txt") is CommandDecision.ALLOW
|
|
assert policy.evaluate("rm -rf build/") is CommandDecision.ALLOW
|
|
|
|
|
|
def test_hard_deny_wins_when_command_also_matches_approval(asking_policy: ShellCommandPolicy) -> None:
|
|
assert asking_policy.evaluate("unlink old.txt && rm -rf /") is CommandDecision.HARD_DENY
|
|
|
|
|
|
def test_matcher_failure_is_fail_closed(policy: ShellCommandPolicy) -> None:
|
|
def broken_matcher(command: str) -> bool:
|
|
raise RuntimeError("broken")
|
|
|
|
policy.register_approval_matcher("broken", broken_matcher)
|
|
|
|
assert policy.evaluate("echo harmless") is CommandDecision.HARD_DENY
|
|
|
|
|
|
@pytest.mark.parametrize("command", ["echo 'unterminated", "echo trailing\\"])
|
|
def test_shell_parse_failure_is_fail_closed(policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.HARD_DENY
|
|
|
|
|
|
class _RecordingExecutor(SandboxExecutor):
|
|
def __init__(self, *, sandboxed: bool) -> None:
|
|
self._sandboxed = sandboxed
|
|
self.commands: list[str] = []
|
|
|
|
@property
|
|
def is_sandboxed(self) -> bool:
|
|
return self._sandboxed
|
|
|
|
async def exec(self, command: str, **kwargs) -> ExecResult:
|
|
self.commands.append(command)
|
|
return ExecResult(stdout="ok", stderr="", exit_code=0)
|
|
|
|
|
|
class _ApprovalResponder:
|
|
def __init__(self, answers: list[bool]) -> None:
|
|
self.answers = answers
|
|
self.requests: list[dict] = []
|
|
|
|
async def await_approval(self, **request) -> ApprovalOutcome:
|
|
self.requests.append(request)
|
|
allowed = self.answers.pop(0)
|
|
return ApprovalOutcome(choice=ApprovalChoice.ALLOW if allowed else ApprovalChoice.DENY)
|
|
|
|
|
|
def _gated_exec(executor, responder, tmp_path, *, mode: str = "ask", families=()):
|
|
"""The real dispatch path: gate at the registry door, ExecTool behind it."""
|
|
builtin = BuiltinRulings()
|
|
for name, matcher in families:
|
|
builtin._policy.register_approval_matcher(name, matcher)
|
|
gate = PermissionGate(
|
|
config_source=lambda: PermissionsConfig(mode=mode),
|
|
builtin=builtin,
|
|
allow_ask=True,
|
|
)
|
|
registry = ToolRegistry(permission_gate=gate)
|
|
registry.register(ExecTool(executor=executor, working_dir=str(tmp_path)))
|
|
start_permission_turn(responder, conversation_id="session-a", turn_id="turn-a")
|
|
return registry
|
|
|
|
|
|
async def test_direct_delete_executes_once_after_approval(tmp_path) -> None:
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
responder = _ApprovalResponder([True])
|
|
registry = _gated_exec(executor, responder, tmp_path)
|
|
|
|
result = await registry.execute("exec", {"command": "rm file.txt"})
|
|
|
|
assert "Exit code: 0" in result
|
|
assert executor.commands == ["rm file.txt"]
|
|
assert responder.requests == [
|
|
{
|
|
"conversation_id": "session-a",
|
|
"turn_id": "turn-a",
|
|
"tool_call_id": "",
|
|
"command": "rm file.txt",
|
|
"description": "Approve this action: rm file.txt",
|
|
"suggested_pattern": "",
|
|
"kind": "shell.exec",
|
|
"family": "",
|
|
"origin": "",
|
|
"origin_name": "",
|
|
"evidence": {"command": "rm file.txt", "cwd": str(tmp_path)},
|
|
}
|
|
]
|
|
|
|
|
|
async def test_direct_delete_without_responder_is_denied(tmp_path) -> None:
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
registry = _gated_exec(executor, None, tmp_path)
|
|
|
|
result = await registry.execute("exec", {"command": "unlink file.txt"})
|
|
|
|
assert result.retryable is False
|
|
assert result.blocks_call is True
|
|
assert result.continuation is Continuation.CONTINUE
|
|
assert "requires user approval" in str(result)
|
|
assert "not interactive" in str(result)
|
|
assert "Do not retry" in str(result)
|
|
assert executor.commands == []
|
|
|
|
|
|
async def test_denied_command_is_not_prompted_again_in_same_turn(tmp_path) -> None:
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
responder = _ApprovalResponder([False])
|
|
registry = _gated_exec(executor, responder, tmp_path)
|
|
|
|
first = await registry.execute("exec", {"command": "find tmp -delete"})
|
|
second = await registry.execute("exec", {"command": "find tmp -delete"})
|
|
|
|
assert first.retryable is False
|
|
assert first.blocks_call is True
|
|
assert first.continuation is Continuation.CONTINUE
|
|
assert "denied" in str(first).lower()
|
|
assert "denied" in str(second).lower()
|
|
assert executor.commands == []
|
|
assert len(responder.requests) == 1
|
|
|
|
|
|
async def test_allow_once_does_not_cover_a_second_execution(tmp_path) -> None:
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
responder = _ApprovalResponder([True, False])
|
|
registry = _gated_exec(executor, responder, tmp_path)
|
|
|
|
first = await registry.execute("exec", {"command": "rm file.txt"})
|
|
second = await registry.execute("exec", {"command": "rm file.txt"})
|
|
|
|
assert "Exit code: 0" in first
|
|
assert "denied" in str(second).lower()
|
|
assert executor.commands == ["rm file.txt"]
|
|
assert len(responder.requests) == 2
|
|
|
|
|
|
async def test_new_turn_can_prompt_for_a_previously_denied_command(tmp_path) -> None:
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
responder = _ApprovalResponder([False, True])
|
|
registry = _gated_exec(executor, responder, tmp_path)
|
|
|
|
first = await registry.execute("exec", {"command": "unlink file.txt"})
|
|
start_permission_turn(responder, conversation_id="session-a", turn_id="turn-b")
|
|
second = await registry.execute("exec", {"command": "unlink file.txt"})
|
|
|
|
assert "denied" in str(first).lower()
|
|
assert "Exit code: 0" in second
|
|
assert executor.commands == ["unlink file.txt"]
|
|
assert len(responder.requests) == 2
|
|
|
|
|
|
async def test_hard_denied_command_never_requests_approval(tmp_path) -> None:
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
responder = _ApprovalResponder([True])
|
|
registry = _gated_exec(executor, responder, tmp_path)
|
|
|
|
result = await registry.execute("exec", {"command": "rm -rf /"})
|
|
|
|
assert result.retryable is False
|
|
assert result.blocks_call is True
|
|
assert result.continuation is Continuation.CONTINUE
|
|
assert "blocked" in str(result)
|
|
assert "Do not retry" in str(result)
|
|
assert responder.requests == []
|
|
assert executor.commands == []
|
|
|
|
|
|
async def test_a_sandboxed_executor_earns_no_relaxation(tmp_path) -> None:
|
|
# Boxlite mounts the real workspace read-write into the VM, so a
|
|
# filesystem catastrophe inside it reaches host data. The deny list
|
|
# holds through a sandboxed executor exactly as through a direct one.
|
|
executor = _RecordingExecutor(sandboxed=True)
|
|
responder = _ApprovalResponder([False])
|
|
registry = _gated_exec(executor, responder, tmp_path, mode="full")
|
|
|
|
result = await registry.execute("exec", {"command": "rm -rf /"})
|
|
|
|
assert result.blocks_call is True
|
|
assert "blocked" in str(result)
|
|
assert responder.requests == []
|
|
assert executor.commands == []
|
|
|
|
|
|
class TestTheTargetIsWhatMakesADeleteUnconditional:
|
|
"""Only a recursive delete aimed at ``/`` or the home tree is unconditional.
|
|
|
|
Everything the machine holds lives under those two paths, so no mode, rule
|
|
or click can rescue them. An ordinary recursive delete (``rm -rf build/``)
|
|
is daily work: it answers to the tiers, and to the deletion family on a
|
|
surface that declares one.
|
|
"""
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"rm -f parse_turns.py turns.json stats.json",
|
|
"rm scratch.txt",
|
|
"rm -f build.py app_css.css",
|
|
"cd /work && rm -f a.py",
|
|
],
|
|
)
|
|
def test_naming_the_files_asks_on_a_declaring_surface(
|
|
self, asking_policy: ShellCommandPolicy, command: str
|
|
) -> None:
|
|
assert asking_policy.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"sudo rm -rf /",
|
|
"bash -c 'rm -rf /'",
|
|
"rm -fr /",
|
|
"rm -R ~/",
|
|
"rm --recursive ~",
|
|
# A separated recursive flag must not walk past the check.
|
|
"rm -f -r /",
|
|
],
|
|
)
|
|
def test_catastrophic_targets_stay_unconditional(self, policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.HARD_DENY
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
["rm -rf build", "rm -fr build", "rm -R build", "rm --recursive build", "rm -f -r build"],
|
|
)
|
|
def test_an_ordinary_recursive_delete_answers_to_the_tiers(self, policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
def test_a_recursive_delete_quoted_inside_another_command_is_still_text(self, policy: ShellCommandPolicy) -> None:
|
|
"""The old regexp searched the raw string, so mentioning the command
|
|
was as forbidden as running it."""
|
|
assert policy.evaluate("echo rm -rf nope") is CommandDecision.ALLOW
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"timeout 5 rm -rf /",
|
|
"timeout -s KILL 5 rm -rf /",
|
|
"timeout --kill-after=2 5s rm -rf ~",
|
|
"nice rm -rf /",
|
|
"nice -n 10 rm -rf /",
|
|
"ionice -c 3 rm -rf ~",
|
|
"time rm -rf /",
|
|
"setsid rm -rf /",
|
|
"stdbuf -oL rm -rf /*",
|
|
"sudo timeout 5 rm -rf /",
|
|
"env FOO=1 timeout 5 rm -rf /",
|
|
"xargs sh -c 'rm -rf /'",
|
|
"find / -name '*' -exec rm -rf {} +",
|
|
],
|
|
)
|
|
def test_a_wrapper_does_not_launder_a_catastrophic_delete(self, policy: ShellCommandPolicy, command: str) -> None:
|
|
"""A program that runs another program must not hide the delete behind
|
|
itself: the wrapper stripping reads through to the target."""
|
|
assert policy.evaluate(command) is CommandDecision.HARD_DENY
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
["xargs rm -rf < list.txt", "git ls-files -o | xargs rm -rf", "xargs -0 -n1 rm -rf", "xargs -I{} rm -rf {}"],
|
|
)
|
|
def test_stdin_fed_targets_are_past_the_tripwire(self, policy: ShellCommandPolicy, command: str) -> None:
|
|
"""The unconditional list is a lexical tripwire, not a boundary: targets
|
|
arriving on stdin are invisible to it, exactly like a delete written in
|
|
Python. Those runs answer to the tiers (and the sandbox), which is the
|
|
actual boundary."""
|
|
assert policy.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"xargs rm -f a.py",
|
|
"timeout 5 rm a.py",
|
|
],
|
|
)
|
|
def test_a_wrapped_named_delete_still_only_asks(self, asking_policy: ShellCommandPolicy, command: str) -> None:
|
|
assert asking_policy.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# The search path is benign, so path inspection alone would clear it;
|
|
# the catastrophic target rides the -exec command, which has to be
|
|
# read as its own argv.
|
|
"find . -exec rm -rf --no-preserve-root / {} +",
|
|
"find . -type f -execdir rm -rf / {} ;",
|
|
'find . -exec sh -c "rm -rf /" {} +',
|
|
# The delete rides a shell the carried command spawns: what find
|
|
# walks under / is deleted all the same, one wrapper deeper.
|
|
"find / -exec sh -c 'rm -rf \"$1\"' sh {} +",
|
|
# Or a recognized command runner: timeout/nice hand the walk on.
|
|
"find / -exec timeout 5 rm -rf --no-preserve-root {} +",
|
|
"find / -execdir nice -n 5 rm -rf {} +",
|
|
"find . -exec timeout 5 rm -rf / {} +",
|
|
# Multiple search roots: the catastrophic one is not first, so
|
|
# reading only paths[0] would clear it.
|
|
"find /home / -delete",
|
|
"find /tmp/scratch ~ -exec rm {} +",
|
|
],
|
|
)
|
|
def test_a_find_exec_carrying_a_root_delete_is_unconditional(
|
|
self, policy: ShellCommandPolicy, command: str
|
|
) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.HARD_DENY
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# -exec of an ordinary delete: the carried command targets what find
|
|
# matched under a benign path, not the root, so it is daily work.
|
|
'find . -name "*.log" -exec rm -rf {} +',
|
|
"find build -execdir rm {} ;",
|
|
# / here is -path's argument, not a search root: the walk starts
|
|
# at . and deletes nothing outside it.
|
|
"find . -path / -delete",
|
|
],
|
|
)
|
|
def test_an_ordinary_find_exec_answers_to_the_tiers(self, policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"rmdir /s C:\\",
|
|
"rd /s /q C:\\",
|
|
"del /f /s /q C:\\*",
|
|
"rmdir /s %USERPROFILE%",
|
|
"del /s C:",
|
|
"cmd /c rmdir /s C:\\",
|
|
"cmd.exe /d /c del /f /s /q C:\\*",
|
|
'cmd /c "rmdir /s %USERPROFILE%"',
|
|
"echo done\nrmdir /s %USERPROFILE%",
|
|
" rmdir /s %USERPROFILE%",
|
|
# Quoting the target is ordinary cmd syntax, not a disguise.
|
|
'rmdir /s "%USERPROFILE%"',
|
|
'del /f /s /q "C:\\*"',
|
|
],
|
|
)
|
|
def test_a_recursive_windows_delete_of_a_root_is_unconditional(
|
|
self, policy: ShellCommandPolicy, command: str
|
|
) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.HARD_DENY
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# Ordinary Windows cleanup: no /s, or /s over a named subdir. The
|
|
# Unix parallel is rm -f a.txt and rm -rf build -- both go to tiers.
|
|
"del /f scratch.txt",
|
|
"del /q old.log",
|
|
"rmdir /s build",
|
|
"rmdir /s C:\\Users\\me\\project",
|
|
"del /f /q C:\\proj\\build",
|
|
# cmd in argument position is text, not a wrapper, and a quoted
|
|
# ampersand separates nothing.
|
|
"echo cmd /c rmdir /s C:",
|
|
'echo "safe & rmdir /s C: "',
|
|
# A quoted named subdir keeps its backslashes and stays ordinary.
|
|
'rmdir /s "C:\\Users\\me\\my project"',
|
|
],
|
|
)
|
|
def test_ordinary_windows_deletes_answer_to_the_tiers(self, policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# `rm` appears, but never in the command position -- reading the
|
|
# whole argv for the word would refuse all three, and a hard deny
|
|
# is the one decision no approval can rescue.
|
|
"timeout 5 grep -r rm .",
|
|
"xargs grep -r foo",
|
|
"nice -n 5 pytest -q",
|
|
],
|
|
)
|
|
def test_a_wrapper_does_not_invent_a_delete(self, policy: ShellCommandPolicy, command: str) -> None:
|
|
assert policy.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
|
|
class TestExternalEffectFamilies:
|
|
"""The opt-in group, and where its line is drawn.
|
|
|
|
The built-in policy asks about exactly one family -- deletion -- which fits a
|
|
terminal the reader is already watching. Behind an editor nothing is on
|
|
screen, so ``git push``, ``npm install`` and ``curl -o`` would run unannounced.
|
|
The group registered by ``raven acp`` closes that, and the tests below are as
|
|
much about what it does *not* ask for: a prompt on every build and every
|
|
documentation fetch trains the reader to approve without looking, which costs
|
|
more than it buys.
|
|
"""
|
|
|
|
@pytest.fixture
|
|
def asking(self) -> ShellCommandPolicy:
|
|
from raven.permissions.shell_policy import DELETE_MATCHERS, EXTERNAL_EFFECT_MATCHERS
|
|
|
|
policy = ShellCommandPolicy(deny_patterns=[])
|
|
for name, matcher in DELETE_MATCHERS:
|
|
policy.register_approval_matcher(name, matcher)
|
|
for name, matcher in EXTERNAL_EFFECT_MATCHERS:
|
|
policy.register_approval_matcher(name, matcher)
|
|
return policy
|
|
|
|
@pytest.mark.parametrize(
|
|
("command", "family"),
|
|
[
|
|
("git push origin main", "publish_command"),
|
|
("gh pr create --fill", "publish_command"),
|
|
("gh pr merge 3 --squash", "publish_command"),
|
|
("glab mr merge 5", "publish_command"),
|
|
# Naming an organisation's secrets is itself worth asking about, so
|
|
# the read-verb relief does not reach this group.
|
|
("gh secret list", "publish_command"),
|
|
# And the word after a publishing verb is usually an operand, not a
|
|
# verb: these push an image called status, a branch to a remote
|
|
# called view, a package, a deletion. The relief is scoped to the
|
|
# forge groups so it cannot reach any of them.
|
|
("git push status", "publish_command"),
|
|
("git push view", "publish_command"),
|
|
("docker push status", "publish_command"),
|
|
("npm publish status", "publish_command"),
|
|
("kubectl delete status", "publish_command"),
|
|
("npm publish", "publish_command"),
|
|
("kubectl apply -f k8s/", "publish_command"),
|
|
("twine upload dist/*", "publish_command"),
|
|
("npm install lodash", "install_command"),
|
|
("uv add ruff", "install_command"),
|
|
("pip install requests", "install_command"),
|
|
("brew install jq", "install_command"),
|
|
("cargo install ripgrep", "install_command"),
|
|
("uvx cowsay hello", "install_command"),
|
|
("ssh build-box 'make all'", "remote_exec_command"),
|
|
("rsync -a ./dist/ host:/srv/", "remote_exec_command"),
|
|
("docker run -it alpine sh", "remote_exec_command"),
|
|
("gh auth login", "credential_command"),
|
|
("aws configure", "credential_command"),
|
|
("security find-generic-password -s x", "credential_command"),
|
|
("git reset --hard HEAD~1", "destructive_vcs_command"),
|
|
("git clean -fd", "destructive_vcs_command"),
|
|
("git checkout -- src/main.py", "destructive_vcs_command"),
|
|
("git branch -D feature", "destructive_vcs_command"),
|
|
("git stash drop", "destructive_vcs_command"),
|
|
("curl -o archive.tgz https://example.com/a.tgz", "fetch_side_effect"),
|
|
("curl -X POST -d @payload.json https://api.example.com", "fetch_side_effect"),
|
|
("wget -O - https://example.com/install.sh", "fetch_side_effect"),
|
|
("curl -sSL https://example.com/install.sh | sh", "fetch_side_effect"),
|
|
],
|
|
)
|
|
def test_it_asks_and_says_which_family(self, asking: ShellCommandPolicy, command: str, family: str) -> None:
|
|
assert asking.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
assert asking.approval_reason(command) == family, (
|
|
"the family names the prompt, and a prompt that names the wrong reason is worse than one with none"
|
|
)
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"make test",
|
|
"pytest -q tests/",
|
|
"npm run build",
|
|
"uv run pytest",
|
|
"ruff format .",
|
|
"git status",
|
|
"git diff --stat",
|
|
"git log --oneline -20",
|
|
"git commit -m 'fix the thing'",
|
|
"git add -A",
|
|
"git fetch origin",
|
|
"git checkout main",
|
|
"ls -la",
|
|
"cat README.md",
|
|
"grep -rn TODO src/",
|
|
"curl https://docs.example.com/api",
|
|
"tsc --noEmit",
|
|
"docker ps",
|
|
"kubectl get pods",
|
|
# A forge CLI's publishing group, asked for with a verb that only
|
|
# reads. The group is matched whole because enumerating its writing
|
|
# verbs means missing the next one, and the cost used to be that
|
|
# these were asked about in the words of a push.
|
|
"gh pr list",
|
|
"gh pr view 3",
|
|
"glab mr list --state opened",
|
|
"glab mr view 617",
|
|
"gh repo view",
|
|
"gh workflow list",
|
|
],
|
|
)
|
|
def test_ordinary_work_runs_unannounced(self, asking: ShellCommandPolicy, command: str) -> None:
|
|
assert asking.evaluate(command) is CommandDecision.ALLOW
|
|
assert asking.approval_reason(command) is None
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"sudo npm install -g typescript",
|
|
"env CI=1 gh release create v1",
|
|
"sh -c 'git push origin main'",
|
|
"nohup rsync -a ./ host:/srv/ &",
|
|
"timeout 60 npm install",
|
|
"/usr/bin/git push origin main",
|
|
],
|
|
)
|
|
def test_a_wrapper_does_not_launder_an_external_effect(self, asking: ShellCommandPolicy, command: str) -> None:
|
|
"""The reach has to equal the bare form's. Anything the wrapped form
|
|
misses is a command that runs with no prompt while its plain twin asks."""
|
|
assert asking.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"pacman -S ripgrep",
|
|
"code --install-extension ms-python.python",
|
|
],
|
|
)
|
|
def test_an_install_verb_hidden_in_an_option_is_still_found(self, asking: ShellCommandPolicy, command: str) -> None:
|
|
"""Two package managers put the verb in a flag rather than a word. A
|
|
matcher that only read words would let them through while every other
|
|
install asked."""
|
|
assert asking.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
assert asking.approval_reason(command) == "install_command"
|
|
|
|
def test_a_git_subcommand_destructive_with_no_flag_at_all(self, asking: ShellCommandPolicy) -> None:
|
|
"""``filter-branch`` rewrites history unconditionally -- no flag makes it
|
|
safe, so the family carries no flag list for it and the bare form fires."""
|
|
assert asking.evaluate("git filter-branch --msg-filter cat") is CommandDecision.REQUIRE_APPROVAL
|
|
assert asking.approval_reason("git filter-branch --msg-filter cat") == "destructive_vcs_command"
|
|
|
|
def test_a_destructive_flag_without_its_subcommand_does_not_fire(self, asking: ShellCommandPolicy) -> None:
|
|
"""``--hard`` belongs to ``reset``. Matching the flag alone would ask about
|
|
anything else that happens to carry it."""
|
|
assert asking.evaluate("git log --hard") is CommandDecision.ALLOW
|
|
assert asking.evaluate("git status -f") is CommandDecision.ALLOW
|
|
|
|
@pytest.mark.parametrize("command", ["rm -rf /", "shutdown -h now", "reboot"])
|
|
def test_a_token_classified_hard_deny_has_no_reason_to_explain(
|
|
self, asking: ShellCommandPolicy, command: str
|
|
) -> None:
|
|
"""The other hard-deny path: these are refused by token inspection rather
|
|
than by a deny pattern, and a refusal has no prompt to describe."""
|
|
assert asking.evaluate(command) is CommandDecision.HARD_DENY
|
|
assert asking.approval_reason(command) is None
|
|
|
|
def test_an_empty_segment_does_not_break_the_walk(self, asking: ShellCommandPolicy) -> None:
|
|
"""A wrapper with nothing after it, and an empty compound segment. Both
|
|
occur in real command strings and neither names an executable."""
|
|
assert asking.evaluate("sudo") is CommandDecision.ALLOW
|
|
assert asking.evaluate("env") is CommandDecision.ALLOW
|
|
assert asking.evaluate(";; git push") is CommandDecision.REQUIRE_APPROVAL
|
|
|
|
def test_a_command_inside_a_nested_shell_is_still_classified(self, asking: ShellCommandPolicy) -> None:
|
|
assert asking.evaluate("""sh -c "sh -c 'git push'" """) is CommandDecision.REQUIRE_APPROVAL
|
|
|
|
def test_unparseable_quoting_closes_the_gate(self, asking: ShellCommandPolicy) -> None:
|
|
"""``shlex`` raises "No closing quotation" on an unbalanced quote, which
|
|
reaches the policy's fail-closed branch. Refusing is the right direction:
|
|
a command string the classifier cannot read is one whose effect it cannot
|
|
bound, and the alternative is running it unexamined."""
|
|
assert asking.evaluate("git status 'unbalanced") is CommandDecision.HARD_DENY
|
|
assert asking.approval_reason("git status 'unbalanced") is None
|
|
|
|
def test_the_walk_stops_at_a_fixed_depth(self) -> None:
|
|
"""Called directly with the depth already at the bound, because reaching
|
|
it through real shell quoting takes five alternating quote levels that no
|
|
command has. What the bound buys is termination: without it a crafted
|
|
string could recurse until the stack ran out."""
|
|
from raven.permissions.shell_policy import _MAX_EMBEDDED_SHELL_DEPTH, _iter_argv
|
|
|
|
shallow = list(_iter_argv("sh -c 'git push'"))
|
|
at_bound = list(_iter_argv("sh -c 'git push'", _depth=_MAX_EMBEDDED_SHELL_DEPTH))
|
|
|
|
assert ["git", "push"] in shallow, "the inner command is reached below the bound"
|
|
assert ["git", "push"] not in at_bound, "and not descended into at it"
|
|
assert at_bound == [["sh", "-c", "git push"]], "the outer argv is still yielded"
|
|
|
|
def test_hard_deny_still_outranks_the_new_families(
|
|
self,
|
|
) -> None:
|
|
"""Ordering is security-sensitive: a matcher must never turn an
|
|
unconditionally forbidden command into an approvable one."""
|
|
from raven.permissions.shell_policy import EXTERNAL_EFFECT_MATCHERS
|
|
|
|
policy = ShellCommandPolicy(deny_patterns=[r"\bmkfs\b"])
|
|
for name, matcher in EXTERNAL_EFFECT_MATCHERS:
|
|
policy.register_approval_matcher(name, matcher)
|
|
|
|
assert policy.evaluate("mkfs.ext4 /dev/sda1 && git push") is CommandDecision.HARD_DENY
|
|
assert policy.approval_reason("mkfs.ext4 /dev/sda1") is None, "a refusal has no prompt to explain"
|
|
assert policy.evaluate("rm -rf / && npm publish") is CommandDecision.HARD_DENY
|
|
|
|
def test_the_default_policy_asks_about_none_of_them(self, policy: ShellCommandPolicy) -> None:
|
|
"""The group is opt-in. A terminal user watching their own shell does not
|
|
need a prompt before ``git push``, and adding one would change behaviour
|
|
for every existing surface."""
|
|
for command in ("git push origin main", "npm install lodash", "ssh box ls"):
|
|
assert policy.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
def test_a_faulty_matcher_closes_the_gate_and_explains_nothing(self, policy: ShellCommandPolicy) -> None:
|
|
def _broken(command: str) -> bool:
|
|
raise RuntimeError("matcher is wrong")
|
|
|
|
policy.register_approval_matcher("broken", _broken)
|
|
|
|
assert policy.evaluate("echo hi") is CommandDecision.HARD_DENY
|
|
assert policy.approval_reason("echo hi") is None
|
|
|
|
|
|
async def test_the_prompt_names_the_family_that_fired(tmp_path) -> None:
|
|
"""The description was a constant before the families existed -- it read
|
|
"Delete files using a shell command" for whatever was being asked about,
|
|
which was accurate only while deletion was the one registered family."""
|
|
from raven.permissions.shell_policy import EXTERNAL_EFFECT_MATCHERS
|
|
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
responder = _ApprovalResponder([True])
|
|
registry = _gated_exec(executor, responder, tmp_path, families=EXTERNAL_EFFECT_MATCHERS)
|
|
|
|
await registry.execute("exec", {"command": "git push origin main"})
|
|
|
|
assert responder.requests[0]["description"] == "Publish or push work to a remote"
|
|
assert executor.commands == ["git push origin main"]
|
|
|
|
|
|
async def test_an_unregistered_family_still_gets_a_usable_prompt(tmp_path) -> None:
|
|
"""A surface can register a matcher this table has no description for. The
|
|
fallback is deliberately vague rather than a guess: naming the wrong reason
|
|
is worse than naming none."""
|
|
executor = _RecordingExecutor(sandboxed=False)
|
|
responder = _ApprovalResponder([False])
|
|
families = (("house_style", lambda command: command.startswith("weird")),)
|
|
registry = _gated_exec(executor, responder, tmp_path, families=families)
|
|
|
|
result = await registry.execute("exec", {"command": "weird --thing"})
|
|
|
|
assert responder.requests[0]["description"] == "Run a command that needs your approval"
|
|
assert "denied" in str(result).lower()
|
|
assert executor.commands == []
|
|
|
|
|
|
class TestAGlobalOptionValueIsNotASubcommand:
|
|
"""The gap that let a hand-written command through the boundary added here.
|
|
|
|
``_subcommands`` skipped options but not their values, so a value was counted
|
|
as one of the words it was looking for and the budget ran out before the
|
|
verb. ``git --git-dir X --work-tree Y push`` therefore read as the two paths
|
|
and never saw ``push``: an ALLOW for the exact command that bare ``git push``
|
|
prompts about. Nothing below is adversarial -- every shape is one a person
|
|
types, and two of them (``aws --profile``, ``git --git-dir``) are the normal
|
|
way to drive those tools from outside their own tree.
|
|
"""
|
|
|
|
@pytest.fixture
|
|
def asking(self) -> ShellCommandPolicy:
|
|
from raven.permissions.shell_policy import DELETE_MATCHERS, EXTERNAL_EFFECT_MATCHERS
|
|
|
|
policy = ShellCommandPolicy(deny_patterns=[])
|
|
for name, matcher in DELETE_MATCHERS:
|
|
policy.register_approval_matcher(name, matcher)
|
|
for name, matcher in EXTERNAL_EFFECT_MATCHERS:
|
|
policy.register_approval_matcher(name, matcher)
|
|
return policy
|
|
|
|
@pytest.mark.parametrize(
|
|
("command", "family"),
|
|
[
|
|
("git --git-dir /tmp/repo/.git --work-tree /tmp/repo push origin main", "publish_command"),
|
|
("git -C /repo --no-pager push", "publish_command"),
|
|
("aws --profile prod --region us-east-1 s3 cp ./x s3://bucket/x", "publish_command"),
|
|
("kubectl --namespace kube-system --context prod apply -f x.yaml", "publish_command"),
|
|
("gh --repo owner/name pr create --fill", "publish_command"),
|
|
("docker --host tcp://build:2375 push registry/image", "publish_command"),
|
|
("npm --prefix /srv/app install lodash", "install_command"),
|
|
("git --git-dir /tmp/r/.git reset --hard HEAD~1", "destructive_vcs_command"),
|
|
],
|
|
)
|
|
def test_the_verb_is_found_past_its_global_options(
|
|
self, asking: ShellCommandPolicy, command: str, family: str
|
|
) -> None:
|
|
assert asking.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
assert asking.approval_reason(command) == family
|
|
|
|
def test_an_attached_value_consumes_nothing_extra(self, asking: ShellCommandPolicy) -> None:
|
|
"""``--git-dir=X`` carries its value in the same word. Consuming a
|
|
following token for it would eat the verb instead of the value, which is
|
|
the same bug pointing the other way."""
|
|
assert asking.approval_reason("git --git-dir=/tmp/r/.git push") == "publish_command"
|
|
|
|
def test_a_double_dash_ends_the_options(self, asking: ShellCommandPolicy) -> None:
|
|
"""After ``--`` a word that looks like an option is an argument, so the
|
|
table must not keep consuming past it."""
|
|
assert asking.approval_reason("git -C /repo push -- --not-an-option") == "publish_command"
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# The two options here are documented AWS globals that a table of
|
|
# "options certain to take a value" did not happen to list. Review
|
|
# found this shape against the table version, and it is the reason
|
|
# the word budget is gone rather than the table extended: no table of
|
|
# every option of every tool can be complete, so nothing that decides
|
|
# whether to ask may depend on one being complete.
|
|
"aws --query '{}' --cli-binary-format raw-in-base64-out s3 cp ./x s3://bucket/x",
|
|
"aws --profile p --region r --output json --query x s3 cp a b",
|
|
"kubectl -n ns --context c --kubeconfig k --as u apply -f x.yaml",
|
|
"git -c a.b=c -c d.e=f --git-dir /r/.git --work-tree /r push origin main",
|
|
],
|
|
)
|
|
def test_no_number_of_option_values_can_hide_the_verb(self, asking: ShellCommandPolicy, command: str) -> None:
|
|
assert asking.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
assert asking.approval_reason(command) is not None
|
|
|
|
def test_a_quoted_argument_of_metacharacters_is_not_a_command_boundary(self, asking: ShellCommandPolicy) -> None:
|
|
"""Found while fixing the case above, and it is the more serious half.
|
|
|
|
The segmenter decided a token was a boundary when every character in it
|
|
was a shell metacharacter, which a *quoted* argument can satisfy.
|
|
``aws --query '{}' ... s3 cp`` was therefore cut in two at its own
|
|
argument, and the second piece began with an option -- so ``argv[0]``,
|
|
which every family matcher keys on to find its table, was an option
|
|
rather than an executable, and no family could fire at all. Extending the
|
|
option table would not have touched this: the command never reached the
|
|
table as one command.
|
|
"""
|
|
from raven.permissions.shell_policy import _iter_argv
|
|
|
|
argvs = list(_iter_argv("aws --query '{}' --cli-binary-format raw s3 cp ./x s3://b/x"))
|
|
|
|
assert len(argvs) == 1, f"one command, not {len(argvs)}: {argvs}"
|
|
assert argvs[0][0] == "aws", "the executable has to survive segmentation"
|
|
|
|
@pytest.mark.parametrize(
|
|
("command", "decision"),
|
|
[
|
|
# The shapes that rely on a bare ``{}`` being an ordinary word, and
|
|
# the real operators that must still split. Both directions, because
|
|
# the fix moves the line between them.
|
|
("{ rm file.txt; }", CommandDecision.REQUIRE_APPROVAL),
|
|
("xargs -I{} rm -rf {}", CommandDecision.REQUIRE_APPROVAL),
|
|
(r'find . -name "*.log" -exec rm {} \;', CommandDecision.REQUIRE_APPROVAL),
|
|
("rm -rf / && git push", CommandDecision.HARD_DENY),
|
|
],
|
|
)
|
|
def test_the_operators_that_must_still_split_still_split(
|
|
self, asking: ShellCommandPolicy, command: str, decision: CommandDecision
|
|
) -> None:
|
|
assert asking.evaluate(command) is decision
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# A repository directory literally named ``|``. Review initialized one
|
|
# and confirmed ``git -C '|' status`` works, so this is a command a
|
|
# person can run, not a contrivance. Under the whole-token rule this
|
|
# segmented as [['git', '-C'], ['push', 'origin', 'main']]: the quoted
|
|
# argument read as a pipeline operator, and the piece holding ``push``
|
|
# started at a word that is not an executable.
|
|
"git -C '|' push origin main",
|
|
# zsh accepts ``|&`` and runs the right-hand side as the pipeline. It
|
|
# was on no operator list, so the whole thing stayed one token inside
|
|
# the ``-c`` string and nothing looked past it.
|
|
"zsh -c 'echo ignored |& git push origin main'",
|
|
"git -C '&&' push",
|
|
"sh -c 'true; git push origin main'",
|
|
],
|
|
)
|
|
def test_a_quoted_operator_is_an_argument_and_a_real_one_is_a_boundary(
|
|
self, asking: ShellCommandPolicy, command: str
|
|
) -> None:
|
|
"""The reason segmentation now reads the raw text instead of tokens.
|
|
|
|
``shlex`` strips quote provenance, so once ``git -C '|' push`` has been
|
|
tokenised there is no information left that distinguishes the argument
|
|
from the operator. No rule written over tokens can tell them apart; the
|
|
fix had to move to where the quoting is still visible.
|
|
"""
|
|
assert asking.evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
assert asking.approval_reason(command) == "publish_command"
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
["echo a|grep b", "make test && echo done", "echo '|' ", "echo 'a && b'"],
|
|
)
|
|
def test_the_quoting_rules_do_not_invent_boundaries_or_lose_them(
|
|
self, asking: ShellCommandPolicy, command: str
|
|
) -> None:
|
|
"""Both directions of the same scanner: a real operator still splits, and
|
|
a quoted one still does not, without either turning ordinary work into a
|
|
prompt."""
|
|
assert asking.evaluate(command) is CommandDecision.ALLOW
|
|
|
|
def test_an_unknown_option_over_reads_rather_than_under_reads(self, asking: ShellCommandPolicy) -> None:
|
|
"""No table lists every option of every tool. An unconsumed value becomes
|
|
a candidate word, which can only make the policy ask about more than it
|
|
must -- the direction a security boundary is allowed to fail in. This one
|
|
passes before the fix too; it is here to pin the fallback, because the
|
|
obvious "consume the next token after any option" would break it."""
|
|
assert asking.evaluate("git --future-flag somevalue push") is CommandDecision.REQUIRE_APPROVAL
|
|
|
|
|
|
class TestSandboxingDoesNotRelaxClassification:
|
|
"""The classifier has no sandbox input: the Boxlite VM mounts the real
|
|
workspace read-write, so the filesystem rules a sandbox supposedly
|
|
contains still reach host data through that mount."""
|
|
|
|
def _asking(self) -> ShellCommandPolicy:
|
|
from raven.permissions.shell_policy import EXTERNAL_EFFECT_MATCHERS
|
|
|
|
policy = ShellCommandPolicy(deny_patterns=[r"\bmkfs\b"])
|
|
for name, matcher in EXTERNAL_EFFECT_MATCHERS:
|
|
policy.register_approval_matcher(name, matcher)
|
|
return policy
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
["git push origin main", "npm install lodash", "ssh host ls", "curl -o out https://example.com"],
|
|
)
|
|
def test_external_effects_ask(self, command: str) -> None:
|
|
assert self._asking().evaluate(command) is CommandDecision.REQUIRE_APPROVAL
|
|
assert self._asking().approval_reason(command) is not None
|
|
|
|
@pytest.mark.parametrize("command", ["rm -rf /", "shutdown now", "mkfs.ext4 /dev/sda1"])
|
|
def test_the_deny_list_holds(self, command: str) -> None:
|
|
assert self._asking().evaluate(command) is CommandDecision.HARD_DENY
|
|
|
|
|
|
def test_the_exec_prompt_shows_the_command_where_it_would_run(tmp_path) -> None:
|
|
tool = ExecTool(working_dir=str(tmp_path))
|
|
|
|
assert tool.approval_kind == "shell.exec"
|
|
assert tool.approval_evidence({"command": "rm a"}) == {"command": "rm a", "cwd": str(tmp_path)}
|
|
assert tool.approval_evidence({"command": "rm a", "working_dir": "/srv"}) == {"command": "rm a", "cwd": "/srv"}
|
|
assert tool.approval_evidence({"command": "rm a", "machine": "prod"}) == {"command": "rm a", "machine": "prod"}
|
|
|
|
|
|
def test_the_listing_root_is_where_the_command_would_run(tmp_path) -> None:
|
|
"""The directory listed around a command has to be the one the command ran
|
|
in, or the listing describes a tree the command never touched: a per-call
|
|
``working_dir`` moves it, and a registered machine takes it off this disk."""
|
|
tool = ExecTool(working_dir=str(tmp_path))
|
|
|
|
assert tool.listing_root({"command": "make"}) == tmp_path
|
|
assert tool.listing_root({"command": "make", "working_dir": "/srv"}) == Path("/srv")
|
|
assert tool.listing_root({"command": "make", "machine": "prod"}) is None
|