Handsome-wzw 7376de6828
feat(*): a machines hub tab, and registering a machine with a password Raven keeps (#1005)
## Summary

Add a Machines tab to the Agent Hub and let owners register an SSH
machine with either a key or a password.

- The Machines tab reuses the existing machine cards, details and
registration form. The hub remembers the last selected module. Machine
details show the login method, and removal explains that the locally
stored password is deleted.
- Password registrations keep a 0600 credential file under the Raven
home's 0700 keys directory. Registration, node installation, file tools,
tunnels and Oncall use the shared SSH askpass transport. Passwords are
excluded from command lines, registry rows and responses, and included
in held-secret scrubbing.
- Password connections authenticate independently, disable inherited
jump hosts, and answer only the registered account's password prompts.
Oncall cannot redirect a stored password to another endpoint. Windows
rejects this mode before creating credentials because its OpenSSH cannot
execute Raven's POSIX helper. Key authentication remains supported as
before.
- Concurrent registrations preserve the successful request's password.
Each probe uses its own temporary 0600 credential file, which is
scrubbed and removed afterwards. Password writes receive a file
identity; cleanup checks ownership under the writer's lock. Successful
registration checks duplicates again and publishes its password with the
registry row under that lock. Both completion orders, RPC cancellation
and CLI aborts are covered.

The RPC contract adds optional `MachineRow.auth` and password
registration parameters. English and Chinese machine documentation, the
runtime context and changelog describe the resulting behavior.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [x] User-facing docs or screenshots are updated when needed

After rebasing onto the latest main:

```bash
uv run --no-sync pytest tests/test_ops_connection_add.py tests/test_ops_connections.py tests/test_ops_transport.py tests/test_cli_ops_connection_commands.py tests/test_rpc_machines.py tests/test_held_secrets_machine_passwords.py tests/test_acp_remote.py tests/test_node_client.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_import_cycle_budget.py -q -n 0 --tb=short --cov=raven.ops.connection_add --cov=raven.ops.transport --cov=raven.cli.ops_connection_commands --cov=raven.rpc.methods.machines --cov-report=json:/tmp/pr1005-fix-coverage.json
```

Result: 932 passed, 6 skipped. The skipped cases require zsh, tcsh or
csh, which are absent on this Linux host.

- `uv run --no-sync python scripts/coverage_gate.py --coverage-json
/tmp/pr1005-fix-coverage.json diff --base-ref HEAD^ --threshold 99`:
100% (94/94 executable changed lines) for the concurrency fix.
- `uv run --no-sync pre-commit run --from-ref HEAD^ --to-ref HEAD`:
passed, including formatting, file checks and private-key detection.
- `uv run --no-sync ty check raven/ops/connection_add.py
raven/ops/transport.py raven/cli/ops_connection_commands.py
raven/rpc/methods/machines.py`: passed.
- `uv run --no-sync lint-imports`: all 10 contracts passed.
- `uv run --no-sync python scripts/check_source_language.py
origin/main..HEAD`, `uv run --no-sync python
scripts/check_commit_messages.py origin/main..HEAD`, `make
check-large-files`, and `git diff --check origin/main...HEAD`: passed.

The original RPC failure/cancellation and CLI-abort reproductions failed
before the fix. Additional regressions cover the reverse completion
order, probes with competing passwords, temporary-secret scrubbing,
duplicate registration and failed registry writes. The complete rebased
source was compared with the original PR plus main, and the concurrency
fix retained its patch identity.

## Risk

- [x] Security impact considered: stored and temporary passwords are
owner-readable and scrubbed from tool output. Unknown SSH host keys are
accepted on the first connection, as before; the guides explain
verifying the host from a terminal first on an untrusted network.
Passwords shorter than six characters or containing CR/LF are rejected.
- [x] Backward compatibility considered: key registration remains the
default and `MachineRow.auth` is optional. Password publication and
cleanup now use file locks and atomic replacement; temporary probe files
are removed when the probe ends.
- [x] Rollback path is clear: revert this PR to restore the earlier
registration flow. Removing a machine deletes the password stored under
its id. Existing password rows require migration to key authentication
if reverting the feature.

## Related Issues

N/A

---------

Co-authored-by: Claude (claude-fable-5-1) <noreply@anthropic.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: MiniMax (minimax-m3.1-flash-free) <noreply@minimax.io>
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 12:23:40 +08:00

9.3 KiB

Manage long-running work with Oncall

Model training, solver runs, and parameter sweeps often need repeated status checks after submission, decisions about whether to continue or adjust the work, and a final report backed by measured results.

This guide shows how to use Raven-Oncall to manage that workflow within a declared budget and allowed actions: prepare a machine, delegate the task, follow its progress, stop when needed, and verify the outcome. Start with the bounded local-job example below before entrusting a longer run.

The initial delegation returns after the first turn, not when the job finishes. A DAG node returning from Oncall is therefore not proof that a remote artifact or final report is ready.

When to use Oncall

When a job needs multiple rounds of observation or adjustment, Raven-Oncall can execute it, inspect outputs, decide the next step within the declared budget and allowed actions, and report the measured outcome.

A campaign records that goal, target machine, execution setup, baseline, budget, trials, and follow-up state. The agent's scheduler can arrange another look; a scheduled look is not the job itself.

Prepare the agent and machine

Check Raven-Oncall through Agent Integrations, including its model credentials. Keep the host and agent runtime/scheduler available for follow-up work. Closing a browser is different from stopping the resident runtime.

The host manages a machine registry:

raven ops connection list
raven ops connection add
raven ops connection doctor

add is interactive and contacts the selected local/SSH machine by default. It records its identity, access parameters, installed software, working paths, budget unit, and concurrency. A non-interactive local example, after replacing the executable and project paths:

raven ops connection add --id local-lab --name "Local lab" \
  --transport local --software "Python at /absolute/path/to/python" \
  --budget-unit minute --concurrency 1 --path /absolute/path/to/project \
  --non-interactive

The registry is the path selected by RAVEN_CONNECTIONS. Otherwise a sub-agent reads the one in the raven home (RAVEN_HOME, or ~/.raven) that the host hands it, and any other instance -- the host itself, including one started with --config -- reads the connections.json beside its config when there is one, else the home. A first registration lands in the home. Sub-agents run on a rendered config in a state directory of their own and inherit the home, so they read the owner's registry rather than a copy of it. SSH registration can also write a managed alias in ~/.ssh/config; the key path is recorded, not the private key bytes. A machine registered with a password (Working on registered machines) runs campaigns too: their ssh and their rsync log in with the password Raven keeps for it, through the same options and askpass hook as every other connection, and a campaign carries the connection id and auth, never the password. The address and account are the registry's: a campaign meta that gives such a connection another host, port or user is refused before any ssh starts, since the kept password goes only to the machine it was registered for. Do not put keys or passwords in a task message.

An agent can register a machine too. The ops_connection_add tool (on in raven-oncall and raven-code through tools.connectionAdd; off elsewhere) takes what the owner said in conversation -- this computer or another, what they call it, an address if another -- fills a port, user or key they left out from their own ssh config, reaches the machine, and only then writes the row. When the registry lists no machine that fits, the agent asks the owner rather than looking for a way in with a raw ssh command. It never asks for a password: a machine that takes one is the owner's to register, from the Agent Hub's Machines tab or with raven ops connection add --auth password.

doctor validates registry usability; it is not a fresh execution test of every remote environment. --skip-probe on add skips contacting the machine and explicitly leaves connectivity unverified. The Oncall agent, not the host's DAG registry, decides how to run a campaign on that machine.

Give a bounded work order

Start with a disposable local job and provide the following information:

Field Example
Machine and directory Registered “Local lab”; a scratch project directory
Goal Run the supplied benchmark and report whether the target was reached
Command and environment Exact existing script, interpreter, and required inputs
Baseline and metric Previous measured value, units, and whether higher/lower is better
Resource budget Total machine minutes and allowed concurrency
Allowed actions Read outputs and retry only the listed parameter changes
Stop/escalation conditions Stop on budget exhaustion; ask before changing dependencies
Delivery Return evidence, output locations, and remaining uncertainty

Example request:

Delegate this to Raven-Oncall on the registered Local lab machine. Use the scratch project and existing smoke script I attached. Inspect and declare the setup before submitting. Allow at most ten machine minutes and one job at a time. Do not install packages or alter the baseline. Watch the result, stop if the budget is exhausted, and report the measured outcome and logs.

These are user requirements to translate into a campaign, not literal CLI flags. Inspect the declared setup: a budget mentioned only in prose is not evidence that a configured meter will enforce it.

What the agent does

Declare goal/setup -> submit or observe -> inspect status and outputs
    -> schedule another look / adjust allowed parameters / ask the owner
    -> stop or finish -> deliver the evidence

Relevant model tools inside Raven-Oncall, not host shell subcommands:

Tool Role
ops_declare Record the campaign's target, setup, baseline, and budget
ops_submit Submit a trial under the declared setup and checks
ops_tune_status / ops_outputs Inspect campaign state and output evidence
ops_check_later Schedule the next observation
ops_campaigns / ops_note Find campaigns and record decisions
ops_ask_owner Evaluate a request for human intervention
ops_kill / ops_finish Stop work or close the campaign with a report

The next-look key is the campaign: a new scheduled look replaces the pending one rather than adding unlimited polling jobs. If scheduling is unavailable or has no usable route, the tool reports that manual checking is needed. Do not interpret that notice as a successful unattended watch.

Budgets and approval boundaries

Campaigns can meter compute, wall-clock watch time, or observation count. Units and concurrency matter: ten GPU-minutes is not necessarily ten elapsed minutes. A campaign with no declared budget has no total-budget stop. These meters do not cap model-provider bills or every external cloud charge.

Oncall's declaration, execution checks, and escalation policies are distinct from host tool permissions. In particular, an allowed ops_ask_owner returns text for the model to deliver using messaging; it does not prove the owner received it. General messaging remains a separate tool.

The shipped Oncall config does not enable a sandbox by default and does not confine all execution to the project directory. Use scoped machine accounts, scratch paths, and an isolated environment before enabling unattended writes. Host approval does not imply a separate human review of every child command.

Follow up, stop, and verify

Use instance collaboration to ask the same agent for campaign status, the pending observation, and the latest output evidence. When requesting a stop, name the campaign and ask it to check running jobs, cancel further observation, and report what actually stopped.

Closing a UI, canceling a host turn, or forgetting an instance is not a remote job kill. After a runtime interruption, inspect the saved ledger, remote processes, and scheduling route before resubmitting. Persisted campaign files do not guarantee every transport automatically resumes its watch.

Accept a final report only when it names the goal, baseline where applicable, actual measurements, termination reason, and retrievable output locations. A “done” message without evidence is not a benchmark result.

Troubleshooting

Symptom Check
Agent cannot select a machine Registry id/name, access fields, and software paths
Submission refused Declared setup, resource availability, budget, and immutable baseline checks
First reply arrives but the job continues Expected first-turn behavior; inspect the campaign
No later update Resident runtime, scheduler, wake route, and delivery channel
Budget seems different from elapsed time Meter, unit, concurrency, and recorded consumption
Restart left uncertain state Read ledger and remote status before retrying or killing anything

Implementation evidence lives in agents/raven-oncall/plugins/oncall-flow/, raven/ops/connections.py, raven/ops/connection_add.py, raven/agent/tools/connection_add.py, and raven/cli/ops_connection_commands.py.