Builds the execution half of the Geek Room "Build a Benchmarking Platform
for AI Agents, Not Just Models" theme: every submission now carries a real
execution trace (steps, outbound/tool-call count, latency p50/p95, tokens +
cost when the endpoint reports them, error/timeout counts) and an opt-in,
deterministically seeded fault-injection stress test (injected timeouts,
5xx responses, malformed answers at configurable rates) that reports
robustness: accuracy under faults vs. the clean run.
- benchmark-scoring.ts: instrumentRunner wraps any Runner with tracing;
endpointRunner gains an optional onOutcome hook + parses real
usage/cost_usd from the submitter's response body; mulberry32-seeded
fault injection with disjoint timeout/server_error/malformed buckets.
- benchmark-submissions.server.ts: wires tracing into both runner kinds,
runs a second faulted pass when requested, persists {clean trace,
robustness} + the fault-injection config on two new additive, nullable
columns (migrations/0012_benchmark_execution_trace.sql).
- API route + leaderboard feed: pass faultInjection through, return
trace/robustness/faultInjection on every submission.
- UI: /submit-model gets a "Stress-test with fault injection" checkbox and
Execution trace / Robustness under faults cards; /leaderboard's gold-suite
tab gets p50 ms / Outbound calls / Errors / Robustness columns for
community submissions.
Verified end to end: a real HTTP submission against a running dev server
(fault injection on, real trace + robustness persisted and read back from
the public leaderboard feed) and endpointRunner against a real local HTTP
agent (93 real round trips; real measured latency/tokens/cost; a 60%-fault
pass provably short-circuiting before the real network call) — see
hackathons/geekroom-2026/verify-execution-trace.mjs and
HACKATHON-SUBMISSION.md's "Real execution-trace + fault-injection run".
Re-recorded demo.mp4/poster.png (real Playwright capture, real ffmpeg
conversion) to show the new flow. npm test (138/1 skip/0 fail), npm run
gold (61/0), npm run typecheck (clean) all unchanged.