caelum0x c1bd918882 feat(otsone): add eval-external for third-party typed-decision models
New `otsone eval-external` CLI command and laya_mlx.eval_external module let
any third party point OpenJev's existing gold-case calibration report
(accuracy, ECE/adaptive-ECE/MCE/Brier/NLL, per-type reliability) at their OWN
typed-decision model or API, not just OpenJev's own ONNX checkpoints -- the
real gap between an internal eval script and a benchmarking platform for AI
agents (targets the Geek Room Global AI Hackathon's published theme of the
same name). HttpEngine posts {state, questions} to a live endpoint;
AnswersFileEngine replays precomputed predictions offline. Zero changes to
the underlying eval_gold/calibration scoring math. 22 new tests, all passing;
zero regressions in the existing otsone suite (216 tests).
2026-11-07 03:58:50 +03:00
..