2 Commits

Author SHA1 Message Date
f74a5ee73b feat(ledger-core): add Prometheus counter for audit-mirror failures
Closes docs/THREAT-MODEL.md Known-gaps #3's named TODO: ledger-core had
no Prometheus endpoint at all, so a sustained audit-log outage was
only a tracing::warn! log line, nothing alertable. Adds
ledger_core_audit_mirror_spawn_append_failures_total{kind}, served on
GET /metrics at LEDGER_CORE_METRICS_ADDR (default localhost:9099),
pre-registered at zero per kind so absence vs zero is distinguishable.
Wires the new endpoint into docker-compose.yml and the Prometheus
scrape config, which already named this exact gap as the trigger to
add the target. Fixes a render() visibility bug and a test assumption
that unobserved CounterVec label values still emit TYPE/HELP lines,
both left broken by an earlier interrupted pass on this same change.
2026-10-23 07:35:50 +03:00
13abc31645 feat(ration): observability, infra-as-code, security hardening, and test depth
Phase 2 sections 2-6 and 8 of docs/SCALE-EXPANSION.md:

- pkg/telemetry + pkg/metrics: OpenTelemetry trace propagation and
  Prometheus metrics across every service hop (CLI -> agent -> gateway ->
  card-issuance/tenant-registry/auth -> ledger-core), tracing added to
  ledger-core itself via services/ledger-core/src/telemetry.rs.
- infra/: otel-collector + Prometheus + Grafana docker-compose stack, a
  Terraform ECS Fargate module + dev/staging/prod envs (plan-only, no
  apply — real AWS spend is an owner action), and a dev-only mTLS CA
  (infra/dev-ca) wired through pkg/mtls + services/ledger-core/src/mtls.rs.
- services/audit-log (from the prior commit) gets its async mirror writer
  in services/ledger-core/src/audit_mirror.rs, and services/gateway gets
  a per-tenant token-bucket rate limiter (internal/ratelimit).
- Test depth: a Rust proptest suite proving concurrent ApplyEntry never
  breaches the reserve floor (tests/apply_entry_proptest.rs, passing), a Go
  fuzz seed corpus for the policy envelope (rules_fuzz_test.go), a libFuzzer
  harness for the C WAL frame parser (services/wal/fuzz, deliberately not
  wired into `make test` per its own Makefile comment - needs a
  libFuzzer-bundled clang), and a real chaos test
  (scripts/chaos/kill-ledger-mid-write.sh) that SIGKILLs ledger-core
  mid-write under concurrent load and verifies via ListEntries that
  nothing was lost or double-applied.
- docs/formal/ledger.tla: a TLA+ spec of the ledger state machine (balance
  >= reserve_floor safety, HOLD-eventually-resolves liveness) plus a TLC
  model config - not run through TLC in this pass, that's a precise
  documented TODO, not faked.
- docs/adr/0001-0004, docs/RUNBOOK.md, docs/SLO.md, docs/THREAT-MODEL.md.

Verified: `make build && make test` clean (C WAL, Rust incl. the new
proptest suite, all 10 Go modules) from a cold target.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 12:25:58 +03:00