lawsynth/specs/parallel-evaluation
caelum0x fab9a4a45a feat: reach + performance + docs — Python analysis client, parallel eval, sensitivity, guide
Batch 7 of expansion-v2. Extends the product across surfaces and adds an
analysis primitive. Deterministic/offline/std-only (Rust) or graceful-absence
(Python).

python/lawsynth analysis.py (new): Python client for the engine's stability,
control, and domains analyses — shells out to the lawsynth CLI (--json) and
parses to typed dataclasses. No numerics re-implemented in Python; the Rust
engine is the single source of truth. Locates the binary like the benchmarks,
raises MissingBinaryError/CliError honestly, and is fully parse-testable
offline. 20 tests; full SDK suite 114 passed.

crates/lawsynth-features: opt-in std::thread-parallel feature-library
evaluation, bit-identical to serial for ALL thread counts (row-chunked via
thread::scope, ordered assembly, no parallel float reduction). Verified via
f64::to_bits across {1,2,3,4,7,8} threads x 10 library variants x uneven row
counts. Measured 3.6x at 8 threads; honestly sublinear (allocation-bound,
~few-thousand-row crossover). 28 tests + benchmark.

crates/lawsynth-sensitivity (new): forward sensitivity analysis. Integrates the
variational equations Ṡ_j = J_x·S_j + f_θj (J_x from the analytic Jacobian, f_θ
by symbolic differentiation) on the augmented system with fixed-step RK4.
Analytic check to <1e-6; finite-difference cross-check at 1e-6 for every
parameter on logistic + Lotka-Volterra. 22 tests.

docs/guide (new): verified end-to-end guide — getting-started, workflow
(discover→simplify→stability→control→export→validate→uncertainty), and a
determinism demo. All 17 documented commands run exit-0 via run_all.sh; honest
about real behaviors (already-minimal law simplifies to 0 nodes saved, CLI
bootstrap doesn't surface per-coef intervals, center points are inconclusive).
README links the guide.

specs/{parallel-evaluation,sensitivity-analysis}: boundary specs.
2026-08-18 16:14:46 +03:00
..

Deterministic parallel feature-library evaluation (v1)

This directory specifies the opt-in, multi-threaded evaluation path for feature-library materialization implemented in crates/lawsynth-features (FeatureLibrary::evaluate_parallel). It is a boundary specification in the house style: it states what a conforming implementation MUST do.

Motivation

Building the candidate matrix Θ(X) from a dataset is the hot inner loop of discovery. Each row of Θ is the candidate terms evaluated at one sample, and rows are independent of one another. The serial FeatureLibrary::evaluate walks rows in order on a single thread; on large datasets and rich libraries this is embarrassingly parallel work left on the table.

The parallel path spreads that per-row work across OS threads. Its non-negotiable requirement is bit-identity: the parallel matrix MUST equal the serial matrix to the last bit, for every thread count. Discovery downstream (sparse regression, symbolic search) is deterministic and content-addressed; a parallel path that changed a single low bit would break reproducibility. Speed is a bonus, determinism is the contract.

Requirements

  1. Parallelism is ONLY over independent rows. A conforming parallel evaluation MUST split the row range 0..rows into contiguous chunks and evaluate each chunk on its own worker. It MUST NOT introduce any cross-row reduction, atomics-based accumulation, or float summation whose order depends on threading. The per-row kernel — environment construction followed by term-by-term evaluation — MUST be the exact same code, in the exact same operation order, as the serial path. In the reference implementation the serial and parallel paths call one shared evaluate_rows kernel precisely so they cannot drift apart.

  2. Assembly is ordered. Chunk outputs MUST be concatenated in ascending row order, independent of the order in which workers finish. Because each row runs identical float operations regardless of which thread computes it, and the rows are reassembled in their original order, the concatenated matrix is bit-identical to serial. This is the whole correctness argument: identical per-row ops + ordered assembly ⇒ identical bits.

  3. Deterministic chunk boundaries. The split MUST depend only on rows and the requested thread count, never on scheduling. The reference rule (row_partitions): the chunk count is clamp(threads, 1, rows), the base chunk size is rows / chunks, and the first rows % chunks chunks each take one extra row. The ranges tile 0..rows exactly — contiguous, ordered, no gaps, no overlaps, no empty chunks.

  4. Serial fallback with no threads spawned. threads == 0 or threads == 1, a dataset of 0 or 1 rows, or any partition that degenerates to a single chunk MUST run the serial path directly, spawning no threads. threads MUST be capped at the row count so no worker receives an empty range.

  5. Result is invariant to thread count. The number of threads MUST affect speed ONLY, never the result. Identical inputs MUST yield bit-identical output for every thread count k — including k larger than the row count and k that does not divide the rows evenly (an uneven final chunk). This is the headline test: evaluate_parallel(.., k) compared to evaluate(..) via f64::to_bits on every entry, for several libraries and dataset sizes.

  6. Offline and std-only. The path MUST be deterministic and offline (net.offline = true). It MUST use only the standard library (std::thread::scope, std::sync) — NO external crates (no rayon). Scoped threads borrow the dataset and library directly; no Arc cloning of data is required.

  7. Honest limits. Any speedup claim MUST be backed by a reproducible measurement (the parallel_eval bench times serial vs 1/2/4/8 threads on a large synthetic dataset and asserts each parallel result is bit-identical to serial before reporting). Speedup is bounded by available cores and, for cheap per-term work, by memory bandwidth. Small inputs do NOT benefit: thread spawn and join overhead dominates below a crossover row count, and the serial fallback covers the trivial cases. No parallel float reduction is used — precisely to preserve bit-identity — so there is no reduction-tree speedup to claim, only the linear-ish row fan-out bounded by core count.

Non-goals

Per-term (intra-row) parallelism, a global thread pool, SIMD vectorization, and work-stealing schedulers are out of scope for this boundary. They may be added as extensions with their own contracts, provided they preserve the bit-identity guarantee above.