Performance notes

Phase 4 — compute offload

Every number below is measured by benchmarks/offload_ceiling.py (48 pure-Python CPU payloads of ~15 ms, 6 overlapping per busy window, full sim.run() wall clock including lazy pool startup, vs the inline sequential reference). Machine: Apple M5 (4P+6E), macOS, CPython 3.14.2t / 3.14.4. Correctness is unconditional — strict-mode traces are bitwise-equal to the inline reference on every backend and worker count, under randomized worker latency, gated in CI; the numbers here are the honest performance record.

The max-vs-sum ceiling

With strict completion slots, a busy window costs the max of its overlapping offloads on a big-enough pool versus their sum inline — so the best possible speedup is the number of offloads that overlap in simulated time, never the worker count. A model that offloads serially (next submitted after the previous resolves) has a ceiling of 1×.

Free-threaded build (3.14.2t), 6-way overlap:

Workers	threads	processes
2	2.08×	1.66×
4	2.69×	2.52×
8	2.74×	2.77×

GIL build (3.14.4):

Workers	threads	processes
2	1.01×	1.66×
4	1.00×	2.31×
8	0.99×	2.51×

No-overlap regime (one offload in flight at a time, 2 workers): threads 0.99–1.03×, processes 1.07–1.09× — pure dispatch overhead, no possible win.

Why the thread column scales here but anti-scales for replications: the benchmark payload is a locally-bound arithmetic loop, exactly the shape the Phase 2 probe measured at 3.54× (below). An offload payload that leans on random.py globals or other shared module-level objects will hit the same refcount contention as the replication engine loop — thread-backend offload scaling is payload-dependent on 3.14.2t, while backend="processes" scales regardless of payload shape. On GIL builds, thread offload of pure-Python payloads is flat 1.0× by construction (the GIL serializes them); it only pays off for payloads that release the GIL (NumPy, C extensions).

Slowdown regimes (offload-specific)

• No overlap in simulated time — the regime above: every offload pays dispatch and transport for zero concurrency. Offload only work that overlaps other offloads (or would, on a pool).
• Cheap payloads — a payload cheaper than its dispatch (pickle + IPC on transport backends, ~1 ms scale) makes the ratio worse; the benchmark's ~15 ms payloads are near the practical floor for the process backend.
• Pure-Python payloads on threads — flat on GIL builds; contention-bound on 3.14.2t unless the payload avoids shared-object hot paths (see above).
• A payload that never returns blocks forever — Future.cancel() cannot interrupt running work on any executor, so a hung payload blocks its strict slot (and OffloadPool.close) indefinitely. There is no offload timeout in 1.0; keep payloads finite.
• Nested pools — inside an Experiment worker, backend="auto" resolves to inline (nproc × nproc oversubscription never pays); pooled offload inside workers is explicit opt-in. Spike-verified on 3.14.2: threads and interpreters nest anywhere, processes nest inside thread and process workers, and process pools inside subinterpreter workers are broken upstream (BrokenProcessPool) — llmsim rejects that combination at construction with an actionable error.

CI gate

Correctness gates are strict (conformance, adversarial-tie equivalence, and jitter suites in tests/). The only enforced timing assertion (benchmarks/test_offload_ceiling.py) is a catastrophic-regression floor: 6-way-overlap speedup at 2 process workers must stay ≥1.25× inline (measured 1.66× on both builds; an overlap-machinery regression reads ~1.0×), skipped on shared macOS runners per the Phase 2 noise findings.

Real-time mode (4.2): pacing overhead and drift regimes

rt.run adds ~0.1 µs per event over sim.run when no sleeping is needed (measured: 10k zero-work events, 4.0 ms paced vs 3.0 ms unpaced, Apple M5, 3.14.2t) — pacing itself is never the bottleneck. What actually causes drift, in decreasing order of likelihood:

• Event density above the sleep-granularity floor. time.sleep(1 ms) overshoots by ~260 µs (median) on macOS; when consecutive events are closer together in wall terms (inter-event sim time × factor) than roughly the platform's sleep overshoot, each step accumulates lateness. Keep factor × mean event spacing comfortably above ~1 ms of wall time, or run strict=False and let bursts hurry.
• Overlong offload payloads. A strict slot blocks wall-clock until its payload finishes; a payload that outlives its slot's real-time budget (delay × factor) converts directly into drift and, under strict=True, a RealtimeDriftError at the next event. The synergy only pays when the payload fits the budget — measure the payload, then pick the slot.
• Interpreter pauses. Free-threaded GC is stop-the-world; a collection or a cold import mid-run reads as drift. The one-factor slack absorbs routine pauses; sustained lateness is surfaced, never silently absorbed.

Strict-mode tolerance is one factor unit of wall time (SimPy 3's rule): zero tolerance would false-positive on scheduler jitter, and a full factor of slack still catches every systematic overrun.

Phase 3 — single-run conservative PDES

Every number below is measured by benchmarks/pdes_scaling.py (a balanced serpentine grid conveyor, ~410k events, full topo.run() wall clock including barriers, vs the sequential reference runner). Machine: Apple M5 (4P+6E), macOS, CPython 3.14.2t unless noted. Correctness is unconditional — the bitwise trace-equivalence suite gates every build; the numbers here are the honest performance record.

Measured shard scaling (3.14.2t, lookahead = segment transit)

Shards	Speedup vs sequential reference
1	1.06×
2	1.10×
4	0.95×
8	0.45×

Status of the ≥3×-at-8-shards roadmap figure: not demonstrated on current interpreters, as the Phase 3 spec anticipated. PDES shards are threads running the Phase 1 engine hot loop concurrently, and CPython 3.14.2t's reference-count contention on shared objects (measured in Phase 2, below) caps thread-parallel DES at ~1.1× regardless of the synchronizer's quality. The figure is recorded as interpreter-dependent and will be asserted when a CPython build scales pure-Python event loops across threads. GIL build for comparison: 0.90× at 2 shards — correct, time-sliced, prominently warned, exactly as documented.

Lookahead degradation (4-shard ring, local event spacing 0.25)

Lookahead / spacing	Wall clock vs L=4× sequential
4×	0.10×
2×	0.05×
1×	0.02×
0.5×	0.01×
0.25×	0.01×

The textbook conservative-PDES slowdown regime: the horizon advances at most one lookahead per round, so halving lookahead doubles the number of barrier rounds — the measured wall clock doubles down the whole sweep. With lookahead at or below the mean event spacing, a sharded run is dominated by synchronization and is strictly slower than sequential. Use llmsim.parallel.pdes.analyze() on a sequential trace to estimate the window economics before partitioning.

Slowdown regimes (PDES-specific)

• Low lookahead — see the sweep above; lookahead must be a large multiple of the mean event spacing for windows to carry meaningful work.
• Unbalanced shards — the window cost is the busiest shard's; a 90/10 split caps speedup at ~1.1× no matter the core count (analyze() reports this as balance_speedup).
• GIL builds — time-sliced threads: correct, warned at runtime, never faster than sequential.
• 3.14.2t thread contention — the same interpreter-level ceiling as Phase 2's thread backend (next section); today this dominates every other regime on free-threaded builds.

CI gate

Correctness gates are strict (bitwise equivalence at 1/2/4/8 shards, the jittered soak job). The only enforced timing assertion (benchmarks/test_pdes_scaling.py) is a catastrophic-regression floor: a 2-shard run must stay ≥0.5× sequential (measured 1.10× on 3.14t, 0.90× on the GIL build), skipped on shared macOS runners per the Phase 2 noise findings.

Phase 2 — parallel replications

Every number below is measured by benchmarks/replication_scaling.py (64 replications of an ~20 ms M/M/1, Experiment.run wall clock including executor startup and result transport, vs a pure in-process sequential loop). Machine: Apple M5, 10 physical cores (4 performance + 6 efficiency), macOS, CPython 3.14.2 / 3.14.2t. Ratios are the portable signal; absolute times are hardware-specific.

Measured replication throughput vs worker count

Free-threaded build (3.14.2t, GIL off — auto picks threads):

Workers	threads	processes	interpreters
2	1.33×	—	—
4	1.29×	2.80×	2.67×
8	0.98×	3.68×	2.92×
10	—	3.70×	—

GIL build (3.14.2 — auto picks processes):

Workers	processes	interpreters
1	0.94×	—
2	1.87×	—
4	2.86×	3.01×
8	3.85×	4.31×
10	3.83×	—

Context for the ceiling: an embarrassingly parallel pure-CPU baseline (no llmsim, no allocation-heavy inner loop) measures 4.69× at 8 workers on this machine — the 4P+6E core asymmetry, not coordination overhead, is the dominant cap. The process backend reaches ~80–90% of that ceiling; per-core efficiency at low worker counts is 0.94–0.99× (2 workers) and 0.72–0.77× (4 workers).

Status of the ≥6×-on-8-cores roadmap target: not yet demonstrated, honestly. This 10-core machine is heterogeneous (4P+6E); even perfect scaling cannot show 6× at 8 workers here. The measured per-core efficiencies above (0.94×/core at 2 workers, 0.72–0.77×/core at 4) are consistent with ≥6× on 8 homogeneous performance cores, but per the docs-honesty rule that number will be recorded only when measured on such a machine.

Slowdown regimes (read before choosing a backend)

• Threads on 3.14.2t anti-scale for DES event loops. The engine's and random.py's hot paths repeatedly touch objects shared by every thread (module-level C functions like heapq.heappush, code-object float constants, random.py globals), and reference-count cache-line contention on those objects currently outweighs the parallelism: the benchmark model measures 1.33× at 2 workers, then degrades (0.98× at 8). An isolated probe makes the mechanism unambiguous: a loop through pure-Python Random.expovariate scales 0.37× at 8 threads, while the same math with thread-locally bound log/constants scales 3.54×, and a bound C-method loop 3.19×. Until upstream free-threading contention improves, use backend="processes" for replication studies even on 3.14t.
• Process startup dominates small studies. A worker pays interpreter spawn + model import (macOS spawn is the worst case). A study whose total sequential runtime is under ~1 s can easily be slower with processes than sequentially (workers=1 measures 0.94×). Size studies to seconds, or keep max_workers low for tiny ones.
• Result transport is per-replication overhead. On the process and interpreter backends every config and result crosses a pickle boundary (zstd-compressed when spool=True); returning multi-MB traces per replication shifts time from simulation to serialization. Return KPIs, not raw traces, when you can.
• Efficiency cores flatten the top of every curve. Beyond the number of performance cores, each extra worker adds a fraction of a core's throughput (see the ceiling note above); max_workers = physical cores is the sweet spot, and more than that is wasted coordination.

CI gate

CI cannot see many cores — shared runners have ~2 physical cores behind their reported vCPUs — so the enforced regression gate (benchmarks/test_replication_scaling.py) runs 2 workers on the process backend and requires ≥1.35× total speedup: real scaling measures 1.6–2.0× at 2 workers while an accidental coordinator serialization reads ~1.0×, so the gate trips on a genuine regression rather than SMT or runner noise. The gate pins processes because the thread backend's 3.14t anti-scaling above is an interpreter-level regime, not an llmsim regression a gate could act on.

Memory (RSS) guidance

Measured on the benchmark model, 3.14t thread backend: coordinator base ~30 MB; each concurrent replication adds ~3–4 MB peak RSS. The process backend pays the interpreter base per worker instead. Free-threaded builds carry larger object headers and a stop-the-world collector, so budget ~10–20% extra for the same model on 3.14t, and prefer Experiment(..., spool=True) (zstd-compressed results, decompressed on access) when per-replication outputs are large. Worker-count rule of thumb: max_workers = physical cores, reduced if workers × per-worker RSS approaches available memory.

Phase 1 — sequential core

llmsim's Phase 1 goal is an honest single-thread win over SimPy 3 to justify the clean break, before any parallelism. This note records the measured numbers and — per the project's docs-honesty rule — the regimes where llmsim does not lead.

What is measured

Every number here is measured, never asserted. The benchmark harness (benchmarks/test_llmsim_benchmarks.py) runs each of the three canonical models on both engines, on the same machine, in the same process:

1. KPI equivalence — for a fixed seed, each llmsim model produces the exact same KPI as its SimPy 3 reference. llmsim reproduces SimPy 3's event ordering and RNG draw sequence bit-for-bit, so this is exact equality, not tolerance.
2. Determinism — the same seed yields the same KPI on repeated llmsim runs.
3. Speedup — a head-to-head, best-of-N wall-clock timing (SimPy time / llmsim time).

Measured single-thread speedup

Representative numbers on an Apple M-series laptop (Darwin-arm64, CPython 3.14, GIL build), best-of-9 wall clock. Absolute times are hardware-specific; the ratio is the portable signal.

Model	Work profile	SimPy 3	llmsim	Speedup
grid_conveyor	object-heavy (10 000 Resources, ~16 k events)	~29.4 ms	~23.9 ms	~1.23×
mm1_queue	call-bound (~12 k events, thin per-event work)	~7.9 ms	~7.7 ms	~1.03×
machine_shop	call-bound (preemptive repair, ~parity churn)	~3.06 ms	~2.99 ms	~1.02×

Where llmsim leads, and where it is at parity

• Object-heavy models lead clearly. grid_conveyor allocates thousands of resources and events; llmsim's __slots__-based, minimal-garbage objects cut per-object memory and GC pressure, yielding ~1.23× — clearing the ≥1.2× design target. The advantage grows with the number of live events and resources.
• Call-bound models run at parity. mm1_queue and machine_shop spend most of their time in generator send()/throw() and heapq push/pop — operations that are identical in both engines (near-identical call counts: ~3.95 M vs ~3.98 M for mm1_queue). There is no 1.2× to be had there without out-running an inner loop that llmsim and SimPy 3 share; llmsim lands a few percent ahead (~1.02–1.03×), i.e. at parity with no regression.

This is why the CI gate is tiered: grid_conveyor must show a clear lead, while the call-bound models are gated only against a meaningful regression. The enforced CI floors carry margin below the measured ratios so the gate catches a real regression without flaking on shared-runner timing noise: grid_conveyor measures ~1.24–1.38× and is gated at ≥1.15×; mm1_queue/machine_shop measure ~1.02–1.03× and are gated at ≥0.9× (no meaningful regression). The timing floors are enforced on Linux CI and local machines; shared macOS runners show ~20% spread on best-of-15 timings (the same commit measured 0.79–0.89× there while benching 1.0–1.03× locally and on ubuntu), so the timing assertion is skipped on macOS CI — KPI-equality and determinism still run everywhere. The honest summary is: llmsim matches SimPy 3 on lightweight models and pulls ahead as models get heavier — exactly where the later share-nothing parallelism (Phases 2–4) then multiplies the win across cores.

Caveats

• Micro-benchmark timings vary with hardware, thermal state, and shared CI runners; the harness compares best-of-N runs of both engines on the same machine in the same process to keep the ratio meaningful.
• Numbers above are the GIL build. The free-threaded build (3.14t) carries a small single-thread overhead for both engines equally, so the ratio is preserved; the free-threaded payoff arrives in Phase 2 with parallel replications, not in this sequential core.