Parallel

The share-nothing parallelism surface: parallel replications of a whole model, the execution-backend abstraction, single-run PDES sharding, and the compute offload pool. Every symbol here is re-exported from the top-level llmsim package.

Parallel replications

Experiment

Experiment(factory, configs, *, master_seed, spool=False)

N replications x M configs of one model factory, reproducibly.

Parameters:

Name	Type	Description	Default
factory	Factory	A module-level callable factory(stream, config) (or factory(stream, config, cancel) to opt into cooperative cancellation). Validated for importability at construction.	required
configs	Sequence[Any]	The experiment's configurations; each must survive the chosen backend's transport (preflighted per run when the backend pickles).	required
master_seed	int	The explicit study seed (required, keyword-only; no ambient default). Every replication's stream derives from it via the seed tree, so the same seed reproduces the same study.	required
spool	bool	When True, worker results are zstd-compressed pickle bytes decompressed lazily on access -- bounds peak coordinator memory for large per-replication outputs.	False

Use as a context manager to cancel any in-flight work on exit::

with Experiment(my_model, configs, master_seed=42) as exp:
    for result in exp.iter_results(replications=100):
        if good_enough(result):
            break  # __exit__ cancels the rest

Validate the factory and root the seed tree at master_seed.

master_seed property

master_seed

The explicit study seed the whole seed tree derives from.

run

run(replications, *, backend='auto', max_workers=None)

Run the full study and return results keyed by identity.

The returned dict maps (config_index, replication_index) to :class:ReplicationResult and iterates in sorted key order -- completion order never leaks into the result set.

iter_results

iter_results(
    replications, *, backend="auto", max_workers=None
)

Yield results as they complete (each exactly once, identity-keyed).

Aggregation stays order-insensitive because every result carries its (config_index, replication_index); only the arrival order is completion-dependent. After :meth:cancel, replications that already finished are still yielded, cancelled ones are not, and no new work is dispatched.

cancel

cancel()

Cancel the in-flight run (one API, two granularities).

Stops dispatching queued work and Future.cancel()-s pending submissions on every backend. On the thread backend the shared token additionally fires, so a cancellation-aware factory stops mid-replication; a factory that owns its own sim.run() -- and any interpreter/process worker -- finishes its current replication first (replication granularity). Already-collected results stay valid.

ReplicationResult

ReplicationResult(
    config_index,
    replication_index,
    seed,
    payload,
    *,
    spooled=False,
)

One replication's identity, seed, and returned value.

Results are keyed and compared by (config_index, replication_index) plus the derived seed and the factory's value; completion order is nowhere in the type, by construction.

Wrap one replication's payload (raw, or zstd-spooled bytes).

spooled property

spooled

Whether the value is held as zstd-compressed bytes (spool=True).

value property

value

The factory's returned value.

When the experiment spools (spool=True), the value is stored as zstd-compressed pickle bytes to bound peak memory and decompressed on each access; hold onto the returned object if you read it repeatedly.

run_replications

run_replications(
    factory,
    configs,
    *,
    master_seed,
    replications,
    backend="auto",
    max_workers=None,
)

Run one study functionally: construct, run, and return the result set.

Equivalent to Experiment(factory, configs, master_seed=master_seed).run(replications, backend=backend, max_workers=max_workers).

ReplicationError

Bases: RuntimeError

One replication failed (fail-fast contract).

Names the offending (config_index, replication_index) and chains the worker's original exception as __cause__. Raising instead of returning a partial result set keeps a study's output all-or-nothing.

Execution backends

ExecutionBackend

ExecutionBackend(kind)

One executor-shaped interface over threads, interpreters, and processes.

Instances are thin and stateless: they know their concrete kind, how to create the matching concurrent.futures executor, and the two traits the replication runner branches on (whether payloads cross a pickle boundary, and whether workers can share a live cancellation token).

Wrap the concrete backend kind ("auto" is not accepted here).

requires_transport property

requires_transport

Whether payloads cross a pickle boundary on this backend.

supports_shared_cancellation property

supports_shared_cancellation

Whether workers can observe a live parent token mid-replication.

resolve classmethod

resolve(backend='auto')

Resolve a backend name, honoring an explicit choice verbatim.

"auto" picks "threads" on the free-threaded build (GIL disabled) and "processes" on a GIL build.

cancel_token

cancel_token()

Return the cancellation token this backend's workers should receive.

A live shared token on the thread backend (mid-replication cancel for token-aware factories); the inert base token elsewhere, where isolated workers cannot observe the parent. Callers fire either uniformly via token._cancel().

executor

executor(max_workers=None)

Create this backend's executor with max_workers workers.

max_workers=None defaults to os.process_cpu_count() on every backend, so worker count is uniform and predictable rather than executor-specific.

CancelToken

The cancellation handle a factory may consult between step() calls.

This base token is inert -- :attr:cancelled is always False -- and picklable; it is what factories receive on the interpreter and process backends, where a parent's cancellation cannot be observed mid-run. On the thread backend the factory receives a live shared token instead, so checking :attr:cancelled between steps gives mid-replication cancellation. Factories annotate the parameter as CancelToken and need not know which they got.

cancelled property

cancelled

Whether the experiment has been cancelled (always False here).

FactoryValidationError

Bases: TypeError

The factory cannot be transported to workers by reference.

Raised at :class:~llmsim.parallel.replicate.Experiment construction when the factory is not an importable module-level callable (a lambda, a local function or closure, or anything not reachable as module.qualified_name). The fix is always the same: define the factory with def at the top level of an importable module.

TransportError

Bases: RuntimeError

A config or result cannot cross the backend's transport boundary.

Names the offending config (and replication, when known) so a failing study points at the exact object to fix -- never a silent partial run.

PDES sharding

ShardedSim

ShardedSim(shards, *, master_seed, debug=False)

One large model partitioned into channel-connected shards.

Parameters:

Name	Type	Description	Default
shards	int	Number of shards; each gets its own Sim and (in threaded mode) its own thread.	required
master_seed	int	The explicit study seed (required, keyword-only). Shard i's Sim adopts the domain-separated stream SeedTree(master_seed).shard_rng(i).	required
debug	bool	Construct shard Sims with the owner-thread debug guard on (also enabled globally by LLMSIM_DEBUG=1).	False

Declare a topology of shards empty shards.

master_seed property

master_seed

The explicit study seed all shard streams derive from.

shard

shard(shard_index)

Register the builder for shard_index (decorator).

run

run(until, *, mode='threads')

Run the topology until every event below until has executed.

mode="threads" runs one thread per shard under the safe-window synchronizer; mode="sequential" runs the identical window algorithm on the calling thread — the reference oracle, bitwise trace-equal to the threaded mode for the same master seed. Events at exactly until (and later) are not executed, matching Sim.run(until=...).

ShardPorts

ShardPorts(shard_index, sim, registry)

A shard builder's window onto the topology's channels.

Handed to each builder as its second argument; every endpoint the shard uses must be declared through it, which is what makes construction-time wiring validation possible.

Bind the ports view to one shard's sim and the shared registry.

out

out(name, *, lookahead)

Declare this shard as the sender of channel name.

inbox

inbox(name)

Declare this shard as the receiver of channel name.

ShardError

Bases: RuntimeError

One shard failed during a run (fail-fast contract).

Names the failing shard index and chains the original exception as __cause__; all other shard threads are cancelled and joined before this is raised, mirroring the Phase 2 replication failure contract.

TopologyError

Bases: ValueError

The sharded topology is mis-wired.

Raised at construction/validation time — duplicate or missing shard builders, channel names without exactly one sender and one receiver, endpoints declared on the wrong shard, or a channel from a shard to itself. Never raised mid-run: a topology that validates runs.

LookaheadError

Bases: ValueError

A channel's lookahead contract was violated.

Raised when a channel is declared with a non-positive lookahead (the safe-window algorithm cannot make progress without positive lookahead on every channel) or when send() is called with delay < lookahead (the lookahead is a promise to the synchronizer; breaking it would let a message land inside an already-executed window).

Channels and run modes

RunMode module-attribute

RunMode = Literal['threads', 'sequential']

Channel

Channel(*, name, lookahead, sim)

The sender-side endpoint of one directed inter-shard link.

Owned by exactly one shard; send() may only be called from that shard's processes while it executes a window. The lookahead is the channel's normative promise: every message carries timestamp >= sender_now + lookahead.

Create the endpoint; the topology wires it via :meth:bind.

A ShardedSim assigns channel ids in sorted-name order after every builder has declared its endpoints, so ids are deterministic regardless of build interleaving.

bind

bind(*, channel_id, mailbox)

Wire the endpoint to its id and destination (topology-internal).

send

send(payload, *, delay)

Send payload to arrive delay after the sender's current time.

Raises:

Type	Description
LookaheadError	if delay < lookahead — the channel's promise to the synchronizer would be broken.

Inbox

Inbox(*, sim, name)

The receiving shard's Store-like endpoint for one channel.

yield inbox.get() in a shard process resolves to the next delivered payload, in the globally deterministic delivery order. Deliveries are performed by the synchronizer at window edges via :meth:deliver: each message becomes a Timeout at its stamped time (an already-successful event carrying the payload) whose processing deposits the payload into this inbox's store, waking any waiting get().

Create the endpoint local to the receiving shard's sim.

get

get()

Return an event resolving to the next delivered payload.

deliver

deliver(message)

Schedule message's payload to land at its timestamp.

Called by the synchronizer while the shard is quiescent, in the normative sorted order — the resulting event-id assignment makes same-timestamp delivery ordering deterministic.

Raises:

Type	Description
RuntimeError	if the message's timestamp is in the shard's past — a causality violation that the safe-window algorithm can never legitimately produce.

Message dataclass

Message(timestamp, channel_id, sequence, payload)

One cross-shard message with its deterministic ordering stamp.

sort_key staticmethod

sort_key(message)

Return the normative delivery order key: (timestamp, channel, seq).

Window analysis

analyze() estimates a partition's parallel-window economics from a sequential trace before you shard — see PDES sharding.

analyze

analyze(traces, *, lookahead)

Estimate achievable PDES speedup for a partition from its traces.

Parameters:

Name	Type	Description	Default
traces	Mapping[int, Sequence[TraceRecord]]	Per-shard trace records (shard index -> records), e.g. from running the topology through the sequential reference runner with a tracer attached per shard.	required
lookahead	float	The window width — use the minimum lookahead the topology's channels would declare.	required

Raises:

Type	Description
ValueError	if traces is empty or lookahead is not positive.

PdesAnalysis dataclass

PdesAnalysis(
    per_shard_events,
    balance_speedup,
    window_count,
    predicted_speedup,
)

The window-model estimate for one proposed partition.

Compute offload

OffloadPool

OffloadPool(sim, *, backend='auto', max_workers=None)

The worker pool behind :meth:~llmsim.core.sim.Sim.offload.

Construct one per Sim before calling run() (attaching a pool mid-run is unsupported)::

sim = Sim(seed=7)
pool = OffloadPool(sim, backend="processes")
...
sim.run()
pool.close()

Parameters:

Name	Type	Description	Default
sim	Sim	The simulation to attach to. A Sim accepts exactly one pool.	required
backend	OffloadBackendName	"inline" runs payloads synchronously on the owning thread (the sequential reference); the Phase 2 names select a worker pool; "auto" resolves like :meth:~llmsim.parallel.backends.ExecutionBackend.resolve -- except inside an Experiment replication worker, where it resolves to "inline" (the nested-pool rule; pooled offload inside workers is explicit opt-in).	'auto'
max_workers	int | None	Pool size; None means os.process_cpu_count().	None

The executor starts lazily on the first pooled submission. Use as a context manager, or call :meth:close, to release workers: outstanding offloads are abandoned, pending futures cancelled, and running payloads finish with their results discarded.

Raises:

Type	Description
RuntimeError	if sim already has a pool, or if the requested backend cannot host nested pools in this worker context (backend="processes" inside an interpreters-backend worker).

Resolve the backend, apply the nested-pool rule, and attach.

kind property

kind

The resolved concrete mode: "inline" or a Phase 2 backend kind.

submit

submit(fn, args, kwargs, *, delay, strict)

Dispatch one payload (the OffloadHandler seam).

Called by :meth:~llmsim.core.sim.Sim.offload; see that method for the user-facing contract.

poll

poll()

Deliver completed non-strict results (called between steps).

drain

drain()

Block for outstanding non-strict work once the schedule empties.

The run-end rule: a non-strict result is never silently dropped -- run() waits wall-clock for at least one outstanding payload, delivers everything then complete, and keeps stepping.

close

close()

Abandon outstanding offloads and shut the executor down.

Pending futures are cancelled; running payloads finish (they cannot be interrupted) and their results are discarded. Idempotent.

OffloadEvent

OffloadEvent(sim, pool, future, qualname, *, strict)

Bases: Event[T]

The completion event of one offloaded payload.

In strict mode the event is born scheduled at its completion slot with a placeholder value; its first callback resolves the real outcome (blocking wall-clock on the worker future if needed) before any waiter runs, so the heap structure -- and therefore the trace -- is identical to the inline reference. In non-strict mode it is born untriggered and delivered by the pool once the owning thread observes the completed future.

:meth:cancel abandons the computation: the pending future is cancelled if not started, and a finished result is discarded, never delivered. Interrupting the last process waiting on this event cancels it automatically (the requirements' cancellation contract).

Create the event for one submitted payload (see OffloadPool).

cancel

cancel()

Abandon the offload: its result or exception is never delivered.

Idempotent; a no-op once the event has processed. Cancels the pending future when the payload has not started; a running or finished payload's outcome is discarded. The event still processes as a discarded outcome (None) -- at its slot in strict mode, at max(now, now + delay) otherwise -- so a process waiting on a cancelled offload resumes with None rather than stranding.

NonStrictOffloadWarning

Bases: RuntimeWarning

A strict=False offload was submitted while debug mode is on.

Non-strict delivery times depend on wall-clock completion, so event ordering is not reproducible across runs -- exactly the property debug mode exists to guard. Emitted per call, naming the payload and call site.