Determinism and Replay

This document explains how the C_AI Platform ensures deterministic replay of assessment runs through snapshot pinning and immutability rules.

Why Determinism Matters

Compliance platforms require:

1. Reproducibility - Same inputs produce same outputs
2. Auditability - Ability to reconstruct exactly what happened
3. Regulatory defense - Prove assessment results are accurate

Snapshot Pinning

When a run is created, the platform captures a snapshot of all configuration versions that affect the run outcome.

What Gets Pinned at Run Creation

Location: server/lib/assessmentPipeline.ts:128-146

const [run] = await db.insert(runs).values({
  tenantId,
  packId,
  environment: snapshot.environment,
  status: RunStatus.IN_PROGRESS,
  // Snapshot pinning (Step 9) - deterministic replay
  snapshotCriteriaVersionId: snapshot.snapshotCriteriaVersionId,
  snapshotCorpusActivationId: snapshot.snapshotCorpusActivationId,
  snapshotRetrievalVersionId: snapshot.snapshotRetrievalVersionId,
  snapshotPromptVersionId: snapshot.snapshotPromptVersionId,
  snapshotChunkingVersionId: snapshot.snapshotChunkingVersionId,
  snapshotAgentVersionId: snapshot.snapshotAgentVersionId
}).returning();

Pinned version IDs (defined in runs table, shared/schema.ts:354-359):

Field	Purpose
snapshotCriteriaVersionId	Which criteria set version
snapshotCorpusActivationId	Which regulatory corpus activation
snapshotRetrievalVersionId	Retrieval engine config version
snapshotPromptVersionId	Prompt template version
snapshotChunkingVersionId	Chunking config version
snapshotAgentVersionId	Agent workflow version

How Pinning Works

Pack resolution (server/lib/packResolver.ts:25-105):

1. resolvePackBinding(tenantId, packId, environment) queries current bindings
2. Returns EffectivePackBinding with all version IDs
3. createRunSnapshot(binding) extracts snapshot fields (lines 107-117)
4. Snapshot is persisted to runs table at run creation

export function createRunSnapshot(binding: EffectivePackBinding) {
  return {
    environment: binding.environment,
    snapshotCriteriaVersionId: binding.criteriaVersionId,
    snapshotCorpusActivationId: binding.corpusActivationId,
    snapshotRetrievalVersionId: binding.retrievalVersionId,
    snapshotPromptVersionId: binding.promptVersionId,
    snapshotChunkingVersionId: binding.chunkingVersionId,
    snapshotAgentVersionId: binding.agentVersionId
  };
}

Immutable Replay Snapshots

At run completion, a full immutable snapshot is captured.

What Gets Frozen

Location: shared/schema.ts:382-397 (run_replay_snapshots table)

Field	Content
assessmentsData	All assessment records (jsonb)
tasksData	All generated tasks (jsonb)
exportsData	All export records (jsonb)
configSnapshot	Frozen configuration state (jsonb)
snapshotHash	SHA256 of snapshot content

Snapshot Status Tracking

Location: shared/schema.ts:306-311

export const SnapshotStatus = {
  PENDING: "PENDING",     // Run not yet completed
  CAPTURED: "CAPTURED",   // Snapshot successfully captured
  FAILED: "FAILED",       // Snapshot capture failed
  LEGACY: "LEGACY"        // Pre-Step 9 run - no snapshot exists
} as const;

The snapshotStatus column on runs table tracks this state.

Replay Guard

The replay guard prevents non-deterministic operations during replay.

How It Works

Location: server/lib/replayGuard.ts

const replayStorage = new AsyncLocalStorage<ReplayContext>();

export function runInReplayMode<T>(runId: string, fn: () => T | Promise<T>) {
  return replayStorage.run({ runId, isReplay: true }, fn);
}

export function assertReplaySafe(operation: string): void {
  const store = replayStorage.getStore();
  if (store?.isReplay) {
    throw new ReplayModeError(operation);
  }
}

Forbidden Operations in Replay Mode

Operations that call assertReplaySafe() will throw ReplayModeError during replay:

Operation	Location	Why Forbidden
generateSimpleEmbedding	assessmentPipeline.ts:66	Non-deterministic
vector search	vectorStore.ts:60	Live DB query
corpus vector search	vectorStore.ts:89	Live DB query

Pack Immutability

Once a pack has ACTIVE bindings, it becomes immutable.

Immutability Service

Location: server/lib/packSafetyService.ts

Key functions:

• isPackScopeActive(scope) - Checks if pack has any ACTIVE binding (lines 60-114)
• enforcePackImmutability(scope) - Throws PackActiveImmutableError if active (lines 120-131)

Pack is ACTIVE if it has any of (documented at lines 55-58):

• corpus_activations with is_active=1 for (tenant_id, pack_id, environment)
• pack_criteria_bindings for (tenant_id, pack_id, environment)
• engine_active_bindings for (tenant_id, pack_id, environment)

Error Response

When attempting to modify an ACTIVE pack:

Location: server/lib/packSafetyService.ts:27-46

export class PackActiveImmutableError extends Error {
  code = "PACK_ACTIVE_IMMUTABLE";
  httpStatus = 409;
  // ...
}

Returns HTTP 409 with error details.

What Must Never Mutate

During a Run

These must remain constant from run start to completion:

Data	Reason
Criteria definitions	Assessment targets
Corpus text units	Regulatory reference
Engine configs	Assessment behavior
Evidence documents	Assessment inputs

After Run Completion

These are frozen permanently:

Data	Location
run_replay_snapshots	Immutable table with runUnique constraint
Pinned version IDs on runs	Set at creation, never updated
Audit log entries	Append-only (server/lib/audit.ts)

Replay Process

For CAPTURED Runs

1. Load run_replay_snapshots by runId
2. Return frozen assessmentsData, tasksData, exportsData
3. No live queries executed

For FAILED Runs

1. Check snapshotStatus = FAILED
2. Return error: "REPLAY_UNAVAILABLE: Snapshot capture failed"
3. Operator must investigate original failure

For LEGACY Runs

1. Check snapshotStatus = LEGACY
2. Fall back to live table queries with warning
3. Results may differ from original run

Uniqueness Constraints

Environment-scoped uniqueness prevents conflicts:

Corpus activations (shared/schema.ts):

CREATE UNIQUE INDEX corpus_activations_unique_active_idx 
  ON corpus_activations (tenant_id, pack_id, environment) 
  WHERE is_active = 1;

Criteria bindings:

CREATE UNIQUE INDEX pack_criteria_bindings_env_unique 
  ON pack_criteria_bindings (tenant_id, pack_id, environment);

Engine bindings:

CREATE UNIQUE INDEX engine_binding_env_unique 
  ON engine_active_bindings (tenant_id, pack_id, environment, config_type);

Testing Determinism

To verify deterministic replay:

1. Complete a run (status: COMPLETED, snapshotStatus: CAPTURED)
2. Request replay via admin API
3. Compare replay output hash with original snapshotHash
4. Hashes must match exactly

Related Pages

• Architecture Overview - System design
• Data Model - Schema reference
• Testing and Debugging - How to reproduce runs