wifi-densepose/crates/ruvector-mincut/docs/adr/ROADMAP-coherence-gate-implementation.md

Implementation Roadmap: Anytime-Valid Coherence Gate

Version: 1.0 Date: 2026-01-17 Related: ADR-001, DDC-001

Executive Summary

This document provides a phased implementation roadmap for the Anytime-Valid Coherence Gate (AVCG), integrating:

1. Dynamic Min-Cut (existing, enhanced)
2. Online Conformal Prediction (new)
3. E-Values/E-Processes (new)

The implementation is designed for incremental delivery with each phase providing standalone value.

---

Phase 0: Preparation (Current State Analysis)

Existing Infrastructure ✅

Component	Location	Status
SubpolynomialMinCut	src/subpolynomial/mod.rs	Production-ready
WitnessTree	src/witness/mod.rs	Production-ready
CutCertificate	src/certificate/mod.rs	Production-ready
DeterministicLocalKCut	src/localkcut/	Production-ready
GateController	mincut-gated-transformer/src/gate.rs	Production-ready
GatePacket	mincut-gated-transformer/src/packets.rs	Production-ready

Dependencies to Add

# Cargo.toml additions for ruvector-mincut
[dependencies]
# Statistics
statrs = "0.17"           # Statistical distributions
rand = "0.8"              # Random number generation
rand_distr = "0.4"        # Probability distributions

# Serialization for receipts
serde_json = "1.0"
bincode = "1.3"
blake3 = "1.5"            # Fast cryptographic hashing

# Optional: async support
tokio = { version = "1", features = ["sync"], optional = true }

---

Phase 1: E-Process Foundation

Goal: Implement core e-value and e-process infrastructure.

Task 1.1: E-Value Module

Create src/eprocess/evalue.rs:

/// Core e-value type with validity guarantees
pub struct EValue {
    value: f64,
    /// Null hypothesis under which E[e] ≤ 1
    null: NullHypothesis,
    /// Computation timestamp
    timestamp: u64,
}

/// Supported null hypotheses
pub enum NullHypothesis {
    /// P(unsafe outcome) ≤ p0
    ActionSafety { p0: f64 },
    /// Current state ~ reference distribution
    StateStability { reference: DistributionId },
    /// Policy matches reference
    PolicyConsistency { reference: PolicyId },
}

impl EValue {
    /// Create from likelihood ratio
    pub fn from_likelihood_ratio(
        likelihood_h1: f64,
        likelihood_h0: f64,
    ) -> Self;

    /// Create mixture e-value for robustness
    pub fn from_mixture(
        components: &[EValue],
        weights: &[f64],
    ) -> Self;

    /// Verify E[e] ≤ 1 property empirically
    pub fn verify_validity(&self, samples: &[f64]) -> bool;
}

Task 1.2: E-Process Module

Create src/eprocess/process.rs:

/// E-process for continuous monitoring
pub struct EProcess {
    /// Current accumulated value
    current: f64,
    /// History for audit
    history: Vec<EValue>,
    /// Update rule
    update_rule: UpdateRule,
}

pub enum UpdateRule {
    /// E_t = Π e_i (aggressive)
    Product,
    /// E_t = (1/t) Σ e_i (conservative)
    Average,
    /// E_t = λe_t + (1-λ)E_{t-1}
    ExponentialMoving { lambda: f64 },
    /// E_t = Σ w_j E_t^{(j)}
    Mixture { weights: Vec<f64> },
}

impl EProcess {
    pub fn new(rule: UpdateRule) -> Self;
    pub fn update(&mut self, e: EValue);
    pub fn current_value(&self) -> f64;

    /// Check stopping condition
    pub fn should_stop(&self, threshold: f64) -> bool;

    /// Export for audit
    pub fn to_evidence_receipt(&self) -> EvidenceReceipt;
}

Task 1.3: Stopping Rules

Create src/eprocess/stopping.rs:

/// Anytime-valid stopping rule
pub struct StoppingRule {
    /// Threshold for rejection
    reject_threshold: f64,  // typically 1/α
    /// Threshold for acceptance (optional)
    accept_threshold: Option<f64>,
}

impl StoppingRule {
    /// Check if we can stop now
    pub fn can_stop(&self, e_process: &EProcess) -> StoppingDecision;

    /// Get confidence at current stopping time
    pub fn confidence_at_stop(&self, e_process: &EProcess) -> f64;
}

pub enum StoppingDecision {
    /// Continue accumulating evidence
    Continue,
    /// Reject null (evidence of incoherence)
    Reject { confidence: f64 },
    /// Accept null (evidence of coherence)
    Accept { confidence: f64 },
}

Deliverables Phase 1

• src/eprocess/mod.rs - module organization
• src/eprocess/evalue.rs - e-value implementation
• src/eprocess/process.rs - e-process implementation
• src/eprocess/stopping.rs - stopping rules
• src/eprocess/mixture.rs - mixture e-values
• Unit tests with ≥95% coverage
• Integration with CutCertificate

Acceptance Criteria Phase 1

• E[e] ≤ 1 verified for all implemented e-value types
• E-process maintains supermartingale property
• Stopping rule provides valid Type I error control
• Computation time < 1ms for single e-value

---

Phase 2: Conformal Prediction

Goal: Implement online conformal prediction with shift adaptation.

Task 2.1: Prediction Set Core

Create src/conformal/prediction_set.rs:

/// Conformal prediction set
pub struct PredictionSet<T> {
    /// Elements in the set
    elements: Vec<T>,
    /// Coverage target
    coverage: f64,
    /// Non-conformity scores
    scores: Vec<f64>,
}

impl<T> PredictionSet<T> {
    /// Check if outcome is in set
    pub fn contains(&self, outcome: &T) -> bool;

    /// Get set size (measure of uncertainty)
    pub fn size(&self) -> usize;

    /// Get normalized uncertainty measure
    pub fn uncertainty(&self) -> f64;
}

Task 2.2: Non-Conformity Scores

Create src/conformal/scores.rs:

/// Non-conformity score function
pub trait NonConformityScore {
    type Input;
    type Output;

    fn score(&self, input: &Self::Input, output: &Self::Output) -> f64;
}

/// Absolute residual score
pub struct AbsoluteResidual<P: Predictor> {
    predictor: P,
}

/// Normalized residual score
pub struct NormalizedResidual<P: Predictor + UncertaintyEstimator> {
    predictor: P,
}

/// Conformalized Quantile Regression (CQR)
pub struct CQRScore<Q: QuantilePredictor> {
    quantile_predictor: Q,
}

Task 2.3: Online Conformal with Adaptation

Create src/conformal/online.rs:

/// Online conformal predictor with shift adaptation
pub struct OnlineConformal<S: NonConformityScore> {
    score_fn: S,
    /// Calibration buffer
    calibration: RingBuffer<f64>,
    /// Current quantile
    quantile: f64,
    /// Adaptation method
    adaptation: AdaptationMethod,
}

pub enum AdaptationMethod {
    /// Adaptive Conformal Inference
    ACI { learning_rate: f64 },
    /// Retrospective adjustment
    Retrospective { window: usize },
    /// Conformal Optimistic Prediction
    COP { cdf_estimator: Box<dyn CDFEstimator> },
}

impl<S: NonConformityScore> OnlineConformal<S> {
    /// Generate prediction set
    pub fn predict(&self, input: &S::Input) -> PredictionSet<S::Output>;

    /// Update with observed outcome
    pub fn update(&mut self, input: &S::Input, outcome: &S::Output);

    /// Get current coverage estimate
    pub fn coverage_estimate(&self) -> f64;
}

Task 2.4: CORE RL-Based Adaptation

Create src/conformal/core.rs:

/// CORE: RL-based conformal adaptation
pub struct COREConformal<S: NonConformityScore> {
    base: OnlineConformal<S>,
    /// RL agent for quantile adjustment
    agent: QuantileAgent,
    /// Coverage as reward signal
    coverage_target: f64,
}

/// Simple TD-learning agent for quantile adjustment
struct QuantileAgent {
    q_value: f64,
    learning_rate: f64,
    discount: f64,
}

impl<S: NonConformityScore> COREConformal<S> {
    /// Predict with RL-adjusted quantile
    pub fn predict(&self, input: &S::Input) -> PredictionSet<S::Output>;

    /// Update agent and base conformal
    pub fn update(&mut self, input: &S::Input, outcome: &S::Output, covered: bool);
}

Deliverables Phase 2

• src/conformal/mod.rs - module organization
• src/conformal/prediction_set.rs - prediction set types
• src/conformal/scores.rs - non-conformity scores
• src/conformal/online.rs - online conformal with ACI
• src/conformal/retrospective.rs - retrospective adjustment
• src/conformal/cop.rs - Conformal Optimistic Prediction
• src/conformal/core.rs - RL-based adaptation
• Unit tests with ≥90% coverage

Acceptance Criteria Phase 2

• Marginal coverage ≥ 1 - α on exchangeable data
• Coverage maintained under gradual shift (δ < 0.1/step)
• Recovery within 100 steps after abrupt shift
• Prediction latency < 10ms

---

Phase 3: Gate Integration

Goal: Integrate all components into unified gate controller.

Task 3.1: Anytime Gate Policy

Create src/anytime_gate/policy.rs:

/// Policy for anytime-valid gate
pub struct AnytimeGatePolicy {
    /// E-process thresholds
    pub e_deny_threshold: f64,      // τ_deny
    pub e_permit_threshold: f64,    // τ_permit

    /// Conformal thresholds
    pub uncertainty_threshold: f64,  // θ_uncertainty
    pub confidence_threshold: f64,   // θ_confidence

    /// Min-cut thresholds (from existing GatePolicy)
    pub lambda_min: u32,
    pub boundary_max: u16,

    /// Adaptation settings
    pub adaptive_thresholds: bool,
    pub threshold_learning_rate: f64,
}

Task 3.2: Unified Gate Controller

Create src/anytime_gate/controller.rs:

/// Unified anytime-valid coherence gate
pub struct AnytimeGateController<S: NonConformityScore> {
    /// Existing min-cut infrastructure
    mincut: SubpolynomialMinCut,

    /// Conformal predictor
    conformal: OnlineConformal<S>,

    /// E-process for evidence
    e_process: EProcess,

    /// Policy
    policy: AnytimeGatePolicy,
}

impl<S: NonConformityScore> AnytimeGateController<S> {
    /// Evaluate gate for action
    pub fn evaluate(&mut self, action: &Action, context: &Context) -> GateResult;

    /// Update after observing outcome
    pub fn update(&mut self, action: &Action, outcome: &Outcome);

    /// Generate witness receipt
    pub fn receipt(&self, decision: &GateDecision) -> WitnessReceipt;
}

pub struct GateResult {
    pub decision: GateDecision,

    // From min-cut
    pub cut_value: f64,
    pub witness_partition: Option<WitnessPartition>,

    // From conformal
    pub prediction_set_size: f64,
    pub uncertainty: f64,

    // From e-process
    pub e_value: f64,
    pub evidence_sufficient: bool,
}

Task 3.3: Witness Receipt

Create src/anytime_gate/receipt.rs:

/// Cryptographically sealed witness receipt
#[derive(Serialize, Deserialize)]
pub struct WitnessReceipt {
    /// Receipt metadata
    pub id: Uuid,
    pub timestamp: u64,
    pub action_id: ActionId,
    pub decision: GateDecision,

    /// Structural witness (from min-cut)
    pub structural: StructuralWitness,

    /// Predictive witness (from conformal)
    pub predictive: PredictiveWitness,

    /// Evidential witness (from e-process)
    pub evidential: EvidentialWitness,

    /// Cryptographic seal
    pub hash: [u8; 32],
    pub signature: Option<[u8; 64]>,
}

#[derive(Serialize, Deserialize)]
pub struct StructuralWitness {
    pub cut_value: f64,
    pub partition_hash: [u8; 32],
    pub critical_edge_count: usize,
}

#[derive(Serialize, Deserialize)]
pub struct PredictiveWitness {
    pub prediction_set_size: usize,
    pub coverage_target: f64,
    pub adaptation_rate: f64,
}

#[derive(Serialize, Deserialize)]
pub struct EvidentialWitness {
    pub e_value: f64,
    pub e_process_cumulative: f64,
    pub null_hypothesis: String,
    pub stopping_valid: bool,
}

impl WitnessReceipt {
    pub fn seal(&mut self) {
        self.hash = blake3::hash(&self.to_bytes()).into();
    }

    pub fn verify(&self) -> bool {
        self.hash == blake3::hash(&self.to_bytes_without_hash()).into()
    }
}

Deliverables Phase 3

• src/anytime_gate/mod.rs - module organization
• src/anytime_gate/policy.rs - gate policy
• src/anytime_gate/controller.rs - unified controller
• src/anytime_gate/decision.rs - decision types
• src/anytime_gate/receipt.rs - witness receipts
• Integration tests with full pipeline
• Benchmarks for latency validation

Acceptance Criteria Phase 3

• Gate latency p99 < 50ms
• All three signals integrated correctly
• Witness receipts pass verification
• Graceful degradation on component failure

---

Phase 4: Production Hardening

Goal: Production-ready implementation with monitoring and optimization.

Task 4.1: Performance Optimization

• SIMD-optimized e-value computation
• Lazy evaluation for conformal sets
• Batched graph updates for min-cut
• Memory-mapped receipt storage

Task 4.2: Monitoring & Alerting

• Prometheus metrics for gate decisions
• Coverage drift detection
• E-process anomaly alerts
• Latency histogram tracking

Task 4.3: Operational Tooling

• Receipt query API
• Threshold tuning dashboard
• A/B testing framework for policy comparison
• Incident replay from receipts

Task 4.4: Documentation

• API documentation
• Operator runbook
• Threshold tuning guide
• Troubleshooting guide

---

Timeline Summary

Phase	Duration	Dependencies	Deliverable
Phase 0	Complete	-	Requirements analysis
Phase 1	2 weeks	None	E-process module
Phase 2	3 weeks	Phase 1	Conformal module
Phase 3	2 weeks	Phase 1, 2	Unified gate
Phase 4	2 weeks	Phase 3	Production hardening

Total estimated effort: 9 weeks

---

Risk Register

Risk	Probability	Impact	Mitigation
E-value power too low	Medium	High	Mixture e-values; tuned alternatives
Conformal sets too large	Medium	Medium	COP for tighter sets; better base predictor
Latency exceeds budget	Low	High	Early profiling; lazy evaluation
Integration complexity	Medium	Medium	Phased delivery; isolated modules
Threshold tuning difficulty	High	Medium	Adaptive thresholds; meta-learning

---

Success Metrics

Metric	Target	Measurement
False deny rate	< 5%	Simulation
Missed unsafe rate	< 0.1%	Simulation
Gate latency p99	< 50ms	Production
Coverage maintenance	≥ 85%	Production
Receipt verification pass	100%	Audit

---

References

1. El-Hayek, Henzinger, Li. arXiv:2512.13105 (Dec 2025)
2. Online Conformal with Retrospective. arXiv:2511.04275 (Nov 2025)
3. Ramdas, Wang. "Hypothesis Testing with E-values" (2025)
4. ICML 2025 Tutorial on SAVI
5. Distribution-informed Conformal (COP). arXiv:2512.07770 (Dec 2025)