dearsky/wifi-densepose

Fork 0

Files

ruv cd5943df23 Merge commit 'd803bfe2b1fe7f5e219e50ac20d6801a0a58ac75' as 'vendor/ruvector'

2026-02-28 14:39:40 -05:00

15 KiB

Raw Permalink Blame History

EXO-AI 2025: Comprehensive Test Strategy

Test Agent Status

Status: ⏳ WAITING FOR CRATES Last Updated: 2025-11-29 Agent: Unit Test Specialist

Overview

This document defines the comprehensive testing strategy for the EXO-AI 2025 cognitive substrate platform. Testing will follow Test-Driven Development (TDD) principles with a focus on quality, coverage, and maintainability.

1. Test Pyramid Architecture

         /\
        /E2E\          <- 10% - Full system integration
       /------\
      /Integr. \       <- 30% - Cross-crate interactions
     /----------\
    /   Unit     \     <- 60% - Core functionality
   /--------------\

Coverage Targets

Unit Tests: 85%+ coverage
Integration Tests: 70%+ coverage
E2E Tests: Key user scenarios
Performance Tests: All critical paths
Security Tests: All trust boundaries

2. Per-Crate Test Strategy

2.1 exo-core Tests

Module: Core traits and types Test Focus: Trait contracts, type safety, error handling

// tests/core_traits_test.rs
#[cfg(test)]
mod substrate_backend_tests {
    use exo_core::*;

    #[test]
    fn test_substrate_backend_trait_bounds() {
        // Verify Send + Sync bounds
    }

    #[test]
    fn test_pattern_construction() {
        // Validate Pattern type construction
    }

    #[test]
    fn test_topological_query_variants() {
        // Test all TopologicalQuery enum variants
    }
}

Test Categories:

✅ Trait bound validation
✅ Type construction and validation
✅ Enum variant coverage
✅ Error type completeness
✅ Serialization/deserialization

2.2 exo-manifold Tests

Module: Learned manifold engine Test Focus: Neural network operations, gradient descent, forgetting

// tests/manifold_engine_test.rs
#[cfg(test)]
mod manifold_tests {
    use exo_manifold::*;
    use burn::backend::NdArray;

    #[test]
    fn test_manifold_retrieve_convergence() {
        // Test gradient descent converges
        let backend = NdArray::<f32>::default();
        let engine = ManifoldEngine::<NdArray<f32>>::new(config);

        let query = Tensor::from_floats([0.1, 0.2, 0.3]);
        let results = engine.retrieve(query, 5);

        assert_eq!(results.len(), 5);
        // Verify convergence metrics
    }

    #[test]
    fn test_manifold_deform_gradient_update() {
        // Test deformation updates weights correctly
    }

    #[test]
    fn test_strategic_forgetting() {
        // Test low-salience region smoothing
    }
}

Test Categories:

✅ Gradient descent convergence
✅ Manifold deformation mechanics
✅ Forgetting kernel application
✅ Tensor Train compression (if enabled)
✅ SIREN layer functionality
✅ Fourier feature encoding

2.3 exo-hypergraph Tests

Module: Hypergraph substrate Test Focus: Hyperedge operations, topology queries, TDA

// tests/hypergraph_test.rs
#[cfg(test)]
mod hypergraph_tests {
    use exo_hypergraph::*;

    #[test]
    fn test_create_hyperedge() {
        let mut substrate = HypergraphSubstrate::new();

        // Add entities
        let e1 = substrate.add_entity("concept_a");
        let e2 = substrate.add_entity("concept_b");
        let e3 = substrate.add_entity("concept_c");

        // Create hyperedge
        let relation = Relation::new("connects");
        let hyperedge = substrate.create_hyperedge(
            &[e1, e2, e3],
            &relation
        ).unwrap();

        assert!(substrate.hyperedge_exists(hyperedge));
    }

    #[test]
    fn test_persistent_homology_0d() {
        // Test connected components (0-dim homology)
    }

    #[test]
    fn test_persistent_homology_1d() {
        // Test 1-dimensional holes (cycles)
    }

    #[test]
    fn test_betti_numbers() {
        // Test Betti number computation
    }

    #[test]
    fn test_sheaf_consistency() {
        // Test sheaf consistency check
    }
}

Test Categories:

✅ Hyperedge CRUD operations
✅ Entity index management
✅ Relation type indexing
✅ Persistent homology (0D, 1D, 2D)
✅ Betti number computation
✅ Sheaf consistency checks
✅ Simplicial complex operations

2.4 exo-temporal Tests

Module: Temporal memory coordinator Test Focus: Causal queries, consolidation, anticipation

// tests/temporal_memory_test.rs
#[cfg(test)]
mod temporal_tests {
    use exo_temporal::*;

    #[test]
    fn test_causal_cone_past() {
        let mut memory = TemporalMemory::new();

        // Store patterns with causal relationships
        let p1 = memory.store(pattern1, &[]).unwrap();
        let p2 = memory.store(pattern2, &[p1]).unwrap();
        let p3 = memory.store(pattern3, &[p2]).unwrap();

        // Query past cone
        let results = memory.causal_query(
            &query,
            SubstrateTime::now(),
            CausalConeType::Past
        );

        assert!(results.iter().all(|r| r.timestamp <= SubstrateTime::now()));
    }

    #[test]
    fn test_memory_consolidation() {
        // Test short-term to long-term consolidation
    }

    #[test]
    fn test_salience_computation() {
        // Test salience scoring
    }

    #[test]
    fn test_anticipatory_prefetch() {
        // Test predictive retrieval
    }
}

Test Categories:

✅ Causal cone queries (past, future, light-cone)
✅ Causal graph construction
✅ Memory consolidation logic
✅ Salience computation
✅ Anticipatory pre-fetch
✅ Temporal knowledge graph (TKG)
✅ Strategic decay

2.5 exo-federation Tests

Module: Federated cognitive mesh Test Focus: Consensus, CRDT, post-quantum crypto

// tests/federation_test.rs
#[cfg(test)]
mod federation_tests {
    use exo_federation::*;

    #[test]
    fn test_post_quantum_handshake() {
        let node1 = FederatedMesh::new(config1);
        let node2 = FederatedMesh::new(config2);

        let token = node1.join_federation(&node2.address()).await.unwrap();

        assert!(token.is_valid());
        assert!(token.has_shared_secret());
    }

    #[test]
    fn test_byzantine_consensus_sufficient_votes() {
        // Test consensus with 2f+1 agreement
    }

    #[test]
    fn test_byzantine_consensus_insufficient_votes() {
        // Test consensus failure with < 2f+1
    }

    #[test]
    fn test_crdt_reconciliation() {
        // Test conflict-free merge
    }

    #[test]
    fn test_onion_routing() {
        // Test privacy-preserving query routing
    }
}

Test Categories:

✅ Post-quantum key exchange (Kyber)
✅ Byzantine fault tolerance (PBFT)
✅ CRDT reconciliation (G-Set, LWW)
✅ Onion-routed queries
✅ Federation token management
✅ Encrypted channel operations

2.6 exo-backend-classical Tests

Module: Classical backend (ruvector integration) Test Focus: ruvector SDK consumption, trait implementation

// tests/classical_backend_test.rs
#[cfg(test)]
mod classical_backend_tests {
    use exo_backend_classical::*;
    use exo_core::SubstrateBackend;

    #[test]
    fn test_similarity_search() {
        let backend = ClassicalBackend::new(config);

        let query = vec![0.1, 0.2, 0.3, 0.4];
        let results = backend.similarity_search(&query, 10, None).unwrap();

        assert_eq!(results.len(), 10);
        // Verify ruvector integration
    }

    #[test]
    fn test_manifold_deform_as_insert() {
        // Test classical discrete insert
    }

    #[test]
    fn test_hyperedge_query_basic() {
        // Test basic hyperedge support
    }
}

Test Categories:

✅ ruvector-core integration
✅ ruvector-graph integration
✅ ruvector-gnn integration
✅ SubstrateBackend trait impl
✅ Error handling and conversion
✅ Filter support

3. Integration Tests

3.1 Cross-Crate Integration

// tests/integration/manifold_hypergraph_test.rs
#[test]
fn test_manifold_with_hypergraph() {
    // Test manifold engine with hypergraph substrate
    let backend = ClassicalBackend::new(config);
    let manifold = ManifoldEngine::new(backend.clone());
    let hypergraph = HypergraphSubstrate::new(backend);

    // Store patterns in manifold
    // Create hyperedges linking patterns
    // Query across both substrates
}

3.2 Temporal-Federation Integration

// tests/integration/temporal_federation_test.rs
#[test]
async fn test_federated_temporal_query() {
    // Test temporal queries across federation
    let node1 = setup_federated_node(config1);
    let node2 = setup_federated_node(config2);

    // Join federation
    // Store temporal patterns on node1
    // Query from node2 with causal constraints
}

4. Performance Tests

4.1 Benchmarks

// benches/manifold_bench.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn bench_manifold_retrieve(c: &mut Criterion) {
    let engine = setup_manifold_engine();
    let query = generate_random_query();

    c.bench_function("manifold_retrieve_k10", |b| {
        b.iter(|| engine.retrieve(black_box(query.clone()), 10))
    });
}

criterion_group!(benches, bench_manifold_retrieve);
criterion_main!(benches);

Benchmark Categories:

Manifold retrieval (k=1, 10, 100)
Hyperedge creation and query
Causal cone queries
Byzantine consensus latency
Memory consolidation throughput

4.2 Performance Targets

Operation	Target Latency	Target Throughput
Manifold Retrieve (k=10)	<10ms	>1000 qps
Hyperedge Creation	<1ms	>10000 ops/s
Causal Query	<20ms	>500 qps
Byzantine Commit	<100ms	>100 commits/s
Consolidation	<1s	Batch operation

5. Property-Based Testing

// tests/property/manifold_properties.rs
use proptest::prelude::*;

proptest! {
    #[test]
    fn prop_manifold_retrieve_always_returns_k_or_less(
        query in prop::collection::vec(any::<f32>(), 128),
        k in 1usize..100
    ) {
        let engine = setup_engine();
        let results = engine.retrieve(Tensor::from_floats(&query), k);
        prop_assert!(results.len() <= k);
    }

    #[test]
    fn prop_hyperedge_creation_preserves_entities(
        entities in prop::collection::vec(any::<u64>(), 2..10)
    ) {
        let mut substrate = HypergraphSubstrate::new();
        let hyperedge = substrate.create_hyperedge(&entities, &Relation::default())?;
        let retrieved = substrate.get_hyperedge_entities(hyperedge)?;
        prop_assert_eq!(entities, retrieved);
    }
}

6. Security Tests

6.1 Cryptographic Tests

// tests/security/crypto_test.rs
#[test]
fn test_kyber_key_exchange_correctness() {
    // Test post-quantum key exchange produces same shared secret
}

#[test]
fn test_onion_routing_privacy() {
    // Test intermediate nodes cannot decrypt payload
}

6.2 Fuzzing Targets

// fuzz/fuzz_targets/manifold_input.rs
#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    if data.len() % 4 == 0 {
        let floats: Vec<f32> = data.chunks_exact(4)
            .map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]]))
            .collect();

        let engine = setup_engine();
        let _ = engine.retrieve(Tensor::from_floats(&floats), 10);
    }
});

7. Test Execution Plan

7.1 CI/CD Pipeline

# .github/workflows/test.yml
name: Test Suite

on: [push, pull_request]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: dtolnay/rust-toolchain@stable
      - run: cargo test --all-features

  integration-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: cargo test --test '*' --all-features

  benchmarks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: cargo bench --all-features

  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: cargo tarpaulin --all-features --out Lcov
      - uses: coverallsapp/github-action@master

7.2 Local Test Commands

# Run all tests
cargo test --all-features

# Run tests for specific crate
cargo test -p exo-manifold

# Run with coverage
cargo tarpaulin --all-features

# Run benchmarks
cargo bench

# Run property tests
cargo test --features proptest

# Run security tests
cargo test --test security_*

8. Test Data Management

8.1 Fixtures

// tests/fixtures/mod.rs
pub fn sample_pattern() -> Pattern {
    Pattern {
        embedding: vec![0.1, 0.2, 0.3, 0.4],
        metadata: Metadata::default(),
        timestamp: SubstrateTime::from_unix(1000),
        antecedents: vec![],
    }
}

pub fn sample_hypergraph() -> HypergraphSubstrate {
    let mut substrate = HypergraphSubstrate::new();
    // Populate with test data
    substrate
}

8.2 Mock Backends

// tests/mocks/mock_backend.rs
pub struct MockSubstrateBackend {
    responses: HashMap<Query, Vec<SearchResult>>,
}

impl SubstrateBackend for MockSubstrateBackend {
    type Error = MockError;

    fn similarity_search(&self, query: &[f32], k: usize, _: Option<&Filter>)
        -> Result<Vec<SearchResult>, Self::Error>
    {
        Ok(self.responses.get(query).cloned().unwrap_or_default())
    }
}

9. Test Metrics & Reporting

9.1 Coverage Reports

# Generate HTML coverage report
cargo tarpaulin --all-features --out Html --output-dir coverage/

# View coverage
open coverage/index.html

9.2 Test Result Dashboard

Jenkins/GitHub Actions: Automated test runs
Coverage Tracking: Coveralls/Codecov integration
Performance Tracking: Criterion benchmark graphs
Security Scanning: Cargo audit in CI

10. Testing Schedule

Phase 1: Core Foundation (Week 1-2)

✅ exo-core unit tests
✅ Basic trait implementations
✅ Type validation

Phase 2: Substrate Components (Week 3-4)

✅ exo-manifold tests
✅ exo-hypergraph tests
✅ exo-temporal tests

Phase 3: Distribution (Week 5-6)

✅ exo-federation tests
✅ Integration tests
✅ Performance benchmarks

Phase 4: Optimization (Week 7-8)

✅ Property-based tests
✅ Fuzzing campaigns
✅ Security audits

11. Test Maintenance

11.1 Test Review Checklist

All public APIs have unit tests
Integration tests cover cross-crate interactions
Performance benchmarks exist for critical paths
Error cases are tested
Edge cases are covered
Tests are deterministic (no flaky tests)
Test names clearly describe what is tested
Test data is documented

11.2 Continuous Improvement

Weekly: Review test coverage reports
Monthly: Update performance baselines
Quarterly: Security audit and fuzzing campaigns

15 KiB Raw Permalink Blame History

EXO-AI 2025: Comprehensive Test Strategy

Test Agent Status

Overview

1. Test Pyramid Architecture

Coverage Targets

2. Per-Crate Test Strategy

2.1 exo-core Tests

2.2 exo-manifold Tests

2.3 exo-hypergraph Tests

2.4 exo-temporal Tests

2.5 exo-federation Tests

2.6 exo-backend-classical Tests

3. Integration Tests

3.1 Cross-Crate Integration

3.2 Temporal-Federation Integration

4. Performance Tests

4.1 Benchmarks

4.2 Performance Targets

5. Property-Based Testing

6. Security Tests

6.1 Cryptographic Tests

6.2 Fuzzing Targets

7. Test Execution Plan

7.1 CI/CD Pipeline

7.2 Local Test Commands

8. Test Data Management

8.1 Fixtures

8.2 Mock Backends

9. Test Metrics & Reporting

9.1 Coverage Reports

9.2 Test Result Dashboard

10. Testing Schedule

Phase 1: Core Foundation (Week 1-2)

Phase 2: Substrate Components (Week 3-4)

Phase 3: Distribution (Week 5-6)

Phase 4: Optimization (Week 7-8)

11. Test Maintenance

11.1 Test Review Checklist

11.2 Continuous Improvement

References

15 KiB

Raw Permalink Blame History