Merge commit 'd803bfe2b1fe7f5e219e50ac20d6801a0a58ac75' as 'vendor/ruvector'

vendor/ruvector/examples/exo-ai-2025/architecture/ARCHITECTURE.md

@@ -0,0 +1,805 @@
+# EXO-AI 2025: System Architecture
+
+## SPARC Phase 3: Architecture Design
+
+### Executive Summary
+
+This document defines the modular architecture for an experimental cognitive substrate platform, consuming the ruvector ecosystem as an SDK while exploring technologies projected for 2035-2060.
+
+---
+
+## 1. Architectural Principles
+
+### 1.1 Core Design Tenets
+
+| Principle | Description | Implementation |
+|-----------|-------------|----------------|
+| **SDK Consumer** | No modifications to ruvector crates | Clean dependency boundaries |
+| **Backend Agnostic** | Hardware abstraction via traits | PIM, neuromorphic, photonic backends |
+| **Substrate-First** | Data and compute unified | In-memory operations where possible |
+| **Topology Native** | Hypergraph as primary structure | Edges span arbitrary entity sets |
+| **Temporal Coherent** | Causal memory by default | Every operation timestamped |
+
+### 1.2 Layer Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    APPLICATION LAYER                             │
+│  ┌─────────────┐ ┌──────────────┐ ┌───────────────────────────┐ │
+│  │ Agent SDK   │ │ Query Engine │ │ Federation Gateway        │ │
+│  └─────────────┘ └──────────────┘ └───────────────────────────┘ │
+├─────────────────────────────────────────────────────────────────┤
+│                    SUBSTRATE LAYER                               │
+│  ┌─────────────┐ ┌──────────────┐ ┌───────────────────────────┐ │
+│  │ Manifold    │ │ Hypergraph   │ │ Temporal Memory           │ │
+│  │ Engine      │ │ Substrate    │ │ Coordinator               │ │
+│  └─────────────┘ └──────────────┘ └───────────────────────────┘ │
+├─────────────────────────────────────────────────────────────────┤
+│                    BACKEND ABSTRACTION                           │
+│  ┌─────────────┐ ┌──────────────┐ ┌───────────────────────────┐ │
+│  │ Classical   │ │ Neuromorphic │ │ Photonic                  │ │
+│  │ (ruvector)  │ │ (Future)     │ │ (Future)                  │ │
+│  └─────────────┘ └──────────────┘ └───────────────────────────┘ │
+├─────────────────────────────────────────────────────────────────┤
+│                    INFRASTRUCTURE                                │
+│  ┌─────────────┐ ┌──────────────┐ ┌───────────────────────────┐ │
+│  │ WASM        │ │ NAPI-RS      │ │ Native                    │ │
+│  │ Runtime     │ │ Bindings     │ │ Binaries                  │ │
+│  └─────────────┘ └──────────────┘ └───────────────────────────┘ │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 2. Module Design
+
+### 2.1 Core Modules
+
+```
+exo-ai-2025/
+├── crates/
+│   ├── exo-core/              # Core traits and types
+│   ├── exo-manifold/          # Learned manifold engine
+│   ├── exo-hypergraph/        # Hypergraph substrate
+│   ├── exo-temporal/          # Temporal memory coordinator
+│   ├── exo-federation/        # Federated mesh networking
+│   ├── exo-backend-classical/ # Classical backend (ruvector)
+│   ├── exo-backend-sim/       # Neuromorphic/photonic simulator
+│   ├── exo-wasm/              # WASM bindings
+│   └── exo-node/              # NAPI-RS bindings
+├── examples/
+├── docs/
+└── research/
+```
+
+### 2.2 exo-core: Foundational Traits
+
+```rust
+//! Core trait definitions for backend abstraction
+
+/// Backend trait for substrate compute operations
+pub trait SubstrateBackend: Send + Sync {
+    type Error: std::error::Error;
+
+    /// Execute similarity search on substrate
+    fn similarity_search(
+        &self,
+        query: &[f32],
+        k: usize,
+        filter: Option<&Filter>,
+    ) -> Result<Vec<SearchResult>, Self::Error>;
+
+    /// Deform manifold to incorporate new pattern
+    fn manifold_deform(
+        &self,
+        pattern: &Pattern,
+        learning_rate: f32,
+    ) -> Result<ManifoldDelta, Self::Error>;
+
+    /// Execute hyperedge query
+    fn hyperedge_query(
+        &self,
+        query: &TopologicalQuery,
+    ) -> Result<HyperedgeResult, Self::Error>;
+}
+
+/// Temporal context for causal operations
+pub trait TemporalContext {
+    /// Get current substrate time
+    fn now(&self) -> SubstrateTime;
+
+    /// Query with causal cone constraints
+    fn causal_query(
+        &self,
+        query: &Query,
+        cone: &CausalCone,
+    ) -> Result<Vec<CausalResult>, Error>;
+
+    /// Predictive pre-fetch based on anticipated queries
+    fn anticipate(&self, hints: &[AnticipationHint]) -> Result<(), Error>;
+}
+
+/// Pattern representation in substrate
+#[derive(Clone, Debug)]
+pub struct Pattern {
+    /// Vector embedding
+    pub embedding: Vec<f32>,
+    /// Metadata
+    pub metadata: Metadata,
+    /// Temporal origin
+    pub timestamp: SubstrateTime,
+    /// Causal antecedents
+    pub antecedents: Vec<PatternId>,
+}
+
+/// Topological query specification
+#[derive(Clone, Debug)]
+pub enum TopologicalQuery {
+    /// Find persistent homology features
+    PersistentHomology {
+        dimension: usize,
+        epsilon_range: (f32, f32),
+    },
+    /// Find N-dimensional holes in structure
+    BettiNumbers {
+        max_dimension: usize,
+    },
+    /// Sheaf consistency check
+    SheafConsistency {
+        local_sections: Vec<SectionId>,
+    },
+}
+```
+
+### 2.3 exo-manifold: Learned Representation Engine
+
+```rust
+//! Continuous manifold storage replacing discrete indices
+
+use burn::prelude::*;
+use crate::core::{Pattern, SubstrateBackend, ManifoldDelta};
+
+/// Implicit Neural Representation for manifold storage
+pub struct ManifoldEngine<B: Backend> {
+    /// Neural network representing the manifold
+    network: LearnedManifold<B>,
+    /// Tensor Train decomposition for compression
+    tt_decomposition: Option<TensorTrainConfig>,
+    /// Consolidation scheduler
+    consolidation: ConsolidationPolicy,
+}
+
+impl<B: Backend> ManifoldEngine<B> {
+    /// Query manifold via gradient descent
+    pub fn retrieve(
+        &self,
+        query: Tensor<B, 1>,
+        k: usize,
+    ) -> Vec<(Pattern, f32)> {
+        // Initialize at query position
+        let mut position = query.clone();
+
+        // Gradient descent toward relevant memories
+        for _ in 0..self.config.max_descent_steps {
+            let relevance = self.network.forward(position.clone());
+            let gradient = relevance.backward();
+            position = position - self.config.learning_rate * gradient;
+
+            if gradient.norm() < self.config.convergence_threshold {
+                break;
+            }
+        }
+
+        // Extract patterns from converged region
+        self.extract_patterns_near(position, k)
+    }
+
+    /// Continuous manifold deformation (replaces insert)
+    pub fn deform(&mut self, pattern: Pattern, salience: f32) {
+        let embedding = Tensor::from_floats(&pattern.embedding);
+
+        // Deformation = gradient update to manifold weights
+        let loss = self.deformation_loss(embedding, salience);
+        let gradients = loss.backward();
+
+        self.optimizer.step(gradients);
+    }
+
+    /// Strategic forgetting via manifold smoothing
+    pub fn forget(&mut self, region: &ManifoldRegion, decay_rate: f32) {
+        // Smooth the manifold in low-salience regions
+        self.apply_forgetting_kernel(region, decay_rate);
+    }
+}
+
+/// Learned manifold network architecture
+#[derive(Module)]
+pub struct LearnedManifold<B: Backend> {
+    /// SIREN-style sinusoidal layers
+    layers: Vec<SirenLayer<B>>,
+    /// Fourier feature encoding
+    fourier_features: FourierEncoding<B>,
+}
+```
+
+### 2.4 exo-hypergraph: Topological Substrate
+
+```rust
+//! Hypergraph substrate for higher-order relations
+
+use petgraph::Graph;
+use simplicial_topology::SimplicialComplex;
+use ruvector_graph::{GraphDatabase, HyperedgeSupport};
+
+/// Hypergraph substrate extending ruvector-graph
+pub struct HypergraphSubstrate {
+    /// Base graph from ruvector-graph
+    base: GraphDatabase,
+    /// Hyperedge index (relations spanning >2 entities)
+    hyperedges: HyperedgeIndex,
+    /// Simplicial complex for TDA
+    topology: SimplicialComplex,
+    /// Sheaf structure for consistency
+    sheaf: Option<SheafStructure>,
+}
+
+impl HypergraphSubstrate {
+    /// Create hyperedge spanning multiple entities
+    pub fn create_hyperedge(
+        &mut self,
+        entities: &[EntityId],
+        relation: &Relation,
+    ) -> Result<HyperedgeId, Error> {
+        // Validate entity existence
+        for entity in entities {
+            self.base.get_node(*entity)?;
+        }
+
+        // Create hyperedge in index
+        let hyperedge_id = self.hyperedges.insert(entities, relation);
+
+        // Update simplicial complex
+        self.topology.add_simplex(entities);
+
+        // Update sheaf sections if enabled
+        if let Some(ref mut sheaf) = self.sheaf {
+            sheaf.update_sections(hyperedge_id, entities)?;
+        }
+
+        Ok(hyperedge_id)
+    }
+
+    /// Topological query: find persistent features
+    pub fn persistent_homology(
+        &self,
+        dimension: usize,
+        epsilon_range: (f32, f32),
+    ) -> PersistenceDiagram {
+        use teia::persistence::compute_persistence;
+
+        let filtration = self.topology.filtration(epsilon_range);
+        compute_persistence(&filtration, dimension)
+    }
+
+    /// Query Betti numbers (topological invariants)
+    pub fn betti_numbers(&self, max_dim: usize) -> Vec<usize> {
+        (0..=max_dim)
+            .map(|d| self.topology.betti_number(d))
+            .collect()
+    }
+
+    /// Sheaf consistency: check local-to-global coherence
+    pub fn check_sheaf_consistency(
+        &self,
+        sections: &[SectionId],
+    ) -> SheafConsistencyResult {
+        match &self.sheaf {
+            Some(sheaf) => sheaf.check_consistency(sections),
+            None => SheafConsistencyResult::NotConfigured,
+        }
+    }
+}
+
+/// Hyperedge index structure
+struct HyperedgeIndex {
+    /// Hyperedge storage
+    edges: DashMap<HyperedgeId, Hyperedge>,
+    /// Inverted index: entity -> hyperedges containing it
+    entity_index: DashMap<EntityId, Vec<HyperedgeId>>,
+    /// Relation type index
+    relation_index: DashMap<RelationType, Vec<HyperedgeId>>,
+}
+```
+
+### 2.5 exo-temporal: Causal Memory Coordinator
+
+```rust
+//! Temporal memory with causal structure
+
+use std::collections::BTreeMap;
+use ruvector_core::VectorIndex;
+
+/// Temporal memory coordinator
+pub struct TemporalMemory {
+    /// Short-term volatile memory
+    short_term: ShortTermBuffer,
+    /// Long-term consolidated memory
+    long_term: LongTermStore,
+    /// Causal graph tracking antecedent relationships
+    causal_graph: CausalGraph,
+    /// Temporal knowledge graph (Zep-inspired)
+    tkg: TemporalKnowledgeGraph,
+}
+
+impl TemporalMemory {
+    /// Store with causal context
+    pub fn store(
+        &mut self,
+        pattern: Pattern,
+        antecedents: &[PatternId],
+    ) -> Result<PatternId, Error> {
+        // Add to short-term buffer
+        let id = self.short_term.insert(pattern.clone());
+
+        // Record causal relationships
+        for antecedent in antecedents {
+            self.causal_graph.add_edge(*antecedent, id);
+        }
+
+        // Update TKG with temporal relations
+        self.tkg.add_temporal_fact(id, &pattern, antecedents)?;
+
+        // Schedule consolidation if buffer full
+        if self.short_term.should_consolidate() {
+            self.trigger_consolidation();
+        }
+
+        Ok(id)
+    }
+
+    /// Causal cone query: retrieve within light-cone constraints
+    pub fn causal_query(
+        &self,
+        query: &Query,
+        reference_time: SubstrateTime,
+        cone_type: CausalConeType,
+    ) -> Vec<CausalResult> {
+        // Determine valid time range based on cone
+        let time_range = match cone_type {
+            CausalConeType::Past => (SubstrateTime::MIN, reference_time),
+            CausalConeType::Future => (reference_time, SubstrateTime::MAX),
+            CausalConeType::LightCone { velocity } => {
+                self.compute_light_cone(reference_time, velocity)
+            }
+        };
+
+        // Query with temporal filter
+        self.long_term
+            .search_with_time_range(query, time_range)
+            .into_iter()
+            .map(|r| CausalResult {
+                pattern: r.pattern,
+                causal_distance: self.causal_graph.distance(r.id, query.origin),
+                temporal_distance: (r.timestamp - reference_time).abs(),
+            })
+            .collect()
+    }
+
+    /// Anticipatory pre-fetch for predictive retrieval
+    pub fn anticipate(&mut self, hints: &[AnticipationHint]) {
+        for hint in hints {
+            // Pre-compute likely future queries
+            let predicted_queries = self.predict_future_queries(hint);
+
+            // Warm cache with predicted results
+            for query in predicted_queries {
+                self.prefetch_cache.insert(query.hash(),
+                    self.long_term.search(&query));
+            }
+        }
+    }
+
+    /// Memory consolidation: short-term -> long-term
+    fn consolidate(&mut self) {
+        // Identify salient patterns
+        let salient = self.short_term
+            .drain()
+            .filter(|p| p.salience > self.consolidation_threshold);
+
+        // Compress via manifold integration
+        for pattern in salient {
+            self.long_term.integrate(pattern);
+        }
+
+        // Strategic forgetting in long-term
+        self.long_term.decay_low_salience(self.decay_rate);
+    }
+}
+
+/// Causal graph for tracking antecedent relationships
+struct CausalGraph {
+    /// Forward edges: cause -> effects
+    forward: DashMap<PatternId, Vec<PatternId>>,
+    /// Backward edges: effect -> causes
+    backward: DashMap<PatternId, Vec<PatternId>>,
+}
+```
+
+### 2.6 exo-federation: Distributed Cognitive Mesh
+
+```rust
+//! Federated substrate with cryptographic sovereignty
+
+use ruvector_raft::{RaftNode, RaftConfig};
+use ruvector_cluster::ClusterManager;
+use kyberlib::{keypair, encapsulate, decapsulate};
+
+/// Federated cognitive mesh
+pub struct FederatedMesh {
+    /// Local substrate instance
+    local: Arc<SubstrateInstance>,
+    /// Raft consensus for local cluster
+    consensus: RaftNode,
+    /// Federation gateway
+    gateway: FederationGateway,
+    /// Post-quantum keypair
+    pq_keys: PostQuantumKeypair,
+}
+
+impl FederatedMesh {
+    /// Join federation with cryptographic handshake
+    pub async fn join_federation(
+        &mut self,
+        peer: &PeerAddress,
+    ) -> Result<FederationToken, Error> {
+        // Post-quantum key exchange
+        let (ciphertext, shared_secret) = encapsulate(&peer.public_key)?;
+
+        // Establish encrypted channel
+        let channel = self.gateway.establish_channel(
+            peer,
+            ciphertext,
+            shared_secret,
+        ).await?;
+
+        // Exchange federation capabilities
+        let token = channel.negotiate_federation().await?;
+
+        Ok(token)
+    }
+
+    /// Federated query with privacy preservation
+    pub async fn federated_query(
+        &self,
+        query: &Query,
+        scope: FederationScope,
+    ) -> Vec<FederatedResult> {
+        // Route through onion network for intent privacy
+        let onion_query = self.gateway.onion_wrap(query, scope)?;
+
+        // Broadcast to federation peers
+        let responses = self.gateway.broadcast(onion_query).await;
+
+        // CRDT reconciliation for eventual consistency
+        let reconciled = self.reconcile_crdt(responses)?;
+
+        reconciled
+    }
+
+    /// Byzantine fault tolerant consensus on shared state
+    pub async fn byzantine_commit(
+        &self,
+        update: &StateUpdate,
+    ) -> Result<CommitProof, Error> {
+        // Require 2f+1 agreement for n=3f+1 nodes
+        let threshold = (self.peer_count() * 2 / 3) + 1;
+
+        // Propose update
+        let proposal = self.consensus.propose(update)?;
+
+        // Collect votes
+        let votes = self.gateway.collect_votes(proposal).await;
+
+        if votes.len() >= threshold {
+            Ok(CommitProof::from_votes(votes))
+        } else {
+            Err(Error::InsufficientConsensus)
+        }
+    }
+}
+
+/// Post-quantum cryptographic keypair
+struct PostQuantumKeypair {
+    /// CRYSTALS-Kyber public key
+    public: [u8; 1184],
+    /// CRYSTALS-Kyber secret key
+    secret: [u8; 2400],
+}
+```
+
+---
+
+## 3. Backend Abstraction Layer
+
+### 3.1 Classical Backend (ruvector SDK)
+
+```rust
+//! Classical backend consuming ruvector crates
+
+use ruvector_core::{VectorIndex, HnswConfig};
+use ruvector_graph::GraphDatabase;
+use ruvector_gnn::GnnLayer;
+
+/// Classical substrate backend using ruvector
+pub struct ClassicalBackend {
+    /// Vector index from ruvector-core
+    vector_index: VectorIndex,
+    /// Graph database from ruvector-graph
+    graph_db: GraphDatabase,
+    /// GNN layer from ruvector-gnn
+    gnn: Option<GnnLayer>,
+}
+
+impl SubstrateBackend for ClassicalBackend {
+    type Error = ruvector_core::Error;
+
+    fn similarity_search(
+        &self,
+        query: &[f32],
+        k: usize,
+        filter: Option<&Filter>,
+    ) -> Result<Vec<SearchResult>, Self::Error> {
+        // Direct delegation to ruvector-core
+        let results = match filter {
+            Some(f) => self.vector_index.search_with_filter(query, k, f)?,
+            None => self.vector_index.search(query, k)?,
+        };
+
+        Ok(results.into_iter().map(SearchResult::from).collect())
+    }
+
+    fn manifold_deform(
+        &self,
+        pattern: &Pattern,
+        _learning_rate: f32,
+    ) -> Result<ManifoldDelta, Self::Error> {
+        // Classical backend: discrete insert
+        let id = self.vector_index.insert(&pattern.embedding, &pattern.metadata)?;
+
+        Ok(ManifoldDelta::DiscreteInsert { id })
+    }
+
+    fn hyperedge_query(
+        &self,
+        query: &TopologicalQuery,
+    ) -> Result<HyperedgeResult, Self::Error> {
+        // Use ruvector-graph hyperedge support
+        match query {
+            TopologicalQuery::PersistentHomology { .. } => {
+                // Compute via graph traversal
+                unimplemented!("TDA on classical backend")
+            }
+            TopologicalQuery::BettiNumbers { .. } => {
+                // Approximate via connected components
+                unimplemented!("Betti numbers on classical backend")
+            }
+            TopologicalQuery::SheafConsistency { .. } => {
+                // Not supported on classical backend
+                Ok(HyperedgeResult::NotSupported)
+            }
+        }
+    }
+}
+```
+
+### 3.2 Future Backend Traits
+
+```rust
+//! Placeholder traits for future hardware backends
+
+/// Processing-in-Memory backend interface
+pub trait PimBackend: SubstrateBackend {
+    /// Execute operation in memory bank
+    fn execute_in_memory(&self, op: &MemoryOperation) -> Result<(), Error>;
+
+    /// Query memory bank location for data
+    fn data_location(&self, pattern_id: PatternId) -> MemoryBank;
+}
+
+/// Neuromorphic backend interface
+pub trait NeuromorphicBackend: SubstrateBackend {
+    /// Encode vector as spike train
+    fn encode_spikes(&self, vector: &[f32]) -> SpikeTrain;
+
+    /// Decode spike train to vector
+    fn decode_spikes(&self, spikes: &SpikeTrain) -> Vec<f32>;
+
+    /// Submit spike computation
+    fn submit_spike_compute(&self, input: SpikeTrain) -> Result<SpikeTrain, Error>;
+}
+
+/// Photonic backend interface
+pub trait PhotonicBackend: SubstrateBackend {
+    /// Optical matrix-vector multiply
+    fn optical_matmul(&self, matrix: &OpticalMatrix, vector: &[f32]) -> Vec<f32>;
+
+    /// Configure optical interference pattern
+    fn configure_mzi(&self, config: &MziConfig) -> Result<(), Error>;
+}
+```
+
+---
+
+## 4. WASM & NAPI-RS Integration
+
+### 4.1 WASM Module Structure
+
+```rust
+//! WASM bindings for browser/edge deployment
+
+use wasm_bindgen::prelude::*;
+use crate::core::{Pattern, Query};
+
+#[wasm_bindgen]
+pub struct ExoSubstrate {
+    inner: Arc<SubstrateInstance>,
+}
+
+#[wasm_bindgen]
+impl ExoSubstrate {
+    #[wasm_bindgen(constructor)]
+    pub fn new(config: JsValue) -> Result<ExoSubstrate, JsError> {
+        let config: SubstrateConfig = serde_wasm_bindgen::from_value(config)?;
+        let inner = SubstrateInstance::new(config)?;
+        Ok(Self { inner: Arc::new(inner) })
+    }
+
+    #[wasm_bindgen]
+    pub async fn query(&self, embedding: Float32Array, k: u32) -> Result<JsValue, JsError> {
+        let query = Query::from_embedding(embedding.to_vec());
+        let results = self.inner.search(query, k as usize).await?;
+        Ok(serde_wasm_bindgen::to_value(&results)?)
+    }
+
+    #[wasm_bindgen]
+    pub fn store(&self, pattern: JsValue) -> Result<String, JsError> {
+        let pattern: Pattern = serde_wasm_bindgen::from_value(pattern)?;
+        let id = self.inner.store(pattern)?;
+        Ok(id.to_string())
+    }
+}
+```
+
+### 4.2 NAPI-RS Bindings
+
+```rust
+//! Node.js bindings via NAPI-RS
+
+use napi::bindgen_prelude::*;
+use napi_derive::napi;
+
+#[napi]
+pub struct ExoSubstrateNode {
+    inner: Arc<RwLock<SubstrateInstance>>,
+}
+
+#[napi]
+impl ExoSubstrateNode {
+    #[napi(constructor)]
+    pub fn new(config: serde_json::Value) -> Result<Self> {
+        let config: SubstrateConfig = serde_json::from_value(config)?;
+        let inner = SubstrateInstance::new(config)?;
+        Ok(Self { inner: Arc::new(RwLock::new(inner)) })
+    }
+
+    #[napi]
+    pub async fn search(&self, embedding: Float32Array, k: u32) -> Result<Vec<SearchResultJs>> {
+        let guard = self.inner.read().await;
+        let results = guard.search(
+            Query::from_embedding(embedding.to_vec()),
+            k as usize,
+        ).await?;
+        Ok(results.into_iter().map(SearchResultJs::from).collect())
+    }
+
+    #[napi]
+    pub async fn hypergraph_query(&self, query: String) -> Result<serde_json::Value> {
+        let guard = self.inner.read().await;
+        let topo_query: TopologicalQuery = serde_json::from_str(&query)?;
+        let result = guard.hypergraph.query(&topo_query).await?;
+        Ok(serde_json::to_value(result)?)
+    }
+}
+```
+
+---
+
+## 5. Deployment Targets
+
+### 5.1 Build Configurations
+
+```toml
+# Cargo.toml feature flags
+
+[features]
+default = ["classical-backend"]
+
+# Backends
+classical-backend = ["ruvector-core", "ruvector-graph", "ruvector-gnn"]
+sim-neuromorphic = []
+sim-photonic = []
+
+# Deployment targets
+wasm = ["wasm-bindgen", "getrandom/js"]
+napi = ["napi", "napi-derive"]
+
+# Experimental features
+tensor-train = []
+sheaf-consistency = []
+post-quantum = ["kyberlib", "pqcrypto"]
+```
+
+### 5.2 Platform Matrix
+
+| Target | Backend | Features | Size |
+|--------|---------|----------|------|
+| `wasm32-unknown-unknown` | Classical (memory-only) | Core substrate | ~2MB |
+| `x86_64-unknown-linux-gnu` | Classical (full) | All features | ~15MB |
+| `aarch64-apple-darwin` | Classical (full) | All features | ~12MB |
+| Node.js (NAPI) | Classical (full) | All features | ~8MB |
+
+---
+
+## 6. Future Architecture Extensions
+
+### 6.1 PIM Integration Path
+
+```
+Phase 1: Abstraction (Current)
+├── Define PimBackend trait
+├── Implement simulation mode
+└── Profile classical baseline
+
+Phase 2: Emulation
+├── UPMEM SDK integration
+├── Performance modeling
+└── Hybrid execution strategies
+
+Phase 3: Native Hardware
+├── Custom PIM firmware
+├── Memory bank allocation
+└── Direct execution path
+```
+
+### 6.2 Consciousness Metrics (Research)
+
+```rust
+//! Experimental: Integrated Information metrics
+
+/// Compute Phi (integrated information) for substrate region
+pub fn compute_phi(
+    substrate: &SubstrateRegion,
+    partition_strategy: PartitionStrategy,
+) -> f64 {
+    // Compute information generated by whole
+    let whole_info = substrate.effective_information();
+
+    // Compute information generated by parts
+    let partitions = partition_strategy.partition(substrate);
+    let parts_info: f64 = partitions
+        .iter()
+        .map(|p| p.effective_information())
+        .sum();
+
+    // Phi = whole - parts (simplified IIT measure)
+    (whole_info - parts_info).max(0.0)
+}
+```
+
+---
+
+## References
+
+- SPARC Specification: `specs/SPECIFICATION.md`
+- Research Papers: `research/PAPERS.md`
+- Rust Libraries: `research/RUST_LIBRARIES.md`

vendor/ruvector/examples/exo-ai-2025/architecture/PSEUDOCODE.md

@@ -0,0 +1,645 @@
+# EXO-AI 2025: Pseudocode Design
+
+## SPARC Phase 2: Algorithm Design
+
+This document presents high-level pseudocode for the core algorithms in the EXO-AI cognitive substrate.
+
+---
+
+## 1. Learned Manifold Engine
+
+### 1.1 Manifold Retrieval via Gradient Descent
+
+```pseudocode
+FUNCTION ManifoldRetrieve(query_vector, k, manifold_network):
+    // Initialize search position at query
+    position = query_vector
+    visited_positions = []
+
+    // Gradient descent toward high-relevance regions
+    FOR step IN 1..MAX_DESCENT_STEPS:
+        // Forward pass through learned manifold
+        relevance_field = manifold_network.forward(position)
+
+        // Compute gradient of relevance
+        gradient = manifold_network.backward(relevance_field)
+
+        // Update position following relevance gradient
+        position = position - LEARNING_RATE * gradient
+        visited_positions.append(position)
+
+        // Check convergence
+        IF norm(gradient) < CONVERGENCE_THRESHOLD:
+            BREAK
+
+    // Extract k nearest patterns from converged region
+    results = []
+    FOR pos IN visited_positions.last(k):
+        patterns = ExtractPatternsNear(pos, manifold_network)
+        results.extend(patterns)
+
+    RETURN TopK(results, k)
+```
+
+### 1.2 Continuous Manifold Deformation
+
+```pseudocode
+FUNCTION ManifoldDeform(pattern, salience, manifold_network, optimizer):
+    // No discrete insert - continuous deformation instead
+
+    // Encode pattern as tensor
+    embedding = Tensor(pattern.embedding)
+
+    // Compute deformation loss
+    // Loss = how much the manifold needs to change to represent this pattern
+    current_relevance = manifold_network.forward(embedding)
+    target_relevance = salience
+    deformation_loss = (current_relevance - target_relevance)^2
+
+    // Add regularization for manifold smoothness
+    smoothness_loss = ManifoldCurvatureRegularizer(manifold_network)
+    total_loss = deformation_loss + LAMBDA * smoothness_loss
+
+    // Gradient update to manifold weights
+    gradients = total_loss.backward()
+    optimizer.step(gradients)
+
+    // Return delta for logging
+    RETURN ManifoldDelta(embedding, salience, total_loss)
+```
+
+### 1.3 Strategic Forgetting
+
+```pseudocode
+FUNCTION StrategicForget(manifold_network, decay_rate, salience_threshold):
+    // Identify low-salience regions
+    low_salience_regions = []
+
+    FOR region IN manifold_network.sample_regions():
+        avg_salience = ComputeAverageSalience(region)
+        IF avg_salience < salience_threshold:
+            low_salience_regions.append(region)
+
+    // Apply smoothing kernel to low-salience regions
+    // This effectively "forgets" by reducing specificity
+    FOR region IN low_salience_regions:
+        ForgetKernel = GaussianKernel(sigma=decay_rate)
+        manifold_network.apply_kernel(region, ForgetKernel)
+
+    // Optional: prune near-zero weights
+    manifold_network.prune_weights(threshold=1e-6)
+```
+
+---
+
+## 2. Hypergraph Substrate
+
+### 2.1 Hyperedge Creation
+
+```pseudocode
+FUNCTION CreateHyperedge(entities, relation, hypergraph):
+    // Validate all entities exist
+    FOR entity IN entities:
+        IF NOT hypergraph.base_graph.contains(entity):
+            RAISE EntityNotFoundError(entity)
+
+    // Generate hyperedge ID
+    hyperedge_id = GenerateUUID()
+
+    // Create hyperedge record
+    hyperedge = Hyperedge(
+        id = hyperedge_id,
+        entities = entities,
+        relation = relation,
+        created_at = NOW(),
+        weight = 1.0
+    )
+
+    // Insert into hyperedge storage
+    hypergraph.hyperedges.insert(hyperedge_id, hyperedge)
+
+    // Update inverted index (entity -> hyperedges)
+    FOR entity IN entities:
+        hypergraph.entity_index[entity].append(hyperedge_id)
+
+    // Update relation type index
+    hypergraph.relation_index[relation.type].append(hyperedge_id)
+
+    // Update simplicial complex for TDA
+    simplex = entities.as_simplex()
+    hypergraph.topology.add_simplex(simplex)
+
+    RETURN hyperedge_id
+```
+
+### 2.2 Persistent Homology Computation
+
+```pseudocode
+FUNCTION ComputePersistentHomology(hypergraph, dimension, epsilon_range):
+    // Build filtration (nested sequence of simplicial complexes)
+    filtration = BuildFiltration(hypergraph.topology, epsilon_range)
+
+    // Initialize boundary matrix for column reduction
+    boundary_matrix = BuildBoundaryMatrix(filtration, dimension)
+
+    // Column reduction algorithm (standard persistent homology)
+    reduced_matrix = ColumnReduction(boundary_matrix)
+
+    // Extract persistence pairs
+    pairs = []
+    FOR col_j IN reduced_matrix.columns:
+        IF reduced_matrix.low(j) != NULL:
+            i = reduced_matrix.low(j)
+            birth = filtration.birth_time(i)
+            death = filtration.birth_time(j)
+            pairs.append((birth, death))
+        ELSE IF col_j is a cycle:
+            birth = filtration.birth_time(j)
+            death = INFINITY  // Essential feature
+            pairs.append((birth, death))
+
+    // Build persistence diagram
+    diagram = PersistenceDiagram(
+        pairs = pairs,
+        dimension = dimension
+    )
+
+    RETURN diagram
+
+FUNCTION ColumnReduction(matrix):
+    // Standard algorithm from computational topology
+    FOR j IN 1..matrix.num_cols:
+        WHILE EXISTS j' < j WITH low(j') = low(j):
+            // Add column j' to column j to reduce
+            matrix.column(j) = matrix.column(j) XOR matrix.column(j')
+    RETURN matrix
+```
+
+### 2.3 Sheaf Consistency Check
+
+```pseudocode
+FUNCTION CheckSheafConsistency(sheaf, sections):
+    // Sheaf consistency: local sections should agree on overlaps
+
+    inconsistencies = []
+
+    // Check all pairs of overlapping sections
+    FOR (section_a, section_b) IN Pairs(sections):
+        overlap = section_a.domain.intersect(section_b.domain)
+
+        IF overlap.is_empty():
+            CONTINUE
+
+        // Restriction maps
+        restricted_a = sheaf.restrict(section_a, overlap)
+        restricted_b = sheaf.restrict(section_b, overlap)
+
+        // Check agreement
+        IF NOT ApproximatelyEqual(restricted_a, restricted_b, tolerance=EPSILON):
+            inconsistencies.append(
+                SheafInconsistency(
+                    sections = (section_a, section_b),
+                    overlap = overlap,
+                    discrepancy = Distance(restricted_a, restricted_b)
+                )
+            )
+
+    IF inconsistencies.is_empty():
+        RETURN SheafConsistencyResult.Consistent
+    ELSE:
+        RETURN SheafConsistencyResult.Inconsistent(inconsistencies)
+```
+
+---
+
+## 3. Temporal Memory Coordinator
+
+### 3.1 Causal Cone Query
+
+```pseudocode
+FUNCTION CausalQuery(query, reference_time, cone_type, temporal_memory):
+    // Determine valid time range based on causal cone
+    SWITCH cone_type:
+        CASE Past:
+            time_range = (MIN_TIME, reference_time)
+        CASE Future:
+            time_range = (reference_time, MAX_TIME)
+        CASE LightCone(velocity):
+            // Relativistic constraint: |delta_x| <= c * |delta_t|
+            time_range = ComputeLightCone(reference_time, query.origin, velocity)
+
+    // Filter candidates by time range
+    candidates = temporal_memory.long_term.filter_by_time(time_range)
+
+    // Similarity search within temporal constraint
+    similarities = []
+    FOR candidate IN candidates:
+        sim = CosineSimilarity(query.embedding, candidate.embedding)
+        causal_dist = temporal_memory.causal_graph.shortest_path(
+            query.origin,
+            candidate.id
+        )
+        similarities.append((candidate, sim, causal_dist))
+
+    // Rank by combined temporal and causal relevance
+    scored = []
+    FOR (candidate, sim, causal_dist) IN similarities:
+        temporal_score = 1.0 / (1.0 + abs(candidate.timestamp - reference_time))
+        causal_score = 1.0 / (1.0 + causal_dist) IF causal_dist != INF ELSE 0.0
+
+        combined = ALPHA * sim + BETA * temporal_score + GAMMA * causal_score
+        scored.append((candidate, combined))
+
+    RETURN sorted(scored, by=combined, descending=True)
+```
+
+### 3.2 Memory Consolidation
+
+```pseudocode
+FUNCTION Consolidate(temporal_memory):
+    // Biological-inspired memory consolidation
+    // Short-term -> Long-term with salience filtering
+
+    // Compute salience for all short-term items
+    salience_scores = []
+    FOR item IN temporal_memory.short_term:
+        salience = ComputeSalience(item, temporal_memory)
+        salience_scores.append((item, salience))
+
+    // Salience computation factors:
+    // - Frequency of access
+    // - Recency of access
+    // - Causal importance (how many things depend on it)
+    // - Surprise (deviation from expected)
+
+    FUNCTION ComputeSalience(item, memory):
+        access_freq = memory.access_counts[item.id]
+        recency = 1.0 / (1.0 + (NOW() - item.last_accessed))
+        causal_importance = memory.causal_graph.out_degree(item.id)
+        surprise = ComputeSurprise(item, memory.long_term)
+
+        RETURN W1*access_freq + W2*recency + W3*causal_importance + W4*surprise
+
+    // Filter by salience threshold
+    salient_items = [item FOR (item, s) IN salience_scores IF s > THRESHOLD]
+
+    // Integrate into long-term (manifold deformation)
+    FOR item IN salient_items:
+        temporal_memory.long_term.manifold.deform(item, salience)
+
+    // Strategic forgetting for low-salience items
+    FOR item IN temporal_memory.short_term:
+        IF item NOT IN salient_items:
+            // Don't integrate - let it decay
+            PASS
+
+    // Clear short-term buffer
+    temporal_memory.short_term.clear()
+
+    // Decay low-salience regions in long-term
+    temporal_memory.long_term.strategic_forget(DECAY_RATE)
+```
+
+### 3.3 Predictive Anticipation
+
+```pseudocode
+FUNCTION Anticipate(hints, temporal_memory):
+    // Pre-compute likely future queries based on hints
+    // This enables "predictive retrieval before queries are issued"
+
+    predicted_queries = []
+
+    FOR hint IN hints:
+        SWITCH hint.type:
+            CASE SequentialPattern:
+                // If A then B pattern detected
+                recent = temporal_memory.recent_queries()
+                FOR pattern IN temporal_memory.sequential_patterns:
+                    IF pattern.matches_prefix(recent):
+                        predicted = pattern.next_likely_query()
+                        predicted_queries.append(predicted)
+
+            CASE TemporalCycle:
+                // Time-of-day or periodic patterns
+                current_phase = GetTemporalPhase(NOW())
+                historical = temporal_memory.queries_at_phase(current_phase)
+                predicted_queries.extend(historical.top_k(5))
+
+            CASE CausalChain:
+                // Causal dependencies predict next queries
+                current_context = hint.current_context
+                downstream = temporal_memory.causal_graph.downstream(current_context)
+                FOR node IN downstream:
+                    predicted_queries.append(QueryFor(node))
+
+    // Pre-fetch and cache
+    FOR query IN predicted_queries:
+        cache_key = Hash(query)
+        IF cache_key NOT IN temporal_memory.prefetch_cache:
+            result = temporal_memory.long_term.search(query)
+            temporal_memory.prefetch_cache[cache_key] = result
+```
+
+---
+
+## 4. Federated Cognitive Mesh
+
+### 4.1 Post-Quantum Federation Handshake
+
+```pseudocode
+FUNCTION JoinFederation(local_node, peer_address):
+    // CRYSTALS-Kyber key exchange
+
+    // Generate ephemeral keypair
+    (local_public, local_secret) = Kyber.KeyGen()
+
+    // Send public key to peer
+    SendMessage(peer_address, FederationRequest(local_public))
+
+    // Receive peer's encapsulated shared secret
+    response = ReceiveMessage(peer_address)
+    ciphertext = response.ciphertext
+
+    // Decapsulate to get shared secret
+    shared_secret = Kyber.Decapsulate(ciphertext, local_secret)
+
+    // Derive session keys from shared secret
+    (encrypt_key, mac_key) = DeriveKeys(shared_secret)
+
+    // Establish encrypted channel
+    channel = EncryptedChannel(peer_address, encrypt_key, mac_key)
+
+    // Exchange capabilities and negotiate federation terms
+    local_caps = local_node.capabilities()
+    peer_caps = channel.exchange(local_caps)
+
+    terms = NegotiateFederationTerms(local_caps, peer_caps)
+
+    // Create federation token
+    token = FederationToken(
+        peer = peer_address,
+        channel = channel,
+        terms = terms,
+        expires = NOW() + TOKEN_VALIDITY
+    )
+
+    RETURN token
+```
+
+### 4.2 Onion-Routed Query
+
+```pseudocode
+FUNCTION OnionQuery(query, destination, relay_nodes, local_keys):
+    // Privacy-preserving query routing through onion network
+
+    // Build onion layers (innermost to outermost)
+    layers = [destination] + relay_nodes  // Reverse order for wrapping
+
+    // Start with plaintext query
+    current_payload = SerializeQuery(query)
+
+    // Wrap in layers
+    FOR node IN layers:
+        // Encrypt with node's public key
+        encrypted = AsymmetricEncrypt(current_payload, node.public_key)
+
+        // Add routing header
+        header = OnionHeader(
+            next_hop = node.address,
+            payload_type = "onion_layer"
+        )
+
+        current_payload = header + encrypted
+
+    // Send to first relay
+    first_relay = relay_nodes.last()
+    SendMessage(first_relay, current_payload)
+
+    // Receive response (also onion-wrapped)
+    encrypted_response = ReceiveMessage(first_relay)
+
+    // Unwrap response layers
+    current_response = encrypted_response
+    FOR node IN reverse(relay_nodes):
+        current_response = AsymmetricDecrypt(current_response, local_keys.secret)
+
+    // Final decryption with destination's response
+    result = DeserializeResponse(current_response)
+
+    RETURN result
+```
+
+### 4.3 CRDT Reconciliation
+
+```pseudocode
+FUNCTION ReconcileCRDT(responses, local_state):
+    // Conflict-free merge of federated query results
+
+    // Use G-Set CRDT for search results (grow-only set)
+    merged_results = GSet()
+
+    FOR response IN responses:
+        FOR result IN response.results:
+            // G-Set merge: union operation
+            merged_results.add(result)
+
+    // For rankings, use LWW-Register (last-writer-wins)
+    ranking_map = LWWMap()
+
+    FOR response IN responses:
+        FOR (result_id, score, timestamp) IN response.rankings:
+            ranking_map.set(result_id, score, timestamp)
+
+    // Combine: results from G-Set, scores from LWW-Map
+    final_results = []
+    FOR result IN merged_results:
+        score = ranking_map.get(result.id)
+        final_results.append((result, score))
+
+    // Sort by reconciled scores
+    final_results.sort(by=score, descending=True)
+
+    RETURN final_results
+```
+
+### 4.4 Byzantine Fault Tolerant Commit
+
+```pseudocode
+FUNCTION ByzantineCommit(update, federation):
+    // PBFT-style consensus for state updates
+    n = federation.node_count()
+    f = (n - 1) / 3  // Maximum Byzantine faults tolerable
+    threshold = 2*f + 1  // Required agreement
+
+    // Phase 1: Pre-prepare (leader proposes)
+    IF federation.is_leader():
+        proposal = SignedProposal(update, sequence_number=NEXT_SEQ)
+        Broadcast(federation.nodes, PrePrepare(proposal))
+
+    // Phase 2: Prepare (nodes acknowledge receipt)
+    pre_prepare = ReceivePrePrepare()
+    IF ValidateProposal(pre_prepare):
+        prepare_msg = Prepare(pre_prepare.digest, federation.local_id)
+        Broadcast(federation.nodes, prepare_msg)
+
+    // Collect prepare messages
+    prepares = CollectMessages(type=Prepare, count=threshold)
+
+    IF len(prepares) < threshold:
+        RETURN CommitResult.InsufficientPrepares
+
+    // Phase 3: Commit (nodes commit to proposal)
+    commit_msg = Commit(pre_prepare.digest, federation.local_id)
+    Broadcast(federation.nodes, commit_msg)
+
+    // Collect commit messages
+    commits = CollectMessages(type=Commit, count=threshold)
+
+    IF len(commits) >= threshold:
+        // Execute update
+        federation.apply_update(update)
+        proof = CommitProof(commits)
+        RETURN CommitResult.Success(proof)
+    ELSE:
+        RETURN CommitResult.InsufficientCommits
+```
+
+---
+
+## 5. Backend Abstraction
+
+### 5.1 Backend Selection
+
+```pseudocode
+FUNCTION SelectBackend(requirements, available_backends):
+    // Automatic backend selection based on requirements
+
+    scored_backends = []
+
+    FOR backend IN available_backends:
+        score = 0.0
+
+        // Evaluate against requirements
+        IF requirements.latency_target:
+            latency_score = 1.0 / backend.expected_latency
+            score += W_LATENCY * latency_score
+
+        IF requirements.energy_target:
+            energy_score = 1.0 / backend.expected_energy
+            score += W_ENERGY * energy_score
+
+        IF requirements.accuracy_target:
+            accuracy_score = backend.expected_accuracy
+            score += W_ACCURACY * accuracy_score
+
+        IF requirements.scale_target:
+            scale_score = backend.max_scale / requirements.scale_target
+            score += W_SCALE * min(scale_score, 1.0)
+
+        // Check hard constraints
+        IF requirements.wasm_required AND NOT backend.supports_wasm:
+            CONTINUE
+
+        IF requirements.post_quantum_required AND NOT backend.supports_pq:
+            CONTINUE
+
+        scored_backends.append((backend, score))
+
+    // Select highest scoring backend
+    best_backend = max(scored_backends, by=score)
+
+    RETURN best_backend
+```
+
+### 5.2 Hybrid Execution
+
+```pseudocode
+FUNCTION HybridExecute(operation, backends):
+    // Execute across multiple backends, combine results
+
+    // Partition operation if possible
+    partitions = PartitionOperation(operation)
+
+    // Assign partitions to backends based on suitability
+    assignments = []
+    FOR partition IN partitions:
+        best_backend = SelectBackendForPartition(partition, backends)
+        assignments.append((partition, best_backend))
+
+    // Execute in parallel
+    futures = []
+    FOR (partition, backend) IN assignments:
+        future = backend.execute_async(partition)
+        futures.append(future)
+
+    // Await all results
+    results = AwaitAll(futures)
+
+    // Merge partition results
+    merged = MergePartitionResults(results, operation.type)
+
+    RETURN merged
+```
+
+---
+
+## 6. Consciousness Metrics (Research)
+
+### 6.1 Phi (Integrated Information) Approximation
+
+```pseudocode
+FUNCTION ApproximatePhi(substrate_region):
+    // Compute integrated information (IIT-inspired)
+    // Full Phi computation is intractable; this is an approximation
+
+    // Step 1: Compute whole-system effective information
+    whole_state = substrate_region.current_state()
+    perturbed_states = []
+    FOR _ IN 1..NUM_PERTURBATIONS:
+        perturbed = ApplyRandomPerturbation(whole_state)
+        evolved = substrate_region.evolve(perturbed)
+        perturbed_states.append(evolved)
+
+    whole_EI = MutualInformation(whole_state, perturbed_states)
+
+    // Step 2: Find minimum information partition (MIP)
+    partitions = GeneratePartitions(substrate_region)
+    min_partition_EI = INFINITY
+
+    FOR partition IN partitions:
+        partition_EI = 0.0
+        FOR part IN partition:
+            part_state = part.current_state()
+            part_perturbed = [ApplyRandomPerturbation(part_state) FOR _ IN 1..NUM_PERTURBATIONS]
+            part_evolved = [part.evolve(p) FOR p IN part_perturbed]
+            partition_EI += MutualInformation(part_state, part_evolved)
+
+        IF partition_EI < min_partition_EI:
+            min_partition_EI = partition_EI
+            mip = partition
+
+    // Step 3: Phi = whole - minimum partition
+    phi = whole_EI - min_partition_EI
+
+    RETURN max(phi, 0.0)  // Phi cannot be negative
+```
+
+---
+
+## Summary
+
+These pseudocode algorithms define the core computational patterns for the EXO-AI cognitive substrate:
+
+| Component | Key Algorithm | Complexity |
+|-----------|---------------|------------|
+| Manifold Engine | Gradient descent retrieval | O(k × d × steps) |
+| Hypergraph | Persistent homology | O(n³) worst case |
+| Temporal Memory | Causal cone query | O(n × log n) |
+| Federation | Byzantine consensus | O(n²) messages |
+| Phi Metric | Partition enumeration | O(B(n)) Bell numbers |
+
+Where:
+- k = number of results
+- d = embedding dimension
+- n = number of entities/nodes
+- steps = gradient descent iterations