Squashed 'vendor/ruvector/' content from commit b64c2172

git-subtree-dir: vendor/ruvector git-subtree-split: b64c21726f2bb37286d9ee36a7869fef60cc6900
2026-02-28 14:39:40 -05:00
commit d803bfe2b1
7854 changed files with 3522914 additions and 0 deletions
--- a/examples/exo-ai-2025/research/07-causal-emergence/RESEARCH.md
+++ b/examples/exo-ai-2025/research/07-causal-emergence/RESEARCH.md
@@ -0,0 +1,583 @@
+# Causal Emergence: Comprehensive Literature Review
+## Nobel-Level Research Synthesis (2023-2025)
+
+**Research Focus**: Computational approaches to detecting and measuring causal emergence in complex systems, with applications to consciousness science.
+
+**Research Date**: December 4, 2025
+
+---
+
+## Executive Summary
+
+Causal emergence represents a paradigm shift in understanding complex systems, demonstrating that macroscopic descriptions can possess stronger causal relationships than their underlying microscopic components. This review synthesizes cutting-edge research (2023-2025) on effective information measurement, hierarchical causation, and computational detection of emergence, with implications for consciousness science and artificial intelligence.
+
+**Key Insight**: The connection between causal emergence and consciousness may be measurable through hierarchical coarse-graining algorithms running in O(log n) time.
+
+---
+
+## 1. Erik Hoel's Causal Emergence Theory
+
+### 1.1 Foundational Framework
+
+Erik Hoel developed a formal theory demonstrating that macroscales of systems can exhibit **stronger causal relationships** than their underlying microscale components. This challenges reductionist assumptions in neuroscience and physics.
+
+**Core Principle**: Causal emergence occurs when a higher-scale description of a system has greater **effective information (EI)** than the micro-level description.
+
+### 1.2 Effective Information (EI)
+
+**Definition**: Mutual information between interventions by an experimenter and their effects, measured following maximum-entropy interventions.
+
+**Mathematical Formulation**:
+```
+EI = I(X; Y) where X = max-entropy interventions, Y = observed effects
+```
+
+**Key Property**: EI quantifies the informativeness of causal relationships across different scales of description.
+
+### 1.3 Causal Emergence 2.0 (March 2025)
+
+Hoel's latest work (arXiv:2503.13395) provides revolutionary updates:
+
+1. **Axiomatic Foundation**: Grounds emergence in fundamental principles of causation
+2. **Multiscale Structure**: Treats different scales as slices of a higher-dimensional object
+3. **Error Correction Framework**: Macroscales add error correction to causal relationships
+4. **Unique Causal Contributions**: Distinguishes which scales possess unique causal power
+
+**Breakthrough Insight**: "Macroscales are encodings that add error correction to causal relationships. Emergence IS this added error correction."
+
+### 1.4 Machine Learning Applications
+
+**Neural Information Squeezer Plus (NIS+)** (2024):
+- Automatically identifies causal emergence in data
+- Directly maximizes effective information
+- Successfully tested on simulated data and real brain recordings
+- Functions as a "machine observer" with internal model
+
+---
+
+## 2. Coarse-Graining and Multi-Scale Analysis
+
+### 2.1 Information Closure Theory of Consciousness (ICT)
+
+**Key Finding**: Only information processed at specific scales of coarse-graining appears available for conscious awareness.
+
+**Non-Trivial Information Closure (NTIC)**:
+- Conscious experiences correlate with coarse-grained neural states (population firing patterns)
+- Level of consciousness corresponds to degree of NTIC
+- Information at lower levels is fine-grained but not consciously accessible
+
+### 2.2 SVD-Based Dynamical Reversibility (2024/2025)
+
+Novel framework from Nature npj Complexity:
+
+**Key Insight**: Causal emergence arises from redundancy in information pathways, represented by irreversible and correlated dynamics.
+
+**Quantification**: CE = potential maximal efficiency increase for dynamical reversibility or information transmission
+
+**Method**: Uses Singular Value Decomposition (SVD) of Markov chain transition matrices to identify optimal coarse-graining.
+
+### 2.3 Dynamical Independence (DI) in Neural Models (2024)
+
+Breakthrough from bioRxiv (2024.10.21.619355):
+
+**Principle**: A dimensionally-reduced macroscopic variable is emergent to the extent it behaves as an independent dynamical process, distinct from micro-level dynamics.
+
+**Application**: Successfully captures emergent structure in biophysical neural models through integration-segregation interplay.
+
+### 2.4 Graph Neural Networks for Coarse-Graining (2025)
+
+Nature Communications approach:
+- Uses GNNs to identify optimal component groupings
+- Preserves information flow under compression
+- Merges nodes with similar structural properties and redundant roles
+- **Low computational complexity** - critical for O(log n) implementations
+
+---
+
+## 3. Hierarchical Causation in AI Systems
+
+### 3.1 State of Causal AI (2025)
+
+**Paradigm Shift**: From correlation-based ML to causation-based reasoning.
+
+**Judea Pearl's Ladder of Causation**:
+1. **Association** (L1): P(Y|X) - seeing/observing
+2. **Intervention** (L2): P(Y|do(X)) - doing/intervening
+3. **Counterfactuals** (L3): P(Y_x|X',Y') - imagining/reasoning
+
+**Key Principle**: "No causes in, no causes out" - data alone cannot provide causal conclusions without causal assumptions.
+
+### 3.2 Neural Causal Abstractions (Xia & Bareinboim)
+
+**Causal Hierarchy Theorem (CHT)**:
+- Models trained on lower layers of causal hierarchy have inherent limitations
+- Higher-level abstractions cannot be inferred from lower-level training alone
+
+**Abstract Causal Hierarchy Theorem**:
+- Given constructive abstraction function τ
+- If high-level model is Li-τ consistent with low-level model
+- High-level model will almost never be Lj-τ consistent for j > i
+
+**Implication**: Each level of causal abstraction requires separate treatment - cannot simply "emerge" from training on lower levels.
+
+### 3.3 Brain-Inspired Hierarchical Processing
+
+**Neurobiological Pattern**:
+- **Bottom level** (sensory cortex): Processes signals as separate sources
+- **Higher levels**: Integrates signals based on potential common sources
+- **Structure**: Reflects progressive processing of uncertainty regarding signal sources
+
+**AI Application**: Hierarchical causal inference demonstrates similar characteristics.
+
+---
+
+## 4. Information-Theoretic Measures
+
+### 4.1 Granger Causality and Transfer Entropy
+
+**Foundational Relationship**:
+```
+For Gaussian variables: Granger Causality ≡ Transfer Entropy
+```
+
+**Granger Causality**: X "G-causes" Y if past of X helps predict future of Y beyond what past of Y alone provides.
+
+**Transfer Entropy (TE)**: Information-theoretic measure of time-directed information transfer.
+
+**Key Advantage of TE**: Handles non-linear signals where Granger causality assumptions break down.
+
+**Trade-off**: TE requires more samples for accurate estimation.
+
+### 4.2 Partial Information Decomposition (PID)
+
+**Breakthrough Framework** (Trends in Cognitive Sciences, 2024):
+
+Splits information into constituent elements:
+1. **Unique Information**: Provided by one source alone
+2. **Redundant Information**: Provided by multiple sources
+3. **Synergistic Information**: Requires combination of sources
+
+**Application to Transfer Entropy**:
+- Identify sources with past of regions X and Y
+- Target: future of Y
+- Decompose information flow into unique, redundant, and synergistic components
+
+**Neuroscience Impact**: Redefining understanding of integrative brain function and neural organization.
+
+### 4.3 Directed Information Theory
+
+**Framework**: Adequate for neuroscience applications like connectivity inference.
+
+**Network Measures**: Can assess Granger causality graphs of stochastic processes.
+
+**Key Tools**:
+- Transfer entropy for directed information flow
+- Mutual information for undirected relationships
+- Conditional mutual information for mediated relationships
+
+---
+
+## 5. Integrated Information Theory (IIT)
+
+### 5.1 Core Framework
+
+**Central Claim**: Consciousness is equivalent to a system's intrinsic cause-effect power.
+
+**Φ (Phi)**: Quantifies integrated information - the degree to which a system's causal structure is irreducible.
+
+**Principle of Being**: "To exist requires being able to take and make a difference" - operational existence IS causal power.
+
+### 5.2 Causal Power Measurement
+
+**Method**: Extract probability distributions from transition probability matrices (TPMs).
+
+**Integrated Information Calculation**:
+```
+Φ = D(p^system || p^partitioned)
+```
+Where D is KL divergence between intact and partitioned distributions.
+
+**Maximally Integrated Conceptual Structure (MICS)**:
+- Generated by system = conscious experience
+- Φ value of MICS = level of consciousness
+
+### 5.3 IIT 4.0 (2024-2025)
+
+**Status**: Leading framework in neuroscience of consciousness.
+
+**Recent Developments**:
+- 16 peer-reviewed empirical studies testing core claims
+- Ongoing debate about empirical validation vs theoretical legitimacy
+- Computational intractability remains major limitation
+
+**Philosophical Grounding** (2025):
+- Connected to Kantian philosophy
+- Identity between experience and Φ-structure as constitutive a priori principle
+
+### 5.4 Computational Challenges
+
+**Problem**: Calculating Φ is computationally intractable for complex systems.
+
+**Implications**:
+- Limits empirical validation
+- Restricts application to real neural networks
+- Motivates search for approximation algorithms
+
+**Opportunity**: O(log n) hierarchical approaches could provide practical solutions.
+
+---
+
+## 6. Renormalization Group and Emergence
+
+### 6.1 Physical RG Framework
+
+**Core Concept**: Systematically retains 'slow' degrees of freedom while integrating out fast ones.
+
+**Reveals**: Universal properties independent of microscopic details.
+
+**Application to Networks**: Distinguishes scale-free from scale-invariant structures.
+
+### 6.2 Deep Learning and RG Connections
+
+**Key Insight**: Unsupervised deep learning implements **Kadanoff Real Space Variational Renormalization Group** (1975).
+
+**Implication**: Success of deep learning relates to fundamental physics concepts.
+
+**Structure**: Decimation RG resembles hierarchical deep network architecture.
+
+### 6.3 Neural Network Renormalization Group (NeuralRG)
+
+**Architecture**:
+- Deep generative model using variational RG approach
+- Type of normalizing flow
+- Composed of layers of bijectors (realNVP implementation)
+
+**Inference Process**:
+1. Each layer separates entangled variables into independent ones
+2. Decimator layers keep only one independent variable
+3. This IS the renormalization group operation
+
+**Training**: Learns optimal RG transformations from data without prior knowledge.
+
+### 6.4 Information-Theoretic RG
+
+**Characterization**: Model-independent, based on constant entropy loss rate across scales.
+
+**Application**:
+- Identifies relevant degrees of freedom automatically
+- Executes RG steps iteratively
+- Distinguishes critical points of phase transitions
+- Separates relevant from irrelevant details
+
+---
+
+## 7. Computational Complexity and Optimization
+
+### 7.1 The O(log n) Opportunity
+
+**Challenge**: Most causal measures scale poorly with system size.
+
+**Solution Pathway**: Hierarchical coarse-graining with logarithmic depth.
+
+**Key Enabler**: SIMD vectorization of information-theoretic calculations.
+
+### 7.2 Hierarchical Decomposition
+
+**Strategy**:
+```
+Level 0: n micro-states
+Level 1: n/k coarse-grained states (k-way merging)
+Level 2: n/k² states
+...
+Level log_k(n): 1 macro-state
+```
+
+**Depth**: O(log n) for k-way branching.
+
+**Computation per Level**: Can be parallelized via SIMD.
+
+### 7.3 SIMD Acceleration Opportunities
+
+**Mutual Information**:
+- Probability table operations vectorizable
+- Entropy calculations via parallel reduction
+- KL divergence computable in batches
+
+**Transfer Entropy**:
+- Time-lagged correlation matrices via SIMD
+- Conditional probabilities in parallel
+- Multiple lag values simultaneously
+
+**Effective Information**:
+- Intervention distributions pre-computed
+- Effect probabilities batched
+- MI calculations vectorized
+
+---
+
+## 8. Breakthrough Connections to Consciousness
+
+### 8.1 The Scale-Consciousness Hypothesis
+
+**Observation**: Conscious experience correlates with specific scales of neural coarse-graining, not raw micro-states.
+
+**Mechanism**: Information Closure at macro-scales creates integrated, irreducible causal structures.
+
+**Testable Prediction**: Systems with high NTIC at intermediate scales should exhibit behavioral signatures of consciousness.
+
+### 8.2 Causal Power as Consciousness Metric
+
+**IIT Claim**: Φ (integrated information) = degree of consciousness.
+
+**Causal Emergence Addition**: Φ should be maximal at the emergent macro-scale, not micro-scale.
+
+**Synthesis**: Consciousness requires BOTH:
+1. High integrated information (IIT)
+2. Causal emergence from micro to macro (Hoel)
+
+### 8.3 Hierarchical Causal Consciousness (Novel)
+
+**Hypothesis**: Consciousness is hierarchical causal emergence with feedback.
+
+**Components**:
+1. **Bottom-up emergence**: Micro → Macro via coarse-graining
+2. **Top-down causation**: Macro constraints on micro dynamics
+3. **Circular causality**: Each level affects levels above and below
+4. **Maximal EI**: At the conscious scale
+
+**Mathematical Signature**:
+```
+Consciousness ∝ max_scale(EI(scale)) × Φ(scale) × Feedback_strength(scale)
+```
+
+### 8.4 Detection Algorithm
+
+**Input**: Neural activity time series
+**Output**: Consciousness score and optimal scale
+
+**Steps**:
+1. Hierarchical coarse-graining (O(log n) levels)
+2. Compute EI at each level (SIMD-accelerated)
+3. Compute Φ at each level (approximation)
+4. Detect feedback loops (transfer entropy)
+5. Identify scale with maximum combined score
+
+**Complexity**: O(n log n) with SIMD, vs O(n²) or worse for naive approaches.
+
+---
+
+## 9. Critical Gaps and Open Questions
+
+### 9.1 Theoretical Gaps
+
+1. **Optimal Coarse-Graining**: No universally agreed-upon method for finding the "right" macro-scale
+2. **Causal vs Correlational**: Distinction sometimes blurred in practice
+3. **Temporal Dynamics**: Most frameworks assume static or Markovian systems
+4. **Quantum Systems**: Causal emergence in quantum mechanics poorly understood
+
+### 9.2 Computational Challenges
+
+1. **Scalability**: IIT's Φ calculation intractable for realistic brain models
+2. **Data Requirements**: Transfer entropy needs large sample sizes
+3. **Non-stationarity**: Real neural data violates stationarity assumptions
+4. **Validation**: Ground truth for consciousness unavailable
+
+### 9.3 Empirical Questions
+
+1. **Anesthesia**: Does causal emergence disappear under anesthesia?
+2. **Development**: How does emergence change from infant to adult brain?
+3. **Lesions**: Do focal brain lesions reduce emergence more than diffuse damage?
+4. **Cross-Species**: What is the emergence profile of different animals?
+
+---
+
+## 10. Research Synthesis: Key Takeaways
+
+### 10.1 Convergent Findings
+
+1. **Multi-scale is Essential**: Single-scale descriptions miss critical causal structure
+2. **Coarse-graining Matters**: The WAY we aggregate matters as much as THAT we aggregate
+3. **Information Theory Works**: Mutual information, transfer entropy, and EI capture emergence
+4. **Computation is Feasible**: Hierarchical algorithms can achieve O(log n) complexity
+5. **Consciousness Connection**: Multiple theories converge on causal power at macro-scales
+
+### 10.2 Novel Opportunities
+
+1. **SIMD Acceleration**: Modern CPUs/GPUs can massively parallelize information calculations
+2. **Hierarchical Methods**: Tree-like decompositions enable logarithmic complexity
+3. **Neural Networks**: Can learn optimal coarse-graining functions from data
+4. **Hybrid Approaches**: Combine IIT, causal emergence, and PID into unified framework
+5. **Real-time Detection**: O(log n) algorithms could monitor consciousness in clinical settings
+
+### 10.3 Implementation Priorities
+
+**Immediate** (High Impact, Feasible):
+1. SIMD-accelerated effective information calculation
+2. Hierarchical coarse-graining with k-way merging
+3. Transfer entropy with parallel lag computation
+4. Automated emergence detection via NeuralRG-inspired networks
+
+**Medium-term** (High Impact, Challenging):
+1. Approximate Φ calculation at multiple scales
+2. Bidirectional causal analysis (bottom-up + top-down)
+3. Temporal dynamics and non-stationarity handling
+4. Validation on neuroscience datasets (fMRI, EEG, spike trains)
+
+**Long-term** (Transformative):
+1. Unified consciousness detection system
+2. Cross-species comparative emergence profiles
+3. Therapeutic applications (coma, anesthesia monitoring)
+4. AI consciousness assessment
+
+---
+
+## 11. Computational Framework Design
+
+### 11.1 Architecture
+
+```
+RuVector Causal Emergence Module
+├── effective_information.rs     # EI calculation (SIMD)
+├── coarse_graining.rs           # Multi-scale aggregation
+├── causal_hierarchy.rs          # Hierarchical structure
+├── emergence_detection.rs       # Automatic scale selection
+├── transfer_entropy.rs          # Directed information flow
+├── integrated_information.rs    # Φ approximation
+└── consciousness_metric.rs      # Combined scoring
+```
+
+### 11.2 Key Algorithms
+
+**1. Hierarchical EI Calculation**:
+```rust
+fn hierarchical_ei(data: &[f32], k: usize) -> Vec<f32> {
+    let mut ei_per_scale = Vec::new();
+    let mut current = data.to_vec();
+
+    while current.len() > 1 {
+        // SIMD-accelerated EI at this scale
+        ei_per_scale.push(compute_ei_simd(&current));
+        // k-way coarse-graining
+        current = coarse_grain_k_way(&current, k);
+    }
+
+    ei_per_scale  // O(log_k n) levels
+}
+```
+
+**2. Optimal Scale Detection**:
+```rust
+fn detect_emergent_scale(ei_per_scale: &[f32]) -> (usize, f32) {
+    // Find scale with maximum EI
+    let (scale, &max_ei) = ei_per_scale.iter()
+        .enumerate()
+        .max_by(|a, b| a.1.partial_cmp(b.1).unwrap())
+        .unwrap();
+
+    (scale, max_ei)
+}
+```
+
+**3. Consciousness Score**:
+```rust
+fn consciousness_score(
+    ei: f32,
+    phi: f32,
+    feedback: f32
+) -> f32 {
+    ei * phi * feedback.ln()  // Log-scale feedback
+}
+```
+
+### 11.3 Performance Targets
+
+- **EI Calculation**: 1M state transitions/second (SIMD)
+- **Coarse-graining**: 10M elements/second (parallel)
+- **Hierarchy Construction**: O(log n) depth, 100M elements
+- **Total Pipeline**: 100K time steps analyzed per second
+
+---
+
+## 12. Nobel-Level Research Question
+
+### Does Consciousness Require Causal Emergence?
+
+**Hypothesis**: Consciousness is not merely integrated information (IIT) or information closure (ICT) alone, but specifically the **causal emergence** of integrated information at a macro-scale.
+
+**Predictions**:
+1. **Under anesthesia**: EI at macro-scale drops, even if micro-scale activity continues
+2. **In minimally conscious states**: Intermediate EI, between unconscious and fully conscious
+3. **Cross-species**: Emergence scale correlates with cognitive complexity
+4. **Artificial systems**: High IIT without emergence ≠ consciousness (zombie AI)
+
+**Test Method**:
+1. Record neural activity (EEG/MEG/fMRI) during:
+   - Wake
+   - Sleep (various stages)
+   - Anesthesia
+   - Vegetative state
+   - Minimally conscious state
+
+2. For each state:
+   - Compute hierarchical EI across scales
+   - Identify emergent scale
+   - Measure integrated information Φ
+   - Quantify feedback strength
+
+3. Compare:
+   - Does emergent scale correlate with subjective reports?
+   - Does max EI predict consciousness better than total information?
+   - Can we detect consciousness transitions in real-time?
+
+**Expected Outcome**: Emergent-scale causal power is **necessary and sufficient** for consciousness, providing a computational bridge between subjective experience and objective measurement.
+
+**Impact**: Would enable:
+- Objective consciousness detection in unresponsive patients
+- Monitoring anesthesia depth in surgery
+- Assessing animal consciousness ethically
+- Determining if AI systems are conscious
+- Therapeutic interventions for disorders of consciousness
+
+---
+
+## Sources
+
+### Erik Hoel's Causal Emergence Theory
+- [Emergence and Causality in Complex Systems: PMC](https://pmc.ncbi.nlm.nih.gov/articles/PMC10887681/)
+- [Causal Emergence 2.0: arXiv](https://arxiv.org/abs/2503.13395)
+- [A Primer on Causal Emergence - Erik Hoel](https://www.theintrinsicperspective.com/p/a-primer-on-causal-emergence)
+- [Emergence as Conversion of Information - Royal Society](https://royalsocietypublishing.org/doi/abs/10.1098/rsta.2021.0150)
+
+### Coarse-Graining and Multi-Scale Analysis
+- [Information Closure Theory - PMC](https://pmc.ncbi.nlm.nih.gov/articles/PMC7374725/)
+- [Dynamical Reversibility - npj Complexity](https://www.nature.com/articles/s44260-025-00028-0)
+- [Emergent Dynamics in Neural Models - bioRxiv](https://www.biorxiv.org/content/10.1101/2024.10.21.619355v2)
+- [Coarse-graining Network Flow - Nature Communications](https://www.nature.com/articles/s41467-025-56034-2)
+
+### Hierarchical Causation in AI
+- [Causal AI Book](https://causalai-book.net/)
+- [Neural Causal Abstractions - Xia & Bareinboim](https://causalai.net/r101.pdf)
+- [State of Causal AI in 2025](https://sonicviz.com/2025/02/16/the-state-of-causal-ai-in-2025/)
+- [Implications of Causality in AI - Frontiers](https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2024.1439702/full)
+
+### Information Theory and Decomposition
+- [Granger Causality and Transfer Entropy - PRL](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.103.238701)
+- [Information Decomposition in Neuroscience - Cell](https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(23)00284-X)
+- [Granger Causality in Neuroscience - PMC](https://pmc.ncbi.nlm.nih.gov/articles/PMC4339347/)
+
+### Integrated Information Theory
+- [IIT Wiki v1.0 - June 2024](https://centerforsleepandconsciousness.psychiatry.wisc.edu/wp-content/uploads/2025/09/Hendren-et-al.-2024-IIT-Wiki-Version-1.0.pdf)
+- [Integrated Information Theory - Wikipedia](https://en.wikipedia.org/wiki/Integrated_information_theory)
+- [IIT: Neuroscientific Theory - DUJS](https://sites.dartmouth.edu/dujs/2024/12/16/integrated-information-theory-a-neuroscientific-theory-of-consciousness/)
+
+### Renormalization Group and Deep Learning
+- [Mutual Information and RG - Nature Physics](https://www.nature.com/articles/s41567-018-0081-4)
+- [Deep Learning and RG - Ro's Blog](https://rojefferson.blog/2019/08/04/deep-learning-and-the-renormalization-group/)
+- [NeuralRG - GitHub](https://github.com/li012589/NeuralRG)
+- [Multiscale Network Unfolding - Nature Physics](https://www.nature.com/articles/s41567-018-0072-5)
+
+---
+
+**Document Status**: Comprehensive Literature Review v1.0
+**Last Updated**: December 4, 2025
+**Next Steps**: Implement computational framework in Rust with SIMD optimization