14 KiB
14 KiB
EXO-AI 2025: Research Papers & References
SPARC Research Phase: Academic Foundations
This document catalogs the academic research informing the EXO-AI architecture, organized by domain.
1. Processing-in-Memory (PIM) Architectures
Core Reviews
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| A Comprehensive Review of Processing-in-Memory Architectures for DNNs | MDPI Computers | 2024 | Chiplet-based PIM designs, dataflow optimization |
| Neural-PIM: Efficient Processing-In-Memory | arXiv | 2022 | Neural network acceleration in DRAM |
| PRIME: Processing-in-Memory for Neural Networks | ISCA | 2016 | ReRAM-based crossbar computation |
| PIMCoSim: Hardware/Software Co-Simulator | MDPI Electronics | 2024 | Simulation framework for PIM exploration |
Key Findings
- UPMEM achieves 23x performance over GPU when memory oversubscription required
- SRAM-PIM with value-level and bit-level sparsity (DB-PIM framework)
- ReRAM crossbars enable ~10x gain over SRAM-based accelerators
UPMEM Architecture
First commercially available PIM: DRAM + in-order cores (DPUs) on same chip.
2. Neuromorphic Computing & Vector Search
Neuromorphic Hardware
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Roadmap to Neuromorphic Computing with Emerging Technologies | arXiv | 2024 | Technology roadmap for neuromorphic systems |
| Neuromorphic Computing for Robotic Vision | Nature Comm. Eng. | 2025 | Event-driven vision processing |
| Survey of Neuromorphic Computing and Neural Networks in Hardware | arXiv | 2017 | Comprehensive hardware survey |
Key Hardware Platforms
- SpiNNaker: Millions of processing cores (Manchester)
- TrueNorth: IBM's commercial neuromorphic chip
- Loihi: Intel research chip with online learning
- BrainScaleS: European analog-digital hybrid
HNSW Advances
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Down with the Hierarchy: Hub Highway Hypothesis | arXiv | 2024 | Hubs maintain hierarchy function, not layers |
| Efficient Vector Search on Disaggregated Memory (d-HNSW) | arXiv | 2025 | Disaggregated memory architecture |
| WebANNS: ANN Search in Web Browsers | arXiv | 2025 | Browser-based vector search |
3. Implicit Neural Representations (INR)
Core Research
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Where Do We Stand with INRs? Technical Survey | arXiv | 2024 | Four-category taxonomy of INR techniques |
| FR-INR: Fourier Reparameterized Training | CVPR | 2024 | Fourier bases for MLP weight composition |
| Neural Experts: Mixture of Experts for INRs | NeurIPS | 2024 | MoE for local piece-wise continuous functions |
| inr2vec: Compact Latent Representation for INRs | CVPR | 2023 | Embeddings for INR-based retrieval |
Key INR Methods
- SIREN: Sinusoidal activation networks
- WIRE: Wavelet implicit representations
- GAUSS: Gaussian activation functions
- FINER: Frequency-enhanced representations
Retrieval Performance
inr2vec shows 1.8 mAP gap vs PointNet++ on 3D retrieval benchmarks.
4. Hypergraph & Topological Data Analysis
Hypergraph Neural Networks
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| EasyHypergraph: Fast Higher-Order Network Analysis | Nature HSS Comm. | 2025 | Memory-efficient hypergraph analysis |
| DPHGNN: Dual Perspective Hypergraph Neural Networks | KDD | 2024 | Dual-perspective message passing |
| Hypergraph Computation Survey | Engineering | 2024 | Comprehensive hypergraph computation survey |
Topological Deep Learning
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Topological Deep Learning: New Frontier for Relational Learning | PMC | 2024 | Position paper on TDL paradigm |
| ICML TDL Challenge 2024: Beyond the Graph Domain | ICML | 2024 | 52 submissions on topological liftings |
| Simplicial Homology Theories for Hypergraphs | arXiv | 2024 | Survey of hypergraph homology |
Key Software
- TopoX Suite: TopoNetX, TopoEmbedX, TopoModelX (Python)
- DHG: DeepHypergraph for learning on hypergraphs
- HyperNetX: Hypergraph computations
- XGI: Hypergraphs and simplicial complexes
5. Temporal Memory & Causal Inference
Agent Memory Architectures
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Mem0: Production-Ready AI Agents with Scalable LTM | arXiv | 2024 | Causal relationships for decision-making |
| Zep: Temporal Knowledge Graph for Agent Memory | arXiv | 2025 | TKG-based memory with Graphiti engine |
| Memory Architectures in Long-Term AI Agents | ResearchGate | 2025 | 47% improvement in temporal reasoning |
| Evaluating Very Long-Term Conversational Memory | ResearchGate | 2024 | Long-term temporal/causal dynamics |
Key Findings
- Zep outperforms MemGPT on Deep Memory Retrieval benchmark
- Mem0g adds graph-based memory representations
- TKGs model relationship start/change/end for causality tracking
Causal Inference + Deep Learning
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Causal Inference Meets Deep Learning: Survey | PMC | 2024 | PFC working memory for causal reasoning |
6. Federated Learning & Distributed Consensus
Federated Learning
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Secure and Fair Federated Learning via Consensus Incentive | MDPI Mathematics | 2024 | Byzantine-resistant FL |
| FL Assisted Distributed Energy Optimization | IET RPG | 2024 | Consensus + innovations approach |
| Comprehensive Review of FL Challenges | J. Big Data | 2025 | Data preparation viewpoint |
CRDT Fundamentals
| Resource | Key Contribution |
|---|---|
| CRDT Dictionary: Field Guide | Comprehensive CRDT taxonomy |
| CRDT Wiki (Dremio) | Strong eventual consistency |
Key Algorithms
- HyFDCA: Hybrid Federated Dual Coordinate Ascent (2024)
- Gossip protocols for decentralized aggregation
- Version vectors for causal tracking in CRDTs
7. Photonic Computing
Silicon Photonics for AI
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| MIT Photonic Processor for Ultrafast AI | MIT News | 2024 | Sub-nanosecond classification, 92% accuracy |
| Silicon Photonics for Scalable AI Hardware | IEEE JSTQE | 2025 | Wafer-scale ONN integration |
| Hundred-Layer Photonic Deep Learning | Nature Comm. | 2025 | SLiM chip: 200+ layer depth |
| All-Optical CNN with Phase Change Materials | Sci. Reports | 2025 | GST-based active waveguides |
Key Characteristics
- Sub-nanosecond latency
- Minimal energy loss (photons don't generate heat like electrons)
- THz bandwidth potential
- 3.2 Tbps achieved on silicon slow-light modulator
8. ReRAM & Memristor Computing
Analog In-Memory Compute
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Programming Memristor Arrays with Arbitrary Precision | Science | 2024 | 16Mb floating-point RRAM, 31.2 TFLOPS/W |
| Memristive Memory Augmented Neural Network | Nature Comm. | 2022 | Hashing and similarity search in crossbars |
| Wafer-Scale Memristive Passive Crossbar | Nature Comm. | 2025 | Brain-scale neuromorphic computing |
| 4K-Memristor Analog-Grade Crossbar | Nature Comm. | 2021 | Foundational analog VMM work |
Vector Similarity Search
- TCAM functionality in analog crossbar
- Hamming distance via degree-of-mismatch output
- Massively parallel in-memory similarity computation
9. Sheaf Theory & Category Theory for ML
Sheaf Neural Networks
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Sheaf Theory: From Deep Geometry to Deep Learning | arXiv | 2025 | Comprehensive sheaf applications survey |
| Sheaf4Rec: Recommender Systems | arXiv | 2023 | 8.53% F1@10 improvement, 37% faster |
| Sheaf Neural Networks with Connection Laplacians | ICML | 2022 | Learnable sheaf Laplacians |
| Categorical Deep Learning: Algebraic Theory of All Architectures | arXiv | 2024 | Monads + 2-categories for neural networks |
Key Concepts
- Sheaf: Local-to-global consistency structure
- Sheaf Laplacian: Diffusion operator on sheaf-decorated graphs
- Neural Sheaf Diffusion: Learning sheaf structure from data
10. Consciousness & Integrated Information
IIT Research
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| IIT 4.0: Phenomenal Existence in Physical Terms | PLOS Comp. Bio. | 2023 | Updated axioms, postulates, measures |
| How to be an IIT Theorist Without Losing Your Body | Frontiers | 2024 | Embodied IIT considerations |
Key Metrics
- Φ (Phi): Integrated information measure
- Reentrant architecture: Feedback loops required for consciousness
- Controversy: Empirical testability debates (2023-2025)
11. Thermodynamic Limits
Landauer Bound & Reversible Computing
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Fundamental Energy Limits and Reversible Computing | Sandia | 2017 | DOE reversible computing roadmap |
| Adiabatic Computing for Optimal Thermodynamic Efficiency | arXiv | 2023 | Optimal information processing bounds |
| Fundamental Energy Cost of Finite-Time Parallelizable Computing | Nature Comm. | 2023 | Parallelization thermodynamics |
Key Numbers
- Landauer limit: ~0.018 eV (2.9×10⁻²¹ J) per bit erasure at room temp
- Current CMOS: 1000x above theoretical minimum
- Reversible computing: 4000x efficiency potential
- Vaire Computing: Commercial reversible chips by 2027-2028
12. Multi-Modal Foundation Models
Unified Architectures
| Paper | Venue | Year | Key Contribution |
|---|---|---|---|
| Unified Multimodal Understanding and Generation | arXiv | 2025 | Any-to-any multimodal models |
| Show-o: Single Transformer for Multimodal | GitHub | 2024 | Unified understanding + generation |
| Multi-Modal Latent Space Learning for CoT Reasoning | AAAI | 2024 | Chain-of-thought across modalities |
Key Models (2024-2025)
- Chameleon: Mixed-modal early fusion (Meta)
- Emu3: Next-token prediction for all modalities
- Janus/JanusFlow: Decoupled visual encoding
- SEED-X: Multi-granularity comprehension
Summary Statistics
| Category | Papers Reviewed | Key Takeaway |
|---|---|---|
| PIM/Near-Memory | 8 | 23x GPU performance, commercial availability |
| Neuromorphic | 12 | 1000x energy reduction potential |
| INR/Learned Manifolds | 6 | Continuous representations for storage |
| Hypergraph/TDA | 10 | Higher-order relations, topological queries |
| Temporal Memory | 6 | TKGs for causal agent memory |
| Federated/CRDT | 5 | Decentralized consensus, eventual consistency |
| Photonic | 5 | Sub-ns latency, 92% accuracy demonstrated |
| Memristor | 5 | 31.2 TFLOPS/W efficiency |
| Sheaf/Category | 6 | 8.5% improvement on recommender tasks |
| Consciousness | 3 | IIT 4.0 framework, Φ measures |
| Thermodynamics | 4 | 4000x reversible computing potential |
| Multi-Modal | 5 | Unified latent spaces emerging |