diff --git a/README.md b/README.md index b48e4f2..f179a52 100644 --- a/README.md +++ b/README.md @@ -707,51 +707,174 @@ The full RuVector ecosystem includes 90+ crates. See [github.com/ruvnet/ruvector
๐Ÿ—๏ธ System Architecture โ€” End-to-end data flow from CSI capture to REST/WebSocket API -``` -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ WiFi Router โ”‚ โ”‚ WiFi Router โ”‚ โ”‚ WiFi Router โ”‚ -โ”‚ (CSI Source) โ”‚ โ”‚ (CSI Source) โ”‚ โ”‚ (CSI Source) โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ โ”‚ โ”‚ - โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ - โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” - โ”‚ CSI Data Collector โ”‚ - โ”‚ (Hardware Interface) โ”‚ - โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ - โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” - โ”‚ Signal Processor โ”‚ - โ”‚ (RuVector + Phase San.) โ”‚ - โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ - โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” - โ”‚ Graph Transformer โ”‚ - โ”‚ (DensePose + GNN Head) โ”‚ - โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ - โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” - โ”‚ Vital Signs + Tracker โ”‚ - โ”‚ (Breathing, Heart, Pose) โ”‚ - โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ - โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” - โ”‚ โ”‚ โ”‚ -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ REST API โ”‚ โ”‚ WebSocket API โ”‚ โ”‚ Analytics โ”‚ -โ”‚ (Axum / FastAPI) โ”‚ โ”‚ (Real-time Stream)โ”‚ โ”‚ (Fall Detection) โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +### End-to-End Pipeline + +```mermaid +graph TB + subgraph HW ["๐Ÿ“ก Hardware Layer"] + direction LR + R1["WiFi Router 1
CSI Source"] + R2["WiFi Router 2
CSI Source"] + R3["WiFi Router 3
CSI Source"] + ESP["ESP32-S3 Mesh
20 Hz ยท 56 subcarriers"] + WIN["Windows WiFi
RSSI scanning"] + end + + subgraph INGEST ["โšก Ingestion"] + AGG["Aggregator
UDP :5005 ยท ADR-018 frames"] + BRIDGE["Bridge
I/Q โ†’ amplitude + phase"] + end + + subgraph SIGNAL ["๐Ÿ”ฌ Signal Processing โ€” RuVector v2.0.4"] + direction TB + PHASE["Phase Sanitization
SpotFi conjugate multiply"] + HAMPEL["Hampel Filter
Outlier rejection ยท ฯƒ=3"] + SUBSEL["Subcarrier Selection
ruvector-mincut ยท sensitive/insensitive split"] + SPEC["Spectrogram
ruvector-attn-mincut ยท gated STFT"] + FRESNEL["Fresnel Geometry
ruvector-solver ยท TX-body-RX distance"] + BVP["Body Velocity Profile
ruvector-attention ยท weighted BVP"] + end + + subgraph ML ["๐Ÿง  Neural Pipeline"] + direction TB + GRAPH["Graph Transformer
17 COCO keypoints ยท 16 edges"] + CROSS["Cross-Attention
CSI features โ†’ body pose"] + SONA["SONA Adapter
LoRA rank-4 ยท EWC++"] + end + + subgraph VITAL ["๐Ÿ’“ Vital Signs"] + direction LR + BREATH["Breathing
0.1โ€“0.5 Hz ยท FFT peak"] + HEART["Heart Rate
0.8โ€“2.0 Hz ยท FFT peak"] + MOTION["Motion Level
Variance + band power"] + end + + subgraph API ["๐ŸŒ Output Layer"] + direction LR + REST["REST API
Axum :3000 ยท 6 endpoints"] + WS["WebSocket
:3001 ยท real-time stream"] + ANALYTICS["Analytics
Fall ยท Activity ยท START triage"] + UI["Web UI
Three.js ยท Gaussian splats"] + end + + R1 & R2 & R3 --> AGG + ESP --> AGG + WIN --> BRIDGE + AGG --> BRIDGE + BRIDGE --> PHASE + PHASE --> HAMPEL + HAMPEL --> SUBSEL + SUBSEL --> SPEC + SPEC --> FRESNEL + FRESNEL --> BVP + BVP --> GRAPH + GRAPH --> CROSS + CROSS --> SONA + SONA --> BREATH & HEART & MOTION + BREATH & HEART & MOTION --> REST & WS & ANALYTICS + WS --> UI + + style HW fill:#1a1a2e,stroke:#e94560,color:#eee + style INGEST fill:#16213e,stroke:#0f3460,color:#eee + style SIGNAL fill:#0f3460,stroke:#533483,color:#eee + style ML fill:#533483,stroke:#e94560,color:#eee + style VITAL fill:#2d132c,stroke:#e94560,color:#eee + style API fill:#1a1a2e,stroke:#0f3460,color:#eee ``` -| Component | Description | -|-----------|-------------| -| **CSI Processor** | Extracts Channel State Information from WiFi signals (ESP32 or RSSI) | -| **Signal Processor** | RuVector-powered phase sanitization, Hampel filter, Fresnel model | -| **Graph Transformer** | GNN body-graph reasoning with cross-attention CSI-to-pose mapping | -| **Vital Signs** | FFT-based breathing (0.1-0.5 Hz) and heartbeat (0.8-2.0 Hz) extraction | -| **REST API** | Axum (Rust) or FastAPI (Python) for data access and control | -| **WebSocket** | Real-time pose, sensing, and vital sign streaming | -| **Analytics** | Fall detection, activity recognition, START triage | +### Signal Processing Detail + +```mermaid +graph LR + subgraph RAW ["Raw CSI Frame"] + IQ["I/Q Samples
56โ€“192 subcarriers ร— N antennas"] + end + + subgraph CLEAN ["Phase Cleanup"] + CONJ["Conjugate Multiply
Remove carrier freq offset"] + UNWRAP["Phase Unwrap
Remove 2ฯ€ discontinuities"] + HAMPEL2["Hampel Filter
Remove impulse noise"] + end + + subgraph SELECT ["Subcarrier Intelligence"] + MINCUT["Min-Cut Partition
ruvector-mincut"] + GATE["Attention Gate
ruvector-attn-mincut"] + end + + subgraph EXTRACT ["Feature Extraction"] + STFT["STFT Spectrogram
Time-frequency decomposition"] + FRESNELZ["Fresnel Zones
ruvector-solver"] + BVPE["BVP Estimation
ruvector-attention"] + end + + subgraph OUT ["Output Features"] + AMP["Amplitude Matrix"] + PHASE2["Phase Matrix"] + DOPPLER["Doppler Shifts"] + VITALS["Vital Band Power"] + end + + IQ --> CONJ --> UNWRAP --> HAMPEL2 + HAMPEL2 --> MINCUT --> GATE + GATE --> STFT --> FRESNELZ --> BVPE + BVPE --> AMP & PHASE2 & DOPPLER & VITALS + + style RAW fill:#0d1117,stroke:#58a6ff,color:#c9d1d9 + style CLEAN fill:#161b22,stroke:#58a6ff,color:#c9d1d9 + style SELECT fill:#161b22,stroke:#d29922,color:#c9d1d9 + style EXTRACT fill:#161b22,stroke:#3fb950,color:#c9d1d9 + style OUT fill:#0d1117,stroke:#8b949e,color:#c9d1d9 +``` + +### Deployment Topology + +```mermaid +graph TB + subgraph EDGE ["Edge (ESP32-S3 Mesh)"] + E1["Node 1
Kitchen"] + E2["Node 2
Living room"] + E3["Node 3
Bedroom"] + end + + subgraph SERVER ["Server (Rust ยท 132 MB Docker)"] + SENSE["Sensing Server
:3000 REST ยท :3001 WS ยท :5005 UDP"] + RVF["RVF Model
Progressive 3-layer load"] + STORE["Time-Series Store
In-memory ring buffer"] + end + + subgraph CLIENT ["Clients"] + BROWSER["Browser
Three.js UI ยท Gaussian splats"] + MOBILE["Mobile App
WebSocket stream"] + DASH["Dashboard
REST polling"] + IOT["Home Automation
MQTT bridge"] + end + + E1 -->|"UDP :5005
ADR-018 frames"| SENSE + E2 -->|"UDP :5005"| SENSE + E3 -->|"UDP :5005"| SENSE + SENSE <--> RVF + SENSE <--> STORE + SENSE -->|"WS :3001
real-time JSON"| BROWSER & MOBILE + SENSE -->|"REST :3000
on-demand"| DASH & IOT + + style EDGE fill:#1a1a2e,stroke:#e94560,color:#eee + style SERVER fill:#16213e,stroke:#533483,color:#eee + style CLIENT fill:#0f3460,stroke:#0f3460,color:#eee +``` + +| Component | Crate / Module | Description | +|-----------|---------------|-------------| +| **Aggregator** | `wifi-densepose-hardware` | ESP32 UDP listener, ADR-018 frame parser, I/Q โ†’ amplitude/phase bridge | +| **Signal Processor** | `wifi-densepose-signal` | SpotFi phase sanitization, Hampel filter, STFT spectrogram, Fresnel geometry, BVP | +| **Subcarrier Selection** | `ruvector-mincut` + `ruvector-attn-mincut` | Dynamic sensitive/insensitive partitioning, attention-gated noise suppression | +| **Fresnel Solver** | `ruvector-solver` | Sparse Neumann series O(sqrt(n)) for TX-body-RX distance estimation | +| **Graph Transformer** | `wifi-densepose-train` | COCO BodyGraph (17 kp, 16 edges), cross-attention CSIโ†’pose, GCN message passing | +| **SONA** | `sona` crate | Micro-LoRA (rank-4) adaptation, EWC++ catastrophic forgetting prevention | +| **Vital Signs** | `wifi-densepose-signal` | FFT-based breathing (0.1-0.5 Hz) and heartbeat (0.8-2.0 Hz) extraction | +| **REST API** | `wifi-densepose-sensing-server` | Axum server: `/api/v1/sensing`, `/health`, `/vital-signs`, `/bssid`, `/sona` | +| **WebSocket** | `wifi-densepose-sensing-server` | Real-time pose, sensing, and vital sign streaming on `:3001` | +| **Analytics** | `wifi-densepose-mat` | Fall detection, activity recognition, START triage (WiFi-Mat disaster module) | +| **Web UI** | `ui/` | Three.js scene, Gaussian splat visualization, signal dashboard |