dearsky/wifi-densepose
1
0
Fork 0
Code Issues 20 Pull Requests 5 Actions Packages Projects Releases 1 Wiki Activity

feat: Add 12 ADRs for RuVector RVF integration and proof-of-reality

Browse Source 
Comprehensive architecture decision records for integrating ruvnet/ruvector
into wifi-densepose, covering:

- ADR-002: Master integration strategy (phased rollout, new crate design)
- ADR-003: RVF cognitive containers for CSI data persistence
- ADR-004: HNSW vector search replacing fixed-threshold detection
- ADR-005: SONA self-learning with LoRA + EWC++ for online adaptation
- ADR-006: GNN-enhanced pattern recognition with temporal modeling
- ADR-007: Post-quantum cryptography (ML-DSA-65 hybrid signatures)
- ADR-008: Raft consensus for multi-AP distributed coordination
- ADR-009: RVF WASM runtime for edge/browser/IoT deployment
- ADR-010: Witness chains for tamper-evident audit trails
- ADR-011: Mock elimination and proof-of-reality (fixes np.random.rand
           placeholders, ships CSI capture + SHA-256 verified pipeline)
- ADR-012: ESP32 CSI sensor mesh ($54 starter kit specification)
- ADR-013: Feature-level sensing on commodity gear (zero-cost RSSI path)

ADR-011 directly addresses the credibility gap by cataloging every
mock/placeholder in the Python codebase and specifying concrete fixes.

https://claude.ai/code/session_01Ki7pvEZtJDvqJkmyn6B714
This commit is contained in:
Claude
2026-02-28 06:13:04 +00:00
parent 16c50abca3
commit 337dd9652f
12 changed files with 3520 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

318
docs/adr/ADR-012-esp32-csi-sensor-mesh.md Normal file
View File

@@ -0,0 +1,318 @@
# ADR-012: ESP32 CSI Sensor Mesh for Distributed Sensing
## Status
Proposed
## Date
2026-02-28
## Context
### The Hardware Reality Gap
WiFi-DensePose's Rust and Python pipelines implement real signal processing (FFT, phase unwrapping, Doppler extraction, correlation features), but the system currently has no defined path from **physical WiFi hardware → CSI bytes → pipeline input**. The `csi_extractor.py` and `router_interface.py` modules contain placeholder parsers that return `np.random.rand()` instead of real parsed data (see ADR-011).
To close this gap, we need a concrete, affordable, reproducible hardware platform that produces real CSI data and streams it into the existing pipeline.
### Why ESP32
| Factor | ESP32/ESP32-S3 | Intel 5300 (iwl5300) | Atheros AR9580 |
|--------|---------------|---------------------|----------------|
| Cost | ~$5-15/node | ~$50-100 (used NIC) | ~$30-60 (used NIC) |
| Availability | Mass produced, in stock | Discontinued, eBay only | Discontinued, eBay only |
| CSI Support | Official ESP-IDF API | Linux CSI Tool (kernel mod) | Atheros CSI Tool |
| Form Factor | Standalone MCU | Requires PCIe/Mini-PCIe host | Requires PCIe host |
| Deployment | Battery/USB, wireless | Desktop/laptop only | Desktop/laptop only |
| Antenna Config | 1-2 TX, 1-2 RX | 3 TX, 3 RX (MIMO) | 3 TX, 3 RX (MIMO) |
| Subcarriers | 52-56 (802.11n) | 30 (compressed) | 56 (full) |
| Fidelity | Lower (consumer SoC) | Higher (dedicated NIC) | Higher (dedicated NIC) |
**ESP32 wins on deployability**: It's the only option where a stranger can buy nodes on Amazon, flash firmware, and have a working CSI mesh in an afternoon. Intel 5300 and Atheros cards require specific hardware, kernel modifications, and legacy OS versions.
### ESP-IDF CSI API
Espressif provides official CSI support through three key functions:
```c
// 1. Configure what CSI data to capture
wifi_csi_config_t csi_config = {
.lltf_en = true, // Long Training Field (best for CSI)
.htltf_en = true, // HT-LTF
.stbc_htltf2_en = true, // STBC HT-LTF2
.ltf_merge_en = true, // Merge LTFs
.channel_filter_en = false,
.manu_scale = false,
};
esp_wifi_set_csi_config(&csi_config);
// 2. Register callback for received CSI data
esp_wifi_set_csi_rx_cb(csi_data_callback, NULL);
// 3. Enable CSI collection
esp_wifi_set_csi(true);
// Callback receives:
void csi_data_callback(void *ctx, wifi_csi_info_t *info) {
// info->rx_ctrl: RSSI, noise_floor, channel, secondary_channel, etc.
// info->buf: Raw CSI data (I/Q pairs per subcarrier)
// info->len: Length of CSI data buffer
// Typical: 112 bytes = 56 subcarriers × 2 (I,Q) × 1 byte each
}
```
## Decision
We will build an ESP32 CSI Sensor Mesh as the primary hardware integration path, with a full stack from firmware to aggregator to Rust pipeline to visualization.
### System Architecture
```
┌─────────────────────────────────────────────────────────────────────┐
│ ESP32 CSI Sensor Mesh │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ ESP32 │ │ ESP32 │ │ ESP32 │ ... (3-6 nodes) │
│ │ Node 1 │ │ Node 2 │ │ Node 3 │ │
│ │ │ │ │ │ │ │
│ │ CSI Rx │ │ CSI Rx │ │ CSI Rx │ ← WiFi frames from │
│ │ FFT │ │ FFT │ │ FFT │ consumer router │
│ │ Features │ │ Features │ │ Features │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ │ UDP/TCP stream (WiFi or secondary channel) │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────┐ │
│ │ Aggregator │ │
│ │ (Laptop / Raspberry Pi / Seed device) │ │
│ │ │ │
│ │ 1. Receive CSI streams from all nodes │ │
│ │ 2. Timestamp alignment (per-node) │ │
│ │ 3. Feature-level fusion │ │
│ │ 4. Feed into Rust/Python pipeline │ │
│ │ 5. Serve WebSocket to visualization │ │
│ └──────────────────┬──────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────┐ │
│ │ WiFi-DensePose Pipeline │ │
│ │ │ │
│ │ CsiProcessor → FeatureExtractor → │ │
│ │ MotionDetector → PoseEstimator → │ │
│ │ Three.js Visualization │ │
│ └─────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
### Node Firmware Specification
**ESP-IDF project**: `firmware/esp32-csi-node/`
```
firmware/esp32-csi-node/
├── CMakeLists.txt
├── sdkconfig.defaults # Menuconfig defaults with CSI enabled
├── main/
│ ├── CMakeLists.txt
│ ├── main.c # Entry point, WiFi init, CSI callback
│ ├── csi_collector.c # CSI data collection and buffering
│ ├── csi_collector.h
│ ├── feature_extract.c # On-device FFT and feature extraction
│ ├── feature_extract.h
│ ├── stream_sender.c # UDP stream to aggregator
│ ├── stream_sender.h
│ ├── config.h # Node configuration (SSID, aggregator IP)
│ └── Kconfig.projbuild # Menuconfig options
├── components/
│ └── esp_dsp/ # Espressif DSP library for FFT
└── README.md # Flash instructions
```
**On-device processing** (reduces bandwidth, node does pre-processing):
```c
// feature_extract.c
typedef struct {
uint32_t timestamp_ms; // Local monotonic timestamp
uint8_t node_id; // This node's ID
int8_t rssi; // Received signal strength
int8_t noise_floor; // Noise floor estimate
uint8_t channel; // WiFi channel
float amplitude[56]; // |CSI| per subcarrier (from I/Q)
float phase[56]; // arg(CSI) per subcarrier
float doppler_energy; // Motion energy from temporal FFT
float breathing_band; // 0.1-0.5 Hz band power
float motion_band; // 0.5-3 Hz band power
} csi_feature_frame_t;
// Size: ~470 bytes per frame
// At 100 Hz: ~47 KB/s per node, ~280 KB/s for 6 nodes
```
**Key firmware design decisions**:
1. **Feature extraction on-device**: Raw CSI I/Q → amplitude + phase + spectral bands. This cuts bandwidth from raw ~11 KB/frame to ~470 bytes/frame.
2. **Monotonic timestamps**: Each node uses its own monotonic clock. No NTP synchronization attempted between nodes - clock drift is handled at the aggregator by fusing features, not raw phases (see "Clock Drift" section below).
3. **UDP streaming**: Low-latency, loss-tolerant. Missing frames are acceptable; ordering is maintained via sequence numbers.
4. **Configurable sampling rate**: 10-100 Hz via menuconfig. 100 Hz for motion detection, 10 Hz sufficient for occupancy.
### Aggregator Specification
The aggregator runs on any machine with WiFi/Ethernet to the nodes:
```rust
// In wifi-densepose-rs, new module: crates/wifi-densepose-hardware/src/esp32/
pub struct Esp32Aggregator {
/// UDP socket listening for node streams
socket: UdpSocket,
/// Per-node state (last timestamp, feature buffer, drift estimate)
nodes: HashMap<u8, NodeState>,
/// Ring buffer of fused feature frames
fused_buffer: VecDeque<FusedFrame>,
/// Channel to pipeline
pipeline_tx: mpsc::Sender<CsiData>,
}
/// Fused frame from all nodes for one time window
pub struct FusedFrame {
/// Timestamp (aggregator local, monotonic)
timestamp: Instant,
/// Per-node features (may have gaps if node dropped)
node_features: Vec<Option<CsiFeatureFrame>>,
/// Cross-node correlation (computed by aggregator)
cross_node_correlation: Array2<f64>,
/// Fused motion energy (max across nodes)
fused_motion_energy: f64,
/// Fused breathing band (coherent sum where phase aligns)
fused_breathing_band: f64,
}
```
### Clock Drift Handling
ESP32 crystal oscillators drift ~20-50 ppm. Over 1 hour, two nodes may diverge by 72-180ms. This makes raw phase alignment across nodes impossible.
**Solution**: Feature-level fusion, not signal-level fusion.
```
Signal-level (WRONG for ESP32):
Align raw I/Q samples across nodes → requires <1µs sync → impractical
Feature-level (CORRECT for ESP32):
Each node: raw CSI → amplitude + phase + spectral features (local)
Aggregator: collect features → correlate → fuse decisions
No cross-node phase alignment needed
```
Specifically:
- **Motion energy**: Take max across nodes (any node seeing motion = motion)
- **Breathing band**: Use node with highest SNR as primary, others as corroboration
- **Location**: Cross-node amplitude ratios estimate position (no phase needed)
### Sensing Capabilities by Deployment
| Capability | 1 Node | 3 Nodes | 6 Nodes | Evidence |
|-----------|--------|---------|---------|----------|
| Presence detection | Good | Excellent | Excellent | Single-node RSSI variance |
| Coarse motion | Good | Excellent | Excellent | Doppler energy |
| Room-level location | None | Good | Excellent | Amplitude ratios |
| Respiration | Marginal | Good | Good | 0.1-0.5 Hz band, placement-sensitive |
| Heartbeat | Poor | Poor-Marginal | Marginal | Requires ideal placement, low noise |
| Multi-person count | None | Marginal | Good | Spatial diversity |
| Pose estimation | None | Poor | Marginal | Requires model + sufficient diversity |
**Honest assessment**: ESP32 CSI is lower fidelity than Intel 5300 or Atheros. Heartbeat detection is placement-sensitive and unreliable. Respiration works with good placement. Motion and presence are solid.
### Failure Modes and Mitigations
| Failure Mode | Severity | Mitigation |
|-------------|----------|------------|
| Multipath dominates in cluttered rooms | High | Mesh diversity: 3+ nodes from different angles |
| Person occludes path between node and router | Medium | Mesh: other nodes still have clear paths |
| Clock drift ruins cross-node fusion | Medium | Feature-level fusion only; no cross-node phase alignment |
| UDP packet loss during high traffic | Low | Sequence numbers, interpolation for gaps <100ms |
| ESP32 WiFi driver bugs with CSI | Medium | Pin ESP-IDF version, test on known-good boards |
| Node power failure | Low | Aggregator handles missing nodes gracefully |
### Bill of Materials (Starter Kit)
| Item | Quantity | Unit Cost | Total |
|------|----------|-----------|-------|
| ESP32-S3-DevKitC-1 | 3 | $10 | $30 |
| USB-A to USB-C cables | 3 | $3 | $9 |
| USB power adapter (multi-port) | 1 | $15 | $15 |
| Consumer WiFi router (any) | 1 | $0 (existing) | $0 |
| Aggregator (laptop or Pi 4) | 1 | $0 (existing) | $0 |
| **Total** | | | **$54** |
### Minimal Build Spec (Clone-Flash-Run)
```
# Step 1: Flash one node (requires ESP-IDF installed)
cd firmware/esp32-csi-node
idf.py set-target esp32s3
idf.py menuconfig # Set WiFi SSID/password, aggregator IP
idf.py build flash monitor
# Step 2: Run aggregator (Docker)
docker compose -f docker-compose.esp32.yml up
# Step 3: Verify with proof bundle
# Aggregator captures 10 seconds, produces feature JSON, verifies hash
docker exec aggregator python verify_esp32.py
# Step 4: Open visualization
open http://localhost:3000 # Three.js dashboard
```
### Proof of Reality for ESP32
```
firmware/esp32-csi-node/proof/
├── captured_csi_10sec.bin # Real 10-second CSI capture from ESP32
├── captured_csi_meta.json # Board: ESP32-S3-DevKitC, ESP-IDF: 5.2, Router: TP-Link AX1800
├── expected_features.json # Feature extraction output
├── expected_features.sha256 # Hash verification
└── capture_photo.jpg # Photo of actual hardware setup
```
## Consequences
### Positive
- **$54 starter kit**: Lowest possible barrier to real CSI data
- **Mass available hardware**: ESP32 boards are in stock globally
- **Real data path**: Eliminates every `np.random.rand()` placeholder with actual hardware input
- **Proof artifact**: Captured CSI + expected hash proves the pipeline processes real data
- **Scalable mesh**: Add nodes for more coverage without changing software
- **Feature-level fusion**: Avoids the impossible problem of cross-node phase synchronization
### Negative
- **Lower fidelity than research NICs**: ESP32 CSI is noisier than Intel 5300
- **Heartbeat detection unreliable**: Micro-Doppler resolution insufficient for consistent heartbeat
- **ESP-IDF learning curve**: Firmware development requires embedded C knowledge
- **WiFi interference**: Nodes sharing the same channel as data traffic adds noise
- **Placement sensitivity**: Respiration detection requires careful node positioning
### Interaction with Other ADRs
- **ADR-011** (Proof of Reality): ESP32 provides the real CSI capture for the proof bundle
- **ADR-008** (Distributed Consensus): Mesh nodes can use simplified Raft for configuration distribution
- **ADR-003** (RVF Containers): Aggregator stores CSI features in RVF format
- **ADR-004** (HNSW): Environment fingerprints from ESP32 mesh feed HNSW index
## References
- [Espressif ESP-CSI Repository](https://github.com/espressif/esp-csi)
- [ESP-IDF WiFi CSI API](https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-guides/wifi.html#wi-fi-channel-state-information)
- [ESP32 CSI Research Papers](https://ieeexplore.ieee.org/document/9439871)
- [Wi-Fi Sensing with ESP32: A Tutorial](https://arxiv.org/abs/2207.07859)
- ADR-011: Python Proof-of-Reality and Mock Elimination