wifi-densepose/docs/adr/ADR-012-esp32-csi-sensor-mesh.md

ADR-012: ESP32 CSI Sensor Mesh for Distributed Sensing

Status

Proposed

Date

2026-02-28

Context

The Hardware Reality Gap

WiFi-DensePose's Rust and Python pipelines implement real signal processing (FFT, phase unwrapping, Doppler extraction, correlation features), but the system currently has no defined path from physical WiFi hardware → CSI bytes → pipeline input. The csi_extractor.py and router_interface.py modules contain placeholder parsers that return np.random.rand() instead of real parsed data (see ADR-011).

To close this gap, we need a concrete, affordable, reproducible hardware platform that produces real CSI data and streams it into the existing pipeline.

Why ESP32

Factor	ESP32/ESP32-S3	Intel 5300 (iwl5300)	Atheros AR9580
Cost	~$5-15/node	~$50-100 (used NIC)	~$30-60 (used NIC)
Availability	Mass produced, in stock	Discontinued, eBay only	Discontinued, eBay only
CSI Support	Official ESP-IDF API	Linux CSI Tool (kernel mod)	Atheros CSI Tool
Form Factor	Standalone MCU	Requires PCIe/Mini-PCIe host	Requires PCIe host
Deployment	Battery/USB, wireless	Desktop/laptop only	Desktop/laptop only
Antenna Config	1-2 TX, 1-2 RX	3 TX, 3 RX (MIMO)	3 TX, 3 RX (MIMO)
Subcarriers	52-56 (802.11n)	30 (compressed)	56 (full)
Fidelity	Lower (consumer SoC)	Higher (dedicated NIC)	Higher (dedicated NIC)

ESP32 wins on deployability: It's the only option where a stranger can buy nodes on Amazon, flash firmware, and have a working CSI mesh in an afternoon. Intel 5300 and Atheros cards require specific hardware, kernel modifications, and legacy OS versions.

ESP-IDF CSI API

Espressif provides official CSI support through three key functions:

// 1. Configure what CSI data to capture
wifi_csi_config_t csi_config = {
    .lltf_en = true,         // Long Training Field (best for CSI)
    .htltf_en = true,        // HT-LTF
    .stbc_htltf2_en = true,  // STBC HT-LTF2
    .ltf_merge_en = true,    // Merge LTFs
    .channel_filter_en = false,
    .manu_scale = false,
};
esp_wifi_set_csi_config(&csi_config);

// 2. Register callback for received CSI data
esp_wifi_set_csi_rx_cb(csi_data_callback, NULL);

// 3. Enable CSI collection
esp_wifi_set_csi(true);

// Callback receives:
void csi_data_callback(void *ctx, wifi_csi_info_t *info) {
    // info->rx_ctrl: RSSI, noise_floor, channel, secondary_channel, etc.
    // info->buf: Raw CSI data (I/Q pairs per subcarrier)
    // info->len: Length of CSI data buffer
    // Typical: 112 bytes = 56 subcarriers × 2 (I,Q) × 1 byte each
}

Decision

We will build an ESP32 CSI Sensor Mesh as the primary hardware integration path, with a full stack from firmware to aggregator to Rust pipeline to visualization.

System Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                   ESP32 CSI Sensor Mesh                              │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐                          │
│  │ ESP32    │  │ ESP32    │  │ ESP32    │  ... (3-6 nodes)         │
│  │ Node 1  │  │ Node 2  │  │ Node 3  │                          │
│  │          │  │          │  │          │                          │
│  │ CSI Rx   │  │ CSI Rx   │  │ CSI Rx   │  ← WiFi frames from    │
│  │ FFT      │  │ FFT      │  │ FFT      │     consumer router     │
│  │ Features │  │ Features │  │ Features │                          │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘                          │
│       │              │              │                                │
│       │    UDP/TCP stream (WiFi or secondary channel)               │
│       │              │              │                                │
│       ▼              ▼              ▼                                │
│  ┌─────────────────────────────────────────┐                        │
│  │           Aggregator                     │                        │
│  │  (Laptop / Raspberry Pi / Seed device)  │                        │
│  │                                          │                        │
│  │  1. Receive CSI streams from all nodes  │                        │
│  │  2. Timestamp alignment (per-node)       │                        │
│  │  3. Feature-level fusion                │                        │
│  │  4. Feed into Rust/Python pipeline      │                        │
│  │  5. Serve WebSocket to visualization    │                        │
│  └──────────────────┬──────────────────────┘                        │
│                      │                                               │
│                      ▼                                               │
│  ┌─────────────────────────────────────────┐                        │
│  │        WiFi-DensePose Pipeline           │                        │
│  │                                          │                        │
│  │  CsiProcessor → FeatureExtractor →      │                        │
│  │  MotionDetector → PoseEstimator →       │                        │
│  │  Three.js Visualization                 │                        │
│  └─────────────────────────────────────────┘                        │
└─────────────────────────────────────────────────────────────────────┘

Node Firmware Specification

ESP-IDF project: firmware/esp32-csi-node/

firmware/esp32-csi-node/
├── CMakeLists.txt
├── sdkconfig.defaults      # Menuconfig defaults with CSI enabled
├── main/
│   ├── CMakeLists.txt
│   ├── main.c              # Entry point, WiFi init, CSI callback
│   ├── csi_collector.c     # CSI data collection and buffering
│   ├── csi_collector.h
│   ├── feature_extract.c   # On-device FFT and feature extraction
│   ├── feature_extract.h
│   ├── stream_sender.c     # UDP stream to aggregator
│   ├── stream_sender.h
│   ├── config.h             # Node configuration (SSID, aggregator IP)
│   └── Kconfig.projbuild   # Menuconfig options
├── components/
│   └── esp_dsp/            # Espressif DSP library for FFT
└── README.md               # Flash instructions

On-device processing (reduces bandwidth, node does pre-processing):

// feature_extract.c
typedef struct {
    uint32_t timestamp_ms;      // Local monotonic timestamp
    uint8_t  node_id;           // This node's ID
    int8_t   rssi;              // Received signal strength
    int8_t   noise_floor;       // Noise floor estimate
    uint8_t  channel;           // WiFi channel
    float    amplitude[56];     // |CSI| per subcarrier (from I/Q)
    float    phase[56];         // arg(CSI) per subcarrier
    float    doppler_energy;    // Motion energy from temporal FFT
    float    breathing_band;    // 0.1-0.5 Hz band power
    float    motion_band;       // 0.5-3 Hz band power
} csi_feature_frame_t;
// Size: ~470 bytes per frame
// At 100 Hz: ~47 KB/s per node, ~280 KB/s for 6 nodes

Key firmware design decisions:

1. Feature extraction on-device: Raw CSI I/Q → amplitude + phase + spectral bands. This cuts bandwidth from raw ~11 KB/frame to ~470 bytes/frame.

2. Monotonic timestamps: Each node uses its own monotonic clock. No NTP synchronization attempted between nodes - clock drift is handled at the aggregator by fusing features, not raw phases (see "Clock Drift" section below).

3. UDP streaming: Low-latency, loss-tolerant. Missing frames are acceptable; ordering is maintained via sequence numbers.

4. Configurable sampling rate: 10-100 Hz via menuconfig. 100 Hz for motion detection, 10 Hz sufficient for occupancy.

Aggregator Specification

The aggregator runs on any machine with WiFi/Ethernet to the nodes:

// In wifi-densepose-rs, new module: crates/wifi-densepose-hardware/src/esp32/
pub struct Esp32Aggregator {
    /// UDP socket listening for node streams
    socket: UdpSocket,

    /// Per-node state (last timestamp, feature buffer, drift estimate)
    nodes: HashMap<u8, NodeState>,

    /// Ring buffer of fused feature frames
    fused_buffer: VecDeque<FusedFrame>,

    /// Channel to pipeline
    pipeline_tx: mpsc::Sender<CsiData>,
}

/// Fused frame from all nodes for one time window
pub struct FusedFrame {
    /// Timestamp (aggregator local, monotonic)
    timestamp: Instant,

    /// Per-node features (may have gaps if node dropped)
    node_features: Vec<Option<CsiFeatureFrame>>,

    /// Cross-node correlation (computed by aggregator)
    cross_node_correlation: Array2<f64>,

    /// Fused motion energy (max across nodes)
    fused_motion_energy: f64,

    /// Fused breathing band (coherent sum where phase aligns)
    fused_breathing_band: f64,
}

Clock Drift Handling

ESP32 crystal oscillators drift ~20-50 ppm. Over 1 hour, two nodes may diverge by 72-180ms. This makes raw phase alignment across nodes impossible.

Solution: Feature-level fusion, not signal-level fusion.

Signal-level (WRONG for ESP32):
  Align raw I/Q samples across nodes → requires <1µs sync → impractical

Feature-level (CORRECT for ESP32):
  Each node: raw CSI → amplitude + phase + spectral features (local)
  Aggregator: collect features → correlate → fuse decisions
  No cross-node phase alignment needed

Specifically:

• Motion energy: Take max across nodes (any node seeing motion = motion)
• Breathing band: Use node with highest SNR as primary, others as corroboration
• Location: Cross-node amplitude ratios estimate position (no phase needed)

Sensing Capabilities by Deployment

Capability	1 Node	3 Nodes	6 Nodes	Evidence
Presence detection	Good	Excellent	Excellent	Single-node RSSI variance
Coarse motion	Good	Excellent	Excellent	Doppler energy
Room-level location	None	Good	Excellent	Amplitude ratios
Respiration	Marginal	Good	Good	0.1-0.5 Hz band, placement-sensitive
Heartbeat	Poor	Poor-Marginal	Marginal	Requires ideal placement, low noise
Multi-person count	None	Marginal	Good	Spatial diversity
Pose estimation	None	Poor	Marginal	Requires model + sufficient diversity

Honest assessment: ESP32 CSI is lower fidelity than Intel 5300 or Atheros. Heartbeat detection is placement-sensitive and unreliable. Respiration works with good placement. Motion and presence are solid.

Failure Modes and Mitigations

Failure Mode	Severity	Mitigation
Multipath dominates in cluttered rooms	High	Mesh diversity: 3+ nodes from different angles
Person occludes path between node and router	Medium	Mesh: other nodes still have clear paths
Clock drift ruins cross-node fusion	Medium	Feature-level fusion only; no cross-node phase alignment
UDP packet loss during high traffic	Low	Sequence numbers, interpolation for gaps <100ms
ESP32 WiFi driver bugs with CSI	Medium	Pin ESP-IDF version, test on known-good boards
Node power failure	Low	Aggregator handles missing nodes gracefully

Bill of Materials (Starter Kit)

Item	Quantity	Unit Cost	Total
ESP32-S3-DevKitC-1	3	$10	$30
USB-A to USB-C cables	3	$3	$9
USB power adapter (multi-port)	1	$15	$15
Consumer WiFi router (any)	1	$0 (existing)	$0
Aggregator (laptop or Pi 4)	1	$0 (existing)	$0
Total			$54

Minimal Build Spec (Clone-Flash-Run)

# Step 1: Flash one node (requires ESP-IDF installed)
cd firmware/esp32-csi-node
idf.py set-target esp32s3
idf.py menuconfig  # Set WiFi SSID/password, aggregator IP
idf.py build flash monitor

# Step 2: Run aggregator (Docker)
docker compose -f docker-compose.esp32.yml up

# Step 3: Verify with proof bundle
# Aggregator captures 10 seconds, produces feature JSON, verifies hash
docker exec aggregator python verify_esp32.py

# Step 4: Open visualization
open http://localhost:3000  # Three.js dashboard

Proof of Reality for ESP32

firmware/esp32-csi-node/proof/
├── captured_csi_10sec.bin          # Real 10-second CSI capture from ESP32
├── captured_csi_meta.json          # Board: ESP32-S3-DevKitC, ESP-IDF: 5.2, Router: TP-Link AX1800
├── expected_features.json          # Feature extraction output
├── expected_features.sha256        # Hash verification
└── capture_photo.jpg               # Photo of actual hardware setup

Consequences

Positive

• $54 starter kit: Lowest possible barrier to real CSI data
• Mass available hardware: ESP32 boards are in stock globally
• Real data path: Eliminates every np.random.rand() placeholder with actual hardware input
• Proof artifact: Captured CSI + expected hash proves the pipeline processes real data
• Scalable mesh: Add nodes for more coverage without changing software
• Feature-level fusion: Avoids the impossible problem of cross-node phase synchronization

Negative

• Lower fidelity than research NICs: ESP32 CSI is noisier than Intel 5300
• Heartbeat detection unreliable: Micro-Doppler resolution insufficient for consistent heartbeat
• ESP-IDF learning curve: Firmware development requires embedded C knowledge
• WiFi interference: Nodes sharing the same channel as data traffic adds noise
• Placement sensitivity: Respiration detection requires careful node positioning

Interaction with Other ADRs

• ADR-011 (Proof of Reality): ESP32 provides the real CSI capture for the proof bundle
• ADR-008 (Distributed Consensus): Mesh nodes can use simplified Raft for configuration distribution
• ADR-003 (RVF Containers): Aggregator stores CSI features in RVF format
• ADR-004 (HNSW): Environment fingerprints from ESP32 mesh feed HNSW index

References

• Espressif ESP-CSI Repository
• ESP-IDF WiFi CSI API
• ESP32 CSI Research Papers
• Wi-Fi Sensing with ESP32: A Tutorial
• ADR-011: Python Proof-of-Reality and Mock Elimination