dearsky/wifi-densepose
1
0
Fork 0
Code Issues 20 Pull Requests 5 Actions Packages Projects Releases 1 Wiki Activity

Research: Deployment Feasibility & Use Cases #57

New Issue
Closed
opened 2026-03-01 15:22:36 +08:00 by bryannwu · 2 comments
bryannwu commented 2026-03-01 15:22:36 +08:00 (Migrated from github.com)
Copy Link

Overview

This issue tracks a research effort to evaluate the deployment feasibility of wifi-densepose and identify practical use cases.

Research Areas

1. Hardware Requirements

  • What WiFi hardware (routers/APs) is required? Does it need custom firmware?
  • Minimum spec for the compute node (CPU/GPU requirements for real-time inference)?
  • Does it work with commodity off-the-shelf hardware, or does it need specific chipsets (e.g., 802.11ac/ax with CSI extraction support)?

2. Deployment Topologies

  • Single-room setup (one TX, one RX antenna pair)
  • Multi-room coverage — can multiple AP pairs be coordinated?
  • Edge deployment on low-power devices (RPi, NVIDIA Jetson, Mac mini M-series)?
  • Cloud-offload mode vs fully local inference?

3. Use Cases (to investigate)

  • Home: fall detection for elderly, occupancy sensing for smart home automation
  • Commercial: people counting in retail, workspace utilization analytics
  • Security: perimeter monitoring without cameras (privacy-preserving)
  • Health: contactless vital sign monitoring (breathing rate, heart rate estimation)
  • Accessibility: gesture recognition for controlling smart home devices

4. Privacy & Regulatory Considerations

  • No camera = better privacy, but CSI data can still reveal behavioral patterns
  • Are there legal/regulatory constraints around passive WiFi sensing in occupied spaces?
  • How does this compare to radar-based sensing (e.g., Google Soli, mmWave)?

5. Integration Points

  • Home Assistant / MQTT integration
  • REST API for downstream consumers
  • Compatibility with existing DensePose model weights vs custom-trained models

6. Known Limitations to Document

  • Range / accuracy degradation with walls, furniture, multi-occupant scenarios
  • Latency baseline for real-time use cases
  • Training data requirements if custom environments need fine-tuning

Deliverables

  • Hardware compatibility matrix
  • Deployment guide for at least one end-to-end scenario
  • Performance benchmarks (latency, accuracy vs ground truth)
  • List of viable use cases with effort/value assessment

Related

## Overview This issue tracks a research effort to evaluate the deployment feasibility of wifi-densepose and identify practical use cases. ## Research Areas ### 1. Hardware Requirements - What WiFi hardware (routers/APs) is required? Does it need custom firmware? - Minimum spec for the compute node (CPU/GPU requirements for real-time inference)? - Does it work with commodity off-the-shelf hardware, or does it need specific chipsets (e.g., 802.11ac/ax with CSI extraction support)? ### 2. Deployment Topologies - Single-room setup (one TX, one RX antenna pair) - Multi-room coverage — can multiple AP pairs be coordinated? - Edge deployment on low-power devices (RPi, NVIDIA Jetson, Mac mini M-series)? - Cloud-offload mode vs fully local inference? ### 3. Use Cases (to investigate) - **Home**: fall detection for elderly, occupancy sensing for smart home automation - **Commercial**: people counting in retail, workspace utilization analytics - **Security**: perimeter monitoring without cameras (privacy-preserving) - **Health**: contactless vital sign monitoring (breathing rate, heart rate estimation) - **Accessibility**: gesture recognition for controlling smart home devices ### 4. Privacy & Regulatory Considerations - No camera = better privacy, but CSI data can still reveal behavioral patterns - Are there legal/regulatory constraints around passive WiFi sensing in occupied spaces? - How does this compare to radar-based sensing (e.g., Google Soli, mmWave)? ### 5. Integration Points - Home Assistant / MQTT integration - REST API for downstream consumers - Compatibility with existing DensePose model weights vs custom-trained models ### 6. Known Limitations to Document - Range / accuracy degradation with walls, furniture, multi-occupant scenarios - Latency baseline for real-time use cases - Training data requirements if custom environments need fine-tuning ## Deliverables - [ ] Hardware compatibility matrix - [ ] Deployment guide for at least one end-to-end scenario - [ ] Performance benchmarks (latency, accuracy vs ground truth) - [ ] List of viable use cases with effort/value assessment ## Related - Project README: https://github.com/ruvnet/wifi-densepose
bryannwu commented 2026-03-01 15:29:55 +08:00 (Migrated from github.com)
Copy Link

Closing — created by mistake.

Closing — created by mistake.
ruvnet commented 2026-03-01 15:34:42 +08:00 (Migrated from github.com)
Copy Link

Deployment Feasibility & Use Cases — Deep Research Report

Thanks for the thorough research framework, @bryannwu. I went through every relevant source file, ADR, and test to give you grounded answers with specific code references. This is long — use the section links.

1. Hardware Requirements

What WiFi Hardware Is Actually Supported

Hardware Data Type Subcarriers Sample Rate Cost Status
ESP32-S3 Raw CSI (I/Q) 56 (20 MHz) 20-100 Hz ~$8/node Parser + aggregator complete; firmware pending
Atheros AR9580 Raw CSI 56-114 100+ Hz ~$30 Supported via CsiFrame (hardware-agnostic)
Intel 5300 Raw CSI 30 ~100 Hz ~$20 Supported via CsiFrame
Any Windows WiFi RSSI only N/A ~2 Hz $0 Fully working (netsh wlan show interfaces)
macOS CoreWLAN RSSI + noise floor N/A ~0.3-0.5 Hz $0 Proposed (ADR-025)

Key distinction: CSI-capable hardware (ESP32, Atheros, Intel 5300) enables pose estimation, vital signs, and through-wall sensing. Commodity WiFi (Windows/macOS) only provides RSSI-based presence detection — significantly less capable. This is a physics limitation, not a software one.

ESP32 details (wifi-densepose-hardware/src/esp32_parser.rs):

  • Binary frame format with magic number 0xC5110001
  • Up to 4 antennas, 256 subcarriers max
  • UDP streaming on port 5005 with per-node sequence tracking
  • Multi-node aggregation with 1024-frame channel capacity

Windows multi-BSSID pipeline (main.rs:467-527):

  • Parses netsh wlan show networks mode=bssid for multiple visible APs
  • Registry: 32 BSSIDs max, 30-second staleness window
  • Enhanced features: motion detection, breathing estimation, posture classification, signal quality gating

Compute Requirements

Metric Value Source
Docker image 132 MB Dockerfile.rust multi-stage build
Binary size 2.1 MB Single stripped Rust binary
Throughput 54,000 frames/sec README benchmark (release build)
Per-frame latency ~100 us Derived from throughput
Breathing buffer RAM 3.4-6.7 MB/zone With ruvector compression (breathing.rs:19-22)
Embedding model 60 KB at INT8 Fits on ESP32 (520 KB SRAM)
GPU Not required All inference is CPU-based

Bottom line: A Raspberry Pi 4 or any modern laptop can run the sensing server. No GPU needed. The Docker image runs on anything that supports linux/amd64.

Commodity Hardware — What You Actually Get

On a laptop with no special hardware ($0 cost):

  • Presence detection (ABSENT / PRESENT_STILL / ACTIVE)
  • Motion classification (None, Fine, Gross, Periodic, Tremor)
  • Multi-BSSID signal quality monitoring
  • Breathing estimation (marginal quality at 2 Hz sample rate)
  • No heartbeat (needs >20 Hz — impossible with RSSI)
  • No pose estimation (needs CSI subcarriers)
  • No through-wall sensing (needs CSI phase information)

With ESP32-S3 nodes (~$8-54 for a 3-6 node mesh):

  • Everything above, plus:
  • 17-keypoint pose estimation (with trained model)
  • Breathing rate (6-30 BPM, FFT-based, vital_signs.rs)
  • Heart rate (40-120 BPM, micro-Doppler, heartbeat.rs)
  • Through-wall presence detection
  • Fresnel zone body tracking

2. Deployment Topologies

Single-Room (Core Model)

The default deployment. One sensing server, one or more WiFi nodes in a single space.

[ESP32 node] --UDP:5005--> [sensing-server] --HTTP:3000--> [browser UI]
                                            --WS:3000----> [WebSocket clients]

CLI: sensing-server --source esp32 --udp-port 5005
Or Docker: docker run -p 3000:3000 ruvnet/wifi-densepose:latest

Multi-Room / Multi-AP

Supported through the WiFi-Mat module (wifi-densepose-mat crate):

  • Trilateration from 3+ sensors (triangulation.rs:47-285) using log-distance path loss model (indoor path loss exponent = 3.0)
  • TDoA triangulation via Neumann solver for time-difference positioning (triangulation.rs:388-470)
  • GDOP calculation for position uncertainty estimation (triangulation.rs:246-284, max threshold 5.0m)
  • Zone-based scanning with configurable bounds (rectangle, circle, polygon) (domain/scan_zone.rs)
  • Multi-node aggregation supports 256 node IDs with per-node sequence gap detection

Edge Deployment

Platform Feasibility Notes
ESP32-S3 Yes Model fits in 60 KB at INT8, <2ms inference per frame
Raspberry Pi 4 Yes 2.1 MB binary, CPU-only, ~54K fps headroom
NVIDIA Jetson Yes Overkill but works — GPU unused, pure CPU inference
Mac mini M-series Yes Runs Docker or native; macOS RSSI adapter proposed (ADR-025)
Browser/WASM Planned RVF WASM runtime proposed in ADR-009

RVF model containers (rvf_container.rs) enable portable edge deployment:

  • 64-byte aligned segments with CRC32 integrity
  • Progressive 3-layer loading: Layer A (instant start) -> B (refinement) -> C (full fidelity)
  • Segment types: weights, quantization codebook, metadata, embeddings, LoRA adapters
  • SONA profiles for per-room adaptation with only 1,792 parameters per room

Cloud vs Local

The architecture is entirely local-first. I searched every crate — there are zero external API calls, zero cloud dependencies, zero telemetry. All processing happens on the sensing server. Data never leaves the local machine unless a WebSocket client connects. This is by design for privacy.

3. Use Cases — What Is Actually Implemented

Fully Implemented and Testable Today

Use Case Module Key Metric Hardware Needed
Presence detection classifier.py, main.rs:449-456 3 states, 0.87 confidence Any WiFi
Motion classification movement.rs:29-31 5 levels (None to Tremor) Any WiFi
Breathing rate vital_signs.rs, breathing.rs 6-30 BPM, FFT-based ESP32 CSI
Heart rate heartbeat.rs:82-96 40-120 BPM, micro-Doppler ESP32 CSI
Pose estimation graph_transformer.rs, main.rs 17 COCO keypoints ESP32 CSI + trained model
Through-wall detection fresnel.rs, MAT module 5m depth, 30cm concrete ESP32 CSI
Survivor search (SAR) wifi-densepose-mat START triage, 3D localization Multi-AP ESP32 mesh
Self-supervised fingerprinting embedding.rs (ADR-024) 128-dim vectors, 53K params ESP32 CSI
Multi-BSSID sensing main.rs:507-527 32 BSSIDs, quality-gated Windows WiFi

Disaster/SAR Detail (WiFi-Mat)

The wifi-densepose-mat crate is a complete disaster response system:

  • 5 disaster types: earthquake, building collapse, avalanche, flood, mine
  • START triage protocol: Immediate / Delayed / Minor / Deceased
  • Breathing + heartbeat + movement ensemble classifier
  • 3D localization through debris via trilateration
  • ML-based debris attenuation prediction by material type
  • Priority-based alert system with dispatch and acknowledgment
  • Full domain event sourcing: SurvivorDetected, AlertGenerated, ZoneScanCompleted

Use Case Feasibility Matrix

Use Case Commodity WiFi ESP32 CSI Multi-AP
People counting Yes (coarse) Yes (3-5/AP) Yes (15-20)
Fall detection Partial (motion only) Yes (pose-based) Yes
Elderly monitoring Partial (presence) Yes (vitals + pose) Yes
Workspace utilization Yes Yes Yes
Retail foot traffic Yes Yes Yes (with zones)
Perimeter security Yes (motion) Yes (through-wall) Yes
Contactless vitals No Yes Yes
Gesture recognition No Partial (needs training) Partial
Home automation (occupancy) Yes Yes Yes
Smart HVAC Yes Yes Yes

4. Privacy and Regulatory Considerations

What CSI Data Actually Is

CSI is complex-valued I/Q pairs per WiFi subcarrier — raw numbers, not images. From csi_frame.rs:128-137:

pub struct SubcarrierData {
    pub i: i16,   // In-phase component
    pub q: i16,   // Quadrature component
    pub index: i16,
}

These get converted to amplitude (sqrt(I^2 + Q^2)) and phase (atan2(Q, I)) — still just arrays of floating-point numbers. No camera, no video, no audio, no pixels are ever captured or processed.

What Gets Transmitted Over WebSocket

The SensingUpdate message (main.rs:162-201) contains only:

  • Timestamp and source label
  • Node RSSI values and positions
  • Statistical features: mean, variance, spectral band power
  • Classification: motion level, presence boolean, confidence float
  • Signal field: 20x20 grid of floats
  • Vital signs (optional): BPM numbers and confidence scores

All numerical signal measurements. No PII, no biometrics in the legal sense, no images, no audio.

Privacy Advantages Over Alternatives

Factor WiFi CSI Camera mmWave Radar Wearable
Captures images No Yes No No
Records audio No Often No No
Identifies faces No Yes No N/A
Works through walls Yes No Limited N/A
Requires consent device No No No Yes
GDPR video applies No Yes No No
HIPAA imaging applies No Yes No No
Data is human-readable No (float arrays) Yes (video) No Varies

Regulatory Landscape

WiFi sensing occupies a regulatory gray area:

  • Not video surveillance under any jurisdiction — no camera involved
  • Not wiretapping — WiFi CSI is a physical-layer measurement of existing RF, not intercepted communications
  • Behavioral inference risk: CSI data CAN reveal activity patterns (sleeping, cooking, bathroom visits) even without images. This could raise concerns under GDPR Article 9 (behavioral profiling) depending on interpretation
  • FCC/CE: Uses existing WiFi transmissions — no additional RF emission, no spectrum licensing
  • Comparable to: Radar-based sensing (Google Soli, mmWave) which the FCC approved for in-home use

Local-Only Architecture

All processing is on the local machine. No cloud dependencies in any crate. No external API calls. No telemetry. No analytics. The Docker container runs fully self-contained. Data never leaves the premises unless you explicitly connect a remote WebSocket client.

5. Integration Points

REST API (28 endpoints)

Sensing Server (main.rs routes):

GET  /health                        -> server status
GET  /health/live                   -> liveness probe
GET  /health/ready                  -> readiness probe
GET  /health/version                -> build version
GET  /health/metrics                -> performance metrics
GET  /api/v1/info                   -> features, backend type
GET  /api/v1/sensing/latest         -> live CSI sensing data
GET  /api/v1/vital-signs            -> breathing + heartbeat BPM
GET  /api/v1/pose/current           -> 17 COCO keypoints
GET  /api/v1/pose/stats             -> detection statistics
GET  /api/v1/pose/zones/summary     -> per-zone person count
GET  /api/v1/model/info             -> RVF container metadata
GET  /api/v1/model/layers           -> progressive loading status
GET  /api/v1/model/segments         -> RVF segment list
GET  /api/v1/sona/profiles          -> SONA adaptation profiles
POST /api/v1/sona/activate          -> activate per-room profile
GET  /api/v1/stream/status          -> active clients, FPS

MAT REST API (wifi-densepose-mat/src/api/mod.rs):

GET  /api/v1/mat/events             -> list disaster events
POST /api/v1/mat/events             -> create event
GET  /api/v1/mat/events/{id}        -> event details
GET  /api/v1/mat/events/{id}/zones  -> zones
GET  /api/v1/mat/events/{id}/survivors -> survivors (filterable)
GET  /api/v1/mat/events/{id}/alerts -> alerts (filterable)
POST /api/v1/mat/alerts/{id}/ack    -> acknowledge alert
POST /api/v1/mat/scan/csi           -> push raw CSI data
POST /api/v1/mat/scan/control       -> start/stop/pause scanning
GET  /api/v1/mat/scan/status        -> pipeline status
GET  /api/v1/mat/events/domain      -> domain event log

WebSocket Streaming

Endpoint Data Format
ws://host:3000/ws/sensing Real-time sensing updates JSON per tick
ws://host:3000/ws/pose COCO 17-keypoint skeleton JSON
ws://host/ws/mat/stream Survivor/alert events JSON with subscribe/unsubscribe

Data Formats

  • All API responses: JSON
  • WebSocket messages: JSON (no binary protocol)
  • ESP32 ingestion: Binary UDP (ADR-018 format, 20-byte header + I/Q payload)
  • Model packaging: RVF binary (64-byte aligned segments, CRC32)

MQTT / Home Assistant

Not currently implemented. No MQTT client, no Home Assistant integration exists anywhere in the codebase. This is a clear integration gap for smart home deployments and would be a valuable contribution. The REST API and WebSocket streaming provide the data — an MQTT bridge would be straightforward to build.

6. Known Limitations (Honest Assessment)

Physics Constraints

Constraint Value Source
People per AP ~3-5 (56 subcarriers) Physics limit, not software
Multi-AP scaling Linear (~15-20 with 4 APs) Triangulation math
Max sensing range ~10m open field README
Through-wall depth ~5m, 30cm concrete MAT default config
Breathing displacement 4-12mm (detectable) fresnel.rs:88-206
Heartbeat displacement ~0.5mm (much harder) heartbeat.rs:133-136
Min breathing samples 40 (~2s at 20 Hz) vital_signs.rs:28
Min heartbeat samples 30 (~1.5s at 20 Hz) vital_signs.rs:29

Sample Rate Limitations

Source Rate Suitable For
ESP32 CSI 20-100 Hz Everything including heartbeat
Windows WiFi ~2 Hz Presence, motion, marginal breathing
macOS (proposed) ~0.3-0.5 Hz Presence and motion only

Windows WiFi at 2 Hz is explicitly noted as "insufficient for heartbeat" (vital_signs.rs:92).

Accuracy Caveats

  1. Amplitude-based detection is fragile in dynamic environments (issue #51). The rule-based classifier uses fixed thresholds and works best in enclosed, static rooms. The Rust CSI pipeline mitigates this with Hampel filtering, conjugate multiplication, and dynamic subcarrier partitioning — but has not been validated cross-scene.

  2. Heartbeat is much harder than breathing: 0.5mm vs 10mm chest displacement, significantly lower SNR. Confidence threshold is stricter (0.4 vs 0.3).

  3. Pose estimation requires a trained model. The current derive_pose_from_sensing function generates COCO keypoints from signal-field heuristics, not from trained weights. True pose estimation requires loading a trained RVF model via --model. No pre-trained model is shipped. Training requires the MM-Fi dataset and compute time.

  4. No cross-scene validation data exists yet. The system has 229+ unit tests and deterministic proof bundles, but no published metrics comparing accuracy across different rooms, AP placements, or subjects.

What Is Not Yet Built

Component Status ADR
ESP32 firmware (.bin) Pending ADR-018
macOS CoreWLAN adapter Proposed ADR-025
Pre-trained model weights Not shipped ADR-023
MQTT / Home Assistant bridge Not started --
Cross-scene benchmarks Not done --
WASM runtime for browser Proposed ADR-009

Deliverables Checklist

Per your requested deliverables:

  • Hardware compatibility matrix — Section 1, covering ESP32, Atheros, Intel 5300, Windows WiFi, macOS
  • Deployment guide for end-to-end scenario — Docker quick start exists; a dedicated guide for ESP32-to-dashboard would be valuable
  • Performance benchmarks vs ground truth — Unit-level benchmarks exist (54K fps, breathing within 2 BPM tolerance); no ground-truth comparison against camera-based systems yet
  • Use case list with effort/value — Section 3 feasibility matrix

Recommended Next Steps

For someone looking to deploy this:

  1. Quickest path to value: docker run -p 3000:3000 ruvnet/wifi-densepose:latest — see the UI, hit the REST API, verify the signal processing chain works
  2. Commodity WiFi demo: Run on Windows laptop — presence detection and motion classification work today with zero hardware cost
  3. Full CSI path: Get 3x ESP32-S3 boards (~$24), flash firmware (once ADR-018 firmware is complete), and you have breathing/heartbeat/pose sensing
  4. Integration: The REST API and WebSocket streaming are production-ready; an MQTT bridge for Home Assistant would be the highest-impact community contribution

Happy to discuss any specific area in more depth.

-- ruFlo AI

## Deployment Feasibility & Use Cases — Deep Research Report Thanks for the thorough research framework, @bryannwu. I went through every relevant source file, ADR, and test to give you grounded answers with specific code references. This is long — use the section links. --- ### 1. Hardware Requirements #### What WiFi Hardware Is Actually Supported | Hardware | Data Type | Subcarriers | Sample Rate | Cost | Status | |----------|----------|-------------|-------------|------|--------| | **ESP32-S3** | Raw CSI (I/Q) | 56 (20 MHz) | 20-100 Hz | ~$8/node | Parser + aggregator complete; firmware pending | | **Atheros AR9580** | Raw CSI | 56-114 | 100+ Hz | ~$30 | Supported via `CsiFrame` (hardware-agnostic) | | **Intel 5300** | Raw CSI | 30 | ~100 Hz | ~$20 | Supported via `CsiFrame` | | **Any Windows WiFi** | RSSI only | N/A | ~2 Hz | $0 | Fully working (`netsh wlan show interfaces`) | | **macOS CoreWLAN** | RSSI + noise floor | N/A | ~0.3-0.5 Hz | $0 | Proposed (ADR-025) | **Key distinction**: CSI-capable hardware (ESP32, Atheros, Intel 5300) enables pose estimation, vital signs, and through-wall sensing. Commodity WiFi (Windows/macOS) only provides RSSI-based presence detection — significantly less capable. This is a physics limitation, not a software one. **ESP32 details** (`wifi-densepose-hardware/src/esp32_parser.rs`): - Binary frame format with magic number `0xC5110001` - Up to 4 antennas, 256 subcarriers max - UDP streaming on port 5005 with per-node sequence tracking - Multi-node aggregation with 1024-frame channel capacity **Windows multi-BSSID pipeline** (`main.rs:467-527`): - Parses `netsh wlan show networks mode=bssid` for multiple visible APs - Registry: 32 BSSIDs max, 30-second staleness window - Enhanced features: motion detection, breathing estimation, posture classification, signal quality gating #### Compute Requirements | Metric | Value | Source | |--------|-------|--------| | Docker image | 132 MB | `Dockerfile.rust` multi-stage build | | Binary size | 2.1 MB | Single stripped Rust binary | | Throughput | 54,000 frames/sec | README benchmark (release build) | | Per-frame latency | ~100 us | Derived from throughput | | Breathing buffer RAM | 3.4-6.7 MB/zone | With ruvector compression (`breathing.rs:19-22`) | | Embedding model | 60 KB at INT8 | Fits on ESP32 (520 KB SRAM) | | GPU | **Not required** | All inference is CPU-based | **Bottom line**: A Raspberry Pi 4 or any modern laptop can run the sensing server. No GPU needed. The Docker image runs on anything that supports `linux/amd64`. #### Commodity Hardware — What You Actually Get On a laptop with no special hardware ($0 cost): - Presence detection (ABSENT / PRESENT_STILL / ACTIVE) - Motion classification (None, Fine, Gross, Periodic, Tremor) - Multi-BSSID signal quality monitoring - Breathing estimation (marginal quality at 2 Hz sample rate) - No heartbeat (needs >20 Hz — impossible with RSSI) - No pose estimation (needs CSI subcarriers) - No through-wall sensing (needs CSI phase information) With ESP32-S3 nodes (~$8-54 for a 3-6 node mesh): - Everything above, plus: - 17-keypoint pose estimation (with trained model) - Breathing rate (6-30 BPM, FFT-based, `vital_signs.rs`) - Heart rate (40-120 BPM, micro-Doppler, `heartbeat.rs`) - Through-wall presence detection - Fresnel zone body tracking --- ### 2. Deployment Topologies #### Single-Room (Core Model) The default deployment. One sensing server, one or more WiFi nodes in a single space. ``` [ESP32 node] --UDP:5005--> [sensing-server] --HTTP:3000--> [browser UI] --WS:3000----> [WebSocket clients] ``` CLI: `sensing-server --source esp32 --udp-port 5005` Or Docker: `docker run -p 3000:3000 ruvnet/wifi-densepose:latest` #### Multi-Room / Multi-AP Supported through the **WiFi-Mat module** (`wifi-densepose-mat` crate): - **Trilateration** from 3+ sensors (`triangulation.rs:47-285`) using log-distance path loss model (indoor path loss exponent = 3.0) - **TDoA triangulation** via Neumann solver for time-difference positioning (`triangulation.rs:388-470`) - **GDOP calculation** for position uncertainty estimation (`triangulation.rs:246-284`, max threshold 5.0m) - **Zone-based scanning** with configurable bounds (rectangle, circle, polygon) (`domain/scan_zone.rs`) - Multi-node aggregation supports 256 node IDs with per-node sequence gap detection #### Edge Deployment | Platform | Feasibility | Notes | |----------|------------|-------| | **ESP32-S3** | Yes | Model fits in 60 KB at INT8, <2ms inference per frame | | **Raspberry Pi 4** | Yes | 2.1 MB binary, CPU-only, ~54K fps headroom | | **NVIDIA Jetson** | Yes | Overkill but works — GPU unused, pure CPU inference | | **Mac mini M-series** | Yes | Runs Docker or native; macOS RSSI adapter proposed (ADR-025) | | **Browser/WASM** | Planned | RVF WASM runtime proposed in ADR-009 | **RVF model containers** (`rvf_container.rs`) enable portable edge deployment: - 64-byte aligned segments with CRC32 integrity - Progressive 3-layer loading: Layer A (instant start) -> B (refinement) -> C (full fidelity) - Segment types: weights, quantization codebook, metadata, embeddings, LoRA adapters - SONA profiles for per-room adaptation with only 1,792 parameters per room #### Cloud vs Local **The architecture is entirely local-first.** I searched every crate — there are zero external API calls, zero cloud dependencies, zero telemetry. All processing happens on the sensing server. Data never leaves the local machine unless a WebSocket client connects. This is by design for privacy. --- ### 3. Use Cases — What Is Actually Implemented #### Fully Implemented and Testable Today | Use Case | Module | Key Metric | Hardware Needed | |----------|--------|-----------|----------------| | **Presence detection** | `classifier.py`, `main.rs:449-456` | 3 states, 0.87 confidence | Any WiFi | | **Motion classification** | `movement.rs:29-31` | 5 levels (None to Tremor) | Any WiFi | | **Breathing rate** | `vital_signs.rs`, `breathing.rs` | 6-30 BPM, FFT-based | ESP32 CSI | | **Heart rate** | `heartbeat.rs:82-96` | 40-120 BPM, micro-Doppler | ESP32 CSI | | **Pose estimation** | `graph_transformer.rs`, `main.rs` | 17 COCO keypoints | ESP32 CSI + trained model | | **Through-wall detection** | `fresnel.rs`, MAT module | 5m depth, 30cm concrete | ESP32 CSI | | **Survivor search (SAR)** | `wifi-densepose-mat` | START triage, 3D localization | Multi-AP ESP32 mesh | | **Self-supervised fingerprinting** | `embedding.rs` (ADR-024) | 128-dim vectors, 53K params | ESP32 CSI | | **Multi-BSSID sensing** | `main.rs:507-527` | 32 BSSIDs, quality-gated | Windows WiFi | #### Disaster/SAR Detail (WiFi-Mat) The `wifi-densepose-mat` crate is a complete disaster response system: - 5 disaster types: earthquake, building collapse, avalanche, flood, mine - START triage protocol: Immediate / Delayed / Minor / Deceased - Breathing + heartbeat + movement ensemble classifier - 3D localization through debris via trilateration - ML-based debris attenuation prediction by material type - Priority-based alert system with dispatch and acknowledgment - Full domain event sourcing: `SurvivorDetected`, `AlertGenerated`, `ZoneScanCompleted` #### Use Case Feasibility Matrix | Use Case | Commodity WiFi | ESP32 CSI | Multi-AP | |----------|---------------|-----------|----------| | People counting | Yes (coarse) | Yes (3-5/AP) | Yes (15-20) | | Fall detection | Partial (motion only) | Yes (pose-based) | Yes | | Elderly monitoring | Partial (presence) | Yes (vitals + pose) | Yes | | Workspace utilization | Yes | Yes | Yes | | Retail foot traffic | Yes | Yes | Yes (with zones) | | Perimeter security | Yes (motion) | Yes (through-wall) | Yes | | Contactless vitals | No | Yes | Yes | | Gesture recognition | No | Partial (needs training) | Partial | | Home automation (occupancy) | Yes | Yes | Yes | | Smart HVAC | Yes | Yes | Yes | --- ### 4. Privacy and Regulatory Considerations #### What CSI Data Actually Is CSI is complex-valued I/Q pairs per WiFi subcarrier — raw numbers, not images. From `csi_frame.rs:128-137`: ```rust pub struct SubcarrierData { pub i: i16, // In-phase component pub q: i16, // Quadrature component pub index: i16, } ``` These get converted to amplitude (`sqrt(I^2 + Q^2)`) and phase (`atan2(Q, I)`) — still just arrays of floating-point numbers. **No camera, no video, no audio, no pixels are ever captured or processed.** #### What Gets Transmitted Over WebSocket The `SensingUpdate` message (`main.rs:162-201`) contains only: - Timestamp and source label - Node RSSI values and positions - Statistical features: mean, variance, spectral band power - Classification: motion level, presence boolean, confidence float - Signal field: 20x20 grid of floats - Vital signs (optional): BPM numbers and confidence scores All numerical signal measurements. No PII, no biometrics in the legal sense, no images, no audio. #### Privacy Advantages Over Alternatives | Factor | WiFi CSI | Camera | mmWave Radar | Wearable | |--------|---------|--------|-------------|----------| | Captures images | No | **Yes** | No | No | | Records audio | No | Often | No | No | | Identifies faces | No | **Yes** | No | N/A | | Works through walls | Yes | No | Limited | N/A | | Requires consent device | No | No | No | **Yes** | | GDPR video applies | No | **Yes** | No | No | | HIPAA imaging applies | No | **Yes** | No | No | | Data is human-readable | No (float arrays) | **Yes** (video) | No | Varies | #### Regulatory Landscape WiFi sensing occupies a regulatory gray area: - **Not video surveillance** under any jurisdiction — no camera involved - **Not wiretapping** — WiFi CSI is a physical-layer measurement of existing RF, not intercepted communications - **Behavioral inference risk**: CSI data CAN reveal activity patterns (sleeping, cooking, bathroom visits) even without images. This could raise concerns under GDPR Article 9 (behavioral profiling) depending on interpretation - **FCC/CE**: Uses existing WiFi transmissions — no additional RF emission, no spectrum licensing - **Comparable to**: Radar-based sensing (Google Soli, mmWave) which the FCC approved for in-home use #### Local-Only Architecture All processing is on the local machine. No cloud dependencies in any crate. No external API calls. No telemetry. No analytics. The Docker container runs fully self-contained. Data never leaves the premises unless you explicitly connect a remote WebSocket client. --- ### 5. Integration Points #### REST API (28 endpoints) **Sensing Server** (`main.rs` routes): ``` GET /health -> server status GET /health/live -> liveness probe GET /health/ready -> readiness probe GET /health/version -> build version GET /health/metrics -> performance metrics GET /api/v1/info -> features, backend type GET /api/v1/sensing/latest -> live CSI sensing data GET /api/v1/vital-signs -> breathing + heartbeat BPM GET /api/v1/pose/current -> 17 COCO keypoints GET /api/v1/pose/stats -> detection statistics GET /api/v1/pose/zones/summary -> per-zone person count GET /api/v1/model/info -> RVF container metadata GET /api/v1/model/layers -> progressive loading status GET /api/v1/model/segments -> RVF segment list GET /api/v1/sona/profiles -> SONA adaptation profiles POST /api/v1/sona/activate -> activate per-room profile GET /api/v1/stream/status -> active clients, FPS ``` **MAT REST API** (`wifi-densepose-mat/src/api/mod.rs`): ``` GET /api/v1/mat/events -> list disaster events POST /api/v1/mat/events -> create event GET /api/v1/mat/events/{id} -> event details GET /api/v1/mat/events/{id}/zones -> zones GET /api/v1/mat/events/{id}/survivors -> survivors (filterable) GET /api/v1/mat/events/{id}/alerts -> alerts (filterable) POST /api/v1/mat/alerts/{id}/ack -> acknowledge alert POST /api/v1/mat/scan/csi -> push raw CSI data POST /api/v1/mat/scan/control -> start/stop/pause scanning GET /api/v1/mat/scan/status -> pipeline status GET /api/v1/mat/events/domain -> domain event log ``` #### WebSocket Streaming | Endpoint | Data | Format | |----------|------|--------| | `ws://host:3000/ws/sensing` | Real-time sensing updates | JSON per tick | | `ws://host:3000/ws/pose` | COCO 17-keypoint skeleton | JSON | | `ws://host/ws/mat/stream` | Survivor/alert events | JSON with subscribe/unsubscribe | #### Data Formats - All API responses: **JSON** - WebSocket messages: **JSON** (no binary protocol) - ESP32 ingestion: **Binary UDP** (ADR-018 format, 20-byte header + I/Q payload) - Model packaging: **RVF binary** (64-byte aligned segments, CRC32) #### MQTT / Home Assistant **Not currently implemented.** No MQTT client, no Home Assistant integration exists anywhere in the codebase. This is a clear integration gap for smart home deployments and would be a valuable contribution. The REST API and WebSocket streaming provide the data — an MQTT bridge would be straightforward to build. --- ### 6. Known Limitations (Honest Assessment) #### Physics Constraints | Constraint | Value | Source | |-----------|-------|--------| | People per AP | ~3-5 (56 subcarriers) | Physics limit, not software | | Multi-AP scaling | Linear (~15-20 with 4 APs) | Triangulation math | | Max sensing range | ~10m open field | README | | Through-wall depth | ~5m, 30cm concrete | MAT default config | | Breathing displacement | 4-12mm (detectable) | `fresnel.rs:88-206` | | Heartbeat displacement | ~0.5mm (much harder) | `heartbeat.rs:133-136` | | Min breathing samples | 40 (~2s at 20 Hz) | `vital_signs.rs:28` | | Min heartbeat samples | 30 (~1.5s at 20 Hz) | `vital_signs.rs:29` | #### Sample Rate Limitations | Source | Rate | Suitable For | |--------|------|-------------| | ESP32 CSI | 20-100 Hz | Everything including heartbeat | | Windows WiFi | ~2 Hz | Presence, motion, marginal breathing | | macOS (proposed) | ~0.3-0.5 Hz | Presence and motion only | Windows WiFi at 2 Hz is explicitly noted as "insufficient for heartbeat" (`vital_signs.rs:92`). #### Accuracy Caveats 1. **Amplitude-based detection is fragile in dynamic environments** (issue #51). The rule-based classifier uses fixed thresholds and works best in enclosed, static rooms. The Rust CSI pipeline mitigates this with Hampel filtering, conjugate multiplication, and dynamic subcarrier partitioning — but has not been validated cross-scene. 2. **Heartbeat is much harder than breathing**: 0.5mm vs 10mm chest displacement, significantly lower SNR. Confidence threshold is stricter (0.4 vs 0.3). 3. **Pose estimation requires a trained model**. The current `derive_pose_from_sensing` function generates COCO keypoints from signal-field heuristics, not from trained weights. True pose estimation requires loading a trained RVF model via `--model`. **No pre-trained model is shipped.** Training requires the MM-Fi dataset and compute time. 4. **No cross-scene validation data exists yet.** The system has 229+ unit tests and deterministic proof bundles, but no published metrics comparing accuracy across different rooms, AP placements, or subjects. #### What Is Not Yet Built | Component | Status | ADR | |-----------|--------|-----| | ESP32 firmware (.bin) | Pending | ADR-018 | | macOS CoreWLAN adapter | Proposed | ADR-025 | | Pre-trained model weights | Not shipped | ADR-023 | | MQTT / Home Assistant bridge | Not started | -- | | Cross-scene benchmarks | Not done | -- | | WASM runtime for browser | Proposed | ADR-009 | --- ### Deliverables Checklist Per your requested deliverables: - [x] **Hardware compatibility matrix** — Section 1, covering ESP32, Atheros, Intel 5300, Windows WiFi, macOS - [ ] **Deployment guide for end-to-end scenario** — Docker quick start exists; a dedicated guide for ESP32-to-dashboard would be valuable - [ ] **Performance benchmarks vs ground truth** — Unit-level benchmarks exist (54K fps, breathing within 2 BPM tolerance); no ground-truth comparison against camera-based systems yet - [x] **Use case list with effort/value** — Section 3 feasibility matrix --- ### Recommended Next Steps For someone looking to deploy this: 1. **Quickest path to value**: `docker run -p 3000:3000 ruvnet/wifi-densepose:latest` — see the UI, hit the REST API, verify the signal processing chain works 2. **Commodity WiFi demo**: Run on Windows laptop — presence detection and motion classification work today with zero hardware cost 3. **Full CSI path**: Get 3x ESP32-S3 boards (~$24), flash firmware (once ADR-018 firmware is complete), and you have breathing/heartbeat/pose sensing 4. **Integration**: The REST API and WebSocket streaming are production-ready; an MQTT bridge for Home Assistant would be the highest-impact community contribution Happy to discuss any specific area in more depth. -- ruFlo AI
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something isn't working
 documentation
Improvements or additions to documentation
 duplicate
This issue or pull request already exists
 enhancement
New feature or request
 good first issue
Good for newcomers
 help wanted
Extra attention is needed
 invalid
This doesn't seem right
 question
Further information is requested
 security
Security-related issues and hardening
 wontfix
This will not be worked on
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dearsky
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dearsky/wifi-densepose#57
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?