feat: Add macOS CoreWLAN WiFi sensing adapter and user guide
- Introduced ADR-025 documenting the implementation of a macOS CoreWLAN sensing adapter using a Swift helper binary and Rust integration. - Added a new user guide detailing installation, usage, and hardware setup for WiFi DensePose, including Docker and source build instructions. - Included sections on data sources, REST API reference, WebSocket streaming, and vital sign detection. - Documented hardware requirements and troubleshooting steps for various setups.
This commit is contained in:
ruv
57
README.md
57
README.md
@@ -41,6 +41,17 @@ docker run -p 3000:3000 ruvnet/wifi-densepose:latest
|
||||
|
||||
---
|
||||
|
||||
## 📖 Documentation
|
||||
|
||||
| Document | Description |
|
||||
|----------|-------------|
|
||||
| [User Guide](docs/user-guide.md) | Step-by-step guide: installation, first run, API usage, hardware setup, training |
|
||||
| [WiFi-Mat User Guide](docs/wifi-mat-user-guide.md) | Disaster response module: search & rescue, START triage |
|
||||
| [Build Guide](docs/build-guide.md) | Building from source (Rust and Python) |
|
||||
| [Architecture Decisions](docs/adr/) | 24 ADRs covering signal processing, training, hardware, security |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Key Features
|
||||
|
||||
| | Feature | What It Means |
|
||||
@@ -317,6 +328,44 @@ docker run --rm -v $(pwd):/out ruvnet/wifi-densepose:latest --export-rvf /out/mo
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Rust Crates</strong> — Individual crates on crates.io</summary>
|
||||
|
||||
The Rust workspace consists of 14 crates, all published to [crates.io](https://crates.io/):
|
||||
|
||||
```bash
|
||||
# Add individual crates to your Cargo.toml
|
||||
cargo add wifi-densepose-core # Types, traits, errors
|
||||
cargo add wifi-densepose-signal # CSI signal processing (6 SOTA algorithms)
|
||||
cargo add wifi-densepose-nn # Neural inference (ONNX, PyTorch, Candle)
|
||||
cargo add wifi-densepose-vitals # Vital sign extraction (breathing + heart rate)
|
||||
cargo add wifi-densepose-mat # Disaster response (MAT survivor detection)
|
||||
cargo add wifi-densepose-hardware # ESP32, Intel 5300, Atheros sensors
|
||||
cargo add wifi-densepose-train # Training pipeline (MM-Fi dataset)
|
||||
cargo add wifi-densepose-wifiscan # Multi-BSSID WiFi scanning
|
||||
```
|
||||
|
||||
| Crate | Description | RuVector | crates.io |
|
||||
|-------|-------------|----------|-----------|
|
||||
| [`wifi-densepose-core`](https://crates.io/crates/wifi-densepose-core) | Foundation types, traits, and utilities | -- | [](https://crates.io/crates/wifi-densepose-core) |
|
||||
| [`wifi-densepose-signal`](https://crates.io/crates/wifi-densepose-signal) | SOTA CSI signal processing (SpotFi, FarSense, Widar 3.0) | `mincut`, `attn-mincut`, `attention`, `solver` | [](https://crates.io/crates/wifi-densepose-signal) |
|
||||
| [`wifi-densepose-nn`](https://crates.io/crates/wifi-densepose-nn) | Multi-backend inference (ONNX, PyTorch, Candle) | -- | [](https://crates.io/crates/wifi-densepose-nn) |
|
||||
| [`wifi-densepose-train`](https://crates.io/crates/wifi-densepose-train) | Training pipeline with MM-Fi dataset (NeurIPS 2023) | **All 5** | [](https://crates.io/crates/wifi-densepose-train) |
|
||||
| [`wifi-densepose-mat`](https://crates.io/crates/wifi-densepose-mat) | Mass Casualty Assessment Tool (disaster survivor detection) | `solver`, `temporal-tensor` | [](https://crates.io/crates/wifi-densepose-mat) |
|
||||
| [`wifi-densepose-vitals`](https://crates.io/crates/wifi-densepose-vitals) | Vital signs: breathing (6-30 BPM), heart rate (40-120 BPM) | -- | [](https://crates.io/crates/wifi-densepose-vitals) |
|
||||
| [`wifi-densepose-hardware`](https://crates.io/crates/wifi-densepose-hardware) | ESP32, Intel 5300, Atheros CSI sensor interfaces | -- | [](https://crates.io/crates/wifi-densepose-hardware) |
|
||||
| [`wifi-densepose-wifiscan`](https://crates.io/crates/wifi-densepose-wifiscan) | Multi-BSSID WiFi scanning (Windows-enhanced) | -- | [](https://crates.io/crates/wifi-densepose-wifiscan) |
|
||||
| [`wifi-densepose-wasm`](https://crates.io/crates/wifi-densepose-wasm) | WebAssembly bindings for browser deployment | -- | [](https://crates.io/crates/wifi-densepose-wasm) |
|
||||
| [`wifi-densepose-sensing-server`](https://crates.io/crates/wifi-densepose-sensing-server) | Axum server: UDP ingestion, WebSocket broadcast | -- | [](https://crates.io/crates/wifi-densepose-sensing-server) |
|
||||
| [`wifi-densepose-cli`](https://crates.io/crates/wifi-densepose-cli) | Command-line tool for MAT disaster scanning | -- | [](https://crates.io/crates/wifi-densepose-cli) |
|
||||
| [`wifi-densepose-api`](https://crates.io/crates/wifi-densepose-api) | REST + WebSocket API layer | -- | [](https://crates.io/crates/wifi-densepose-api) |
|
||||
| [`wifi-densepose-config`](https://crates.io/crates/wifi-densepose-config) | Configuration management | -- | [](https://crates.io/crates/wifi-densepose-config) |
|
||||
| [`wifi-densepose-db`](https://crates.io/crates/wifi-densepose-db) | Database persistence (PostgreSQL, SQLite, Redis) | -- | [](https://crates.io/crates/wifi-densepose-db) |
|
||||
|
||||
All crates integrate with [RuVector v2.0.4](https://github.com/ruvnet/ruvector) for graph algorithms and neural network optimization.
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
@@ -548,8 +597,8 @@ cargo bench --package wifi-densepose-signal
|
||||
| **Confidence** | 0.0-1.0 per sign | Spectral coherence + signal quality |
|
||||
|
||||
```bash
|
||||
./target/release/sensing-server --source simulate --ui-path ../../ui
|
||||
curl http://localhost:8080/api/v1/vital-signs
|
||||
./target/release/sensing-server --source simulate --http-port 3000 --ws-port 3001 --ui-path ../../ui
|
||||
curl http://localhost:3000/api/v1/vital-signs
|
||||
```
|
||||
|
||||
See [ADR-021](docs/adr/ADR-021-vital-sign-detection-rvdna-pipeline.md).
|
||||
@@ -1026,9 +1075,9 @@ GET /api/v1/model/sona/profiles # SONA profiles
|
||||
POST /api/v1/model/sona/activate # Activate SONA profile
|
||||
```
|
||||
|
||||
WebSocket: `ws://localhost:8765/ws/sensing` (real-time sensing + vital signs)
|
||||
WebSocket: `ws://localhost:3001/ws/sensing` (real-time sensing + vital signs)
|
||||
|
||||
> Default ports: HTTP 8080, WS 8765. Docker images remap to 3000/3001 via `--http-port` / `--ws-port`.
|
||||
> Default ports (Docker): HTTP 3000, WS 3001. Binary defaults: HTTP 8080, WS 8765. Override with `--http-port` / `--ws-port`.
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
315
docs/adr/ADR-025-macos-corewlan-wifi-sensing.md
Normal file
315
docs/adr/ADR-025-macos-corewlan-wifi-sensing.md
Normal file
@@ -0,0 +1,315 @@
|
||||
# ADR-025: macOS CoreWLAN WiFi Sensing via Swift Helper Bridge
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Status** | Proposed |
|
||||
| **Date** | 2026-03-01 |
|
||||
| **Deciders** | ruv |
|
||||
| **Codename** | **ORCA** — OS-native Radio Channel Acquisition |
|
||||
| **Relates to** | ADR-013 (Feature-Level Sensing Commodity Gear), ADR-022 (Windows WiFi Enhanced Fidelity), ADR-014 (SOTA Signal Processing), ADR-018 (ESP32 Dev Implementation) |
|
||||
| **Issue** | [#56](https://github.com/ruvnet/wifi-densepose/issues/56) |
|
||||
| **Build/Test Target** | Mac Mini (M2 Pro, macOS 26.3) |
|
||||
|
||||
---
|
||||
|
||||
## 1. Context
|
||||
|
||||
### 1.1 The Gap: macOS Is a Silent Fallback
|
||||
|
||||
The `--source auto` path in `sensing-server` probes for ESP32 UDP, then Windows `netsh`, then falls back to simulated mode. macOS users hit the simulation path silently — there is no macOS WiFi adapter. This is the only major desktop platform without real WiFi sensing support.
|
||||
|
||||
### 1.2 Platform Constraints (macOS 26.3+)
|
||||
|
||||
| Constraint | Detail |
|
||||
|------------|--------|
|
||||
| **`airport` CLI removed** | Apple removed `/System/Library/PrivateFrameworks/.../airport` in macOS 15. No CLI fallback exists. |
|
||||
| **CoreWLAN is the only path** | `CWWiFiClient` (Swift/ObjC) is the supported API for WiFi scanning. Returns RSSI, channel, SSID, noise, PHY mode, security. |
|
||||
| **BSSIDs redacted** | macOS privacy policy redacts MAC addresses from `CWNetwork.bssid` unless the app has Location Services + WiFi entitlement. Apps without entitlement see `nil` for BSSID. |
|
||||
| **No raw CSI** | Apple does not expose CSI or per-subcarrier data. macOS WiFi sensing is RSSI-only, same tier as Windows `netsh`. |
|
||||
| **Scan rate** | `CWInterface.scanForNetworks()` takes ~2-4 seconds. Effective rate: ~0.3-0.5 Hz without caching. |
|
||||
| **Permissions** | Location Services prompt required for BSSID access. Without it, SSID + RSSI + channel still available. |
|
||||
|
||||
### 1.3 The Opportunity: Multi-AP RSSI Diversity
|
||||
|
||||
Same principle as ADR-022 (Windows): visible APs serve as pseudo-subcarriers. A typical indoor environment exposes 10-30+ SSIDs across 2.4 GHz and 5 GHz bands. Each AP's RSSI responds differently to human movement based on geometry, creating spatial diversity.
|
||||
|
||||
| Source | Effective Subcarriers | Sample Rate | Capabilities |
|
||||
|--------|----------------------|-------------|-------------|
|
||||
| ESP32-S3 (CSI) | 56-192 | 20 Hz | Full: pose, vitals, through-wall |
|
||||
| Windows `netsh` (ADR-022) | 10-30 BSSIDs | ~2 Hz | Presence, motion, coarse breathing |
|
||||
| **macOS CoreWLAN (this ADR)** | **10-30 SSIDs** | **~0.3-0.5 Hz** | **Presence, motion** |
|
||||
|
||||
The lower scan rate vs Windows is offset by higher signal quality — CoreWLAN returns calibrated dBm (not percentage) plus noise floor, enabling proper SNR computation.
|
||||
|
||||
### 1.4 Why Swift Subprocess (Not FFI)
|
||||
|
||||
| Approach | Complexity | Maintenance | Build | Verdict |
|
||||
|----------|-----------|-------------|-------|---------|
|
||||
| **Swift CLI → JSON → stdout** | Low | Independent binary, versionable | `swiftc` (ships with Xcode CLT) | **Chosen** |
|
||||
| ObjC FFI via `cc` crate | Medium | Fragile header bindings, ABI churn | Requires Xcode headers | Rejected |
|
||||
| `objc2` crate (Rust ObjC bridge) | High | CoreWLAN not in upstream `objc2-frameworks` | Requires manual class definitions | Rejected |
|
||||
| `swift-bridge` crate | High | Young ecosystem, async bridging unsupported | Requires Swift build integration in Cargo | Rejected |
|
||||
|
||||
The `Command::new()` + parse JSON pattern is proven — it's exactly what `NetshBssidScanner` does for Windows. The subprocess boundary also isolates Apple framework dependencies from the Rust build graph.
|
||||
|
||||
### 1.5 SOTA: Platform-Adaptive WiFi Sensing
|
||||
|
||||
Recent work validates multi-platform RSSI-based sensing:
|
||||
|
||||
- **WiFind** (2024): Cross-platform WiFi fingerprinting using RSSI vectors from heterogeneous hardware. Demonstrates that normalization across scan APIs (dBm, percentage, raw) is critical for model portability.
|
||||
- **WiGesture** (2025): RSSI variance-based gesture recognition achieving 89% accuracy on commodity hardware with 15+ APs. Shows that temporal RSSI variance alone carries significant motion information.
|
||||
- **CrossSense** (2024): Transfer learning from CSI-rich hardware to RSSI-only devices. Pre-trained signal features transfer with 78% effectiveness, validating multi-tier hardware strategy.
|
||||
|
||||
---
|
||||
|
||||
## 2. Decision
|
||||
|
||||
Implement a **macOS CoreWLAN sensing adapter** as a Swift helper binary + Rust adapter pair, following the established `NetshBssidScanner` subprocess pattern from ADR-022. Real RSSI data flows through the existing 8-stage `WindowsWifiPipeline` (which operates on `BssidObservation` structs regardless of platform origin).
|
||||
|
||||
### 2.1 Design Principles
|
||||
|
||||
1. **Subprocess isolation** — Swift binary is a standalone tool, built and versioned independently of the Rust workspace.
|
||||
2. **Same domain types** — macOS adapter produces `Vec<BssidObservation>`, identical to the Windows path. All downstream processing reuses as-is.
|
||||
3. **SSID:channel as synthetic BSSID** — When real BSSIDs are redacted (no Location Services), `sha256(ssid + channel)[:12]` generates a stable pseudo-BSSID. Documented limitation: same-SSID same-channel APs collapse to one observation.
|
||||
4. **`#[cfg(target_os = "macos")]` gating** — macOS-specific code compiles only on macOS. Windows and Linux builds are unaffected.
|
||||
5. **Graceful degradation** — If the Swift helper is not found or fails, `--source auto` skips macOS WiFi and falls back to simulated mode with a clear warning.
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture
|
||||
|
||||
### 3.1 Component Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ macOS WiFi Sensing Path │
|
||||
│ │
|
||||
│ ┌──────────────────────┐ ┌───────────────────────────────────┐│
|
||||
│ │ Swift Helper Binary │ │ Rust Adapter + Existing Pipeline ││
|
||||
│ │ (tools/macos-wifi- │ │ ││
|
||||
│ │ scan/main.swift) │ │ MacosCoreWlanScanner ││
|
||||
│ │ │ │ │ ││
|
||||
│ │ CWWiFiClient │JSON │ ▼ ││
|
||||
│ │ scanForNetworks() ──┼────►│ Vec<BssidObservation> ││
|
||||
│ │ interface() │ │ │ ││
|
||||
│ │ │ │ ▼ ││
|
||||
│ │ Outputs: │ │ BssidRegistry ││
|
||||
│ │ - ssid │ │ │ ││
|
||||
│ │ - rssi (dBm) │ │ ▼ ││
|
||||
│ │ - noise (dBm) │ │ WindowsWifiPipeline (reused) ││
|
||||
│ │ - channel │ │ [8-stage signal intelligence] ││
|
||||
│ │ - band (2.4/5/6) │ │ │ ││
|
||||
│ │ - phy_mode │ │ ▼ ││
|
||||
│ │ - bssid (if avail) │ │ SensingUpdate → REST/WS ││
|
||||
│ └──────────────────────┘ └───────────────────────────────────┘│
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 3.2 Swift Helper Binary
|
||||
|
||||
**File:** `rust-port/wifi-densepose-rs/tools/macos-wifi-scan/main.swift`
|
||||
|
||||
```swift
|
||||
// Modes:
|
||||
// (no args) → Full scan, output JSON array to stdout
|
||||
// --probe → Quick availability check, output {"available": true/false}
|
||||
// --connected → Connected network info only
|
||||
//
|
||||
// Output schema (scan mode):
|
||||
// [
|
||||
// {
|
||||
// "ssid": "MyNetwork",
|
||||
// "rssi": -52,
|
||||
// "noise": -90,
|
||||
// "channel": 36,
|
||||
// "band": "5GHz",
|
||||
// "phy_mode": "802.11ax",
|
||||
// "bssid": "aa:bb:cc:dd:ee:ff" | null,
|
||||
// "security": "wpa2_personal"
|
||||
// }
|
||||
// ]
|
||||
```
|
||||
|
||||
**Build:**
|
||||
|
||||
```bash
|
||||
# Requires Xcode Command Line Tools (xcode-select --install)
|
||||
cd tools/macos-wifi-scan
|
||||
swiftc -framework CoreWLAN -framework Foundation -O -o macos-wifi-scan main.swift
|
||||
```
|
||||
|
||||
**Build script:** `tools/macos-wifi-scan/build.sh`
|
||||
|
||||
### 3.3 Rust Adapter
|
||||
|
||||
**File:** `crates/wifi-densepose-wifiscan/src/adapter/macos_scanner.rs`
|
||||
|
||||
```rust
|
||||
// #[cfg(target_os = "macos")]
|
||||
|
||||
pub struct MacosCoreWlanScanner {
|
||||
helper_path: PathBuf, // Resolved at construction: $PATH or sibling of server binary
|
||||
}
|
||||
|
||||
impl MacosCoreWlanScanner {
|
||||
pub fn new() -> Result<Self, WifiScanError> // Finds helper or errors
|
||||
pub fn probe() -> bool // Runs --probe, returns availability
|
||||
pub fn scan_sync(&self) -> Result<Vec<BssidObservation>, WifiScanError>
|
||||
pub fn connected_sync(&self) -> Result<Option<BssidObservation>, WifiScanError>
|
||||
}
|
||||
```
|
||||
|
||||
**Key mappings:**
|
||||
|
||||
| CoreWLAN field | → | BssidObservation field | Transform |
|
||||
|----------------|---|----------------------|-----------|
|
||||
| `rssi` (dBm) | → | `signal_dbm` | Direct (CoreWLAN gives calibrated dBm) |
|
||||
| `rssi` (dBm) | → | `amplitude` | `rssi_to_amplitude()` (existing) |
|
||||
| `noise` (dBm) | → | `snr` | `rssi - noise` (new field, macOS advantage) |
|
||||
| `channel` | → | `channel` | Direct |
|
||||
| `band` | → | `band` | `BandType::from_channel()` (existing) |
|
||||
| `phy_mode` | → | `radio_type` | Map string → `RadioType` enum |
|
||||
| `bssid` | → | `bssid_id` | Direct if available, else `sha256(ssid:channel)[:12]` |
|
||||
| `ssid` | → | `ssid` | Direct |
|
||||
|
||||
### 3.4 Sensing Server Integration
|
||||
|
||||
**File:** `crates/wifi-densepose-sensing-server/src/main.rs`
|
||||
|
||||
| Function | Purpose |
|
||||
|----------|---------|
|
||||
| `probe_macos_wifi()` | Calls `MacosCoreWlanScanner::probe()`, returns bool |
|
||||
| `macos_wifi_task()` | Async loop: scan → build `BssidObservation` vec → feed into `BssidRegistry` + `WindowsWifiPipeline` → emit `SensingUpdate`. Same structure as `windows_wifi_task()`. |
|
||||
|
||||
**Auto-detection order (updated):**
|
||||
|
||||
```
|
||||
1. ESP32 UDP probe (port 5005) → --source esp32
|
||||
2. Windows netsh probe → --source wifi (Windows)
|
||||
3. macOS CoreWLAN probe [NEW] → --source wifi (macOS)
|
||||
4. Simulated fallback → --source simulated
|
||||
```
|
||||
|
||||
### 3.5 Pipeline Reuse
|
||||
|
||||
The existing 8-stage `WindowsWifiPipeline` (ADR-022) operates entirely on `BssidObservation` / `MultiApFrame` types:
|
||||
|
||||
| Stage | Reusable? | Notes |
|
||||
|-------|-----------|-------|
|
||||
| 1. Predictive Gating | Yes | Filters static APs by temporal variance |
|
||||
| 2. Attention Weighting | Yes | Weights APs by motion sensitivity |
|
||||
| 3. Spatial Correlation | Yes | Cross-AP signal correlation |
|
||||
| 4. Motion Estimation | Yes | RSSI variance → motion level |
|
||||
| 5. Breathing Extraction | **Marginal** | 0.3 Hz scan rate is below Nyquist for breathing (0.1-0.5 Hz). May detect very slow breathing only. |
|
||||
| 6. Quality Gating | Yes | Rejects low-confidence estimates |
|
||||
| 7. Fingerprint Matching | Yes | Location/posture classification |
|
||||
| 8. Orchestration | Yes | Fuses all stages |
|
||||
|
||||
**Limitation:** CoreWLAN scan rate (~0.3-0.5 Hz) is significantly slower than `netsh` (~2 Hz). Breathing extraction (stage 5) will have reduced accuracy. Motion and presence detection remain effective since they depend on variance over longer windows.
|
||||
|
||||
---
|
||||
|
||||
## 4. Files
|
||||
|
||||
### 4.1 New Files
|
||||
|
||||
| File | Purpose | Lines (est.) |
|
||||
|------|---------|-------------|
|
||||
| `tools/macos-wifi-scan/main.swift` | CoreWLAN scanner, JSON output | ~120 |
|
||||
| `tools/macos-wifi-scan/build.sh` | Build script (`swiftc` invocation) | ~15 |
|
||||
| `crates/wifi-densepose-wifiscan/src/adapter/macos_scanner.rs` | Rust adapter: spawn helper, parse JSON, produce `BssidObservation` | ~200 |
|
||||
|
||||
### 4.2 Modified Files
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `crates/wifi-densepose-wifiscan/src/adapter/mod.rs` | Add `#[cfg(target_os = "macos")] pub mod macos_scanner;` + re-export |
|
||||
| `crates/wifi-densepose-wifiscan/src/lib.rs` | Add `MacosCoreWlanScanner` re-export |
|
||||
| `crates/wifi-densepose-sensing-server/src/main.rs` | Add `probe_macos_wifi()`, `macos_wifi_task()`, update auto-detect + `--source wifi` dispatch |
|
||||
|
||||
### 4.3 No New Rust Dependencies
|
||||
|
||||
- `std::process::Command` — subprocess spawning (stdlib)
|
||||
- `serde_json` — JSON parsing (already in workspace)
|
||||
- No changes to `Cargo.toml`
|
||||
|
||||
---
|
||||
|
||||
## 5. Verification Plan
|
||||
|
||||
All verification on Mac Mini (M2 Pro, macOS 26.3).
|
||||
|
||||
### 5.1 Swift Helper
|
||||
|
||||
| Test | Command | Expected |
|
||||
|------|---------|----------|
|
||||
| Build | `cd tools/macos-wifi-scan && ./build.sh` | Produces `macos-wifi-scan` binary |
|
||||
| Probe | `./macos-wifi-scan --probe` | `{"available": true}` |
|
||||
| Scan | `./macos-wifi-scan` | JSON array with real SSIDs, RSSI in dBm, channels |
|
||||
| Connected | `./macos-wifi-scan --connected` | Single JSON object for connected network |
|
||||
| No WiFi | Disable WiFi → `./macos-wifi-scan` | `{"available": false}` or empty array |
|
||||
|
||||
### 5.2 Rust Adapter
|
||||
|
||||
| Test | Method | Expected |
|
||||
|------|--------|----------|
|
||||
| Unit: JSON parsing | `#[test]` with fixture JSON | Correct `BssidObservation` values |
|
||||
| Unit: synthetic BSSID | `#[test]` with nil bssid input | Stable `sha256(ssid:channel)[:12]` |
|
||||
| Unit: helper not found | `#[test]` with bad path | `WifiScanError::ProcessError` |
|
||||
| Integration: real scan | `cargo test` on Mac Mini | Live observations from CoreWLAN |
|
||||
|
||||
### 5.3 End-to-End
|
||||
|
||||
| Step | Command | Verify |
|
||||
|------|---------|--------|
|
||||
| 1 | `cargo build --release` (Mac Mini) | Clean build, no warnings |
|
||||
| 2 | `cargo test --workspace` | All existing tests pass + new macOS tests |
|
||||
| 3 | `./target/release/sensing-server --source wifi` | Server starts, logs `source: wifi (macOS CoreWLAN)` |
|
||||
| 4 | `curl http://localhost:8080/api/v1/sensing/latest` | `source: "wifi:<SSID>"`, real RSSI values |
|
||||
| 5 | `curl http://localhost:8080/api/v1/vital-signs` | Motion detection responds to physical movement |
|
||||
| 6 | Open UI at `http://localhost:8080` | Signal field updates with real RSSI variation |
|
||||
| 7 | `--source auto` | Auto-detects macOS WiFi, does not fall back to simulated |
|
||||
|
||||
### 5.4 Cross-Platform Regression
|
||||
|
||||
| Platform | Build | Expected |
|
||||
|----------|-------|----------|
|
||||
| macOS (Mac Mini) | `cargo build --release` | macOS adapter compiled, works |
|
||||
| Windows | `cargo build --release` | macOS adapter skipped (`#[cfg]`), Windows path unchanged |
|
||||
| Linux | `cargo build --release` | macOS adapter skipped, ESP32/simulated paths unchanged |
|
||||
|
||||
---
|
||||
|
||||
## 6. Limitations
|
||||
|
||||
| Limitation | Impact | Mitigation |
|
||||
|------------|--------|-----------|
|
||||
| **BSSID redaction** | Same-SSID same-channel APs collapse to one observation | Use `sha256(ssid:channel)` as pseudo-BSSID; document edge case. Rare in practice (mesh networks). |
|
||||
| **Slow scan rate** (~0.3 Hz) | Breathing extraction unreliable (below Nyquist) | Motion/presence still work. Breathing marked low-confidence. Future: cache + connected AP fast-poll hybrid. |
|
||||
| **Requires Swift helper in PATH** | Extra build step for source builds | `build.sh` provided. Docker image pre-bundles it. Clear error message when missing. |
|
||||
| **Location Services for BSSID** | Full BSSID requires user permission prompt | System degrades gracefully to SSID:channel pseudo-BSSID without permission. |
|
||||
| **No CSI** | Cannot match ESP32 pose estimation accuracy | Expected — this is RSSI-tier sensing (presence + motion). Same limitation as Windows. |
|
||||
|
||||
---
|
||||
|
||||
## 7. Future Work
|
||||
|
||||
| Enhancement | Description | Depends On |
|
||||
|-------------|-------------|-----------|
|
||||
| **Fast-poll connected AP** | Poll connected AP's RSSI at ~10 Hz via `CWInterface.rssiValue()` (no full scan needed) | CoreWLAN `rssiValue()` performance testing |
|
||||
| **Linux `iw` adapter** | Same subprocess pattern with `iw dev wlan0 scan` output | Linux machine for testing |
|
||||
| **Unified `RssiPipeline` rename** | Rename `WindowsWifiPipeline` → `RssiPipeline` to reflect multi-platform use | ADR-022 update |
|
||||
| **802.11bf sensing** | Apple may expose CSI via 802.11bf in future macOS | Apple framework availability |
|
||||
| **Docker macOS image** | Pre-built macOS Docker image with Swift helper bundled | Docker multi-arch build |
|
||||
|
||||
---
|
||||
|
||||
## 8. References
|
||||
|
||||
- [Apple CoreWLAN Documentation](https://developer.apple.com/documentation/corewlan)
|
||||
- [CWWiFiClient](https://developer.apple.com/documentation/corewlan/cwwificlient) — Primary WiFi interface API
|
||||
- [CWNetwork](https://developer.apple.com/documentation/corewlan/cwnetwork) — Scan result type (SSID, RSSI, channel, noise)
|
||||
- [macOS 15 airport removal](https://developer.apple.com/forums/thread/732431) — Apple Developer Forums
|
||||
- ADR-022: Windows WiFi Enhanced Fidelity (analogous platform adapter)
|
||||
- ADR-013: Feature-Level Sensing from Commodity Gear
|
||||
- Issue [#56](https://github.com/ruvnet/wifi-densepose/issues/56): macOS support request
|
||||
632
docs/user-guide.md
Normal file
632
docs/user-guide.md
Normal file
@@ -0,0 +1,632 @@
|
||||
# WiFi DensePose User Guide
|
||||
|
||||
WiFi DensePose turns commodity WiFi signals into real-time human pose estimation, vital sign monitoring, and presence detection. This guide walks you through installation, first run, API usage, hardware setup, and model training.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Prerequisites](#prerequisites)
|
||||
2. [Installation](#installation)
|
||||
- [Docker (Recommended)](#docker-recommended)
|
||||
- [From Source (Rust)](#from-source-rust)
|
||||
- [From Source (Python)](#from-source-python)
|
||||
- [Guided Installer](#guided-installer)
|
||||
3. [Quick Start](#quick-start)
|
||||
- [30-Second Demo (Docker)](#30-second-demo-docker)
|
||||
- [Verify the System Works](#verify-the-system-works)
|
||||
4. [Data Sources](#data-sources)
|
||||
- [Simulated Mode (No Hardware)](#simulated-mode-no-hardware)
|
||||
- [Windows WiFi (RSSI Only)](#windows-wifi-rssi-only)
|
||||
- [ESP32-S3 (Full CSI)](#esp32-s3-full-csi)
|
||||
5. [REST API Reference](#rest-api-reference)
|
||||
6. [WebSocket Streaming](#websocket-streaming)
|
||||
7. [Web UI](#web-ui)
|
||||
8. [Vital Sign Detection](#vital-sign-detection)
|
||||
9. [CLI Reference](#cli-reference)
|
||||
10. [Training a Model](#training-a-model)
|
||||
11. [RVF Model Containers](#rvf-model-containers)
|
||||
12. [Hardware Setup](#hardware-setup)
|
||||
- [ESP32-S3 Mesh](#esp32-s3-mesh)
|
||||
- [Intel 5300 / Atheros NIC](#intel-5300--atheros-nic)
|
||||
13. [Docker Compose (Multi-Service)](#docker-compose-multi-service)
|
||||
14. [Troubleshooting](#troubleshooting)
|
||||
15. [FAQ](#faq)
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Requirement | Minimum | Recommended |
|
||||
|-------------|---------|-------------|
|
||||
| **OS** | Windows 10, macOS 10.15, Ubuntu 18.04 | Latest stable |
|
||||
| **RAM** | 4 GB | 8 GB+ |
|
||||
| **Disk** | 2 GB free | 5 GB free |
|
||||
| **Docker** (for Docker path) | Docker 20+ | Docker 24+ |
|
||||
| **Rust** (for source build) | 1.70+ | 1.85+ |
|
||||
| **Python** (for legacy v1) | 3.8+ | 3.11+ |
|
||||
|
||||
**Hardware for live sensing (optional):**
|
||||
|
||||
| Option | Cost | Capabilities |
|
||||
|--------|------|-------------|
|
||||
| ESP32-S3 mesh (3-6 boards) | ~$54 | Full CSI: pose, breathing, heartbeat, presence |
|
||||
| Intel 5300 / Atheros AR9580 | $50-100 | Full CSI with 3x3 MIMO (Linux only) |
|
||||
| Any WiFi laptop | $0 | RSSI-only: coarse presence and motion detection |
|
||||
|
||||
No hardware? The system runs in **simulated mode** with synthetic CSI data.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### Docker (Recommended)
|
||||
|
||||
The fastest path. No toolchain installation needed.
|
||||
|
||||
```bash
|
||||
docker pull ruvnet/wifi-densepose:latest
|
||||
```
|
||||
|
||||
Image size: ~132 MB. Contains the Rust sensing server, Three.js UI, and all signal processing.
|
||||
|
||||
### From Source (Rust)
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ruvnet/wifi-densepose.git
|
||||
cd wifi-densepose/rust-port/wifi-densepose-rs
|
||||
|
||||
# Build
|
||||
cargo build --release
|
||||
|
||||
# Verify (runs 542+ tests)
|
||||
cargo test --workspace
|
||||
```
|
||||
|
||||
The compiled binary is at `target/release/sensing-server`.
|
||||
|
||||
### From Source (Python)
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ruvnet/wifi-densepose.git
|
||||
cd wifi-densepose
|
||||
|
||||
pip install -r requirements.txt
|
||||
pip install -e .
|
||||
|
||||
# Or via PyPI
|
||||
pip install wifi-densepose
|
||||
pip install wifi-densepose[gpu] # GPU acceleration
|
||||
pip install wifi-densepose[all] # All optional deps
|
||||
```
|
||||
|
||||
### Guided Installer
|
||||
|
||||
An interactive installer that detects your hardware and recommends a profile:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ruvnet/wifi-densepose.git
|
||||
cd wifi-densepose
|
||||
./install.sh
|
||||
```
|
||||
|
||||
Available profiles: `verify`, `python`, `rust`, `browser`, `iot`, `docker`, `field`, `full`.
|
||||
|
||||
Non-interactive:
|
||||
```bash
|
||||
./install.sh --profile rust --yes
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 30-Second Demo (Docker)
|
||||
|
||||
```bash
|
||||
# Pull and run
|
||||
docker run -p 3000:3000 -p 3001:3001 ruvnet/wifi-densepose:latest
|
||||
|
||||
# Open the UI in your browser
|
||||
# http://localhost:3000
|
||||
```
|
||||
|
||||
You will see a Three.js visualization with:
|
||||
- 3D body skeleton (17 COCO keypoints)
|
||||
- Signal amplitude heatmap
|
||||
- Phase plot
|
||||
- Vital signs panel (breathing + heartbeat)
|
||||
|
||||
### Verify the System Works
|
||||
|
||||
Open a second terminal and test the API:
|
||||
|
||||
```bash
|
||||
# Health check
|
||||
curl http://localhost:3000/health
|
||||
# Expected: {"status":"ok","source":"simulated","clients":0}
|
||||
|
||||
# Latest sensing frame
|
||||
curl http://localhost:3000/api/v1/sensing/latest
|
||||
|
||||
# Vital signs
|
||||
curl http://localhost:3000/api/v1/vital-signs
|
||||
|
||||
# Pose estimation (17 COCO keypoints)
|
||||
curl http://localhost:3000/api/v1/pose/current
|
||||
|
||||
# Server build info
|
||||
curl http://localhost:3000/api/v1/info
|
||||
```
|
||||
|
||||
All endpoints return JSON. In simulated mode, data is generated from a deterministic reference signal.
|
||||
|
||||
---
|
||||
|
||||
## Data Sources
|
||||
|
||||
The `--source` flag controls where CSI data comes from.
|
||||
|
||||
### Simulated Mode (No Hardware)
|
||||
|
||||
Default in Docker. Generates synthetic CSI data exercising the full pipeline.
|
||||
|
||||
```bash
|
||||
# Docker
|
||||
docker run -p 3000:3000 ruvnet/wifi-densepose:latest
|
||||
# (--source simulated is the default)
|
||||
|
||||
# From source
|
||||
./target/release/sensing-server --source simulated --http-port 3000 --ws-port 3001
|
||||
```
|
||||
|
||||
### Windows WiFi (RSSI Only)
|
||||
|
||||
Uses `netsh wlan` to capture RSSI from nearby access points. No special hardware needed, but capabilities are limited to coarse presence and motion detection (no pose estimation or vital signs).
|
||||
|
||||
```bash
|
||||
# From source (Windows only)
|
||||
./target/release/sensing-server --source windows --http-port 3000 --ws-port 3001 --tick-ms 500
|
||||
|
||||
# Docker (requires --network host on Windows)
|
||||
docker run --network host ruvnet/wifi-densepose:latest --source windows --tick-ms 500
|
||||
```
|
||||
|
||||
See [Tutorial #36](https://github.com/ruvnet/wifi-densepose/issues/36) for a walkthrough.
|
||||
|
||||
### ESP32-S3 (Full CSI)
|
||||
|
||||
Real Channel State Information at 20 Hz with 56-192 subcarriers. Required for pose estimation, vital signs, and through-wall sensing.
|
||||
|
||||
```bash
|
||||
# From source
|
||||
./target/release/sensing-server --source esp32 --udp-port 5005 --http-port 3000 --ws-port 3001
|
||||
|
||||
# Docker
|
||||
docker run -p 3000:3000 -p 3001:3001 -p 5005:5005/udp ruvnet/wifi-densepose:latest --source esp32
|
||||
```
|
||||
|
||||
The ESP32 nodes stream binary CSI frames over UDP to port 5005. See [Hardware Setup](#esp32-s3-mesh) for flashing instructions.
|
||||
|
||||
---
|
||||
|
||||
## REST API Reference
|
||||
|
||||
Base URL: `http://localhost:3000` (Docker) or `http://localhost:8080` (binary default).
|
||||
|
||||
| Method | Endpoint | Description | Example Response |
|
||||
|--------|----------|-------------|-----------------|
|
||||
| `GET` | `/health` | Server health check | `{"status":"ok","source":"simulated","clients":0}` |
|
||||
| `GET` | `/api/v1/sensing/latest` | Latest CSI sensing frame (amplitude, phase, motion) | JSON with subcarrier arrays |
|
||||
| `GET` | `/api/v1/vital-signs` | Breathing rate + heart rate + confidence | `{"breathing_bpm":16.2,"heart_bpm":72.1,"confidence":0.87}` |
|
||||
| `GET` | `/api/v1/pose/current` | 17 COCO keypoints (x, y, z, confidence) | Array of 17 joint positions |
|
||||
| `GET` | `/api/v1/info` | Server version, build info, uptime | JSON metadata |
|
||||
| `GET` | `/api/v1/bssid` | Multi-BSSID WiFi registry | List of detected access points |
|
||||
| `GET` | `/api/v1/model/layers` | Progressive model loading status | Layer A/B/C load state |
|
||||
| `GET` | `/api/v1/model/sona/profiles` | SONA adaptation profiles | List of environment profiles |
|
||||
| `POST` | `/api/v1/model/sona/activate` | Activate a SONA profile for a specific room | `{"profile":"kitchen"}` |
|
||||
|
||||
### Example: Get Vital Signs
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:3000/api/v1/vital-signs | python -m json.tool
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"breathing_bpm": 16.2,
|
||||
"heart_bpm": 72.1,
|
||||
"breathing_confidence": 0.87,
|
||||
"heart_confidence": 0.63,
|
||||
"motion_level": 0.12,
|
||||
"timestamp_ms": 1709312400000
|
||||
}
|
||||
```
|
||||
|
||||
### Example: Get Pose
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:3000/api/v1/pose/current | python -m json.tool
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"persons": [
|
||||
{
|
||||
"id": 0,
|
||||
"keypoints": [
|
||||
{"name": "nose", "x": 0.52, "y": 0.31, "z": 0.0, "confidence": 0.91},
|
||||
{"name": "left_eye", "x": 0.54, "y": 0.29, "z": 0.0, "confidence": 0.88}
|
||||
]
|
||||
}
|
||||
],
|
||||
"frame_id": 1024,
|
||||
"timestamp_ms": 1709312400000
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## WebSocket Streaming
|
||||
|
||||
Real-time sensing data is available via WebSocket.
|
||||
|
||||
**URL:** `ws://localhost:3001/ws/sensing` (Docker) or `ws://localhost:8765/ws/sensing` (binary default).
|
||||
|
||||
### Python Example
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
import websockets
|
||||
import json
|
||||
|
||||
async def stream():
|
||||
uri = "ws://localhost:3001/ws/sensing"
|
||||
async with websockets.connect(uri) as ws:
|
||||
async for message in ws:
|
||||
data = json.loads(message)
|
||||
persons = data.get("persons", [])
|
||||
vitals = data.get("vital_signs", {})
|
||||
print(f"Persons: {len(persons)}, "
|
||||
f"Breathing: {vitals.get('breathing_bpm', 'N/A')} BPM")
|
||||
|
||||
asyncio.run(stream())
|
||||
```
|
||||
|
||||
### JavaScript Example
|
||||
|
||||
```javascript
|
||||
const ws = new WebSocket("ws://localhost:3001/ws/sensing");
|
||||
|
||||
ws.onmessage = (event) => {
|
||||
const data = JSON.parse(event.data);
|
||||
console.log("Persons:", data.persons?.length ?? 0);
|
||||
console.log("Breathing:", data.vital_signs?.breathing_bpm, "BPM");
|
||||
};
|
||||
|
||||
ws.onerror = (err) => console.error("WebSocket error:", err);
|
||||
```
|
||||
|
||||
### curl (single frame)
|
||||
|
||||
```bash
|
||||
# Requires wscat (npm install -g wscat)
|
||||
wscat -c ws://localhost:3001/ws/sensing
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Web UI
|
||||
|
||||
The built-in Three.js UI is served at `http://localhost:3000/` (Docker) or the configured HTTP port.
|
||||
|
||||
**What you see:**
|
||||
|
||||
| Panel | Description |
|
||||
|-------|-------------|
|
||||
| 3D Body View | Rotatable wireframe skeleton with 17 COCO keypoints |
|
||||
| Signal Heatmap | 56 subcarriers color-coded by amplitude |
|
||||
| Phase Plot | Per-subcarrier phase values over time |
|
||||
| Doppler Bars | Motion band power indicators |
|
||||
| Vital Signs | Live breathing rate (BPM) and heart rate (BPM) |
|
||||
| Dashboard | System stats, throughput, connected WebSocket clients |
|
||||
|
||||
The UI updates in real-time via the WebSocket connection.
|
||||
|
||||
---
|
||||
|
||||
## Vital Sign Detection
|
||||
|
||||
The system extracts breathing rate and heart rate from CSI signal fluctuations using FFT peak detection.
|
||||
|
||||
| Sign | Frequency Band | Range | Method |
|
||||
|------|---------------|-------|--------|
|
||||
| Breathing | 0.1-0.5 Hz | 6-30 BPM | Bandpass filter + FFT peak |
|
||||
| Heart rate | 0.8-2.0 Hz | 40-120 BPM | Bandpass filter + FFT peak |
|
||||
|
||||
**Requirements:**
|
||||
- CSI-capable hardware (ESP32-S3 or research NIC) for accurate readings
|
||||
- Subject within ~3-5 meters of an access point
|
||||
- Relatively stationary subject (large movements mask vital sign oscillations)
|
||||
|
||||
**Simulated mode** produces synthetic vital sign data for testing.
|
||||
|
||||
---
|
||||
|
||||
## CLI Reference
|
||||
|
||||
The Rust sensing server binary accepts the following flags:
|
||||
|
||||
| Flag | Default | Description |
|
||||
|------|---------|-------------|
|
||||
| `--source` | `auto` | Data source: `auto`, `simulated`, `windows`, `esp32` |
|
||||
| `--http-port` | `8080` | HTTP port for REST API and UI |
|
||||
| `--ws-port` | `8765` | WebSocket port |
|
||||
| `--udp-port` | `5005` | UDP port for ESP32 CSI frames |
|
||||
| `--ui-path` | (none) | Path to UI static files directory |
|
||||
| `--tick-ms` | `50` | Simulated frame interval (milliseconds) |
|
||||
| `--benchmark` | off | Run vital sign benchmark (1000 frames) and exit |
|
||||
| `--train` | off | Train a model from dataset |
|
||||
| `--dataset` | (none) | Path to dataset directory (MM-Fi or Wi-Pose) |
|
||||
| `--dataset-type` | `mmfi` | Dataset format: `mmfi` or `wipose` |
|
||||
| `--epochs` | `100` | Training epochs |
|
||||
| `--export-rvf` | (none) | Export RVF model container and exit |
|
||||
| `--save-rvf` | (none) | Save model state to RVF on shutdown |
|
||||
| `--model` | (none) | Load a trained `.rvf` model for inference |
|
||||
| `--load-rvf` | (none) | Load model config from RVF container |
|
||||
| `--progressive` | off | Enable progressive 3-layer model loading |
|
||||
|
||||
### Common Invocations
|
||||
|
||||
```bash
|
||||
# Simulated mode with UI (development)
|
||||
./target/release/sensing-server --source simulated --http-port 3000 --ws-port 3001 --ui-path ../../ui
|
||||
|
||||
# ESP32 hardware mode
|
||||
./target/release/sensing-server --source esp32 --udp-port 5005
|
||||
|
||||
# Windows WiFi RSSI
|
||||
./target/release/sensing-server --source windows --tick-ms 500
|
||||
|
||||
# Run benchmark
|
||||
./target/release/sensing-server --benchmark
|
||||
|
||||
# Train and export model
|
||||
./target/release/sensing-server --train --dataset data/ --epochs 100 --save-rvf model.rvf
|
||||
|
||||
# Load trained model with progressive loading
|
||||
./target/release/sensing-server --model model.rvf --progressive
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Training a Model
|
||||
|
||||
The training pipeline is implemented in pure Rust (7,832 lines, zero external ML dependencies).
|
||||
|
||||
### Step 1: Obtain a Dataset
|
||||
|
||||
The system supports two public WiFi CSI datasets:
|
||||
|
||||
| Dataset | Source | Format | Subjects | Environments |
|
||||
|---------|--------|--------|----------|-------------|
|
||||
| [MM-Fi](https://mmfi.github.io/) | NeurIPS 2023 | `.npy` | 40 | 4 rooms |
|
||||
| [Wi-Pose](https://github.com/aiot-lab/Wi-Pose) | AAAI 2024 | `.mat` | 8 | 3 rooms |
|
||||
|
||||
Download and place in a `data/` directory.
|
||||
|
||||
### Step 2: Train
|
||||
|
||||
```bash
|
||||
# From source
|
||||
./target/release/sensing-server --train --dataset data/ --dataset-type mmfi --epochs 100 --save-rvf model.rvf
|
||||
|
||||
# Via Docker (mount your data directory)
|
||||
docker run --rm \
|
||||
-v $(pwd)/data:/data \
|
||||
-v $(pwd)/output:/output \
|
||||
ruvnet/wifi-densepose:latest \
|
||||
--train --dataset /data --epochs 100 --export-rvf /output/model.rvf
|
||||
```
|
||||
|
||||
The pipeline runs 8 phases:
|
||||
1. Dataset loading (MM-Fi `.npy` or Wi-Pose `.mat`)
|
||||
2. Subcarrier resampling (114->56 or 30->56)
|
||||
3. Graph transformer construction (17 COCO keypoints, 16 bone edges)
|
||||
4. Cross-attention training (CSI features -> body pose)
|
||||
5. Composite loss optimization (MSE + CE + UV + temporal + bone + symmetry)
|
||||
6. SONA adaptation (micro-LoRA + EWC++)
|
||||
7. Sparse inference optimization (hot/cold neuron partitioning)
|
||||
8. RVF model packaging
|
||||
|
||||
### Step 3: Use the Trained Model
|
||||
|
||||
```bash
|
||||
./target/release/sensing-server --model model.rvf --progressive --source esp32
|
||||
```
|
||||
|
||||
Progressive loading enables instant startup (Layer A loads in <5ms with basic inference), with full model loading in the background.
|
||||
|
||||
---
|
||||
|
||||
## RVF Model Containers
|
||||
|
||||
The RuVector Format (RVF) packages a trained model into a single self-contained binary file.
|
||||
|
||||
### Export
|
||||
|
||||
```bash
|
||||
./target/release/sensing-server --export-rvf model.rvf
|
||||
```
|
||||
|
||||
### Load
|
||||
|
||||
```bash
|
||||
./target/release/sensing-server --model model.rvf --progressive
|
||||
```
|
||||
|
||||
### Contents
|
||||
|
||||
An RVF file contains: model weights, HNSW vector index, quantization codebooks, SONA adaptation profiles, Ed25519 training proof, and vital sign filter parameters.
|
||||
|
||||
### Deployment Targets
|
||||
|
||||
| Target | Quantization | Size | Load Time |
|
||||
|--------|-------------|------|-----------|
|
||||
| ESP32 / IoT | int4 | ~0.7 MB | <5ms |
|
||||
| Mobile / WASM | int8 | ~6-10 MB | ~200-500ms |
|
||||
| Field (WiFi-Mat) | fp16 | ~62 MB | ~2s |
|
||||
| Server / Cloud | f32 | ~50+ MB | ~3s |
|
||||
|
||||
---
|
||||
|
||||
## Hardware Setup
|
||||
|
||||
### ESP32-S3 Mesh
|
||||
|
||||
A 3-6 node ESP32-S3 mesh provides full CSI at 20 Hz. Total cost: ~$54 for a 3-node setup.
|
||||
|
||||
**What you need:**
|
||||
- 3-6x ESP32-S3 development boards (~$8 each)
|
||||
- A WiFi router (the CSI source)
|
||||
- A computer running the sensing server
|
||||
|
||||
**Flashing firmware:**
|
||||
|
||||
Pre-built binaries are available at [Releases](https://github.com/ruvnet/wifi-densepose/releases/tag/v0.1.0-esp32).
|
||||
|
||||
```bash
|
||||
# Flash an ESP32-S3 (requires esptool: pip install esptool)
|
||||
python -m esptool --chip esp32s3 --port COM7 --baud 460800 \
|
||||
write-flash --flash-mode dio --flash-size 4MB \
|
||||
0x0 bootloader.bin 0x8000 partition-table.bin 0x10000 esp32-csi-node.bin
|
||||
```
|
||||
|
||||
**Provisioning:**
|
||||
|
||||
```bash
|
||||
python scripts/provision.py --port COM7 \
|
||||
--ssid "YourWiFi" --password "YourPassword" --target-ip 192.168.1.20
|
||||
```
|
||||
|
||||
Replace `192.168.1.20` with the IP of the machine running the sensing server.
|
||||
|
||||
**Start the aggregator:**
|
||||
|
||||
```bash
|
||||
# From source
|
||||
./target/release/sensing-server --source esp32 --udp-port 5005 --http-port 3000 --ws-port 3001
|
||||
|
||||
# Docker
|
||||
docker run -p 3000:3000 -p 3001:3001 -p 5005:5005/udp ruvnet/wifi-densepose:latest --source esp32
|
||||
```
|
||||
|
||||
See [ADR-018](../docs/adr/ADR-018-esp32-dev-implementation.md) and [Tutorial #34](https://github.com/ruvnet/wifi-densepose/issues/34).
|
||||
|
||||
### Intel 5300 / Atheros NIC
|
||||
|
||||
These research NICs provide full CSI on Linux with firmware/driver modifications.
|
||||
|
||||
| NIC | Driver | Platform | Setup |
|
||||
|-----|--------|----------|-------|
|
||||
| Intel 5300 | `iwl-csi` | Linux | Custom firmware, ~$15 used |
|
||||
| Atheros AR9580 | `ath9k` patch | Linux | Kernel patch, ~$20 used |
|
||||
|
||||
These are advanced setups. See the respective driver documentation for installation.
|
||||
|
||||
---
|
||||
|
||||
## Docker Compose (Multi-Service)
|
||||
|
||||
For production deployments with both Rust and Python services:
|
||||
|
||||
```bash
|
||||
cd docker
|
||||
docker compose up
|
||||
```
|
||||
|
||||
This starts:
|
||||
- Rust sensing server on ports 3000 (HTTP), 3001 (WS), 5005 (UDP)
|
||||
- Python legacy server on ports 8080 (HTTP), 8765 (WS)
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Docker: "Connection refused" on localhost:3000
|
||||
|
||||
Make sure you're mapping the ports correctly:
|
||||
|
||||
```bash
|
||||
docker run -p 3000:3000 -p 3001:3001 ruvnet/wifi-densepose:latest
|
||||
```
|
||||
|
||||
The `-p 3000:3000` maps host port 3000 to container port 3000.
|
||||
|
||||
### Docker: No WebSocket data in UI
|
||||
|
||||
Add the WebSocket port mapping:
|
||||
|
||||
```bash
|
||||
docker run -p 3000:3000 -p 3001:3001 ruvnet/wifi-densepose:latest
|
||||
```
|
||||
|
||||
### ESP32: No data arriving
|
||||
|
||||
1. Verify the ESP32 is connected to the same WiFi network
|
||||
2. Check the target IP matches the sensing server machine: `python scripts/provision.py --port COM7 --target-ip <YOUR_IP>`
|
||||
3. Verify UDP port 5005 is not blocked by firewall
|
||||
4. Test with: `nc -lu 5005` (Linux) or similar UDP listener
|
||||
|
||||
### Build: Rust compilation errors
|
||||
|
||||
Ensure Rust 1.70+ is installed:
|
||||
```bash
|
||||
rustup update stable
|
||||
rustc --version
|
||||
```
|
||||
|
||||
### Windows: RSSI mode shows no data
|
||||
|
||||
Run the terminal as Administrator (required for `netsh wlan` access).
|
||||
|
||||
### Vital signs show 0 BPM
|
||||
|
||||
- Vital sign detection requires CSI-capable hardware (ESP32 or research NIC)
|
||||
- RSSI-only mode (Windows WiFi) does not have sufficient resolution for vital signs
|
||||
- In simulated mode, synthetic vital signs are generated after a few seconds of warm-up
|
||||
|
||||
---
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Do I need special hardware to try this?**
|
||||
No. Run `docker run -p 3000:3000 ruvnet/wifi-densepose:latest` and open `http://localhost:3000`. Simulated mode exercises the full pipeline with synthetic data.
|
||||
|
||||
**Q: Can consumer WiFi laptops do pose estimation?**
|
||||
No. Consumer WiFi exposes only RSSI (one number per access point), not CSI (56+ complex subcarrier values per frame). RSSI supports coarse presence and motion detection. Full pose estimation requires CSI-capable hardware like an ESP32-S3 ($8) or a research NIC.
|
||||
|
||||
**Q: How accurate is the pose estimation?**
|
||||
Accuracy depends on hardware and environment. With a 3-node ESP32 mesh in a single room, the system tracks 17 COCO keypoints. The core algorithm follows the CMU "DensePose From WiFi" paper ([arXiv:2301.00250](https://arxiv.org/abs/2301.00250)). See the paper for quantitative evaluations.
|
||||
|
||||
**Q: Does it work through walls?**
|
||||
Yes. WiFi signals penetrate non-metallic materials (drywall, wood, concrete up to ~30cm). Metal walls/doors significantly attenuate the signal. The effective through-wall range is approximately 5 meters.
|
||||
|
||||
**Q: How many people can it track?**
|
||||
Each access point can distinguish ~3-5 people with 56 subcarriers. Multi-AP deployments multiply linearly (e.g., 4 APs cover ~15-20 people). There is no hard software limit; the practical ceiling is signal physics.
|
||||
|
||||
**Q: Is this privacy-preserving?**
|
||||
The system uses WiFi radio signals, not cameras. No images or video are captured or stored. However, it does track human position, movement, and vital signs, which is personal data subject to applicable privacy regulations.
|
||||
|
||||
**Q: What's the Python vs Rust difference?**
|
||||
The Rust implementation (v2) is 810x faster than Python (v1) for the full CSI pipeline. The Docker image is 132 MB vs 569 MB. Rust is the primary and recommended runtime. Python v1 remains available for legacy workflows.
|
||||
|
||||
---
|
||||
|
||||
## Further Reading
|
||||
|
||||
- [Architecture Decision Records](../docs/adr/) - 24 ADRs covering all design decisions
|
||||
- [WiFi-Mat Disaster Response Guide](wifi-mat-user-guide.md) - Search & rescue module
|
||||
- [Build Guide](build-guide.md) - Detailed build instructions
|
||||
- [RuVector](https://github.com/ruvnet/ruvector) - Signal intelligence crate ecosystem
|
||||
- [CMU DensePose From WiFi](https://arxiv.org/abs/2301.00250) - The foundational research paper
|
||||
Reference in New Issue
Block a user