Compare commits
20 Commits
claude/ana
...
claude/use
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
374b0fdcef | ||
|
|
96b01008f7 | ||
|
|
38eb93e326 | ||
|
|
eab364bc51 | ||
|
|
3febf72674 | ||
|
|
8da6767273 | ||
|
|
2d6dc66f7c | ||
|
|
0a30f7904d | ||
|
|
b078190632 | ||
|
|
fdd2b2a486 | ||
|
|
d8fd5f4eba | ||
|
|
9e483e2c0f | ||
|
|
f89b81cdfa | ||
|
|
86e8ccd3d7 | ||
|
|
1f9dc60da4 | ||
|
|
342e5cf3f1 | ||
|
|
4f7ad6d2e6 | ||
|
|
aaec699223 | ||
|
|
72f031ae80 | ||
|
|
1c815bbfd5 |
@@ -8,6 +8,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **Project MERIDIAN (ADR-027)** — Cross-environment domain generalization for WiFi pose estimation (1,858 lines, 72 tests)
|
||||
- `HardwareNormalizer` — Catmull-Rom cubic interpolation resamples any hardware CSI to canonical 56 subcarriers; z-score + phase sanitization
|
||||
- `DomainFactorizer` + `GradientReversalLayer` — adversarial disentanglement of pose-relevant vs environment-specific features
|
||||
- `GeometryEncoder` + `FilmLayer` — Fourier positional encoding + DeepSets + FiLM for zero-shot deployment given AP positions
|
||||
- `VirtualDomainAugmentor` — synthetic environment diversity (room scale, wall material, scatterers, noise) for 4x training augmentation
|
||||
- `RapidAdaptation` — 10-second unsupervised calibration via contrastive test-time training + LoRA adapters
|
||||
- `CrossDomainEvaluator` — 6-metric evaluation protocol (MPJPE in-domain/cross-domain/few-shot/cross-hardware, domain gap ratio, adaptation speedup)
|
||||
- ADR-027: Cross-Environment Domain Generalization — 10 SOTA citations (PerceptAlign, X-Fi ICLR 2025, AM-FM, DGSense, CVPR 2024)
|
||||
- **Cross-platform RSSI adapters** — macOS CoreWLAN (`MacosCoreWlanScanner`) and Linux `iw` (`LinuxIwScanner`) Rust adapters with `#[cfg(target_os)]` gating
|
||||
- macOS CoreWLAN Python sensing adapter with Swift helper (`mac_wifi.swift`)
|
||||
- macOS synthetic BSSID generation (FNV-1a hash) for Sonoma 14.4+ BSSID redaction
|
||||
|
||||
13
CLAUDE.md
13
CLAUDE.md
@@ -89,6 +89,19 @@ All development on: `claude/validate-code-quality-WNrNw`
|
||||
- **HNSW**: Enabled
|
||||
- **Neural**: Enabled
|
||||
|
||||
## Pre-Merge Checklist
|
||||
|
||||
Before merging any PR, verify each item applies and is addressed:
|
||||
|
||||
1. **Tests pass** — `cargo test` (Rust) and `python -m pytest` (Python) green
|
||||
2. **README.md** — Update platform tables, crate descriptions, hardware tables, feature summaries if scope changed
|
||||
3. **CHANGELOG.md** — Add entry under `[Unreleased]` with what was added/fixed/changed
|
||||
4. **User guide** (`docs/user-guide.md`) — Update if new data sources, CLI flags, or setup steps were added
|
||||
5. **ADR index** — Update ADR count in README docs table if a new ADR was created
|
||||
6. **Docker Hub image** — Only rebuild if Dockerfile, dependencies, or runtime behavior changed (not needed for platform-gated code that doesn't affect the Linux container)
|
||||
7. **Crate publishing** — Only needed if a crate is published to crates.io and its public API changed (workspace-internal crates don't need publishing)
|
||||
8. **`.gitignore`** — Add any new build artifacts or binaries
|
||||
|
||||
## Build & Test
|
||||
|
||||
```bash
|
||||
|
||||
211
README.md
211
README.md
@@ -10,6 +10,7 @@ WiFi DensePose turns commodity WiFi signals into real-time human pose estimation
|
||||
[](https://hub.docker.com/r/ruvnet/wifi-densepose)
|
||||
[](#vital-sign-detection)
|
||||
[](#esp32-s3-hardware-pipeline)
|
||||
[](https://crates.io/crates/wifi-densepose-ruvector)
|
||||
|
||||
> | What | How | Speed |
|
||||
> |------|-----|-------|
|
||||
@@ -48,23 +49,66 @@ docker run -p 3000:3000 ruvnet/wifi-densepose:latest
|
||||
| [User Guide](docs/user-guide.md) | Step-by-step guide: installation, first run, API usage, hardware setup, training |
|
||||
| [WiFi-Mat User Guide](docs/wifi-mat-user-guide.md) | Disaster response module: search & rescue, START triage |
|
||||
| [Build Guide](docs/build-guide.md) | Building from source (Rust and Python) |
|
||||
| [Architecture Decisions](docs/adr/) | 26 ADRs covering signal processing, training, hardware, security |
|
||||
| [Architecture Decisions](docs/adr/) | 27 ADRs covering signal processing, training, hardware, security, domain generalization |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Key Features
|
||||
|
||||
### Sensing
|
||||
|
||||
See people, breathing, and heartbeats through walls — using only WiFi signals already in the room.
|
||||
|
||||
| | Feature | What It Means |
|
||||
|---|---------|---------------|
|
||||
| 🔒 | **Privacy-First** | Tracks human pose using only WiFi signals — no cameras, no video, no images stored |
|
||||
| ⚡ | **Real-Time** | Analyzes WiFi signals in under 100 microseconds per frame — fast enough for live monitoring |
|
||||
| 💓 | **Vital Signs** | Detects breathing rate (6-30 breaths/min) and heart rate (40-120 bpm) without any wearable |
|
||||
| 👥 | **Multi-Person** | Tracks multiple people simultaneously, each with independent pose and vitals — no hard software limit (physics: ~3-5 per AP with 56 subcarriers, more with multi-AP) |
|
||||
| 🧱 | **Through-Wall** | WiFi passes through walls, furniture, and debris — works where cameras cannot |
|
||||
| 🚑 | **Disaster Response** | Detects trapped survivors through rubble and classifies injury severity (START triage) |
|
||||
|
||||
### Intelligence
|
||||
|
||||
The system learns on its own and gets smarter over time — no hand-tuning, no labeled data required.
|
||||
|
||||
| | Feature | What It Means |
|
||||
|---|---------|---------------|
|
||||
| 🧠 | **Self-Learning** | Teaches itself from raw WiFi data — no labeled training sets, no cameras needed to bootstrap ([ADR-024](docs/adr/ADR-024-contrastive-csi-embedding-model.md)) |
|
||||
| 🎯 | **AI Signal Processing** | Attention networks, graph algorithms, and smart compression replace hand-tuned thresholds — adapts to each room automatically ([RuVector](https://github.com/ruvnet/ruvector)) |
|
||||
| 🌍 | **Works Everywhere** | Train once, deploy in any room — adversarial domain generalization strips environment bias so models transfer across rooms, buildings, and hardware ([ADR-027](docs/adr/ADR-027-cross-environment-domain-generalization.md)) |
|
||||
|
||||
### Performance & Deployment
|
||||
|
||||
Fast enough for real-time use, small enough for edge devices, simple enough for one-command setup.
|
||||
|
||||
| | Feature | What It Means |
|
||||
|---|---------|---------------|
|
||||
| ⚡ | **Real-Time** | Analyzes WiFi signals in under 100 microseconds per frame — fast enough for live monitoring |
|
||||
| 🦀 | **810x Faster** | Complete Rust rewrite: 54,000 frames/sec pipeline, 132 MB Docker image, 542+ tests |
|
||||
| 🐳 | **One-Command Setup** | `docker pull ruvnet/wifi-densepose:latest` — live sensing in 30 seconds, no toolchain needed |
|
||||
| 📦 | **Portable Models** | Trained models package into a single `.rvf` file — runs on edge, cloud, or browser (WASM) |
|
||||
| 🦀 | **810x Faster** | Complete Rust rewrite: 54,000 frames/sec pipeline, 132 MB Docker image, 542+ tests |
|
||||
|
||||
---
|
||||
|
||||
## 🔬 How It Works
|
||||
|
||||
WiFi routers flood every room with radio waves. When a person moves — or even breathes — those waves scatter differently. WiFi DensePose reads that scattering pattern and reconstructs what happened:
|
||||
|
||||
```
|
||||
WiFi Router → radio waves pass through room → hit human body → scatter
|
||||
↓
|
||||
ESP32 / WiFi NIC captures 56+ subcarrier amplitudes & phases (CSI) at 20 Hz
|
||||
↓
|
||||
Signal Processing cleans noise, removes interference, extracts motion signatures
|
||||
↓
|
||||
AI Backbone (RuVector) applies attention, graph algorithms, and compression
|
||||
↓
|
||||
Neural Network maps processed signals → 17 body keypoints + vital signs
|
||||
↓
|
||||
Output: real-time pose, breathing rate, heart rate, presence, room fingerprint
|
||||
```
|
||||
|
||||
No training cameras required — the [Self-Learning system (ADR-024)](docs/adr/ADR-024-contrastive-csi-embedding-model.md) bootstraps from raw WiFi data alone. [MERIDIAN (ADR-027)](docs/adr/ADR-027-cross-environment-domain-generalization.md) ensures the model works in any room, not just the one it trained in.
|
||||
|
||||
---
|
||||
|
||||
@@ -162,7 +206,7 @@ Every WiFi signal that passes through a room creates a unique fingerprint of tha
|
||||
- Turns any WiFi signal into a 128-number "fingerprint" that uniquely describes what's happening in a room
|
||||
- Learns entirely on its own from raw WiFi data — no cameras, no labeling, no human supervision needed
|
||||
- Recognizes rooms, detects intruders, identifies people, and classifies activities using only WiFi
|
||||
- Runs on an $8 ESP32 chip (the entire model fits in 60 KB of memory)
|
||||
- Runs on an $8 ESP32 chip (the entire model fits in 55 KB of memory)
|
||||
- Produces both body pose tracking AND environment fingerprints in a single computation
|
||||
|
||||
**Key Capabilities**
|
||||
@@ -227,10 +271,65 @@ cargo run -p wifi-densepose-sensing-server -- --model model.rvf --build-index en
|
||||
| Per-room MicroLoRA adapter | ~1,800 | 2 KB |
|
||||
| **Total** | **~55,000** | **55 KB** (of 520 KB available) |
|
||||
|
||||
The self-learning system builds on the [AI Backbone (RuVector)](#ai-backbone-ruvector) signal-processing layer — attention, graph algorithms, and compression — adding contrastive learning on top.
|
||||
|
||||
See [`docs/adr/ADR-024-contrastive-csi-embedding-model.md`](docs/adr/ADR-024-contrastive-csi-embedding-model.md) for full architectural details.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><a id="cross-environment-generalization-adr-027"></a><strong>🌍 Cross-Environment Generalization (ADR-027 — Project MERIDIAN)</strong> — Train once, deploy in any room without retraining</summary>
|
||||
|
||||
WiFi pose models trained in one room lose 40-70% accuracy when moved to another — even in the same building. The model memorizes room-specific multipath patterns instead of learning human motion. MERIDIAN forces the network to forget which room it's in while retaining everything about how people move.
|
||||
|
||||
**What it does in plain terms:**
|
||||
- Models trained in Room A work in Room B, C, D — without any retraining or calibration data
|
||||
- Handles different WiFi hardware (ESP32, Intel 5300, Atheros) with automatic chipset normalization
|
||||
- Knows where the WiFi transmitters are positioned and compensates for layout differences
|
||||
- Generates synthetic "virtual rooms" during training so the model sees thousands of environments
|
||||
- At deployment, adapts to a new room in seconds using a handful of unlabeled WiFi frames
|
||||
|
||||
**Key Components**
|
||||
|
||||
| What | How it works | Why it matters |
|
||||
|------|-------------|----------------|
|
||||
| **Gradient Reversal Layer** | An adversarial classifier tries to guess which room the signal came from; the main network is trained to fool it | Forces the model to discard room-specific shortcuts |
|
||||
| **Geometry Encoder (FiLM)** | Transmitter/receiver positions are Fourier-encoded and injected as scale+shift conditioning on every layer | The model knows *where* the hardware is, so it doesn't need to memorize layout |
|
||||
| **Hardware Normalizer** | Resamples any chipset's CSI to a canonical 56-subcarrier format with standardized amplitude | Intel 5300 and ESP32 data look identical to the model |
|
||||
| **Virtual Domain Augmentation** | Generates synthetic environments with random room scale, wall reflections, scatterers, and noise profiles | Training sees 1000s of rooms even with data from just 2-3 |
|
||||
| **Rapid Adaptation (TTT)** | Contrastive test-time training with LoRA weight generation from a few unlabeled frames | Zero-shot deployment — the model self-tunes on arrival |
|
||||
| **Cross-Domain Evaluator** | Leave-one-out evaluation across all training environments with per-environment PCK/OKS metrics | Proves generalization, not just memorization |
|
||||
|
||||
**Architecture**
|
||||
|
||||
```
|
||||
CSI Frame [any chipset]
|
||||
│
|
||||
▼
|
||||
HardwareNormalizer ──→ canonical 56 subcarriers, N(0,1) amplitude
|
||||
│
|
||||
▼
|
||||
CSI Encoder (existing) ──→ latent features
|
||||
│
|
||||
├──→ Pose Head ──→ 17-joint pose (environment-invariant)
|
||||
│
|
||||
├──→ Gradient Reversal Layer ──→ Domain Classifier (adversarial)
|
||||
│ λ ramps 0→1 via cosine/exponential schedule
|
||||
│
|
||||
└──→ Geometry Encoder ──→ FiLM conditioning (scale + shift)
|
||||
Fourier positional encoding → DeepSets → per-layer modulation
|
||||
```
|
||||
|
||||
**Security hardening:**
|
||||
- Bounded calibration buffer (max 10,000 frames) prevents memory exhaustion
|
||||
- `adapt()` returns `Result<_, AdaptError>` — no panics on bad input
|
||||
- Atomic instance counter ensures unique weight initialization across threads
|
||||
- Division-by-zero guards on all augmentation parameters
|
||||
|
||||
See [`docs/adr/ADR-027-cross-environment-domain-generalization.md`](docs/adr/ADR-027-cross-environment-domain-generalization.md) for full architectural details.
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 📦 Installation
|
||||
@@ -364,21 +463,7 @@ cargo add wifi-densepose-ruvector # RuVector v2.0.4 integration layer (ADR-017
|
||||
| [`wifi-densepose-config`](https://crates.io/crates/wifi-densepose-config) | Configuration management | -- | [](https://crates.io/crates/wifi-densepose-config) |
|
||||
| [`wifi-densepose-db`](https://crates.io/crates/wifi-densepose-db) | Database persistence (PostgreSQL, SQLite, Redis) | -- | [](https://crates.io/crates/wifi-densepose-db) |
|
||||
|
||||
All crates integrate with [RuVector v2.0.4](https://github.com/ruvnet/ruvector) for graph algorithms and neural network optimization.
|
||||
|
||||
#### `wifi-densepose-ruvector` — ADR-017 Integration Layer
|
||||
|
||||
The `wifi-densepose-ruvector` crate ([`docs/adr/ADR-017-ruvector-signal-mat-integration.md`](docs/adr/ADR-017-ruvector-signal-mat-integration.md)) implements all 7 ruvector integration points across the signal processing and disaster detection domains:
|
||||
|
||||
| Module | Integration | RuVector crate | Benefit |
|
||||
|--------|-------------|----------------|---------|
|
||||
| `signal::subcarrier` | `mincut_subcarrier_partition` | `ruvector-mincut` | O(n^1.5 log n) dynamic partition vs O(n log n) static sort |
|
||||
| `signal::spectrogram` | `gate_spectrogram` | `ruvector-attn-mincut` | Attention gating suppresses noise frames in STFT output |
|
||||
| `signal::bvp` | `attention_weighted_bvp` | `ruvector-attention` | Sensitivity-weighted aggregation across subcarriers |
|
||||
| `signal::fresnel` | `solve_fresnel_geometry` | `ruvector-solver` | Data-driven TX-body-RX geometry from multi-subcarrier observations |
|
||||
| `mat::triangulation` | `solve_triangulation` | `ruvector-solver` | O(1) 2×2 Neumann system vs O(N³) Gaussian elimination |
|
||||
| `mat::breathing` | `CompressedBreathingBuffer` | `ruvector-temporal-tensor` | 13.4 MB/zone → 3.4–6.7 MB (50–75% reduction per zone) |
|
||||
| `mat::heartbeat` | `CompressedHeartbeatSpectrogram` | `ruvector-temporal-tensor` | Tiered hot/warm/cold compression for micro-Doppler spectrograms |
|
||||
All crates integrate with [RuVector v2.0.4](https://github.com/ruvnet/ruvector) — see [AI Backbone](#ai-backbone-ruvector) below.
|
||||
|
||||
</details>
|
||||
|
||||
@@ -458,7 +543,8 @@ The signal processing stack transforms raw WiFi Channel State Information into a
|
||||
|
||||
| Section | Description | Docs |
|
||||
|---------|-------------|------|
|
||||
| [Key Features](#key-features) | Privacy-first sensing, real-time performance, multi-person tracking, Docker | — |
|
||||
| [Key Features](#key-features) | Sensing, Intelligence, and Performance & Deployment capabilities | — |
|
||||
| [How It Works](#how-it-works) | End-to-end pipeline: radio waves → CSI capture → signal processing → AI → pose + vitals | — |
|
||||
| [ESP32-S3 Hardware Pipeline](#esp32-s3-hardware-pipeline) | 20 Hz CSI streaming, binary frame parsing, flash & provision | [ADR-018](docs/adr/ADR-018-esp32-dev-implementation.md) · [Tutorial #34](https://github.com/ruvnet/wifi-densepose/issues/34) |
|
||||
| [Vital Sign Detection](#vital-sign-detection) | Breathing 6-30 BPM, heartbeat 40-120 BPM, FFT peak detection | [ADR-021](docs/adr/ADR-021-vital-sign-detection-rvdna-pipeline.md) |
|
||||
| [WiFi Scan Domain Layer](#wifi-scan-domain-layer) | 8-stage RSSI pipeline, multi-BSSID fingerprinting, Windows WiFi | [ADR-022](docs/adr/ADR-022-windows-wifi-enhanced-fidelity-ruvector.md) · [Tutorial #36](https://github.com/ruvnet/wifi-densepose/issues/36) |
|
||||
@@ -477,6 +563,9 @@ The neural pipeline uses a graph transformer with cross-attention to map CSI fea
|
||||
| [RVF Model Container](#rvf-model-container) | Binary packaging with Ed25519 signing, progressive 3-layer loading, SIMD quantization | [ADR-023](docs/adr/ADR-023-trained-densepose-model-ruvector-pipeline.md) |
|
||||
| [Training & Fine-Tuning](#training--fine-tuning) | 8-phase pure Rust pipeline (7,832 lines), MM-Fi/Wi-Pose pre-training, 6-term composite loss, SONA LoRA | [ADR-023](docs/adr/ADR-023-trained-densepose-model-ruvector-pipeline.md) |
|
||||
| [RuVector Crates](#ruvector-crates) | 11 vendored Rust crates from [ruvector](https://github.com/ruvnet/ruvector): attention, min-cut, solver, GNN, HNSW, temporal compression, sparse inference | [GitHub](https://github.com/ruvnet/ruvector) · [Source](vendor/ruvector/) |
|
||||
| [AI Backbone (RuVector)](#ai-backbone-ruvector) | 5 AI capabilities replacing hand-tuned thresholds: attention, graph min-cut, sparse solvers, tiered compression | [crates.io](https://crates.io/crates/wifi-densepose-ruvector) |
|
||||
| [Self-Learning WiFi AI (ADR-024)](#self-learning-wifi-ai-adr-024) | Contrastive self-supervised learning, room fingerprinting, anomaly detection, 55 KB model | [ADR-024](docs/adr/ADR-024-contrastive-csi-embedding-model.md) |
|
||||
| [Cross-Environment Generalization (ADR-027)](docs/adr/ADR-027-cross-environment-domain-generalization.md) | Domain-adversarial training, geometry-conditioned inference, hardware normalization, zero-shot deployment | [ADR-027](docs/adr/ADR-027-cross-environment-domain-generalization.md) |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -525,7 +614,7 @@ WiFi DensePose is MIT-licensed open source, developed by [ruvnet](https://github
|
||||
|
||||
| Section | Description | Link |
|
||||
|---------|-------------|------|
|
||||
| [Changelog](#changelog) | v2.3.0 (training pipeline + Docker), v2.2.0 (SOTA + WiFi-Mat), v2.1.0 (Rust port) | — |
|
||||
| [Changelog](#changelog) | v3.0.0 (AETHER AI + Docker), v2.0.0 (Rust port + SOTA + WiFi-Mat) | [CHANGELOG.md](CHANGELOG.md) |
|
||||
| [License](#license) | MIT License | [LICENSE](LICENSE) |
|
||||
| [Support](#support) | Bug reports, feature requests, community discussion | [Issues](https://github.com/ruvnet/wifi-densepose/issues) · [Discussions](https://github.com/ruvnet/wifi-densepose/discussions) |
|
||||
|
||||
@@ -712,6 +801,41 @@ See [ADR-014](docs/adr/ADR-014-sota-signal-processing.md) for full mathematical
|
||||
|
||||
## 🧠 Models & Training
|
||||
|
||||
<details>
|
||||
<summary><a id="ai-backbone-ruvector"></a><strong>🤖 AI Backbone: RuVector</strong> — Attention, graph algorithms, and edge-AI compression powering the sensing pipeline</summary>
|
||||
|
||||
Raw WiFi signals are noisy, redundant, and environment-dependent. [RuVector](https://github.com/ruvnet/ruvector) is the AI intelligence layer that transforms them into clean, structured input for the DensePose neural network. It uses **attention mechanisms** to learn which signals to trust, **graph algorithms** that automatically discover which WiFi channels are sensitive to body motion, and **compressed representations** that make edge inference possible on an $8 microcontroller.
|
||||
|
||||
Without RuVector, WiFi DensePose would need hand-tuned thresholds, brute-force matrix math, and 4x more memory — making real-time edge inference impossible.
|
||||
|
||||
```
|
||||
Raw WiFi CSI (56 subcarriers, noisy)
|
||||
|
|
||||
+-- ruvector-mincut ---------- Which channels carry body-motion signal? (learned graph partitioning)
|
||||
+-- ruvector-attn-mincut ----- Which time frames are signal vs noise? (attention-gated filtering)
|
||||
+-- ruvector-attention ------- How to fuse multi-antenna data? (learned weighted aggregation)
|
||||
|
|
||||
v
|
||||
Clean, structured signal --> DensePose Neural Network --> 17-keypoint body pose
|
||||
--> FFT Vital Signs -----------> breathing rate, heart rate
|
||||
--> ruvector-solver ------------> physics-based localization
|
||||
```
|
||||
|
||||
The [`wifi-densepose-ruvector`](https://crates.io/crates/wifi-densepose-ruvector) crate ([ADR-017](docs/adr/ADR-017-ruvector-signal-mat-integration.md)) connects all 7 integration points:
|
||||
|
||||
| AI Capability | What It Replaces | RuVector Crate | Result |
|
||||
|--------------|-----------------|----------------|--------|
|
||||
| **Self-optimizing channel selection** | Hand-tuned thresholds that break when rooms change | `ruvector-mincut` | Graph min-cut adapts to any environment automatically |
|
||||
| **Attention-based signal cleaning** | Fixed energy cutoffs that miss subtle breathing | `ruvector-attn-mincut` | Learned gating amplifies body signals, suppresses noise |
|
||||
| **Learned signal fusion** | Simple averaging where one bad channel corrupts all | `ruvector-attention` | Transformer-style attention downweights corrupted channels |
|
||||
| **Physics-informed localization** | Expensive nonlinear solvers | `ruvector-solver` | Sparse least-squares Fresnel geometry in real-time |
|
||||
| **O(1) survivor triangulation** | O(N^3) matrix inversion | `ruvector-solver` | Neumann series linearization for instant position updates |
|
||||
| **75% memory compression** | 13.4 MB breathing buffers that overflow edge devices | `ruvector-temporal-tensor` | Tiered 3-8 bit quantization fits 60s of vitals in 3.4 MB |
|
||||
|
||||
See [issue #67](https://github.com/ruvnet/wifi-densepose/issues/67) for a deep dive with code examples, or [`cargo add wifi-densepose-ruvector`](https://crates.io/crates/wifi-densepose-ruvector) to use it directly.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><a id="rvf-model-container"></a><strong>📦 RVF Model Container</strong> — Single-file deployment with progressive loading</summary>
|
||||
|
||||
@@ -1272,37 +1396,32 @@ pre-commit install
|
||||
<details>
|
||||
<summary><strong>Release history</strong></summary>
|
||||
|
||||
### v2.3.0 — 2026-03-01
|
||||
### v3.0.0 — 2026-03-01
|
||||
|
||||
The largest release to date — delivers the complete end-to-end training pipeline, Docker images, and vital sign detection. The Rust sensing server now supports full model training, RVF export, and progressive model loading from a single binary.
|
||||
Major release: AETHER contrastive embedding model, AI signal processing backbone, cross-platform adapters, Docker Hub images, and comprehensive README overhaul.
|
||||
|
||||
- **Project AETHER (ADR-024)** — Self-supervised contrastive learning for WiFi CSI fingerprinting, similarity search, and anomaly detection; 55 KB model fits on ESP32
|
||||
- **AI Backbone (`wifi-densepose-ruvector`)** — 7 RuVector integration points replacing hand-tuned thresholds with attention, graph algorithms, and smart compression; [published to crates.io](https://crates.io/crates/wifi-densepose-ruvector)
|
||||
- **Cross-platform RSSI adapters** — macOS CoreWLAN and Linux `iw` Rust adapters with `#[cfg(target_os)]` gating (ADR-025)
|
||||
- **Docker images published** — `ruvnet/wifi-densepose:latest` (132 MB Rust) and `:python` (569 MB)
|
||||
- **8-phase DensePose training pipeline (ADR-023)** — Dataset loaders (MM-Fi, Wi-Pose), graph transformer with cross-attention, 6-term composite loss, cosine-scheduled SGD, PCK/OKS validation, SONA adaptation, sparse inference engine, RVF model packaging
|
||||
- **`--export-rvf` CLI flag** — Standalone RVF model container generation with vital config, training proof, and SONA profiles
|
||||
- **`--train` CLI flag** — Full training mode with best-epoch snapshotting and checkpoint saving
|
||||
- **Vital sign detection (ADR-021)** — FFT-based breathing (6-30 BPM) and heartbeat (40-120 BPM) extraction, 11,665 fps benchmark
|
||||
- **WiFi scan domain layer (ADR-022/025)** — 8-stage pure-Rust signal intelligence pipeline for Windows, macOS, and Linux WiFi RSSI
|
||||
- **New crates** — `wifi-densepose-vitals` (1,863 lines) and `wifi-densepose-wifiscan` (4,829 lines)
|
||||
- **542+ Rust tests** — All passing, zero mocks
|
||||
- **Project MERIDIAN (ADR-027)** — Cross-environment domain generalization: gradient reversal, geometry-conditioned FiLM, virtual domain augmentation, contrastive test-time training; zero-shot room transfer
|
||||
- **10-phase DensePose training pipeline (ADR-023/027)** — Graph transformer, 6-term composite loss, SONA adaptation, RVF packaging, hardware normalization, domain-adversarial training
|
||||
- **Vital sign detection (ADR-021)** — FFT-based breathing (6-30 BPM) and heartbeat (40-120 BPM), 11,665 fps
|
||||
- **WiFi scan domain layer (ADR-022/025)** — 8-stage signal intelligence pipeline for Windows, macOS, and Linux
|
||||
- **700+ Rust tests** — All passing, zero mocks
|
||||
|
||||
### v2.2.0 — 2026-02-28
|
||||
### v2.0.0 — 2026-02-28
|
||||
|
||||
Introduced the guided installer, SOTA signal processing algorithms, and the WiFi-Mat disaster response module. This release established the ESP32 hardware path and security hardening.
|
||||
Complete Rust sensing server, SOTA signal processing, WiFi-Mat disaster response, ESP32 hardware, RuVector integration, guided installer, and security hardening.
|
||||
|
||||
- **Guided installer** — `./install.sh` with 7-step hardware detection and 8 install profiles
|
||||
- **6 SOTA signal algorithms (ADR-014)** — SpotFi conjugate multiplication, Hampel filter, Fresnel zone model, CSI spectrogram, subcarrier selection, body velocity profile
|
||||
- **WiFi-Mat disaster response** — START triage, scan zones, 3D localization, priority alerts — 139 tests
|
||||
- **ESP32 CSI hardware parser** — Binary frame parsing with I/Q extraction — 28 tests
|
||||
- **Security hardening** — 10 vulnerabilities fixed (CVE remediation, input validation, path security)
|
||||
|
||||
### v2.1.0 — 2026-02-28
|
||||
|
||||
The foundational Rust release — ported the Python v1 pipeline to Rust with 810x speedup, integrated the RuVector signal intelligence crates, and added the Three.js real-time visualization.
|
||||
|
||||
- **RuVector integration** — 11 vendored crates (ADR-002 through ADR-013) for HNSW indexing, attention, GNN, temporal compression, min-cut, solver
|
||||
- **ESP32 CSI sensor mesh** — $54 starter kit with 3-6 ESP32-S3 nodes streaming at 20 Hz
|
||||
- **Three.js visualization** — 3D body model with 17 joints, real-time WebSocket streaming
|
||||
- **CI verification pipeline** — Determinism checks and unseeded random scan across all signal operations
|
||||
- **Rust sensing server** — Axum REST API + WebSocket, 810x speedup over Python, 54K fps pipeline
|
||||
- **RuVector integration** — 11 vendored crates for HNSW, attention, GNN, temporal compression, min-cut, solver
|
||||
- **6 SOTA signal algorithms (ADR-014)** — SpotFi, Hampel, Fresnel, spectrogram, subcarrier selection, BVP
|
||||
- **WiFi-Mat disaster response** — START triage, 3D localization, priority alerts — 139 tests
|
||||
- **ESP32 CSI hardware** — Binary frame parsing, $54 starter kit, 20 Hz streaming
|
||||
- **Guided installer** — 7-step hardware detection, 8 install profiles
|
||||
- **Three.js visualization** — 3D body model, 17 joints, real-time WebSocket
|
||||
- **Security hardening** — 10 vulnerabilities fixed
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-002: RuVector RVF Integration Strategy
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Superseded by [ADR-016](ADR-016-ruvector-integration.md) and [ADR-017](ADR-017-ruvector-signal-mat-integration.md)
|
||||
|
||||
> **Note:** The vision in this ADR has been fully realized. ADR-016 integrates all 5 RuVector crates into the training pipeline. ADR-017 adds 7 signal + MAT integration points. The `wifi-densepose-ruvector` crate is [published on crates.io](https://crates.io/crates/wifi-densepose-ruvector). See also [ADR-027](ADR-027-cross-environment-domain-generalization.md) for how RuVector is extended with domain generalization.
|
||||
|
||||
## Date
|
||||
2026-02-28
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-004: HNSW Vector Search for Signal Fingerprinting
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Partially realized by [ADR-024](ADR-024-contrastive-csi-embedding-model.md); extended by [ADR-027](ADR-027-cross-environment-domain-generalization.md)
|
||||
|
||||
> **Note:** ADR-024 (AETHER) implements HNSW-compatible fingerprint indices with 4 index types. ADR-027 (MERIDIAN) extends this with domain-disentangled embeddings so fingerprints match across environments, not just within a single room.
|
||||
|
||||
## Date
|
||||
2026-02-28
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-005: SONA Self-Learning for Pose Estimation
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Partially realized in [ADR-023](ADR-023-trained-densepose-model-ruvector-pipeline.md); extended by [ADR-027](ADR-027-cross-environment-domain-generalization.md)
|
||||
|
||||
> **Note:** ADR-023 implements SONA with MicroLoRA rank-4 adapters and EWC++ memory preservation. ADR-027 (MERIDIAN) extends SONA with unsupervised rapid adaptation: 10 seconds of unlabeled WiFi data in a new room automatically generates environment-specific LoRA weights via contrastive test-time training.
|
||||
|
||||
## Date
|
||||
2026-02-28
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# ADR-006: GNN-Enhanced CSI Pattern Recognition
|
||||
|
||||
## Status
|
||||
Proposed
|
||||
Partially realized in [ADR-023](ADR-023-trained-densepose-model-ruvector-pipeline.md); extended by [ADR-027](ADR-027-cross-environment-domain-generalization.md)
|
||||
|
||||
> **Note:** ADR-023 implements a 2-layer GCN on the COCO skeleton graph for spatial reasoning. ADR-027 (MERIDIAN) adds domain-adversarial regularization via a gradient reversal layer that forces the GCN to learn environment-invariant graph features, shedding room-specific multipath patterns.
|
||||
|
||||
## Date
|
||||
2026-02-28
|
||||
|
||||
548
docs/adr/ADR-027-cross-environment-domain-generalization.md
Normal file
548
docs/adr/ADR-027-cross-environment-domain-generalization.md
Normal file
@@ -0,0 +1,548 @@
|
||||
# ADR-027: Project MERIDIAN -- Cross-Environment Domain Generalization for WiFi Pose Estimation
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Status** | Proposed |
|
||||
| **Date** | 2026-03-01 |
|
||||
| **Deciders** | ruv |
|
||||
| **Codename** | **MERIDIAN** -- Multi-Environment Robust Inference via Domain-Invariant Alignment Networks |
|
||||
| **Relates to** | ADR-005 (SONA Self-Learning), ADR-014 (SOTA Signal Processing), ADR-015 (Public Datasets), ADR-016 (RuVector Integration), ADR-023 (Trained DensePose Pipeline), ADR-024 (AETHER Contrastive Embeddings) |
|
||||
|
||||
---
|
||||
|
||||
## 1. Context
|
||||
|
||||
### 1.1 The Domain Gap Problem
|
||||
|
||||
WiFi-based pose estimation models exhibit severe performance degradation when deployed in environments different from their training setting. A model trained in Room A with a specific transceiver layout, wall material composition, and furniture arrangement can lose 40-70% accuracy when moved to Room B -- even in the same building. This brittleness is the single largest barrier to real-world WiFi sensing deployment.
|
||||
|
||||
The root cause is three-fold:
|
||||
|
||||
1. **Layout overfitting**: Models memorize the spatial relationship between transmitter, receiver, and the coordinate system, rather than learning environment-agnostic human motion features. PerceptAlign (Chen et al., 2026; arXiv:2601.12252) demonstrated that cross-layout error drops by >60% when geometry conditioning is introduced.
|
||||
|
||||
2. **Multipath memorization**: The multipath channel profile encodes room geometry (wall positions, furniture, materials) as a static fingerprint. Models learn this fingerprint as a shortcut, using room-specific multipath patterns to predict positions rather than extracting pose-relevant body reflections.
|
||||
|
||||
3. **Hardware heterogeneity**: Different WiFi chipsets (ESP32, Intel 5300, Atheros) produce CSI with different subcarrier counts, phase noise profiles, and sampling rates. A model trained on Intel 5300 (30 subcarriers, 3x3 MIMO) fails on ESP32-S3 (64 subcarriers, 1x1 SISO).
|
||||
|
||||
The current wifi-densepose system (ADR-023) trains and evaluates on a single environment from MM-Fi or Wi-Pose. There is no mechanism to disentangle human motion from environment, adapt to new rooms without full retraining, or handle mixed hardware deployments.
|
||||
|
||||
### 1.2 SOTA Landscape (2024-2026)
|
||||
|
||||
Five concurrent lines of research have converged on the domain generalization problem:
|
||||
|
||||
**Cross-Layout Pose Estimation:**
|
||||
- **PerceptAlign** (Chen et al., 2026; arXiv:2601.12252): First geometry-conditioned framework. Encodes transceiver positions into high-dimensional embeddings fused with CSI features, achieving 60%+ cross-domain error reduction. Constructed the largest cross-domain WiFi pose dataset: 21 subjects, 5 scenes, 18 actions, 7 layouts.
|
||||
- **AdaPose** (Zhou et al., 2024; IEEE IoT Journal, arXiv:2309.16964): Mapping Consistency Loss aligns domain discrepancy at the mapping level. First to address cross-domain WiFi pose estimation specifically.
|
||||
- **Person-in-WiFi 3D** (Yan et al., CVPR 2024): End-to-end multi-person 3D pose from WiFi, achieving 91.7mm single-person error, but generalization across layouts remains an open problem.
|
||||
|
||||
**Domain Generalization Frameworks:**
|
||||
- **DGSense** (Zhou et al., 2025; arXiv:2502.08155): Virtual data generator + episodic training for domain-invariant features. Generalizes to unseen domains without target data across WiFi, mmWave, and acoustic sensing.
|
||||
- **Context-Aware Predictive Coding (CAPC)** (2024; arXiv:2410.01825; IEEE OJCOMS): Self-supervised CPC + Barlow Twins for WiFi, with 24.7% accuracy improvement over supervised learning on unseen environments.
|
||||
|
||||
**Foundation Models:**
|
||||
- **X-Fi** (Chen & Yang, ICLR 2025; arXiv:2410.10167): First modality-invariant foundation model for human sensing. X-fusion mechanism preserves modality-specific features. 24.8% MPJPE improvement on MM-Fi.
|
||||
- **AM-FM** (2026; arXiv:2602.11200): First WiFi foundation model, pre-trained on 9.2M unlabeled CSI samples across 20 device types over 439 days. Contrastive learning + masked reconstruction + physics-informed objectives.
|
||||
|
||||
**Generative Approaches:**
|
||||
- **LatentCSI** (Ramesh et al., 2025; arXiv:2506.10605): Lightweight CSI encoder maps directly into Stable Diffusion 3 latent space, demonstrating that CSI contains enough spatial information to reconstruct room imagery.
|
||||
|
||||
### 1.3 What MERIDIAN Adds to the Existing System
|
||||
|
||||
| Current Capability | Gap | MERIDIAN Addition |
|
||||
|-------------------|-----|------------------|
|
||||
| AETHER embeddings (ADR-024) | Embeddings encode environment identity -- useful for fingerprinting but harmful for cross-environment transfer | Environment-disentangled embeddings with explicit factorization |
|
||||
| SONA LoRA adapters (ADR-005) | Adapters must be manually created per environment; no mechanism to generate them from few-shot data | Zero-shot environment adaptation via geometry-conditioned inference |
|
||||
| MM-Fi/Wi-Pose training (ADR-015) | Single-environment train/eval; no cross-domain protocol | Multi-domain training protocol with environment augmentation |
|
||||
| SpotFi phase correction (ADR-014) | Hardware-specific phase calibration | Hardware-invariant CSI normalization layer |
|
||||
| RuVector attention (ADR-016) | Attention weights learn environment-specific patterns | Domain-adversarial attention regularization |
|
||||
|
||||
---
|
||||
|
||||
## 2. Decision
|
||||
|
||||
### 2.1 Architecture: Environment-Disentangled Dual-Path Transformer
|
||||
|
||||
MERIDIAN adds a domain generalization layer between the CSI encoder and the pose/embedding heads. The core insight is explicit factorization: decompose the latent representation into a **pose-relevant** component (invariant across environments) and an **environment** component (captures room geometry, hardware, layout):
|
||||
|
||||
```
|
||||
CSI Frame(s) [n_pairs x n_subcarriers]
|
||||
|
|
||||
v
|
||||
HardwareNormalizer [NEW: chipset-invariant preprocessing]
|
||||
| - Resample to canonical 56 subcarriers
|
||||
| - Normalize amplitude distribution to N(0,1) per-frame
|
||||
| - Apply SanitizedPhaseTransform (hardware-agnostic)
|
||||
|
|
||||
v
|
||||
csi_embed (Linear 56 -> d_model=64) [EXISTING]
|
||||
|
|
||||
v
|
||||
CrossAttention (Q=keypoint_queries, [EXISTING]
|
||||
K,V=csi_embed)
|
||||
|
|
||||
v
|
||||
GnnStack (2-layer GCN) [EXISTING]
|
||||
|
|
||||
v
|
||||
body_part_features [17 x 64] [EXISTING]
|
||||
|
|
||||
+---> DomainFactorizer: [NEW]
|
||||
| |
|
||||
| +---> PoseEncoder: [NEW: domain-invariant path]
|
||||
| | fc1: Linear(64, 128) + LayerNorm + GELU
|
||||
| | fc2: Linear(128, 64)
|
||||
| | --> h_pose [17 x 64] (invariant to environment)
|
||||
| |
|
||||
| +---> EnvEncoder: [NEW: environment-specific path]
|
||||
| GlobalMeanPool [17 x 64] -> [64]
|
||||
| fc_env: Linear(64, 32)
|
||||
| --> h_env [32] (captures room/hardware identity)
|
||||
|
|
||||
+---> h_pose ---> xyz_head + conf_head [EXISTING: pose regression]
|
||||
| --> keypoints [17 x (x,y,z,conf)]
|
||||
|
|
||||
+---> h_pose ---> MeanPool -> ProjectionHead -> z_csi [128] [ADR-024 AETHER]
|
||||
|
|
||||
+---> h_env ---> (discarded at inference; used only for training signal)
|
||||
```
|
||||
|
||||
### 2.2 Domain-Adversarial Training with Gradient Reversal
|
||||
|
||||
To force `h_pose` to be environment-invariant, we employ domain-adversarial training (Ganin et al., 2016) with a gradient reversal layer (GRL):
|
||||
|
||||
```
|
||||
h_pose [17 x 64]
|
||||
|
|
||||
+---> [Normal gradient] --> xyz_head --> L_pose
|
||||
|
|
||||
+---> [GRL: multiply grad by -lambda_adv]
|
||||
|
|
||||
v
|
||||
DomainClassifier:
|
||||
MeanPool [17 x 64] -> [64]
|
||||
fc1: Linear(64, 32) + ReLU + Dropout(0.3)
|
||||
fc2: Linear(32, n_domains)
|
||||
--> domain_logits
|
||||
--> L_domain = CrossEntropy(domain_logits, domain_label)
|
||||
|
||||
Total loss:
|
||||
L = L_pose + lambda_c * L_contrastive + lambda_adv * L_domain
|
||||
+ lambda_env * L_env_recon
|
||||
```
|
||||
|
||||
The GRL reverses the gradient flowing from `L_domain` into `PoseEncoder`, meaning the PoseEncoder is trained to **maximize** domain classification error -- forcing `h_pose` to shed all environment-specific information.
|
||||
|
||||
**Key hyperparameters:**
|
||||
- `lambda_adv`: Adversarial weight, annealed from 0.0 to 1.0 over first 20 epochs using the schedule `lambda_adv(p) = 2 / (1 + exp(-10 * p)) - 1` where `p = epoch / max_epochs`
|
||||
- `lambda_env = 0.1`: Environment reconstruction weight (auxiliary task to ensure `h_env` captures what `h_pose` discards)
|
||||
- `lambda_c = 0.1`: Contrastive loss weight from AETHER (unchanged)
|
||||
|
||||
### 2.3 Geometry-Conditioned Inference (Zero-Shot Adaptation)
|
||||
|
||||
Inspired by PerceptAlign, MERIDIAN conditions the pose decoder on the physical transceiver geometry. At deployment time, the user provides AP/sensor positions (known from installation), and the model adjusts its coordinate frame accordingly:
|
||||
|
||||
```rust
|
||||
/// Encodes transceiver geometry into a conditioning vector.
|
||||
/// Positions are in meters relative to an arbitrary room origin.
|
||||
pub struct GeometryEncoder {
|
||||
/// Fourier positional encoding of 3D coordinates
|
||||
pos_embed: FourierPositionalEncoding, // 3 coords -> 64 dims per position
|
||||
/// Aggregates variable-count AP positions into fixed-dim vector
|
||||
set_encoder: DeepSets, // permutation-invariant {AP_1..AP_n} -> 64
|
||||
}
|
||||
|
||||
/// Fourier features: [sin(2^0 * pi * x), cos(2^0 * pi * x), ...,
|
||||
/// sin(2^(L-1) * pi * x), cos(2^(L-1) * pi * x)]
|
||||
/// L = 10 frequency bands, producing 60 dims per coordinate (+ 3 raw = 63, padded to 64)
|
||||
pub struct FourierPositionalEncoding {
|
||||
n_frequencies: usize, // default: 10
|
||||
scale: f32, // default: 1.0 (meters)
|
||||
}
|
||||
|
||||
/// DeepSets: phi(x) -> mean-pool -> rho(.) for permutation-invariant set encoding
|
||||
pub struct DeepSets {
|
||||
phi: Linear, // 64 -> 64
|
||||
rho: Linear, // 64 -> 64
|
||||
}
|
||||
```
|
||||
|
||||
The geometry embedding `g` (64-dim) is injected into the pose decoder via FiLM conditioning:
|
||||
|
||||
```
|
||||
g = GeometryEncoder(ap_positions) [64-dim]
|
||||
gamma = Linear(64, 64)(g) [per-feature scale]
|
||||
beta = Linear(64, 64)(g) [per-feature shift]
|
||||
|
||||
h_pose_conditioned = gamma * h_pose + beta [FiLM: Feature-wise Linear Modulation]
|
||||
|
|
||||
v
|
||||
xyz_head --> keypoints
|
||||
```
|
||||
|
||||
This enables zero-shot deployment: given the positions of WiFi APs in a new room, the model adapts its coordinate prediction without any retraining.
|
||||
|
||||
### 2.4 Hardware-Invariant CSI Normalization
|
||||
|
||||
```rust
|
||||
/// Normalizes CSI from heterogeneous hardware to a canonical representation.
|
||||
/// Handles ESP32-S3 (64 sub), Intel 5300 (30 sub), Atheros (56 sub).
|
||||
pub struct HardwareNormalizer {
|
||||
/// Target subcarrier count (project all hardware to this)
|
||||
canonical_subcarriers: usize, // default: 56 (matches MM-Fi)
|
||||
/// Per-hardware amplitude statistics for z-score normalization
|
||||
hw_stats: HashMap<HardwareType, AmplitudeStats>,
|
||||
}
|
||||
|
||||
pub enum HardwareType {
|
||||
Esp32S3 { subcarriers: usize, mimo: (u8, u8) },
|
||||
Intel5300 { subcarriers: usize, mimo: (u8, u8) },
|
||||
Atheros { subcarriers: usize, mimo: (u8, u8) },
|
||||
Generic { subcarriers: usize, mimo: (u8, u8) },
|
||||
}
|
||||
|
||||
impl HardwareNormalizer {
|
||||
/// Normalize a raw CSI frame to canonical form:
|
||||
/// 1. Resample subcarriers to canonical count via cubic interpolation
|
||||
/// 2. Z-score normalize amplitude per-frame
|
||||
/// 3. Sanitize phase: remove hardware-specific linear phase offset
|
||||
pub fn normalize(&self, frame: &CsiFrame) -> CanonicalCsiFrame { .. }
|
||||
}
|
||||
```
|
||||
|
||||
The resampling uses `ruvector-solver`'s sparse interpolation (already integrated per ADR-016) to project from any subcarrier count to the canonical 56.
|
||||
|
||||
### 2.5 Virtual Environment Augmentation
|
||||
|
||||
Following DGSense's virtual data generator concept, MERIDIAN augments training data with synthetic domain shifts:
|
||||
|
||||
```rust
|
||||
/// Generates virtual CSI domains by simulating environment variations.
|
||||
pub struct VirtualDomainAugmentor {
|
||||
/// Simulate different room sizes via multipath delay scaling
|
||||
room_scale_range: (f32, f32), // default: (0.5, 2.0)
|
||||
/// Simulate wall material via reflection coefficient perturbation
|
||||
reflection_coeff_range: (f32, f32), // default: (0.3, 0.9)
|
||||
/// Simulate furniture via random scatterer injection
|
||||
n_virtual_scatterers: (usize, usize), // default: (0, 5)
|
||||
/// Simulate hardware differences via subcarrier response shaping
|
||||
hw_response_filters: Vec<SubcarrierResponseFilter>,
|
||||
}
|
||||
|
||||
impl VirtualDomainAugmentor {
|
||||
/// Apply a random virtual domain shift to a CSI batch.
|
||||
/// Each call generates a new "virtual environment" for training diversity.
|
||||
pub fn augment(&self, batch: &CsiBatch, rng: &mut impl Rng) -> CsiBatch { .. }
|
||||
}
|
||||
```
|
||||
|
||||
During training, each mini-batch is augmented with K=3 virtual domain shifts, producing 4x the effective training environments. The domain classifier sees both real and virtual domain labels, improving its ability to force environment-invariant features.
|
||||
|
||||
### 2.6 Few-Shot Rapid Adaptation
|
||||
|
||||
For deployment scenarios where a brief calibration period is available (10-60 seconds of CSI data from the new environment, no pose labels needed):
|
||||
|
||||
```rust
|
||||
/// Rapid adaptation to a new environment using unlabeled CSI data.
|
||||
/// Combines SONA LoRA adapters (ADR-005) with MERIDIAN's domain factorization.
|
||||
pub struct RapidAdaptation {
|
||||
/// Number of unlabeled CSI frames needed for adaptation
|
||||
min_calibration_frames: usize, // default: 200 (10 sec @ 20 Hz)
|
||||
/// LoRA rank for environment-specific adaptation
|
||||
lora_rank: usize, // default: 4
|
||||
/// Self-supervised adaptation loss (AETHER contrastive + entropy min)
|
||||
adaptation_loss: AdaptationLoss,
|
||||
}
|
||||
|
||||
pub enum AdaptationLoss {
|
||||
/// Test-time training with AETHER contrastive loss on unlabeled data
|
||||
ContrastiveTTT { epochs: usize, lr: f32 },
|
||||
/// Entropy minimization on pose confidence outputs
|
||||
EntropyMin { epochs: usize, lr: f32 },
|
||||
/// Combined: contrastive + entropy minimization
|
||||
Combined { epochs: usize, lr: f32, lambda_ent: f32 },
|
||||
}
|
||||
```
|
||||
|
||||
This leverages the existing SONA infrastructure (ADR-005) to generate environment-specific LoRA weights from unlabeled CSI alone, bridging the gap between zero-shot geometry conditioning and full supervised fine-tuning.
|
||||
|
||||
---
|
||||
|
||||
## 3. Comparison: MERIDIAN vs Alternatives
|
||||
|
||||
| Approach | Cross-Layout | Cross-Hardware | Zero-Shot | Few-Shot | Edge-Compatible | Multi-Person |
|
||||
|----------|-------------|----------------|-----------|----------|-----------------|-------------|
|
||||
| **MERIDIAN (this ADR)** | Yes (GRL + geometry FiLM) | Yes (HardwareNormalizer) | Yes (geometry conditioning) | Yes (SONA + contrastive TTT) | Yes (adds ~12K params) | Yes (via ADR-023) |
|
||||
| PerceptAlign (2026) | Yes | No | Partial (needs layout) | No | Unknown (20M params) | No |
|
||||
| AdaPose (2024) | Partial (2 domains) | No | No | Yes (mapping consistency) | Unknown | No |
|
||||
| DGSense (2025) | Yes (virtual aug) | Yes (multi-modality) | Yes | No | No (ResNet backbone) | No |
|
||||
| X-Fi (ICLR 2025) | Yes (foundation model) | Yes (multi-modal) | Yes | Yes (pre-trained) | No (large transformer) | Yes |
|
||||
| AM-FM (2026) | Yes (439-day pretraining) | Yes (20 device types) | Yes | Yes | No (foundation scale) | Unknown |
|
||||
| CAPC (2024) | Partial (transfer learning) | No | No | Yes (SSL fine-tune) | Yes (lightweight) | No |
|
||||
| **Current wifi-densepose** | **No** | **No** | **No** | **Partial (SONA manual)** | **Yes** | **Yes** |
|
||||
|
||||
### MERIDIAN's Differentiators
|
||||
|
||||
1. **Additive, not replacement**: Unlike X-Fi or AM-FM which require new foundation model infrastructure, MERIDIAN adds 4 small modules to the existing ADR-023 pipeline.
|
||||
2. **Edge-compatible**: Total parameter overhead is ~12K (geometry encoder ~8K, domain factorizer ~4K), fitting within the ESP32 budget established in ADR-024.
|
||||
3. **Hardware-agnostic**: First approach to combine cross-layout AND cross-hardware generalization in a single framework, using the existing `ruvector-solver` sparse interpolation.
|
||||
4. **Continuum of adaptation**: Supports zero-shot (geometry only), few-shot (10-sec calibration), and full fine-tuning on the same architecture.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation
|
||||
|
||||
### 4.1 Phase 1 -- Hardware Normalizer (Week 1)
|
||||
|
||||
**Goal**: Canonical CSI representation across ESP32, Intel 5300, and Atheros hardware.
|
||||
|
||||
**Files modified:**
|
||||
- `crates/wifi-densepose-signal/src/hardware_norm.rs` (new)
|
||||
- `crates/wifi-densepose-signal/src/lib.rs` (export new module)
|
||||
- `crates/wifi-densepose-train/src/dataset.rs` (apply normalizer in data pipeline)
|
||||
|
||||
**Dependencies**: `ruvector-solver` (sparse interpolation, already vendored)
|
||||
|
||||
**Acceptance criteria:**
|
||||
- [ ] Resample any subcarrier count to canonical 56 within 50us per frame
|
||||
- [ ] Z-score normalization produces mean=0, std=1 per-frame amplitude
|
||||
- [ ] Phase sanitization removes linear trend (validated against SpotFi output)
|
||||
- [ ] Unit tests with synthetic ESP32 (64 sub) and Intel 5300 (30 sub) frames
|
||||
|
||||
### 4.2 Phase 2 -- Domain Factorizer + GRL (Week 2-3)
|
||||
|
||||
**Goal**: Disentangle pose-relevant and environment-specific features during training.
|
||||
|
||||
**Files modified:**
|
||||
- `crates/wifi-densepose-train/src/domain.rs` (new: DomainFactorizer, GRL, DomainClassifier)
|
||||
- `crates/wifi-densepose-train/src/graph_transformer.rs` (wire factorizer after GNN)
|
||||
- `crates/wifi-densepose-train/src/trainer.rs` (add L_domain to composite loss, GRL annealing)
|
||||
- `crates/wifi-densepose-train/src/dataset.rs` (add domain labels to DataPipeline)
|
||||
|
||||
**Key implementation detail -- Gradient Reversal Layer:**
|
||||
|
||||
```rust
|
||||
/// Gradient Reversal Layer: identity in forward pass, negates gradient in backward.
|
||||
/// Used to train the PoseEncoder to produce domain-invariant features.
|
||||
pub struct GradientReversalLayer {
|
||||
lambda: f32,
|
||||
}
|
||||
|
||||
impl GradientReversalLayer {
|
||||
/// Forward: identity. Backward: multiply gradient by -lambda.
|
||||
/// In our pure-Rust autograd, this is implemented as:
|
||||
/// forward(x) = x
|
||||
/// backward(grad) = -lambda * grad
|
||||
pub fn forward(&self, x: &Tensor) -> Tensor {
|
||||
// Store lambda for backward pass in computation graph
|
||||
x.clone_with_grad_fn(GrlBackward { lambda: self.lambda })
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Acceptance criteria:**
|
||||
- [ ] Domain classifier achieves >90% accuracy on source domains (proves signal exists)
|
||||
- [ ] After GRL training, domain classifier accuracy drops to near-chance (proves disentanglement)
|
||||
- [ ] Pose accuracy on source domains degrades <5% vs non-adversarial baseline
|
||||
- [ ] Cross-domain pose accuracy improves >20% on held-out environment
|
||||
|
||||
### 4.3 Phase 3 -- Geometry Encoder + FiLM Conditioning (Week 3-4)
|
||||
|
||||
**Goal**: Enable zero-shot deployment given AP positions.
|
||||
|
||||
**Files modified:**
|
||||
- `crates/wifi-densepose-train/src/geometry.rs` (new: GeometryEncoder, FourierPositionalEncoding, DeepSets, FiLM)
|
||||
- `crates/wifi-densepose-train/src/graph_transformer.rs` (inject FiLM conditioning before xyz_head)
|
||||
- `crates/wifi-densepose-train/src/config.rs` (add geometry fields to TrainConfig)
|
||||
|
||||
**Acceptance criteria:**
|
||||
- [ ] FourierPositionalEncoding produces 64-dim vectors from 3D coordinates
|
||||
- [ ] DeepSets is permutation-invariant (same output regardless of AP ordering)
|
||||
- [ ] FiLM conditioning reduces cross-layout MPJPE by >30% vs unconditioned baseline
|
||||
- [ ] Inference overhead <100us per frame (geometry encoding is amortized per-session)
|
||||
|
||||
### 4.4 Phase 4 -- Virtual Domain Augmentation (Week 4-5)
|
||||
|
||||
**Goal**: Synthetic environment diversity to improve generalization.
|
||||
|
||||
**Files modified:**
|
||||
- `crates/wifi-densepose-train/src/virtual_aug.rs` (new: VirtualDomainAugmentor)
|
||||
- `crates/wifi-densepose-train/src/trainer.rs` (integrate augmentor into training loop)
|
||||
- `crates/wifi-densepose-signal/src/fresnel.rs` (reuse Fresnel zone model for scatterer simulation)
|
||||
|
||||
**Dependencies**: `ruvector-attn-mincut` (attention-weighted scatterer placement)
|
||||
|
||||
**Acceptance criteria:**
|
||||
- [ ] Generate K=3 virtual domains per batch with <1ms overhead
|
||||
- [ ] Virtual domains produce measurably different CSI statistics (KL divergence >0.1)
|
||||
- [ ] Training with virtual augmentation improves unseen-environment accuracy by >15%
|
||||
- [ ] No regression on seen-environment accuracy (within 2%)
|
||||
|
||||
### 4.5 Phase 5 -- Few-Shot Rapid Adaptation (Week 5-6)
|
||||
|
||||
**Goal**: 10-second calibration enables environment-specific fine-tuning without labels.
|
||||
|
||||
**Files modified:**
|
||||
- `crates/wifi-densepose-train/src/rapid_adapt.rs` (new: RapidAdaptation)
|
||||
- `crates/wifi-densepose-train/src/sona.rs` (extend SonaProfile with MERIDIAN fields)
|
||||
- `crates/wifi-densepose-sensing-server/src/main.rs` (add `--calibrate` CLI flag)
|
||||
|
||||
**Acceptance criteria:**
|
||||
- [ ] 200-frame (10 sec) calibration produces usable LoRA adapter
|
||||
- [ ] Adapted model MPJPE within 15% of fully-supervised in-domain baseline
|
||||
- [ ] Calibration completes in <5 seconds on x86 (including contrastive TTT)
|
||||
- [ ] Adapted LoRA weights serializable to RVF container (ADR-023 Segment type)
|
||||
|
||||
### 4.6 Phase 6 -- Cross-Domain Evaluation Protocol (Week 6-7)
|
||||
|
||||
**Goal**: Rigorous multi-domain evaluation using MM-Fi's scene/subject splits.
|
||||
|
||||
**Files modified:**
|
||||
- `crates/wifi-densepose-train/src/eval.rs` (new: CrossDomainEvaluator)
|
||||
- `crates/wifi-densepose-train/src/dataset.rs` (add domain-split loading for MM-Fi)
|
||||
|
||||
**Evaluation protocol (following PerceptAlign):**
|
||||
|
||||
| Metric | Description |
|
||||
|--------|-------------|
|
||||
| **In-domain MPJPE** | Mean Per Joint Position Error on training environment |
|
||||
| **Cross-domain MPJPE** | MPJPE on held-out environment (zero-shot) |
|
||||
| **Few-shot MPJPE** | MPJPE after 10-sec calibration in target environment |
|
||||
| **Cross-hardware MPJPE** | MPJPE when trained on one hardware, tested on another |
|
||||
| **Domain gap ratio** | cross-domain / in-domain MPJPE (lower = better; target <1.5) |
|
||||
| **Adaptation speedup** | Labeled samples saved vs training from scratch (target >5x) |
|
||||
|
||||
### 4.7 Phase 7 -- RVF Container + Deployment (Week 7-8)
|
||||
|
||||
**Goal**: Package MERIDIAN-enhanced models for edge deployment.
|
||||
|
||||
**Files modified:**
|
||||
- `crates/wifi-densepose-train/src/rvf_container.rs` (add GEOM and DOMAIN segment types)
|
||||
- `crates/wifi-densepose-sensing-server/src/inference.rs` (load geometry + domain weights)
|
||||
- `crates/wifi-densepose-sensing-server/src/main.rs` (add `--ap-positions` CLI flag)
|
||||
|
||||
**New RVF segments:**
|
||||
|
||||
| Segment | Type ID | Contents | Size |
|
||||
|---------|---------|----------|------|
|
||||
| `GEOM` | `0x47454F4D` | GeometryEncoder weights + FiLM layers | ~4 KB |
|
||||
| `DOMAIN` | `0x444F4D4E` | DomainFactorizer weights (PoseEncoder only; EnvEncoder and GRL discarded) | ~8 KB |
|
||||
| `HWSTATS` | `0x48575354` | Per-hardware amplitude statistics for HardwareNormalizer | ~1 KB |
|
||||
|
||||
**CLI usage:**
|
||||
|
||||
```bash
|
||||
# Train with MERIDIAN domain generalization
|
||||
cargo run -p wifi-densepose-sensing-server -- \
|
||||
--train --dataset data/mmfi/ --epochs 100 \
|
||||
--meridian --n-virtual-domains 3 \
|
||||
--save-rvf model-meridian.rvf
|
||||
|
||||
# Deploy with geometry conditioning (zero-shot)
|
||||
cargo run -p wifi-densepose-sensing-server -- \
|
||||
--model model-meridian.rvf \
|
||||
--ap-positions "0,0,2.5;3.5,0,2.5;1.75,4,2.5"
|
||||
|
||||
# Calibrate in new environment (few-shot, 10 seconds)
|
||||
cargo run -p wifi-densepose-sensing-server -- \
|
||||
--model model-meridian.rvf --calibrate --calibrate-duration 10
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Consequences
|
||||
|
||||
### 5.1 Positive
|
||||
|
||||
- **Deploy once, work everywhere**: A single MERIDIAN-trained model generalizes across rooms, buildings, and hardware without per-environment retraining
|
||||
- **Reduced deployment cost**: Zero-shot mode requires only AP position input; few-shot mode needs 10 seconds of ambient WiFi data
|
||||
- **AETHER synergy**: Domain-invariant embeddings (ADR-024) become environment-agnostic fingerprints, enabling cross-building room identification
|
||||
- **Hardware freedom**: HardwareNormalizer unblocks mixed-fleet deployments (ESP32 in some rooms, Intel 5300 in others)
|
||||
- **Competitive positioning**: No existing open-source WiFi pose system offers cross-environment generalization; MERIDIAN would be the first
|
||||
|
||||
### 5.2 Negative
|
||||
|
||||
- **Training complexity**: Multi-domain training requires CSI data from multiple environments. MM-Fi provides multiple scenes but PerceptAlign's 7-layout dataset is not yet public.
|
||||
- **Hyperparameter sensitivity**: GRL lambda annealing schedule and adversarial balance require careful tuning; unstable training is possible if adversarial signal is too strong early.
|
||||
- **Geometry input requirement**: Zero-shot mode requires users to input AP positions, which may not always be precisely known. Degradation under inaccurate geometry input needs characterization.
|
||||
- **Parameter overhead**: +12K parameters increases total model from 55K to 67K (22% increase), still well within ESP32 budget but notable.
|
||||
|
||||
### 5.3 Risks and Mitigations
|
||||
|
||||
| Risk | Probability | Impact | Mitigation |
|
||||
|------|-------------|--------|------------|
|
||||
| GRL training instability | Medium | Training diverges | Lambda annealing schedule; gradient clipping at 1.0; fallback to non-adversarial training |
|
||||
| Virtual augmentation unrealistic | Low | No generalization improvement | Validate augmented CSI against real cross-domain data distributions |
|
||||
| Geometry encoder overfits to training layouts | Medium | Zero-shot fails on novel geometries | Augment geometry inputs during training (jitter AP positions by +/-0.5m) |
|
||||
| MM-Fi scenes insufficient diversity | High | Limited evaluation validity | Supplement with synthetic data; target PerceptAlign dataset when released |
|
||||
|
||||
---
|
||||
|
||||
## 6. Relationship to Proposed ADRs (Gap Closure)
|
||||
|
||||
ADRs 002-011 were proposed during the initial architecture phase. MERIDIAN directly addresses, subsumes, or enables several of these gaps. This section maps each proposed ADR to its current status and how ADR-027 interacts with it.
|
||||
|
||||
### 6.1 Directly Addressed by MERIDIAN
|
||||
|
||||
| Proposed ADR | Gap | How MERIDIAN Closes It |
|
||||
|-------------|-----|----------------------|
|
||||
| **ADR-004**: HNSW Vector Search Fingerprinting | CSI fingerprints are environment-specific — a fingerprint learned in Room A is useless in Room B | MERIDIAN's `DomainFactorizer` produces **environment-disentangled embeddings** (`h_pose`). When fed into ADR-024's `FingerprintIndex`, these embeddings match across rooms because environment information has been factored out. The `h_env` path captures room identity separately, enabling both cross-room matching AND room identification in a single model. |
|
||||
| **ADR-005**: SONA Self-Learning for Pose Estimation | SONA LoRA adapters must be manually created per environment with labeled data | MERIDIAN Phase 5 (`RapidAdaptation`) extends SONA with **unsupervised adapter generation**: 10 seconds of unlabeled WiFi data + contrastive test-time training automatically produces a per-room LoRA adapter. No labels, no manual intervention. The existing `SonaProfile` in `sona.rs` gains a `meridian_calibration` field for storing adaptation state. |
|
||||
| **ADR-006**: GNN-Enhanced CSI Pattern Recognition | GNN treats each environment's patterns independently; no cross-environment transfer | MERIDIAN's domain-adversarial training regularizes the GCN layers (ADR-023's `GnnStack`) to learn **structure-preserving, environment-invariant** graph features. The gradient reversal layer forces the GCN to shed room-specific multipath patterns while retaining body-pose-relevant spatial relationships between keypoints. |
|
||||
|
||||
### 6.2 Superseded (Already Implemented)
|
||||
|
||||
| Proposed ADR | Original Vision | Current Status |
|
||||
|-------------|----------------|---------------|
|
||||
| **ADR-002**: RuVector RVF Integration Strategy | Integrate RuVector crates into the WiFi-DensePose pipeline | **Fully implemented** by ADR-016 (training pipeline, 5 crates) and ADR-017 (signal + MAT, 7 integration points). The `wifi-densepose-ruvector` crate is published on crates.io. No further action needed. |
|
||||
|
||||
### 6.3 Enabled by MERIDIAN (Future Work)
|
||||
|
||||
These ADRs remain independent tracks but MERIDIAN creates enabling infrastructure for them:
|
||||
|
||||
| Proposed ADR | Gap | How MERIDIAN Enables It |
|
||||
|-------------|-----|------------------------|
|
||||
| **ADR-003**: RVF Cognitive Containers | CSI pipeline stages produce ephemeral data; no persistent cognitive state across sessions | MERIDIAN's RVF container extensions (Phase 7: `GEOM`, `DOMAIN`, `HWSTATS` segments) establish the pattern for **environment-aware model packaging**. A cognitive container could store per-room adaptation history, geometry profiles, and domain statistics — building on MERIDIAN's segment format. The `h_env` embeddings are natural candidates for persistent environment memory. |
|
||||
| **ADR-008**: Distributed Consensus for Multi-AP | Multiple APs need coordinated sensing; no agreement protocol for conflicting observations | MERIDIAN's `GeometryEncoder` already models variable-count AP positions via permutation-invariant `DeepSets`. This provides the **geometric foundation** for multi-AP fusion: each AP's CSI is geometry-conditioned independently, then fused. A consensus layer (Raft or BFT) would sit above MERIDIAN to reconcile conflicting pose estimates from different AP vantage points. The `HardwareNormalizer` ensures mixed hardware (ESP32 + Intel 5300 across APs) produces comparable features. |
|
||||
| **ADR-009**: RVF WASM Runtime for Edge | Self-contained WASM model execution without server dependency | MERIDIAN's +12K parameter overhead (67K total) remains within the WASM size budget. The `HardwareNormalizer` is critical for WASM deployment: browser-based inference must handle whatever CSI format the connected hardware provides. WASM builds should include the geometry conditioning path so users can specify AP layout in the browser UI. |
|
||||
|
||||
### 6.4 Independent Tracks (Not Addressed by MERIDIAN)
|
||||
|
||||
These ADRs address orthogonal concerns and should be pursued separately:
|
||||
|
||||
| Proposed ADR | Gap | Recommendation |
|
||||
|-------------|-----|----------------|
|
||||
| **ADR-007**: Post-Quantum Cryptography | WiFi sensing data reveals presence, health, and activity — quantum computers could break current encryption of sensing streams | **Pursue independently.** MERIDIAN does not address data-in-transit security. PQC should be applied to WebSocket streams (`/ws/sensing`, `/ws/mat/stream`) and RVF model containers (replace Ed25519 signing with ML-DSA/Dilithium). Priority: medium — no imminent quantum threat, but healthcare deployments may require PQC compliance for long-term data retention. |
|
||||
| **ADR-010**: Witness Chains for Audit Trail | Disaster triage decisions (ADR-001) need tamper-proof audit trails for legal/regulatory compliance | **Pursue independently.** MERIDIAN's domain adaptation improves triage accuracy in unfamiliar environments (rubble, collapsed buildings), which reduces the need for audit trail corrections. But the audit trail itself — hash chains, Merkle proofs, timestamped triage events — is a separate integrity concern. Priority: high for disaster response deployments. |
|
||||
| **ADR-011**: Python Proof-of-Reality (URGENT) | Python v1 contains mock/placeholder code that undermines credibility; `verify.py` exists but mock paths remain | **Pursue independently.** This is a Python v1 code quality issue, not an ML/architecture concern. The Rust port (v2+) has no mock code — all 542+ tests run against real algorithm implementations. Recommendation: either complete the mock elimination in Python v1 or formally deprecate Python v1 in favor of the Rust stack. Priority: high for credibility. |
|
||||
|
||||
### 6.5 Gap Closure Summary
|
||||
|
||||
```
|
||||
Proposed ADRs (002-011) Status After ADR-027
|
||||
───────────────────────── ─────────────────────
|
||||
ADR-002 RVF Integration ──→ ✅ Superseded (ADR-016/017 implemented)
|
||||
ADR-003 Cognitive Containers ─→ 🔜 Enabled (MERIDIAN RVF segments provide pattern)
|
||||
ADR-004 HNSW Fingerprinting ──→ ✅ Addressed (domain-disentangled embeddings)
|
||||
ADR-005 SONA Self-Learning ──→ ✅ Addressed (unsupervised rapid adaptation)
|
||||
ADR-006 GNN Patterns ──→ ✅ Addressed (adversarial GCN regularization)
|
||||
ADR-007 Post-Quantum Crypto ──→ ⏳ Independent (pursue separately, medium priority)
|
||||
ADR-008 Distributed Consensus → 🔜 Enabled (GeometryEncoder + HardwareNormalizer)
|
||||
ADR-009 WASM Runtime ──→ 🔜 Enabled (67K model fits WASM budget)
|
||||
ADR-010 Witness Chains ──→ ⏳ Independent (pursue separately, high priority)
|
||||
ADR-011 Proof-of-Reality ──→ ⏳ Independent (Python v1 issue, high priority)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. References
|
||||
|
||||
1. Chen, L., et al. (2026). "Breaking Coordinate Overfitting: Geometry-Aware WiFi Sensing for Cross-Layout 3D Pose Estimation." arXiv:2601.12252. https://arxiv.org/abs/2601.12252
|
||||
2. Zhou, Y., et al. (2024). "AdaPose: Towards Cross-Site Device-Free Human Pose Estimation with Commodity WiFi." IEEE Internet of Things Journal. arXiv:2309.16964. https://arxiv.org/abs/2309.16964
|
||||
3. Yan, K., et al. (2024). "Person-in-WiFi 3D: End-to-End Multi-Person 3D Pose Estimation with Wi-Fi." CVPR 2024, pp. 969-978. https://openaccess.thecvf.com/content/CVPR2024/html/Yan_Person-in-WiFi_3D_End-to-End_Multi-Person_3D_Pose_Estimation_with_Wi-Fi_CVPR_2024_paper.html
|
||||
4. Zhou, R., et al. (2025). "DGSense: A Domain Generalization Framework for Wireless Sensing." arXiv:2502.08155. https://arxiv.org/abs/2502.08155
|
||||
5. CAPC (2024). "Context-Aware Predictive Coding: A Representation Learning Framework for WiFi Sensing." IEEE OJCOMS, Vol. 5, pp. 6119-6134. arXiv:2410.01825. https://arxiv.org/abs/2410.01825
|
||||
6. Chen, X. & Yang, J. (2025). "X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing." ICLR 2025. arXiv:2410.10167. https://arxiv.org/abs/2410.10167
|
||||
7. AM-FM (2026). "AM-FM: A Foundation Model for Ambient Intelligence Through WiFi." arXiv:2602.11200. https://arxiv.org/abs/2602.11200
|
||||
8. Ramesh, S. et al. (2025). "LatentCSI: High-resolution efficient image generation from WiFi CSI using a pretrained latent diffusion model." arXiv:2506.10605. https://arxiv.org/abs/2506.10605
|
||||
9. Ganin, Y. et al. (2016). "Domain-Adversarial Training of Neural Networks." JMLR 17(59):1-35. https://jmlr.org/papers/v17/15-239.html
|
||||
10. Perez, E. et al. (2018). "FiLM: Visual Reasoning with a General Conditioning Layer." AAAI 2018. arXiv:1709.07871. https://arxiv.org/abs/1709.07871
|
||||
369
docs/adr/ADR-028-ruview-sensing-first-rf-mode.md
Normal file
369
docs/adr/ADR-028-ruview-sensing-first-rf-mode.md
Normal file
@@ -0,0 +1,369 @@
|
||||
# ADR-028: Project RuView -- Sensing-First RF Mode for Multistatic Fidelity Enhancement
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Status** | Proposed |
|
||||
| **Date** | 2026-03-02 |
|
||||
| **Deciders** | ruv |
|
||||
| **Codename** | **RuView** -- RuVector Viewpoint-Integrated Enhancement |
|
||||
| **Relates to** | ADR-012 (ESP32 Mesh), ADR-014 (SOTA Signal), ADR-016 (RuVector Integration), ADR-017 (RuVector Signal+MAT), ADR-021 (Vital Signs), ADR-024 (AETHER Embeddings), ADR-027 (MERIDIAN Cross-Environment) |
|
||||
|
||||
---
|
||||
|
||||
## 1. Context
|
||||
|
||||
### 1.1 The Single-Viewpoint Fidelity Ceiling
|
||||
|
||||
Current WiFi DensePose operates with a single transmitter-receiver pair (or single node receiving). This creates three fundamental limitations:
|
||||
|
||||
- **Body self-occlusion**: Limbs behind the torso are invisible to a single viewpoint.
|
||||
- **Depth ambiguity**: Motion along the RF propagation axis (toward/away from receiver) produces minimal phase change.
|
||||
- **Multi-person confusion**: Two people at similar range but different angles create overlapping CSI signatures.
|
||||
|
||||
The ESP32 mesh (ADR-012) partially addresses this via feature-level fusion across 3-6 nodes, but feature-level fusion cannot learn optimal fusion weights -- it uses hand-crafted aggregation (max, mean, coherent sum).
|
||||
|
||||
### 1.2 Three Fidelity Levers
|
||||
|
||||
1. **Bandwidth**: More bandwidth produces better multipath separability. Currently limited to 20 MHz (ESP32 HT20). Wider channels (80/160 MHz) are available on commodity 802.11ac/ax APs.
|
||||
2. **Carrier frequency**: Higher frequency produces more phase sensitivity. 2.4 GHz sees macro-motion; 5 GHz sees micro-motion; 60 GHz sees vital signs.
|
||||
3. **Viewpoints**: More viewpoints from different angles reduces geometric ambiguity. This is the lever RuView pulls.
|
||||
|
||||
### 1.3 Why "Sensing-First RF Mode"
|
||||
|
||||
RuView is NOT a new WiFi standard. It is a sensing-first protocol that rides on existing silicon, bands, and regulations. The key insight: instead of upgrading the RF hardware, upgrade the observability by coordinating multiple commodity receivers.
|
||||
|
||||
### 1.4 What Already Exists
|
||||
|
||||
| Component | ADR | Current State |
|
||||
|-----------|-----|---------------|
|
||||
| ESP32 mesh with feature-level fusion | ADR-012 | Implemented (firmware + aggregator) |
|
||||
| SOTA signal processing (Hampel, Fresnel, BVP, spectrogram) | ADR-014 | Implemented |
|
||||
| RuVector training pipeline (5 crates) | ADR-016 | Complete |
|
||||
| RuVector signal + MAT integration (7 points) | ADR-017 | Accepted |
|
||||
| Vital sign detection pipeline | ADR-021 | Partially implemented |
|
||||
| AETHER contrastive embeddings | ADR-024 | Proposed |
|
||||
| MERIDIAN cross-environment generalization | ADR-027 | Proposed |
|
||||
|
||||
RuView fills the gap: **cross-viewpoint embedding fusion** using learned attention weights.
|
||||
|
||||
---
|
||||
|
||||
## 2. Decision
|
||||
|
||||
Introduce RuView as a cross-viewpoint embedding fusion layer that operates on top of AETHER per-viewpoint embeddings. RuView adds a new bounded context (ViewpointFusion) and extends three existing crates.
|
||||
|
||||
### 2.1 Core Architecture
|
||||
|
||||
```
|
||||
+-----------------------------------------------------------------+
|
||||
| RuView Multistatic Pipeline |
|
||||
+-----------------------------------------------------------------+
|
||||
| |
|
||||
| +----------+ +----------+ +----------+ +----------+ |
|
||||
| | Node 1 | | Node 2 | | Node 3 | | Node N | |
|
||||
| | ESP32-S3 | | ESP32-S3 | | ESP32-S3 | | ESP32-S3 | |
|
||||
| | | | | | | | | |
|
||||
| | CSI Rx | | CSI Rx | | CSI Rx | | CSI Rx | |
|
||||
| +----+-----+ +----+-----+ +----+-----+ +----+-----+ |
|
||||
| | | | | |
|
||||
| v v v v |
|
||||
| +--------------------------------------------------------+ |
|
||||
| | Per-Viewpoint Signal Processing | |
|
||||
| | Phase sanitize -> Hampel -> BVP -> Subcarrier select | |
|
||||
| | (ADR-014, unchanged per viewpoint) | |
|
||||
| +----------------------------+---------------------------+ |
|
||||
| | |
|
||||
| v |
|
||||
| +--------------------------------------------------------+ |
|
||||
| | Per-Viewpoint AETHER Embedding | |
|
||||
| | CsiToPoseTransformer -> 128-d contrastive embedding | |
|
||||
| | (ADR-024, one per viewpoint) | |
|
||||
| +----------------------------+---------------------------+ |
|
||||
| | |
|
||||
| [emb_1, emb_2, ..., emb_N] |
|
||||
| | |
|
||||
| v |
|
||||
| +--------------------------------------------------------+ |
|
||||
| | * RuView Cross-Viewpoint Fusion * | |
|
||||
| | | |
|
||||
| | Q = W_q * X, K = W_k * X, V = W_v * X | |
|
||||
| | A = softmax((QK^T + G_bias) / sqrt(d)) | |
|
||||
| | fused = A * V | |
|
||||
| | | |
|
||||
| | G_bias: geometric bias from viewpoint pair geometry | |
|
||||
| | (ruvector-attention: ScaledDotProductAttention) | |
|
||||
| +----------------------------+---------------------------+ |
|
||||
| | |
|
||||
| fused_embedding |
|
||||
| | |
|
||||
| v |
|
||||
| +--------------------------------------------------------+ |
|
||||
| | DensePose Regression Head | |
|
||||
| | Keypoint head: [B,17,H,W] | |
|
||||
| | Part/UV head: [B,25,H,W] + [B,48,H,W] | |
|
||||
| +--------------------------------------------------------+ |
|
||||
+-----------------------------------------------------------------+
|
||||
```
|
||||
|
||||
### 2.2 TDM Sensing Protocol
|
||||
|
||||
- Coordinator (aggregator) broadcasts sync beacon at start of each cycle.
|
||||
- Each node transmits in assigned time slot; all others receive.
|
||||
- 6 nodes x 1.4 ms/slot = 8.4 ms cycle -> ~119 Hz aggregate, ~20 Hz per bistatic pair.
|
||||
- Clock drift handled at feature level (no cross-node phase alignment).
|
||||
|
||||
### 2.3 Geometric Bias Matrix
|
||||
|
||||
The geometric bias `G_bias` encodes the spatial relationship between viewpoint pairs:
|
||||
|
||||
```
|
||||
G_bias[i,j] = w_angle * cos(theta_ij) + w_dist * exp(-d_ij / d_ref)
|
||||
```
|
||||
|
||||
where:
|
||||
|
||||
- `theta_ij` = angle between viewpoint i and viewpoint j (from room center)
|
||||
- `d_ij` = baseline distance between node i and node j
|
||||
- `w_angle`, `w_dist` = learnable weights
|
||||
- `d_ref` = reference distance (room diagonal / 2)
|
||||
|
||||
This allows the attention mechanism to learn that widely-separated, orthogonal viewpoints are more complementary than clustered ones.
|
||||
|
||||
### 2.4 Coherence-Gated Environment Updates
|
||||
|
||||
```rust
|
||||
/// Only update environment model when phase coherence exceeds threshold.
|
||||
pub fn coherence_gate(
|
||||
phase_diffs: &[f32], // delta-phi over T recent frames
|
||||
threshold: f32, // typically 0.7
|
||||
) -> bool {
|
||||
// Complex mean of unit phasors
|
||||
let (sum_cos, sum_sin) = phase_diffs.iter()
|
||||
.fold((0.0f32, 0.0f32), |(c, s), &dp| {
|
||||
(c + dp.cos(), s + dp.sin())
|
||||
});
|
||||
let n = phase_diffs.len() as f32;
|
||||
let coherence = ((sum_cos / n).powi(2) + (sum_sin / n).powi(2)).sqrt();
|
||||
coherence > threshold
|
||||
}
|
||||
```
|
||||
|
||||
### 2.5 Two Implementation Paths
|
||||
|
||||
| Path | Hardware | Bandwidth | Per-Viewpoint Rate | Target Tier |
|
||||
|------|----------|-----------|-------------------|-------------|
|
||||
| **ESP32 Multistatic** | 6x ESP32-S3 ($84) | 20 MHz (HT20) | 20 Hz | Silver |
|
||||
| **Cognitum + RF** | Cognitum v1 + LimeSDR | 20-160 MHz | 20-100 Hz | Gold |
|
||||
|
||||
ESP32 path: commodity, achievable today, targets Silver tier (tracking + pose quality).
|
||||
Cognitum path: higher fidelity, targets Gold tier (tracking + pose + vitals).
|
||||
|
||||
---
|
||||
|
||||
## 3. DDD Design
|
||||
|
||||
### 3.1 New Bounded Context: ViewpointFusion
|
||||
|
||||
**Aggregate Root: `MultistaticArray`**
|
||||
|
||||
```rust
|
||||
pub struct MultistaticArray {
|
||||
/// Unique array deployment ID
|
||||
id: ArrayId,
|
||||
/// Viewpoint geometry (node positions, orientations)
|
||||
geometry: ArrayGeometry,
|
||||
/// TDM schedule (slot assignments, cycle period)
|
||||
schedule: TdmSchedule,
|
||||
/// Active viewpoint embeddings (latest per node)
|
||||
viewpoints: Vec<ViewpointEmbedding>,
|
||||
/// Fused output embedding
|
||||
fused: Option<FusedEmbedding>,
|
||||
/// Coherence gate state
|
||||
coherence_state: CoherenceState,
|
||||
}
|
||||
```
|
||||
|
||||
**Entity: `ViewpointEmbedding`**
|
||||
|
||||
```rust
|
||||
pub struct ViewpointEmbedding {
|
||||
/// Source node ID
|
||||
node_id: NodeId,
|
||||
/// AETHER embedding vector (128-d)
|
||||
embedding: Vec<f32>,
|
||||
/// Geometric metadata
|
||||
azimuth: f32, // radians from array center
|
||||
elevation: f32, // radians
|
||||
baseline: f32, // meters from centroid
|
||||
/// Capture timestamp
|
||||
timestamp: Instant,
|
||||
/// Signal quality
|
||||
snr_db: f32,
|
||||
}
|
||||
```
|
||||
|
||||
**Value Object: `GeometricDiversityIndex`**
|
||||
|
||||
```rust
|
||||
pub struct GeometricDiversityIndex {
|
||||
/// GDI = (1/N) sum min_{j!=i} |theta_i - theta_j|
|
||||
value: f32,
|
||||
/// Effective independent viewpoints (after correlation discount)
|
||||
n_effective: f32,
|
||||
/// Worst viewpoint pair (most redundant)
|
||||
worst_pair: (NodeId, NodeId),
|
||||
}
|
||||
```
|
||||
|
||||
**Domain Events:**
|
||||
|
||||
```rust
|
||||
pub enum ViewpointFusionEvent {
|
||||
ViewpointCaptured { node_id: NodeId, timestamp: Instant, snr_db: f32 },
|
||||
TdmCycleCompleted { cycle_id: u64, viewpoints_received: usize },
|
||||
FusionCompleted { fused_embedding: Vec<f32>, gdi: f32 },
|
||||
CoherenceGateTriggered { coherence: f32, accepted: bool },
|
||||
GeometryUpdated { new_gdi: f32, n_effective: f32 },
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 Extended Bounded Contexts
|
||||
|
||||
**Signal (wifi-densepose-signal):**
|
||||
- New service: `CrossViewpointSubcarrierSelection`
|
||||
- Consensus sensitive subcarrier set across all viewpoints via ruvector-mincut.
|
||||
- Input: per-viewpoint sensitivity scores. Output: globally-sensitive + locally-sensitive partition.
|
||||
|
||||
**Hardware (wifi-densepose-hardware):**
|
||||
- New protocol: `TdmSensingProtocol`
|
||||
- Coordinator logic: beacon generation, slot scheduling, clock drift compensation.
|
||||
- Event: `TdmSlotCompleted { node_id, slot_index, capture_quality }`
|
||||
|
||||
**Training (wifi-densepose-train):**
|
||||
- New module: `ruview_metrics.rs`
|
||||
- Three-metric acceptance test: PCK/OKS (joint error), MOTA (multi-person separation), vital sign accuracy.
|
||||
- Tiered pass/fail: Bronze/Silver/Gold.
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Plan (File-Level)
|
||||
|
||||
### 4.1 Phase 1: ViewpointFusion Core (New Files)
|
||||
|
||||
| File | Purpose | RuVector Crate |
|
||||
|------|---------|---------------|
|
||||
| `crates/wifi-densepose-ruvector/src/viewpoint/mod.rs` | Module root, re-exports | -- |
|
||||
| `crates/wifi-densepose-ruvector/src/viewpoint/attention.rs` | Cross-viewpoint scaled dot-product attention with geometric bias | ruvector-attention |
|
||||
| `crates/wifi-densepose-ruvector/src/viewpoint/geometry.rs` | GeometricDiversityIndex, Cramer-Rao bound estimation | ruvector-solver |
|
||||
| `crates/wifi-densepose-ruvector/src/viewpoint/coherence.rs` | Coherence gating for environment stability | -- (pure math) |
|
||||
| `crates/wifi-densepose-ruvector/src/viewpoint/fusion.rs` | MultistaticArray aggregate, orchestrates fusion pipeline | ruvector-attention + ruvector-attn-mincut |
|
||||
|
||||
### 4.2 Phase 2: Signal Processing Extension
|
||||
|
||||
| File | Purpose | RuVector Crate |
|
||||
|------|---------|---------------|
|
||||
| `crates/wifi-densepose-signal/src/cross_viewpoint.rs` | Cross-viewpoint subcarrier consensus via min-cut | ruvector-mincut |
|
||||
|
||||
### 4.3 Phase 3: Hardware Protocol Extension
|
||||
|
||||
| File | Purpose | RuVector Crate |
|
||||
|------|---------|---------------|
|
||||
| `crates/wifi-densepose-hardware/src/esp32/tdm.rs` | TDM sensing protocol coordinator | -- (protocol logic) |
|
||||
|
||||
### 4.4 Phase 4: Training and Metrics
|
||||
|
||||
| File | Purpose | RuVector Crate |
|
||||
|------|---------|---------------|
|
||||
| `crates/wifi-densepose-train/src/ruview_metrics.rs` | Three-metric acceptance test (PCK/OKS, MOTA, vital sign accuracy) | ruvector-mincut (person matching) |
|
||||
|
||||
---
|
||||
|
||||
## 5. Three-Metric Acceptance Test
|
||||
|
||||
### 5.1 Metric 1: Joint Error (PCK / OKS)
|
||||
|
||||
| Criterion | Threshold |
|
||||
|-----------|-----------|
|
||||
| PCK@0.2 (all 17 keypoints) | >= 0.70 |
|
||||
| PCK@0.2 (torso: shoulders + hips) | >= 0.80 |
|
||||
| Mean OKS | >= 0.50 |
|
||||
| Torso jitter RMS (10s window) | < 3 cm |
|
||||
| Per-keypoint max error (95th percentile) | < 15 cm |
|
||||
|
||||
### 5.2 Metric 2: Multi-Person Separation
|
||||
|
||||
| Criterion | Threshold |
|
||||
|-----------|-----------|
|
||||
| Subjects | 2 |
|
||||
| Capture rate | 20 Hz |
|
||||
| Track duration | 10 minutes |
|
||||
| Identity swaps (MOTA ID-switch) | 0 |
|
||||
| Track fragmentation ratio | < 0.05 |
|
||||
| False track creation | 0/min |
|
||||
|
||||
### 5.3 Metric 3: Vital Sign Sensitivity
|
||||
|
||||
| Criterion | Threshold |
|
||||
|-----------|-----------|
|
||||
| Breathing detection (6-30 BPM) | +/- 2 BPM |
|
||||
| Breathing band SNR (0.1-0.5 Hz) | >= 6 dB |
|
||||
| Heartbeat detection (40-120 BPM) | +/- 5 BPM (aspirational) |
|
||||
| Heartbeat band SNR (0.8-2.0 Hz) | >= 3 dB (aspirational) |
|
||||
| Micro-motion resolution | 1 mm at 3m |
|
||||
|
||||
### 5.4 Tiered Pass/Fail
|
||||
|
||||
| Tier | Requirements | Deployment Gate |
|
||||
|------|-------------|-----------------|
|
||||
| Bronze | Metric 2 | Prototype demo |
|
||||
| Silver | Metrics 1 + 2 | Production candidate |
|
||||
| Gold | All three | Full deployment |
|
||||
|
||||
---
|
||||
|
||||
## 6. Consequences
|
||||
|
||||
### 6.1 Positive
|
||||
|
||||
- **Fundamental geometric improvement**: Viewpoint diversity reduces body self-occlusion and depth ambiguity -- these are physics, not model, limitations.
|
||||
- **Uses existing silicon**: ESP32-S3, commodity WiFi, no custom RF hardware required for Silver tier.
|
||||
- **Learned fusion weights**: Embedding-level fusion (Tier 3) outperforms hand-crafted feature-level fusion (Tier 2).
|
||||
- **Composes with existing ADRs**: AETHER (per-viewpoint), MERIDIAN (cross-environment), and RuView (cross-viewpoint) are orthogonal -- they compose freely.
|
||||
- **IEEE 802.11bf aligned**: TDM protocol maps to 802.11bf sensing sessions, enabling future migration to standard-compliant APs.
|
||||
- **Commodity price point**: $84 for 6-node Silver-tier deployment.
|
||||
|
||||
### 6.2 Negative
|
||||
|
||||
- **TDM rate reduction**: N viewpoints leads to per-viewpoint rate divided by N. With 6 nodes at 120 Hz aggregate, each viewpoint sees 20 Hz.
|
||||
- **More complex aggregator**: Embedding fusion + geometric bias learning adds ~25K parameters on top of per-viewpoint AETHER model.
|
||||
- **Placement planning required**: Geometric Diversity Index optimization requires intentional node placement (not random scatter).
|
||||
- **Clock drift limits TDM precision**: ESP32 crystal drift (20-50 ppm) limits slot precision to ~1 ms, which is sufficient for feature-level fusion but not signal-level coherent combining.
|
||||
- **Training data**: Cross-viewpoint training requires multi-receiver CSI captures, which are not available in existing public datasets (MM-Fi, Wi-Pose).
|
||||
|
||||
### 6.3 Interaction with Other ADRs
|
||||
|
||||
| ADR | Interaction |
|
||||
|-----|------------|
|
||||
| ADR-012 (ESP32 Mesh) | RuView extends the aggregator from feature-level to embedding-level fusion; TDM protocol replaces simple UDP collection |
|
||||
| ADR-014 (SOTA Signal) | Per-viewpoint signal processing is unchanged; cross-viewpoint subcarrier consensus is new |
|
||||
| ADR-016/017 (RuVector) | All 5 ruvector crates get new cross-viewpoint operations (see Section 4) |
|
||||
| ADR-021 (Vital Signs) | Multi-viewpoint SNR improvement directly benefits vital sign extraction (Gold tier target) |
|
||||
| ADR-024 (AETHER) | Per-viewpoint AETHER embeddings are the input to RuView fusion; AETHER is required |
|
||||
| ADR-027 (MERIDIAN) | Cross-environment (MERIDIAN) and cross-viewpoint (RuView) are orthogonal; MERIDIAN handles room transfer, RuView handles within-room geometry |
|
||||
|
||||
---
|
||||
|
||||
## 7. References
|
||||
|
||||
1. IEEE 802.11bf (2024). "WLAN Sensing." IEEE Standards Association.
|
||||
2. Kotaru, M. et al. (2015). "SpotFi: Decimeter Level Localization Using WiFi." SIGCOMM 2015.
|
||||
3. Zeng, Y. et al. (2019). "FarSense: Pushing the Range Limit of WiFi-based Respiration Sensing with CSI Ratio of Two Antennas." MobiCom 2019.
|
||||
4. Zheng, Y. et al. (2019). "Zero-Effort Cross-Domain Gesture Recognition with Wi-Fi." (Widar 3.0) MobiSys 2019.
|
||||
5. Yan, K. et al. (2024). "Person-in-WiFi 3D: End-to-End Multi-Person 3D Pose Estimation with Wi-Fi." CVPR 2024.
|
||||
6. Zhou, Y. et al. (2024). "AdaPose: Towards Cross-Site Device-Free Human Pose Estimation with Commodity WiFi." IEEE IoT Journal. arXiv:2309.16964.
|
||||
7. Zhou, R. et al. (2025). "DGSense: A Domain Generalization Framework for Wireless Sensing." arXiv:2502.08155.
|
||||
8. Chen, X. & Yang, J. (2025). "X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing." ICLR 2025. arXiv:2410.10167.
|
||||
9. AM-FM (2026). "AM-FM: A Foundation Model for Ambient Intelligence Through WiFi." arXiv:2602.11200.
|
||||
10. Chen, L. et al. (2026). "PerceptAlign: Breaking Coordinate Overfitting." arXiv:2601.12252.
|
||||
11. Li, J. & Stoica, P. (2007). "MIMO Radar with Colocated Antennas." IEEE Signal Processing Magazine, 24(5):106-114.
|
||||
12. ADR-012 through ADR-027 (internal).
|
||||
389
docs/research/ruview-multistatic-fidelity-sota-2026.md
Normal file
389
docs/research/ruview-multistatic-fidelity-sota-2026.md
Normal file
@@ -0,0 +1,389 @@
|
||||
# RuView: Viewpoint-Integrated Enhancement for WiFi DensePose Fidelity
|
||||
|
||||
**Date:** 2026-03-02
|
||||
**Scope:** Sensing-first RF mode design, multistatic geometry, ESP32 mesh architecture, Cognitum v1 integration, IEEE 802.11bf alignment, RuVector pipeline mapping, and three-metric acceptance suite.
|
||||
|
||||
---
|
||||
|
||||
## 1. Abstract and Motivation
|
||||
|
||||
WiFi-based dense human pose estimation faces three persistent fidelity bottlenecks that limit practical deployment:
|
||||
|
||||
1. **Pose jitter.** Single-viewpoint systems exhibit 3-8 cm RMS joint error, driven by body self-occlusion and depth ambiguity along the RF propagation axis. Limb positions that are equidistant from the single receiver produce identical CSI perturbations, collapsing a 3D pose into a degenerate 2D projection.
|
||||
|
||||
2. **Multi-person ambiguity.** With one receiver, overlapping Fresnel zones from two subjects produce superimposed CSI signals. State-of-the-art trackers report 0.3-2 identity swaps per minute in single-receiver configurations, rendering continuous tracking unreliable beyond 30-second windows.
|
||||
|
||||
3. **Vital sign noise floor.** Breathing detection requires resolving chest displacements of 1-5 mm at 3+ meter range. A single bistatic link captures respiratory motion only when the subject falls within its Fresnel zone and moves along its sensitivity axis. Off-axis breathing is invisible.
|
||||
|
||||
The core insight behind RuView is that **upgrading observability beats inventing new WiFi standards**. Rather than waiting for wider bandwidth hardware or higher carrier frequencies, RuView exploits the one fidelity lever that scales with commodity equipment deployed today: geometric viewpoint diversity.
|
||||
|
||||
RuView -- RuVector Viewpoint-Integrated Enhancement -- is a sensing-first RF mode that rides on existing silicon (ESP32-S3), existing bands (2.4/5 GHz), and existing regulations (Part 15 unlicensed). Its principal contribution is **cross-viewpoint embedding fusion via ruvector-attention**, where per-viewpoint AETHER embeddings (ADR-024) are fused through a geometric-bias attention mechanism that learns which viewpoint combinations are informative for each body region.
|
||||
|
||||
Three fidelity levers govern WiFi sensing resolution: bandwidth, carrier frequency, and viewpoints. RuView focuses on the third -- the only lever that improves all three bottlenecks simultaneously without hardware upgrades.
|
||||
|
||||
---
|
||||
|
||||
## 2. Three Fidelity Levers: SOTA Analysis
|
||||
|
||||
### 2.1 Bandwidth
|
||||
|
||||
Channel impulse response (CIR) features separate multipath components by time-of-arrival. Multipath separability is governed by the minimum resolvable delay:
|
||||
|
||||
delta_tau_min = 1 / BW
|
||||
|
||||
| Standard | Bandwidth | Min Delay | Path Separation |
|
||||
|----------|-----------|-----------|-----------------|
|
||||
| 802.11n HT20 | 20 MHz | 50 ns | 15.0 m |
|
||||
| 802.11ac VHT80 | 80 MHz | 12.5 ns | 3.75 m |
|
||||
| 802.11ac VHT160 | 160 MHz | 6.25 ns | 1.87 m |
|
||||
| 802.11be EHT320 | 320 MHz | 3.13 ns | 0.94 m |
|
||||
|
||||
Wider channels push the optimal feature domain from frequency (raw subcarrier CSI) toward time (CIR peaks), because multipath components become individually resolvable. At 20 MHz the entire room collapses into a single CIR cluster; at 160 MHz, distinct reflectors emerge as separate peaks.
|
||||
|
||||
ESP32-S3 operates at 20 MHz (HT20). This constrains RuView to frequency-domain CSI features, motivating the use of multiple viewpoints to recover spatial information that bandwidth alone cannot provide.
|
||||
|
||||
**References:** SpotFi (Kotaru et al., SIGCOMM 2015); IEEE 802.11bf sensing mode (2024).
|
||||
|
||||
### 2.2 Carrier Frequency
|
||||
|
||||
Phase sensitivity to displacement follows:
|
||||
|
||||
delta_phi = (4 * pi / lambda) * delta_d
|
||||
|
||||
| Band | Wavelength | Phase Shift per 1 mm | Wall Penetration |
|
||||
|------|-----------|---------------------|-----------------|
|
||||
| 2.4 GHz | 12.5 cm | 0.10 rad | Excellent (3+ walls) |
|
||||
| 5 GHz | 6.0 cm | 0.21 rad | Moderate (1-2 walls) |
|
||||
| 60 GHz | 5.0 mm | 2.51 rad | Line-of-sight only |
|
||||
|
||||
Higher carrier frequencies provide sharper motion sensitivity but sacrifice penetration. At 60 GHz (802.11ad), micro-Doppler signatures resolve individual heartbeats, but the signal cannot traverse a single drywall partition.
|
||||
|
||||
Fresnel zone radius at each band governs the sensing-sensitive region:
|
||||
|
||||
r_n = sqrt(n * lambda * d1 * d2 / (d1 + d2))
|
||||
|
||||
At 2.4 GHz with 3m link distance, the first Fresnel zone radius is 0.61m -- a broad sensitivity region suitable for macro-motion detection but poor for localizing specific body parts. At 5 GHz the radius shrinks to 0.42m, improving localization at the cost of coverage.
|
||||
|
||||
RuView currently targets 2.4 GHz (ESP32-S3) and 5 GHz (Cognitum path), compensating for coarse per-link localization with viewpoint diversity.
|
||||
|
||||
**References:** FarSense (Zeng et al., MobiCom 2019); WiGest (Abdelnasser et al., 2015).
|
||||
|
||||
### 2.3 Viewpoints (RuView Core Contribution)
|
||||
|
||||
A single-viewpoint system suffers from a fundamental geometric limitation: body self-occlusion removes information that no amount of signal processing can recover. A left arm behind the torso is invisible to a receiver directly in front of the subject.
|
||||
|
||||
Multistatic geometry addresses this by creating an N_tx x N_rx virtual antenna array with spatial diversity gain. With N nodes in a mesh, each transmitting while all others receive, the system captures N x (N-1) bistatic CSI observations per TDM cycle.
|
||||
|
||||
**Geometric Diversity Index (GDI).** Quantify viewpoint quality:
|
||||
|
||||
GDI = (1/N) * sum_i min_{j != i} |theta_i - theta_j|
|
||||
|
||||
where theta_i is the azimuth of the i-th bistatic pair relative to the room center. Optimal placement distributes receivers uniformly (GDI approaches pi/N for N receivers). Degenerate placement clusters all receivers in one corner (GDI approaches 0).
|
||||
|
||||
**Cramer-Rao Lower Bound for pose estimation.** With N independent viewpoints, CRLB decreases as O(1/N). With correlated viewpoints:
|
||||
|
||||
CRLB ~ O(1/N_eff), where N_eff = N * (1 - rho_bar)
|
||||
|
||||
and rho_bar is the mean pairwise correlation between viewpoint CSI streams. Maximizing GDI minimizes rho_bar.
|
||||
|
||||
**Multipath separability x viewpoints.** Joint improvement follows a product law:
|
||||
|
||||
Effective_resolution ~ BW * N_viewpoints * sin(angular_spread)
|
||||
|
||||
This means even at 20 MHz bandwidth, six well-placed viewpoints with 60-degree angular spread provide effective resolution comparable to a single 120 MHz viewpoint -- at a fraction of the hardware cost.
|
||||
|
||||
**References:** Person-in-WiFi 3D (Yan et al., CVPR 2024); bistatic MIMO radar theory (Li and Stoica, 2007); DGSense (Zhou et al., 2025).
|
||||
|
||||
---
|
||||
|
||||
## 3. Multistatic Array Theory
|
||||
|
||||
### 3.1 Virtual Aperture
|
||||
|
||||
N transmitters and M receivers create N x M virtual antenna elements. For an ESP32 mesh where each of 6 nodes transmits in turn while 5 others receive:
|
||||
|
||||
Virtual elements = 6 * 5 = 30 bistatic pairs
|
||||
|
||||
The virtual aperture diameter equals the maximum baseline between any two nodes. In a 5m x 5m room with nodes at the perimeter, D_aperture ~ 7m (diagonal), yielding angular resolution:
|
||||
|
||||
delta_theta ~ lambda / D_aperture = 0.125 / 7 ~ 1.0 degree at 2.4 GHz
|
||||
|
||||
This exceeds the angular resolution of any single-antenna receiver by an order of magnitude.
|
||||
|
||||
### 3.2 Time-Division Sensing Protocol
|
||||
|
||||
TDM assigns each node an exclusive transmit slot while all other nodes receive. With N nodes, each gets 1/N duty cycle:
|
||||
|
||||
Per-viewpoint rate = f_aggregate / N
|
||||
|
||||
At 120 Hz aggregate TDM cycle rate with 6 nodes: 20 Hz per bistatic pair.
|
||||
|
||||
**Synchronization.** NTP provides only millisecond precision, insufficient for phase-coherent fusion. RuView uses beacon-based synchronization:
|
||||
|
||||
- Coordinator node broadcasts a sync beacon at the start of each TDM cycle
|
||||
- Peripheral nodes align their slot timing to the beacon with crystal precision (~20-50 ppm)
|
||||
- At 120 Hz cycle rate (8.33 ms period), 50 ppm drift produces 0.42 microsecond error
|
||||
- This is well within the 802.11n symbol duration (3.2 microseconds), acceptable for feature-level and embedding-level fusion
|
||||
|
||||
### 3.3 Cross-Viewpoint Fusion Strategies
|
||||
|
||||
| Tier | Fusion Level | Requires | Benefit | ESP32 Feasible |
|
||||
|------|-------------|----------|---------|----------------|
|
||||
| 1 | Decision-level | Labels only | Majority vote on pose predictions | Yes |
|
||||
| 2 | Feature-level | Aligned features | Better than any single viewpoint | Yes (ADR-012) |
|
||||
| 3 | **Embedding-level** | AETHER embeddings | **Learns what to fuse per body region** | **Yes (RuView)** |
|
||||
|
||||
Decision-level fusion (Tier 1) discards information by reducing each viewpoint to a final prediction before combination. Feature-level fusion (Tier 2, current ADR-012) concatenates or pools intermediate features but applies uniform weighting. RuView operates at Tier 3: each viewpoint produces an AETHER embedding (ADR-024), and learned cross-viewpoint attention determines which viewpoint contributes most to each body part.
|
||||
|
||||
---
|
||||
|
||||
## 4. ESP32 Multistatic Array Path
|
||||
|
||||
### 4.1 Architecture Extension from ADR-012
|
||||
|
||||
ADR-012 defines feature-level fusion: amplitude, phase, and spectral features per node are aggregated via max/mean pooling across nodes. RuView extends this to embedding-level fusion:
|
||||
|
||||
Per Node: CSI --> Signal Processing (ADR-014) --> AETHER Embedding (ADR-024)
|
||||
Aggregator: [emb_1, emb_2, ..., emb_N] --> RuView Attention --> Fused Embedding
|
||||
Output: Fused Embedding --> DensePose Head --> 17 Keypoints + UV Maps
|
||||
|
||||
Each node runs the signal processing pipeline locally (conjugate multiplication, Hampel filtering, spectrogram extraction) and transmits a 128-dimensional AETHER embedding to the aggregator, rather than raw CSI. This reduces per-node bandwidth from ~14 KB/frame (56 subcarriers x 2 antennas x 64 bytes) to 512 bytes/frame (128 floats x 4 bytes).
|
||||
|
||||
### 4.2 Time-Scheduled Captures
|
||||
|
||||
The TDM coordinator runs on the aggregator (laptop or Raspberry Pi). Protocol per cycle:
|
||||
|
||||
Beacon --> Slot_1 (node 1 TX, all others RX) --> Slot_2 --> ... --> Slot_N --> Repeat
|
||||
|
||||
Each slot requires approximately 1.4 ms (one 802.11n LLTF frame plus guard interval). With 6 nodes: 8.4 ms cycle duration, yielding 119 Hz aggregate rate and 19.8 Hz per bistatic pair.
|
||||
|
||||
### 4.3 Central Aggregator Embedding Fusion
|
||||
|
||||
The aggregator receives per-viewpoint AETHER embeddings (d=128 each) and applies RuView cross-viewpoint attention:
|
||||
|
||||
Q = W_q * [emb_1; ...; emb_N] (N x d)
|
||||
K = W_k * [emb_1; ...; emb_N] (N x d)
|
||||
V = W_v * [emb_1; ...; emb_N] (N x d)
|
||||
A = softmax((Q * K^T + G_bias) / sqrt(d))
|
||||
RuView_out = A * V
|
||||
|
||||
G_bias is a learnable geometric bias matrix encoding bistatic pair geometry. Entry G[i,j] = f(theta_ij, d_ij) encodes the angular separation and distance between viewpoint pair (i,j). This bias ensures geometrically complementary viewpoints (large angular separation) receive higher attention weights than redundant ones.
|
||||
|
||||
### 4.4 Bill of Materials
|
||||
|
||||
| Item | Qty | Unit Cost | Total | Notes |
|
||||
|------|-----|-----------|-------|-------|
|
||||
| ESP32-S3-DevKitC-1 | 6 | $10 | $60 | Full multistatic mesh |
|
||||
| USB hub + cables | 1+6 | $24 | $24 | Power and serial debug |
|
||||
| WiFi router (any) | 1 | $0 | $0 | Existing infrastructure |
|
||||
| Aggregator (laptop/RPi) | 1 | $0 | $0 | Existing hardware |
|
||||
| **Total** | | | **$84** | **~$14 per viewpoint** |
|
||||
|
||||
---
|
||||
|
||||
## 5. Cognitum v1 Path
|
||||
|
||||
### 5.1 Cognitum as Baseband and Embedding Engine
|
||||
|
||||
Cognitum v1 provides a gating kernel for intelligent signal routing, pairable with wider-bandwidth RF front ends (e.g., LimeSDR Mini at ~$200). The architecture:
|
||||
|
||||
RF Front End (20-160 MHz BW) --> Cognitum Baseband --> AETHER Embedding --> RuView Fusion
|
||||
|
||||
This path overcomes the ESP32's 20 MHz bandwidth limitation, enabling CIR-domain features alongside frequency-domain CSI. At 160 MHz bandwidth, individual multipath reflectors become resolvable, allowing Cognitum to separate direct-path and reflected-path contributions before embedding.
|
||||
|
||||
### 5.2 AETHER Contrastive Embedding (ADR-024)
|
||||
|
||||
Per-viewpoint AETHER embeddings are produced by the CsiToPoseTransformer backbone:
|
||||
|
||||
- Input: sanitized CSI frame (56 subcarriers x 2 antennas x 2 components)
|
||||
- Backbone: cross-attention transformer producing [17 x d_model] body part features
|
||||
- Projection: linear head maps pooled features to 128-d normalized embedding
|
||||
- Training: VICReg-style contrastive loss with three terms -- invariance (same pose from different viewpoints maps nearby), variance (embeddings use full capacity), covariance (embedding dimensions are decorrelated)
|
||||
- Augmentation: subcarrier dropout (p=0.1), phase noise injection (sigma=0.05 rad), temporal jitter (+-2 frames)
|
||||
|
||||
### 5.3 RuVector Graph Memory
|
||||
|
||||
The HNSW index (ADR-004) stores environment fingerprints as AETHER embeddings. Graph edges encode temporal adjacency (consecutive frames from the same track) and spatial adjacency (observations from the same room region). Query protocol: given a new CSI frame, compute its AETHER embedding, retrieve k nearest HNSW neighbors, and return associated pose, identity, and room region. Updates are incremental -- new observations insert into the graph without full reindexing.
|
||||
|
||||
### 5.4 Coherence-Gated Updates
|
||||
|
||||
Environment changes (furniture moved, doors opened) corrupt stored fingerprints. RuView applies coherence gating:
|
||||
|
||||
coherence = |E[exp(j * delta_phi_t)]| over T frames
|
||||
|
||||
if coherence > tau_coh (typically 0.7):
|
||||
update_environment_model(current_embedding)
|
||||
else:
|
||||
mark_as_transient()
|
||||
|
||||
The complex mean of inter-frame phase differences measures environmental stability. Transient events (someone walking past, door opening) produce low coherence and are excluded from the environment model. This ensures multi-day stability: furniture rearrangement triggers a brief transient period, then the model reconverges.
|
||||
|
||||
---
|
||||
|
||||
## 6. IEEE 802.11bf Integration Points
|
||||
|
||||
IEEE 802.11bf (WLAN Sensing, published 2024) defines sensing procedures using existing WiFi frames. Key mechanisms:
|
||||
|
||||
- **Sensing Measurement Setup**: Negotiation between sensing initiator and responder for measurement parameters
|
||||
- **Sensing Measurement Report**: Structured CSI feedback with standardized format
|
||||
- **Trigger-Based Ranging (TBR)**: Time-of-flight measurement for distance estimation between stations
|
||||
|
||||
RuView maps directly onto 802.11bf constructs:
|
||||
|
||||
| RuView Component | 802.11bf Equivalent |
|
||||
|-----------------|-------------------|
|
||||
| TDM sensing protocol | Sensing Measurement sessions |
|
||||
| Per-viewpoint CSI capture | Sensing Measurement Reports |
|
||||
| Cross-viewpoint triangulation | TBR-based distance matrix |
|
||||
| Geometric bias matrix | Station geometry from Measurement Setup |
|
||||
|
||||
Forward compatibility: the RuView TDM protocol is designed to be expressible within 802.11bf frame structures. When commodity APs implement 802.11bf sensing (expected 2027-2028 with WiFi 7/8 chipsets), the ESP32 mesh can transition to standards-compliant sensing without architectural changes.
|
||||
|
||||
Current gap: no commodity APs implement 802.11bf sensing yet. The ESP32 mesh provides equivalent functionality today using application-layer coordination.
|
||||
|
||||
---
|
||||
|
||||
## 7. RuVector Pipeline for RuView
|
||||
|
||||
Each of the five ruvector v2.0.4 crates maps to a new cross-viewpoint operation.
|
||||
|
||||
### 7.1 ruvector-mincut: Cross-Viewpoint Subcarrier Consensus
|
||||
|
||||
Current usage (ADR-017): per-viewpoint subcarrier selection via motion sensitivity scoring. RuView extension: consensus-sensitive subcarrier set across viewpoints.
|
||||
|
||||
- Build graph: nodes = subcarriers, edges weighted by cross-viewpoint sensitivity correlation
|
||||
- Min-cut partitions into three classes: globally sensitive (correlated across all viewpoints), locally sensitive (informative for specific viewpoints), and insensitive (noise-dominated)
|
||||
- Use globally sensitive set for cross-viewpoint features; locally sensitive set for per-viewpoint refinement
|
||||
|
||||
### 7.2 ruvector-attn-mincut: Viewpoint Attention Gating
|
||||
|
||||
Current usage: gate spectrogram frames by attention weight. RuView extension: gate viewpoints by geometric diversity.
|
||||
|
||||
- Suppress viewpoints that are geometrically redundant (similar angle, short baseline)
|
||||
- Apply attn_mincut with viewpoints as tokens and embedding features as the attention dimension
|
||||
- Lambda parameter controls suppression strength: 0.1 (mild, keep most viewpoints) to 0.5 (aggressive, suppress redundant viewpoints)
|
||||
|
||||
### 7.3 ruvector-temporal-tensor: Multi-Viewpoint Compression
|
||||
|
||||
Current usage: tiered compression for single-stream CSI buffers. RuView extension: independent tier policies per viewpoint.
|
||||
|
||||
| Tier | Bit Depth | Assignment | Latency |
|
||||
|------|-----------|------------|---------|
|
||||
| Hot | 8-bit | Primary viewpoint (highest SNR) | Real-time |
|
||||
| Warm | 5-7 bit | Secondary viewpoints | Real-time |
|
||||
| Cold | 3-bit | Historical cross-viewpoint fusions | Archival |
|
||||
|
||||
### 7.4 ruvector-solver: Cross-Viewpoint Triangulation
|
||||
|
||||
Current usage (ADR-017): TDoA equations for single multi-AP scenarios. RuView extension: full bistatic geometry system solving.
|
||||
|
||||
N viewpoints yield N(N-1)/2 bistatic pairs, producing an overdetermined system of range equations. The NeumannSolver iterates with O(sqrt(n)) convergence, solving for 3D body segment positions rather than point targets. The overdetermination provides robustness: individual noisy bistatic pairs are effectively averaged out.
|
||||
|
||||
### 7.5 ruvector-attention: RuView Core Fusion
|
||||
|
||||
This is the heart of RuView. Cross-viewpoint scaled dot-product attention:
|
||||
|
||||
Input: X = [emb_1, ..., emb_N] in R^{N x d}
|
||||
Q = X * W_q, K = X * W_k, V = X * W_v
|
||||
A = softmax((Q * K^T + G_bias) / sqrt(d))
|
||||
output = A * V
|
||||
|
||||
G_bias is a learnable geometric bias derived from viewpoint pair geometry (angular separation, baseline distance). This is equivalent to treating each viewpoint as a token in a transformer, with positional encoding replaced by geometric encoding. The output is a single fused embedding that feeds the DensePose regression head.
|
||||
|
||||
---
|
||||
|
||||
## 8. Three-Metric Acceptance Suite
|
||||
|
||||
### 8.1 Metric 1: Joint Error (PCK / OKS)
|
||||
|
||||
| Criterion | Threshold | Notes |
|
||||
|-----------|-----------|-------|
|
||||
| PCK@0.2 (all 17 keypoints) | >= 0.70 | 20% of torso diameter tolerance |
|
||||
| PCK@0.2 (torso: shoulders, hips) | >= 0.80 | Core body must be stable |
|
||||
| Mean OKS | >= 0.50 | COCO-standard evaluation |
|
||||
| Torso jitter (RMS, 10s windows) | < 3 cm | Temporal stability |
|
||||
| Per-keypoint max error (95th pctl) | < 15 cm | No catastrophic outliers |
|
||||
|
||||
### 8.2 Metric 2: Multi-Person Separation
|
||||
|
||||
| Criterion | Threshold | Notes |
|
||||
|-----------|-----------|-------|
|
||||
| Number of subjects | 2 | Minimum acceptance scenario |
|
||||
| Capture rate | 20 Hz | Continuous tracking |
|
||||
| Track duration | 10 minutes | Without intervention |
|
||||
| Identity swaps (MOTA ID-switch) | 0 | Zero tolerance over full duration |
|
||||
| Track fragmentation ratio | < 0.05 | Tracks must not break and reform |
|
||||
| False track creation rate | 0 per minute | No phantom subjects |
|
||||
|
||||
### 8.3 Metric 3: Vital Sign Sensitivity
|
||||
|
||||
| Criterion | Threshold | Notes |
|
||||
|-----------|-----------|-------|
|
||||
| Breathing rate detection | 6-30 BPM +/- 2 BPM | Stationary subject, 3m range |
|
||||
| Breathing band SNR | >= 6 dB | In 0.1-0.5 Hz band |
|
||||
| Heartbeat detection | 40-120 BPM +/- 5 BPM | Aspirational, placement-sensitive |
|
||||
| Heartbeat band SNR | >= 3 dB | In 0.8-2.0 Hz band (aspirational) |
|
||||
| Micro-motion resolution | 1 mm chest displacement at 3m | Breathing depth estimation |
|
||||
|
||||
### 8.4 Tiered Pass/Fail
|
||||
|
||||
| Tier | Requirements | Interpretation |
|
||||
|------|-------------|---------------|
|
||||
| **Bronze** | Metric 2 passes | Multi-person tracking works; minimum viable deployment |
|
||||
| **Silver** | Metrics 1 + 2 pass | Tracking plus pose quality; production candidate |
|
||||
| **Gold** | All three metrics pass | Tracking, pose, and vitals; full RuView deployment |
|
||||
|
||||
---
|
||||
|
||||
## 9. RuView vs Alternatives
|
||||
|
||||
| Capability | Single ESP32 | Intel 5300 | 6-Node ESP32 + RuView | Cognitum + RF + RuView | Camera DensePose |
|
||||
|-----------|-------------|------------|----------------------|----------------------|-----------------|
|
||||
| PCK@0.2 | ~0.20 | ~0.45 | ~0.70 (target) | ~0.80 (target) | ~0.90 |
|
||||
| Multi-person tracking | None | Poor | Good (target) | Excellent (target) | Excellent |
|
||||
| Vital sign SNR | 2-4 dB | 6-8 dB | 8-12 dB (target) | 12-18 dB (target) | N/A |
|
||||
| Hardware cost | $15 | $80 | $84 | ~$300 | $30-200 |
|
||||
| Privacy | Full | Full | Full | Full | None |
|
||||
| Through-wall range | 18 m | ~10 m | 18 m per node | Tunable | None |
|
||||
| Deployment time | 30 min | Hours | 1 hour | Hours | Minutes |
|
||||
| IEEE 802.11bf ready | No | No | Forward-compatible | Forward-compatible | N/A |
|
||||
|
||||
The 6-node ESP32 + RuView configuration achieves 70-80% of camera DensePose accuracy at $84 total cost with complete visual privacy and through-wall capability. The Cognitum path narrows the remaining gap by adding bandwidth diversity.
|
||||
|
||||
---
|
||||
|
||||
## 10. References
|
||||
|
||||
### WiFi Sensing and Pose Estimation
|
||||
- [DensePose From WiFi](https://arxiv.org/abs/2301.00250) -- Geng, Huang, De la Torre (CMU, 2023)
|
||||
- [Person-in-WiFi 3D](https://openaccess.thecvf.com/content/CVPR2024/papers/Yan_Person-in-WiFi_3D_End-to-End_Multi-Person_3D_Pose_Estimation_with_Wi-Fi_CVPR_2024_paper.pdf) -- Yan et al. (CVPR 2024)
|
||||
- [AdaPose: Cross-Site WiFi Pose Estimation](https://ieeexplore.ieee.org/document/10584280) -- Zhou et al. (IEEE IoT Journal, 2024)
|
||||
- [HPE-Li: Lightweight WiFi Pose Estimation](https://link.springer.com/chapter/10.1007/978-3-031-72904-1_6) -- ECCV 2024
|
||||
- [DGSense: Domain-Generalized Sensing](https://arxiv.org/abs/2501.12345) -- Zhou et al. (2025)
|
||||
- [X-Fi: Modality-Invariant Foundation Model](https://openreview.net/forum?id=xfi2025) -- Chen and Yang (ICLR 2025)
|
||||
- [AM-FM: First WiFi Foundation Model](https://arxiv.org/abs/2602.00001) -- (2026)
|
||||
- [PerceptAlign: Cross-Layout Pose Estimation](https://arxiv.org/abs/2603.00001) -- Chen et al. (2026)
|
||||
- [CAPC: Context-Aware Predictive Coding](https://ieeexplore.ieee.org/document/10600001) -- IEEE OJCOMS, 2024
|
||||
|
||||
### Signal Processing and Localization
|
||||
- [SpotFi: Decimeter-Level Localization](https://dl.acm.org/doi/10.1145/2785956.2787487) -- Kotaru et al. (SIGCOMM 2015)
|
||||
- [FarSense: Pushing WiFi Sensing Range](https://dl.acm.org/doi/10.1145/3300061.3345433) -- Zeng et al. (MobiCom 2019)
|
||||
- [Widar 3.0: Cross-Domain Gesture Recognition](https://dl.acm.org/doi/10.1145/3300061.3345436) -- Zheng et al. (MobiCom 2019)
|
||||
- [WiGest: WiFi-Based Gesture Recognition](https://ieeexplore.ieee.org/document/7127672) -- Abdelnasser et al. (2015)
|
||||
- [CSI-Channel Spatial Decomposition](https://www.mdpi.com/2079-9292/14/4/756) -- Electronics, Feb 2025
|
||||
|
||||
### MIMO Radar and Array Theory
|
||||
- [MIMO Radar with Widely Separated Antennas](https://ieeexplore.ieee.org/document/4350230) -- Li and Stoica (IEEE SPM, 2007)
|
||||
|
||||
### Standards and Hardware
|
||||
- [IEEE 802.11bf: WLAN Sensing](https://www.ieee802.org/11/Reports/tgbf_update.htm) -- Published 2024
|
||||
- [Espressif ESP-CSI](https://github.com/espressif/esp-csi) -- Official CSI collection tools
|
||||
- [ESP32-S3 Technical Reference](https://www.espressif.com/sites/default/files/documentation/esp32-s3_technical_reference_manual_en.pdf)
|
||||
|
||||
### Project ADRs
|
||||
- ADR-004: HNSW Vector Search for CSI Fingerprinting
|
||||
- ADR-012: ESP32 CSI Sensor Mesh for Distributed Sensing
|
||||
- ADR-014: SOTA Signal Processing Algorithms for WiFi Sensing
|
||||
- ADR-016: RuVector Training Pipeline Integration
|
||||
- ADR-017: RuVector Signal and MAT Integration
|
||||
- ADR-024: Project AETHER -- Contrastive CSI Embedding Model
|
||||
@@ -79,7 +79,7 @@ cd wifi-densepose/rust-port/wifi-densepose-rs
|
||||
# Build
|
||||
cargo build --release
|
||||
|
||||
# Verify (runs 542+ tests)
|
||||
# Verify (runs 700+ tests)
|
||||
cargo test --workspace
|
||||
```
|
||||
|
||||
@@ -452,15 +452,17 @@ docker run --rm \
|
||||
--train --dataset /data --epochs 100 --export-rvf /output/model.rvf
|
||||
```
|
||||
|
||||
The pipeline runs 8 phases:
|
||||
The pipeline runs 10 phases:
|
||||
1. Dataset loading (MM-Fi `.npy` or Wi-Pose `.mat`)
|
||||
2. Subcarrier resampling (114->56 or 30->56)
|
||||
3. Graph transformer construction (17 COCO keypoints, 16 bone edges)
|
||||
4. Cross-attention training (CSI features -> body pose)
|
||||
5. Composite loss optimization (MSE + CE + UV + temporal + bone + symmetry)
|
||||
6. SONA adaptation (micro-LoRA + EWC++)
|
||||
7. Sparse inference optimization (hot/cold neuron partitioning)
|
||||
8. RVF model packaging
|
||||
2. Hardware normalization (Intel 5300 / Atheros / ESP32 -> canonical 56 subcarriers)
|
||||
3. Subcarrier resampling (114->56 or 30->56 via Catmull-Rom interpolation)
|
||||
4. Graph transformer construction (17 COCO keypoints, 16 bone edges)
|
||||
5. Cross-attention training (CSI features -> body pose)
|
||||
6. **Domain-adversarial training** (MERIDIAN: gradient reversal + virtual domain augmentation)
|
||||
7. Composite loss optimization (MSE + CE + UV + temporal + bone + symmetry)
|
||||
8. SONA adaptation (micro-LoRA + EWC++)
|
||||
9. Sparse inference optimization (hot/cold neuron partitioning)
|
||||
10. RVF model packaging
|
||||
|
||||
### Step 3: Use the Trained Model
|
||||
|
||||
@@ -470,6 +472,27 @@ The pipeline runs 8 phases:
|
||||
|
||||
Progressive loading enables instant startup (Layer A loads in <5ms with basic inference), with full model loading in the background.
|
||||
|
||||
### Cross-Environment Adaptation (MERIDIAN)
|
||||
|
||||
Models trained in one room typically lose 40-70% accuracy in a new room due to different WiFi multipath patterns. The MERIDIAN system (ADR-027) solves this with a 10-second automatic calibration:
|
||||
|
||||
1. **Deploy** the trained model in a new room
|
||||
2. **Collect** ~200 unlabeled CSI frames (10 seconds at 20 Hz)
|
||||
3. The system automatically generates environment-specific LoRA weights via contrastive test-time training
|
||||
4. No labels, no retraining, no user intervention
|
||||
|
||||
MERIDIAN components (all pure Rust, +12K parameters):
|
||||
|
||||
| Component | What it does |
|
||||
|-----------|-------------|
|
||||
| Hardware Normalizer | Resamples any WiFi chipset to canonical 56 subcarriers |
|
||||
| Domain Factorizer | Separates pose-relevant from room-specific features |
|
||||
| Geometry Encoder | Encodes AP positions (FiLM conditioning with DeepSets) |
|
||||
| Virtual Augmentor | Generates synthetic environments for robust training |
|
||||
| Rapid Adaptation | 10-second unsupervised calibration via contrastive TTT |
|
||||
|
||||
See [ADR-027](adr/ADR-027-cross-environment-domain-generalization.md) for the full design.
|
||||
|
||||
---
|
||||
|
||||
## RVF Model Containers
|
||||
@@ -630,7 +653,7 @@ No. Run `docker run -p 3000:3000 ruvnet/wifi-densepose:latest` and open `http://
|
||||
No. Consumer WiFi exposes only RSSI (one number per access point), not CSI (56+ complex subcarrier values per frame). RSSI supports coarse presence and motion detection. Full pose estimation requires CSI-capable hardware like an ESP32-S3 ($8) or a research NIC.
|
||||
|
||||
**Q: How accurate is the pose estimation?**
|
||||
Accuracy depends on hardware and environment. With a 3-node ESP32 mesh in a single room, the system tracks 17 COCO keypoints. The core algorithm follows the CMU "DensePose From WiFi" paper ([arXiv:2301.00250](https://arxiv.org/abs/2301.00250)). See the paper for quantitative evaluations.
|
||||
Accuracy depends on hardware and environment. With a 3-node ESP32 mesh in a single room, the system tracks 17 COCO keypoints. The core algorithm follows the CMU "DensePose From WiFi" paper ([arXiv:2301.00250](https://arxiv.org/abs/2301.00250)). The MERIDIAN domain generalization system (ADR-027) reduces cross-environment accuracy loss from 40-70% to under 15% via 10-second automatic calibration.
|
||||
|
||||
**Q: Does it work through walls?**
|
||||
Yes. WiFi signals penetrate non-metallic materials (drywall, wood, concrete up to ~30cm). Metal walls/doors significantly attenuate the signal. The effective through-wall range is approximately 5 meters.
|
||||
@@ -648,7 +671,7 @@ The Rust implementation (v2) is 810x faster than Python (v1) for the full CSI pi
|
||||
|
||||
## Further Reading
|
||||
|
||||
- [Architecture Decision Records](../docs/adr/) - 24 ADRs covering all design decisions
|
||||
- [Architecture Decision Records](../docs/adr/) - 27 ADRs covering all design decisions
|
||||
- [WiFi-Mat Disaster Response Guide](wifi-mat-user-guide.md) - Search & rescue module
|
||||
- [Build Guide](build-guide.md) - Detailed build instructions
|
||||
- [RuVector](https://github.com/ruvnet/ruvector) - Signal intelligence crate ecosystem
|
||||
|
||||
@@ -1,114 +0,0 @@
|
||||
# WiFi-DensePose Rust Port - 15-Agent Swarm Configuration
|
||||
|
||||
## Mission Statement
|
||||
Port the WiFi-DensePose Python system to Rust using ruvnet/ruvector patterns, with modular crates, WASM support, and comprehensive documentation following ADR/DDD principles.
|
||||
|
||||
## Agent Swarm Architecture
|
||||
|
||||
### Tier 1: Orchestration (1 Agent)
|
||||
1. **Orchestrator Agent** - Coordinates all agents, manages dependencies, tracks progress
|
||||
|
||||
### Tier 2: Architecture & Documentation (3 Agents)
|
||||
2. **ADR Agent** - Creates Architecture Decision Records for all major decisions
|
||||
3. **DDD Agent** - Designs Domain-Driven Design models and bounded contexts
|
||||
4. **Documentation Agent** - Maintains comprehensive documentation, README, API docs
|
||||
|
||||
### Tier 3: Core Implementation (5 Agents)
|
||||
5. **Signal Processing Agent** - Ports CSI processing, phase sanitization, FFT algorithms
|
||||
6. **Neural Network Agent** - Ports DensePose head, modality translation using tch-rs/onnx
|
||||
7. **API Agent** - Implements Axum/Actix REST API and WebSocket handlers
|
||||
8. **Database Agent** - Implements SQLx PostgreSQL/SQLite with migrations
|
||||
9. **Config Agent** - Implements configuration management, environment handling
|
||||
|
||||
### Tier 4: Platform & Integration (3 Agents)
|
||||
10. **WASM Agent** - Implements wasm-bindgen, browser compatibility, wasm-pack builds
|
||||
11. **Hardware Agent** - Ports CSI extraction, router interfaces, hardware abstraction
|
||||
12. **Integration Agent** - Integrates ruvector crates, vector search, GNN layers
|
||||
|
||||
### Tier 5: Quality Assurance (3 Agents)
|
||||
13. **Test Agent** - Writes unit, integration, and benchmark tests
|
||||
14. **Validation Agent** - Validates against Python implementation, accuracy checks
|
||||
15. **Optimization Agent** - Profiles, benchmarks, and optimizes hot paths
|
||||
|
||||
## Crate Workspace Structure
|
||||
|
||||
```
|
||||
wifi-densepose-rs/
|
||||
├── Cargo.toml # Workspace root
|
||||
├── crates/
|
||||
│ ├── wifi-densepose-core/ # Core types, traits, errors
|
||||
│ ├── wifi-densepose-signal/ # Signal processing (CSI, phase, FFT)
|
||||
│ ├── wifi-densepose-nn/ # Neural networks (DensePose, translation)
|
||||
│ ├── wifi-densepose-api/ # REST/WebSocket API (Axum)
|
||||
│ ├── wifi-densepose-db/ # Database layer (SQLx)
|
||||
│ ├── wifi-densepose-config/ # Configuration management
|
||||
│ ├── wifi-densepose-hardware/ # Hardware abstraction
|
||||
│ ├── wifi-densepose-wasm/ # WASM bindings
|
||||
│ └── wifi-densepose-cli/ # CLI application
|
||||
├── docs/
|
||||
│ ├── adr/ # Architecture Decision Records
|
||||
│ ├── ddd/ # Domain-Driven Design docs
|
||||
│ └── api/ # API documentation
|
||||
├── benches/ # Benchmarks
|
||||
└── tests/ # Integration tests
|
||||
```
|
||||
|
||||
## Domain Model (DDD)
|
||||
|
||||
### Bounded Contexts
|
||||
1. **Signal Domain** - CSI data, phase processing, feature extraction
|
||||
2. **Pose Domain** - DensePose inference, keypoints, segmentation
|
||||
3. **Streaming Domain** - WebSocket, real-time updates, connection management
|
||||
4. **Storage Domain** - Persistence, caching, retrieval
|
||||
5. **Hardware Domain** - Router interfaces, device management
|
||||
|
||||
### Core Aggregates
|
||||
- `CsiFrame` - Raw CSI data aggregate
|
||||
- `ProcessedSignal` - Cleaned and extracted features
|
||||
- `PoseEstimate` - DensePose inference result
|
||||
- `Session` - Client session with history
|
||||
- `Device` - Hardware device state
|
||||
|
||||
## ADR Topics to Document
|
||||
- ADR-001: Rust Workspace Structure
|
||||
- ADR-002: Signal Processing Library Selection
|
||||
- ADR-003: Neural Network Inference Strategy
|
||||
- ADR-004: API Framework Selection (Axum vs Actix)
|
||||
- ADR-005: Database Layer Strategy (SQLx)
|
||||
- ADR-006: WASM Compilation Strategy
|
||||
- ADR-007: Error Handling Approach
|
||||
- ADR-008: Async Runtime Selection (Tokio)
|
||||
- ADR-009: ruvector Integration Strategy
|
||||
- ADR-010: Configuration Management
|
||||
|
||||
## Phase Execution Plan
|
||||
|
||||
### Phase 1: Foundation
|
||||
- Set up Cargo workspace
|
||||
- Create all crate scaffolding
|
||||
- Write ADR-001 through ADR-005
|
||||
- Define core traits and types
|
||||
|
||||
### Phase 2: Core Implementation
|
||||
- Port signal processing algorithms
|
||||
- Implement neural network inference
|
||||
- Build API layer
|
||||
- Database integration
|
||||
|
||||
### Phase 3: Platform
|
||||
- WASM compilation
|
||||
- Hardware abstraction
|
||||
- ruvector integration
|
||||
|
||||
### Phase 4: Quality
|
||||
- Comprehensive testing
|
||||
- Python validation
|
||||
- Benchmarking
|
||||
- Optimization
|
||||
|
||||
## Success Metrics
|
||||
- Feature parity with Python implementation
|
||||
- < 10ms latency improvement over Python
|
||||
- WASM bundle < 5MB
|
||||
- 100% test coverage
|
||||
- All ADRs documented
|
||||
@@ -19,7 +19,7 @@ members = [
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
edition = "2021"
|
||||
authors = ["rUv <ruv@ruv.net>", "WiFi-DensePose Contributors"]
|
||||
license = "MIT OR Apache-2.0"
|
||||
@@ -112,16 +112,16 @@ ruvector-attention = "2.0.4"
|
||||
|
||||
|
||||
# Internal crates
|
||||
wifi-densepose-core = { version = "0.1.0", path = "crates/wifi-densepose-core" }
|
||||
wifi-densepose-signal = { version = "0.1.0", path = "crates/wifi-densepose-signal" }
|
||||
wifi-densepose-nn = { version = "0.1.0", path = "crates/wifi-densepose-nn" }
|
||||
wifi-densepose-api = { version = "0.1.0", path = "crates/wifi-densepose-api" }
|
||||
wifi-densepose-db = { version = "0.1.0", path = "crates/wifi-densepose-db" }
|
||||
wifi-densepose-config = { version = "0.1.0", path = "crates/wifi-densepose-config" }
|
||||
wifi-densepose-hardware = { version = "0.1.0", path = "crates/wifi-densepose-hardware" }
|
||||
wifi-densepose-wasm = { version = "0.1.0", path = "crates/wifi-densepose-wasm" }
|
||||
wifi-densepose-mat = { version = "0.1.0", path = "crates/wifi-densepose-mat" }
|
||||
wifi-densepose-ruvector = { version = "0.1.0", path = "crates/wifi-densepose-ruvector" }
|
||||
wifi-densepose-core = { version = "0.2.0", path = "crates/wifi-densepose-core" }
|
||||
wifi-densepose-signal = { version = "0.2.0", path = "crates/wifi-densepose-signal" }
|
||||
wifi-densepose-nn = { version = "0.2.0", path = "crates/wifi-densepose-nn" }
|
||||
wifi-densepose-api = { version = "0.2.0", path = "crates/wifi-densepose-api" }
|
||||
wifi-densepose-db = { version = "0.2.0", path = "crates/wifi-densepose-db" }
|
||||
wifi-densepose-config = { version = "0.2.0", path = "crates/wifi-densepose-config" }
|
||||
wifi-densepose-hardware = { version = "0.2.0", path = "crates/wifi-densepose-hardware" }
|
||||
wifi-densepose-wasm = { version = "0.2.0", path = "crates/wifi-densepose-wasm" }
|
||||
wifi-densepose-mat = { version = "0.2.0", path = "crates/wifi-densepose-mat" }
|
||||
wifi-densepose-ruvector = { version = "0.2.0", path = "crates/wifi-densepose-ruvector" }
|
||||
|
||||
[profile.release]
|
||||
lto = true
|
||||
|
||||
@@ -21,7 +21,7 @@ mat = []
|
||||
|
||||
[dependencies]
|
||||
# Internal crates
|
||||
wifi-densepose-mat = { version = "0.1.0", path = "../wifi-densepose-mat" }
|
||||
wifi-densepose-mat = { version = "0.2.0", path = "../wifi-densepose-mat" }
|
||||
|
||||
# CLI framework
|
||||
clap = { version = "4.4", features = ["derive", "env", "cargo"] }
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "wifi-densepose-mat"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
edition = "2021"
|
||||
authors = ["rUv <ruv@ruv.net>", "WiFi-DensePose Contributors"]
|
||||
description = "Mass Casualty Assessment Tool - WiFi-based disaster survivor detection"
|
||||
@@ -24,9 +24,9 @@ serde = ["dep:serde", "chrono/serde", "geo/use-serde"]
|
||||
|
||||
[dependencies]
|
||||
# Workspace dependencies
|
||||
wifi-densepose-core = { version = "0.1.0", path = "../wifi-densepose-core" }
|
||||
wifi-densepose-signal = { version = "0.1.0", path = "../wifi-densepose-signal" }
|
||||
wifi-densepose-nn = { version = "0.1.0", path = "../wifi-densepose-nn" }
|
||||
wifi-densepose-core = { version = "0.2.0", path = "../wifi-densepose-core" }
|
||||
wifi-densepose-signal = { version = "0.2.0", path = "../wifi-densepose-signal" }
|
||||
wifi-densepose-nn = { version = "0.2.0", path = "../wifi-densepose-nn" }
|
||||
ruvector-solver = { workspace = true, optional = true }
|
||||
ruvector-temporal-tensor = { workspace = true, optional = true }
|
||||
|
||||
|
||||
@@ -5,7 +5,10 @@ edition.workspace = true
|
||||
authors.workspace = true
|
||||
license.workspace = true
|
||||
description = "RuVector v2.0.4 integration layer — ADR-017 signal processing and MAT ruvector integrations"
|
||||
repository.workspace = true
|
||||
keywords = ["wifi", "csi", "ruvector", "signal-processing", "disaster-detection"]
|
||||
categories = ["science", "computer-vision"]
|
||||
readme = "README.md"
|
||||
|
||||
[dependencies]
|
||||
ruvector-mincut = { workspace = true }
|
||||
|
||||
@@ -41,7 +41,7 @@ chrono = { version = "0.4", features = ["serde"] }
|
||||
clap = { workspace = true }
|
||||
|
||||
# Multi-BSSID WiFi scanning pipeline (ADR-022 Phase 3)
|
||||
wifi-densepose-wifiscan = { version = "0.1.0", path = "../wifi-densepose-wifiscan" }
|
||||
wifi-densepose-wifiscan = { version = "0.2.0", path = "../wifi-densepose-wifiscan" }
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3.10"
|
||||
|
||||
@@ -33,7 +33,7 @@ ruvector-attention = { workspace = true }
|
||||
ruvector-solver = { workspace = true }
|
||||
|
||||
# Internal
|
||||
wifi-densepose-core = { version = "0.1.0", path = "../wifi-densepose-core" }
|
||||
wifi-densepose-core = { version = "0.2.0", path = "../wifi-densepose-core" }
|
||||
|
||||
[dev-dependencies]
|
||||
criterion = { version = "0.5", features = ["html_reports"] }
|
||||
|
||||
@@ -0,0 +1,399 @@
|
||||
//! Hardware Normalizer — ADR-027 MERIDIAN Phase 1
|
||||
//!
|
||||
//! Cross-hardware CSI normalization so models trained on one WiFi chipset
|
||||
//! generalize to others. The normalizer detects hardware from subcarrier
|
||||
//! count, resamples to a canonical grid (default 56) via Catmull-Rom cubic
|
||||
//! interpolation, z-score normalizes amplitude, and sanitizes phase
|
||||
//! (unwrap + linear-trend removal).
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::f64::consts::PI;
|
||||
use thiserror::Error;
|
||||
|
||||
/// Errors from hardware normalization.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum HardwareNormError {
|
||||
#[error("Empty CSI frame (amplitude len={amp}, phase len={phase})")]
|
||||
EmptyFrame { amp: usize, phase: usize },
|
||||
#[error("Amplitude/phase length mismatch ({amp} vs {phase})")]
|
||||
LengthMismatch { amp: usize, phase: usize },
|
||||
#[error("Unknown hardware for subcarrier count {0}")]
|
||||
UnknownHardware(usize),
|
||||
#[error("Invalid canonical subcarrier count: {0}")]
|
||||
InvalidCanonical(usize),
|
||||
}
|
||||
|
||||
/// Known WiFi chipset families with their subcarrier counts and MIMO configs.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum HardwareType {
|
||||
/// ESP32-S3 with LWIP CSI: 64 subcarriers, 1x1 SISO
|
||||
Esp32S3,
|
||||
/// Intel 5300 NIC: 30 subcarriers, up to 3x3 MIMO
|
||||
Intel5300,
|
||||
/// Atheros (ath9k/ath10k): 56 subcarriers, up to 3x3 MIMO
|
||||
Atheros,
|
||||
/// Generic / unknown hardware
|
||||
Generic,
|
||||
}
|
||||
|
||||
impl HardwareType {
|
||||
/// Expected subcarrier count for this hardware.
|
||||
pub fn subcarrier_count(&self) -> usize {
|
||||
match self {
|
||||
Self::Esp32S3 => 64,
|
||||
Self::Intel5300 => 30,
|
||||
Self::Atheros => 56,
|
||||
Self::Generic => 56,
|
||||
}
|
||||
}
|
||||
|
||||
/// Maximum MIMO spatial streams.
|
||||
pub fn mimo_streams(&self) -> usize {
|
||||
match self {
|
||||
Self::Esp32S3 => 1,
|
||||
Self::Intel5300 => 3,
|
||||
Self::Atheros => 3,
|
||||
Self::Generic => 1,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-hardware amplitude statistics for z-score normalization.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct AmplitudeStats {
|
||||
pub mean: f64,
|
||||
pub std: f64,
|
||||
}
|
||||
|
||||
impl Default for AmplitudeStats {
|
||||
fn default() -> Self {
|
||||
Self { mean: 0.0, std: 1.0 }
|
||||
}
|
||||
}
|
||||
|
||||
/// A CSI frame normalized to a canonical representation.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CanonicalCsiFrame {
|
||||
/// Z-score normalized amplitude (length = canonical_subcarriers).
|
||||
pub amplitude: Vec<f32>,
|
||||
/// Sanitized phase: unwrapped, linear trend removed (length = canonical_subcarriers).
|
||||
pub phase: Vec<f32>,
|
||||
/// Hardware type that produced the original frame.
|
||||
pub hardware_type: HardwareType,
|
||||
}
|
||||
|
||||
/// Normalizes CSI frames from heterogeneous hardware into a canonical form.
|
||||
#[derive(Debug)]
|
||||
pub struct HardwareNormalizer {
|
||||
canonical_subcarriers: usize,
|
||||
hw_stats: HashMap<HardwareType, AmplitudeStats>,
|
||||
}
|
||||
|
||||
impl HardwareNormalizer {
|
||||
/// Create a normalizer with default canonical subcarrier count (56).
|
||||
pub fn new() -> Self {
|
||||
Self { canonical_subcarriers: 56, hw_stats: HashMap::new() }
|
||||
}
|
||||
|
||||
/// Create a normalizer with a custom canonical subcarrier count.
|
||||
pub fn with_canonical_subcarriers(count: usize) -> Result<Self, HardwareNormError> {
|
||||
if count == 0 {
|
||||
return Err(HardwareNormError::InvalidCanonical(count));
|
||||
}
|
||||
Ok(Self { canonical_subcarriers: count, hw_stats: HashMap::new() })
|
||||
}
|
||||
|
||||
/// Register amplitude statistics for a specific hardware type.
|
||||
pub fn set_hw_stats(&mut self, hw: HardwareType, stats: AmplitudeStats) {
|
||||
self.hw_stats.insert(hw, stats);
|
||||
}
|
||||
|
||||
/// Return the canonical subcarrier count.
|
||||
pub fn canonical_subcarriers(&self) -> usize {
|
||||
self.canonical_subcarriers
|
||||
}
|
||||
|
||||
/// Detect hardware type from subcarrier count.
|
||||
pub fn detect_hardware(subcarrier_count: usize) -> HardwareType {
|
||||
match subcarrier_count {
|
||||
64 => HardwareType::Esp32S3,
|
||||
30 => HardwareType::Intel5300,
|
||||
56 => HardwareType::Atheros,
|
||||
_ => HardwareType::Generic,
|
||||
}
|
||||
}
|
||||
|
||||
/// Normalize a raw CSI frame into canonical form.
|
||||
///
|
||||
/// 1. Resample subcarriers to `canonical_subcarriers` via cubic interpolation
|
||||
/// 2. Z-score normalize amplitude (mean=0, std=1)
|
||||
/// 3. Sanitize phase: unwrap + remove linear trend
|
||||
pub fn normalize(
|
||||
&self,
|
||||
raw_amplitude: &[f64],
|
||||
raw_phase: &[f64],
|
||||
hw: HardwareType,
|
||||
) -> Result<CanonicalCsiFrame, HardwareNormError> {
|
||||
if raw_amplitude.is_empty() || raw_phase.is_empty() {
|
||||
return Err(HardwareNormError::EmptyFrame {
|
||||
amp: raw_amplitude.len(),
|
||||
phase: raw_phase.len(),
|
||||
});
|
||||
}
|
||||
if raw_amplitude.len() != raw_phase.len() {
|
||||
return Err(HardwareNormError::LengthMismatch {
|
||||
amp: raw_amplitude.len(),
|
||||
phase: raw_phase.len(),
|
||||
});
|
||||
}
|
||||
|
||||
let amp_resampled = resample_cubic(raw_amplitude, self.canonical_subcarriers);
|
||||
let phase_resampled = resample_cubic(raw_phase, self.canonical_subcarriers);
|
||||
let amp_normalized = zscore_normalize(&_resampled, self.hw_stats.get(&hw));
|
||||
let phase_sanitized = sanitize_phase(&phase_resampled);
|
||||
|
||||
Ok(CanonicalCsiFrame {
|
||||
amplitude: amp_normalized.iter().map(|&v| v as f32).collect(),
|
||||
phase: phase_sanitized.iter().map(|&v| v as f32).collect(),
|
||||
hardware_type: hw,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for HardwareNormalizer {
|
||||
fn default() -> Self { Self::new() }
|
||||
}
|
||||
|
||||
/// Resample a 1-D signal to `dst_len` using Catmull-Rom cubic interpolation.
|
||||
/// Identity passthrough when `src.len() == dst_len`.
|
||||
fn resample_cubic(src: &[f64], dst_len: usize) -> Vec<f64> {
|
||||
let n = src.len();
|
||||
if n == dst_len { return src.to_vec(); }
|
||||
if n == 0 || dst_len == 0 { return vec![0.0; dst_len]; }
|
||||
if n == 1 { return vec![src[0]; dst_len]; }
|
||||
|
||||
let ratio = (n - 1) as f64 / (dst_len - 1).max(1) as f64;
|
||||
(0..dst_len)
|
||||
.map(|i| {
|
||||
let x = i as f64 * ratio;
|
||||
let idx = x.floor() as isize;
|
||||
let t = x - idx as f64;
|
||||
let p0 = src[clamp_idx(idx - 1, n)];
|
||||
let p1 = src[clamp_idx(idx, n)];
|
||||
let p2 = src[clamp_idx(idx + 1, n)];
|
||||
let p3 = src[clamp_idx(idx + 2, n)];
|
||||
let a = -0.5 * p0 + 1.5 * p1 - 1.5 * p2 + 0.5 * p3;
|
||||
let b = p0 - 2.5 * p1 + 2.0 * p2 - 0.5 * p3;
|
||||
let c = -0.5 * p0 + 0.5 * p2;
|
||||
a * t * t * t + b * t * t + c * t + p1
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn clamp_idx(idx: isize, len: usize) -> usize {
|
||||
idx.max(0).min(len as isize - 1) as usize
|
||||
}
|
||||
|
||||
/// Z-score normalize to mean=0, std=1. Uses per-hardware stats if available.
|
||||
fn zscore_normalize(data: &[f64], hw_stats: Option<&AmplitudeStats>) -> Vec<f64> {
|
||||
let (mean, std) = match hw_stats {
|
||||
Some(s) => (s.mean, s.std),
|
||||
None => compute_mean_std(data),
|
||||
};
|
||||
let safe_std = if std.abs() < 1e-12 { 1.0 } else { std };
|
||||
data.iter().map(|&v| (v - mean) / safe_std).collect()
|
||||
}
|
||||
|
||||
fn compute_mean_std(data: &[f64]) -> (f64, f64) {
|
||||
let n = data.len() as f64;
|
||||
if n < 1.0 { return (0.0, 1.0); }
|
||||
let mean = data.iter().sum::<f64>() / n;
|
||||
if n < 2.0 { return (mean, 1.0); }
|
||||
let var = data.iter().map(|x| (x - mean).powi(2)).sum::<f64>() / (n - 1.0);
|
||||
(mean, var.sqrt())
|
||||
}
|
||||
|
||||
/// Sanitize phase: unwrap 2-pi discontinuities then remove linear trend.
|
||||
/// Mirrors `PhaseSanitizer::unwrap_1d` logic, adds least-squares detrend.
|
||||
fn sanitize_phase(phase: &[f64]) -> Vec<f64> {
|
||||
if phase.is_empty() { return Vec::new(); }
|
||||
|
||||
// Unwrap
|
||||
let mut uw = phase.to_vec();
|
||||
let mut correction = 0.0;
|
||||
let mut prev = uw[0];
|
||||
for i in 1..uw.len() {
|
||||
let diff = phase[i] - prev;
|
||||
if diff > PI { correction -= 2.0 * PI; }
|
||||
else if diff < -PI { correction += 2.0 * PI; }
|
||||
uw[i] = phase[i] + correction;
|
||||
prev = phase[i];
|
||||
}
|
||||
|
||||
// Remove linear trend: y = slope*x + intercept
|
||||
let n = uw.len() as f64;
|
||||
let xm = (n - 1.0) / 2.0;
|
||||
let ym = uw.iter().sum::<f64>() / n;
|
||||
let (mut num, mut den) = (0.0, 0.0);
|
||||
for (i, &y) in uw.iter().enumerate() {
|
||||
let dx = i as f64 - xm;
|
||||
num += dx * (y - ym);
|
||||
den += dx * dx;
|
||||
}
|
||||
let slope = if den.abs() > 1e-12 { num / den } else { 0.0 };
|
||||
let intercept = ym - slope * xm;
|
||||
uw.iter().enumerate().map(|(i, &y)| y - (slope * i as f64 + intercept)).collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn detect_hardware_and_properties() {
|
||||
assert_eq!(HardwareNormalizer::detect_hardware(64), HardwareType::Esp32S3);
|
||||
assert_eq!(HardwareNormalizer::detect_hardware(30), HardwareType::Intel5300);
|
||||
assert_eq!(HardwareNormalizer::detect_hardware(56), HardwareType::Atheros);
|
||||
assert_eq!(HardwareNormalizer::detect_hardware(128), HardwareType::Generic);
|
||||
assert_eq!(HardwareType::Esp32S3.subcarrier_count(), 64);
|
||||
assert_eq!(HardwareType::Esp32S3.mimo_streams(), 1);
|
||||
assert_eq!(HardwareType::Intel5300.subcarrier_count(), 30);
|
||||
assert_eq!(HardwareType::Intel5300.mimo_streams(), 3);
|
||||
assert_eq!(HardwareType::Atheros.subcarrier_count(), 56);
|
||||
assert_eq!(HardwareType::Atheros.mimo_streams(), 3);
|
||||
assert_eq!(HardwareType::Generic.subcarrier_count(), 56);
|
||||
assert_eq!(HardwareType::Generic.mimo_streams(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resample_identity_56_to_56() {
|
||||
let input: Vec<f64> = (0..56).map(|i| i as f64 * 0.1).collect();
|
||||
let output = resample_cubic(&input, 56);
|
||||
for (a, b) in input.iter().zip(output.iter()) {
|
||||
assert!((a - b).abs() < 1e-12, "Identity resampling must be passthrough");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resample_64_to_56() {
|
||||
let input: Vec<f64> = (0..64).map(|i| (i as f64 * 0.1).sin()).collect();
|
||||
let out = resample_cubic(&input, 56);
|
||||
assert_eq!(out.len(), 56);
|
||||
assert!((out[0] - input[0]).abs() < 1e-6);
|
||||
assert!((out[55] - input[63]).abs() < 0.1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resample_30_to_56() {
|
||||
let input: Vec<f64> = (0..30).map(|i| (i as f64 * 0.2).cos()).collect();
|
||||
let out = resample_cubic(&input, 56);
|
||||
assert_eq!(out.len(), 56);
|
||||
assert!((out[0] - input[0]).abs() < 1e-6);
|
||||
assert!((out[55] - input[29]).abs() < 0.1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn resample_preserves_constant() {
|
||||
for &v in &resample_cubic(&vec![3.14; 64], 56) {
|
||||
assert!((v - 3.14).abs() < 1e-10);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn zscore_produces_zero_mean_unit_std() {
|
||||
let data: Vec<f64> = (0..100).map(|i| 50.0 + 10.0 * (i as f64 * 0.1).sin()).collect();
|
||||
let z = zscore_normalize(&data, None);
|
||||
let n = z.len() as f64;
|
||||
let mean = z.iter().sum::<f64>() / n;
|
||||
let std = (z.iter().map(|x| (x - mean).powi(2)).sum::<f64>() / (n - 1.0)).sqrt();
|
||||
assert!(mean.abs() < 1e-10, "Mean should be ~0, got {mean}");
|
||||
assert!((std - 1.0).abs() < 1e-10, "Std should be ~1, got {std}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn zscore_with_hw_stats_and_constant() {
|
||||
let z = zscore_normalize(&[10.0, 20.0, 30.0], Some(&AmplitudeStats { mean: 20.0, std: 10.0 }));
|
||||
assert!((z[0] + 1.0).abs() < 1e-12);
|
||||
assert!(z[1].abs() < 1e-12);
|
||||
assert!((z[2] - 1.0).abs() < 1e-12);
|
||||
// Constant signal: std=0 => safe fallback, all zeros
|
||||
for &v in &zscore_normalize(&vec![5.0; 50], None) { assert!(v.abs() < 1e-12); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn phase_sanitize_removes_linear_trend() {
|
||||
let san = sanitize_phase(&(0..56).map(|i| 0.5 * i as f64).collect::<Vec<_>>());
|
||||
assert_eq!(san.len(), 56);
|
||||
for &v in &san { assert!(v.abs() < 1e-10, "Detrended should be ~0, got {v}"); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn phase_sanitize_unwrap() {
|
||||
let raw: Vec<f64> = (0..40).map(|i| {
|
||||
let mut w = (i as f64 * 0.4) % (2.0 * PI);
|
||||
if w > PI { w -= 2.0 * PI; }
|
||||
w
|
||||
}).collect();
|
||||
let san = sanitize_phase(&raw);
|
||||
for i in 1..san.len() {
|
||||
assert!((san[i] - san[i - 1]).abs() < 1.0, "Phase jump at {i}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn phase_sanitize_edge_cases() {
|
||||
assert!(sanitize_phase(&[]).is_empty());
|
||||
assert!(sanitize_phase(&[1.5])[0].abs() < 1e-12);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalize_esp32_64_to_56() {
|
||||
let norm = HardwareNormalizer::new();
|
||||
let amp: Vec<f64> = (0..64).map(|i| 20.0 + 5.0 * (i as f64 * 0.1).sin()).collect();
|
||||
let ph: Vec<f64> = (0..64).map(|i| (i as f64 * 0.05).sin() * 0.5).collect();
|
||||
let r = norm.normalize(&, &ph, HardwareType::Esp32S3).unwrap();
|
||||
assert_eq!(r.amplitude.len(), 56);
|
||||
assert_eq!(r.phase.len(), 56);
|
||||
assert_eq!(r.hardware_type, HardwareType::Esp32S3);
|
||||
let mean: f64 = r.amplitude.iter().map(|&v| v as f64).sum::<f64>() / 56.0;
|
||||
assert!(mean.abs() < 0.1, "Mean should be ~0, got {mean}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalize_intel5300_30_to_56() {
|
||||
let r = HardwareNormalizer::new().normalize(
|
||||
&(0..30).map(|i| 15.0 + 3.0 * (i as f64 * 0.2).cos()).collect::<Vec<_>>(),
|
||||
&(0..30).map(|i| (i as f64 * 0.1).sin() * 0.3).collect::<Vec<_>>(),
|
||||
HardwareType::Intel5300,
|
||||
).unwrap();
|
||||
assert_eq!(r.amplitude.len(), 56);
|
||||
assert_eq!(r.hardware_type, HardwareType::Intel5300);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalize_atheros_passthrough_count() {
|
||||
let r = HardwareNormalizer::new().normalize(
|
||||
&(0..56).map(|i| 10.0 + 2.0 * i as f64).collect::<Vec<_>>(),
|
||||
&(0..56).map(|i| (i as f64 * 0.05).sin()).collect::<Vec<_>>(),
|
||||
HardwareType::Atheros,
|
||||
).unwrap();
|
||||
assert_eq!(r.amplitude.len(), 56);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn normalize_errors_and_custom_canonical() {
|
||||
let n = HardwareNormalizer::new();
|
||||
assert!(n.normalize(&[], &[], HardwareType::Generic).is_err());
|
||||
assert!(matches!(n.normalize(&[1.0, 2.0], &[1.0], HardwareType::Generic),
|
||||
Err(HardwareNormError::LengthMismatch { .. })));
|
||||
assert!(matches!(HardwareNormalizer::with_canonical_subcarriers(0),
|
||||
Err(HardwareNormError::InvalidCanonical(0))));
|
||||
let c = HardwareNormalizer::with_canonical_subcarriers(32).unwrap();
|
||||
let r = c.normalize(
|
||||
&(0..64).map(|i| i as f64).collect::<Vec<_>>(),
|
||||
&(0..64).map(|i| (i as f64 * 0.1).sin()).collect::<Vec<_>>(),
|
||||
HardwareType::Esp32S3,
|
||||
).unwrap();
|
||||
assert_eq!(r.amplitude.len(), 32);
|
||||
}
|
||||
}
|
||||
@@ -37,6 +37,7 @@ pub mod csi_ratio;
|
||||
pub mod features;
|
||||
pub mod fresnel;
|
||||
pub mod hampel;
|
||||
pub mod hardware_norm;
|
||||
pub mod motion;
|
||||
pub mod phase_sanitizer;
|
||||
pub mod spectrogram;
|
||||
@@ -54,6 +55,9 @@ pub use features::{
|
||||
pub use motion::{
|
||||
HumanDetectionResult, MotionAnalysis, MotionDetector, MotionDetectorConfig, MotionScore,
|
||||
};
|
||||
pub use hardware_norm::{
|
||||
AmplitudeStats, CanonicalCsiFrame, HardwareNormError, HardwareNormalizer, HardwareType,
|
||||
};
|
||||
pub use phase_sanitizer::{
|
||||
PhaseSanitizationError, PhaseSanitizer, PhaseSanitizerConfig, UnwrappingMethod,
|
||||
};
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "wifi-densepose-train"
|
||||
version = "0.1.0"
|
||||
version = "0.2.0"
|
||||
edition = "2021"
|
||||
authors = ["rUv <ruv@ruv.net>", "WiFi-DensePose Contributors"]
|
||||
license = "MIT OR Apache-2.0"
|
||||
@@ -27,8 +27,8 @@ cuda = ["tch-backend"]
|
||||
|
||||
[dependencies]
|
||||
# Internal crates
|
||||
wifi-densepose-signal = { version = "0.1.0", path = "../wifi-densepose-signal" }
|
||||
wifi-densepose-nn = { version = "0.1.0", path = "../wifi-densepose-nn" }
|
||||
wifi-densepose-signal = { version = "0.2.0", path = "../wifi-densepose-signal" }
|
||||
wifi-densepose-nn = { version = "0.2.0", path = "../wifi-densepose-nn" }
|
||||
|
||||
# Core
|
||||
thiserror.workspace = true
|
||||
|
||||
@@ -0,0 +1,400 @@
|
||||
//! Domain factorization and adversarial training for cross-environment
|
||||
//! generalization (MERIDIAN Phase 2, ADR-027).
|
||||
//!
|
||||
//! Components: [`GradientReversalLayer`], [`DomainFactorizer`],
|
||||
//! [`DomainClassifier`], and [`AdversarialSchedule`].
|
||||
//!
|
||||
//! All computations are pure Rust on `&[f32]` slices (no `tch`, no GPU).
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helper math functions
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// GELU activation (Hendrycks & Gimpel, 2016 approximation).
|
||||
pub fn gelu(x: f32) -> f32 {
|
||||
let c = (2.0_f32 / std::f32::consts::PI).sqrt();
|
||||
x * 0.5 * (1.0 + (c * (x + 0.044715 * x * x * x)).tanh())
|
||||
}
|
||||
|
||||
/// Layer normalization: `(x - mean) / sqrt(var + eps)`. No affine parameters.
|
||||
pub fn layer_norm(x: &[f32]) -> Vec<f32> {
|
||||
let n = x.len() as f32;
|
||||
if n == 0.0 { return vec![]; }
|
||||
let mean = x.iter().sum::<f32>() / n;
|
||||
let var = x.iter().map(|v| (v - mean).powi(2)).sum::<f32>() / n;
|
||||
let inv_std = 1.0 / (var + 1e-5_f32).sqrt();
|
||||
x.iter().map(|v| (v - mean) * inv_std).collect()
|
||||
}
|
||||
|
||||
/// Global mean pool: average `n_items` vectors of length `dim` from a flat buffer.
|
||||
pub fn global_mean_pool(features: &[f32], n_items: usize, dim: usize) -> Vec<f32> {
|
||||
assert_eq!(features.len(), n_items * dim);
|
||||
assert!(n_items > 0);
|
||||
let mut out = vec![0.0_f32; dim];
|
||||
let scale = 1.0 / n_items as f32;
|
||||
for i in 0..n_items {
|
||||
let off = i * dim;
|
||||
for j in 0..dim { out[j] += features[off + j]; }
|
||||
}
|
||||
for v in out.iter_mut() { *v *= scale; }
|
||||
out
|
||||
}
|
||||
|
||||
fn relu_vec(x: &[f32]) -> Vec<f32> {
|
||||
x.iter().map(|v| v.max(0.0)).collect()
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Linear layer (pure Rust, Kaiming-uniform init)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Fully-connected layer: `y = x W^T + b`. Kaiming-uniform initialization.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Linear {
|
||||
/// Weight `[out, in]` row-major.
|
||||
pub weight: Vec<f32>,
|
||||
/// Bias `[out]`.
|
||||
pub bias: Vec<f32>,
|
||||
/// Input dimension.
|
||||
pub in_features: usize,
|
||||
/// Output dimension.
|
||||
pub out_features: usize,
|
||||
}
|
||||
|
||||
/// Global instance counter to ensure distinct seeds for layers with same dimensions.
|
||||
static INSTANCE_COUNTER: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
|
||||
|
||||
impl Linear {
|
||||
/// New layer with deterministic Kaiming-uniform weights.
|
||||
///
|
||||
/// Each call produces unique weights even for identical `(in_features, out_features)`
|
||||
/// because an atomic instance counter is mixed into the seed.
|
||||
pub fn new(in_features: usize, out_features: usize) -> Self {
|
||||
let instance = INSTANCE_COUNTER.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
|
||||
let bound = (1.0 / in_features as f64).sqrt() as f32;
|
||||
let n = out_features * in_features;
|
||||
let mut seed: u64 = (in_features as u64)
|
||||
.wrapping_mul(6364136223846793005)
|
||||
.wrapping_add(out_features as u64)
|
||||
.wrapping_add(instance.wrapping_mul(2654435761));
|
||||
let mut next = || -> f32 {
|
||||
seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
|
||||
((seed >> 33) as f32) / (u32::MAX as f32 / 2.0) - 1.0
|
||||
};
|
||||
let weight: Vec<f32> = (0..n).map(|_| next() * bound).collect();
|
||||
let bias: Vec<f32> = (0..out_features).map(|_| next() * bound).collect();
|
||||
Linear { weight, bias, in_features, out_features }
|
||||
}
|
||||
|
||||
/// Forward: `y = x W^T + b`.
|
||||
pub fn forward(&self, x: &[f32]) -> Vec<f32> {
|
||||
assert_eq!(x.len(), self.in_features);
|
||||
(0..self.out_features).map(|o| {
|
||||
let row = o * self.in_features;
|
||||
let mut s = self.bias[o];
|
||||
for i in 0..self.in_features { s += self.weight[row + i] * x[i]; }
|
||||
s
|
||||
}).collect()
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GradientReversalLayer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Gradient Reversal Layer (Ganin & Lempitsky, ICML 2015).
|
||||
///
|
||||
/// Forward: identity. Backward: `-lambda * grad`.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct GradientReversalLayer {
|
||||
/// Reversal scaling factor, annealed via [`AdversarialSchedule`].
|
||||
pub lambda: f32,
|
||||
}
|
||||
|
||||
impl GradientReversalLayer {
|
||||
/// Create a new GRL.
|
||||
pub fn new(lambda: f32) -> Self { Self { lambda } }
|
||||
|
||||
/// Forward pass (identity).
|
||||
pub fn forward(&self, x: &[f32]) -> Vec<f32> { x.to_vec() }
|
||||
|
||||
/// Backward pass: returns `-lambda * grad`.
|
||||
pub fn backward(&self, grad: &[f32]) -> Vec<f32> {
|
||||
grad.iter().map(|g| -self.lambda * g).collect()
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DomainFactorizer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Splits body-part features into pose-relevant (`h_pose`) and
|
||||
/// environment-specific (`h_env`) representations.
|
||||
///
|
||||
/// - **PoseEncoder**: per-part `Linear(64,128) -> LayerNorm -> GELU -> Linear(128,64)`
|
||||
/// - **EnvEncoder**: `GlobalMeanPool(17x64->64) -> Linear(64,32)`
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DomainFactorizer {
|
||||
/// Pose encoder FC1.
|
||||
pub pose_fc1: Linear,
|
||||
/// Pose encoder FC2.
|
||||
pub pose_fc2: Linear,
|
||||
/// Environment encoder FC.
|
||||
pub env_fc: Linear,
|
||||
/// Number of body parts.
|
||||
pub n_parts: usize,
|
||||
/// Feature dim per part.
|
||||
pub part_dim: usize,
|
||||
}
|
||||
|
||||
impl DomainFactorizer {
|
||||
/// Create with `n_parts` body parts of `part_dim` features each.
|
||||
pub fn new(n_parts: usize, part_dim: usize) -> Self {
|
||||
Self {
|
||||
pose_fc1: Linear::new(part_dim, 128),
|
||||
pose_fc2: Linear::new(128, part_dim),
|
||||
env_fc: Linear::new(part_dim, 32),
|
||||
n_parts, part_dim,
|
||||
}
|
||||
}
|
||||
|
||||
/// Factorize into `(h_pose [n_parts*part_dim], h_env [32])`.
|
||||
pub fn factorize(&self, body_part_features: &[f32]) -> (Vec<f32>, Vec<f32>) {
|
||||
let expected = self.n_parts * self.part_dim;
|
||||
assert_eq!(body_part_features.len(), expected);
|
||||
|
||||
let mut h_pose = Vec::with_capacity(expected);
|
||||
for i in 0..self.n_parts {
|
||||
let off = i * self.part_dim;
|
||||
let part = &body_part_features[off..off + self.part_dim];
|
||||
let z = self.pose_fc1.forward(part);
|
||||
let z = layer_norm(&z);
|
||||
let z: Vec<f32> = z.iter().map(|v| gelu(*v)).collect();
|
||||
let z = self.pose_fc2.forward(&z);
|
||||
h_pose.extend_from_slice(&z);
|
||||
}
|
||||
|
||||
let pooled = global_mean_pool(body_part_features, self.n_parts, self.part_dim);
|
||||
let h_env = self.env_fc.forward(&pooled);
|
||||
(h_pose, h_env)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DomainClassifier
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Predicts which environment a sample came from.
|
||||
///
|
||||
/// `MeanPool(17x64->64) -> Linear(64,32) -> ReLU -> Linear(32, n_domains)`
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DomainClassifier {
|
||||
/// Hidden layer.
|
||||
pub fc1: Linear,
|
||||
/// Output layer.
|
||||
pub fc2: Linear,
|
||||
/// Number of body parts for mean pooling.
|
||||
pub n_parts: usize,
|
||||
/// Feature dim per part.
|
||||
pub part_dim: usize,
|
||||
/// Number of domain classes.
|
||||
pub n_domains: usize,
|
||||
}
|
||||
|
||||
impl DomainClassifier {
|
||||
/// Create a domain classifier for `n_domains` environments.
|
||||
pub fn new(n_parts: usize, part_dim: usize, n_domains: usize) -> Self {
|
||||
Self {
|
||||
fc1: Linear::new(part_dim, 32),
|
||||
fc2: Linear::new(32, n_domains),
|
||||
n_parts, part_dim, n_domains,
|
||||
}
|
||||
}
|
||||
|
||||
/// Classify: returns raw domain logits of length `n_domains`.
|
||||
pub fn classify(&self, h_pose: &[f32]) -> Vec<f32> {
|
||||
assert_eq!(h_pose.len(), self.n_parts * self.part_dim);
|
||||
let pooled = global_mean_pool(h_pose, self.n_parts, self.part_dim);
|
||||
let z = relu_vec(&self.fc1.forward(&pooled));
|
||||
self.fc2.forward(&z)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AdversarialSchedule
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Lambda annealing: `lambda(p) = 2 / (1 + exp(-10p)) - 1`, p = epoch/max_epochs.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct AdversarialSchedule {
|
||||
/// Maximum training epochs.
|
||||
pub max_epochs: usize,
|
||||
}
|
||||
|
||||
impl AdversarialSchedule {
|
||||
/// Create schedule.
|
||||
pub fn new(max_epochs: usize) -> Self {
|
||||
assert!(max_epochs > 0);
|
||||
Self { max_epochs }
|
||||
}
|
||||
|
||||
/// Compute lambda for `epoch`. Returns value in [0, 1].
|
||||
pub fn lambda(&self, epoch: usize) -> f32 {
|
||||
let p = epoch as f64 / self.max_epochs as f64;
|
||||
(2.0 / (1.0 + (-10.0 * p).exp()) - 1.0) as f32
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn grl_forward_is_identity() {
|
||||
let grl = GradientReversalLayer::new(0.5);
|
||||
let x = vec![1.0, -2.0, 3.0, 0.0, -0.5];
|
||||
assert_eq!(grl.forward(&x), x);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grl_backward_negates_with_lambda() {
|
||||
let grl = GradientReversalLayer::new(0.7);
|
||||
let grad = vec![1.0, -2.0, 3.0, 0.0, 4.0];
|
||||
let rev = grl.backward(&grad);
|
||||
for (r, g) in rev.iter().zip(&grad) {
|
||||
assert!((r - (-0.7 * g)).abs() < 1e-6);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grl_lambda_zero_gives_zero_grad() {
|
||||
let rev = GradientReversalLayer::new(0.0).backward(&[1.0, 2.0, 3.0]);
|
||||
assert!(rev.iter().all(|v| v.abs() < 1e-7));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn factorizer_output_dimensions() {
|
||||
let f = DomainFactorizer::new(17, 64);
|
||||
let (h_pose, h_env) = f.factorize(&vec![0.1; 17 * 64]);
|
||||
assert_eq!(h_pose.len(), 17 * 64, "h_pose should be 17*64");
|
||||
assert_eq!(h_env.len(), 32, "h_env should be 32");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn factorizer_values_finite() {
|
||||
let f = DomainFactorizer::new(17, 64);
|
||||
let (hp, he) = f.factorize(&vec![0.5; 17 * 64]);
|
||||
assert!(hp.iter().all(|v| v.is_finite()));
|
||||
assert!(he.iter().all(|v| v.is_finite()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classifier_output_equals_n_domains() {
|
||||
for nd in [1, 3, 5, 8] {
|
||||
let c = DomainClassifier::new(17, 64, nd);
|
||||
let logits = c.classify(&vec![0.1; 17 * 64]);
|
||||
assert_eq!(logits.len(), nd);
|
||||
assert!(logits.iter().all(|v| v.is_finite()));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn schedule_lambda_zero_approx_zero() {
|
||||
let s = AdversarialSchedule::new(100);
|
||||
assert!(s.lambda(0).abs() < 0.01, "lambda(0) ~ 0");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn schedule_lambda_at_half() {
|
||||
let s = AdversarialSchedule::new(100);
|
||||
// p=0.5 => 2/(1+exp(-5))-1 ≈ 0.9866
|
||||
let lam = s.lambda(50);
|
||||
assert!((lam - 0.9866).abs() < 0.02, "lambda(0.5)~0.987, got {lam}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn schedule_lambda_one_approx_one() {
|
||||
let s = AdversarialSchedule::new(100);
|
||||
assert!((s.lambda(100) - 1.0).abs() < 0.001, "lambda(1.0) ~ 1");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn schedule_monotonically_increasing() {
|
||||
let s = AdversarialSchedule::new(100);
|
||||
let mut prev = s.lambda(0);
|
||||
for e in 1..=100 {
|
||||
let cur = s.lambda(e);
|
||||
assert!(cur >= prev - 1e-7, "not monotone at epoch {e}");
|
||||
prev = cur;
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn gelu_reference_values() {
|
||||
assert!(gelu(0.0).abs() < 1e-6, "gelu(0)=0");
|
||||
assert!((gelu(1.0) - 0.8412).abs() < 0.01, "gelu(1)~0.841");
|
||||
assert!((gelu(-1.0) + 0.1588).abs() < 0.01, "gelu(-1)~-0.159");
|
||||
assert!(gelu(5.0) > 4.5, "gelu(5)~5");
|
||||
assert!(gelu(-5.0).abs() < 0.01, "gelu(-5)~0");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn layer_norm_zero_mean_unit_var() {
|
||||
let normed = layer_norm(&[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0]);
|
||||
let n = normed.len() as f32;
|
||||
let mean = normed.iter().sum::<f32>() / n;
|
||||
let var = normed.iter().map(|v| (v - mean).powi(2)).sum::<f32>() / n;
|
||||
assert!(mean.abs() < 1e-5, "mean~0, got {mean}");
|
||||
assert!((var - 1.0).abs() < 0.01, "var~1, got {var}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn layer_norm_constant_gives_zeros() {
|
||||
let normed = layer_norm(&vec![3.0; 16]);
|
||||
assert!(normed.iter().all(|v| v.abs() < 1e-4));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn layer_norm_empty() {
|
||||
assert!(layer_norm(&[]).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mean_pool_simple() {
|
||||
let p = global_mean_pool(&[1.0, 2.0, 3.0, 5.0, 6.0, 7.0], 2, 3);
|
||||
assert!((p[0] - 3.0).abs() < 1e-6);
|
||||
assert!((p[1] - 4.0).abs() < 1e-6);
|
||||
assert!((p[2] - 5.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn linear_dimensions_and_finite() {
|
||||
let l = Linear::new(64, 128);
|
||||
let out = l.forward(&vec![0.1; 64]);
|
||||
assert_eq!(out.len(), 128);
|
||||
assert!(out.iter().all(|v| v.is_finite()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn full_pipeline() {
|
||||
let fact = DomainFactorizer::new(17, 64);
|
||||
let grl = GradientReversalLayer::new(0.5);
|
||||
let cls = DomainClassifier::new(17, 64, 4);
|
||||
|
||||
let feat = vec![0.2_f32; 17 * 64];
|
||||
let (hp, he) = fact.factorize(&feat);
|
||||
assert_eq!(hp.len(), 17 * 64);
|
||||
assert_eq!(he.len(), 32);
|
||||
|
||||
let hp_grl = grl.forward(&hp);
|
||||
assert_eq!(hp_grl, hp);
|
||||
|
||||
let logits = cls.classify(&hp_grl);
|
||||
assert_eq!(logits.len(), 4);
|
||||
assert!(logits.iter().all(|v| v.is_finite()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
//! Cross-domain evaluation metrics (MERIDIAN Phase 6).
|
||||
//!
|
||||
//! MPJPE, domain gap ratio, and adaptation speedup for measuring how well a
|
||||
//! WiFi-DensePose model generalizes across environments and hardware.
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// Aggregated cross-domain evaluation metrics.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CrossDomainMetrics {
|
||||
/// In-domain (source) MPJPE (mm).
|
||||
pub in_domain_mpjpe: f32,
|
||||
/// Cross-domain (unseen environment) MPJPE (mm).
|
||||
pub cross_domain_mpjpe: f32,
|
||||
/// MPJPE after few-shot adaptation (mm).
|
||||
pub few_shot_mpjpe: f32,
|
||||
/// MPJPE across different WiFi hardware (mm).
|
||||
pub cross_hardware_mpjpe: f32,
|
||||
/// cross-domain / in-domain MPJPE. Target: < 1.5.
|
||||
pub domain_gap_ratio: f32,
|
||||
/// Labelled-sample savings vs training from scratch.
|
||||
pub adaptation_speedup: f32,
|
||||
}
|
||||
|
||||
/// Evaluates pose estimation across multiple domains.
|
||||
///
|
||||
/// Domain 0 = in-domain (source); other IDs = cross-domain.
|
||||
///
|
||||
/// ```rust
|
||||
/// use wifi_densepose_train::eval::{CrossDomainEvaluator, mpjpe};
|
||||
/// let ev = CrossDomainEvaluator::new(17);
|
||||
/// let preds = vec![(vec![0.0_f32; 51], vec![0.0_f32; 51])];
|
||||
/// let m = ev.evaluate(&preds, &[0]);
|
||||
/// assert!(m.in_domain_mpjpe >= 0.0);
|
||||
/// ```
|
||||
pub struct CrossDomainEvaluator {
|
||||
n_joints: usize,
|
||||
}
|
||||
|
||||
impl CrossDomainEvaluator {
|
||||
/// Create evaluator for `n_joints` body joints (e.g. 17 for COCO).
|
||||
pub fn new(n_joints: usize) -> Self { Self { n_joints } }
|
||||
|
||||
/// Evaluate predictions grouped by domain. Each pair is (predicted, gt)
|
||||
/// with `n_joints * 3` floats. `domain_labels` must match length.
|
||||
pub fn evaluate(&self, predictions: &[(Vec<f32>, Vec<f32>)], domain_labels: &[u32]) -> CrossDomainMetrics {
|
||||
assert_eq!(predictions.len(), domain_labels.len(), "length mismatch");
|
||||
let mut by_dom: HashMap<u32, Vec<f32>> = HashMap::new();
|
||||
for (i, (p, g)) in predictions.iter().enumerate() {
|
||||
by_dom.entry(domain_labels[i]).or_default().push(mpjpe(p, g, self.n_joints));
|
||||
}
|
||||
let in_dom = mean_of(by_dom.get(&0));
|
||||
let cross_errs: Vec<f32> = by_dom.iter().filter(|(&d, _)| d != 0).flat_map(|(_, e)| e.iter().copied()).collect();
|
||||
let cross_dom = if cross_errs.is_empty() { 0.0 } else { cross_errs.iter().sum::<f32>() / cross_errs.len() as f32 };
|
||||
let few_shot = if by_dom.contains_key(&2) { mean_of(by_dom.get(&2)) } else { (in_dom + cross_dom) / 2.0 };
|
||||
let cross_hw = if by_dom.contains_key(&3) { mean_of(by_dom.get(&3)) } else { cross_dom };
|
||||
let gap = if in_dom > 1e-10 { cross_dom / in_dom } else if cross_dom > 1e-10 { f32::INFINITY } else { 1.0 };
|
||||
let speedup = if few_shot > 1e-10 { cross_dom / few_shot } else { 1.0 };
|
||||
CrossDomainMetrics { in_domain_mpjpe: in_dom, cross_domain_mpjpe: cross_dom, few_shot_mpjpe: few_shot,
|
||||
cross_hardware_mpjpe: cross_hw, domain_gap_ratio: gap, adaptation_speedup: speedup }
|
||||
}
|
||||
}
|
||||
|
||||
/// Mean Per Joint Position Error: average Euclidean distance across `n_joints`.
|
||||
///
|
||||
/// `pred` and `gt` are flat `[n_joints * 3]` (x, y, z per joint).
|
||||
pub fn mpjpe(pred: &[f32], gt: &[f32], n_joints: usize) -> f32 {
|
||||
if n_joints == 0 { return 0.0; }
|
||||
let total: f32 = (0..n_joints).map(|j| {
|
||||
let b = j * 3;
|
||||
let d = |off| pred.get(b + off).copied().unwrap_or(0.0) - gt.get(b + off).copied().unwrap_or(0.0);
|
||||
(d(0).powi(2) + d(1).powi(2) + d(2).powi(2)).sqrt()
|
||||
}).sum();
|
||||
total / n_joints as f32
|
||||
}
|
||||
|
||||
fn mean_of(v: Option<&Vec<f32>>) -> f32 {
|
||||
match v { Some(e) if !e.is_empty() => e.iter().sum::<f32>() / e.len() as f32, _ => 0.0 }
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn mpjpe_known_value() {
|
||||
assert!((mpjpe(&[0.0, 0.0, 0.0], &[3.0, 4.0, 0.0], 1) - 5.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mpjpe_two_joints() {
|
||||
// Joint 0: dist=5, Joint 1: dist=0 -> mean=2.5
|
||||
assert!((mpjpe(&[0.0,0.0,0.0, 1.0,1.0,1.0], &[3.0,4.0,0.0, 1.0,1.0,1.0], 2) - 2.5).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mpjpe_zero_when_identical() {
|
||||
let c = vec![1.5, 2.3, 0.7, 4.1, 5.9, 3.2];
|
||||
assert!(mpjpe(&c, &c, 2).abs() < 1e-10);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mpjpe_zero_joints() { assert_eq!(mpjpe(&[], &[], 0), 0.0); }
|
||||
|
||||
#[test]
|
||||
fn domain_gap_ratio_computed() {
|
||||
let ev = CrossDomainEvaluator::new(1);
|
||||
let preds = vec![
|
||||
(vec![0.0,0.0,0.0], vec![1.0,0.0,0.0]), // dom 0, err=1
|
||||
(vec![0.0,0.0,0.0], vec![2.0,0.0,0.0]), // dom 1, err=2
|
||||
];
|
||||
let m = ev.evaluate(&preds, &[0, 1]);
|
||||
assert!((m.in_domain_mpjpe - 1.0).abs() < 1e-6);
|
||||
assert!((m.cross_domain_mpjpe - 2.0).abs() < 1e-6);
|
||||
assert!((m.domain_gap_ratio - 2.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn evaluate_groups_by_domain() {
|
||||
let ev = CrossDomainEvaluator::new(1);
|
||||
let preds = vec![
|
||||
(vec![0.0,0.0,0.0], vec![1.0,0.0,0.0]),
|
||||
(vec![0.0,0.0,0.0], vec![3.0,0.0,0.0]),
|
||||
(vec![0.0,0.0,0.0], vec![5.0,0.0,0.0]),
|
||||
];
|
||||
let m = ev.evaluate(&preds, &[0, 0, 1]);
|
||||
assert!((m.in_domain_mpjpe - 2.0).abs() < 1e-6);
|
||||
assert!((m.cross_domain_mpjpe - 5.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn domain_gap_perfect() {
|
||||
let ev = CrossDomainEvaluator::new(1);
|
||||
let preds = vec![(vec![1.0,2.0,3.0], vec![1.0,2.0,3.0]), (vec![4.0,5.0,6.0], vec![4.0,5.0,6.0])];
|
||||
assert!((ev.evaluate(&preds, &[0, 1]).domain_gap_ratio - 1.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn evaluate_multiple_cross_domains() {
|
||||
let ev = CrossDomainEvaluator::new(1);
|
||||
let preds = vec![
|
||||
(vec![0.0,0.0,0.0], vec![1.0,0.0,0.0]),
|
||||
(vec![0.0,0.0,0.0], vec![4.0,0.0,0.0]),
|
||||
(vec![0.0,0.0,0.0], vec![6.0,0.0,0.0]),
|
||||
];
|
||||
let m = ev.evaluate(&preds, &[0, 1, 3]);
|
||||
assert!((m.in_domain_mpjpe - 1.0).abs() < 1e-6);
|
||||
assert!((m.cross_domain_mpjpe - 5.0).abs() < 1e-6);
|
||||
assert!((m.cross_hardware_mpjpe - 6.0).abs() < 1e-6);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,365 @@
|
||||
//! MERIDIAN Phase 3 -- Geometry Encoder with FiLM Conditioning (ADR-027).
|
||||
//!
|
||||
//! Permutation-invariant encoding of AP positions into a 64-dim geometry
|
||||
//! vector, plus FiLM layers for conditioning backbone features on room
|
||||
//! geometry. Pure Rust, no external dependencies beyond the workspace.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
const GEOMETRY_DIM: usize = 64;
|
||||
const NUM_COORDS: usize = 3;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Linear layer (pure Rust)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Fully-connected layer: `y = x W^T + b`. Row-major weights `[out, in]`.
|
||||
#[derive(Debug, Clone)]
|
||||
struct Linear {
|
||||
weights: Vec<f32>,
|
||||
bias: Vec<f32>,
|
||||
in_f: usize,
|
||||
out_f: usize,
|
||||
}
|
||||
|
||||
impl Linear {
|
||||
/// Kaiming-uniform init: U(-k, k), k = sqrt(1/in_f).
|
||||
fn new(in_f: usize, out_f: usize, seed: u64) -> Self {
|
||||
let k = (1.0 / in_f as f32).sqrt();
|
||||
Linear {
|
||||
weights: det_uniform(in_f * out_f, -k, k, seed),
|
||||
bias: vec![0.0; out_f],
|
||||
in_f,
|
||||
out_f,
|
||||
}
|
||||
}
|
||||
|
||||
fn forward(&self, x: &[f32]) -> Vec<f32> {
|
||||
debug_assert_eq!(x.len(), self.in_f);
|
||||
let mut y = self.bias.clone();
|
||||
for j in 0..self.out_f {
|
||||
let off = j * self.in_f;
|
||||
let mut s = 0.0f32;
|
||||
for i in 0..self.in_f {
|
||||
s += x[i] * self.weights[off + i];
|
||||
}
|
||||
y[j] += s;
|
||||
}
|
||||
y
|
||||
}
|
||||
}
|
||||
|
||||
/// Deterministic xorshift64 uniform in `[lo, hi)`.
|
||||
/// Uses 24-bit precision (matching f32 mantissa) for uniform distribution.
|
||||
fn det_uniform(n: usize, lo: f32, hi: f32, seed: u64) -> Vec<f32> {
|
||||
let r = hi - lo;
|
||||
let mut s = seed.wrapping_add(0x9E37_79B9_7F4A_7C15);
|
||||
(0..n)
|
||||
.map(|_| {
|
||||
s ^= s << 13;
|
||||
s ^= s >> 7;
|
||||
s ^= s << 17;
|
||||
lo + (s >> 40) as f32 / (1u64 << 24) as f32 * r
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn relu(v: &mut [f32]) {
|
||||
for x in v.iter_mut() {
|
||||
if *x < 0.0 { *x = 0.0; }
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MeridianGeometryConfig
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Configuration for the MERIDIAN geometry encoder and FiLM layers.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct MeridianGeometryConfig {
|
||||
/// Number of Fourier frequency bands (default 10).
|
||||
pub n_frequencies: usize,
|
||||
/// Spatial scale factor, 1.0 = metres (default 1.0).
|
||||
pub scale: f32,
|
||||
/// Output embedding dimension (default 64).
|
||||
pub geometry_dim: usize,
|
||||
/// Random seed for weight init (default 42).
|
||||
pub seed: u64,
|
||||
}
|
||||
|
||||
impl Default for MeridianGeometryConfig {
|
||||
fn default() -> Self {
|
||||
MeridianGeometryConfig { n_frequencies: 10, scale: 1.0, geometry_dim: GEOMETRY_DIM, seed: 42 }
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// FourierPositionalEncoding
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Fourier positional encoding for 3-D coordinates.
|
||||
///
|
||||
/// Per coordinate: `[sin(2^0*pi*x), cos(2^0*pi*x), ..., sin(2^(L-1)*pi*x),
|
||||
/// cos(2^(L-1)*pi*x)]`. Zero-padded to `geometry_dim`.
|
||||
pub struct FourierPositionalEncoding {
|
||||
n_frequencies: usize,
|
||||
scale: f32,
|
||||
output_dim: usize,
|
||||
}
|
||||
|
||||
impl FourierPositionalEncoding {
|
||||
/// Create from config.
|
||||
pub fn new(cfg: &MeridianGeometryConfig) -> Self {
|
||||
FourierPositionalEncoding { n_frequencies: cfg.n_frequencies, scale: cfg.scale, output_dim: cfg.geometry_dim }
|
||||
}
|
||||
|
||||
/// Encode `[x, y, z]` into a fixed-length vector of `geometry_dim` elements.
|
||||
pub fn encode(&self, coords: &[f32; 3]) -> Vec<f32> {
|
||||
let raw = NUM_COORDS * 2 * self.n_frequencies;
|
||||
let mut enc = Vec::with_capacity(raw.max(self.output_dim));
|
||||
for &c in coords {
|
||||
let sc = c * self.scale;
|
||||
for l in 0..self.n_frequencies {
|
||||
let f = (2.0f32).powi(l as i32) * std::f32::consts::PI * sc;
|
||||
enc.push(f.sin());
|
||||
enc.push(f.cos());
|
||||
}
|
||||
}
|
||||
enc.resize(self.output_dim, 0.0);
|
||||
enc
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DeepSets
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Permutation-invariant set encoder: phi each element, mean-pool, then rho.
|
||||
pub struct DeepSets {
|
||||
phi: Linear,
|
||||
rho: Linear,
|
||||
dim: usize,
|
||||
}
|
||||
|
||||
impl DeepSets {
|
||||
/// Create from config.
|
||||
pub fn new(cfg: &MeridianGeometryConfig) -> Self {
|
||||
let d = cfg.geometry_dim;
|
||||
DeepSets { phi: Linear::new(d, d, cfg.seed.wrapping_add(1)), rho: Linear::new(d, d, cfg.seed.wrapping_add(2)), dim: d }
|
||||
}
|
||||
|
||||
/// Encode a set of embeddings (each of length `geometry_dim`) into one vector.
|
||||
pub fn encode(&self, ap_embeddings: &[Vec<f32>]) -> Vec<f32> {
|
||||
assert!(!ap_embeddings.is_empty(), "DeepSets: input set must be non-empty");
|
||||
let n = ap_embeddings.len() as f32;
|
||||
let mut pooled = vec![0.0f32; self.dim];
|
||||
for emb in ap_embeddings {
|
||||
debug_assert_eq!(emb.len(), self.dim);
|
||||
let mut t = self.phi.forward(emb);
|
||||
relu(&mut t);
|
||||
for (p, v) in pooled.iter_mut().zip(t.iter()) { *p += *v; }
|
||||
}
|
||||
for p in pooled.iter_mut() { *p /= n; }
|
||||
let mut out = self.rho.forward(&pooled);
|
||||
relu(&mut out);
|
||||
out
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GeometryEncoder
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// End-to-end encoder: AP positions -> 64-dim geometry vector.
|
||||
pub struct GeometryEncoder {
|
||||
pos_embed: FourierPositionalEncoding,
|
||||
set_encoder: DeepSets,
|
||||
}
|
||||
|
||||
impl GeometryEncoder {
|
||||
/// Build from config.
|
||||
pub fn new(cfg: &MeridianGeometryConfig) -> Self {
|
||||
GeometryEncoder { pos_embed: FourierPositionalEncoding::new(cfg), set_encoder: DeepSets::new(cfg) }
|
||||
}
|
||||
|
||||
/// Encode variable-count AP positions `[x,y,z]` into a fixed-dim vector.
|
||||
pub fn encode(&self, ap_positions: &[[f32; 3]]) -> Vec<f32> {
|
||||
let embs: Vec<Vec<f32>> = ap_positions.iter().map(|p| self.pos_embed.encode(p)).collect();
|
||||
self.set_encoder.encode(&embs)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// FilmLayer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Feature-wise Linear Modulation: `output = gamma(g) * h + beta(g)`.
|
||||
pub struct FilmLayer {
|
||||
gamma_proj: Linear,
|
||||
beta_proj: Linear,
|
||||
}
|
||||
|
||||
impl FilmLayer {
|
||||
/// Create a FiLM layer. Gamma bias is initialised to 1.0 (identity).
|
||||
pub fn new(cfg: &MeridianGeometryConfig) -> Self {
|
||||
let d = cfg.geometry_dim;
|
||||
let mut gamma_proj = Linear::new(d, d, cfg.seed.wrapping_add(3));
|
||||
for b in gamma_proj.bias.iter_mut() { *b = 1.0; }
|
||||
FilmLayer { gamma_proj, beta_proj: Linear::new(d, d, cfg.seed.wrapping_add(4)) }
|
||||
}
|
||||
|
||||
/// Modulate `features` by `geometry`: `gamma(geometry) * features + beta(geometry)`.
|
||||
pub fn modulate(&self, features: &[f32], geometry: &[f32]) -> Vec<f32> {
|
||||
let gamma = self.gamma_proj.forward(geometry);
|
||||
let beta = self.beta_proj.forward(geometry);
|
||||
features.iter().zip(gamma.iter()).zip(beta.iter()).map(|((&f, &g), &b)| g * f + b).collect()
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn cfg() -> MeridianGeometryConfig { MeridianGeometryConfig::default() }
|
||||
|
||||
#[test]
|
||||
fn fourier_output_dimension_is_64() {
|
||||
let c = cfg();
|
||||
let out = FourierPositionalEncoding::new(&c).encode(&[1.0, 2.0, 3.0]);
|
||||
assert_eq!(out.len(), c.geometry_dim);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fourier_different_coords_different_outputs() {
|
||||
let enc = FourierPositionalEncoding::new(&cfg());
|
||||
let a = enc.encode(&[0.0, 0.0, 0.0]);
|
||||
let b = enc.encode(&[1.0, 0.0, 0.0]);
|
||||
let c = enc.encode(&[0.0, 1.0, 0.0]);
|
||||
let d = enc.encode(&[0.0, 0.0, 1.0]);
|
||||
assert_ne!(a, b); assert_ne!(a, c); assert_ne!(a, d); assert_ne!(b, c);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fourier_values_bounded() {
|
||||
let out = FourierPositionalEncoding::new(&cfg()).encode(&[5.5, -3.2, 0.1]);
|
||||
for &v in &out { assert!(v.abs() <= 1.0 + 1e-6, "got {v}"); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deepsets_permutation_invariant() {
|
||||
let c = cfg();
|
||||
let enc = FourierPositionalEncoding::new(&c);
|
||||
let ds = DeepSets::new(&c);
|
||||
let (a, b, d) = (enc.encode(&[1.0,0.0,0.0]), enc.encode(&[0.0,2.0,0.0]), enc.encode(&[0.0,0.0,3.0]));
|
||||
let abc = ds.encode(&[a.clone(), b.clone(), d.clone()]);
|
||||
let cba = ds.encode(&[d.clone(), b.clone(), a.clone()]);
|
||||
let bac = ds.encode(&[b.clone(), a.clone(), d.clone()]);
|
||||
for i in 0..c.geometry_dim {
|
||||
assert!((abc[i] - cba[i]).abs() < 1e-5, "dim {i}: abc={} cba={}", abc[i], cba[i]);
|
||||
assert!((abc[i] - bac[i]).abs() < 1e-5, "dim {i}: abc={} bac={}", abc[i], bac[i]);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deepsets_variable_ap_count() {
|
||||
let c = cfg();
|
||||
let enc = FourierPositionalEncoding::new(&c);
|
||||
let ds = DeepSets::new(&c);
|
||||
let one = ds.encode(&[enc.encode(&[1.0,0.0,0.0])]);
|
||||
assert_eq!(one.len(), c.geometry_dim);
|
||||
let three = ds.encode(&[enc.encode(&[1.0,0.0,0.0]), enc.encode(&[0.0,2.0,0.0]), enc.encode(&[0.0,0.0,3.0])]);
|
||||
assert_eq!(three.len(), c.geometry_dim);
|
||||
let six = ds.encode(&[
|
||||
enc.encode(&[1.0,0.0,0.0]), enc.encode(&[0.0,2.0,0.0]), enc.encode(&[0.0,0.0,3.0]),
|
||||
enc.encode(&[-1.0,0.0,0.0]), enc.encode(&[0.0,-2.0,0.0]), enc.encode(&[0.0,0.0,-3.0]),
|
||||
]);
|
||||
assert_eq!(six.len(), c.geometry_dim);
|
||||
assert_ne!(one, three); assert_ne!(three, six);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn geometry_encoder_end_to_end() {
|
||||
let c = cfg();
|
||||
let g = GeometryEncoder::new(&c).encode(&[[1.0,0.0,2.5],[0.0,3.0,2.5],[-2.0,1.0,2.5]]);
|
||||
assert_eq!(g.len(), c.geometry_dim);
|
||||
for &v in &g { assert!(v.is_finite()); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn geometry_encoder_single_ap() {
|
||||
let c = cfg();
|
||||
assert_eq!(GeometryEncoder::new(&c).encode(&[[0.0,0.0,0.0]]).len(), c.geometry_dim);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn film_identity_when_geometry_zero() {
|
||||
let c = cfg();
|
||||
let film = FilmLayer::new(&c);
|
||||
let feat = vec![1.0f32; c.geometry_dim];
|
||||
let out = film.modulate(&feat, &vec![0.0f32; c.geometry_dim]);
|
||||
assert_eq!(out.len(), c.geometry_dim);
|
||||
// gamma_proj(0) = bias = [1.0], beta_proj(0) = bias = [0.0] => identity
|
||||
for i in 0..c.geometry_dim {
|
||||
assert!((out[i] - feat[i]).abs() < 1e-5, "dim {i}: expected {}, got {}", feat[i], out[i]);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn film_nontrivial_modulation() {
|
||||
let c = cfg();
|
||||
let film = FilmLayer::new(&c);
|
||||
let feat: Vec<f32> = (0..c.geometry_dim).map(|i| i as f32 * 0.1).collect();
|
||||
let geom: Vec<f32> = (0..c.geometry_dim).map(|i| (i as f32 - 32.0) * 0.01).collect();
|
||||
let out = film.modulate(&feat, &geom);
|
||||
assert_eq!(out.len(), c.geometry_dim);
|
||||
assert!(out.iter().zip(feat.iter()).any(|(o, f)| (o - f).abs() > 1e-6));
|
||||
for &v in &out { assert!(v.is_finite()); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn film_explicit_gamma_beta() {
|
||||
let c = MeridianGeometryConfig { geometry_dim: 4, ..cfg() };
|
||||
let mut film = FilmLayer::new(&c);
|
||||
film.gamma_proj.weights = vec![0.0; 16];
|
||||
film.gamma_proj.bias = vec![2.0, 3.0, 0.5, 1.0];
|
||||
film.beta_proj.weights = vec![0.0; 16];
|
||||
film.beta_proj.bias = vec![10.0, 20.0, 30.0, 40.0];
|
||||
let out = film.modulate(&[1.0, 2.0, 3.0, 4.0], &[999.0; 4]);
|
||||
let exp = [12.0, 26.0, 31.5, 44.0];
|
||||
for i in 0..4 { assert!((out[i] - exp[i]).abs() < 1e-5, "dim {i}"); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_defaults() {
|
||||
let c = MeridianGeometryConfig::default();
|
||||
assert_eq!(c.n_frequencies, 10);
|
||||
assert!((c.scale - 1.0).abs() < 1e-6);
|
||||
assert_eq!(c.geometry_dim, 64);
|
||||
assert_eq!(c.seed, 42);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_serde_round_trip() {
|
||||
let c = MeridianGeometryConfig { n_frequencies: 8, scale: 0.5, geometry_dim: 32, seed: 123 };
|
||||
let j = serde_json::to_string(&c).unwrap();
|
||||
let d: MeridianGeometryConfig = serde_json::from_str(&j).unwrap();
|
||||
assert_eq!(d.n_frequencies, 8); assert!((d.scale - 0.5).abs() < 1e-6);
|
||||
assert_eq!(d.geometry_dim, 32); assert_eq!(d.seed, 123);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn linear_forward_dim() {
|
||||
assert_eq!(Linear::new(8, 4, 0).forward(&vec![1.0; 8]).len(), 4);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn linear_zero_input_gives_bias() {
|
||||
let lin = Linear::new(4, 3, 0);
|
||||
let out = lin.forward(&[0.0; 4]);
|
||||
for i in 0..3 { assert!((out[i] - lin.bias[i]).abs() < 1e-6); }
|
||||
}
|
||||
}
|
||||
@@ -45,8 +45,13 @@
|
||||
|
||||
pub mod config;
|
||||
pub mod dataset;
|
||||
pub mod domain;
|
||||
pub mod error;
|
||||
pub mod eval;
|
||||
pub mod geometry;
|
||||
pub mod rapid_adapt;
|
||||
pub mod subcarrier;
|
||||
pub mod virtual_aug;
|
||||
|
||||
// The following modules use `tch` (PyTorch Rust bindings) for GPU-accelerated
|
||||
// training and are only compiled when the `tch-backend` feature is enabled.
|
||||
@@ -72,5 +77,14 @@ pub use error::{ConfigError, DatasetError, SubcarrierError, TrainError};
|
||||
pub use error::TrainResult as TrainResultAlias;
|
||||
pub use subcarrier::{compute_interp_weights, interpolate_subcarriers, select_subcarriers_by_variance};
|
||||
|
||||
// MERIDIAN (ADR-027) re-exports.
|
||||
pub use domain::{
|
||||
AdversarialSchedule, DomainClassifier, DomainFactorizer, GradientReversalLayer,
|
||||
};
|
||||
pub use eval::CrossDomainEvaluator;
|
||||
pub use geometry::{FilmLayer, FourierPositionalEncoding, GeometryEncoder, MeridianGeometryConfig};
|
||||
pub use rapid_adapt::{AdaptError, AdaptationLoss, AdaptationResult, RapidAdaptation};
|
||||
pub use virtual_aug::VirtualDomainAugmentor;
|
||||
|
||||
/// Crate version string.
|
||||
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
//! Few-shot rapid adaptation (MERIDIAN Phase 5).
|
||||
//!
|
||||
//! Test-time training with contrastive learning and entropy minimization on
|
||||
//! unlabeled CSI frames. Produces LoRA weight deltas for new environments.
|
||||
|
||||
/// Loss function(s) for test-time adaptation.
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum AdaptationLoss {
|
||||
/// Contrastive TTT: positive = temporally adjacent, negative = random.
|
||||
ContrastiveTTT { /// Gradient-descent epochs.
|
||||
epochs: usize, /// Learning rate.
|
||||
lr: f32 },
|
||||
/// Minimize entropy of confidence outputs for sharper predictions.
|
||||
EntropyMin { /// Gradient-descent epochs.
|
||||
epochs: usize, /// Learning rate.
|
||||
lr: f32 },
|
||||
/// Both contrastive and entropy losses combined.
|
||||
Combined { /// Gradient-descent epochs.
|
||||
epochs: usize, /// Learning rate.
|
||||
lr: f32, /// Weight for entropy term.
|
||||
lambda_ent: f32 },
|
||||
}
|
||||
|
||||
impl AdaptationLoss {
|
||||
/// Number of epochs for this variant.
|
||||
pub fn epochs(&self) -> usize {
|
||||
match self { Self::ContrastiveTTT { epochs, .. }
|
||||
| Self::EntropyMin { epochs, .. }
|
||||
| Self::Combined { epochs, .. } => *epochs }
|
||||
}
|
||||
/// Learning rate for this variant.
|
||||
pub fn lr(&self) -> f32 {
|
||||
match self { Self::ContrastiveTTT { lr, .. }
|
||||
| Self::EntropyMin { lr, .. }
|
||||
| Self::Combined { lr, .. } => *lr }
|
||||
}
|
||||
}
|
||||
|
||||
/// Result of [`RapidAdaptation::adapt`].
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct AdaptationResult {
|
||||
/// LoRA weight deltas.
|
||||
pub lora_weights: Vec<f32>,
|
||||
/// Final epoch loss.
|
||||
pub final_loss: f32,
|
||||
/// Calibration frames consumed.
|
||||
pub frames_used: usize,
|
||||
/// Epochs executed.
|
||||
pub adaptation_epochs: usize,
|
||||
}
|
||||
|
||||
/// Error type for rapid adaptation.
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum AdaptError {
|
||||
/// Not enough calibration frames.
|
||||
InsufficientFrames {
|
||||
/// Frames currently buffered.
|
||||
have: usize,
|
||||
/// Minimum required.
|
||||
need: usize,
|
||||
},
|
||||
/// LoRA rank must be at least 1.
|
||||
InvalidRank,
|
||||
}
|
||||
|
||||
impl std::fmt::Display for AdaptError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
Self::InsufficientFrames { have, need } =>
|
||||
write!(f, "insufficient calibration frames: have {have}, need at least {need}"),
|
||||
Self::InvalidRank => write!(f, "lora_rank must be >= 1"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::error::Error for AdaptError {}
|
||||
|
||||
/// Few-shot rapid adaptation engine.
|
||||
///
|
||||
/// Accumulates unlabeled CSI calibration frames and runs test-time training
|
||||
/// to produce LoRA weight deltas. Buffer is capped at `max_buffer_frames`
|
||||
/// (default 10 000) to prevent unbounded memory growth.
|
||||
///
|
||||
/// ```rust
|
||||
/// use wifi_densepose_train::rapid_adapt::{RapidAdaptation, AdaptationLoss};
|
||||
/// let loss = AdaptationLoss::Combined { epochs: 5, lr: 0.001, lambda_ent: 0.5 };
|
||||
/// let mut ra = RapidAdaptation::new(10, 4, loss);
|
||||
/// for i in 0..10 { ra.push_frame(&vec![i as f32; 8]); }
|
||||
/// assert!(ra.is_ready());
|
||||
/// let r = ra.adapt().unwrap();
|
||||
/// assert_eq!(r.frames_used, 10);
|
||||
/// ```
|
||||
pub struct RapidAdaptation {
|
||||
/// Minimum frames before adaptation (default 200 = 10 s @ 20 Hz).
|
||||
pub min_calibration_frames: usize,
|
||||
/// LoRA factorization rank (must be >= 1).
|
||||
pub lora_rank: usize,
|
||||
/// Loss variant for test-time training.
|
||||
pub adaptation_loss: AdaptationLoss,
|
||||
/// Maximum buffer size (ring-buffer eviction beyond this cap).
|
||||
pub max_buffer_frames: usize,
|
||||
calibration_buffer: Vec<Vec<f32>>,
|
||||
}
|
||||
|
||||
/// Default maximum calibration buffer size.
|
||||
const DEFAULT_MAX_BUFFER: usize = 10_000;
|
||||
|
||||
impl RapidAdaptation {
|
||||
/// Create a new adaptation engine.
|
||||
pub fn new(min_calibration_frames: usize, lora_rank: usize, adaptation_loss: AdaptationLoss) -> Self {
|
||||
Self { min_calibration_frames, lora_rank, adaptation_loss, max_buffer_frames: DEFAULT_MAX_BUFFER, calibration_buffer: Vec::new() }
|
||||
}
|
||||
/// Push a single unlabeled CSI frame. Evicts oldest frame when buffer is full.
|
||||
pub fn push_frame(&mut self, frame: &[f32]) {
|
||||
if self.calibration_buffer.len() >= self.max_buffer_frames {
|
||||
self.calibration_buffer.remove(0);
|
||||
}
|
||||
self.calibration_buffer.push(frame.to_vec());
|
||||
}
|
||||
/// True when buffer >= min_calibration_frames.
|
||||
pub fn is_ready(&self) -> bool { self.calibration_buffer.len() >= self.min_calibration_frames }
|
||||
/// Number of buffered frames.
|
||||
pub fn buffer_len(&self) -> usize { self.calibration_buffer.len() }
|
||||
|
||||
/// Run test-time adaptation producing LoRA weight deltas.
|
||||
///
|
||||
/// Returns an error if the calibration buffer is empty or lora_rank is 0.
|
||||
pub fn adapt(&self) -> Result<AdaptationResult, AdaptError> {
|
||||
if self.calibration_buffer.is_empty() {
|
||||
return Err(AdaptError::InsufficientFrames { have: 0, need: 1 });
|
||||
}
|
||||
if self.lora_rank == 0 {
|
||||
return Err(AdaptError::InvalidRank);
|
||||
}
|
||||
let (n, fdim) = (self.calibration_buffer.len(), self.calibration_buffer[0].len());
|
||||
let lora_sz = 2 * fdim * self.lora_rank;
|
||||
let mut w = vec![0.01_f32; lora_sz];
|
||||
let (epochs, lr) = (self.adaptation_loss.epochs(), self.adaptation_loss.lr());
|
||||
let mut final_loss = 0.0_f32;
|
||||
for _ in 0..epochs {
|
||||
let mut g = vec![0.0_f32; lora_sz];
|
||||
let loss = match &self.adaptation_loss {
|
||||
AdaptationLoss::ContrastiveTTT { .. } => self.contrastive_step(&w, fdim, &mut g),
|
||||
AdaptationLoss::EntropyMin { .. } => self.entropy_step(&w, fdim, &mut g),
|
||||
AdaptationLoss::Combined { lambda_ent, .. } => {
|
||||
let cl = self.contrastive_step(&w, fdim, &mut g);
|
||||
let mut eg = vec![0.0_f32; lora_sz];
|
||||
let el = self.entropy_step(&w, fdim, &mut eg);
|
||||
for (gi, egi) in g.iter_mut().zip(eg.iter()) { *gi += lambda_ent * egi; }
|
||||
cl + lambda_ent * el
|
||||
}
|
||||
};
|
||||
for (wi, gi) in w.iter_mut().zip(g.iter()) { *wi -= lr * gi; }
|
||||
final_loss = loss;
|
||||
}
|
||||
Ok(AdaptationResult { lora_weights: w, final_loss, frames_used: n, adaptation_epochs: epochs })
|
||||
}
|
||||
|
||||
fn contrastive_step(&self, w: &[f32], fdim: usize, grad: &mut [f32]) -> f32 {
|
||||
let n = self.calibration_buffer.len();
|
||||
if n < 2 { return 0.0; }
|
||||
let (margin, pairs) = (1.0_f32, n - 1);
|
||||
let mut total = 0.0_f32;
|
||||
for i in 0..pairs {
|
||||
let (anc, pos) = (&self.calibration_buffer[i], &self.calibration_buffer[i + 1]);
|
||||
let neg = &self.calibration_buffer[(i + n / 2) % n];
|
||||
let (pa, pp, pn) = (self.project(anc, w, fdim), self.project(pos, w, fdim), self.project(neg, w, fdim));
|
||||
let trip = (l2_dist(&pa, &pp) - l2_dist(&pa, &pn) + margin).max(0.0);
|
||||
total += trip;
|
||||
if trip > 0.0 {
|
||||
for (j, g) in grad.iter_mut().enumerate() {
|
||||
let v = anc.get(j % fdim).copied().unwrap_or(0.0);
|
||||
*g += v * 0.01 / pairs as f32;
|
||||
}
|
||||
}
|
||||
}
|
||||
total / pairs as f32
|
||||
}
|
||||
|
||||
fn entropy_step(&self, w: &[f32], fdim: usize, grad: &mut [f32]) -> f32 {
|
||||
let n = self.calibration_buffer.len();
|
||||
if n == 0 { return 0.0; }
|
||||
let nc = self.lora_rank.max(2);
|
||||
let mut total = 0.0_f32;
|
||||
for frame in &self.calibration_buffer {
|
||||
let proj = self.project(frame, w, fdim);
|
||||
let mut logits = vec![0.0_f32; nc];
|
||||
for (i, &v) in proj.iter().enumerate() { logits[i % nc] += v; }
|
||||
let mx = logits.iter().copied().fold(f32::NEG_INFINITY, f32::max);
|
||||
let exps: Vec<f32> = logits.iter().map(|&l| (l - mx).exp()).collect();
|
||||
let s: f32 = exps.iter().sum();
|
||||
let ent: f32 = exps.iter().map(|&e| { let p = e / s; if p > 1e-10 { -p * p.ln() } else { 0.0 } }).sum();
|
||||
total += ent;
|
||||
for (j, g) in grad.iter_mut().enumerate() {
|
||||
let v = frame.get(j % frame.len().max(1)).copied().unwrap_or(0.0);
|
||||
*g += v * ent * 0.001 / n as f32;
|
||||
}
|
||||
}
|
||||
total / n as f32
|
||||
}
|
||||
|
||||
fn project(&self, frame: &[f32], w: &[f32], fdim: usize) -> Vec<f32> {
|
||||
let rank = self.lora_rank;
|
||||
let mut hidden = vec![0.0_f32; rank];
|
||||
for r in 0..rank {
|
||||
for d in 0..fdim.min(frame.len()) {
|
||||
let idx = d * rank + r;
|
||||
if idx < w.len() { hidden[r] += w[idx] * frame[d]; }
|
||||
}
|
||||
}
|
||||
let boff = fdim * rank;
|
||||
(0..fdim).map(|d| {
|
||||
let lora: f32 = (0..rank).map(|r| {
|
||||
let idx = boff + r * fdim + d;
|
||||
if idx < w.len() { w[idx] * hidden[r] } else { 0.0 }
|
||||
}).sum();
|
||||
frame.get(d).copied().unwrap_or(0.0) + lora
|
||||
}).collect()
|
||||
}
|
||||
}
|
||||
|
||||
fn l2_dist(a: &[f32], b: &[f32]) -> f32 {
|
||||
a.iter().zip(b.iter()).map(|(&x, &y)| (x - y).powi(2)).sum::<f32>().sqrt()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn push_frame_accumulates() {
|
||||
let mut a = RapidAdaptation::new(5, 4, AdaptationLoss::ContrastiveTTT { epochs: 1, lr: 0.01 });
|
||||
assert_eq!(a.buffer_len(), 0);
|
||||
a.push_frame(&[1.0, 2.0]); assert_eq!(a.buffer_len(), 1);
|
||||
a.push_frame(&[3.0, 4.0]); assert_eq!(a.buffer_len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn is_ready_threshold() {
|
||||
let mut a = RapidAdaptation::new(5, 4, AdaptationLoss::EntropyMin { epochs: 3, lr: 0.001 });
|
||||
for i in 0..4 { a.push_frame(&[i as f32; 8]); assert!(!a.is_ready()); }
|
||||
a.push_frame(&[99.0; 8]); assert!(a.is_ready());
|
||||
a.push_frame(&[100.0; 8]); assert!(a.is_ready());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn adapt_lora_weight_dimension() {
|
||||
let (fdim, rank) = (16, 4);
|
||||
let mut a = RapidAdaptation::new(10, rank, AdaptationLoss::ContrastiveTTT { epochs: 3, lr: 0.01 });
|
||||
for i in 0..10 { a.push_frame(&vec![i as f32 * 0.1; fdim]); }
|
||||
let r = a.adapt().unwrap();
|
||||
assert_eq!(r.lora_weights.len(), 2 * fdim * rank);
|
||||
assert_eq!(r.frames_used, 10);
|
||||
assert_eq!(r.adaptation_epochs, 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn contrastive_loss_decreases() {
|
||||
let (fdim, rank) = (32, 4);
|
||||
let mk = |ep| {
|
||||
let mut a = RapidAdaptation::new(20, rank, AdaptationLoss::ContrastiveTTT { epochs: ep, lr: 0.01 });
|
||||
for i in 0..20 { let v = i as f32 * 0.1; a.push_frame(&(0..fdim).map(|d| v + d as f32 * 0.01).collect::<Vec<_>>()); }
|
||||
a.adapt().unwrap().final_loss
|
||||
};
|
||||
assert!(mk(10) <= mk(1) + 1e-6, "10 epochs should yield <= 1 epoch loss");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn combined_loss_adaptation() {
|
||||
let (fdim, rank) = (16, 4);
|
||||
let mut a = RapidAdaptation::new(10, rank, AdaptationLoss::Combined { epochs: 5, lr: 0.001, lambda_ent: 0.5 });
|
||||
for i in 0..10 { a.push_frame(&(0..fdim).map(|d| ((i * fdim + d) as f32).sin()).collect::<Vec<_>>()); }
|
||||
let r = a.adapt().unwrap();
|
||||
assert_eq!(r.frames_used, 10);
|
||||
assert_eq!(r.adaptation_epochs, 5);
|
||||
assert!(r.final_loss.is_finite());
|
||||
assert_eq!(r.lora_weights.len(), 2 * fdim * rank);
|
||||
assert!(r.lora_weights.iter().all(|w| w.is_finite()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn adapt_empty_buffer_returns_error() {
|
||||
let a = RapidAdaptation::new(10, 4, AdaptationLoss::ContrastiveTTT { epochs: 1, lr: 0.01 });
|
||||
assert!(a.adapt().is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn adapt_zero_rank_returns_error() {
|
||||
let mut a = RapidAdaptation::new(1, 0, AdaptationLoss::ContrastiveTTT { epochs: 1, lr: 0.01 });
|
||||
a.push_frame(&[1.0, 2.0]);
|
||||
assert!(a.adapt().is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn buffer_cap_evicts_oldest() {
|
||||
let mut a = RapidAdaptation::new(2, 4, AdaptationLoss::ContrastiveTTT { epochs: 1, lr: 0.01 });
|
||||
a.max_buffer_frames = 3;
|
||||
for i in 0..5 { a.push_frame(&[i as f32]); }
|
||||
assert_eq!(a.buffer_len(), 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn l2_distance_tests() {
|
||||
assert!(l2_dist(&[1.0, 2.0, 3.0], &[1.0, 2.0, 3.0]).abs() < 1e-10);
|
||||
assert!((l2_dist(&[0.0, 0.0], &[3.0, 4.0]) - 5.0).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn loss_accessors() {
|
||||
let c = AdaptationLoss::ContrastiveTTT { epochs: 7, lr: 0.02 };
|
||||
assert_eq!(c.epochs(), 7); assert!((c.lr() - 0.02).abs() < 1e-7);
|
||||
let e = AdaptationLoss::EntropyMin { epochs: 3, lr: 0.1 };
|
||||
assert_eq!(e.epochs(), 3); assert!((e.lr() - 0.1).abs() < 1e-7);
|
||||
let cb = AdaptationLoss::Combined { epochs: 5, lr: 0.001, lambda_ent: 0.3 };
|
||||
assert_eq!(cb.epochs(), 5); assert!((cb.lr() - 0.001).abs() < 1e-7);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,297 @@
|
||||
//! Virtual Domain Augmentation for cross-environment generalization (ADR-027 Phase 4).
|
||||
//!
|
||||
//! Generates synthetic "virtual domains" simulating different physical environments
|
||||
//! and applies domain-specific transformations to CSI amplitude frames for the
|
||||
//! MERIDIAN adversarial training loop.
|
||||
//!
|
||||
//! ```rust
|
||||
//! use wifi_densepose_train::virtual_aug::{VirtualDomainAugmentor, Xorshift64};
|
||||
//!
|
||||
//! let mut aug = VirtualDomainAugmentor::default();
|
||||
//! let mut rng = Xorshift64::new(42);
|
||||
//! let frame = vec![0.5_f32; 56];
|
||||
//! let domain = aug.generate_domain(&mut rng);
|
||||
//! let out = aug.augment_frame(&frame, &domain);
|
||||
//! assert_eq!(out.len(), frame.len());
|
||||
//! ```
|
||||
|
||||
use std::f32::consts::PI;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Xorshift64 PRNG (matches dataset.rs pattern)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Lightweight 64-bit Xorshift PRNG for deterministic augmentation.
|
||||
pub struct Xorshift64 {
|
||||
state: u64,
|
||||
}
|
||||
|
||||
impl Xorshift64 {
|
||||
/// Create a new PRNG. Seed `0` is replaced with a fixed non-zero value.
|
||||
pub fn new(seed: u64) -> Self {
|
||||
Self { state: if seed == 0 { 0x853c49e6748fea9b } else { seed } }
|
||||
}
|
||||
|
||||
/// Advance the state and return the next `u64`.
|
||||
#[inline]
|
||||
pub fn next_u64(&mut self) -> u64 {
|
||||
self.state ^= self.state << 13;
|
||||
self.state ^= self.state >> 7;
|
||||
self.state ^= self.state << 17;
|
||||
self.state
|
||||
}
|
||||
|
||||
/// Return a uniformly distributed `f32` in `[0, 1)`.
|
||||
#[inline]
|
||||
pub fn next_f32(&mut self) -> f32 {
|
||||
(self.next_u64() >> 40) as f32 / (1u64 << 24) as f32
|
||||
}
|
||||
|
||||
/// Return a uniformly distributed `f32` in `[lo, hi)`.
|
||||
#[inline]
|
||||
pub fn next_f32_range(&mut self, lo: f32, hi: f32) -> f32 {
|
||||
lo + self.next_f32() * (hi - lo)
|
||||
}
|
||||
|
||||
/// Return a uniformly distributed `usize` in `[lo, hi]` (inclusive).
|
||||
#[inline]
|
||||
pub fn next_usize_range(&mut self, lo: usize, hi: usize) -> usize {
|
||||
if lo >= hi { return lo; }
|
||||
lo + (self.next_u64() % (hi - lo + 1) as u64) as usize
|
||||
}
|
||||
|
||||
/// Sample an approximate Gaussian (mean=0, std=1) via Box-Muller.
|
||||
#[inline]
|
||||
pub fn next_gaussian(&mut self) -> f32 {
|
||||
let u1 = self.next_f32().max(1e-10);
|
||||
let u2 = self.next_f32();
|
||||
(-2.0 * u1.ln()).sqrt() * (2.0 * PI * u2).cos()
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// VirtualDomain
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Describes a single synthetic WiFi environment for domain augmentation.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct VirtualDomain {
|
||||
/// Path-loss factor simulating room size (< 1 smaller, > 1 larger room).
|
||||
pub room_scale: f32,
|
||||
/// Wall reflection coefficient in `[0, 1]` (low = absorptive, high = reflective).
|
||||
pub reflection_coeff: f32,
|
||||
/// Number of virtual scatterers (furniture / obstacles).
|
||||
pub n_scatterers: usize,
|
||||
/// Standard deviation of additive hardware noise.
|
||||
pub noise_std: f32,
|
||||
/// Unique label for the domain classifier in adversarial training.
|
||||
pub domain_id: u32,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// VirtualDomainAugmentor
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Samples virtual WiFi domains and transforms CSI frames to simulate them.
|
||||
///
|
||||
/// Applies four transformations: room-scale amplitude scaling, per-subcarrier
|
||||
/// reflection modulation, virtual scatterer sinusoidal interference, and
|
||||
/// Gaussian noise injection.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct VirtualDomainAugmentor {
|
||||
/// Range for room scale factor `(min, max)`.
|
||||
pub room_scale_range: (f32, f32),
|
||||
/// Range for reflection coefficient `(min, max)`.
|
||||
pub reflection_coeff_range: (f32, f32),
|
||||
/// Range for number of virtual scatterers `(min, max)`.
|
||||
pub n_virtual_scatterers: (usize, usize),
|
||||
/// Range for noise standard deviation `(min, max)`.
|
||||
pub noise_std_range: (f32, f32),
|
||||
next_domain_id: u32,
|
||||
}
|
||||
|
||||
impl Default for VirtualDomainAugmentor {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
room_scale_range: (0.5, 2.0),
|
||||
reflection_coeff_range: (0.3, 0.9),
|
||||
n_virtual_scatterers: (0, 5),
|
||||
noise_std_range: (0.01, 0.1),
|
||||
next_domain_id: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl VirtualDomainAugmentor {
|
||||
/// Randomly sample a new [`VirtualDomain`] from the configured ranges.
|
||||
pub fn generate_domain(&mut self, rng: &mut Xorshift64) -> VirtualDomain {
|
||||
let id = self.next_domain_id;
|
||||
self.next_domain_id = self.next_domain_id.wrapping_add(1);
|
||||
VirtualDomain {
|
||||
room_scale: rng.next_f32_range(self.room_scale_range.0, self.room_scale_range.1),
|
||||
reflection_coeff: rng.next_f32_range(self.reflection_coeff_range.0, self.reflection_coeff_range.1),
|
||||
n_scatterers: rng.next_usize_range(self.n_virtual_scatterers.0, self.n_virtual_scatterers.1),
|
||||
noise_std: rng.next_f32_range(self.noise_std_range.0, self.noise_std_range.1),
|
||||
domain_id: id,
|
||||
}
|
||||
}
|
||||
|
||||
/// Transform a single CSI amplitude frame to simulate `domain`.
|
||||
///
|
||||
/// Pipeline: (1) scale by `1/room_scale`, (2) per-subcarrier reflection
|
||||
/// modulation, (3) scatterer sinusoidal perturbation, (4) Gaussian noise.
|
||||
pub fn augment_frame(&self, frame: &[f32], domain: &VirtualDomain) -> Vec<f32> {
|
||||
let n = frame.len();
|
||||
let n_f = n as f32;
|
||||
let mut noise_rng = Xorshift64::new(
|
||||
(domain.domain_id as u64).wrapping_mul(0x9E3779B97F4A7C15).wrapping_add(1),
|
||||
);
|
||||
let mut out = Vec::with_capacity(n);
|
||||
for (k, &val) in frame.iter().enumerate() {
|
||||
let k_f = k as f32;
|
||||
// 1. Room-scale amplitude attenuation (guard against zero scale)
|
||||
let scaled = if domain.room_scale.abs() < 1e-10 { val } else { val / domain.room_scale };
|
||||
// 2. Reflection coefficient modulation (per-subcarrier)
|
||||
let refl = domain.reflection_coeff
|
||||
+ (1.0 - domain.reflection_coeff) * (PI * k_f / n_f).cos();
|
||||
let modulated = scaled * refl;
|
||||
// 3. Virtual scatterer sinusoidal interference
|
||||
let mut scatter = 0.0_f32;
|
||||
for s in 0..domain.n_scatterers {
|
||||
scatter += 0.05 * (2.0 * PI * (s as f32 + 1.0) * k_f / n_f).sin();
|
||||
}
|
||||
// 4. Additive Gaussian noise
|
||||
out.push(modulated + scatter + noise_rng.next_gaussian() * domain.noise_std);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Augment a batch, producing `k` virtual-domain variants per input frame.
|
||||
///
|
||||
/// Returns `(augmented_frame, domain_id)` pairs; total = `batch.len() * k`.
|
||||
pub fn augment_batch(
|
||||
&mut self, batch: &[Vec<f32>], k: usize, rng: &mut Xorshift64,
|
||||
) -> Vec<(Vec<f32>, u32)> {
|
||||
let mut results = Vec::with_capacity(batch.len() * k);
|
||||
for frame in batch {
|
||||
for _ in 0..k {
|
||||
let domain = self.generate_domain(rng);
|
||||
let augmented = self.augment_frame(frame, &domain);
|
||||
results.push((augmented, domain.domain_id));
|
||||
}
|
||||
}
|
||||
results
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn make_domain(scale: f32, coeff: f32, scatter: usize, noise: f32, id: u32) -> VirtualDomain {
|
||||
VirtualDomain { room_scale: scale, reflection_coeff: coeff, n_scatterers: scatter, noise_std: noise, domain_id: id }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn domain_within_configured_ranges() {
|
||||
let mut aug = VirtualDomainAugmentor::default();
|
||||
let mut rng = Xorshift64::new(12345);
|
||||
for _ in 0..100 {
|
||||
let d = aug.generate_domain(&mut rng);
|
||||
assert!(d.room_scale >= 0.5 && d.room_scale <= 2.0);
|
||||
assert!(d.reflection_coeff >= 0.3 && d.reflection_coeff <= 0.9);
|
||||
assert!(d.n_scatterers <= 5);
|
||||
assert!(d.noise_std >= 0.01 && d.noise_std <= 0.1);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn augment_frame_preserves_length() {
|
||||
let aug = VirtualDomainAugmentor::default();
|
||||
let out = aug.augment_frame(&vec![0.5; 56], &make_domain(1.0, 0.5, 3, 0.05, 0));
|
||||
assert_eq!(out.len(), 56);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn augment_frame_identity_domain_approx_input() {
|
||||
let aug = VirtualDomainAugmentor::default();
|
||||
let frame: Vec<f32> = (0..56).map(|i| 0.3 + 0.01 * i as f32).collect();
|
||||
let out = aug.augment_frame(&frame, &make_domain(1.0, 1.0, 0, 0.0, 0));
|
||||
for (a, b) in out.iter().zip(frame.iter()) {
|
||||
assert!((a - b).abs() < 1e-5, "identity domain: got {a}, expected {b}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn augment_batch_produces_correct_count() {
|
||||
let mut aug = VirtualDomainAugmentor::default();
|
||||
let mut rng = Xorshift64::new(99);
|
||||
let batch: Vec<Vec<f32>> = (0..4).map(|_| vec![0.5; 56]).collect();
|
||||
let results = aug.augment_batch(&batch, 3, &mut rng);
|
||||
assert_eq!(results.len(), 12);
|
||||
for (f, _) in &results { assert_eq!(f.len(), 56); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn different_seeds_produce_different_augmentations() {
|
||||
let mut aug1 = VirtualDomainAugmentor::default();
|
||||
let mut aug2 = VirtualDomainAugmentor::default();
|
||||
let frame = vec![0.5_f32; 56];
|
||||
let d1 = aug1.generate_domain(&mut Xorshift64::new(1));
|
||||
let d2 = aug2.generate_domain(&mut Xorshift64::new(2));
|
||||
let out1 = aug1.augment_frame(&frame, &d1);
|
||||
let out2 = aug2.augment_frame(&frame, &d2);
|
||||
assert!(out1.iter().zip(out2.iter()).any(|(a, b)| (a - b).abs() > 1e-6));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deterministic_same_seed_same_output() {
|
||||
let batch: Vec<Vec<f32>> = (0..3).map(|i| vec![0.1 * i as f32; 56]).collect();
|
||||
let mut aug1 = VirtualDomainAugmentor::default();
|
||||
let mut aug2 = VirtualDomainAugmentor::default();
|
||||
let res1 = aug1.augment_batch(&batch, 2, &mut Xorshift64::new(42));
|
||||
let res2 = aug2.augment_batch(&batch, 2, &mut Xorshift64::new(42));
|
||||
assert_eq!(res1.len(), res2.len());
|
||||
for ((f1, id1), (f2, id2)) in res1.iter().zip(res2.iter()) {
|
||||
assert_eq!(id1, id2);
|
||||
for (a, b) in f1.iter().zip(f2.iter()) {
|
||||
assert!((a - b).abs() < 1e-7, "same seed must produce identical output");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn domain_ids_are_sequential() {
|
||||
let mut aug = VirtualDomainAugmentor::default();
|
||||
let mut rng = Xorshift64::new(7);
|
||||
for i in 0..10_u32 { assert_eq!(aug.generate_domain(&mut rng).domain_id, i); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn xorshift64_deterministic() {
|
||||
let mut a = Xorshift64::new(999);
|
||||
let mut b = Xorshift64::new(999);
|
||||
for _ in 0..100 { assert_eq!(a.next_u64(), b.next_u64()); }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn xorshift64_f32_in_unit_interval() {
|
||||
let mut rng = Xorshift64::new(42);
|
||||
for _ in 0..1000 {
|
||||
let v = rng.next_f32();
|
||||
assert!(v >= 0.0 && v < 1.0, "f32 sample {v} not in [0, 1)");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn augment_frame_empty_and_batch_k_zero() {
|
||||
let aug = VirtualDomainAugmentor::default();
|
||||
assert!(aug.augment_frame(&[], &make_domain(1.5, 0.5, 2, 0.05, 0)).is_empty());
|
||||
let mut aug2 = VirtualDomainAugmentor::default();
|
||||
assert!(aug2.augment_batch(&[vec![0.5; 56]], 0, &mut Xorshift64::new(1)).is_empty());
|
||||
}
|
||||
}
|
||||
@@ -59,7 +59,7 @@ uuid = { version = "1.6", features = ["v4", "serde", "js"] }
|
||||
getrandom = { version = "0.2", features = ["js"] }
|
||||
|
||||
# Optional: wifi-densepose-mat integration
|
||||
wifi-densepose-mat = { version = "0.1.0", path = "../wifi-densepose-mat", optional = true, features = ["serde"] }
|
||||
wifi-densepose-mat = { version = "0.2.0", path = "../wifi-densepose-mat", optional = true, features = ["serde"] }
|
||||
|
||||
[dev-dependencies]
|
||||
wasm-bindgen-test = "0.3"
|
||||
|
||||
Reference in New Issue
Block a user