feat: ADR-024 Contrastive CSI Embedding Model — all 7 phases (#52)
Full implementation of Project AETHER — Contrastive CSI Embedding Model. ## Phases Delivered 1. ProjectionHead (64→128→128) + L2 normalization 2. CsiAugmenter (5 physically-motivated augmentations) 3. InfoNCE contrastive loss + SimCLR pretraining 4. FingerprintIndex (4 index types: env, activity, temporal, person) 5. RVF SEG_EMBED (0x0C) + CLI integration 6. Cross-modal alignment (PoseEncoder + InfoNCE) 7. Deep RuVector: MicroLoRA, EWC++, drift detection, hard-negative mining, SEG_LORA ## Stats - 276 tests passing (191 lib + 51 bin + 16 rvf + 18 vitals) - 3,342 additions across 8 files - Zero unsafe/unwrap/panic/todo stubs - ~55KB INT8 model for ESP32 edge deployment Also fixes deprecated GitHub Actions (v3→v4) and adds feat/* branch CI triggers. Closes #50
This commit was merged in pull request #52.
This commit is contained in:
@@ -0,0 +1,89 @@
|
||||
# wifi-densepose-nn
|
||||
|
||||
[](https://crates.io/crates/wifi-densepose-nn)
|
||||
[](https://docs.rs/wifi-densepose-nn)
|
||||
[](LICENSE)
|
||||
|
||||
Multi-backend neural network inference for WiFi-based DensePose estimation.
|
||||
|
||||
## Overview
|
||||
|
||||
`wifi-densepose-nn` provides the inference engine that maps processed WiFi CSI features to
|
||||
DensePose body surface predictions. It supports three backends -- ONNX Runtime (default),
|
||||
PyTorch via `tch-rs`, and Candle -- so models can run on CPU, CUDA GPU, or TensorRT depending
|
||||
on the deployment target.
|
||||
|
||||
The crate implements two key neural components:
|
||||
|
||||
- **DensePose Head** -- Predicts 24 body part segmentation masks and per-part UV coordinate
|
||||
regression.
|
||||
- **Modality Translator** -- Translates CSI feature embeddings into visual feature space,
|
||||
bridging the domain gap between WiFi signals and image-based pose estimation.
|
||||
|
||||
## Features
|
||||
|
||||
- **ONNX Runtime backend** (default) -- Load and run `.onnx` models with CPU or GPU execution
|
||||
providers.
|
||||
- **PyTorch backend** (`tch-backend`) -- Native PyTorch inference via libtorch FFI.
|
||||
- **Candle backend** (`candle-backend`) -- Pure-Rust inference with `candle-core` and
|
||||
`candle-nn`.
|
||||
- **CUDA acceleration** (`cuda`) -- GPU execution for supported backends.
|
||||
- **TensorRT optimization** (`tensorrt`) -- INT8/FP16 optimized inference via ONNX Runtime.
|
||||
- **Batched inference** -- Process multiple CSI frames in a single forward pass.
|
||||
- **Model caching** -- Memory-mapped model weights via `memmap2`.
|
||||
|
||||
### Feature flags
|
||||
|
||||
| Flag | Default | Description |
|
||||
|-------------------|---------|-------------------------------------|
|
||||
| `onnx` | yes | ONNX Runtime backend |
|
||||
| `tch-backend` | no | PyTorch (tch-rs) backend |
|
||||
| `candle-backend` | no | Candle pure-Rust backend |
|
||||
| `cuda` | no | CUDA GPU acceleration |
|
||||
| `tensorrt` | no | TensorRT via ONNX Runtime |
|
||||
| `all-backends` | no | Enable onnx + tch + candle together |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```rust
|
||||
use wifi_densepose_nn::{InferenceEngine, DensePoseConfig, OnnxBackend};
|
||||
|
||||
// Create inference engine with ONNX backend
|
||||
let config = DensePoseConfig::default();
|
||||
let backend = OnnxBackend::from_file("model.onnx")?;
|
||||
let engine = InferenceEngine::new(backend, config)?;
|
||||
|
||||
// Run inference on a CSI feature tensor
|
||||
let input = ndarray::Array4::zeros((1, 256, 64, 64));
|
||||
let output = engine.infer(&input)?;
|
||||
|
||||
println!("Body parts: {}", output.body_parts.shape()[1]); // 24
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```text
|
||||
wifi-densepose-nn/src/
|
||||
lib.rs -- Re-exports, constants (NUM_BODY_PARTS=24), prelude
|
||||
densepose.rs -- DensePoseHead, DensePoseConfig, DensePoseOutput
|
||||
inference.rs -- Backend trait, InferenceEngine, InferenceOptions
|
||||
onnx.rs -- OnnxBackend, OnnxSession (feature-gated)
|
||||
tensor.rs -- Tensor, TensorShape utilities
|
||||
translator.rs -- ModalityTranslator (CSI -> visual space)
|
||||
error.rs -- NnError, NnResult
|
||||
```
|
||||
|
||||
## Related Crates
|
||||
|
||||
| Crate | Role |
|
||||
|-------|------|
|
||||
| [`wifi-densepose-core`](../wifi-densepose-core) | Foundation types and `NeuralInference` trait |
|
||||
| [`wifi-densepose-signal`](../wifi-densepose-signal) | Produces CSI features consumed by inference |
|
||||
| [`wifi-densepose-train`](../wifi-densepose-train) | Trains the models this crate loads |
|
||||
| [`ort`](https://crates.io/crates/ort) | ONNX Runtime Rust bindings |
|
||||
| [`tch`](https://crates.io/crates/tch) | PyTorch Rust bindings |
|
||||
| [`candle-core`](https://crates.io/crates/candle-core) | Hugging Face pure-Rust ML framework |
|
||||
|
||||
## License
|
||||
|
||||
MIT OR Apache-2.0
|
||||
Reference in New Issue
Block a user