dearsky/wifi-densepose
1
0
Fork 0
Code Issues 20 Pull Requests 5 Actions Packages Projects Releases 1 Wiki Activity

⭐ Tutorial: ESP32-S3 CSI Pipeline End-to-End Setup (ADR-018) #34

New Issue
Open
opened 2026-03-01 02:26:55 +08:00 by ruvnet · 3 comments
ruvnet commented 2026-03-01 02:26:55 +08:00 (Migrated from github.com)
Copy Link

ESP32-S3 CSI Pipeline: Step-by-Step Tutorial (ADR-018)

Complete end-to-end guide for building, flashing, and running the WiFi CSI human presence detection pipeline using an ESP32-S3 and the Rust aggregator.

Verified hardware: ESP32-S3-DevKitC-1 (CP2102 UART, MAC 3C:0F:02:EC:C2:28)
Verified performance: ~20 Hz CSI streaming, 64/128/192 subcarrier frames, RSSI -47 to -88 dBm

Prerequisites

Component Version Install
Docker Desktop 28.x+ https://www.docker.com/products/docker-desktop
esptool 5.x+ pip install esptool
Rust toolchain stable https://rustup.rs
CP210x driver latest https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers
ESP32-S3 board any DevKitC-1, WROOM, or similar

Step 1: Clone the repository

git clone https://github.com/ruvnet/wifi-densepose.git
cd wifi-densepose

Step 2: Build the Rust aggregator

cd rust-port/wifi-densepose-rs
cargo build -p wifi-densepose-hardware --bin aggregator --release

The binary will be at target/release/aggregator (or aggregator.exe on Windows).

Step 3: Configure WiFi credentials

Create or edit firmware/esp32-csi-node/sdkconfig.defaults:

CONFIG_IDF_TARGET="esp32s3"
CONFIG_ESP_WIFI_CSI_ENABLED=y
CONFIG_CSI_NODE_ID=1
CONFIG_CSI_WIFI_SSID="YOUR_WIFI_SSID"
CONFIG_CSI_WIFI_PASSWORD="YOUR_WIFI_PASSWORD"
CONFIG_CSI_TARGET_IP="192.168.1.20"
CONFIG_CSI_TARGET_PORT=5005
CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y

Replace:

  • YOUR_WIFI_SSID with your WiFi network name
  • YOUR_WIFI_PASSWORD with your WiFi password
  • 192.168.1.20 with the IP of the machine running the aggregator

Security: sdkconfig.defaults is gitignored. Never commit real credentials.

Step 4: Build firmware with Docker

cd firmware/esp32-csi-node

# Linux/macOS:
docker run --rm -v "$(pwd):/project" -w /project \
  espressif/idf:v5.2 bash -c "idf.py set-target esp32s3 && idf.py build"

# Windows (Git Bash):
MSYS_NO_PATHCONV=1 docker run --rm -v "$(pwd -W)://project" -w //project \
  espressif/idf:v5.2 bash -c "idf.py set-target esp32s3 && idf.py build"

Build output appears in build/ (~960 compilation steps, takes 3-5 minutes first time).

Step 5: Find your serial port

OS Command Typical Port
Windows Device Manager > Ports COM7
Linux ls /dev/ttyUSB* /dev/ttyUSB0
macOS ls /dev/cu.usbserial* /dev/cu.usbserial-0001

Step 6: Flash firmware to ESP32-S3

cd firmware/esp32-csi-node/build

python -m esptool --chip esp32s3 --port COM7 --baud 460800 \
  --before default-reset --after hard-reset \
  write-flash --flash-mode dio --flash-freq 80m --flash-size 4MB \
  0x0 bootloader/bootloader.bin \
  0x8000 partition_table/partition-table.bin \
  0x10000 esp32-csi-node.bin

Replace COM7 with your serial port. Expected output:

Hash of data verified.
Leaving...
Hard resetting via RTS pin...

Step 7: Open firewall (Windows only)

Run in elevated PowerShell:

netsh advfirewall firewall add rule name="ESP32 CSI" dir=in action=allow protocol=UDP localport=5005

Step 8: Run the aggregator

cargo run -p wifi-densepose-hardware --bin aggregator -- --bind 0.0.0.0:5005 --verbose

Expected output when ESP32 connects:

Listening on 0.0.0.0:5005...
  [148 bytes from 192.168.1.71:60764]
[node:1 seq:0] sc=64 rssi=-49 amp=9.5
  [276 bytes from 192.168.1.71:60764]
[node:1 seq:1] sc=128 rssi=-64 amp=16.0

Step 9: Verify presence detection

Walk near the ESP32 and observe changes in amplitude and RSSI values. Expect:

  • ~20 frames/sec streaming rate
  • Amplitude variance increases when someone is moving near the antenna
  • RSSI fluctuates with proximity changes
  • Frame types: 64 sc (148 B), 128 sc (276 B), 192 sc (404 B)

Step 10: Multi-node setup (optional)

For multi-node coverage, repeat Steps 3-6 for additional ESP32-S3 boards with different CONFIG_CSI_NODE_ID values (2, 3, ...). All nodes stream to the same aggregator.

Architecture

ESP32-S3 Node(s)                     Host Machine
+-------------------+               +-------------------+
| WiFi CSI callback |  UDP/5005     | aggregator binary |
| (promiscuous mode)|  ---------->  | (Rust, clap CLI)  |
| ADR-018 serialize |  ADR-018      | Esp32CsiParser    |
| stream_sender.c   |  binary       | CsiFrame output   |
+-------------------+               +-------------------+

ADR-018 Binary Frame Format

Offset  Size  Field
0       4     Magic: 0xC5110001 (LE)
4       1     Node ID
5       1     Number of antennas
6       2     Number of subcarriers (LE u16)
8       4     Frequency MHz (LE u32)
12      4     Sequence number (LE u32)
16      1     RSSI (i8)
17      1     Noise floor (i8)
18      2     Reserved
20      N*2   I/Q pairs (n_antennas * n_subcarriers * 2 bytes)

Troubleshooting

Symptom Cause Fix
No serial output Wrong baud rate Use 115200 for ESP32 monitor
WiFi connection fails Wrong SSID/password Check sdkconfig.defaults
No UDP frames arrive Firewall blocking Add UDP 5005 inbound rule (Step 7)
CSI callback not firing Promiscuous mode issue Verify esp_wifi_set_promiscuous(true) in csi_collector.c
Parse errors in aggregator Firmware/parser version mismatch Rebuild both from same commit
Docker path errors (Windows) MSYS path conversion Use MSYS_NO_PATHCONV=1 prefix
esptool not found Not installed pip install esptool

Verified Test Results

  • 20 Rust tests pass (cargo test -p wifi-densepose-hardware)
  • 6 Python tests pass (pytest v1/tests/)
  • 693 frames captured in 18 seconds (~21.6 fps)
  • Sequence numbers contiguous (zero frame loss)
  • Presence detection confirmed with live motion scoring

Related

  • ADR-018: ESP32 Development Implementation
  • Commit 92a5182: feat(adr-018) ESP32-S3 firmware, Rust aggregator, and live CSI pipeline
  • firmware/esp32-csi-node/README.md: Firmware-specific documentation
## ESP32-S3 CSI Pipeline: Step-by-Step Tutorial (ADR-018) Complete end-to-end guide for building, flashing, and running the WiFi CSI human presence detection pipeline using an ESP32-S3 and the Rust aggregator. **Verified hardware:** ESP32-S3-DevKitC-1 (CP2102 UART, MAC 3C:0F:02:EC:C2:28) **Verified performance:** ~20 Hz CSI streaming, 64/128/192 subcarrier frames, RSSI -47 to -88 dBm --- ### Prerequisites | Component | Version | Install | |-----------|---------|---------| | Docker Desktop | 28.x+ | https://www.docker.com/products/docker-desktop | | esptool | 5.x+ | `pip install esptool` | | Rust toolchain | stable | https://rustup.rs | | CP210x driver | latest | https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers | | ESP32-S3 board | any | DevKitC-1, WROOM, or similar | --- ### Step 1: Clone the repository ```bash git clone https://github.com/ruvnet/wifi-densepose.git cd wifi-densepose ``` ### Step 2: Build the Rust aggregator ```bash cd rust-port/wifi-densepose-rs cargo build -p wifi-densepose-hardware --bin aggregator --release ``` The binary will be at `target/release/aggregator` (or `aggregator.exe` on Windows). ### Step 3: Configure WiFi credentials Create or edit `firmware/esp32-csi-node/sdkconfig.defaults`: ```ini CONFIG_IDF_TARGET="esp32s3" CONFIG_ESP_WIFI_CSI_ENABLED=y CONFIG_CSI_NODE_ID=1 CONFIG_CSI_WIFI_SSID="YOUR_WIFI_SSID" CONFIG_CSI_WIFI_PASSWORD="YOUR_WIFI_PASSWORD" CONFIG_CSI_TARGET_IP="192.168.1.20" CONFIG_CSI_TARGET_PORT=5005 CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y ``` Replace: - `YOUR_WIFI_SSID` with your WiFi network name - `YOUR_WIFI_PASSWORD` with your WiFi password - `192.168.1.20` with the IP of the machine running the aggregator > **Security:** `sdkconfig.defaults` is gitignored. Never commit real credentials. ### Step 4: Build firmware with Docker ```bash cd firmware/esp32-csi-node # Linux/macOS: docker run --rm -v "$(pwd):/project" -w /project \ espressif/idf:v5.2 bash -c "idf.py set-target esp32s3 && idf.py build" # Windows (Git Bash): MSYS_NO_PATHCONV=1 docker run --rm -v "$(pwd -W)://project" -w //project \ espressif/idf:v5.2 bash -c "idf.py set-target esp32s3 && idf.py build" ``` Build output appears in `build/` (~960 compilation steps, takes 3-5 minutes first time). ### Step 5: Find your serial port | OS | Command | Typical Port | |----|---------|--------------| | Windows | Device Manager > Ports | COM7 | | Linux | `ls /dev/ttyUSB*` | /dev/ttyUSB0 | | macOS | `ls /dev/cu.usbserial*` | /dev/cu.usbserial-0001 | ### Step 6: Flash firmware to ESP32-S3 ```bash cd firmware/esp32-csi-node/build python -m esptool --chip esp32s3 --port COM7 --baud 460800 \ --before default-reset --after hard-reset \ write-flash --flash-mode dio --flash-freq 80m --flash-size 4MB \ 0x0 bootloader/bootloader.bin \ 0x8000 partition_table/partition-table.bin \ 0x10000 esp32-csi-node.bin ``` Replace `COM7` with your serial port. Expected output: ``` Hash of data verified. Leaving... Hard resetting via RTS pin... ``` ### Step 7: Open firewall (Windows only) Run in elevated PowerShell: ```powershell netsh advfirewall firewall add rule name="ESP32 CSI" dir=in action=allow protocol=UDP localport=5005 ``` ### Step 8: Run the aggregator ```bash cargo run -p wifi-densepose-hardware --bin aggregator -- --bind 0.0.0.0:5005 --verbose ``` Expected output when ESP32 connects: ``` Listening on 0.0.0.0:5005... [148 bytes from 192.168.1.71:60764] [node:1 seq:0] sc=64 rssi=-49 amp=9.5 [276 bytes from 192.168.1.71:60764] [node:1 seq:1] sc=128 rssi=-64 amp=16.0 ``` ### Step 9: Verify presence detection Walk near the ESP32 and observe changes in amplitude and RSSI values. Expect: - **~20 frames/sec** streaming rate - **Amplitude variance increases** when someone is moving near the antenna - **RSSI fluctuates** with proximity changes - **Frame types:** 64 sc (148 B), 128 sc (276 B), 192 sc (404 B) ### Step 10: Multi-node setup (optional) For multi-node coverage, repeat Steps 3-6 for additional ESP32-S3 boards with different `CONFIG_CSI_NODE_ID` values (2, 3, ...). All nodes stream to the same aggregator. --- ### Architecture ``` ESP32-S3 Node(s) Host Machine +-------------------+ +-------------------+ | WiFi CSI callback | UDP/5005 | aggregator binary | | (promiscuous mode)| ----------> | (Rust, clap CLI) | | ADR-018 serialize | ADR-018 | Esp32CsiParser | | stream_sender.c | binary | CsiFrame output | +-------------------+ +-------------------+ ``` ### ADR-018 Binary Frame Format ``` Offset Size Field 0 4 Magic: 0xC5110001 (LE) 4 1 Node ID 5 1 Number of antennas 6 2 Number of subcarriers (LE u16) 8 4 Frequency MHz (LE u32) 12 4 Sequence number (LE u32) 16 1 RSSI (i8) 17 1 Noise floor (i8) 18 2 Reserved 20 N*2 I/Q pairs (n_antennas * n_subcarriers * 2 bytes) ``` ### Troubleshooting | Symptom | Cause | Fix | |---------|-------|-----| | No serial output | Wrong baud rate | Use 115200 for ESP32 monitor | | WiFi connection fails | Wrong SSID/password | Check sdkconfig.defaults | | No UDP frames arrive | Firewall blocking | Add UDP 5005 inbound rule (Step 7) | | CSI callback not firing | Promiscuous mode issue | Verify `esp_wifi_set_promiscuous(true)` in csi_collector.c | | Parse errors in aggregator | Firmware/parser version mismatch | Rebuild both from same commit | | Docker path errors (Windows) | MSYS path conversion | Use `MSYS_NO_PATHCONV=1` prefix | | `esptool` not found | Not installed | `pip install esptool` | ### Verified Test Results - 20 Rust tests pass (`cargo test -p wifi-densepose-hardware`) - 6 Python tests pass (`pytest v1/tests/`) - 693 frames captured in 18 seconds (~21.6 fps) - Sequence numbers contiguous (zero frame loss) - Presence detection confirmed with live motion scoring ### Related - ADR-018: ESP32 Development Implementation - Commit `92a5182`: feat(adr-018) ESP32-S3 firmware, Rust aggregator, and live CSI pipeline - `firmware/esp32-csi-node/README.md`: Firmware-specific documentation
ruvnet commented 2026-03-01 12:25:43 +08:00 (Migrated from github.com)
Copy Link

Update: ESP32 Pipeline Now Includes Vital Signs + Trained Model Support

The Rust sensing server that this tutorial connects to has been significantly expanded. When following this tutorial with ESP32-S3 hardware, you now get additional capabilities beyond basic CSI streaming:

New with ESP32 CSI data

  • Vital sign detection: Real-time breathing rate (6–30 BPM) and heart rate (48–120 BPM) extracted from CSI amplitude and phase variance
  • Graph transformer pipeline: COCO 17-keypoint skeleton with GNN message passing over the body graph
  • REST endpoint: GET /api/v1/vital-signs returns live breathing/heartbeat data
  • WebSocket: Every broadcast now includes vital_signs alongside pose data

New CLI flags for the sensing server

# Run with ESP32 + vital signs
./target/release/sensing-server --source esp32 --udp-port 5005

# Run benchmark to verify performance
./target/release/sensing-server --benchmark
# => 9,520 frames/sec

# Load a trained model (when available)
./target/release/sensing-server --source esp32 --model wifi-densepose-v1.rvf --progressive

The ESP32 firmware and flash steps in this tutorial remain unchanged — the improvements are all server-side.

— Ruflo AI

## Update: ESP32 Pipeline Now Includes Vital Signs + Trained Model Support The Rust sensing server that this tutorial connects to has been significantly expanded. When following this tutorial with ESP32-S3 hardware, you now get additional capabilities beyond basic CSI streaming: ### New with ESP32 CSI data - **Vital sign detection**: Real-time breathing rate (6–30 BPM) and heart rate (48–120 BPM) extracted from CSI amplitude and phase variance - **Graph transformer pipeline**: COCO 17-keypoint skeleton with GNN message passing over the body graph - **REST endpoint**: `GET /api/v1/vital-signs` returns live breathing/heartbeat data - **WebSocket**: Every broadcast now includes `vital_signs` alongside pose data ### New CLI flags for the sensing server ```bash # Run with ESP32 + vital signs ./target/release/sensing-server --source esp32 --udp-port 5005 # Run benchmark to verify performance ./target/release/sensing-server --benchmark # => 9,520 frames/sec # Load a trained model (when available) ./target/release/sensing-server --source esp32 --model wifi-densepose-v1.rvf --progressive ``` The ESP32 firmware and flash steps in this tutorial remain unchanged — the improvements are all server-side. — Ruflo AI
ruvnet commented 2026-03-01 12:46:00 +08:00 (Migrated from github.com)
Copy Link

Update: Docker + Rust Sensing Server Available

You can now skip the local Rust toolchain and run the sensing server directly from Docker:

# Pull and run (132 MB, includes UI + RuVector signal processing)
docker pull ruvnet/wifi-densepose:latest
docker run -p 3000:3000 -p 3001:3001 -p 5005:5005/udp ruvnet/wifi-densepose:latest

# With ESP32 hardware connected (forward UDP port):
docker run --network host ruvnet/wifi-densepose:latest --source esp32

The Docker image includes:

  • Rust Axum server with RuVector signal processing
  • Vital sign detection (breathing + heartbeat via FFT)
  • RVF model container support (--load-rvf, --export-rvf)
  • Full 8-phase DensePose training pipeline (ADR-023)
  • Three.js UI served at http://localhost:3000

RVF model package (portable, single-file deployment):

docker run --rm -v $(pwd):/out ruvnet/wifi-densepose:latest --export-rvf /out/wifi-densepose-v1.rvf

The ESP32 tutorial steps remain the same — only Step 8 (run aggregator) can now be replaced with the Docker command above.

Current test coverage

  • 229 sensing server tests passing (147 lib + 82 integration)
  • 11,665 fps vital sign benchmark (release build)
  • Docker image: 132 MB (multi-stage, debian:bookworm-slim)
## Update: Docker + Rust Sensing Server Available You can now skip the local Rust toolchain and run the sensing server directly from Docker: ```bash # Pull and run (132 MB, includes UI + RuVector signal processing) docker pull ruvnet/wifi-densepose:latest docker run -p 3000:3000 -p 3001:3001 -p 5005:5005/udp ruvnet/wifi-densepose:latest # With ESP32 hardware connected (forward UDP port): docker run --network host ruvnet/wifi-densepose:latest --source esp32 ``` The Docker image includes: - Rust Axum server with RuVector signal processing - Vital sign detection (breathing + heartbeat via FFT) - RVF model container support (`--load-rvf`, `--export-rvf`) - Full 8-phase DensePose training pipeline (ADR-023) - Three.js UI served at http://localhost:3000 **RVF model package** (portable, single-file deployment): ```bash docker run --rm -v $(pwd):/out ruvnet/wifi-densepose:latest --export-rvf /out/wifi-densepose-v1.rvf ``` The ESP32 tutorial steps remain the same — only Step 8 (run aggregator) can now be replaced with the Docker command above. ### Current test coverage - 229 sensing server tests passing (147 lib + 82 integration) - 11,665 fps vital sign benchmark (release build) - Docker image: 132 MB (multi-stage, debian:bookworm-slim)
👍 1
ywz978020607 commented 2026-03-01 13:40:26 +08:00 (Migrated from github.com)
Copy Link

Is there any real testing results? datasets or images or screenshots？

Is there any real testing results? datasets or images or screenshots？
👍 7
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
Something isn't working
 documentation
Improvements or additions to documentation
 duplicate
This issue or pull request already exists
 enhancement
New feature or request
 good first issue
Good for newcomers
 help wanted
Extra attention is needed
 invalid
This doesn't seem right
 question
Further information is requested
 security
Security-related issues and hardening
 wontfix
This will not be worked on
No Label documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dearsky
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dearsky/wifi-densepose#34
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?