git-subtree-dir: vendor/ruvector git-subtree-split: b64c21726f2bb37286d9ee36a7869fef60cc6900
263 lines
8.0 KiB
Markdown
263 lines
8.0 KiB
Markdown
# getrandom Resolution - SUCCESSFUL ✅
|
|
|
|
**Date**: 2025-12-09
|
|
**Status**: RESOLVED
|
|
**Build**: Successful
|
|
**Time to Resolution**: ~2 hours
|
|
|
|
---
|
|
|
|
## 🎯 Problem Statement
|
|
|
|
RvLite could not compile to WASM with `ruvector-core` due to conflicting `getrandom` versions:
|
|
|
|
- **getrandom 0.2.16** (needs "js" feature for WASM)
|
|
- **getrandom 0.3.4** (needs "wasm_js" cfg flag)
|
|
|
|
**Root Cause**: `hnsw_rs 0.3.3` → `rand 0.9` → `getrandom 0.3`
|
|
Meanwhile, rest of ecosystem uses `rand 0.8` → `getrandom 0.2`
|
|
|
|
---
|
|
|
|
## ✅ Solution Implemented
|
|
|
|
### 1. **Patched hnsw_rs** (Avoided by disabling)
|
|
Created `/workspaces/ruvector/patches/hnsw_rs/` with modified `Cargo.toml`:
|
|
```toml
|
|
rand = { version = "0.8" } # Changed from 0.9
|
|
```
|
|
|
|
Added to workspace `Cargo.toml`:
|
|
```toml
|
|
[patch.crates-io]
|
|
hnsw_rs = { path = "./patches/hnsw_rs" }
|
|
```
|
|
|
|
**Result**: This prevented getrandom 0.3, but wasn't needed since we disabled HNSW entirely.
|
|
|
|
### 2. **Disabled HNSW in ruvector-core** ✅ PRIMARY FIX
|
|
Modified `rvlite/Cargo.toml`:
|
|
```toml
|
|
ruvector-core = {
|
|
path = "../ruvector-core",
|
|
default-features = false, # ← Critical!
|
|
features = ["memory-only"]
|
|
}
|
|
```
|
|
|
|
**Why this worked**:
|
|
- `ruvector-core` default features include `hnsw = ["hnsw_rs"]`
|
|
- By disabling defaults, we avoid `hnsw_rs` → `mmap-rs` → platform-specific code
|
|
- `memory-only` feature provides pure in-memory storage (perfect for WASM)
|
|
|
|
### 3. **Enabled getrandom "js" feature** ✅ CRITICAL FIX
|
|
Added WASM-specific dependency in `rvlite/Cargo.toml`:
|
|
```toml
|
|
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
|
getrandom = { workspace = true, features = ["js"] }
|
|
```
|
|
|
|
**Why this was needed**:
|
|
- Workspace specifying `getrandom` with features doesn't propagate to transitive deps
|
|
- Top-level crate must explicitly enable features for WASM target
|
|
- This ensures `rand 0.8` → `rand_core 0.6` → `getrandom 0.2` gets "js" feature
|
|
|
|
### 4. **Build script** (Created but not required)
|
|
Created `build.rs` with WASM cfg flags:
|
|
```rust
|
|
if target.starts_with("wasm32") {
|
|
println!("cargo:rustc-cfg=getrandom_backend=\"wasm_js\"");
|
|
}
|
|
```
|
|
|
|
**Result**: Not required for getrandom 0.2 approach, but kept for documentation.
|
|
|
|
---
|
|
|
|
## 📊 Build Results
|
|
|
|
### Successful Build Output
|
|
```bash
|
|
$ wasm-pack build --target web --release
|
|
[INFO]: 🎯 Checking for the Wasm target...
|
|
[INFO]: 🌀 Compiling to Wasm...
|
|
Compiling ruvector-core v0.1.21
|
|
Compiling rvlite v0.1.0
|
|
Finished `release` profile [optimized] target(s) in 10.68s
|
|
[INFO]: ✨ Done in 11.29s
|
|
[INFO]: 📦 Your wasm pkg is ready to publish at /workspaces/ruvector/crates/rvlite/pkg.
|
|
```
|
|
|
|
### Bundle Size
|
|
```
|
|
Uncompressed: 41 KB
|
|
Gzipped: 15.90 KB
|
|
Total pkg: 92 KB
|
|
```
|
|
|
|
**Note**: Size is still minimal because `lib.rs` doesn't use ruvector-core APIs yet.
|
|
Tree-shaking removes unused code. Actual size will increase when vector operations are implemented.
|
|
|
|
---
|
|
|
|
## 🔑 Key Learnings
|
|
|
|
### 1. **Default Features Must Be Explicitly Disabled**
|
|
```toml
|
|
# ❌ WRONG - Still enables default features
|
|
ruvector-core = { path = "../ruvector-core", features = ["memory-only"] }
|
|
|
|
# ✅ CORRECT - Only enables memory-only
|
|
ruvector-core = { path = "../ruvector-core", default-features = false, features = ["memory-only"] }
|
|
```
|
|
|
|
### 2. **WASM Feature Propagation**
|
|
- Workspace dependencies with features don't auto-enable for transitive deps
|
|
- Must add target-specific dependency in top-level crate
|
|
- Use `[target.'cfg(target_arch = "wasm32")'.dependencies]`
|
|
|
|
### 3. **getrandom Versions**
|
|
- **v0.2**: Uses `features = ["js"]` for WASM
|
|
- **v0.3**: Uses `features = ["wasm_js"]` AND requires cfg flags
|
|
- Cannot unify across major versions
|
|
|
|
### 4. **WASM Incompatibilities**
|
|
- `mmap-rs`: Requires OS-level memory mapping (not available in WASM)
|
|
- `hnsw_rs`: Depends on `mmap-rs` for persistence
|
|
- Solution: Use `memory-only` features or WASM-specific alternatives
|
|
|
|
### 5. **Feature Flag Architecture**
|
|
ruvector-core has excellent WASM support via features:
|
|
```toml
|
|
[features]
|
|
default = ["simd", "storage", "hnsw"]
|
|
storage = ["redb", "memmap2"] # Not available in WASM
|
|
hnsw = ["hnsw_rs"] # Not available in WASM
|
|
memory-only = [] # Pure in-memory (WASM-compatible)
|
|
```
|
|
|
|
---
|
|
|
|
## 🚀 Next Steps
|
|
|
|
### Phase 1: Basic Vector Operations (Current)
|
|
- ✅ WASM builds successfully
|
|
- ✅ getrandom conflict resolved
|
|
- ⏳ Integrate ruvector-core vector APIs into lib.rs
|
|
- ⏳ Measure actual bundle size with vector operations
|
|
|
|
### Phase 2: Additional WASM Crates
|
|
Integrate existing WASM crates:
|
|
- `ruvector-wasm`: Storage and indexing
|
|
- `ruvector-graph-wasm`: Cypher queries
|
|
- `ruvector-gnn-wasm`: GNN layers
|
|
- `micro-hnsw-wasm`: WASM-compatible HNSW
|
|
|
|
### Phase 3: Query Engines
|
|
Extract from ruvector-postgres:
|
|
- SQL parser and executor
|
|
- SPARQL query engine
|
|
- Cypher integration
|
|
|
|
### Phase 4: Learning Systems
|
|
- Integrate `sona` with WASM features
|
|
- ReasoningBank for self-learning
|
|
|
|
---
|
|
|
|
## 📁 Files Modified
|
|
|
|
### Created
|
|
1. `/workspaces/ruvector/crates/rvlite/build.rs` - WASM cfg configuration
|
|
2. `/workspaces/ruvector/patches/hnsw_rs/` - Patched hnsw_rs with rand 0.8
|
|
3. `/workspaces/ruvector/crates/rvlite/docs/GETRANDOM_RESOLUTION_SUCCESS.md` - This file
|
|
|
|
### Modified
|
|
1. `/workspaces/ruvector/Cargo.toml` - Added `[patch.crates-io]` section
|
|
2. `/workspaces/ruvector/crates/rvlite/Cargo.toml` - Disabled default features, added WASM getrandom
|
|
3. `/workspaces/ruvector/crates/ruvector-wasm/Cargo.toml` - Removed getrandom02 alias
|
|
4. `/workspaces/ruvector/crates/ruvector-graph-wasm/Cargo.toml` - Updated getrandom features
|
|
5. `/workspaces/ruvector/crates/ruvector-gnn-wasm/Cargo.toml` - Updated getrandom features
|
|
|
|
---
|
|
|
|
## 🎓 Solution Comparison
|
|
|
|
From `GETRANDOM_RESOLUTION_STRATEGY.md`:
|
|
|
|
| Option | Status | Effort | Result |
|
|
|--------|--------|--------|--------|
|
|
| A: Exclude deps | ✅ Used | 0 days | POC working (15.90 KB) |
|
|
| B: Patch rand | ⚠️ Created but not needed | 1 hour | hnsw_rs patched, but avoided via features |
|
|
| C: Update to rand 0.9 | ❌ Not needed | N/A | Avoided by disabling HNSW |
|
|
| D: Build script | ⚠️ Created but not needed | 30 min | Works, but target dep sufficient |
|
|
| E: Wait upstream | ❌ Not viable | N/A | Resolved without waiting |
|
|
|
|
**Actual Solution**: Combination of A + D + target-specific dependencies
|
|
|
|
---
|
|
|
|
## 🔧 Technical Deep Dive
|
|
|
|
### Dependency Resolution
|
|
|
|
**Before (Failed)**:
|
|
```
|
|
rvlite
|
|
└─ ruvector-core (default features)
|
|
├─ hnsw_rs 0.3.3
|
|
│ ├─ rand 0.9 → getrandom 0.3 ❌
|
|
│ └─ mmap-rs (not WASM-compatible) ❌
|
|
└─ rand 0.8 → getrandom 0.2 (no "js" feature) ❌
|
|
```
|
|
|
|
**After (Success)**:
|
|
```
|
|
rvlite
|
|
├─ getrandom 0.2 (features = ["js"]) ✅
|
|
└─ ruvector-core (default-features = false, features = ["memory-only"])
|
|
└─ rand 0.8 → getrandom 0.2 (gets "js" via top-level) ✅
|
|
```
|
|
|
|
### Why Target-Specific Dependency Works
|
|
|
|
When you add:
|
|
```toml
|
|
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
|
getrandom = { workspace = true, features = ["js"] }
|
|
```
|
|
|
|
Cargo's feature unification sees:
|
|
1. `rvlite` depends on `getrandom` with `["js"]` (only on WASM)
|
|
2. `rand_core` depends on `getrandom` (no features)
|
|
3. Cargo unifies to: `getrandom` with `["js"]` ✅
|
|
|
|
This works because feature unification is additive within the same version.
|
|
|
|
---
|
|
|
|
## 🎯 Success Metrics
|
|
|
|
- ✅ WASM builds without errors
|
|
- ✅ No getrandom version conflicts
|
|
- ✅ No dependency on incompatible crates (mmap-rs)
|
|
- ✅ Bundle size remains optimal (15.90 KB for POC)
|
|
- ✅ ruvector-core integrated and ready to use
|
|
- ✅ Build time < 12 seconds
|
|
- ✅ Tree-shaking working (unused code removed)
|
|
|
|
---
|
|
|
|
## 📚 References
|
|
|
|
- [getrandom WASM support](https://docs.rs/getrandom/latest/getrandom/#webassembly-support)
|
|
- [Cargo target dependencies](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#platform-specific-dependencies)
|
|
- [Feature unification](https://doc.rust-lang.org/cargo/reference/features.html#feature-unification)
|
|
- [wasm-pack guide](https://rustwasm.github.io/wasm-pack/)
|
|
|
|
---
|
|
|
|
**Resolution Date**: 2025-12-09
|
|
**Status**: ✅ COMPLETE
|
|
**Ready for**: Vector operations implementation
|