Squashed 'vendor/ruvector/' content from commit b64c2172
git-subtree-dir: vendor/ruvector git-subtree-split: b64c21726f2bb37286d9ee36a7869fef60cc6900
This commit is contained in:
ruv
383
crates/rvlite/docs/GETRANDOM_RESOLUTION_STRATEGY.md
Normal file
383
crates/rvlite/docs/GETRANDOM_RESOLUTION_STRATEGY.md
Normal file
@@ -0,0 +1,383 @@
|
||||
# getrandom Resolution Strategy
|
||||
|
||||
**Status**: Blocked - Complex dependency conflict
|
||||
**Priority**: High (blocks integration with ruvector-core)
|
||||
**Date**: 2025-12-09
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Problem Summary
|
||||
|
||||
RvLite cannot compile to WASM with `ruvector-core` due to conflicting `getrandom` versions in the dependency tree:
|
||||
|
||||
```
|
||||
getrandom 0.2.16 (needs "js" feature)
|
||||
← rand_core 0.6.4
|
||||
← rand 0.8.5
|
||||
← ruvector-core
|
||||
|
||||
getrandom 0.3.4 (needs "wasm_js" cfg flag)
|
||||
← (some other dependency)
|
||||
```
|
||||
|
||||
**Error**:
|
||||
```
|
||||
error: the wasm*-unknown-unknown targets are not supported by default,
|
||||
you may need to enable the "js" feature.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Root Cause Analysis
|
||||
|
||||
### Dependency Chain
|
||||
|
||||
```
|
||||
ruvector-core
|
||||
└─ rand 0.8.5 (workspace dependency)
|
||||
└─ rand_core 0.6.4
|
||||
└─ getrandom 0.2.16 ❌ No WASM support without feature
|
||||
|
||||
something-else
|
||||
└─ getrandom 0.3.4 ❌ Requires cfg flag, not just feature
|
||||
```
|
||||
|
||||
### Why Features Aren't Working
|
||||
|
||||
1. **getrandom 0.2** requires `features = ["js"]` for WASM
|
||||
- We set this in workspace dependencies ✅
|
||||
- BUT: Feature isn't being propagated through `rand` → `rand_core` → `getrandom`
|
||||
|
||||
2. **getrandom 0.3** requires BOTH:
|
||||
- `features = ["wasm_js"]` ✅
|
||||
- `RUSTFLAGS='--cfg getrandom_backend="wasm_js"'` ❌
|
||||
|
||||
3. **Cargo feature unification** doesn't work across major versions
|
||||
- Can't unify 0.2 and 0.3 features
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Attempted Solutions
|
||||
|
||||
### ❌ Attempt 1: Update rand to 0.9
|
||||
```toml
|
||||
rand = "0.9" # Uses getrandom 0.3
|
||||
```
|
||||
**Result**: FAILED - Other crates require `rand = "^0.8"`
|
||||
- ruvector-router-core
|
||||
- statistical (via ruvector-bench)
|
||||
|
||||
### ❌ Attempt 2: Force getrandom 0.2 everywhere
|
||||
```toml
|
||||
getrandom = { version = "0.2", features = ["js"] }
|
||||
```
|
||||
**Result**: FAILED - Some dependency pulls in getrandom 0.3
|
||||
|
||||
### ❌ Attempt 3: Configure both versions
|
||||
```toml
|
||||
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
||||
getrandom = { workspace = true, features = ["js"] }
|
||||
```
|
||||
**Result**: FAILED - Features not propagated correctly
|
||||
|
||||
### ❌ Attempt 4: Cargo patch
|
||||
```toml
|
||||
[patch.crates-io]
|
||||
getrandom = { version = "0.2", features = ["js"] }
|
||||
```
|
||||
**Result**: FAILED - Can't patch to same source
|
||||
|
||||
---
|
||||
|
||||
## ✅ Viable Solutions (Ranked by Effort)
|
||||
|
||||
### Option A: Exclude Problematic Dependencies (Immediate)
|
||||
|
||||
**Approach**: Build rvlite without dependencies that require getrandom
|
||||
|
||||
**Changes**:
|
||||
1. Temporarily exclude `ruvector-core` (already done in POC)
|
||||
2. Build minimal WASM package
|
||||
3. Integrate other features after getrandom is resolved
|
||||
|
||||
**Pros**:
|
||||
- ✅ POC already works (15.90 KB)
|
||||
- ✅ Proves architecture
|
||||
- ✅ Can publish minimal version immediately
|
||||
|
||||
**Cons**:
|
||||
- ❌ No vector operations yet
|
||||
- ❌ Delays full feature integration
|
||||
|
||||
**Timeline**: Already complete
|
||||
**Recommendation**: ★★★ Use for v0.1.0 release
|
||||
|
||||
---
|
||||
|
||||
### Option B: Fork and Patch rand (1-2 days)
|
||||
|
||||
**Approach**: Create workspace patch of `rand` that explicitly enables getrandom features
|
||||
|
||||
```toml
|
||||
[patch.crates-io]
|
||||
rand = { path = "./patches/rand-0.8.5-wasm-fix" }
|
||||
```
|
||||
|
||||
**Changes**:
|
||||
1. Fork `rand 0.8.5` to workspace
|
||||
2. Update its `Cargo.toml` to explicitly enable getrandom/js:
|
||||
```toml
|
||||
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
||||
getrandom = { version = "0.2", features = ["js"] }
|
||||
```
|
||||
3. Test and validate
|
||||
|
||||
**Pros**:
|
||||
- ✅ Minimal changes
|
||||
- ✅ Works with existing rand 0.8 ecosystem
|
||||
- ✅ Can upstream patch to rand maintainers
|
||||
|
||||
**Cons**:
|
||||
- ⚠️ Requires maintaining fork
|
||||
- ⚠️ Need to keep in sync with upstream
|
||||
|
||||
**Timeline**: 1-2 days
|
||||
**Recommendation**: ★★★★ Best mid-term solution
|
||||
|
||||
---
|
||||
|
||||
### Option C: Update All Dependencies to rand 0.9 (3-5 days)
|
||||
|
||||
**Approach**: Update or fork all crates that require rand 0.8
|
||||
|
||||
**Changes**:
|
||||
1. Update `ruvector-router-core` to use rand 0.9
|
||||
2. Replace or update `statistical` dependency in ruvector-bench
|
||||
3. Test all affected crates
|
||||
4. Update documentation
|
||||
|
||||
**Pros**:
|
||||
- ✅ Clean long-term solution
|
||||
- ✅ Uses latest dependencies
|
||||
- ✅ Better WASM support
|
||||
|
||||
**Cons**:
|
||||
- ❌ Requires changes to multiple crates
|
||||
- ❌ Risk of breaking changes
|
||||
- ❌ Significant testing required
|
||||
|
||||
**Timeline**: 3-5 days
|
||||
**Recommendation**: ★★ Long-term solution
|
||||
|
||||
---
|
||||
|
||||
### Option D: Use Build Script for WASM Target (2-3 hours)
|
||||
|
||||
**Approach**: Add build.rs script to configure getrandom for WASM builds
|
||||
|
||||
**Changes**:
|
||||
```rust
|
||||
// crates/rvlite/build.rs
|
||||
fn main() {
|
||||
if cfg!(target_arch = "wasm32") {
|
||||
println!("cargo:rustc-cfg=getrandom_backend=\"wasm_js\"");
|
||||
// Force getrandom to use js backend
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
- ✅ Quick to implement
|
||||
- ✅ No dependency changes
|
||||
- ✅ Works for rvlite specifically
|
||||
|
||||
**Cons**:
|
||||
- ⚠️ Only fixes rvlite, not other WASM crates
|
||||
- ⚠️ May not work for transitive dependencies
|
||||
- ⚠️ Build script complexity
|
||||
|
||||
**Timeline**: 2-3 hours
|
||||
**Recommendation**: ★★★ Worth trying first
|
||||
|
||||
---
|
||||
|
||||
### Option E: Wait for Upstream Fixes (Unknown timeline)
|
||||
|
||||
**Approach**: Report issue to rand/getrandom maintainers and wait
|
||||
|
||||
**Pros**:
|
||||
- ✅ Cleanest solution
|
||||
- ✅ Benefits entire ecosystem
|
||||
|
||||
**Cons**:
|
||||
- ❌ Unknown timeline
|
||||
- ❌ Blocks rvlite development
|
||||
- ❌ No guarantee of fix
|
||||
|
||||
**Timeline**: Unknown (weeks to months)
|
||||
**Recommendation**: ★ Not viable for immediate progress
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Recommended Path Forward
|
||||
|
||||
### Phase 1: Immediate (Today)
|
||||
|
||||
**Use Option A** - Ship POC as v0.1.0:
|
||||
- ✅ Minimal WASM package (15.90 KB)
|
||||
- ✅ Validates architecture
|
||||
- ✅ Demonstrates browser integration
|
||||
- ✅ Proves build system works
|
||||
|
||||
**Deliverable**: v0.1.0-poc published to npm
|
||||
|
||||
### Phase 2: Short-term (This Week)
|
||||
|
||||
**Try Option D** - Build script approach:
|
||||
1. Add build.rs to rvlite (2-3 hours)
|
||||
2. Test WASM compilation
|
||||
3. If works → integrate ruvector-core
|
||||
4. If fails → proceed to Option B
|
||||
|
||||
**Deliverable**: rvlite with ruvector-core integration OR documented failure
|
||||
|
||||
### Phase 3: Medium-term (Next Week)
|
||||
|
||||
**Implement Option B** - Fork and patch rand:
|
||||
1. Create `patches/rand-0.8.5-wasm` directory
|
||||
2. Add explicit getrandom feature enablement
|
||||
3. Apply patch via `[patch.crates-io]`
|
||||
4. Validate all WASM crates compile
|
||||
5. Submit upstream PR to rand
|
||||
|
||||
**Deliverable**: Full ruvector-core integration working
|
||||
|
||||
### Phase 4: Long-term (Future)
|
||||
|
||||
**Migrate to Option C** - Update to rand 0.9:
|
||||
- Update dependencies as ecosystem stabilizes
|
||||
- Remove patches when upstream fixes land
|
||||
- Clean up temporary workarounds
|
||||
|
||||
---
|
||||
|
||||
## 📝 Implementation Notes
|
||||
|
||||
### For Option D (Build Script):
|
||||
|
||||
```rust
|
||||
// crates/rvlite/build.rs
|
||||
use std::env;
|
||||
|
||||
fn main() {
|
||||
let target = env::var("TARGET").unwrap();
|
||||
|
||||
if target.starts_with("wasm32") {
|
||||
// Configure getrandom for WASM
|
||||
println!("cargo:rustc-env=GETRANDOM_BACKEND=wasm_js");
|
||||
println!("cargo:rustc-cfg=getrandom_backend=\"wasm_js\"");
|
||||
|
||||
// Force feature propagation
|
||||
println!("cargo:rustc-check-cfg=cfg(getrandom_backend, values(\"wasm_js\"))");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### For Option B (Patch):
|
||||
|
||||
1. Clone rand 0.8.5:
|
||||
```bash
|
||||
mkdir -p patches
|
||||
cd patches
|
||||
git clone https://github.com/rust-random/rand.git
|
||||
cd rand
|
||||
git checkout 0.8.5
|
||||
```
|
||||
|
||||
2. Modify `Cargo.toml`:
|
||||
```toml
|
||||
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
||||
getrandom = { version = "0.2", features = ["js"] }
|
||||
```
|
||||
|
||||
3. Apply patch in workspace:
|
||||
```toml
|
||||
[patch.crates-io]
|
||||
rand = { path = "./patches/rand" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing Strategy
|
||||
|
||||
After implementing any solution:
|
||||
|
||||
1. **Build Test**:
|
||||
```bash
|
||||
wasm-pack build --target web --release
|
||||
```
|
||||
|
||||
2. **Size Check**:
|
||||
```bash
|
||||
ls -lh pkg/*.wasm
|
||||
gzip -c pkg/*.wasm | wc -c
|
||||
```
|
||||
|
||||
3. **Browser Test**:
|
||||
```bash
|
||||
python3 -m http.server 8000
|
||||
# Open examples/demo.html
|
||||
```
|
||||
|
||||
4. **Integration Test**:
|
||||
```rust
|
||||
#[wasm_bindgen_test]
|
||||
fn test_ruvector_core_integration() {
|
||||
let db = RvLite::new().unwrap();
|
||||
// Test vector operations from ruvector-core
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Decision Matrix
|
||||
|
||||
| Option | Effort | Risk | Timeline | Maintainability | Recommendation |
|
||||
|--------|--------|------|----------|-----------------|----------------|
|
||||
| A: Exclude deps | None | Low | 0 days | High | ★★★ (v0.1.0) |
|
||||
| B: Patch rand | Low | Low | 1-2 days | Medium | ★★★★ (Best) |
|
||||
| C: Update deps | High | Medium | 3-5 days | High | ★★ (Future) |
|
||||
| D: Build script | Very Low | Medium | 2-3 hours | Medium | ★★★ (Try first) |
|
||||
| E: Wait upstream | None | High | Unknown | High | ★ (Not viable) |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Action Plan
|
||||
|
||||
**Immediate Next Steps**:
|
||||
|
||||
1. **Document current state** ✅ (this document)
|
||||
2. **Try Option D** (build script) - 2-3 hours
|
||||
3. **If D fails, implement Option B** (patch) - 1-2 days
|
||||
4. **Ship v0.1.0 with Option A** if needed for milestone
|
||||
|
||||
**Success Criteria**:
|
||||
- [ ] WASM builds with ruvector-core
|
||||
- [ ] getrandom works in browser
|
||||
- [ ] No WASM compilation errors
|
||||
- [ ] Bundle size < 1 MB with ruvector-core
|
||||
|
||||
---
|
||||
|
||||
## 📚 References
|
||||
|
||||
- [getrandom 0.2 WASM docs](https://docs.rs/getrandom/0.2/getrandom/#webassembly-support)
|
||||
- [getrandom 0.3 WASM docs](https://docs.rs/getrandom/0.3/getrandom/#webassembly-support)
|
||||
- [Cargo patch documentation](https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html#the-patch-section)
|
||||
- [wasm-bindgen guide](https://rustwasm.github.io/wasm-bindgen/)
|
||||
|
||||
---
|
||||
|
||||
**Status**: Awaiting decision on path forward
|
||||
**Next Action**: Try Option D (build script approach)
|
||||
**Estimated Resolution**: 1-3 days depending on chosen option
|
||||
Reference in New Issue
Block a user