git-subtree-dir: vendor/ruvector git-subtree-split: b64c21726f2bb37286d9ee36a7869fef60cc6900
203 lines
5.8 KiB
Markdown
203 lines
5.8 KiB
Markdown
# ruvector-wasm Integration Status
|
|
|
|
## Summary
|
|
|
|
The ruvector-wasm crate has been updated to integrate ruvector-collections and ruvector-filter functionality. However, compilation is currently blocked by pre-existing issues in ruvector-core.
|
|
|
|
## Changes Made
|
|
|
|
### 1. Cargo.toml Updates
|
|
|
|
#### Added Dependencies:
|
|
```toml
|
|
ruvector-collections = { path = "../ruvector-collections", optional = true }
|
|
ruvector-filter = { path = "../ruvector-filter", optional = true }
|
|
getrandom02 = { package = "getrandom", version = "0.2", features = ["js"] }
|
|
```
|
|
|
|
#### Added Features:
|
|
```toml
|
|
[features]
|
|
collections = ["dep:ruvector-collections", "dep:ruvector-filter"]
|
|
```
|
|
|
|
#### WASM Configuration:
|
|
```toml
|
|
[target.'cfg(target_arch = "wasm32")'.dependencies]
|
|
getrandom = { workspace = true, features = ["wasm_js"] }
|
|
```
|
|
|
|
### 2. src/lib.rs Updates
|
|
|
|
#### Added CollectionManager (Lines 411-587):
|
|
- `new(base_path: Option<String>)` - Create collection manager
|
|
- `create_collection(name, dimensions, metric)` - Create new collection
|
|
- `list_collections()` - List all collections
|
|
- `delete_collection(name)` - Delete a collection
|
|
- `get_collection(name)` - Get collection's VectorDB
|
|
- `create_alias(alias, collection)` - Create an alias
|
|
- `delete_alias(alias)` - Delete an alias
|
|
- `list_aliases()` - List all aliases
|
|
|
|
#### Added FilterBuilder (Lines 591-799):
|
|
- `eq(field, value)` - Equality filter
|
|
- `ne(field, value)` - Not-equal filter
|
|
- `gt(field, value)` - Greater-than filter
|
|
- `gte(field, value)` - Greater-than-or-equal filter
|
|
- `lt(field, value)` - Less-than filter
|
|
- `lte(field, value)` - Less-than-or-equal filter
|
|
- `in_values(field, values)` - IN filter
|
|
- `match_text(field, text)` - Text match filter
|
|
- `geo_radius(field, lat, lon, radius_m)` - Geo radius filter
|
|
- `and(filters)` - AND combinator
|
|
- `or(filters)` - OR combinator
|
|
- `not(filter)` - NOT combinator
|
|
- `exists(field)` - Field exists filter
|
|
- `is_null(field)` - Field is null filter
|
|
- `to_json()` - Convert to JavaScript object
|
|
- `get_fields()` - Get referenced field names
|
|
|
|
## Current Issues
|
|
|
|
### Compilation Blockers
|
|
|
|
The ruvector-core crate has conditional compilation issues that prevent WASM builds:
|
|
|
|
1. **redb dependency**: Code in `error.rs` uses `redb` types without `#[cfg(feature = "storage")]` guards
|
|
2. **hnsw_rs dependency**: Code in `index/hnsw.rs` uses `hnsw_rs` without `#[cfg(feature = "hnsw")]` guards
|
|
3. **uuid dependency**: Some code uses `uuid::Uuid` without proper feature guards
|
|
|
|
### Architectural Limitations
|
|
|
|
**Collections and Filter in WASM**: The ruvector-collections crate relies on file I/O and memory-mapped files (via mmap-rs), which are not available in browser WASM environments. These features are marked as optional and require the `collections` feature to be enabled.
|
|
|
|
## Usage
|
|
|
|
### Standard WASM Build (Browser):
|
|
```bash
|
|
cd crates/ruvector-wasm
|
|
cargo build --target wasm32-unknown-unknown --release
|
|
```
|
|
|
|
This builds only the core VectorDB functionality without collections or filter support.
|
|
|
|
### WASM with Collections (WASI/Server):
|
|
```bash
|
|
cargo build --target wasm32-unknown-unknown --release --features collections
|
|
```
|
|
|
|
**Note**: This requires a WASM runtime with file system support (e.g., WASI) and will not work in browsers.
|
|
|
|
## JavaScript API Examples
|
|
|
|
### CollectionManager:
|
|
```javascript
|
|
import { CollectionManager } from 'ruvector-wasm';
|
|
|
|
// Create manager
|
|
const manager = new CollectionManager();
|
|
|
|
// Create collection
|
|
manager.createCollection("documents", 384, "cosine");
|
|
|
|
// List collections
|
|
const collections = manager.listCollections();
|
|
|
|
// Create alias
|
|
manager.createAlias("current_docs", "documents");
|
|
|
|
// Get collection
|
|
const db = manager.getCollection("current_docs");
|
|
|
|
// Use the VectorDB
|
|
const id = db.insert(vector, "doc1", { title: "Hello" });
|
|
```
|
|
|
|
### FilterBuilder:
|
|
```javascript
|
|
import { FilterBuilder } from 'ruvector-wasm';
|
|
|
|
// Simple equality filter
|
|
const filter1 = FilterBuilder.eq("status", "active");
|
|
|
|
// Complex filter
|
|
const filter2 = FilterBuilder.and([
|
|
FilterBuilder.eq("status", "active"),
|
|
FilterBuilder.or([
|
|
FilterBuilder.gte("age", 18),
|
|
FilterBuilder.lt("priority", 10)
|
|
])
|
|
]);
|
|
|
|
// Geo filter
|
|
const filter3 = FilterBuilder.geoRadius(
|
|
"location",
|
|
40.7128, // latitude
|
|
-74.0060, // longitude
|
|
1000 // radius in meters
|
|
);
|
|
|
|
// Convert to JSON for use with search
|
|
const filterJson = filter.toJson();
|
|
const results = db.search(queryVector, 10, filterJson);
|
|
```
|
|
|
|
## Required Fixes
|
|
|
|
To make this fully functional, the following changes are needed in ruvector-core:
|
|
|
|
### 1. Add cfg guards to error.rs:
|
|
```rust
|
|
#[cfg(feature = "storage")]
|
|
impl From<redb::Error> for RuvectorError {
|
|
// ...
|
|
}
|
|
```
|
|
|
|
### 2. Add cfg guards to index/hnsw.rs:
|
|
```rust
|
|
#[cfg(feature = "hnsw")]
|
|
use hnsw_rs::prelude::*;
|
|
|
|
#[cfg(feature = "hnsw")]
|
|
pub struct HnswIndex {
|
|
// ...
|
|
}
|
|
```
|
|
|
|
### 3. Ensure memory-only feature works:
|
|
The `memory-only` feature should be a complete alternative that doesn't require redb or hnsw_rs.
|
|
|
|
## Files Modified
|
|
|
|
1. `/home/user/ruvector/crates/ruvector-wasm/Cargo.toml`
|
|
2. `/home/user/ruvector/crates/ruvector-wasm/src/lib.rs`
|
|
3. `/home/user/ruvector/Cargo.toml` (attempted patch section, later removed)
|
|
|
|
## Verification
|
|
|
|
Once ruvector-core's conditional compilation issues are fixed, verify with:
|
|
|
|
```bash
|
|
# Check basic WASM build
|
|
cargo check --target wasm32-unknown-unknown
|
|
|
|
# Check with collections feature (requires WASI)
|
|
cargo check --target wasm32-unknown-unknown --features collections
|
|
|
|
# Build release
|
|
cargo build --target wasm32-unknown-unknown --release
|
|
|
|
# Run WASM tests
|
|
wasm-pack test --node
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
1. Fix ruvector-core conditional compilation issues
|
|
2. Add proper cfg guards for all optional dependencies
|
|
3. Test WASM builds with and without collections feature
|
|
4. Add WASM-specific tests for CollectionManager and FilterBuilder
|
|
5. Document WASI requirements for collections feature
|
|
6. Consider creating a pure in-memory alternative to collections for browser use
|