Squashed 'vendor/ruvector/' content from commit b64c2172
git-subtree-dir: vendor/ruvector git-subtree-split: b64c21726f2bb37286d9ee36a7869fef60cc6900
This commit is contained in:
557
npm/packages/ospipe/README.md
Normal file
557
npm/packages/ospipe/README.md
Normal file
@@ -0,0 +1,557 @@
|
||||
# @ruvector/ospipe
|
||||
|
||||
**RuVector-enhanced personal AI memory SDK for Screenpipe**
|
||||
|
||||
[](https://www.npmjs.com/package/@ruvector/ospipe)
|
||||
[](https://www.npmjs.com/package/@ruvector/ospipe-wasm)
|
||||
[](https://crates.io/crates/ospipe)
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[](https://webassembly.org/)
|
||||
|
||||
---
|
||||
|
||||
## What is OSpipe?
|
||||
|
||||
[Screenpipe](https://github.com/screenpipe/screenpipe) is an open-source desktop application that continuously records your screen, audio, and UI interactions locally. It builds a searchable timeline of everything you see, hear, and do on your computer. Out of the box, Screenpipe stores its data in SQLite with FTS5 full-text indexing -- effective for keyword lookups, but limited to literal string matching. If you search for "auth discussion," you will not find a frame that says "we talked about login security."
|
||||
|
||||
OSpipe replaces Screenpipe's storage and search backend with the [RuVector](https://github.com/ruvnet/ruvector) ecosystem -- a collection of 70+ Rust crates providing HNSW vector search, graph neural networks, attention mechanisms, delta-change tracking, and more. Instead of keyword matching, OSpipe embeds every captured frame into a high-dimensional vector space and performs approximate nearest neighbor search, delivering true semantic recall. A query like *"what was that API we discussed in standup?"* will surface the relevant audio transcription even if those exact words never appeared.
|
||||
|
||||
Everything stays local and private. OSpipe processes all data on-device with no cloud dependency. The safety gate automatically detects and redacts PII -- credit card numbers, Social Security numbers, and email addresses -- before content ever reaches the vector store. A cosine-similarity deduplication window prevents consecutive identical frames (like a static desktop) from bloating storage. Age-based quantization progressively compresses older embeddings from 32-bit floats down to 1-bit binary, cutting long-term memory usage by 97%.
|
||||
|
||||
**Ask your computer what you saw, heard, and did -- with semantic understanding.**
|
||||
|
||||
---
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
npm install @ruvector/ospipe
|
||||
```
|
||||
|
||||
Also available:
|
||||
|
||||
| Package | Install | Description |
|
||||
|---------|---------|-------------|
|
||||
| [`@ruvector/ospipe`](https://www.npmjs.com/package/@ruvector/ospipe) | `npm install @ruvector/ospipe` | TypeScript SDK for Node.js and browser |
|
||||
| [`@ruvector/ospipe-wasm`](https://www.npmjs.com/package/@ruvector/ospipe-wasm) | `npm install @ruvector/ospipe-wasm` | WASM bindings (145 KB) for browser-only use |
|
||||
| [`ospipe`](https://crates.io/crates/ospipe) | `cargo add ospipe` | Rust crate with full pipeline |
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **Semantic Vector Search** -- HNSW index via `ruvector-core` with 61us p50 query latency
|
||||
- **Knowledge Graph** -- Cypher queries over extracted entities (people, apps, topics, meetings)
|
||||
- **Temporal Deltas** -- track how content changed over time with delta-behavior analysis
|
||||
- **Attention Streaming** -- real-time SSE stream of attention-weighted events
|
||||
- **PII Safety Gate** -- automatic redaction of credit card numbers, SSNs, and email addresses before storage
|
||||
- **Frame Deduplication** -- cosine similarity sliding window eliminates near-duplicate captures
|
||||
- **Query Router** -- automatically routes queries to the optimal backend (Semantic, Keyword, Graph, Temporal, or Hybrid)
|
||||
- **Hybrid Search** -- weighted combination of semantic vector similarity and keyword term overlap
|
||||
- **WASM Support** -- runs entirely in the browser with bundles from 11.8KB (micro) to 350KB (full)
|
||||
- **Configurable Quantization** -- 4-tier age-based compression: f32 -> int8 -> product -> binary (97% savings)
|
||||
- **Retry + Timeout** -- exponential backoff, AbortSignal support, configurable timeout
|
||||
- **Screenpipe Compatible** -- backward-compatible `queryScreenpipe()` for existing code
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
OSpipe Ingestion Pipeline
|
||||
=========================
|
||||
|
||||
Screenpipe -----> Capture -----> Safety Gate -----> Dedup -----> Embed -----> VectorStore
|
||||
(Screen/Audio/UI) (CapturedFrame) (PII Redaction) (Cosine Window) (HNSW) |
|
||||
|
|
||||
Search Router <------------+
|
||||
| | | | |
|
||||
Semantic Keyword Graph Temporal Hybrid
|
||||
```
|
||||
|
||||
Frames flow left to right through the ingestion pipeline. Each captured frame passes through:
|
||||
|
||||
1. **Safety Gate** -- PII detection and redaction; content may be allowed, redacted, or denied
|
||||
2. **Deduplication** -- cosine similarity check against a sliding window of recent embeddings
|
||||
3. **Embedding** -- text content is encoded into a normalized vector
|
||||
4. **Vector Store** -- the embedding is indexed for approximate nearest neighbor retrieval
|
||||
|
||||
Queries enter through the **Search Router**, which analyzes the query string and dispatches to the optimal backend.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### TypeScript SDK
|
||||
|
||||
```typescript
|
||||
import { OsPipe } from "@ruvector/ospipe";
|
||||
|
||||
const client = new OsPipe({ baseUrl: "http://localhost:3030" });
|
||||
|
||||
// Semantic search across everything you've seen, heard, and done
|
||||
const results = await client.queryRuVector(
|
||||
"what did we discuss about authentication?"
|
||||
);
|
||||
|
||||
for (const hit of results) {
|
||||
console.log(`[${hit.score.toFixed(3)}] ${hit.content}`);
|
||||
console.log(` app: ${hit.metadata.app}, time: ${hit.timestamp}`);
|
||||
}
|
||||
```
|
||||
|
||||
### WASM (Browser)
|
||||
|
||||
```javascript
|
||||
import { OsPipeWasm } from "@ruvector/ospipe-wasm";
|
||||
|
||||
// Initialize with 384-dimensional embeddings
|
||||
const pipe = new OsPipeWasm(384);
|
||||
|
||||
// Embed and insert content
|
||||
const embedding = pipe.embed_text("meeting notes about auth migration to OAuth2");
|
||||
pipe.insert("frame-001", embedding, '{"app":"Chrome","window":"Jira"}', Date.now());
|
||||
|
||||
// Embed a query and search
|
||||
const queryEmbedding = pipe.embed_text("what was the auth discussion about?");
|
||||
const results = pipe.search(queryEmbedding, 5);
|
||||
console.log("Results:", results);
|
||||
|
||||
// Safety check before storage
|
||||
const safety = pipe.safety_check("my card is 4111-1111-1111-1111");
|
||||
console.log("Safety:", safety); // "deny"
|
||||
|
||||
// Query routing
|
||||
const route = pipe.route_query("what happened yesterday?");
|
||||
console.log("Route:", route); // "Temporal"
|
||||
|
||||
// Pipeline statistics
|
||||
console.log("Stats:", pipe.stats());
|
||||
```
|
||||
|
||||
### Start the OSpipe Server
|
||||
|
||||
```bash
|
||||
# Using the Rust binary
|
||||
cargo install ospipe
|
||||
ospipe-server --port 3030
|
||||
|
||||
# Or build from source
|
||||
cargo build -p ospipe --release --bin ospipe-server
|
||||
./target/release/ospipe-server --port 3030 --data-dir ~/.ospipe
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparison: Screenpipe vs OSpipe
|
||||
|
||||
| Feature | Screenpipe (FTS5) | OSpipe (RuVector) |
|
||||
|---|---|---|
|
||||
| Search Type | Keyword (FTS5) | Semantic + Keyword + Graph + Temporal |
|
||||
| Search Latency | ~1ms (FTS5) | 61us (HNSW p50) |
|
||||
| Content Relations | None | Knowledge Graph (Cypher) |
|
||||
| Temporal Analysis | Basic SQL | Delta-behavior tracking |
|
||||
| PII Protection | Basic | Credit card, SSN, email redaction |
|
||||
| Deduplication | None | Cosine similarity sliding window |
|
||||
| Browser Support | None | WASM (11.8KB - 350KB) |
|
||||
| Quantization | None | 4-tier age-based (f32 -> binary) |
|
||||
| Privacy | Local-first | Local-first + PII redaction |
|
||||
| Query Routing | None | Auto-routes to optimal backend |
|
||||
| Hybrid Search | None | Weighted semantic + keyword fusion |
|
||||
| Metadata Filtering | SQL WHERE | App, time range, content type, monitor |
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
### Constructor
|
||||
|
||||
```typescript
|
||||
const client = new OsPipe({
|
||||
baseUrl: "http://localhost:3030", // OSpipe server URL
|
||||
apiVersion: "v2", // API version ("v1" | "v2")
|
||||
defaultK: 10, // Default number of results
|
||||
hybridWeight: 0.7, // Semantic vs keyword weight (0-1)
|
||||
rerank: true, // Enable MMR deduplication
|
||||
timeout: 10_000, // Request timeout in ms
|
||||
maxRetries: 3, // Retry attempts for 5xx/network errors
|
||||
});
|
||||
```
|
||||
|
||||
### `queryRuVector(query, options?)` -- Semantic Search
|
||||
|
||||
```typescript
|
||||
const results = await client.queryRuVector("user login issues", {
|
||||
k: 5,
|
||||
metric: "cosine", // "cosine" | "euclidean" | "dot"
|
||||
rerank: true, // MMR deduplication
|
||||
confidence: true, // Include confidence bounds
|
||||
filters: {
|
||||
app: "Chrome",
|
||||
contentType: "screen", // "screen" | "audio" | "ui" | "all"
|
||||
timeRange: { start: "2026-02-12T00:00:00Z", end: "2026-02-12T23:59:59Z" },
|
||||
speaker: "Alice",
|
||||
monitor: 0,
|
||||
language: "en",
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Returns `SearchResult[]`:
|
||||
|
||||
```typescript
|
||||
interface SearchResult {
|
||||
id: string;
|
||||
score: number;
|
||||
content: string;
|
||||
source: "screen" | "audio" | "ui";
|
||||
timestamp: string;
|
||||
metadata: {
|
||||
app?: string;
|
||||
window?: string;
|
||||
monitor?: number;
|
||||
speaker?: string;
|
||||
confidence?: number;
|
||||
language?: string;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### `queryGraph(cypher)` -- Knowledge Graph
|
||||
|
||||
```typescript
|
||||
const result = await client.queryGraph(
|
||||
"MATCH (p:Person)-[:MENTIONED_IN]->(m:Meeting) RETURN p, m LIMIT 10"
|
||||
);
|
||||
|
||||
console.log(result.nodes); // GraphNode[] with id, label, type, properties
|
||||
console.log(result.edges); // GraphEdge[] with source, target, type
|
||||
```
|
||||
|
||||
Node types: `App`, `Window`, `Person`, `Topic`, `Meeting`, `Symbol`.
|
||||
|
||||
### `queryDelta(options)` -- Temporal Changes
|
||||
|
||||
```typescript
|
||||
const deltas = await client.queryDelta({
|
||||
app: "VSCode",
|
||||
timeRange: {
|
||||
start: "2026-02-12T09:00:00Z",
|
||||
end: "2026-02-12T17:00:00Z",
|
||||
},
|
||||
includeChanges: true,
|
||||
});
|
||||
|
||||
for (const delta of deltas) {
|
||||
console.log(`${delta.timestamp} [${delta.app}]`);
|
||||
for (const change of delta.changes) {
|
||||
console.log(` -${change.removed} +${change.added}`);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `streamAttention(options?)` -- Real-Time Events
|
||||
|
||||
```typescript
|
||||
for await (const event of client.streamAttention({
|
||||
threshold: 0.5,
|
||||
categories: ["code_change", "meeting_start"],
|
||||
signal: AbortSignal.timeout(60_000),
|
||||
})) {
|
||||
console.log(`[${event.category}] ${event.summary} (${event.attention})`);
|
||||
}
|
||||
```
|
||||
|
||||
Event categories: `code_change`, `person_mention`, `topic_shift`, `context_switch`, `meeting_start`, `meeting_end`.
|
||||
|
||||
### `routeQuery(query)` -- Query Routing
|
||||
|
||||
```typescript
|
||||
const route = await client.routeQuery("who mentioned auth yesterday?");
|
||||
// route: "semantic" | "keyword" | "graph" | "temporal" | "hybrid"
|
||||
```
|
||||
|
||||
### `stats()` -- Pipeline Statistics
|
||||
|
||||
```typescript
|
||||
const stats = await client.stats();
|
||||
// { totalIngested, totalDeduplicated, totalDenied, storageBytes, indexSize, uptime }
|
||||
```
|
||||
|
||||
### `health()` -- Server Health
|
||||
|
||||
```typescript
|
||||
const health = await client.health();
|
||||
// { status: "ok", version: "0.1.0", backends: ["hnsw", "keyword", "graph"] }
|
||||
```
|
||||
|
||||
### `queryScreenpipe(options)` -- Legacy API
|
||||
|
||||
Backward-compatible with `@screenpipe/js`:
|
||||
|
||||
```typescript
|
||||
const results = await client.queryScreenpipe({
|
||||
q: "meeting notes",
|
||||
contentType: "ocr", // "all" | "ocr" | "audio"
|
||||
limit: 20,
|
||||
appName: "Notion",
|
||||
startTime: "2026-02-12T00:00:00Z",
|
||||
endTime: "2026-02-12T23:59:59Z",
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Safety Gate
|
||||
|
||||
<details>
|
||||
<summary>PII Detection Details</summary>
|
||||
|
||||
The safety gate inspects all captured content before it enters the ingestion pipeline. It operates in three modes:
|
||||
|
||||
| Decision | Behavior | When |
|
||||
|---|---|---|
|
||||
| **Allow** | Content stored as-is | No sensitive patterns detected |
|
||||
| **AllowRedacted** | Content stored with PII replaced by tokens | PII detected, redaction enabled |
|
||||
| **Deny** | Content rejected, not stored | Custom deny pattern matched |
|
||||
|
||||
**Detected PII patterns:**
|
||||
|
||||
- **Credit Cards** -- sequences of 13-16 digits (with optional spaces or dashes) -> `[CC_REDACTED]`
|
||||
- **Social Security Numbers** -- XXX-XX-XXXX format -> `[SSN_REDACTED]`
|
||||
- **Email Addresses** -- word@domain.tld patterns -> `[EMAIL_REDACTED]`
|
||||
- **Sensitive Keywords** (WASM) -- `password`, `secret`, `api_key`, `api-key`, `apikey`, `token`, `private_key`, `private-key`
|
||||
|
||||
**WASM safety API:**
|
||||
|
||||
```javascript
|
||||
pipe.safety_check("my card is 4111-1111-1111-1111"); // "deny"
|
||||
pipe.safety_check("set password to foo123"); // "redact"
|
||||
pipe.safety_check("the weather is nice today"); // "allow"
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Configuration Guide
|
||||
|
||||
<details>
|
||||
<summary>Client Configuration</summary>
|
||||
|
||||
All configuration options with defaults:
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|---|---|---|---|
|
||||
| `baseUrl` | `string` | `"http://localhost:3030"` | OSpipe server URL |
|
||||
| `apiVersion` | `"v1" \| "v2"` | `"v2"` | API version |
|
||||
| `defaultK` | `number` | `10` | Default number of results |
|
||||
| `hybridWeight` | `number` | `0.7` | Semantic vs keyword weight (0 = pure keyword, 1 = pure semantic) |
|
||||
| `rerank` | `boolean` | `true` | Enable MMR deduplication |
|
||||
| `timeout` | `number` | `10000` | Request timeout in milliseconds |
|
||||
| `maxRetries` | `number` | `3` | Retry attempts for 5xx/network errors |
|
||||
|
||||
**Retry behavior:** The SDK uses exponential backoff starting at 300ms. Only network errors and HTTP 5xx responses are retried. Client errors (4xx) are never retried. Each request has an independent `AbortController` timeout.
|
||||
|
||||
```typescript
|
||||
// High-throughput configuration
|
||||
const client = new OsPipe({
|
||||
baseUrl: "http://localhost:3030",
|
||||
defaultK: 50,
|
||||
hybridWeight: 0.9, // lean heavily toward semantic
|
||||
timeout: 30_000, // 30s for large result sets
|
||||
maxRetries: 5,
|
||||
});
|
||||
|
||||
// Low-latency configuration
|
||||
const fast = new OsPipe({
|
||||
defaultK: 3,
|
||||
hybridWeight: 1.0, // pure semantic, skip keyword
|
||||
rerank: false, // skip MMR reranking
|
||||
timeout: 2_000,
|
||||
maxRetries: 0, // fail fast, no retries
|
||||
});
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Server Configuration (Rust)</summary>
|
||||
|
||||
The OSpipe server is configured via `OsPipeConfig` with nested subsystem configs. All fields have sensible defaults.
|
||||
|
||||
| Subsystem | Key Fields | Defaults |
|
||||
|-----------|-----------|----------|
|
||||
| **Capture** | `fps`, `audio_chunk_secs`, `excluded_apps`, `skip_private_windows` | 1.0 fps, 30s chunks, excludes 1Password/Keychain |
|
||||
| **Storage** | `embedding_dim`, `hnsw_m`, `hnsw_ef_construction`, `dedup_threshold` | 384 dims, M=32, ef=200, 0.95 threshold |
|
||||
| **Search** | `default_k`, `hybrid_weight`, `mmr_lambda`, `rerank_enabled` | k=10, 0.7 hybrid, 0.5 MMR lambda |
|
||||
| **Safety** | `pii_detection`, `credit_card_redaction`, `ssn_redaction`, `custom_patterns` | All enabled, no custom patterns |
|
||||
|
||||
```bash
|
||||
# Start with defaults
|
||||
ospipe-server --port 3030
|
||||
|
||||
# Custom data directory
|
||||
ospipe-server --port 3030 --data-dir /var/lib/ospipe
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## WASM Deployment
|
||||
|
||||
<details>
|
||||
<summary>Bundle Tiers & Web Worker Setup</summary>
|
||||
|
||||
### Bundle Tiers
|
||||
|
||||
OSpipe provides four WASM bundle sizes depending on which features you need:
|
||||
|
||||
| Tier | Size | Features |
|
||||
|---|---|---|
|
||||
| **Micro** | 11.8KB | Embedding + vector search only |
|
||||
| **Standard** | 225KB | Full pipeline (embed, insert, search, filtered search) |
|
||||
| **Full** | 350KB | + deduplication + safety gate + query routing |
|
||||
| **AI** | 2.5MB | + on-device neural inference (ONNX) |
|
||||
|
||||
### Web Worker Setup
|
||||
|
||||
For best performance, run OSpipe in a Web Worker to avoid blocking the main thread:
|
||||
|
||||
```javascript
|
||||
// worker.js
|
||||
import { OsPipeWasm } from "@ruvector/ospipe-wasm";
|
||||
|
||||
const pipe = new OsPipeWasm(384);
|
||||
|
||||
self.onmessage = (event) => {
|
||||
const { type, payload } = event.data;
|
||||
|
||||
switch (type) {
|
||||
case "insert":
|
||||
const emb = pipe.embed_text(payload.text);
|
||||
pipe.insert(payload.id, emb, JSON.stringify(payload.metadata), Date.now());
|
||||
self.postMessage({ type: "inserted", id: payload.id });
|
||||
break;
|
||||
|
||||
case "search":
|
||||
const queryEmb = pipe.embed_text(payload.query);
|
||||
const results = pipe.search(queryEmb, payload.k || 10);
|
||||
self.postMessage({ type: "results", data: results });
|
||||
break;
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### SharedArrayBuffer
|
||||
|
||||
For multi-threaded WASM (e.g., parallel batch embedding), set the required headers:
|
||||
|
||||
```
|
||||
Cross-Origin-Opener-Policy: same-origin
|
||||
Cross-Origin-Embedder-Policy: require-corp
|
||||
```
|
||||
|
||||
### WASM API Reference
|
||||
|
||||
| Method | Parameters | Returns | Description |
|
||||
|---|---|---|---|
|
||||
| `new(dimension)` | `number` | `OsPipeWasm` | Constructor |
|
||||
| `insert(id, embedding, metadata, timestamp)` | `string, Float32Array, string, number` | `void` | Insert a frame |
|
||||
| `search(query_embedding, k)` | `Float32Array, number` | `JSON array` | Semantic search |
|
||||
| `search_filtered(query_embedding, k, start, end)` | `Float32Array, number, number, number` | `JSON array` | Time-filtered search |
|
||||
| `is_duplicate(embedding, threshold)` | `Float32Array, number` | `boolean` | Deduplication check |
|
||||
| `embed_text(text)` | `string` | `Float32Array` | Hash-based text embedding |
|
||||
| `batch_embed(texts)` | `string[]` | `Float32Array[]` | Batch text embedding |
|
||||
| `safety_check(content)` | `string` | `string` | Returns "allow", "redact", or "deny" |
|
||||
| `route_query(query)` | `string` | `string` | Returns "Semantic", "Keyword", "Graph", or "Temporal" |
|
||||
| `len()` | -- | `number` | Number of stored embeddings |
|
||||
| `stats()` | -- | `string` (JSON) | Pipeline statistics |
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Quantization Tiers
|
||||
|
||||
<details>
|
||||
<summary>Age-Based Memory Compression</summary>
|
||||
|
||||
OSpipe progressively compresses older embeddings to reduce long-term storage costs. The default quantization schedule:
|
||||
|
||||
| Age | Method | Bits/Dim | Memory vs f32 | Description |
|
||||
|---|---|---|---|---|
|
||||
| 0 hours | None (f32) | 32 | 100% | Full precision for recent content |
|
||||
| 24 hours | Scalar (int8) | 8 | 25% | Minimal quality loss, 4x compression |
|
||||
| 1 week | Product | ~2 | ~6% | Codebook-based compression |
|
||||
| 30 days | Binary | 1 | 3% | Single bit per dimension, 97% savings |
|
||||
|
||||
### Memory Estimate
|
||||
|
||||
For 1 million frames at 384 dimensions:
|
||||
|
||||
| Tier | Bytes/Vector | Total (1M vectors) |
|
||||
|---|---|---|
|
||||
| f32 | 1,536 | 1.43 GB |
|
||||
| int8 | 384 | 366 MB |
|
||||
| Product | ~96 | ~91 MB |
|
||||
| Binary | 48 | 46 MB |
|
||||
|
||||
With the default age distribution (most content aging past 30 days), long-term average storage is approximately **50-80 MB per million frames**.
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## RuVector Crate Integration
|
||||
|
||||
OSpipe integrates 10 crates from the [RuVector](https://github.com/ruvnet/ruvector) ecosystem:
|
||||
|
||||
| RuVector Crate | OSpipe Usage | Status |
|
||||
|---|---|---|
|
||||
| `ruvector-core` | HNSW vector storage and nearest neighbor search | Integrated |
|
||||
| `ruvector-filter` | Metadata filtering (app, time, content type) | Integrated |
|
||||
| `ruvector-cluster` | Frame deduplication via cosine similarity | Integrated |
|
||||
| `ruvector-delta-core` | Change tracking and delta-behavior analysis | Integrated |
|
||||
| `ruvector-router-core` | Query routing to optimal search backend | Integrated |
|
||||
| `cognitum-gate-kernel` | AI safety gate decisions (allow/redact/deny) | Integrated |
|
||||
| `ruvector-graph` | Knowledge graph for entity relationships | Integrated |
|
||||
| `ruvector-attention` | Content prioritization and relevance weighting | Integrated |
|
||||
| `ruvector-gnn` | Learned search improvement via graph neural nets | Integrated |
|
||||
| `ruqu-algorithms` | Quantum-inspired search diversity (MMR) | Integrated |
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run all 82 tests
|
||||
cargo test -p ospipe
|
||||
|
||||
# Build for WASM (verify compilation)
|
||||
cargo build -p ospipe --target wasm32-unknown-unknown
|
||||
|
||||
# Build with wasm-pack for JS bindings
|
||||
wasm-pack build examples/OSpipe --target web
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
| Package | Description |
|
||||
|---------|-------------|
|
||||
| [`@ruvector/ospipe-wasm`](https://www.npmjs.com/package/@ruvector/ospipe-wasm) | WASM bindings for browser (145 KB) |
|
||||
| [`ospipe`](https://crates.io/crates/ospipe) | Rust crate with full pipeline |
|
||||
| [`ruvector`](https://www.npmjs.com/package/ruvector) | RuVector vector database |
|
||||
|
||||
- [Full Documentation & ADR](https://github.com/ruvnet/ruvector/tree/main/examples/OSpipe)
|
||||
- [RuVector Ecosystem](https://github.com/ruvnet/ruvector) (70+ Rust crates)
|
||||
- [Screenpipe](https://github.com/screenpipe/screenpipe)
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
58
npm/packages/ospipe/package.json
Normal file
58
npm/packages/ospipe/package.json
Normal file
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"name": "@ruvector/ospipe",
|
||||
"version": "0.1.2",
|
||||
"type": "module",
|
||||
"description": "OSpipe SDK - RuVector-enhanced personal AI memory system for Screenpipe pipes",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"module": "dist/index.js",
|
||||
"exports": {
|
||||
".": {
|
||||
"import": "./dist/index.js",
|
||||
"types": "./dist/index.d.ts"
|
||||
},
|
||||
"./wasm": {
|
||||
"import": "./dist/wasm.js",
|
||||
"types": "./dist/wasm.d.ts"
|
||||
}
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"prepublishOnly": "npm run build"
|
||||
},
|
||||
"keywords": [
|
||||
"ospipe",
|
||||
"screenpipe",
|
||||
"ruvector",
|
||||
"ai-memory",
|
||||
"vector-search",
|
||||
"semantic-search",
|
||||
"pipes"
|
||||
],
|
||||
"author": "RuVector Team",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/ruvnet/ruvector.git",
|
||||
"directory": "examples/OSpipe"
|
||||
},
|
||||
"homepage": "https://github.com/ruvnet/ruvector/tree/main/examples/OSpipe#readme",
|
||||
"bugs": {
|
||||
"url": "https://github.com/ruvnet/ruvector/issues"
|
||||
},
|
||||
"files": [
|
||||
"dist/",
|
||||
"README.md"
|
||||
],
|
||||
"publishConfig": {
|
||||
"access": "public"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@screenpipe/js": ">=0.1.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@screenpipe/js": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
}
|
||||
1
npm/packages/ospipe/src/index.d.ts.map
Normal file
1
npm/packages/ospipe/src/index.d.ts.map
Normal file
File diff suppressed because one or more lines are too long
1
npm/packages/ospipe/src/index.js.map
Normal file
1
npm/packages/ospipe/src/index.js.map
Normal file
File diff suppressed because one or more lines are too long
647
npm/packages/ospipe/src/index.ts
Normal file
647
npm/packages/ospipe/src/index.ts
Normal file
@@ -0,0 +1,647 @@
|
||||
/**
|
||||
* @ruvector/ospipe - RuVector-enhanced personal AI memory SDK
|
||||
*
|
||||
* Extends @screenpipe/js with semantic vector search, knowledge graphs,
|
||||
* temporal queries, and AI safety features powered by the RuVector ecosystem.
|
||||
*
|
||||
* @packageDocumentation
|
||||
*/
|
||||
|
||||
// ---- Types ----
|
||||
|
||||
/** Configuration options for the OsPipe client. */
|
||||
export interface OsPipeConfig {
|
||||
/** OSpipe REST API base URL (default: http://localhost:3030) */
|
||||
baseUrl?: string;
|
||||
/** API version (default: "v2") */
|
||||
apiVersion?: "v1" | "v2";
|
||||
/** Default number of results (default: 10) */
|
||||
defaultK?: number;
|
||||
/** Semantic weight for hybrid search 0-1 (default: 0.7) */
|
||||
hybridWeight?: number;
|
||||
/** Enable MMR deduplication (default: true) */
|
||||
rerank?: boolean;
|
||||
/** Request timeout in milliseconds (default: 10000) */
|
||||
timeout?: number;
|
||||
/** Maximum retries for failed requests (default: 3) */
|
||||
maxRetries?: number;
|
||||
}
|
||||
|
||||
/** Options for semantic vector search queries. */
|
||||
export interface SemanticSearchOptions {
|
||||
/** Number of results to return */
|
||||
k?: number;
|
||||
/** Distance metric */
|
||||
metric?: "cosine" | "euclidean" | "dot";
|
||||
/** Metadata filters */
|
||||
filters?: SearchFilters;
|
||||
/** Enable MMR deduplication */
|
||||
rerank?: boolean;
|
||||
/** Include confidence bounds */
|
||||
confidence?: boolean;
|
||||
}
|
||||
|
||||
/** Filters to narrow search results by metadata. */
|
||||
export interface SearchFilters {
|
||||
/** Filter by application name */
|
||||
app?: string;
|
||||
/** Filter by window title */
|
||||
window?: string;
|
||||
/** Filter by content type */
|
||||
contentType?: "screen" | "audio" | "ui" | "all";
|
||||
/** Filter by time range (ISO 8601 strings) */
|
||||
timeRange?: { start: string; end: string };
|
||||
/** Filter by monitor index */
|
||||
monitor?: number;
|
||||
/** Filter by speaker name (audio content) */
|
||||
speaker?: string;
|
||||
/** Filter by language code */
|
||||
language?: string;
|
||||
}
|
||||
|
||||
/** A single search result from a semantic or keyword query. */
|
||||
export interface SearchResult {
|
||||
/** Unique identifier for the content chunk */
|
||||
id: string;
|
||||
/** Relevance score (higher is more relevant) */
|
||||
score: number;
|
||||
/** The matched content text */
|
||||
content: string;
|
||||
/** Source type of the content */
|
||||
source: "screen" | "audio" | "ui";
|
||||
/** ISO 8601 timestamp when the content was captured */
|
||||
timestamp: string;
|
||||
/** Additional metadata about the content */
|
||||
metadata: {
|
||||
app?: string;
|
||||
window?: string;
|
||||
monitor?: number;
|
||||
speaker?: string;
|
||||
confidence?: number;
|
||||
language?: string;
|
||||
};
|
||||
}
|
||||
|
||||
/** Result of a knowledge graph query. */
|
||||
export interface GraphResult {
|
||||
/** Nodes in the result subgraph */
|
||||
nodes: GraphNode[];
|
||||
/** Edges connecting the nodes */
|
||||
edges: GraphEdge[];
|
||||
}
|
||||
|
||||
/** A node in the knowledge graph. */
|
||||
export interface GraphNode {
|
||||
/** Unique node identifier */
|
||||
id: string;
|
||||
/** Human-readable label */
|
||||
label: string;
|
||||
/** Node type category */
|
||||
type: "App" | "Window" | "Person" | "Topic" | "Meeting" | "Symbol";
|
||||
/** Arbitrary key-value properties */
|
||||
properties: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** An edge in the knowledge graph connecting two nodes. */
|
||||
export interface GraphEdge {
|
||||
/** Source node ID */
|
||||
source: string;
|
||||
/** Target node ID */
|
||||
target: string;
|
||||
/** Relationship type */
|
||||
type: string;
|
||||
/** Arbitrary key-value properties */
|
||||
properties: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** A temporal delta result showing changes over time. */
|
||||
export interface DeltaResult {
|
||||
/** ISO 8601 timestamp of the delta snapshot */
|
||||
timestamp: string;
|
||||
/** Application where the change occurred */
|
||||
app: string;
|
||||
/** List of individual changes */
|
||||
changes: DeltaChange[];
|
||||
}
|
||||
|
||||
/** A single positional change within a delta result. */
|
||||
export interface DeltaChange {
|
||||
/** Character position where the change occurred */
|
||||
position: number;
|
||||
/** Text that was removed */
|
||||
removed: string;
|
||||
/** Text that was added */
|
||||
added: string;
|
||||
}
|
||||
|
||||
/** Options for temporal delta queries. */
|
||||
export interface DeltaQueryOptions {
|
||||
/** Filter by application name */
|
||||
app?: string;
|
||||
/** Filter by file path */
|
||||
file?: string;
|
||||
/** Time range for the delta query (ISO 8601 strings) */
|
||||
timeRange: { start: string; end: string };
|
||||
/** Include full change details (default: false) */
|
||||
includeChanges?: boolean;
|
||||
}
|
||||
|
||||
/** An attention-weighted event from the real-time stream. */
|
||||
export interface AttentionEvent {
|
||||
/** Category of the attention event */
|
||||
category:
|
||||
| "code_change"
|
||||
| "person_mention"
|
||||
| "topic_shift"
|
||||
| "context_switch"
|
||||
| "meeting_start"
|
||||
| "meeting_end";
|
||||
/** Attention score 0-1 (higher = more important) */
|
||||
attention: number;
|
||||
/** Human-readable summary of the event */
|
||||
summary: string;
|
||||
/** ISO 8601 timestamp of the event */
|
||||
timestamp: string;
|
||||
/** The underlying search result that triggered the event */
|
||||
source: SearchResult;
|
||||
}
|
||||
|
||||
/** Pipeline statistics from the OsPipe server. */
|
||||
export interface PipelineStats {
|
||||
/** Total number of ingested content chunks */
|
||||
totalIngested: number;
|
||||
/** Total number of deduplicated (skipped) chunks */
|
||||
totalDeduplicated: number;
|
||||
/** Total number of denied (safety filtered) chunks */
|
||||
totalDenied: number;
|
||||
/** Total storage used in bytes */
|
||||
storageBytes: number;
|
||||
/** Number of entries in the vector index */
|
||||
indexSize: number;
|
||||
/** Server uptime in seconds */
|
||||
uptime: number;
|
||||
}
|
||||
|
||||
/** The resolved query route type. */
|
||||
export type QueryRoute = "semantic" | "keyword" | "graph" | "temporal" | "hybrid";
|
||||
|
||||
// ---- Client ----
|
||||
|
||||
/**
|
||||
* OsPipe client for interacting with the RuVector-enhanced personal AI memory system.
|
||||
*
|
||||
* Provides semantic vector search, knowledge graph queries, temporal delta queries,
|
||||
* attention-weighted streaming, and backward-compatible Screenpipe API access.
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { OsPipe } from "@ruvector/ospipe";
|
||||
*
|
||||
* const client = new OsPipe({ baseUrl: "http://localhost:3030" });
|
||||
*
|
||||
* // Semantic search
|
||||
* const results = await client.queryRuVector("authentication flow");
|
||||
*
|
||||
* // Knowledge graph
|
||||
* const graph = await client.queryGraph("MATCH (a:App)-[:USED_BY]->(p:Person) RETURN a, p");
|
||||
*
|
||||
* // Temporal deltas
|
||||
* const deltas = await client.queryDelta({
|
||||
* timeRange: { start: "2026-02-12T00:00:00Z", end: "2026-02-12T23:59:59Z" },
|
||||
* app: "VSCode",
|
||||
* });
|
||||
* ```
|
||||
*/
|
||||
export class OsPipe {
|
||||
private baseUrl: string;
|
||||
private apiVersion: string;
|
||||
private defaultK: number;
|
||||
private hybridWeight: number;
|
||||
private rerank: boolean;
|
||||
private timeout: number;
|
||||
private maxRetries: number;
|
||||
|
||||
constructor(config: OsPipeConfig = {}) {
|
||||
this.baseUrl = config.baseUrl ?? "http://localhost:3030";
|
||||
this.apiVersion = config.apiVersion ?? "v2";
|
||||
this.defaultK = config.defaultK ?? 10;
|
||||
this.hybridWeight = config.hybridWeight ?? 0.7;
|
||||
this.rerank = config.rerank ?? true;
|
||||
this.timeout = config.timeout ?? 10_000;
|
||||
this.maxRetries = config.maxRetries ?? 3;
|
||||
}
|
||||
|
||||
// ---- Internal Helpers ----
|
||||
|
||||
/**
|
||||
* Fetch with exponential backoff retry and per-request timeout.
|
||||
*
|
||||
* Retries are only attempted for network errors and HTTP 5xx responses.
|
||||
* Client errors (4xx) are never retried.
|
||||
*
|
||||
* @param url - Request URL
|
||||
* @param options - Standard RequestInit options
|
||||
* @param retries - Maximum number of retry attempts (default: this.maxRetries)
|
||||
* @param backoffMs - Initial backoff delay in milliseconds (default: 300)
|
||||
* @returns The fetch Response
|
||||
* @throws {Error} After all retries are exhausted or on a non-retryable error
|
||||
*/
|
||||
private async fetchWithRetry(
|
||||
url: string,
|
||||
options?: RequestInit,
|
||||
retries?: number,
|
||||
backoffMs = 300,
|
||||
): Promise<Response> {
|
||||
const maxAttempts = retries ?? this.maxRetries;
|
||||
|
||||
for (let attempt = 0; attempt <= maxAttempts; attempt++) {
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), this.timeout);
|
||||
|
||||
// Merge the timeout signal with any caller-provided signal.
|
||||
const callerSignal = options?.signal;
|
||||
if (callerSignal?.aborted) {
|
||||
clearTimeout(timeoutId);
|
||||
throw new DOMException("The operation was aborted.", "AbortError");
|
||||
}
|
||||
|
||||
// If the caller provided a signal, listen for its abort to propagate.
|
||||
const onCallerAbort = () => controller.abort();
|
||||
callerSignal?.addEventListener("abort", onCallerAbort, { once: true });
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
...options,
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
// Do not retry client errors (4xx).
|
||||
if (response.status >= 400 && response.status < 500) {
|
||||
return response;
|
||||
}
|
||||
|
||||
// Retry on server errors (5xx).
|
||||
if (response.status >= 500 && attempt < maxAttempts) {
|
||||
await this.sleep(backoffMs * Math.pow(2, attempt));
|
||||
continue;
|
||||
}
|
||||
|
||||
return response;
|
||||
} catch (error: unknown) {
|
||||
// If the caller aborted, propagate immediately without retry.
|
||||
if (callerSignal?.aborted) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
// If this was the last attempt, throw.
|
||||
if (attempt >= maxAttempts) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
// Retry on network / timeout errors.
|
||||
await this.sleep(backoffMs * Math.pow(2, attempt));
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
callerSignal?.removeEventListener("abort", onCallerAbort);
|
||||
}
|
||||
}
|
||||
|
||||
// Unreachable, but satisfies the type checker.
|
||||
throw new Error("fetchWithRetry: unexpected exit");
|
||||
}
|
||||
|
||||
/** Sleep helper for backoff delays. */
|
||||
private sleep(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
// ---- Semantic Vector Search ----
|
||||
|
||||
/**
|
||||
* Perform a semantic vector search across all ingested content.
|
||||
*
|
||||
* Uses RuVector HNSW index for approximate nearest neighbor search
|
||||
* with optional MMR deduplication and metadata filtering.
|
||||
*
|
||||
* @param query - Natural language query string
|
||||
* @param options - Search configuration options
|
||||
* @returns Array of search results ranked by relevance
|
||||
* @throws {Error} If the search request fails
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const results = await client.queryRuVector("user login issues", {
|
||||
* k: 5,
|
||||
* filters: { app: "Chrome", contentType: "screen" },
|
||||
* rerank: true,
|
||||
* });
|
||||
* ```
|
||||
*/
|
||||
async queryRuVector(
|
||||
query: string,
|
||||
options: SemanticSearchOptions = {}
|
||||
): Promise<SearchResult[]> {
|
||||
const k = options.k ?? this.defaultK;
|
||||
const response = await this.fetchWithRetry(
|
||||
`${this.baseUrl}/${this.apiVersion}/search`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
query,
|
||||
mode: "semantic",
|
||||
k,
|
||||
metric: options.metric ?? "cosine",
|
||||
filters: options.filters,
|
||||
rerank: options.rerank ?? this.rerank,
|
||||
confidence: options.confidence ?? false,
|
||||
}),
|
||||
},
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Search failed: ${response.statusText}`);
|
||||
}
|
||||
return (await response.json()) as SearchResult[];
|
||||
}
|
||||
|
||||
// ---- Knowledge Graph Query ----
|
||||
|
||||
/**
|
||||
* Query the knowledge graph using a Cypher-like query language.
|
||||
*
|
||||
* The knowledge graph connects apps, windows, people, topics, meetings,
|
||||
* and code symbols with typed relationships extracted from captured content.
|
||||
*
|
||||
* @param cypher - Cypher query string
|
||||
* @returns Graph result containing matched nodes and edges
|
||||
* @throws {Error} If the graph query fails
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const result = await client.queryGraph(
|
||||
* "MATCH (p:Person)-[:MENTIONED_IN]->(m:Meeting) RETURN p, m LIMIT 10"
|
||||
* );
|
||||
* console.log(result.nodes, result.edges);
|
||||
* ```
|
||||
*/
|
||||
async queryGraph(cypher: string): Promise<GraphResult> {
|
||||
const response = await this.fetchWithRetry(
|
||||
`${this.baseUrl}/${this.apiVersion}/graph`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ query: cypher }),
|
||||
},
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Graph query failed: ${response.statusText}`);
|
||||
}
|
||||
return (await response.json()) as GraphResult;
|
||||
}
|
||||
|
||||
// ---- Temporal Delta Query ----
|
||||
|
||||
/**
|
||||
* Query temporal deltas to see how content changed over time.
|
||||
*
|
||||
* Returns a sequence of diffs showing what was added and removed
|
||||
* within the specified time range, optionally filtered by app or file.
|
||||
*
|
||||
* @param options - Delta query configuration
|
||||
* @returns Array of delta results ordered chronologically
|
||||
* @throws {Error} If the delta query fails
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const deltas = await client.queryDelta({
|
||||
* app: "VSCode",
|
||||
* timeRange: {
|
||||
* start: "2026-02-12T09:00:00Z",
|
||||
* end: "2026-02-12T17:00:00Z",
|
||||
* },
|
||||
* includeChanges: true,
|
||||
* });
|
||||
* ```
|
||||
*/
|
||||
async queryDelta(options: DeltaQueryOptions): Promise<DeltaResult[]> {
|
||||
const response = await this.fetchWithRetry(
|
||||
`${this.baseUrl}/${this.apiVersion}/delta`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(options),
|
||||
},
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Delta query failed: ${response.statusText}`);
|
||||
}
|
||||
return (await response.json()) as DeltaResult[];
|
||||
}
|
||||
|
||||
// ---- Attention-Weighted Stream ----
|
||||
|
||||
/**
|
||||
* Stream attention-weighted events from the OsPipe server.
|
||||
*
|
||||
* Yields events in real-time as they are detected by the attention model.
|
||||
* Events are filtered by threshold and category. Uses Server-Sent Events (SSE).
|
||||
*
|
||||
* @param options - Stream configuration
|
||||
* @returns Async generator of attention events
|
||||
* @throws {Error} If the stream connection fails
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* for await (const event of client.streamAttention({
|
||||
* threshold: 0.5,
|
||||
* categories: ["code_change", "meeting_start"],
|
||||
* })) {
|
||||
* console.log(`[${event.category}] ${event.summary} (attention: ${event.attention})`);
|
||||
* }
|
||||
* ```
|
||||
*/
|
||||
async *streamAttention(
|
||||
options: {
|
||||
/** Minimum attention score to emit (0-1) */
|
||||
threshold?: number;
|
||||
/** Only emit events of these categories */
|
||||
categories?: AttentionEvent["category"][];
|
||||
/** AbortSignal to cancel the stream */
|
||||
signal?: AbortSignal;
|
||||
} = {}
|
||||
): AsyncGenerator<AttentionEvent> {
|
||||
const params = new URLSearchParams();
|
||||
if (options.threshold !== undefined) {
|
||||
params.set("threshold", options.threshold.toString());
|
||||
}
|
||||
if (options.categories) {
|
||||
params.set("categories", options.categories.join(","));
|
||||
}
|
||||
|
||||
if (options.signal?.aborted) {
|
||||
throw new DOMException("The operation was aborted.", "AbortError");
|
||||
}
|
||||
|
||||
const url = `${this.baseUrl}/${this.apiVersion}/stream/attention?${params}`;
|
||||
const response = await this.fetchWithRetry(url, {
|
||||
signal: options.signal,
|
||||
});
|
||||
if (!response.ok || !response.body) {
|
||||
throw new Error(`Attention stream failed: ${response.statusText}`);
|
||||
}
|
||||
|
||||
const reader = response.body.getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = "";
|
||||
|
||||
try {
|
||||
while (true) {
|
||||
if (options.signal?.aborted) {
|
||||
break;
|
||||
}
|
||||
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
const lines = buffer.split("\n");
|
||||
buffer = lines.pop() ?? "";
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.startsWith("data: ")) {
|
||||
const data = line.slice(6).trim();
|
||||
if (data && data !== "[DONE]") {
|
||||
yield JSON.parse(data) as AttentionEvent;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Backward-Compatible Screenpipe API ----
|
||||
|
||||
/**
|
||||
* Query the legacy Screenpipe search API (v1 compatible).
|
||||
*
|
||||
* This method provides backward compatibility with the original @screenpipe/js
|
||||
* query interface. For enhanced features, use {@link queryRuVector} instead.
|
||||
*
|
||||
* @param options - Screenpipe-compatible query options
|
||||
* @returns Array of search results
|
||||
* @throws {Error} If the search request fails
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const results = await client.queryScreenpipe({
|
||||
* q: "meeting notes",
|
||||
* contentType: "ocr",
|
||||
* limit: 20,
|
||||
* appName: "Notion",
|
||||
* });
|
||||
* ```
|
||||
*/
|
||||
async queryScreenpipe(options: {
|
||||
/** Search query string */
|
||||
q: string;
|
||||
/** Content type filter */
|
||||
contentType?: "all" | "ocr" | "audio";
|
||||
/** Maximum number of results */
|
||||
limit?: number;
|
||||
/** Start of time range (ISO 8601) */
|
||||
startTime?: string;
|
||||
/** End of time range (ISO 8601) */
|
||||
endTime?: string;
|
||||
/** Filter by application name */
|
||||
appName?: string;
|
||||
}): Promise<SearchResult[]> {
|
||||
const params = new URLSearchParams({ q: options.q });
|
||||
if (options.contentType) params.set("content_type", options.contentType);
|
||||
if (options.limit) params.set("limit", options.limit.toString());
|
||||
if (options.startTime) params.set("start_time", options.startTime);
|
||||
if (options.endTime) params.set("end_time", options.endTime);
|
||||
if (options.appName) params.set("app_name", options.appName);
|
||||
|
||||
const response = await this.fetchWithRetry(`${this.baseUrl}/search?${params}`);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Screenpipe search failed: ${response.statusText}`);
|
||||
}
|
||||
return (await response.json()) as SearchResult[];
|
||||
}
|
||||
|
||||
// ---- Utilities ----
|
||||
|
||||
/**
|
||||
* Determine the optimal query route for a given query string.
|
||||
*
|
||||
* The router analyzes the query intent and returns the best query mode
|
||||
* (semantic, keyword, graph, temporal, or hybrid).
|
||||
*
|
||||
* @param query - Natural language query to route
|
||||
* @returns The recommended query route
|
||||
* @throws {Error} If the route request fails
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const route = await client.routeQuery("who mentioned authentication yesterday?");
|
||||
* // route === "graph" or "temporal"
|
||||
* ```
|
||||
*/
|
||||
async routeQuery(query: string): Promise<QueryRoute> {
|
||||
const response = await this.fetchWithRetry(
|
||||
`${this.baseUrl}/${this.apiVersion}/route`,
|
||||
{
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ query }),
|
||||
},
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Route failed: ${response.statusText}`);
|
||||
}
|
||||
const result = (await response.json()) as { route: QueryRoute };
|
||||
return result.route;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieve pipeline statistics from the OsPipe server.
|
||||
*
|
||||
* @returns Pipeline statistics including ingestion counts, storage, and uptime
|
||||
* @throws {Error} If the stats request fails
|
||||
*/
|
||||
async stats(): Promise<PipelineStats> {
|
||||
const response = await this.fetchWithRetry(
|
||||
`${this.baseUrl}/${this.apiVersion}/stats`,
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Stats failed: ${response.statusText}`);
|
||||
}
|
||||
return (await response.json()) as PipelineStats;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check the health of the OsPipe server.
|
||||
*
|
||||
* @returns Health status including version and active backends
|
||||
* @throws {Error} If the health check fails
|
||||
*/
|
||||
async health(): Promise<{
|
||||
status: string;
|
||||
version: string;
|
||||
backends: string[];
|
||||
}> {
|
||||
const response = await this.fetchWithRetry(
|
||||
`${this.baseUrl}/${this.apiVersion}/health`,
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`Health check failed: ${response.statusText}`);
|
||||
}
|
||||
return (await response.json()) as { status: string; version: string; backends: string[] };
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Default Export ----
|
||||
|
||||
export default OsPipe;
|
||||
118
npm/packages/ospipe/src/wasm.d.ts
vendored
Normal file
118
npm/packages/ospipe/src/wasm.d.ts
vendored
Normal file
@@ -0,0 +1,118 @@
|
||||
/**
|
||||
* WASM bindings for OsPipe - use in browser-based pipes.
|
||||
*
|
||||
* This module provides a thin wrapper around the @ruvector/ospipe-wasm package,
|
||||
* exposing vector search, embedding, deduplication, and safety checking
|
||||
* capabilities that run entirely client-side via WebAssembly.
|
||||
*
|
||||
* @packageDocumentation
|
||||
*/
|
||||
/** A single search result from the WASM vector index. */
|
||||
export interface WasmSearchResult {
|
||||
/** Unique identifier for the indexed entry */
|
||||
id: string;
|
||||
/** Similarity score (higher is more similar) */
|
||||
score: number;
|
||||
/** JSON-encoded metadata string */
|
||||
metadata: string;
|
||||
}
|
||||
/** Configuration options for WASM initialization. */
|
||||
export interface OsPipeWasmOptions {
|
||||
/** Embedding vector dimension (default: 384) */
|
||||
dimension?: number;
|
||||
}
|
||||
/** The initialized WASM instance interface. */
|
||||
export interface OsPipeWasmInstance {
|
||||
/**
|
||||
* Insert a vector into the index.
|
||||
*
|
||||
* @param id - Unique identifier for the entry
|
||||
* @param embedding - Float32Array embedding vector
|
||||
* @param metadata - JSON-encoded metadata string
|
||||
* @param timestamp - Unix timestamp in milliseconds (default: Date.now())
|
||||
*/
|
||||
insert(id: string, embedding: Float32Array, metadata: string, timestamp?: number): void;
|
||||
/**
|
||||
* Search for the k nearest neighbors to the query embedding.
|
||||
*
|
||||
* @param queryEmbedding - Float32Array query vector
|
||||
* @param k - Number of results to return (default: 10)
|
||||
* @returns Array of search results ranked by similarity
|
||||
*/
|
||||
search(queryEmbedding: Float32Array, k?: number): WasmSearchResult[];
|
||||
/**
|
||||
* Search with a time range filter applied before ranking.
|
||||
*
|
||||
* @param queryEmbedding - Float32Array query vector
|
||||
* @param k - Number of results to return
|
||||
* @param startTime - Start of time range (Unix ms)
|
||||
* @param endTime - End of time range (Unix ms)
|
||||
* @returns Array of filtered search results
|
||||
*/
|
||||
searchFiltered(queryEmbedding: Float32Array, k: number, startTime: number, endTime: number): WasmSearchResult[];
|
||||
/**
|
||||
* Check if an embedding is a near-duplicate of an existing entry.
|
||||
*
|
||||
* @param embedding - Float32Array embedding to check
|
||||
* @param threshold - Similarity threshold 0-1 (default: 0.95)
|
||||
* @returns True if a duplicate is found above the threshold
|
||||
*/
|
||||
isDuplicate(embedding: Float32Array, threshold?: number): boolean;
|
||||
/**
|
||||
* Generate an embedding vector from text using the built-in ONNX model.
|
||||
*
|
||||
* @param text - Input text to embed
|
||||
* @returns Float32Array embedding vector
|
||||
*/
|
||||
embedText(text: string): Float32Array;
|
||||
/**
|
||||
* Run a safety check on content, returning the recommended action.
|
||||
*
|
||||
* @param content - Content string to check
|
||||
* @returns "allow", "redact", or "deny"
|
||||
*/
|
||||
safetyCheck(content: string): "allow" | "redact" | "deny";
|
||||
/**
|
||||
* Route a query to the optimal query type.
|
||||
*
|
||||
* @param query - Natural language query string
|
||||
* @returns Recommended query route type
|
||||
*/
|
||||
routeQuery(query: string): string;
|
||||
/** Number of entries currently in the index. */
|
||||
readonly size: number;
|
||||
/**
|
||||
* Get index statistics as a JSON string.
|
||||
*
|
||||
* @returns JSON-encoded statistics object
|
||||
*/
|
||||
stats(): string;
|
||||
}
|
||||
/**
|
||||
* Load and initialize the OsPipe WASM module.
|
||||
*
|
||||
* This function dynamically imports the @ruvector/ospipe-wasm package,
|
||||
* initializes the WebAssembly module, and returns a typed wrapper
|
||||
* around the raw WASM bindings.
|
||||
*
|
||||
* @param options - WASM initialization options
|
||||
* @returns Initialized WASM instance with typed methods
|
||||
* @throws {Error} If the WASM module fails to load or initialize
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { initOsPipeWasm } from "@ruvector/ospipe/wasm";
|
||||
*
|
||||
* const wasm = await initOsPipeWasm({ dimension: 384 });
|
||||
*
|
||||
* // Embed and insert
|
||||
* const embedding = wasm.embedText("hello world");
|
||||
* wasm.insert("doc-1", embedding, JSON.stringify({ app: "test" }));
|
||||
*
|
||||
* // Search
|
||||
* const query = wasm.embedText("greetings");
|
||||
* const results = wasm.search(query, 5);
|
||||
* ```
|
||||
*/
|
||||
export declare function initOsPipeWasm(options?: OsPipeWasmOptions): Promise<OsPipeWasmInstance>;
|
||||
//# sourceMappingURL=wasm.d.ts.map
|
||||
1
npm/packages/ospipe/src/wasm.d.ts.map
Normal file
1
npm/packages/ospipe/src/wasm.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"wasm.d.ts","sourceRoot":"","sources":["wasm.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,yDAAyD;AACzD,MAAM,WAAW,gBAAgB;IAC/B,8CAA8C;IAC9C,EAAE,EAAE,MAAM,CAAC;IACX,gDAAgD;IAChD,KAAK,EAAE,MAAM,CAAC;IACd,mCAAmC;IACnC,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,qDAAqD;AACrD,MAAM,WAAW,iBAAiB;IAChC,gDAAgD;IAChD,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED,+CAA+C;AAC/C,MAAM,WAAW,kBAAkB;IACjC;;;;;;;OAOG;IACH,MAAM,CAAC,EAAE,EAAE,MAAM,EAAE,SAAS,EAAE,YAAY,EAAE,QAAQ,EAAE,MAAM,EAAE,SAAS,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAExF;;;;;;OAMG;IACH,MAAM,CAAC,cAAc,EAAE,YAAY,EAAE,CAAC,CAAC,EAAE,MAAM,GAAG,gBAAgB,EAAE,CAAC;IAErE;;;;;;;;OAQG;IACH,cAAc,CACZ,cAAc,EAAE,YAAY,EAC5B,CAAC,EAAE,MAAM,EACT,SAAS,EAAE,MAAM,EACjB,OAAO,EAAE,MAAM,GACd,gBAAgB,EAAE,CAAC;IAEtB;;;;;;OAMG;IACH,WAAW,CAAC,SAAS,EAAE,YAAY,EAAE,SAAS,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC;IAElE;;;;;OAKG;IACH,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,YAAY,CAAC;IAEtC;;;;;OAKG;IACH,WAAW,CAAC,OAAO,EAAE,MAAM,GAAG,OAAO,GAAG,QAAQ,GAAG,MAAM,CAAC;IAE1D;;;;;OAKG;IACH,UAAU,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAAC;IAElC,gDAAgD;IAChD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAEtB;;;;OAIG;IACH,KAAK,IAAI,MAAM,CAAC;CACjB;AAED;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AACH,wBAAsB,cAAc,CAClC,OAAO,GAAE,iBAAsB,GAC9B,OAAO,CAAC,kBAAkB,CAAC,CAqE7B"}
|
||||
120
npm/packages/ospipe/src/wasm.js
Normal file
120
npm/packages/ospipe/src/wasm.js
Normal file
@@ -0,0 +1,120 @@
|
||||
"use strict";
|
||||
/**
|
||||
* WASM bindings for OsPipe - use in browser-based pipes.
|
||||
*
|
||||
* This module provides a thin wrapper around the @ruvector/ospipe-wasm package,
|
||||
* exposing vector search, embedding, deduplication, and safety checking
|
||||
* capabilities that run entirely client-side via WebAssembly.
|
||||
*
|
||||
* @packageDocumentation
|
||||
*/
|
||||
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
var desc = Object.getOwnPropertyDescriptor(m, k);
|
||||
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
||||
desc = { enumerable: true, get: function() { return m[k]; } };
|
||||
}
|
||||
Object.defineProperty(o, k2, desc);
|
||||
}) : (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
o[k2] = m[k];
|
||||
}));
|
||||
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
||||
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
||||
}) : function(o, v) {
|
||||
o["default"] = v;
|
||||
});
|
||||
var __importStar = (this && this.__importStar) || (function () {
|
||||
var ownKeys = function(o) {
|
||||
ownKeys = Object.getOwnPropertyNames || function (o) {
|
||||
var ar = [];
|
||||
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
||||
return ar;
|
||||
};
|
||||
return ownKeys(o);
|
||||
};
|
||||
return function (mod) {
|
||||
if (mod && mod.__esModule) return mod;
|
||||
var result = {};
|
||||
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
||||
__setModuleDefault(result, mod);
|
||||
return result;
|
||||
};
|
||||
})();
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.initOsPipeWasm = initOsPipeWasm;
|
||||
/**
|
||||
* Load and initialize the OsPipe WASM module.
|
||||
*
|
||||
* This function dynamically imports the @ruvector/ospipe-wasm package,
|
||||
* initializes the WebAssembly module, and returns a typed wrapper
|
||||
* around the raw WASM bindings.
|
||||
*
|
||||
* @param options - WASM initialization options
|
||||
* @returns Initialized WASM instance with typed methods
|
||||
* @throws {Error} If the WASM module fails to load or initialize
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { initOsPipeWasm } from "@ruvector/ospipe/wasm";
|
||||
*
|
||||
* const wasm = await initOsPipeWasm({ dimension: 384 });
|
||||
*
|
||||
* // Embed and insert
|
||||
* const embedding = wasm.embedText("hello world");
|
||||
* wasm.insert("doc-1", embedding, JSON.stringify({ app: "test" }));
|
||||
*
|
||||
* // Search
|
||||
* const query = wasm.embedText("greetings");
|
||||
* const results = wasm.search(query, 5);
|
||||
* ```
|
||||
*/
|
||||
async function initOsPipeWasm(options = {}) {
|
||||
const dimension = options.dimension ?? 384;
|
||||
// Dynamic import so the WASM package is not required at bundle time.
|
||||
// This allows the main @ruvector/ospipe package to work without WASM.
|
||||
// The @ruvector/ospipe-wasm package provides the compiled WASM bindings.
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
let wasm;
|
||||
try {
|
||||
// Use a variable to prevent TypeScript from resolving the module statically
|
||||
const wasmPkg = "@ruvector/ospipe-wasm";
|
||||
wasm = await Promise.resolve(`${wasmPkg}`).then(s => __importStar(require(s)));
|
||||
}
|
||||
catch {
|
||||
throw new Error("Failed to load @ruvector/ospipe-wasm. " +
|
||||
"Install it with: npm install @ruvector/ospipe-wasm");
|
||||
}
|
||||
await wasm.default();
|
||||
const instance = new wasm.OsPipeWasm(dimension);
|
||||
return {
|
||||
insert(id, embedding, metadata, timestamp) {
|
||||
instance.insert(id, embedding, metadata, timestamp ?? Date.now());
|
||||
},
|
||||
search(queryEmbedding, k = 10) {
|
||||
return instance.search(queryEmbedding, k);
|
||||
},
|
||||
searchFiltered(queryEmbedding, k, startTime, endTime) {
|
||||
return instance.search_filtered(queryEmbedding, k, startTime, endTime);
|
||||
},
|
||||
isDuplicate(embedding, threshold = 0.95) {
|
||||
return instance.is_duplicate(embedding, threshold);
|
||||
},
|
||||
embedText(text) {
|
||||
return new Float32Array(instance.embed_text(text));
|
||||
},
|
||||
safetyCheck(content) {
|
||||
return instance.safety_check(content);
|
||||
},
|
||||
routeQuery(query) {
|
||||
return instance.route_query(query);
|
||||
},
|
||||
get size() {
|
||||
return instance.len();
|
||||
},
|
||||
stats() {
|
||||
return instance.stats();
|
||||
},
|
||||
};
|
||||
}
|
||||
//# sourceMappingURL=wasm.js.map
|
||||
1
npm/packages/ospipe/src/wasm.js.map
Normal file
1
npm/packages/ospipe/src/wasm.js.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"wasm.js","sourceRoot":"","sources":["wasm.ts"],"names":[],"mappings":";AAAA;;;;;;;;GAQG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AA6HH,wCAuEC;AAjGD;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AACI,KAAK,UAAU,cAAc,CAClC,UAA6B,EAAE;IAE/B,MAAM,SAAS,GAAG,OAAO,CAAC,SAAS,IAAI,GAAG,CAAC;IAE3C,qEAAqE;IACrE,sEAAsE;IACtE,yEAAyE;IACzE,8DAA8D;IAC9D,IAAI,IAAS,CAAC;IACd,IAAI,CAAC;QACH,4EAA4E;QAC5E,MAAM,OAAO,GAAG,uBAAuB,CAAC;QACxC,IAAI,GAAG,yBAAuC,OAAO,uCAAC,CAAC;IACzD,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,KAAK,CACb,wCAAwC;YACtC,oDAAoD,CACvD,CAAC;IACJ,CAAC;IACD,MAAM,IAAI,CAAC,OAAO,EAAE,CAAC;IAErB,MAAM,QAAQ,GAAG,IAAI,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IAEhD,OAAO;QACL,MAAM,CACJ,EAAU,EACV,SAAuB,EACvB,QAAgB,EAChB,SAAkB;YAElB,QAAQ,CAAC,MAAM,CAAC,EAAE,EAAE,SAAS,EAAE,QAAQ,EAAE,SAAS,IAAI,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;QACpE,CAAC;QAED,MAAM,CAAC,cAA4B,EAAE,CAAC,GAAG,EAAE;YACzC,OAAO,QAAQ,CAAC,MAAM,CAAC,cAAc,EAAE,CAAC,CAAC,CAAC;QAC5C,CAAC;QAED,cAAc,CACZ,cAA4B,EAC5B,CAAS,EACT,SAAiB,EACjB,OAAe;YAEf,OAAO,QAAQ,CAAC,eAAe,CAAC,cAAc,EAAE,CAAC,EAAE,SAAS,EAAE,OAAO,CAAC,CAAC;QACzE,CAAC;QAED,WAAW,CAAC,SAAuB,EAAE,SAAS,GAAG,IAAI;YACnD,OAAO,QAAQ,CAAC,YAAY,CAAC,SAAS,EAAE,SAAS,CAAC,CAAC;QACrD,CAAC;QAED,SAAS,CAAC,IAAY;YACpB,OAAO,IAAI,YAAY,CAAC,QAAQ,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC,CAAC;QACrD,CAAC;QAED,WAAW,CAAC,OAAe;YACzB,OAAO,QAAQ,CAAC,YAAY,CAAC,OAAO,CAAgC,CAAC;QACvE,CAAC;QAED,UAAU,CAAC,KAAa;YACtB,OAAO,QAAQ,CAAC,WAAW,CAAC,KAAK,CAAC,CAAC;QACrC,CAAC;QAED,IAAI,IAAI;YACN,OAAO,QAAQ,CAAC,GAAG,EAAE,CAAC;QACxB,CAAC;QAED,KAAK;YACH,OAAO,QAAQ,CAAC,KAAK,EAAE,CAAC;QAC1B,CAAC;KACF,CAAC;AACJ,CAAC"}
|
||||
205
npm/packages/ospipe/src/wasm.ts
Normal file
205
npm/packages/ospipe/src/wasm.ts
Normal file
@@ -0,0 +1,205 @@
|
||||
/**
|
||||
* WASM bindings for OsPipe - use in browser-based pipes.
|
||||
*
|
||||
* This module provides a thin wrapper around the @ruvector/ospipe-wasm package,
|
||||
* exposing vector search, embedding, deduplication, and safety checking
|
||||
* capabilities that run entirely client-side via WebAssembly.
|
||||
*
|
||||
* @packageDocumentation
|
||||
*/
|
||||
|
||||
/** A single search result from the WASM vector index. */
|
||||
export interface WasmSearchResult {
|
||||
/** Unique identifier for the indexed entry */
|
||||
id: string;
|
||||
/** Similarity score (higher is more similar) */
|
||||
score: number;
|
||||
/** JSON-encoded metadata string */
|
||||
metadata: string;
|
||||
}
|
||||
|
||||
/** Configuration options for WASM initialization. */
|
||||
export interface OsPipeWasmOptions {
|
||||
/** Embedding vector dimension (default: 384) */
|
||||
dimension?: number;
|
||||
}
|
||||
|
||||
/** The initialized WASM instance interface. */
|
||||
export interface OsPipeWasmInstance {
|
||||
/**
|
||||
* Insert a vector into the index.
|
||||
*
|
||||
* @param id - Unique identifier for the entry
|
||||
* @param embedding - Float32Array embedding vector
|
||||
* @param metadata - JSON-encoded metadata string
|
||||
* @param timestamp - Unix timestamp in milliseconds (default: Date.now())
|
||||
*/
|
||||
insert(id: string, embedding: Float32Array, metadata: string, timestamp?: number): void;
|
||||
|
||||
/**
|
||||
* Search for the k nearest neighbors to the query embedding.
|
||||
*
|
||||
* @param queryEmbedding - Float32Array query vector
|
||||
* @param k - Number of results to return (default: 10)
|
||||
* @returns Array of search results ranked by similarity
|
||||
*/
|
||||
search(queryEmbedding: Float32Array, k?: number): WasmSearchResult[];
|
||||
|
||||
/**
|
||||
* Search with a time range filter applied before ranking.
|
||||
*
|
||||
* @param queryEmbedding - Float32Array query vector
|
||||
* @param k - Number of results to return
|
||||
* @param startTime - Start of time range (Unix ms)
|
||||
* @param endTime - End of time range (Unix ms)
|
||||
* @returns Array of filtered search results
|
||||
*/
|
||||
searchFiltered(
|
||||
queryEmbedding: Float32Array,
|
||||
k: number,
|
||||
startTime: number,
|
||||
endTime: number
|
||||
): WasmSearchResult[];
|
||||
|
||||
/**
|
||||
* Check if an embedding is a near-duplicate of an existing entry.
|
||||
*
|
||||
* @param embedding - Float32Array embedding to check
|
||||
* @param threshold - Similarity threshold 0-1 (default: 0.95)
|
||||
* @returns True if a duplicate is found above the threshold
|
||||
*/
|
||||
isDuplicate(embedding: Float32Array, threshold?: number): boolean;
|
||||
|
||||
/**
|
||||
* Generate an embedding vector from text using the built-in ONNX model.
|
||||
*
|
||||
* @param text - Input text to embed
|
||||
* @returns Float32Array embedding vector
|
||||
*/
|
||||
embedText(text: string): Float32Array;
|
||||
|
||||
/**
|
||||
* Run a safety check on content, returning the recommended action.
|
||||
*
|
||||
* @param content - Content string to check
|
||||
* @returns "allow", "redact", or "deny"
|
||||
*/
|
||||
safetyCheck(content: string): "allow" | "redact" | "deny";
|
||||
|
||||
/**
|
||||
* Route a query to the optimal query type.
|
||||
*
|
||||
* @param query - Natural language query string
|
||||
* @returns Recommended query route type
|
||||
*/
|
||||
routeQuery(query: string): string;
|
||||
|
||||
/** Number of entries currently in the index. */
|
||||
readonly size: number;
|
||||
|
||||
/**
|
||||
* Get index statistics as a JSON string.
|
||||
*
|
||||
* @returns JSON-encoded statistics object
|
||||
*/
|
||||
stats(): string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Load and initialize the OsPipe WASM module.
|
||||
*
|
||||
* This function dynamically imports the @ruvector/ospipe-wasm package,
|
||||
* initializes the WebAssembly module, and returns a typed wrapper
|
||||
* around the raw WASM bindings.
|
||||
*
|
||||
* @param options - WASM initialization options
|
||||
* @returns Initialized WASM instance with typed methods
|
||||
* @throws {Error} If the WASM module fails to load or initialize
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { initOsPipeWasm } from "@ruvector/ospipe/wasm";
|
||||
*
|
||||
* const wasm = await initOsPipeWasm({ dimension: 384 });
|
||||
*
|
||||
* // Embed and insert
|
||||
* const embedding = wasm.embedText("hello world");
|
||||
* wasm.insert("doc-1", embedding, JSON.stringify({ app: "test" }));
|
||||
*
|
||||
* // Search
|
||||
* const query = wasm.embedText("greetings");
|
||||
* const results = wasm.search(query, 5);
|
||||
* ```
|
||||
*/
|
||||
export async function initOsPipeWasm(
|
||||
options: OsPipeWasmOptions = {}
|
||||
): Promise<OsPipeWasmInstance> {
|
||||
const dimension = options.dimension ?? 384;
|
||||
|
||||
// Dynamic import so the WASM package is not required at bundle time.
|
||||
// This allows the main @ruvector/ospipe package to work without WASM.
|
||||
// The @ruvector/ospipe-wasm package provides the compiled WASM bindings.
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
let wasm: any;
|
||||
try {
|
||||
// Use a variable to prevent TypeScript from resolving the module statically
|
||||
const wasmPkg = "@ruvector/ospipe-wasm";
|
||||
wasm = await import(/* webpackIgnore: true */ wasmPkg);
|
||||
} catch {
|
||||
throw new Error(
|
||||
"Failed to load @ruvector/ospipe-wasm. " +
|
||||
"Install it with: npm install @ruvector/ospipe-wasm"
|
||||
);
|
||||
}
|
||||
await wasm.default();
|
||||
|
||||
const instance = new wasm.OsPipeWasm(dimension);
|
||||
|
||||
return {
|
||||
insert(
|
||||
id: string,
|
||||
embedding: Float32Array,
|
||||
metadata: string,
|
||||
timestamp?: number
|
||||
): void {
|
||||
instance.insert(id, embedding, metadata, timestamp ?? Date.now());
|
||||
},
|
||||
|
||||
search(queryEmbedding: Float32Array, k = 10): WasmSearchResult[] {
|
||||
return instance.search(queryEmbedding, k);
|
||||
},
|
||||
|
||||
searchFiltered(
|
||||
queryEmbedding: Float32Array,
|
||||
k: number,
|
||||
startTime: number,
|
||||
endTime: number
|
||||
): WasmSearchResult[] {
|
||||
return instance.search_filtered(queryEmbedding, k, startTime, endTime);
|
||||
},
|
||||
|
||||
isDuplicate(embedding: Float32Array, threshold = 0.95): boolean {
|
||||
return instance.is_duplicate(embedding, threshold);
|
||||
},
|
||||
|
||||
embedText(text: string): Float32Array {
|
||||
return new Float32Array(instance.embed_text(text));
|
||||
},
|
||||
|
||||
safetyCheck(content: string): "allow" | "redact" | "deny" {
|
||||
return instance.safety_check(content) as "allow" | "redact" | "deny";
|
||||
},
|
||||
|
||||
routeQuery(query: string): string {
|
||||
return instance.route_query(query);
|
||||
},
|
||||
|
||||
get size(): number {
|
||||
return instance.len();
|
||||
},
|
||||
|
||||
stats(): string {
|
||||
return instance.stats();
|
||||
},
|
||||
};
|
||||
}
|
||||
17
npm/packages/ospipe/tsconfig.json
Normal file
17
npm/packages/ospipe/tsconfig.json
Normal file
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true,
|
||||
"outDir": "dist",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"lib": ["ES2022", "DOM"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["node_modules", "dist"]
|
||||
}
|
||||
Reference in New Issue
Block a user