Squashed 'vendor/ruvector/' content from commit b64c2172
git-subtree-dir: vendor/ruvector git-subtree-split: b64c21726f2bb37286d9ee36a7869fef60cc6900
This commit is contained in:
12
npm/packages/ruvector/.npmignore
Normal file
12
npm/packages/ruvector/.npmignore
Normal file
@@ -0,0 +1,12 @@
|
||||
src/
|
||||
test/
|
||||
tsconfig.json
|
||||
*.log
|
||||
node_modules/
|
||||
.DS_Store
|
||||
*.tgz
|
||||
*.db
|
||||
.agentic-flow/
|
||||
.claude-flow/
|
||||
.ruvector/
|
||||
ruvector.db
|
||||
221
npm/packages/ruvector/HOOKS.md
Normal file
221
npm/packages/ruvector/HOOKS.md
Normal file
@@ -0,0 +1,221 @@
|
||||
# RuVector Hooks for Claude Code
|
||||
|
||||
Self-learning intelligence hooks that enhance Claude Code with Q-learning, vector memory, and automatic agent routing.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Full setup: hooks + pretrain + optimized agents
|
||||
npx ruvector hooks init --pretrain --build-agents quality
|
||||
|
||||
# Or step by step:
|
||||
npx ruvector hooks init # Setup hooks
|
||||
npx ruvector hooks pretrain # Analyze repository
|
||||
npx ruvector hooks build-agents # Generate agent configs
|
||||
```
|
||||
|
||||
## What It Does
|
||||
|
||||
RuVector hooks integrate with Claude Code to provide:
|
||||
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| **Agent Routing** | Suggests the best agent for each file type based on learned patterns |
|
||||
| **Co-edit Patterns** | Predicts "likely next files" from git history |
|
||||
| **Vector Memory** | Semantic recall of project context |
|
||||
| **Command Analysis** | Risk assessment for bash commands |
|
||||
| **Self-Learning** | Q-learning improves suggestions over time |
|
||||
|
||||
## Commands
|
||||
|
||||
### Initialization
|
||||
|
||||
```bash
|
||||
# Full configuration
|
||||
npx ruvector hooks init
|
||||
|
||||
# With pretrain and agent building
|
||||
npx ruvector hooks init --pretrain --build-agents security
|
||||
|
||||
# Minimal (basic hooks only)
|
||||
npx ruvector hooks init --minimal
|
||||
|
||||
# Options
|
||||
--force # Overwrite existing settings
|
||||
--minimal # Basic hooks only
|
||||
--pretrain # Run pretrain after init
|
||||
--build-agents # Generate optimized agents (quality|speed|security|testing|fullstack)
|
||||
--no-claude-md # Skip CLAUDE.md creation
|
||||
--no-permissions # Skip permissions config
|
||||
--no-env # Skip environment variables
|
||||
--no-gitignore # Skip .gitignore update
|
||||
--no-mcp # Skip MCP server config
|
||||
--no-statusline # Skip status line config
|
||||
```
|
||||
|
||||
### Pretrain
|
||||
|
||||
Analyze your repository to bootstrap intelligence:
|
||||
|
||||
```bash
|
||||
npx ruvector hooks pretrain
|
||||
|
||||
# Options
|
||||
--depth <n> # Git history depth (default: 100)
|
||||
--verbose # Show detailed progress
|
||||
--skip-git # Skip git history analysis
|
||||
--skip-files # Skip file structure analysis
|
||||
```
|
||||
|
||||
**What it learns:**
|
||||
- File type → Agent mapping (`.rs` → rust-developer)
|
||||
- Co-edit patterns from git history
|
||||
- Directory → Agent mapping
|
||||
- Project context memories
|
||||
|
||||
### Build Agents
|
||||
|
||||
Generate optimized `.claude/agents/` configurations:
|
||||
|
||||
```bash
|
||||
npx ruvector hooks build-agents --focus quality
|
||||
|
||||
# Focus modes
|
||||
--focus quality # Code quality, best practices (default)
|
||||
--focus speed # Rapid development, prototyping
|
||||
--focus security # OWASP, input validation, encryption
|
||||
--focus testing # TDD, comprehensive coverage
|
||||
--focus fullstack # Balanced frontend/backend/database
|
||||
|
||||
# Options
|
||||
--output <dir> # Output directory (default: .claude/agents)
|
||||
--format <fmt> # yaml, json, or md (default: yaml)
|
||||
--include-prompts # Include system prompts in agent configs
|
||||
```
|
||||
|
||||
### Verification & Diagnostics
|
||||
|
||||
```bash
|
||||
# Check if hooks are working
|
||||
npx ruvector hooks verify
|
||||
|
||||
# Diagnose and fix issues
|
||||
npx ruvector hooks doctor
|
||||
npx ruvector hooks doctor --fix
|
||||
```
|
||||
|
||||
### Data Management
|
||||
|
||||
```bash
|
||||
# View statistics
|
||||
npx ruvector hooks stats
|
||||
|
||||
# Export intelligence data
|
||||
npx ruvector hooks export -o backup.json
|
||||
npx ruvector hooks export --include-all
|
||||
|
||||
# Import intelligence data
|
||||
npx ruvector hooks import backup.json
|
||||
npx ruvector hooks import backup.json --merge
|
||||
```
|
||||
|
||||
### Memory Operations
|
||||
|
||||
```bash
|
||||
# Store context in vector memory
|
||||
npx ruvector hooks remember "API uses JWT auth" -t project
|
||||
|
||||
# Semantic search memory
|
||||
npx ruvector hooks recall "authentication"
|
||||
|
||||
# Route a task to best agent
|
||||
npx ruvector hooks route "implement user login"
|
||||
```
|
||||
|
||||
## Hook Events
|
||||
|
||||
| Event | Trigger | RuVector Action |
|
||||
|-------|---------|-----------------|
|
||||
| **PreToolUse** | Before Edit/Write/Bash | Agent routing, file analysis, command risk |
|
||||
| **PostToolUse** | After Edit/Write/Bash | Q-learning update, pattern recording |
|
||||
| **SessionStart** | Conversation begins | Load intelligence, display stats |
|
||||
| **Stop** | Conversation ends | Save learning data |
|
||||
| **UserPromptSubmit** | User sends message | Context suggestions |
|
||||
| **PreCompact** | Before context compaction | Preserve important context |
|
||||
| **Notification** | Any notification | Track events for learning |
|
||||
|
||||
## Generated Files
|
||||
|
||||
After running `hooks init`:
|
||||
|
||||
```
|
||||
your-project/
|
||||
├── .claude/
|
||||
│ ├── settings.json # Hooks configuration
|
||||
│ ├── statusline.sh # Status bar script
|
||||
│ └── agents/ # Generated agents (with --build-agents)
|
||||
│ ├── rust-specialist.yaml
|
||||
│ ├── typescript-specialist.yaml
|
||||
│ ├── test-architect.yaml
|
||||
│ └── project-coordinator.yaml
|
||||
├── .ruvector/
|
||||
│ └── intelligence.json # Learning data
|
||||
├── CLAUDE.md # Project documentation
|
||||
└── .gitignore # Updated with .ruvector/
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `RUVECTOR_INTELLIGENCE_ENABLED` | `true` | Enable/disable intelligence |
|
||||
| `RUVECTOR_LEARNING_RATE` | `0.1` | Q-learning rate (0.0-1.0) |
|
||||
| `RUVECTOR_MEMORY_BACKEND` | `rvlite` | Memory storage backend |
|
||||
| `INTELLIGENCE_MODE` | `treatment` | A/B testing mode |
|
||||
|
||||
## Example Output
|
||||
|
||||
### Agent Routing
|
||||
```
|
||||
🧠 Intelligence Analysis:
|
||||
📁 src/api/routes.ts
|
||||
🤖 Recommended: typescript-developer (85% confidence)
|
||||
→ learned from 127 .ts files in repo
|
||||
📎 Likely next files:
|
||||
- src/api/handlers.ts (12 co-edits)
|
||||
- src/types/api.ts (8 co-edits)
|
||||
```
|
||||
|
||||
### Command Analysis
|
||||
```
|
||||
🧠 Command Analysis:
|
||||
📦 Category: rust
|
||||
🏷️ Type: test
|
||||
✅ Risk: LOW
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Run pretrain on existing repos** — Bootstrap intelligence before starting work
|
||||
2. **Use focus modes** — Match agent generation to your current task
|
||||
3. **Export before major changes** — Backup learning data
|
||||
4. **Let it learn** — Intelligence improves with each edit
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
```bash
|
||||
# Check setup
|
||||
npx ruvector hooks verify
|
||||
|
||||
# Fix common issues
|
||||
npx ruvector hooks doctor --fix
|
||||
|
||||
# Reset and reinitialize
|
||||
npx ruvector hooks init --force --pretrain
|
||||
```
|
||||
|
||||
## Links
|
||||
|
||||
- [RuVector GitHub](https://github.com/ruvnet/ruvector)
|
||||
- [npm Package](https://www.npmjs.com/package/ruvector)
|
||||
- [Claude Code Documentation](https://docs.anthropic.com/claude-code)
|
||||
409
npm/packages/ruvector/PACKAGE_SUMMARY.md
Normal file
409
npm/packages/ruvector/PACKAGE_SUMMARY.md
Normal file
@@ -0,0 +1,409 @@
|
||||
# ruvector Package Summary
|
||||
|
||||
## Overview
|
||||
|
||||
The main `ruvector` package provides a unified interface for high-performance vector database operations in Node.js, with automatic platform detection and smart fallback between native (Rust) and WASM implementations.
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
/workspaces/ruvector/npm/packages/ruvector/
|
||||
├── src/ # TypeScript source
|
||||
│ ├── index.ts # Smart loader with platform detection
|
||||
│ └── types.ts # TypeScript type definitions
|
||||
├── dist/ # Compiled JavaScript and types
|
||||
│ ├── index.js # Main entry point
|
||||
│ ├── index.d.ts # Type definitions
|
||||
│ ├── types.js # Compiled types
|
||||
│ └── types.d.ts # Type definitions
|
||||
├── bin/
|
||||
│ └── cli.js # CLI tool
|
||||
├── test/
|
||||
│ ├── mock-implementation.js # Mock VectorDB for testing
|
||||
│ ├── standalone-test.js # Package structure tests
|
||||
│ └── integration.js # Integration tests
|
||||
├── examples/
|
||||
│ ├── api-usage.js # API usage examples
|
||||
│ └── cli-demo.sh # CLI demonstration
|
||||
├── package.json # NPM package configuration
|
||||
├── tsconfig.json # TypeScript configuration
|
||||
└── README.md # Package documentation
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
### 1. Smart Platform Detection
|
||||
|
||||
The package automatically detects and loads the best available implementation:
|
||||
|
||||
```typescript
|
||||
// Tries to load in this order:
|
||||
// 1. @ruvector/core (native Rust, fastest)
|
||||
// 2. @ruvector/wasm (WebAssembly, universal fallback)
|
||||
|
||||
import { VectorDB, getImplementationType, isNative, isWasm } from 'ruvector';
|
||||
|
||||
console.log(getImplementationType()); // 'native' or 'wasm'
|
||||
console.log(isNative()); // true if using native
|
||||
console.log(isWasm()); // true if using WASM
|
||||
```
|
||||
|
||||
### 2. Complete TypeScript Support
|
||||
|
||||
Full type definitions for all APIs:
|
||||
|
||||
```typescript
|
||||
interface VectorEntry {
|
||||
id: string;
|
||||
vector: number[];
|
||||
metadata?: Record<string, any>;
|
||||
}
|
||||
|
||||
interface SearchQuery {
|
||||
vector: number[];
|
||||
k?: number;
|
||||
filter?: Record<string, any>;
|
||||
threshold?: number;
|
||||
}
|
||||
|
||||
interface SearchResult {
|
||||
id: string;
|
||||
score: number;
|
||||
vector: number[];
|
||||
metadata?: Record<string, any>;
|
||||
}
|
||||
|
||||
interface DbOptions {
|
||||
dimension: number;
|
||||
metric?: 'cosine' | 'euclidean' | 'dot';
|
||||
path?: string;
|
||||
autoPersist?: boolean;
|
||||
hnsw?: {
|
||||
m?: number;
|
||||
efConstruction?: number;
|
||||
efSearch?: number;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### 3. VectorDB API
|
||||
|
||||
Comprehensive vector database operations:
|
||||
|
||||
```typescript
|
||||
const db = new VectorDB({
|
||||
dimension: 384,
|
||||
metric: 'cosine'
|
||||
});
|
||||
|
||||
// Insert operations
|
||||
db.insert({ id: 'doc1', vector: [...], metadata: {...} });
|
||||
db.insertBatch([...entries]);
|
||||
|
||||
// Search operations
|
||||
const results = db.search({
|
||||
vector: [...],
|
||||
k: 10,
|
||||
threshold: 0.7
|
||||
});
|
||||
|
||||
// CRUD operations
|
||||
const entry = db.get('doc1');
|
||||
db.updateMetadata('doc1', { updated: true });
|
||||
db.delete('doc1');
|
||||
|
||||
// Database management
|
||||
const stats = db.stats();
|
||||
db.save('./mydb.vec');
|
||||
db.load('./mydb.vec');
|
||||
db.buildIndex();
|
||||
db.optimize();
|
||||
```
|
||||
|
||||
### 4. CLI Tools
|
||||
|
||||
Command-line interface for database operations:
|
||||
|
||||
```bash
|
||||
# Create database
|
||||
ruvector create mydb.vec --dimension 384 --metric cosine
|
||||
|
||||
# Insert vectors
|
||||
ruvector insert mydb.vec vectors.json --batch-size 1000
|
||||
|
||||
# Search
|
||||
ruvector search mydb.vec --vector "[0.1,0.2,...]" --top-k 10
|
||||
|
||||
# Statistics
|
||||
ruvector stats mydb.vec
|
||||
|
||||
# Benchmark
|
||||
ruvector benchmark --num-vectors 10000 --num-queries 1000
|
||||
|
||||
# Info
|
||||
ruvector info
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### Constructor
|
||||
|
||||
```typescript
|
||||
new VectorDB(options: DbOptions): VectorDB
|
||||
```
|
||||
|
||||
### Methods
|
||||
|
||||
- `insert(entry: VectorEntry): void` - Insert single vector
|
||||
- `insertBatch(entries: VectorEntry[]): void` - Batch insert
|
||||
- `search(query: SearchQuery): SearchResult[]` - Search similar vectors
|
||||
- `get(id: string): VectorEntry | null` - Get by ID
|
||||
- `delete(id: string): boolean` - Delete vector
|
||||
- `updateMetadata(id: string, metadata: Record<string, any>): void` - Update metadata
|
||||
- `stats(): DbStats` - Get database statistics
|
||||
- `save(path?: string): void` - Save to disk
|
||||
- `load(path: string): void` - Load from disk
|
||||
- `clear(): void` - Clear all vectors
|
||||
- `buildIndex(): void` - Build HNSW index
|
||||
- `optimize(): void` - Optimize database
|
||||
|
||||
### Utility Functions
|
||||
|
||||
- `getImplementationType(): 'native' | 'wasm'` - Get current implementation
|
||||
- `isNative(): boolean` - Check if using native
|
||||
- `isWasm(): boolean` - Check if using WASM
|
||||
- `getVersion(): { version: string, implementation: string }` - Get version info
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Production Dependencies
|
||||
|
||||
- `commander` (^11.1.0) - CLI framework
|
||||
- `chalk` (^4.1.2) - Terminal styling
|
||||
- `ora` (^5.4.1) - Spinners and progress
|
||||
|
||||
### Optional Dependencies
|
||||
|
||||
- `@ruvector/core` (^0.1.1) - Native Rust bindings (when available)
|
||||
- `@ruvector/wasm` (^0.1.1) - WebAssembly module (fallback)
|
||||
|
||||
### Dev Dependencies
|
||||
|
||||
- `typescript` (^5.3.3) - TypeScript compiler
|
||||
- `@types/node` (^20.10.5) - Node.js type definitions
|
||||
|
||||
## Package.json Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "ruvector",
|
||||
"version": "0.1.1",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"bin": {
|
||||
"ruvector": "./bin/cli.js"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"test": "node test/standalone-test.js"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Build Process
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Build TypeScript
|
||||
npm run build
|
||||
|
||||
# Run tests
|
||||
npm test
|
||||
|
||||
# Package for NPM
|
||||
npm pack
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
The package includes comprehensive tests:
|
||||
|
||||
### 1. Standalone Test (`test/standalone-test.js`)
|
||||
|
||||
Tests package structure and API using mock implementation:
|
||||
- Package structure validation
|
||||
- TypeScript type definitions
|
||||
- VectorDB API functionality
|
||||
- CLI structure
|
||||
- Smart loader logic
|
||||
|
||||
### 2. Integration Test (`test/integration.js`)
|
||||
|
||||
Tests integration with real implementations when available.
|
||||
|
||||
### 3. Mock Implementation (`test/mock-implementation.js`)
|
||||
|
||||
JavaScript-based VectorDB implementation for testing and demonstration purposes.
|
||||
|
||||
## Examples
|
||||
|
||||
### API Usage (`examples/api-usage.js`)
|
||||
|
||||
Demonstrates:
|
||||
- Basic CRUD operations
|
||||
- Batch operations
|
||||
- Semantic search
|
||||
- Different distance metrics
|
||||
- Performance benchmarking
|
||||
- Persistence
|
||||
|
||||
### CLI Demo (`examples/cli-demo.sh`)
|
||||
|
||||
Bash script demonstrating CLI tools.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Simple Vector Search
|
||||
|
||||
```javascript
|
||||
const { VectorDB } = require('ruvector');
|
||||
|
||||
const db = new VectorDB({ dimension: 3 });
|
||||
|
||||
db.insertBatch([
|
||||
{ id: 'cat', vector: [0.9, 0.1, 0.1], metadata: { animal: 'cat' } },
|
||||
{ id: 'dog', vector: [0.1, 0.9, 0.1], metadata: { animal: 'dog' } },
|
||||
{ id: 'tiger', vector: [0.8, 0.2, 0.15], metadata: { animal: 'tiger' } }
|
||||
]);
|
||||
|
||||
const results = db.search({
|
||||
vector: [0.9, 0.1, 0.1],
|
||||
k: 2
|
||||
});
|
||||
|
||||
console.log(results);
|
||||
// [
|
||||
// { id: 'cat', score: 1.0, ... },
|
||||
// { id: 'tiger', score: 0.97, ... }
|
||||
// ]
|
||||
```
|
||||
|
||||
### Semantic Document Search
|
||||
|
||||
```javascript
|
||||
const db = new VectorDB({ dimension: 768, metric: 'cosine' });
|
||||
|
||||
// Insert documents with embeddings (from your embedding model)
|
||||
db.insertBatch([
|
||||
{ id: 'doc1', vector: embedding1, metadata: { title: 'AI Guide' } },
|
||||
{ id: 'doc2', vector: embedding2, metadata: { title: 'Web Dev' } }
|
||||
]);
|
||||
|
||||
// Search with query embedding
|
||||
const results = db.search({
|
||||
vector: queryEmbedding,
|
||||
k: 10,
|
||||
threshold: 0.7
|
||||
});
|
||||
```
|
||||
|
||||
### Persistence
|
||||
|
||||
```javascript
|
||||
const db = new VectorDB({
|
||||
dimension: 384,
|
||||
path: './vectors.db',
|
||||
autoPersist: true
|
||||
});
|
||||
|
||||
// Changes automatically saved
|
||||
db.insert({ id: 'doc1', vector: [...] });
|
||||
|
||||
// Or manual save
|
||||
db.save('./backup.db');
|
||||
|
||||
// Load from disk
|
||||
db.load('./vectors.db');
|
||||
```
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Mock Implementation (JavaScript)
|
||||
- Insert: ~1M vectors/sec (batch)
|
||||
- Search: ~400 queries/sec (1000 vectors, k=10)
|
||||
|
||||
### Native Implementation (Rust)
|
||||
- Insert: ~10M+ vectors/sec (batch)
|
||||
- Search: ~100K+ queries/sec with HNSW index
|
||||
- 150x faster than pgvector
|
||||
|
||||
### WASM Implementation
|
||||
- Insert: ~1M+ vectors/sec (batch)
|
||||
- Search: ~10K+ queries/sec with HNSW index
|
||||
- ~10x faster than pure JavaScript
|
||||
|
||||
## Integration with Other Packages
|
||||
|
||||
This package serves as the main interface and coordinates between:
|
||||
|
||||
1. **@ruvector/core** - Native Rust bindings (napi-rs)
|
||||
- Platform-specific native modules
|
||||
- Maximum performance
|
||||
- Optional dependency
|
||||
|
||||
2. **@ruvector/wasm** - WebAssembly module
|
||||
- Universal compatibility
|
||||
- Near-native performance
|
||||
- Fallback implementation
|
||||
|
||||
## Error Handling
|
||||
|
||||
The package provides clear error messages when implementations are unavailable:
|
||||
|
||||
```
|
||||
Failed to load ruvector: Neither native nor WASM implementation available.
|
||||
Native error: Cannot find module '@ruvector/core'
|
||||
WASM error: Cannot find module '@ruvector/wasm'
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `RUVECTOR_DEBUG=1` - Enable debug logging for implementation loading
|
||||
|
||||
## Next Steps
|
||||
|
||||
To complete the package ecosystem:
|
||||
|
||||
1. **Create @ruvector/core**
|
||||
- napi-rs bindings to Rust code
|
||||
- Platform-specific builds (Linux, macOS, Windows)
|
||||
- Native module packaging
|
||||
|
||||
2. **Create @ruvector/wasm**
|
||||
- wasm-pack build from Rust code
|
||||
- WebAssembly module
|
||||
- Universal compatibility layer
|
||||
|
||||
3. **Update Dependencies**
|
||||
- Add @ruvector/core as optionalDependency
|
||||
- Add @ruvector/wasm as dependency
|
||||
- Configure proper fallback chain
|
||||
|
||||
4. **Publishing**
|
||||
- Publish all three packages to npm
|
||||
- Set up CI/CD for builds
|
||||
- Create platform-specific releases
|
||||
|
||||
## Version
|
||||
|
||||
Current version: **0.1.1**
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
## Repository
|
||||
|
||||
https://github.com/ruvnet/ruvector
|
||||
2228
npm/packages/ruvector/README.md
Normal file
2228
npm/packages/ruvector/README.md
Normal file
File diff suppressed because it is too large
Load Diff
7356
npm/packages/ruvector/bin/cli.js
Executable file
7356
npm/packages/ruvector/bin/cli.js
Executable file
File diff suppressed because it is too large
Load Diff
3059
npm/packages/ruvector/bin/mcp-server.js
Normal file
3059
npm/packages/ruvector/bin/mcp-server.js
Normal file
File diff suppressed because it is too large
Load Diff
211
npm/packages/ruvector/examples/api-usage.js
Executable file
211
npm/packages/ruvector/examples/api-usage.js
Executable file
@@ -0,0 +1,211 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* ruvector API Usage Examples
|
||||
*
|
||||
* This demonstrates how to use ruvector in your Node.js applications
|
||||
*/
|
||||
|
||||
// For this demo, we use the mock implementation
|
||||
// In production, you would use: const { VectorDB } = require('ruvector');
|
||||
const { VectorDB } = require('../test/mock-implementation.js');
|
||||
|
||||
console.log('ruvector API Examples\n');
|
||||
console.log('='.repeat(60));
|
||||
|
||||
// Show info
|
||||
console.log('\nUsing: Mock implementation (for demo purposes)');
|
||||
console.log('In production: npm install ruvector\n');
|
||||
|
||||
// Example 1: Basic usage
|
||||
console.log('Example 1: Basic Vector Operations');
|
||||
console.log('-'.repeat(60));
|
||||
|
||||
const db = new VectorDB({
|
||||
dimension: 3,
|
||||
metric: 'cosine'
|
||||
});
|
||||
|
||||
// Insert some vectors
|
||||
db.insert({
|
||||
id: 'doc1',
|
||||
vector: [1, 0, 0],
|
||||
metadata: { title: 'First Document', category: 'A' }
|
||||
});
|
||||
|
||||
db.insertBatch([
|
||||
{ id: 'doc2', vector: [0, 1, 0], metadata: { title: 'Second Document', category: 'B' } },
|
||||
{ id: 'doc3', vector: [0, 0, 1], metadata: { title: 'Third Document', category: 'C' } },
|
||||
{ id: 'doc4', vector: [0.7, 0.7, 0], metadata: { title: 'Fourth Document', category: 'A' } }
|
||||
]);
|
||||
|
||||
console.log('✓ Inserted 4 vectors');
|
||||
|
||||
// Get stats
|
||||
const stats = db.stats();
|
||||
console.log(`✓ Database has ${stats.count} vectors, dimension ${stats.dimension}`);
|
||||
|
||||
// Search
|
||||
const results = db.search({
|
||||
vector: [1, 0, 0],
|
||||
k: 3
|
||||
});
|
||||
|
||||
console.log(`✓ Search returned ${results.length} results:`);
|
||||
results.forEach((result, i) => {
|
||||
console.log(` ${i + 1}. ${result.id} (score: ${result.score.toFixed(4)}) - ${result.metadata.title}`);
|
||||
});
|
||||
|
||||
// Get by ID
|
||||
const doc = db.get('doc2');
|
||||
console.log(`✓ Retrieved document: ${doc.metadata.title}`);
|
||||
|
||||
// Update metadata
|
||||
db.updateMetadata('doc1', { updated: true, timestamp: Date.now() });
|
||||
console.log('✓ Updated metadata');
|
||||
|
||||
// Delete
|
||||
db.delete('doc3');
|
||||
console.log('✓ Deleted doc3');
|
||||
console.log(`✓ Database now has ${db.stats().count} vectors\n`);
|
||||
|
||||
// Example 2: Semantic Search Simulation
|
||||
console.log('Example 2: Semantic Search Simulation');
|
||||
console.log('-'.repeat(60));
|
||||
|
||||
const semanticDb = new VectorDB({
|
||||
dimension: 5,
|
||||
metric: 'cosine'
|
||||
});
|
||||
|
||||
// Simulate document embeddings
|
||||
const documents = [
|
||||
{ id: 'machine-learning', vector: [0.9, 0.8, 0.1, 0.2, 0.1], metadata: { title: 'Introduction to Machine Learning', topic: 'AI' } },
|
||||
{ id: 'deep-learning', vector: [0.85, 0.9, 0.15, 0.25, 0.1], metadata: { title: 'Deep Learning Fundamentals', topic: 'AI' } },
|
||||
{ id: 'web-dev', vector: [0.1, 0.2, 0.9, 0.8, 0.1], metadata: { title: 'Web Development Guide', topic: 'Web' } },
|
||||
{ id: 'react', vector: [0.15, 0.2, 0.85, 0.9, 0.1], metadata: { title: 'React Tutorial', topic: 'Web' } },
|
||||
{ id: 'database', vector: [0.2, 0.3, 0.3, 0.4, 0.9], metadata: { title: 'Database Design', topic: 'Data' } }
|
||||
];
|
||||
|
||||
semanticDb.insertBatch(documents);
|
||||
console.log(`✓ Indexed ${documents.length} documents`);
|
||||
|
||||
// Search for AI-related content
|
||||
const aiQuery = [0.9, 0.85, 0.1, 0.2, 0.1];
|
||||
const aiResults = semanticDb.search({ vector: aiQuery, k: 2 });
|
||||
|
||||
console.log('\nQuery: AI-related content');
|
||||
console.log('Results:');
|
||||
aiResults.forEach((result, i) => {
|
||||
console.log(` ${i + 1}. ${result.metadata.title} (score: ${result.score.toFixed(4)})`);
|
||||
});
|
||||
|
||||
// Search for Web-related content
|
||||
const webQuery = [0.1, 0.2, 0.9, 0.85, 0.1];
|
||||
const webResults = semanticDb.search({ vector: webQuery, k: 2 });
|
||||
|
||||
console.log('\nQuery: Web-related content');
|
||||
console.log('Results:');
|
||||
webResults.forEach((result, i) => {
|
||||
console.log(` ${i + 1}. ${result.metadata.title} (score: ${result.score.toFixed(4)})`);
|
||||
});
|
||||
|
||||
// Example 3: Different Distance Metrics
|
||||
console.log('\n\nExample 3: Distance Metrics Comparison');
|
||||
console.log('-'.repeat(60));
|
||||
|
||||
const metrics = ['cosine', 'euclidean', 'dot'];
|
||||
const testVectors = [
|
||||
{ id: 'v1', vector: [1, 0, 0] },
|
||||
{ id: 'v2', vector: [0.7, 0.7, 0] },
|
||||
{ id: 'v3', vector: [0, 1, 0] }
|
||||
];
|
||||
|
||||
metrics.forEach(metric => {
|
||||
const metricDb = new VectorDB({ dimension: 3, metric });
|
||||
metricDb.insertBatch(testVectors);
|
||||
|
||||
const results = metricDb.search({ vector: [1, 0, 0], k: 3 });
|
||||
|
||||
console.log(`\n${metric.toUpperCase()} metric:`);
|
||||
results.forEach((result, i) => {
|
||||
console.log(` ${i + 1}. ${result.id}: ${result.score.toFixed(4)}`);
|
||||
});
|
||||
});
|
||||
|
||||
// Example 4: Batch Operations Performance
|
||||
console.log('\n\nExample 4: Batch Operations Performance');
|
||||
console.log('-'.repeat(60));
|
||||
|
||||
const perfDb = new VectorDB({ dimension: 128, metric: 'cosine' });
|
||||
|
||||
// Generate random vectors
|
||||
const numVectors = 1000;
|
||||
const vectors = [];
|
||||
for (let i = 0; i < numVectors; i++) {
|
||||
vectors.push({
|
||||
id: `vec_${i}`,
|
||||
vector: Array.from({ length: 128 }, () => Math.random()),
|
||||
metadata: { index: i, batch: Math.floor(i / 100) }
|
||||
});
|
||||
}
|
||||
|
||||
console.log(`Inserting ${numVectors} vectors...`);
|
||||
const insertStart = Date.now();
|
||||
perfDb.insertBatch(vectors);
|
||||
const insertTime = Date.now() - insertStart;
|
||||
|
||||
console.log(`✓ Inserted ${numVectors} vectors in ${insertTime}ms`);
|
||||
console.log(`✓ Rate: ${Math.round(numVectors / (insertTime / 1000))} vectors/sec`);
|
||||
|
||||
// Search performance
|
||||
const numQueries = 100;
|
||||
console.log(`\nRunning ${numQueries} searches...`);
|
||||
const searchStart = Date.now();
|
||||
|
||||
for (let i = 0; i < numQueries; i++) {
|
||||
const query = {
|
||||
vector: Array.from({ length: 128 }, () => Math.random()),
|
||||
k: 10
|
||||
};
|
||||
perfDb.search(query);
|
||||
}
|
||||
|
||||
const searchTime = Date.now() - searchStart;
|
||||
console.log(`✓ Completed ${numQueries} searches in ${searchTime}ms`);
|
||||
console.log(`✓ Rate: ${Math.round(numQueries / (searchTime / 1000))} queries/sec`);
|
||||
console.log(`✓ Avg latency: ${(searchTime / numQueries).toFixed(2)}ms`);
|
||||
|
||||
// Example 5: Persistence (conceptual, would need real implementation)
|
||||
console.log('\n\nExample 5: Persistence');
|
||||
console.log('-'.repeat(60));
|
||||
|
||||
const persistDb = new VectorDB({
|
||||
dimension: 3,
|
||||
metric: 'cosine',
|
||||
path: './my-vectors.db',
|
||||
autoPersist: true
|
||||
});
|
||||
|
||||
persistDb.insertBatch([
|
||||
{ id: 'p1', vector: [1, 0, 0], metadata: { name: 'First' } },
|
||||
{ id: 'p2', vector: [0, 1, 0], metadata: { name: 'Second' } }
|
||||
]);
|
||||
|
||||
console.log('✓ Created database with auto-persist enabled');
|
||||
console.log('✓ Insert operations will automatically save to disk');
|
||||
console.log('✓ Use db.save(path) for manual saves');
|
||||
console.log('✓ Use db.load(path) to restore from disk');
|
||||
|
||||
// Summary
|
||||
console.log('\n' + '='.repeat(60));
|
||||
console.log('\n✅ All examples completed successfully!');
|
||||
console.log('\nKey Features Demonstrated:');
|
||||
console.log(' • Basic CRUD operations (insert, search, get, update, delete)');
|
||||
console.log(' • Batch operations for better performance');
|
||||
console.log(' • Multiple distance metrics (cosine, euclidean, dot)');
|
||||
console.log(' • Semantic search simulation');
|
||||
console.log(' • Performance benchmarking');
|
||||
console.log(' • Metadata filtering and updates');
|
||||
console.log(' • Persistence (save/load)');
|
||||
console.log('\nFor more examples, see: /workspaces/ruvector/npm/packages/ruvector/examples/');
|
||||
85
npm/packages/ruvector/examples/cli-demo.sh
Executable file
85
npm/packages/ruvector/examples/cli-demo.sh
Executable file
@@ -0,0 +1,85 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ruvector CLI Demo
|
||||
# This demonstrates the CLI functionality with a simple example
|
||||
|
||||
echo "🚀 ruvector CLI Demo"
|
||||
echo "===================="
|
||||
echo ""
|
||||
|
||||
# 1. Show version info
|
||||
echo "1. Checking ruvector info..."
|
||||
ruvector info
|
||||
echo ""
|
||||
|
||||
# 2. Create a database
|
||||
echo "2. Creating a new database..."
|
||||
ruvector create demo.vec --dimension 3 --metric cosine
|
||||
echo ""
|
||||
|
||||
# 3. Create sample data
|
||||
echo "3. Creating sample vectors..."
|
||||
cat > demo-vectors.json << 'EOF'
|
||||
[
|
||||
{
|
||||
"id": "cat",
|
||||
"vector": [0.9, 0.1, 0.1],
|
||||
"metadata": {"animal": "cat", "category": "feline"}
|
||||
},
|
||||
{
|
||||
"id": "dog",
|
||||
"vector": [0.1, 0.9, 0.1],
|
||||
"metadata": {"animal": "dog", "category": "canine"}
|
||||
},
|
||||
{
|
||||
"id": "tiger",
|
||||
"vector": [0.8, 0.2, 0.15],
|
||||
"metadata": {"animal": "tiger", "category": "feline"}
|
||||
},
|
||||
{
|
||||
"id": "wolf",
|
||||
"vector": [0.2, 0.8, 0.15],
|
||||
"metadata": {"animal": "wolf", "category": "canine"}
|
||||
},
|
||||
{
|
||||
"id": "lion",
|
||||
"vector": [0.85, 0.15, 0.1],
|
||||
"metadata": {"animal": "lion", "category": "feline"}
|
||||
}
|
||||
]
|
||||
EOF
|
||||
echo " Created demo-vectors.json with 5 animals"
|
||||
echo ""
|
||||
|
||||
# 4. Insert vectors
|
||||
echo "4. Inserting vectors into database..."
|
||||
ruvector insert demo.vec demo-vectors.json
|
||||
echo ""
|
||||
|
||||
# 5. Show statistics
|
||||
echo "5. Database statistics..."
|
||||
ruvector stats demo.vec
|
||||
echo ""
|
||||
|
||||
# 6. Search for cat-like animals
|
||||
echo "6. Searching for cat-like animals (vector: [0.9, 0.1, 0.1])..."
|
||||
ruvector search demo.vec --vector "[0.9, 0.1, 0.1]" --top-k 3
|
||||
echo ""
|
||||
|
||||
# 7. Search for dog-like animals
|
||||
echo "7. Searching for dog-like animals (vector: [0.1, 0.9, 0.1])..."
|
||||
ruvector search demo.vec --vector "[0.1, 0.9, 0.1]" --top-k 3
|
||||
echo ""
|
||||
|
||||
# 8. Run benchmark
|
||||
echo "8. Running performance benchmark..."
|
||||
ruvector benchmark --dimension 128 --num-vectors 1000 --num-queries 100
|
||||
echo ""
|
||||
|
||||
# Cleanup
|
||||
echo "9. Cleanup (removing demo files)..."
|
||||
rm -f demo.vec demo-vectors.json
|
||||
echo " ✓ Demo files removed"
|
||||
echo ""
|
||||
|
||||
echo "✅ Demo complete!"
|
||||
77
npm/packages/ruvector/package.json
Normal file
77
npm/packages/ruvector/package.json
Normal file
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"name": "ruvector",
|
||||
"version": "0.1.100",
|
||||
"description": "High-performance vector database for Node.js with automatic native/WASM fallback",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"bin": {
|
||||
"ruvector": "./bin/cli.js"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc && cp src/core/onnx/pkg/package.json dist/core/onnx/pkg/",
|
||||
"prepublishOnly": "npm run build",
|
||||
"test": "node test/integration.js"
|
||||
},
|
||||
"keywords": [
|
||||
"vector",
|
||||
"database",
|
||||
"vector-database",
|
||||
"vector-search",
|
||||
"similarity-search",
|
||||
"semantic-search",
|
||||
"embeddings",
|
||||
"hnsw",
|
||||
"ann",
|
||||
"ai",
|
||||
"machine-learning",
|
||||
"rag",
|
||||
"rust",
|
||||
"wasm",
|
||||
"native",
|
||||
"ruv",
|
||||
"ruvector",
|
||||
"attention",
|
||||
"transformer",
|
||||
"flash-attention",
|
||||
"hyperbolic",
|
||||
"sona",
|
||||
"lora",
|
||||
"ewc",
|
||||
"adaptive-learning",
|
||||
"continual-learning",
|
||||
"onnx",
|
||||
"semantic-embeddings",
|
||||
"minilm"
|
||||
],
|
||||
"author": "ruv.io Team <info@ruv.io> (https://ruv.io)",
|
||||
"homepage": "https://ruv.io",
|
||||
"bugs": {
|
||||
"url": "https://github.com/ruvnet/ruvector/issues"
|
||||
},
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "https://github.com/ruvnet/ruvector.git",
|
||||
"directory": "npm/packages/ruvector"
|
||||
},
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.0.0",
|
||||
"@ruvector/attention": "^0.1.3",
|
||||
"@ruvector/core": "^0.1.25",
|
||||
"@ruvector/gnn": "^0.1.22",
|
||||
"@ruvector/sona": "^0.1.4",
|
||||
"chalk": "^4.1.2",
|
||||
"commander": "^11.1.0",
|
||||
"ora": "^5.4.1"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@ruvector/rvf": "^0.1.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.10.5",
|
||||
"typescript": "^5.3.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=14.0.0"
|
||||
}
|
||||
}
|
||||
BIN
npm/packages/ruvector/ruvector-0.1.1.tgz
Normal file
BIN
npm/packages/ruvector/ruvector-0.1.1.tgz
Normal file
Binary file not shown.
52
npm/packages/ruvector/src/analysis/complexity.d.ts
vendored
Normal file
52
npm/packages/ruvector/src/analysis/complexity.d.ts
vendored
Normal file
@@ -0,0 +1,52 @@
|
||||
/**
|
||||
* Complexity Analysis Module - Consolidated code complexity metrics
|
||||
*
|
||||
* Single source of truth for cyclomatic complexity and code metrics.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
export interface ComplexityResult {
|
||||
file: string;
|
||||
lines: number;
|
||||
nonEmptyLines: number;
|
||||
cyclomaticComplexity: number;
|
||||
functions: number;
|
||||
avgFunctionSize: number;
|
||||
maxFunctionComplexity?: number;
|
||||
}
|
||||
export interface ComplexityThresholds {
|
||||
complexity: number;
|
||||
functions: number;
|
||||
lines: number;
|
||||
avgSize: number;
|
||||
}
|
||||
export declare const DEFAULT_THRESHOLDS: ComplexityThresholds;
|
||||
/**
|
||||
* Analyze complexity of a single file
|
||||
*/
|
||||
export declare function analyzeFile(filePath: string, content?: string): ComplexityResult;
|
||||
/**
|
||||
* Analyze complexity of multiple files
|
||||
*/
|
||||
export declare function analyzeFiles(files: string[], maxFiles?: number): ComplexityResult[];
|
||||
/**
|
||||
* Check if complexity exceeds thresholds
|
||||
*/
|
||||
export declare function exceedsThresholds(result: ComplexityResult, thresholds?: ComplexityThresholds): boolean;
|
||||
/**
|
||||
* Get complexity rating
|
||||
*/
|
||||
export declare function getComplexityRating(complexity: number): 'low' | 'medium' | 'high' | 'critical';
|
||||
/**
|
||||
* Filter files exceeding thresholds
|
||||
*/
|
||||
export declare function filterComplex(results: ComplexityResult[], thresholds?: ComplexityThresholds): ComplexityResult[];
|
||||
declare const _default: {
|
||||
DEFAULT_THRESHOLDS: ComplexityThresholds;
|
||||
analyzeFile: typeof analyzeFile;
|
||||
analyzeFiles: typeof analyzeFiles;
|
||||
exceedsThresholds: typeof exceedsThresholds;
|
||||
getComplexityRating: typeof getComplexityRating;
|
||||
filterComplex: typeof filterComplex;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=complexity.d.ts.map
|
||||
1
npm/packages/ruvector/src/analysis/complexity.d.ts.map
Normal file
1
npm/packages/ruvector/src/analysis/complexity.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"complexity.d.ts","sourceRoot":"","sources":["complexity.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAIH,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;IACd,aAAa,EAAE,MAAM,CAAC;IACtB,oBAAoB,EAAE,MAAM,CAAC;IAC7B,SAAS,EAAE,MAAM,CAAC;IAClB,eAAe,EAAE,MAAM,CAAC;IACxB,qBAAqB,CAAC,EAAE,MAAM,CAAC;CAChC;AAED,MAAM,WAAW,oBAAoB;IACnC,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,OAAO,EAAE,MAAM,CAAC;CACjB;AAED,eAAO,MAAM,kBAAkB,EAAE,oBAKhC,CAAC;AAEF;;GAEG;AACH,wBAAgB,WAAW,CAAC,QAAQ,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,MAAM,GAAG,gBAAgB,CAsDhF;AAED;;GAEG;AACH,wBAAgB,YAAY,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,QAAQ,GAAE,MAAY,GAAG,gBAAgB,EAAE,CAExF;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAC/B,MAAM,EAAE,gBAAgB,EACxB,UAAU,GAAE,oBAAyC,GACpD,OAAO,CAOT;AAED;;GAEG;AACH,wBAAgB,mBAAmB,CAAC,UAAU,EAAE,MAAM,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,UAAU,CAK9F;AAED;;GAEG;AACH,wBAAgB,aAAa,CAC3B,OAAO,EAAE,gBAAgB,EAAE,EAC3B,UAAU,GAAE,oBAAyC,GACpD,gBAAgB,EAAE,CAEpB;;;;;;;;;AAED,wBAOE"}
|
||||
147
npm/packages/ruvector/src/analysis/complexity.js
Normal file
147
npm/packages/ruvector/src/analysis/complexity.js
Normal file
@@ -0,0 +1,147 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Complexity Analysis Module - Consolidated code complexity metrics
|
||||
*
|
||||
* Single source of truth for cyclomatic complexity and code metrics.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
var desc = Object.getOwnPropertyDescriptor(m, k);
|
||||
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
||||
desc = { enumerable: true, get: function() { return m[k]; } };
|
||||
}
|
||||
Object.defineProperty(o, k2, desc);
|
||||
}) : (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
o[k2] = m[k];
|
||||
}));
|
||||
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
||||
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
||||
}) : function(o, v) {
|
||||
o["default"] = v;
|
||||
});
|
||||
var __importStar = (this && this.__importStar) || (function () {
|
||||
var ownKeys = function(o) {
|
||||
ownKeys = Object.getOwnPropertyNames || function (o) {
|
||||
var ar = [];
|
||||
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
||||
return ar;
|
||||
};
|
||||
return ownKeys(o);
|
||||
};
|
||||
return function (mod) {
|
||||
if (mod && mod.__esModule) return mod;
|
||||
var result = {};
|
||||
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
||||
__setModuleDefault(result, mod);
|
||||
return result;
|
||||
};
|
||||
})();
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.DEFAULT_THRESHOLDS = void 0;
|
||||
exports.analyzeFile = analyzeFile;
|
||||
exports.analyzeFiles = analyzeFiles;
|
||||
exports.exceedsThresholds = exceedsThresholds;
|
||||
exports.getComplexityRating = getComplexityRating;
|
||||
exports.filterComplex = filterComplex;
|
||||
const fs = __importStar(require("fs"));
|
||||
exports.DEFAULT_THRESHOLDS = {
|
||||
complexity: 10,
|
||||
functions: 30,
|
||||
lines: 500,
|
||||
avgSize: 50,
|
||||
};
|
||||
/**
|
||||
* Analyze complexity of a single file
|
||||
*/
|
||||
function analyzeFile(filePath, content) {
|
||||
try {
|
||||
const fileContent = content ?? (fs.existsSync(filePath) ? fs.readFileSync(filePath, 'utf-8') : '');
|
||||
if (!fileContent) {
|
||||
return { file: filePath, lines: 0, nonEmptyLines: 0, cyclomaticComplexity: 1, functions: 0, avgFunctionSize: 0 };
|
||||
}
|
||||
const lines = fileContent.split('\n');
|
||||
const nonEmptyLines = lines.filter(l => l.trim().length > 0).length;
|
||||
// Count branching statements for cyclomatic complexity
|
||||
const branches = (fileContent.match(/\bif\b/g)?.length || 0) +
|
||||
(fileContent.match(/\belse\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bfor\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bwhile\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bswitch\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bcase\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bcatch\b/g)?.length || 0) +
|
||||
(fileContent.match(/\?\?/g)?.length || 0) +
|
||||
(fileContent.match(/&&/g)?.length || 0) +
|
||||
(fileContent.match(/\|\|/g)?.length || 0) +
|
||||
(fileContent.match(/\?[^:]/g)?.length || 0); // Ternary
|
||||
const cyclomaticComplexity = branches + 1;
|
||||
// Count functions
|
||||
const functionPatterns = [
|
||||
/function\s+\w+/g,
|
||||
/\w+\s*=\s*(?:async\s*)?\(/g,
|
||||
/\w+\s*:\s*(?:async\s*)?\(/g,
|
||||
/(?:async\s+)?(?:public|private|protected)?\s+\w+\s*\([^)]*\)\s*[:{]/g,
|
||||
];
|
||||
let functions = 0;
|
||||
for (const pattern of functionPatterns) {
|
||||
functions += (fileContent.match(pattern) || []).length;
|
||||
}
|
||||
// Deduplicate by rough estimate
|
||||
functions = Math.ceil(functions / 2);
|
||||
const avgFunctionSize = functions > 0 ? Math.round(nonEmptyLines / functions) : nonEmptyLines;
|
||||
return {
|
||||
file: filePath,
|
||||
lines: lines.length,
|
||||
nonEmptyLines,
|
||||
cyclomaticComplexity,
|
||||
functions,
|
||||
avgFunctionSize,
|
||||
};
|
||||
}
|
||||
catch {
|
||||
return { file: filePath, lines: 0, nonEmptyLines: 0, cyclomaticComplexity: 1, functions: 0, avgFunctionSize: 0 };
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Analyze complexity of multiple files
|
||||
*/
|
||||
function analyzeFiles(files, maxFiles = 100) {
|
||||
return files.slice(0, maxFiles).map(f => analyzeFile(f));
|
||||
}
|
||||
/**
|
||||
* Check if complexity exceeds thresholds
|
||||
*/
|
||||
function exceedsThresholds(result, thresholds = exports.DEFAULT_THRESHOLDS) {
|
||||
return (result.cyclomaticComplexity > thresholds.complexity ||
|
||||
result.functions > thresholds.functions ||
|
||||
result.lines > thresholds.lines ||
|
||||
result.avgFunctionSize > thresholds.avgSize);
|
||||
}
|
||||
/**
|
||||
* Get complexity rating
|
||||
*/
|
||||
function getComplexityRating(complexity) {
|
||||
if (complexity <= 5)
|
||||
return 'low';
|
||||
if (complexity <= 10)
|
||||
return 'medium';
|
||||
if (complexity <= 20)
|
||||
return 'high';
|
||||
return 'critical';
|
||||
}
|
||||
/**
|
||||
* Filter files exceeding thresholds
|
||||
*/
|
||||
function filterComplex(results, thresholds = exports.DEFAULT_THRESHOLDS) {
|
||||
return results.filter(r => exceedsThresholds(r, thresholds));
|
||||
}
|
||||
exports.default = {
|
||||
DEFAULT_THRESHOLDS: exports.DEFAULT_THRESHOLDS,
|
||||
analyzeFile,
|
||||
analyzeFiles,
|
||||
exceedsThresholds,
|
||||
getComplexityRating,
|
||||
filterComplex,
|
||||
};
|
||||
//# sourceMappingURL=complexity.js.map
|
||||
1
npm/packages/ruvector/src/analysis/complexity.js.map
Normal file
1
npm/packages/ruvector/src/analysis/complexity.js.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"complexity.js","sourceRoot":"","sources":["complexity.ts"],"names":[],"mappings":";AAAA;;;;;GAKG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AA+BH,kCAsDC;AAKD,oCAEC;AAKD,8CAUC;AAKD,kDAKC;AAKD,sCAKC;AA7HD,uCAAyB;AAmBZ,QAAA,kBAAkB,GAAyB;IACtD,UAAU,EAAE,EAAE;IACd,SAAS,EAAE,EAAE;IACb,KAAK,EAAE,GAAG;IACV,OAAO,EAAE,EAAE;CACZ,CAAC;AAEF;;GAEG;AACH,SAAgB,WAAW,CAAC,QAAgB,EAAE,OAAgB;IAC5D,IAAI,CAAC;QACH,MAAM,WAAW,GAAG,OAAO,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,YAAY,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QACnG,IAAI,CAAC,WAAW,EAAE,CAAC;YACjB,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,KAAK,EAAE,CAAC,EAAE,aAAa,EAAE,CAAC,EAAE,oBAAoB,EAAE,CAAC,EAAE,SAAS,EAAE,CAAC,EAAE,eAAe,EAAE,CAAC,EAAE,CAAC;QACnH,CAAC;QAED,MAAM,KAAK,GAAG,WAAW,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtC,MAAM,aAAa,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,MAAM,CAAC;QAEpE,uDAAuD;QACvD,MAAM,QAAQ,GACZ,CAAC,WAAW,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YAC3C,CAAC,WAAW,CAAC,KAAK,CAAC,WAAW,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YAC7C,CAAC,WAAW,CAAC,KAAK,CAAC,UAAU,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YAC5C,CAAC,WAAW,CAAC,KAAK,CAAC,YAAY,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YAC9C,CAAC,WAAW,CAAC,KAAK,CAAC,aAAa,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YAC/C,CAAC,WAAW,CAAC,KAAK,CAAC,WAAW,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YAC7C,CAAC,WAAW,CAAC,KAAK,CAAC,YAAY,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YAC9C,CAAC,WAAW,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YACzC,CAAC,WAAW,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YACvC,CAAC,WAAW,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC;YACzC,CAAC,WAAW,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,MAAM,IAAI,CAAC,CAAC,CAAC,CAAC,UAAU;QAEzD,MAAM,oBAAoB,GAAG,QAAQ,GAAG,CAAC,CAAC;QAE1C,kBAAkB;QAClB,MAAM,gBAAgB,GAAG;YACvB,iBAAiB;YACjB,4BAA4B;YAC5B,4BAA4B;YAC5B,sEAAsE;SACvE,CAAC;QAEF,IAAI,SAAS,GAAG,CAAC,CAAC;QAClB,KAAK,MAAM,OAAO,IAAI,gBAAgB,EAAE,CAAC;YACvC,SAAS,IAAI,CAAC,WAAW,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC;QACzD,CAAC;QACD,gCAAgC;QAChC,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,GAAG,CAAC,CAAC,CAAC;QAErC,MAAM,eAAe,GAAG,SAAS,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,aAAa,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC;QAE9F,OAAO;YACL,IAAI,EAAE,QAAQ;YACd,KAAK,EAAE,KAAK,CAAC,MAAM;YACnB,aAAa;YACb,oBAAoB;YACpB,SAAS;YACT,eAAe;SAChB,CAAC;IACJ,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,KAAK,EAAE,CAAC,EAAE,aAAa,EAAE,CAAC,EAAE,oBAAoB,EAAE,CAAC,EAAE,SAAS,EAAE,CAAC,EAAE,eAAe,EAAE,CAAC,EAAE,CAAC;IACnH,CAAC;AACH,CAAC;AAED;;GAEG;AACH,SAAgB,YAAY,CAAC,KAAe,EAAE,WAAmB,GAAG;IAClE,OAAO,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,WAAW,CAAC,CAAC,CAAC,CAAC,CAAC;AAC3D,CAAC;AAED;;GAEG;AACH,SAAgB,iBAAiB,CAC/B,MAAwB,EACxB,aAAmC,0BAAkB;IAErD,OAAO,CACL,MAAM,CAAC,oBAAoB,GAAG,UAAU,CAAC,UAAU;QACnD,MAAM,CAAC,SAAS,GAAG,UAAU,CAAC,SAAS;QACvC,MAAM,CAAC,KAAK,GAAG,UAAU,CAAC,KAAK;QAC/B,MAAM,CAAC,eAAe,GAAG,UAAU,CAAC,OAAO,CAC5C,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,SAAgB,mBAAmB,CAAC,UAAkB;IACpD,IAAI,UAAU,IAAI,CAAC;QAAE,OAAO,KAAK,CAAC;IAClC,IAAI,UAAU,IAAI,EAAE;QAAE,OAAO,QAAQ,CAAC;IACtC,IAAI,UAAU,IAAI,EAAE;QAAE,OAAO,MAAM,CAAC;IACpC,OAAO,UAAU,CAAC;AACpB,CAAC;AAED;;GAEG;AACH,SAAgB,aAAa,CAC3B,OAA2B,EAC3B,aAAmC,0BAAkB;IAErD,OAAO,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,iBAAiB,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC;AAC/D,CAAC;AAED,kBAAe;IACb,kBAAkB,EAAlB,0BAAkB;IAClB,WAAW;IACX,YAAY;IACZ,iBAAiB;IACjB,mBAAmB;IACnB,aAAa;CACd,CAAC"}
|
||||
142
npm/packages/ruvector/src/analysis/complexity.ts
Normal file
142
npm/packages/ruvector/src/analysis/complexity.ts
Normal file
@@ -0,0 +1,142 @@
|
||||
/**
|
||||
* Complexity Analysis Module - Consolidated code complexity metrics
|
||||
*
|
||||
* Single source of truth for cyclomatic complexity and code metrics.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
|
||||
export interface ComplexityResult {
|
||||
file: string;
|
||||
lines: number;
|
||||
nonEmptyLines: number;
|
||||
cyclomaticComplexity: number;
|
||||
functions: number;
|
||||
avgFunctionSize: number;
|
||||
maxFunctionComplexity?: number;
|
||||
}
|
||||
|
||||
export interface ComplexityThresholds {
|
||||
complexity: number; // Max cyclomatic complexity
|
||||
functions: number; // Max functions per file
|
||||
lines: number; // Max lines per file
|
||||
avgSize: number; // Max avg function size
|
||||
}
|
||||
|
||||
export const DEFAULT_THRESHOLDS: ComplexityThresholds = {
|
||||
complexity: 10,
|
||||
functions: 30,
|
||||
lines: 500,
|
||||
avgSize: 50,
|
||||
};
|
||||
|
||||
/**
|
||||
* Analyze complexity of a single file
|
||||
*/
|
||||
export function analyzeFile(filePath: string, content?: string): ComplexityResult {
|
||||
try {
|
||||
const fileContent = content ?? (fs.existsSync(filePath) ? fs.readFileSync(filePath, 'utf-8') : '');
|
||||
if (!fileContent) {
|
||||
return { file: filePath, lines: 0, nonEmptyLines: 0, cyclomaticComplexity: 1, functions: 0, avgFunctionSize: 0 };
|
||||
}
|
||||
|
||||
const lines = fileContent.split('\n');
|
||||
const nonEmptyLines = lines.filter(l => l.trim().length > 0).length;
|
||||
|
||||
// Count branching statements for cyclomatic complexity
|
||||
const branches =
|
||||
(fileContent.match(/\bif\b/g)?.length || 0) +
|
||||
(fileContent.match(/\belse\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bfor\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bwhile\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bswitch\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bcase\b/g)?.length || 0) +
|
||||
(fileContent.match(/\bcatch\b/g)?.length || 0) +
|
||||
(fileContent.match(/\?\?/g)?.length || 0) +
|
||||
(fileContent.match(/&&/g)?.length || 0) +
|
||||
(fileContent.match(/\|\|/g)?.length || 0) +
|
||||
(fileContent.match(/\?[^:]/g)?.length || 0); // Ternary
|
||||
|
||||
const cyclomaticComplexity = branches + 1;
|
||||
|
||||
// Count functions
|
||||
const functionPatterns = [
|
||||
/function\s+\w+/g,
|
||||
/\w+\s*=\s*(?:async\s*)?\(/g,
|
||||
/\w+\s*:\s*(?:async\s*)?\(/g,
|
||||
/(?:async\s+)?(?:public|private|protected)?\s+\w+\s*\([^)]*\)\s*[:{]/g,
|
||||
];
|
||||
|
||||
let functions = 0;
|
||||
for (const pattern of functionPatterns) {
|
||||
functions += (fileContent.match(pattern) || []).length;
|
||||
}
|
||||
// Deduplicate by rough estimate
|
||||
functions = Math.ceil(functions / 2);
|
||||
|
||||
const avgFunctionSize = functions > 0 ? Math.round(nonEmptyLines / functions) : nonEmptyLines;
|
||||
|
||||
return {
|
||||
file: filePath,
|
||||
lines: lines.length,
|
||||
nonEmptyLines,
|
||||
cyclomaticComplexity,
|
||||
functions,
|
||||
avgFunctionSize,
|
||||
};
|
||||
} catch {
|
||||
return { file: filePath, lines: 0, nonEmptyLines: 0, cyclomaticComplexity: 1, functions: 0, avgFunctionSize: 0 };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Analyze complexity of multiple files
|
||||
*/
|
||||
export function analyzeFiles(files: string[], maxFiles: number = 100): ComplexityResult[] {
|
||||
return files.slice(0, maxFiles).map(f => analyzeFile(f));
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if complexity exceeds thresholds
|
||||
*/
|
||||
export function exceedsThresholds(
|
||||
result: ComplexityResult,
|
||||
thresholds: ComplexityThresholds = DEFAULT_THRESHOLDS
|
||||
): boolean {
|
||||
return (
|
||||
result.cyclomaticComplexity > thresholds.complexity ||
|
||||
result.functions > thresholds.functions ||
|
||||
result.lines > thresholds.lines ||
|
||||
result.avgFunctionSize > thresholds.avgSize
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get complexity rating
|
||||
*/
|
||||
export function getComplexityRating(complexity: number): 'low' | 'medium' | 'high' | 'critical' {
|
||||
if (complexity <= 5) return 'low';
|
||||
if (complexity <= 10) return 'medium';
|
||||
if (complexity <= 20) return 'high';
|
||||
return 'critical';
|
||||
}
|
||||
|
||||
/**
|
||||
* Filter files exceeding thresholds
|
||||
*/
|
||||
export function filterComplex(
|
||||
results: ComplexityResult[],
|
||||
thresholds: ComplexityThresholds = DEFAULT_THRESHOLDS
|
||||
): ComplexityResult[] {
|
||||
return results.filter(r => exceedsThresholds(r, thresholds));
|
||||
}
|
||||
|
||||
export default {
|
||||
DEFAULT_THRESHOLDS,
|
||||
analyzeFile,
|
||||
analyzeFiles,
|
||||
exceedsThresholds,
|
||||
getComplexityRating,
|
||||
filterComplex,
|
||||
};
|
||||
1
npm/packages/ruvector/src/analysis/index.d.ts.map
Normal file
1
npm/packages/ruvector/src/analysis/index.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["index.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAEH,cAAc,YAAY,CAAC;AAC3B,cAAc,cAAc,CAAC;AAC7B,cAAc,YAAY,CAAC;AAG3B,OAAO,EAAE,OAAO,IAAI,QAAQ,EAAE,MAAM,YAAY,CAAC;AACjD,OAAO,EAAE,OAAO,IAAI,UAAU,EAAE,MAAM,cAAc,CAAC;AACrD,OAAO,EAAE,OAAO,IAAI,QAAQ,EAAE,MAAM,YAAY,CAAC"}
|
||||
1
npm/packages/ruvector/src/analysis/index.js.map
Normal file
1
npm/packages/ruvector/src/analysis/index.js.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"index.js","sourceRoot":"","sources":["index.ts"],"names":[],"mappings":";AAAA;;;;;;;GAOG;;;;;;;;;;;;;;;;;;;;AAEH,6CAA2B;AAC3B,+CAA6B;AAC7B,6CAA2B;AAE3B,qCAAqC;AACrC,uCAAiD;AAAxC,qHAAA,OAAO,OAAY;AAC5B,2CAAqD;AAA5C,yHAAA,OAAO,OAAc;AAC9B,uCAAiD;AAAxC,qHAAA,OAAO,OAAY"}
|
||||
17
npm/packages/ruvector/src/analysis/index.ts
Normal file
17
npm/packages/ruvector/src/analysis/index.ts
Normal file
@@ -0,0 +1,17 @@
|
||||
/**
|
||||
* Analysis Module - Consolidated code analysis utilities
|
||||
*
|
||||
* Single source of truth for:
|
||||
* - Security scanning
|
||||
* - Complexity analysis
|
||||
* - Pattern extraction
|
||||
*/
|
||||
|
||||
export * from './security';
|
||||
export * from './complexity';
|
||||
export * from './patterns';
|
||||
|
||||
// Re-export defaults for convenience
|
||||
export { default as security } from './security';
|
||||
export { default as complexity } from './complexity';
|
||||
export { default as patterns } from './patterns';
|
||||
71
npm/packages/ruvector/src/analysis/patterns.d.ts
vendored
Normal file
71
npm/packages/ruvector/src/analysis/patterns.d.ts
vendored
Normal file
@@ -0,0 +1,71 @@
|
||||
/**
|
||||
* Pattern Extraction Module - Consolidated code pattern detection
|
||||
*
|
||||
* Single source of truth for extracting functions, imports, exports, etc.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
export interface PatternMatch {
|
||||
type: 'function' | 'class' | 'import' | 'export' | 'todo' | 'variable' | 'type';
|
||||
match: string;
|
||||
file: string;
|
||||
line?: number;
|
||||
}
|
||||
export interface FilePatterns {
|
||||
file: string;
|
||||
language: string;
|
||||
functions: string[];
|
||||
classes: string[];
|
||||
imports: string[];
|
||||
exports: string[];
|
||||
todos: string[];
|
||||
variables: string[];
|
||||
}
|
||||
/**
|
||||
* Detect language from file extension
|
||||
*/
|
||||
export declare function detectLanguage(file: string): string;
|
||||
/**
|
||||
* Extract function names from content
|
||||
*/
|
||||
export declare function extractFunctions(content: string): string[];
|
||||
/**
|
||||
* Extract class names from content
|
||||
*/
|
||||
export declare function extractClasses(content: string): string[];
|
||||
/**
|
||||
* Extract import statements from content
|
||||
*/
|
||||
export declare function extractImports(content: string): string[];
|
||||
/**
|
||||
* Extract export statements from content
|
||||
*/
|
||||
export declare function extractExports(content: string): string[];
|
||||
/**
|
||||
* Extract TODO/FIXME comments from content
|
||||
*/
|
||||
export declare function extractTodos(content: string): string[];
|
||||
/**
|
||||
* Extract all patterns from a file
|
||||
*/
|
||||
export declare function extractAllPatterns(filePath: string, content?: string): FilePatterns;
|
||||
/**
|
||||
* Extract patterns from multiple files
|
||||
*/
|
||||
export declare function extractFromFiles(files: string[], maxFiles?: number): FilePatterns[];
|
||||
/**
|
||||
* Convert FilePatterns to PatternMatch array (for native-worker compatibility)
|
||||
*/
|
||||
export declare function toPatternMatches(patterns: FilePatterns): PatternMatch[];
|
||||
declare const _default: {
|
||||
detectLanguage: typeof detectLanguage;
|
||||
extractFunctions: typeof extractFunctions;
|
||||
extractClasses: typeof extractClasses;
|
||||
extractImports: typeof extractImports;
|
||||
extractExports: typeof extractExports;
|
||||
extractTodos: typeof extractTodos;
|
||||
extractAllPatterns: typeof extractAllPatterns;
|
||||
extractFromFiles: typeof extractFromFiles;
|
||||
toPatternMatches: typeof toPatternMatches;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=patterns.d.ts.map
|
||||
1
npm/packages/ruvector/src/analysis/patterns.d.ts.map
Normal file
1
npm/packages/ruvector/src/analysis/patterns.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"patterns.d.ts","sourceRoot":"","sources":["patterns.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAIH,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,UAAU,GAAG,OAAO,GAAG,QAAQ,GAAG,QAAQ,GAAG,MAAM,GAAG,UAAU,GAAG,MAAM,CAAC;IAChF,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAED,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,KAAK,EAAE,MAAM,EAAE,CAAC;IAChB,SAAS,EAAE,MAAM,EAAE,CAAC;CACrB;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAUnD;AAED;;GAEG;AACH,wBAAgB,gBAAgB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CA2B1D;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAmBxD;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAmBxD;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAuBxD;AAED;;GAEG;AACH,wBAAgB,YAAY,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAUtD;AAED;;GAEG;AACH,wBAAgB,kBAAkB,CAAC,QAAQ,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,MAAM,GAAG,YAAY,CA0BnF;AAED;;GAEG;AACH,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,QAAQ,GAAE,MAAY,GAAG,YAAY,EAAE,CAExF;AAED;;GAEG;AACH,wBAAgB,gBAAgB,CAAC,QAAQ,EAAE,YAAY,GAAG,YAAY,EAAE,CAoBvE;;;;;;;;;;;;AAED,wBAUE"}
|
||||
244
npm/packages/ruvector/src/analysis/patterns.js
Normal file
244
npm/packages/ruvector/src/analysis/patterns.js
Normal file
@@ -0,0 +1,244 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Pattern Extraction Module - Consolidated code pattern detection
|
||||
*
|
||||
* Single source of truth for extracting functions, imports, exports, etc.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
var desc = Object.getOwnPropertyDescriptor(m, k);
|
||||
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
||||
desc = { enumerable: true, get: function() { return m[k]; } };
|
||||
}
|
||||
Object.defineProperty(o, k2, desc);
|
||||
}) : (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
o[k2] = m[k];
|
||||
}));
|
||||
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
||||
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
||||
}) : function(o, v) {
|
||||
o["default"] = v;
|
||||
});
|
||||
var __importStar = (this && this.__importStar) || (function () {
|
||||
var ownKeys = function(o) {
|
||||
ownKeys = Object.getOwnPropertyNames || function (o) {
|
||||
var ar = [];
|
||||
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
||||
return ar;
|
||||
};
|
||||
return ownKeys(o);
|
||||
};
|
||||
return function (mod) {
|
||||
if (mod && mod.__esModule) return mod;
|
||||
var result = {};
|
||||
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
||||
__setModuleDefault(result, mod);
|
||||
return result;
|
||||
};
|
||||
})();
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.detectLanguage = detectLanguage;
|
||||
exports.extractFunctions = extractFunctions;
|
||||
exports.extractClasses = extractClasses;
|
||||
exports.extractImports = extractImports;
|
||||
exports.extractExports = extractExports;
|
||||
exports.extractTodos = extractTodos;
|
||||
exports.extractAllPatterns = extractAllPatterns;
|
||||
exports.extractFromFiles = extractFromFiles;
|
||||
exports.toPatternMatches = toPatternMatches;
|
||||
const fs = __importStar(require("fs"));
|
||||
/**
|
||||
* Detect language from file extension
|
||||
*/
|
||||
function detectLanguage(file) {
|
||||
const ext = file.split('.').pop()?.toLowerCase() || '';
|
||||
const langMap = {
|
||||
ts: 'typescript', tsx: 'typescript', js: 'javascript', jsx: 'javascript',
|
||||
rs: 'rust', py: 'python', go: 'go', java: 'java', rb: 'ruby',
|
||||
cpp: 'cpp', c: 'c', h: 'c', hpp: 'cpp', cs: 'csharp',
|
||||
md: 'markdown', json: 'json', yaml: 'yaml', yml: 'yaml',
|
||||
sql: 'sql', sh: 'shell', bash: 'shell', zsh: 'shell',
|
||||
};
|
||||
return langMap[ext] || ext || 'unknown';
|
||||
}
|
||||
/**
|
||||
* Extract function names from content
|
||||
*/
|
||||
function extractFunctions(content) {
|
||||
const patterns = [
|
||||
/function\s+(\w+)/g,
|
||||
/const\s+(\w+)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>/g,
|
||||
/let\s+(\w+)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>/g,
|
||||
/(?:async\s+)?(?:public|private|protected)?\s+(\w+)\s*\([^)]*\)\s*[:{]/g,
|
||||
/(\w+)\s*:\s*(?:async\s*)?\([^)]*\)\s*=>/g,
|
||||
/def\s+(\w+)\s*\(/g, // Python
|
||||
/fn\s+(\w+)\s*[<(]/g, // Rust
|
||||
/func\s+(\w+)\s*\(/g, // Go
|
||||
];
|
||||
const funcs = new Set();
|
||||
const reserved = new Set(['if', 'for', 'while', 'switch', 'catch', 'try', 'else', 'return', 'new', 'class', 'function', 'async', 'await']);
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
const name = match[1];
|
||||
if (name && !reserved.has(name) && name.length > 1) {
|
||||
funcs.add(name);
|
||||
}
|
||||
}
|
||||
}
|
||||
return Array.from(funcs);
|
||||
}
|
||||
/**
|
||||
* Extract class names from content
|
||||
*/
|
||||
function extractClasses(content) {
|
||||
const patterns = [
|
||||
/class\s+(\w+)/g,
|
||||
/interface\s+(\w+)/g,
|
||||
/type\s+(\w+)\s*=/g,
|
||||
/enum\s+(\w+)/g,
|
||||
/struct\s+(\w+)/g,
|
||||
];
|
||||
const classes = new Set();
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
if (match[1])
|
||||
classes.add(match[1]);
|
||||
}
|
||||
}
|
||||
return Array.from(classes);
|
||||
}
|
||||
/**
|
||||
* Extract import statements from content
|
||||
*/
|
||||
function extractImports(content) {
|
||||
const patterns = [
|
||||
/import\s+.*?from\s+['"]([^'"]+)['"]/g,
|
||||
/import\s+['"]([^'"]+)['"]/g,
|
||||
/require\s*\(['"]([^'"]+)['"]\)/g,
|
||||
/from\s+(\w+)\s+import/g, // Python
|
||||
/use\s+(\w+(?:::\w+)*)/g, // Rust
|
||||
];
|
||||
const imports = [];
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
if (match[1])
|
||||
imports.push(match[1]);
|
||||
}
|
||||
}
|
||||
return [...new Set(imports)];
|
||||
}
|
||||
/**
|
||||
* Extract export statements from content
|
||||
*/
|
||||
function extractExports(content) {
|
||||
const patterns = [
|
||||
/export\s+(?:default\s+)?(?:class|function|const|let|var|interface|type|enum)\s+(\w+)/g,
|
||||
/export\s*\{\s*([^}]+)\s*\}/g,
|
||||
/module\.exports\s*=\s*(\w+)/g,
|
||||
/exports\.(\w+)\s*=/g,
|
||||
/pub\s+(?:fn|struct|enum|type)\s+(\w+)/g, // Rust
|
||||
];
|
||||
const exports = [];
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
if (match[1]) {
|
||||
// Handle grouped exports: export { a, b, c }
|
||||
const names = match[1].split(',').map(s => s.trim().split(/\s+as\s+/)[0].trim());
|
||||
exports.push(...names.filter(n => n && /^\w+$/.test(n)));
|
||||
}
|
||||
}
|
||||
}
|
||||
return [...new Set(exports)];
|
||||
}
|
||||
/**
|
||||
* Extract TODO/FIXME comments from content
|
||||
*/
|
||||
function extractTodos(content) {
|
||||
const pattern = /\/\/\s*(TODO|FIXME|HACK|XXX|BUG|NOTE):\s*(.+)/gi;
|
||||
const todos = [];
|
||||
let match;
|
||||
while ((match = pattern.exec(content)) !== null) {
|
||||
todos.push(`${match[1]}: ${match[2].trim()}`);
|
||||
}
|
||||
return todos;
|
||||
}
|
||||
/**
|
||||
* Extract all patterns from a file
|
||||
*/
|
||||
function extractAllPatterns(filePath, content) {
|
||||
try {
|
||||
const fileContent = content ?? (fs.existsSync(filePath) ? fs.readFileSync(filePath, 'utf-8') : '');
|
||||
return {
|
||||
file: filePath,
|
||||
language: detectLanguage(filePath),
|
||||
functions: extractFunctions(fileContent),
|
||||
classes: extractClasses(fileContent),
|
||||
imports: extractImports(fileContent),
|
||||
exports: extractExports(fileContent),
|
||||
todos: extractTodos(fileContent),
|
||||
variables: [], // Could add variable extraction if needed
|
||||
};
|
||||
}
|
||||
catch {
|
||||
return {
|
||||
file: filePath,
|
||||
language: detectLanguage(filePath),
|
||||
functions: [],
|
||||
classes: [],
|
||||
imports: [],
|
||||
exports: [],
|
||||
todos: [],
|
||||
variables: [],
|
||||
};
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Extract patterns from multiple files
|
||||
*/
|
||||
function extractFromFiles(files, maxFiles = 100) {
|
||||
return files.slice(0, maxFiles).map(f => extractAllPatterns(f));
|
||||
}
|
||||
/**
|
||||
* Convert FilePatterns to PatternMatch array (for native-worker compatibility)
|
||||
*/
|
||||
function toPatternMatches(patterns) {
|
||||
const matches = [];
|
||||
for (const func of patterns.functions) {
|
||||
matches.push({ type: 'function', match: func, file: patterns.file });
|
||||
}
|
||||
for (const cls of patterns.classes) {
|
||||
matches.push({ type: 'class', match: cls, file: patterns.file });
|
||||
}
|
||||
for (const imp of patterns.imports) {
|
||||
matches.push({ type: 'import', match: imp, file: patterns.file });
|
||||
}
|
||||
for (const exp of patterns.exports) {
|
||||
matches.push({ type: 'export', match: exp, file: patterns.file });
|
||||
}
|
||||
for (const todo of patterns.todos) {
|
||||
matches.push({ type: 'todo', match: todo, file: patterns.file });
|
||||
}
|
||||
return matches;
|
||||
}
|
||||
exports.default = {
|
||||
detectLanguage,
|
||||
extractFunctions,
|
||||
extractClasses,
|
||||
extractImports,
|
||||
extractExports,
|
||||
extractTodos,
|
||||
extractAllPatterns,
|
||||
extractFromFiles,
|
||||
toPatternMatches,
|
||||
};
|
||||
//# sourceMappingURL=patterns.js.map
|
||||
1
npm/packages/ruvector/src/analysis/patterns.js.map
Normal file
1
npm/packages/ruvector/src/analysis/patterns.js.map
Normal file
File diff suppressed because one or more lines are too long
239
npm/packages/ruvector/src/analysis/patterns.ts
Normal file
239
npm/packages/ruvector/src/analysis/patterns.ts
Normal file
@@ -0,0 +1,239 @@
|
||||
/**
|
||||
* Pattern Extraction Module - Consolidated code pattern detection
|
||||
*
|
||||
* Single source of truth for extracting functions, imports, exports, etc.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
|
||||
export interface PatternMatch {
|
||||
type: 'function' | 'class' | 'import' | 'export' | 'todo' | 'variable' | 'type';
|
||||
match: string;
|
||||
file: string;
|
||||
line?: number;
|
||||
}
|
||||
|
||||
export interface FilePatterns {
|
||||
file: string;
|
||||
language: string;
|
||||
functions: string[];
|
||||
classes: string[];
|
||||
imports: string[];
|
||||
exports: string[];
|
||||
todos: string[];
|
||||
variables: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect language from file extension
|
||||
*/
|
||||
export function detectLanguage(file: string): string {
|
||||
const ext = file.split('.').pop()?.toLowerCase() || '';
|
||||
const langMap: Record<string, string> = {
|
||||
ts: 'typescript', tsx: 'typescript', js: 'javascript', jsx: 'javascript',
|
||||
rs: 'rust', py: 'python', go: 'go', java: 'java', rb: 'ruby',
|
||||
cpp: 'cpp', c: 'c', h: 'c', hpp: 'cpp', cs: 'csharp',
|
||||
md: 'markdown', json: 'json', yaml: 'yaml', yml: 'yaml',
|
||||
sql: 'sql', sh: 'shell', bash: 'shell', zsh: 'shell',
|
||||
};
|
||||
return langMap[ext] || ext || 'unknown';
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract function names from content
|
||||
*/
|
||||
export function extractFunctions(content: string): string[] {
|
||||
const patterns = [
|
||||
/function\s+(\w+)/g,
|
||||
/const\s+(\w+)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>/g,
|
||||
/let\s+(\w+)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>/g,
|
||||
/(?:async\s+)?(?:public|private|protected)?\s+(\w+)\s*\([^)]*\)\s*[:{]/g,
|
||||
/(\w+)\s*:\s*(?:async\s*)?\([^)]*\)\s*=>/g,
|
||||
/def\s+(\w+)\s*\(/g, // Python
|
||||
/fn\s+(\w+)\s*[<(]/g, // Rust
|
||||
/func\s+(\w+)\s*\(/g, // Go
|
||||
];
|
||||
|
||||
const funcs = new Set<string>();
|
||||
const reserved = new Set(['if', 'for', 'while', 'switch', 'catch', 'try', 'else', 'return', 'new', 'class', 'function', 'async', 'await']);
|
||||
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
const name = match[1];
|
||||
if (name && !reserved.has(name) && name.length > 1) {
|
||||
funcs.add(name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return Array.from(funcs);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract class names from content
|
||||
*/
|
||||
export function extractClasses(content: string): string[] {
|
||||
const patterns = [
|
||||
/class\s+(\w+)/g,
|
||||
/interface\s+(\w+)/g,
|
||||
/type\s+(\w+)\s*=/g,
|
||||
/enum\s+(\w+)/g,
|
||||
/struct\s+(\w+)/g,
|
||||
];
|
||||
|
||||
const classes = new Set<string>();
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
if (match[1]) classes.add(match[1]);
|
||||
}
|
||||
}
|
||||
|
||||
return Array.from(classes);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract import statements from content
|
||||
*/
|
||||
export function extractImports(content: string): string[] {
|
||||
const patterns = [
|
||||
/import\s+.*?from\s+['"]([^'"]+)['"]/g,
|
||||
/import\s+['"]([^'"]+)['"]/g,
|
||||
/require\s*\(['"]([^'"]+)['"]\)/g,
|
||||
/from\s+(\w+)\s+import/g, // Python
|
||||
/use\s+(\w+(?:::\w+)*)/g, // Rust
|
||||
];
|
||||
|
||||
const imports: string[] = [];
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
if (match[1]) imports.push(match[1]);
|
||||
}
|
||||
}
|
||||
|
||||
return [...new Set(imports)];
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract export statements from content
|
||||
*/
|
||||
export function extractExports(content: string): string[] {
|
||||
const patterns = [
|
||||
/export\s+(?:default\s+)?(?:class|function|const|let|var|interface|type|enum)\s+(\w+)/g,
|
||||
/export\s*\{\s*([^}]+)\s*\}/g,
|
||||
/module\.exports\s*=\s*(\w+)/g,
|
||||
/exports\.(\w+)\s*=/g,
|
||||
/pub\s+(?:fn|struct|enum|type)\s+(\w+)/g, // Rust
|
||||
];
|
||||
|
||||
const exports: string[] = [];
|
||||
for (const pattern of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(content)) !== null) {
|
||||
if (match[1]) {
|
||||
// Handle grouped exports: export { a, b, c }
|
||||
const names = match[1].split(',').map(s => s.trim().split(/\s+as\s+/)[0].trim());
|
||||
exports.push(...names.filter(n => n && /^\w+$/.test(n)));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return [...new Set(exports)];
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract TODO/FIXME comments from content
|
||||
*/
|
||||
export function extractTodos(content: string): string[] {
|
||||
const pattern = /\/\/\s*(TODO|FIXME|HACK|XXX|BUG|NOTE):\s*(.+)/gi;
|
||||
const todos: string[] = [];
|
||||
|
||||
let match;
|
||||
while ((match = pattern.exec(content)) !== null) {
|
||||
todos.push(`${match[1]}: ${match[2].trim()}`);
|
||||
}
|
||||
|
||||
return todos;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract all patterns from a file
|
||||
*/
|
||||
export function extractAllPatterns(filePath: string, content?: string): FilePatterns {
|
||||
try {
|
||||
const fileContent = content ?? (fs.existsSync(filePath) ? fs.readFileSync(filePath, 'utf-8') : '');
|
||||
|
||||
return {
|
||||
file: filePath,
|
||||
language: detectLanguage(filePath),
|
||||
functions: extractFunctions(fileContent),
|
||||
classes: extractClasses(fileContent),
|
||||
imports: extractImports(fileContent),
|
||||
exports: extractExports(fileContent),
|
||||
todos: extractTodos(fileContent),
|
||||
variables: [], // Could add variable extraction if needed
|
||||
};
|
||||
} catch {
|
||||
return {
|
||||
file: filePath,
|
||||
language: detectLanguage(filePath),
|
||||
functions: [],
|
||||
classes: [],
|
||||
imports: [],
|
||||
exports: [],
|
||||
todos: [],
|
||||
variables: [],
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract patterns from multiple files
|
||||
*/
|
||||
export function extractFromFiles(files: string[], maxFiles: number = 100): FilePatterns[] {
|
||||
return files.slice(0, maxFiles).map(f => extractAllPatterns(f));
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert FilePatterns to PatternMatch array (for native-worker compatibility)
|
||||
*/
|
||||
export function toPatternMatches(patterns: FilePatterns): PatternMatch[] {
|
||||
const matches: PatternMatch[] = [];
|
||||
|
||||
for (const func of patterns.functions) {
|
||||
matches.push({ type: 'function', match: func, file: patterns.file });
|
||||
}
|
||||
for (const cls of patterns.classes) {
|
||||
matches.push({ type: 'class', match: cls, file: patterns.file });
|
||||
}
|
||||
for (const imp of patterns.imports) {
|
||||
matches.push({ type: 'import', match: imp, file: patterns.file });
|
||||
}
|
||||
for (const exp of patterns.exports) {
|
||||
matches.push({ type: 'export', match: exp, file: patterns.file });
|
||||
}
|
||||
for (const todo of patterns.todos) {
|
||||
matches.push({ type: 'todo', match: todo, file: patterns.file });
|
||||
}
|
||||
|
||||
return matches;
|
||||
}
|
||||
|
||||
export default {
|
||||
detectLanguage,
|
||||
extractFunctions,
|
||||
extractClasses,
|
||||
extractImports,
|
||||
extractExports,
|
||||
extractTodos,
|
||||
extractAllPatterns,
|
||||
extractFromFiles,
|
||||
toPatternMatches,
|
||||
};
|
||||
51
npm/packages/ruvector/src/analysis/security.d.ts
vendored
Normal file
51
npm/packages/ruvector/src/analysis/security.d.ts
vendored
Normal file
@@ -0,0 +1,51 @@
|
||||
/**
|
||||
* Security Analysis Module - Consolidated security scanning
|
||||
*
|
||||
* Single source of truth for security patterns and vulnerability detection.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
export interface SecurityPattern {
|
||||
pattern: RegExp;
|
||||
rule: string;
|
||||
severity: 'low' | 'medium' | 'high' | 'critical';
|
||||
message: string;
|
||||
suggestion?: string;
|
||||
}
|
||||
export interface SecurityFinding {
|
||||
file: string;
|
||||
line: number;
|
||||
severity: 'low' | 'medium' | 'high' | 'critical';
|
||||
rule: string;
|
||||
message: string;
|
||||
match?: string;
|
||||
suggestion?: string;
|
||||
}
|
||||
/**
|
||||
* Default security patterns for vulnerability detection
|
||||
*/
|
||||
export declare const SECURITY_PATTERNS: SecurityPattern[];
|
||||
/**
|
||||
* Scan a single file for security issues
|
||||
*/
|
||||
export declare function scanFile(filePath: string, content?: string, patterns?: SecurityPattern[]): SecurityFinding[];
|
||||
/**
|
||||
* Scan multiple files for security issues
|
||||
*/
|
||||
export declare function scanFiles(files: string[], patterns?: SecurityPattern[], maxFiles?: number): SecurityFinding[];
|
||||
/**
|
||||
* Get severity score (for sorting/filtering)
|
||||
*/
|
||||
export declare function getSeverityScore(severity: string): number;
|
||||
/**
|
||||
* Sort findings by severity (highest first)
|
||||
*/
|
||||
export declare function sortBySeverity(findings: SecurityFinding[]): SecurityFinding[];
|
||||
declare const _default: {
|
||||
SECURITY_PATTERNS: SecurityPattern[];
|
||||
scanFile: typeof scanFile;
|
||||
scanFiles: typeof scanFiles;
|
||||
getSeverityScore: typeof getSeverityScore;
|
||||
sortBySeverity: typeof sortBySeverity;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=security.d.ts.map
|
||||
1
npm/packages/ruvector/src/analysis/security.d.ts.map
Normal file
1
npm/packages/ruvector/src/analysis/security.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"security.d.ts","sourceRoot":"","sources":["security.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAIH,MAAM,WAAW,eAAe;IAC9B,OAAO,EAAE,MAAM,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,UAAU,CAAC;IACjD,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,UAAU,CAAC;IACjD,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED;;GAEG;AACH,eAAO,MAAM,iBAAiB,EAAE,eAAe,EA0B9C,CAAC;AAEF;;GAEG;AACH,wBAAgB,QAAQ,CACtB,QAAQ,EAAE,MAAM,EAChB,OAAO,CAAC,EAAE,MAAM,EAChB,QAAQ,GAAE,eAAe,EAAsB,GAC9C,eAAe,EAAE,CA4BnB;AAED;;GAEG;AACH,wBAAgB,SAAS,CACvB,KAAK,EAAE,MAAM,EAAE,EACf,QAAQ,GAAE,eAAe,EAAsB,EAC/C,QAAQ,GAAE,MAAY,GACrB,eAAe,EAAE,CAQnB;AAED;;GAEG;AACH,wBAAgB,gBAAgB,CAAC,QAAQ,EAAE,MAAM,GAAG,MAAM,CAQzD;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,eAAe,EAAE,GAAG,eAAe,EAAE,CAE7E;;;;;;;;AAED,wBAME"}
|
||||
140
npm/packages/ruvector/src/analysis/security.js
Normal file
140
npm/packages/ruvector/src/analysis/security.js
Normal file
@@ -0,0 +1,140 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Security Analysis Module - Consolidated security scanning
|
||||
*
|
||||
* Single source of truth for security patterns and vulnerability detection.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
var desc = Object.getOwnPropertyDescriptor(m, k);
|
||||
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
||||
desc = { enumerable: true, get: function() { return m[k]; } };
|
||||
}
|
||||
Object.defineProperty(o, k2, desc);
|
||||
}) : (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
o[k2] = m[k];
|
||||
}));
|
||||
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
||||
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
||||
}) : function(o, v) {
|
||||
o["default"] = v;
|
||||
});
|
||||
var __importStar = (this && this.__importStar) || (function () {
|
||||
var ownKeys = function(o) {
|
||||
ownKeys = Object.getOwnPropertyNames || function (o) {
|
||||
var ar = [];
|
||||
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
||||
return ar;
|
||||
};
|
||||
return ownKeys(o);
|
||||
};
|
||||
return function (mod) {
|
||||
if (mod && mod.__esModule) return mod;
|
||||
var result = {};
|
||||
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
||||
__setModuleDefault(result, mod);
|
||||
return result;
|
||||
};
|
||||
})();
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.SECURITY_PATTERNS = void 0;
|
||||
exports.scanFile = scanFile;
|
||||
exports.scanFiles = scanFiles;
|
||||
exports.getSeverityScore = getSeverityScore;
|
||||
exports.sortBySeverity = sortBySeverity;
|
||||
const fs = __importStar(require("fs"));
|
||||
/**
|
||||
* Default security patterns for vulnerability detection
|
||||
*/
|
||||
exports.SECURITY_PATTERNS = [
|
||||
// Critical: Hardcoded secrets
|
||||
{ pattern: /password\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-password', severity: 'critical', message: 'Hardcoded password detected', suggestion: 'Use environment variables or secret management' },
|
||||
{ pattern: /api[_-]?key\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-apikey', severity: 'critical', message: 'Hardcoded API key detected', suggestion: 'Use environment variables' },
|
||||
{ pattern: /secret\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-secret', severity: 'critical', message: 'Hardcoded secret detected', suggestion: 'Use environment variables or secret management' },
|
||||
{ pattern: /private[_-]?key\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-private-key', severity: 'critical', message: 'Hardcoded private key detected', suggestion: 'Use secure key management' },
|
||||
// High: Code execution risks
|
||||
{ pattern: /eval\s*\(/g, rule: 'no-eval', severity: 'high', message: 'Avoid eval() - code injection risk', suggestion: 'Use safer alternatives like JSON.parse()' },
|
||||
{ pattern: /exec\s*\(/g, rule: 'no-exec', severity: 'high', message: 'Avoid exec() - command injection risk', suggestion: 'Use execFile or spawn with args array' },
|
||||
{ pattern: /Function\s*\(/g, rule: 'no-function-constructor', severity: 'high', message: 'Avoid Function constructor - code injection risk' },
|
||||
{ pattern: /child_process.*exec\(/g, rule: 'no-shell-exec', severity: 'high', message: 'Shell execution detected', suggestion: 'Use execFile or spawn instead' },
|
||||
// High: SQL injection
|
||||
{ pattern: /SELECT\s+.*\s+FROM.*\+/gi, rule: 'sql-injection-risk', severity: 'high', message: 'Potential SQL injection - string concatenation in query', suggestion: 'Use parameterized queries' },
|
||||
{ pattern: /`SELECT.*\$\{/gi, rule: 'sql-injection-template', severity: 'high', message: 'Template literal in SQL query', suggestion: 'Use parameterized queries' },
|
||||
// Medium: XSS risks
|
||||
{ pattern: /dangerouslySetInnerHTML/g, rule: 'xss-risk', severity: 'medium', message: 'XSS risk: dangerouslySetInnerHTML', suggestion: 'Sanitize content before rendering' },
|
||||
{ pattern: /innerHTML\s*=/g, rule: 'no-inner-html', severity: 'medium', message: 'Avoid innerHTML - XSS risk', suggestion: 'Use textContent or sanitize content' },
|
||||
{ pattern: /document\.write\s*\(/g, rule: 'no-document-write', severity: 'medium', message: 'Avoid document.write - XSS risk' },
|
||||
// Medium: Other risks
|
||||
{ pattern: /\$\{.*\}/g, rule: 'template-injection', severity: 'low', message: 'Template literal detected - verify no injection' },
|
||||
{ pattern: /new\s+RegExp\s*\([^)]*\+/g, rule: 'regex-injection', severity: 'medium', message: 'Dynamic RegExp - potential ReDoS risk', suggestion: 'Validate/sanitize regex input' },
|
||||
{ pattern: /\.on\s*\(\s*['"]error['"]/g, rule: 'unhandled-error', severity: 'low', message: 'Error handler detected - verify proper error handling' },
|
||||
];
|
||||
/**
|
||||
* Scan a single file for security issues
|
||||
*/
|
||||
function scanFile(filePath, content, patterns = exports.SECURITY_PATTERNS) {
|
||||
const findings = [];
|
||||
try {
|
||||
const fileContent = content ?? (fs.existsSync(filePath) ? fs.readFileSync(filePath, 'utf-8') : '');
|
||||
if (!fileContent)
|
||||
return findings;
|
||||
for (const { pattern, rule, severity, message, suggestion } of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(fileContent)) !== null) {
|
||||
const lineNum = fileContent.slice(0, match.index).split('\n').length;
|
||||
findings.push({
|
||||
file: filePath,
|
||||
line: lineNum,
|
||||
severity,
|
||||
rule,
|
||||
message,
|
||||
match: match[0].slice(0, 50),
|
||||
suggestion,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
catch {
|
||||
// Skip unreadable files
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
/**
|
||||
* Scan multiple files for security issues
|
||||
*/
|
||||
function scanFiles(files, patterns = exports.SECURITY_PATTERNS, maxFiles = 100) {
|
||||
const findings = [];
|
||||
for (const file of files.slice(0, maxFiles)) {
|
||||
findings.push(...scanFile(file, undefined, patterns));
|
||||
}
|
||||
return findings;
|
||||
}
|
||||
/**
|
||||
* Get severity score (for sorting/filtering)
|
||||
*/
|
||||
function getSeverityScore(severity) {
|
||||
switch (severity) {
|
||||
case 'critical': return 4;
|
||||
case 'high': return 3;
|
||||
case 'medium': return 2;
|
||||
case 'low': return 1;
|
||||
default: return 0;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Sort findings by severity (highest first)
|
||||
*/
|
||||
function sortBySeverity(findings) {
|
||||
return [...findings].sort((a, b) => getSeverityScore(b.severity) - getSeverityScore(a.severity));
|
||||
}
|
||||
exports.default = {
|
||||
SECURITY_PATTERNS: exports.SECURITY_PATTERNS,
|
||||
scanFile,
|
||||
scanFiles,
|
||||
getSeverityScore,
|
||||
sortBySeverity,
|
||||
};
|
||||
//# sourceMappingURL=security.js.map
|
||||
1
npm/packages/ruvector/src/analysis/security.js.map
Normal file
1
npm/packages/ruvector/src/analysis/security.js.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"security.js","sourceRoot":"","sources":["security.ts"],"names":[],"mappings":";AAAA;;;;;GAKG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAwDH,4BAgCC;AAKD,8BAYC;AAKD,4CAQC;AAKD,wCAEC;AA3HD,uCAAyB;AAoBzB;;GAEG;AACU,QAAA,iBAAiB,GAAsB;IAClD,8BAA8B;IAC9B,EAAE,OAAO,EAAE,iCAAiC,EAAE,IAAI,EAAE,uBAAuB,EAAE,QAAQ,EAAE,UAAU,EAAE,OAAO,EAAE,6BAA6B,EAAE,UAAU,EAAE,gDAAgD,EAAE;IACzM,EAAE,OAAO,EAAE,oCAAoC,EAAE,IAAI,EAAE,qBAAqB,EAAE,QAAQ,EAAE,UAAU,EAAE,OAAO,EAAE,4BAA4B,EAAE,UAAU,EAAE,2BAA2B,EAAE;IACpL,EAAE,OAAO,EAAE,+BAA+B,EAAE,IAAI,EAAE,qBAAqB,EAAE,QAAQ,EAAE,UAAU,EAAE,OAAO,EAAE,2BAA2B,EAAE,UAAU,EAAE,gDAAgD,EAAE;IACnM,EAAE,OAAO,EAAE,wCAAwC,EAAE,IAAI,EAAE,0BAA0B,EAAE,QAAQ,EAAE,UAAU,EAAE,OAAO,EAAE,gCAAgC,EAAE,UAAU,EAAE,2BAA2B,EAAE;IAEjM,6BAA6B;IAC7B,EAAE,OAAO,EAAE,YAAY,EAAE,IAAI,EAAE,SAAS,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,EAAE,oCAAoC,EAAE,UAAU,EAAE,0CAA0C,EAAE;IACnK,EAAE,OAAO,EAAE,YAAY,EAAE,IAAI,EAAE,SAAS,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,EAAE,uCAAuC,EAAE,UAAU,EAAE,uCAAuC,EAAE;IACnK,EAAE,OAAO,EAAE,gBAAgB,EAAE,IAAI,EAAE,yBAAyB,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,EAAE,kDAAkD,EAAE;IAC7I,EAAE,OAAO,EAAE,wBAAwB,EAAE,IAAI,EAAE,eAAe,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,EAAE,0BAA0B,EAAE,UAAU,EAAE,+BAA+B,EAAE;IAEhK,sBAAsB;IACtB,EAAE,OAAO,EAAE,0BAA0B,EAAE,IAAI,EAAE,oBAAoB,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,EAAE,yDAAyD,EAAE,UAAU,EAAE,2BAA2B,EAAE;IAClM,EAAE,OAAO,EAAE,iBAAiB,EAAE,IAAI,EAAE,wBAAwB,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,EAAE,+BAA+B,EAAE,UAAU,EAAE,2BAA2B,EAAE;IAEnK,oBAAoB;IACpB,EAAE,OAAO,EAAE,0BAA0B,EAAE,IAAI,EAAE,UAAU,EAAE,QAAQ,EAAE,QAAQ,EAAE,OAAO,EAAE,mCAAmC,EAAE,UAAU,EAAE,mCAAmC,EAAE;IAC5K,EAAE,OAAO,EAAE,gBAAgB,EAAE,IAAI,EAAE,eAAe,EAAE,QAAQ,EAAE,QAAQ,EAAE,OAAO,EAAE,4BAA4B,EAAE,UAAU,EAAE,qCAAqC,EAAE;IAClK,EAAE,OAAO,EAAE,uBAAuB,EAAE,IAAI,EAAE,mBAAmB,EAAE,QAAQ,EAAE,QAAQ,EAAE,OAAO,EAAE,iCAAiC,EAAE;IAE/H,sBAAsB;IACtB,EAAE,OAAO,EAAE,WAAW,EAAE,IAAI,EAAE,oBAAoB,EAAE,QAAQ,EAAE,KAAK,EAAE,OAAO,EAAE,iDAAiD,EAAE;IACjI,EAAE,OAAO,EAAE,2BAA2B,EAAE,IAAI,EAAE,iBAAiB,EAAE,QAAQ,EAAE,QAAQ,EAAE,OAAO,EAAE,uCAAuC,EAAE,UAAU,EAAE,+BAA+B,EAAE;IACpL,EAAE,OAAO,EAAE,4BAA4B,EAAE,IAAI,EAAE,iBAAiB,EAAE,QAAQ,EAAE,KAAK,EAAE,OAAO,EAAE,uDAAuD,EAAE;CACtJ,CAAC;AAEF;;GAEG;AACH,SAAgB,QAAQ,CACtB,QAAgB,EAChB,OAAgB,EAChB,WAA8B,yBAAiB;IAE/C,MAAM,QAAQ,GAAsB,EAAE,CAAC;IAEvC,IAAI,CAAC;QACH,MAAM,WAAW,GAAG,OAAO,IAAI,CAAC,EAAE,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,YAAY,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QACnG,IAAI,CAAC,WAAW;YAAE,OAAO,QAAQ,CAAC;QAElC,KAAK,MAAM,EAAE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,UAAU,EAAE,IAAI,QAAQ,EAAE,CAAC;YACxE,MAAM,KAAK,GAAG,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,CAAC,CAAC;YACxD,IAAI,KAAK,CAAC;YACV,OAAO,CAAC,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;gBAClD,MAAM,OAAO,GAAG,WAAW,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC;gBACrE,QAAQ,CAAC,IAAI,CAAC;oBACZ,IAAI,EAAE,QAAQ;oBACd,IAAI,EAAE,OAAO;oBACb,QAAQ;oBACR,IAAI;oBACJ,OAAO;oBACP,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC;oBAC5B,UAAU;iBACX,CAAC,CAAC;YACL,CAAC;QACH,CAAC;IACH,CAAC;IAAC,MAAM,CAAC;QACP,wBAAwB;IAC1B,CAAC;IAED,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED;;GAEG;AACH,SAAgB,SAAS,CACvB,KAAe,EACf,WAA8B,yBAAiB,EAC/C,WAAmB,GAAG;IAEtB,MAAM,QAAQ,GAAsB,EAAE,CAAC;IAEvC,KAAK,MAAM,IAAI,IAAI,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,QAAQ,CAAC,EAAE,CAAC;QAC5C,QAAQ,CAAC,IAAI,CAAC,GAAG,QAAQ,CAAC,IAAI,EAAE,SAAS,EAAE,QAAQ,CAAC,CAAC,CAAC;IACxD,CAAC;IAED,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED;;GAEG;AACH,SAAgB,gBAAgB,CAAC,QAAgB;IAC/C,QAAQ,QAAQ,EAAE,CAAC;QACjB,KAAK,UAAU,CAAC,CAAC,OAAO,CAAC,CAAC;QAC1B,KAAK,MAAM,CAAC,CAAC,OAAO,CAAC,CAAC;QACtB,KAAK,QAAQ,CAAC,CAAC,OAAO,CAAC,CAAC;QACxB,KAAK,KAAK,CAAC,CAAC,OAAO,CAAC,CAAC;QACrB,OAAO,CAAC,CAAC,OAAO,CAAC,CAAC;IACpB,CAAC;AACH,CAAC;AAED;;GAEG;AACH,SAAgB,cAAc,CAAC,QAA2B;IACxD,OAAO,CAAC,GAAG,QAAQ,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,gBAAgB,CAAC,CAAC,CAAC,QAAQ,CAAC,GAAG,gBAAgB,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC;AACnG,CAAC;AAED,kBAAe;IACb,iBAAiB,EAAjB,yBAAiB;IACjB,QAAQ;IACR,SAAS;IACT,gBAAgB;IAChB,cAAc;CACf,CAAC"}
|
||||
139
npm/packages/ruvector/src/analysis/security.ts
Normal file
139
npm/packages/ruvector/src/analysis/security.ts
Normal file
@@ -0,0 +1,139 @@
|
||||
/**
|
||||
* Security Analysis Module - Consolidated security scanning
|
||||
*
|
||||
* Single source of truth for security patterns and vulnerability detection.
|
||||
* Used by native-worker.ts and parallel-workers.ts
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
|
||||
export interface SecurityPattern {
|
||||
pattern: RegExp;
|
||||
rule: string;
|
||||
severity: 'low' | 'medium' | 'high' | 'critical';
|
||||
message: string;
|
||||
suggestion?: string;
|
||||
}
|
||||
|
||||
export interface SecurityFinding {
|
||||
file: string;
|
||||
line: number;
|
||||
severity: 'low' | 'medium' | 'high' | 'critical';
|
||||
rule: string;
|
||||
message: string;
|
||||
match?: string;
|
||||
suggestion?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Default security patterns for vulnerability detection
|
||||
*/
|
||||
export const SECURITY_PATTERNS: SecurityPattern[] = [
|
||||
// Critical: Hardcoded secrets
|
||||
{ pattern: /password\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-password', severity: 'critical', message: 'Hardcoded password detected', suggestion: 'Use environment variables or secret management' },
|
||||
{ pattern: /api[_-]?key\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-apikey', severity: 'critical', message: 'Hardcoded API key detected', suggestion: 'Use environment variables' },
|
||||
{ pattern: /secret\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-secret', severity: 'critical', message: 'Hardcoded secret detected', suggestion: 'Use environment variables or secret management' },
|
||||
{ pattern: /private[_-]?key\s*=\s*['"][^'"]+['"]/gi, rule: 'no-hardcoded-private-key', severity: 'critical', message: 'Hardcoded private key detected', suggestion: 'Use secure key management' },
|
||||
|
||||
// High: Code execution risks
|
||||
{ pattern: /eval\s*\(/g, rule: 'no-eval', severity: 'high', message: 'Avoid eval() - code injection risk', suggestion: 'Use safer alternatives like JSON.parse()' },
|
||||
{ pattern: /exec\s*\(/g, rule: 'no-exec', severity: 'high', message: 'Avoid exec() - command injection risk', suggestion: 'Use execFile or spawn with args array' },
|
||||
{ pattern: /Function\s*\(/g, rule: 'no-function-constructor', severity: 'high', message: 'Avoid Function constructor - code injection risk' },
|
||||
{ pattern: /child_process.*exec\(/g, rule: 'no-shell-exec', severity: 'high', message: 'Shell execution detected', suggestion: 'Use execFile or spawn instead' },
|
||||
|
||||
// High: SQL injection
|
||||
{ pattern: /SELECT\s+.*\s+FROM.*\+/gi, rule: 'sql-injection-risk', severity: 'high', message: 'Potential SQL injection - string concatenation in query', suggestion: 'Use parameterized queries' },
|
||||
{ pattern: /`SELECT.*\$\{/gi, rule: 'sql-injection-template', severity: 'high', message: 'Template literal in SQL query', suggestion: 'Use parameterized queries' },
|
||||
|
||||
// Medium: XSS risks
|
||||
{ pattern: /dangerouslySetInnerHTML/g, rule: 'xss-risk', severity: 'medium', message: 'XSS risk: dangerouslySetInnerHTML', suggestion: 'Sanitize content before rendering' },
|
||||
{ pattern: /innerHTML\s*=/g, rule: 'no-inner-html', severity: 'medium', message: 'Avoid innerHTML - XSS risk', suggestion: 'Use textContent or sanitize content' },
|
||||
{ pattern: /document\.write\s*\(/g, rule: 'no-document-write', severity: 'medium', message: 'Avoid document.write - XSS risk' },
|
||||
|
||||
// Medium: Other risks
|
||||
{ pattern: /\$\{.*\}/g, rule: 'template-injection', severity: 'low', message: 'Template literal detected - verify no injection' },
|
||||
{ pattern: /new\s+RegExp\s*\([^)]*\+/g, rule: 'regex-injection', severity: 'medium', message: 'Dynamic RegExp - potential ReDoS risk', suggestion: 'Validate/sanitize regex input' },
|
||||
{ pattern: /\.on\s*\(\s*['"]error['"]/g, rule: 'unhandled-error', severity: 'low', message: 'Error handler detected - verify proper error handling' },
|
||||
];
|
||||
|
||||
/**
|
||||
* Scan a single file for security issues
|
||||
*/
|
||||
export function scanFile(
|
||||
filePath: string,
|
||||
content?: string,
|
||||
patterns: SecurityPattern[] = SECURITY_PATTERNS
|
||||
): SecurityFinding[] {
|
||||
const findings: SecurityFinding[] = [];
|
||||
|
||||
try {
|
||||
const fileContent = content ?? (fs.existsSync(filePath) ? fs.readFileSync(filePath, 'utf-8') : '');
|
||||
if (!fileContent) return findings;
|
||||
|
||||
for (const { pattern, rule, severity, message, suggestion } of patterns) {
|
||||
const regex = new RegExp(pattern.source, pattern.flags);
|
||||
let match;
|
||||
while ((match = regex.exec(fileContent)) !== null) {
|
||||
const lineNum = fileContent.slice(0, match.index).split('\n').length;
|
||||
findings.push({
|
||||
file: filePath,
|
||||
line: lineNum,
|
||||
severity,
|
||||
rule,
|
||||
message,
|
||||
match: match[0].slice(0, 50),
|
||||
suggestion,
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Skip unreadable files
|
||||
}
|
||||
|
||||
return findings;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan multiple files for security issues
|
||||
*/
|
||||
export function scanFiles(
|
||||
files: string[],
|
||||
patterns: SecurityPattern[] = SECURITY_PATTERNS,
|
||||
maxFiles: number = 100
|
||||
): SecurityFinding[] {
|
||||
const findings: SecurityFinding[] = [];
|
||||
|
||||
for (const file of files.slice(0, maxFiles)) {
|
||||
findings.push(...scanFile(file, undefined, patterns));
|
||||
}
|
||||
|
||||
return findings;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get severity score (for sorting/filtering)
|
||||
*/
|
||||
export function getSeverityScore(severity: string): number {
|
||||
switch (severity) {
|
||||
case 'critical': return 4;
|
||||
case 'high': return 3;
|
||||
case 'medium': return 2;
|
||||
case 'low': return 1;
|
||||
default: return 0;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Sort findings by severity (highest first)
|
||||
*/
|
||||
export function sortBySeverity(findings: SecurityFinding[]): SecurityFinding[] {
|
||||
return [...findings].sort((a, b) => getSeverityScore(b.severity) - getSeverityScore(a.severity));
|
||||
}
|
||||
|
||||
export default {
|
||||
SECURITY_PATTERNS,
|
||||
scanFile,
|
||||
scanFiles,
|
||||
getSeverityScore,
|
||||
sortBySeverity,
|
||||
};
|
||||
156
npm/packages/ruvector/src/core/adaptive-embedder.d.ts
vendored
Normal file
156
npm/packages/ruvector/src/core/adaptive-embedder.d.ts
vendored
Normal file
@@ -0,0 +1,156 @@
|
||||
/**
|
||||
* AdaptiveEmbedder - Micro-LoRA Style Optimization for ONNX Embeddings
|
||||
*
|
||||
* Applies continual learning techniques to frozen ONNX embeddings:
|
||||
*
|
||||
* 1. MICRO-LORA ADAPTERS
|
||||
* - Low-rank projection layers (rank 2-8) on top of frozen embeddings
|
||||
* - Domain-specific fine-tuning with minimal parameters
|
||||
* - ~0.1% of base model parameters
|
||||
*
|
||||
* 2. CONTRASTIVE LEARNING
|
||||
* - Files edited together → embeddings closer
|
||||
* - Semantic clustering from trajectories
|
||||
* - Online learning from user behavior
|
||||
*
|
||||
* 3. EWC++ (Elastic Weight Consolidation)
|
||||
* - Prevents catastrophic forgetting
|
||||
* - Consolidates important adaptations
|
||||
* - Fisher information regularization
|
||||
*
|
||||
* 4. MEMORY-AUGMENTED RETRIEVAL
|
||||
* - Episodic memory for context-aware embeddings
|
||||
* - Attention over past similar embeddings
|
||||
* - Domain prototype learning
|
||||
*
|
||||
* Architecture:
|
||||
* ONNX(text) → [frozen 384d] → LoRA_A → LoRA_B → [adapted 384d]
|
||||
* (384×r) (r×384)
|
||||
*/
|
||||
export interface AdaptiveConfig {
|
||||
/** LoRA rank (lower = fewer params, higher = more expressive) */
|
||||
loraRank?: number;
|
||||
/** Learning rate for online updates */
|
||||
learningRate?: number;
|
||||
/** EWC regularization strength */
|
||||
ewcLambda?: number;
|
||||
/** Number of domain prototypes to maintain */
|
||||
numPrototypes?: number;
|
||||
/** Enable contrastive learning from co-edits */
|
||||
contrastiveLearning?: boolean;
|
||||
/** Temperature for contrastive loss */
|
||||
contrastiveTemp?: number;
|
||||
/** Memory capacity for episodic retrieval */
|
||||
memoryCapacity?: number;
|
||||
}
|
||||
export interface LoRAWeights {
|
||||
A: number[][];
|
||||
B: number[][];
|
||||
bias?: number[];
|
||||
}
|
||||
export interface DomainPrototype {
|
||||
domain: string;
|
||||
centroid: number[];
|
||||
count: number;
|
||||
variance: number;
|
||||
}
|
||||
export interface AdaptiveStats {
|
||||
baseModel: string;
|
||||
dimension: number;
|
||||
loraRank: number;
|
||||
loraParams: number;
|
||||
adaptations: number;
|
||||
prototypes: number;
|
||||
memorySize: number;
|
||||
ewcConsolidations: number;
|
||||
contrastiveUpdates: number;
|
||||
}
|
||||
export declare class AdaptiveEmbedder {
|
||||
private config;
|
||||
private lora;
|
||||
private prototypes;
|
||||
private episodic;
|
||||
private onnxReady;
|
||||
private dimension;
|
||||
private adaptationCount;
|
||||
private ewcCount;
|
||||
private contrastiveCount;
|
||||
private coEditBuffer;
|
||||
constructor(config?: AdaptiveConfig);
|
||||
/**
|
||||
* Initialize ONNX backend
|
||||
*/
|
||||
init(): Promise<void>;
|
||||
/**
|
||||
* Generate adaptive embedding
|
||||
* Pipeline: ONNX → LoRA → Prototype Adjustment → Episodic Augmentation
|
||||
*/
|
||||
embed(text: string, options?: {
|
||||
domain?: string;
|
||||
useEpisodic?: boolean;
|
||||
storeInMemory?: boolean;
|
||||
}): Promise<number[]>;
|
||||
/**
|
||||
* Batch embed with adaptation
|
||||
*/
|
||||
embedBatch(texts: string[], options?: {
|
||||
domain?: string;
|
||||
}): Promise<number[][]>;
|
||||
/**
|
||||
* Learn from co-edit pattern (contrastive learning)
|
||||
* Files edited together should have similar embeddings
|
||||
*/
|
||||
learnCoEdit(file1: string, content1: string, file2: string, content2: string): Promise<number>;
|
||||
/**
|
||||
* Process co-edit batch with contrastive loss
|
||||
*/
|
||||
private processCoEditBatch;
|
||||
/**
|
||||
* Learn from trajectory outcome (reinforcement-like)
|
||||
*/
|
||||
learnFromOutcome(context: string, action: string, success: boolean, quality?: number): Promise<void>;
|
||||
/**
|
||||
* EWC consolidation - prevent forgetting important adaptations
|
||||
* OPTIMIZED: Works with Float32Array episodic entries
|
||||
*/
|
||||
consolidate(): Promise<void>;
|
||||
/**
|
||||
* Fallback hash embedding
|
||||
*/
|
||||
private hashEmbed;
|
||||
private normalize;
|
||||
/**
|
||||
* Get statistics
|
||||
*/
|
||||
getStats(): AdaptiveStats;
|
||||
/**
|
||||
* Export learned weights
|
||||
*/
|
||||
export(): {
|
||||
lora: LoRAWeights;
|
||||
prototypes: DomainPrototype[];
|
||||
stats: AdaptiveStats;
|
||||
};
|
||||
/**
|
||||
* Import learned weights
|
||||
*/
|
||||
import(data: {
|
||||
lora?: LoRAWeights;
|
||||
prototypes?: DomainPrototype[];
|
||||
}): void;
|
||||
/**
|
||||
* Reset adaptations
|
||||
*/
|
||||
reset(): void;
|
||||
/**
|
||||
* Get LoRA cache statistics
|
||||
*/
|
||||
getCacheStats(): {
|
||||
size: number;
|
||||
maxSize: number;
|
||||
};
|
||||
}
|
||||
export declare function getAdaptiveEmbedder(config?: AdaptiveConfig): AdaptiveEmbedder;
|
||||
export declare function initAdaptiveEmbedder(config?: AdaptiveConfig): Promise<AdaptiveEmbedder>;
|
||||
export default AdaptiveEmbedder;
|
||||
//# sourceMappingURL=adaptive-embedder.d.ts.map
|
||||
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"adaptive-embedder.d.ts","sourceRoot":"","sources":["adaptive-embedder.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA4BG;AAQH,MAAM,WAAW,cAAc;IAC7B,iEAAiE;IACjE,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,uCAAuC;IACvC,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,kCAAkC;IAClC,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,8CAA8C;IAC9C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,gDAAgD;IAChD,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B,uCAAuC;IACvC,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,6CAA6C;IAC7C,cAAc,CAAC,EAAE,MAAM,CAAC;CACzB;AAED,MAAM,WAAW,WAAW;IAC1B,CAAC,EAAE,MAAM,EAAE,EAAE,CAAC;IACd,CAAC,EAAE,MAAM,EAAE,EAAE,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC;CACjB;AAED,MAAM,WAAW,eAAe;IAC9B,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,EAAE,CAAC;IACnB,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,MAAM,WAAW,aAAa;IAC5B,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,UAAU,EAAE,MAAM,CAAC;IACnB,WAAW,EAAE,MAAM,CAAC;IACpB,UAAU,EAAE,MAAM,CAAC;IACnB,UAAU,EAAE,MAAM,CAAC;IACnB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,kBAAkB,EAAE,MAAM,CAAC;CAC5B;AA8pBD,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,MAAM,CAA2B;IACzC,OAAO,CAAC,IAAI,CAAY;IACxB,OAAO,CAAC,UAAU,CAAkB;IACpC,OAAO,CAAC,QAAQ,CAAiB;IACjC,OAAO,CAAC,SAAS,CAAkB;IACnC,OAAO,CAAC,SAAS,CAAe;IAGhC,OAAO,CAAC,eAAe,CAAa;IACpC,OAAO,CAAC,QAAQ,CAAa;IAC7B,OAAO,CAAC,gBAAgB,CAAa;IAGrC,OAAO,CAAC,YAAY,CAA+E;gBAEvF,MAAM,GAAE,cAAmB;IAiBvC;;OAEG;IACG,IAAI,IAAI,OAAO,CAAC,IAAI,CAAC;IAO3B;;;OAGG;IACG,KAAK,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE;QAClC,MAAM,CAAC,EAAE,MAAM,CAAC;QAChB,WAAW,CAAC,EAAE,OAAO,CAAC;QACtB,aAAa,CAAC,EAAE,OAAO,CAAC;KACzB,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IAmCrB;;OAEG;IACG,UAAU,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,OAAO,CAAC,EAAE;QAC1C,MAAM,CAAC,EAAE,MAAM,CAAC;KACjB,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC;IAsBvB;;;OAGG;IACG,WAAW,CAAC,KAAK,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;IAkBpG;;OAEG;IACH,OAAO,CAAC,kBAAkB;IA+B1B;;OAEG;IACG,gBAAgB,CACpB,OAAO,EAAE,MAAM,EACf,MAAM,EAAE,MAAM,EACd,OAAO,EAAE,OAAO,EAChB,OAAO,GAAE,MAAY,GACpB,OAAO,CAAC,IAAI,CAAC;IAiBhB;;;OAGG;IACG,WAAW,IAAI,OAAO,CAAC,IAAI,CAAC;IAmBlC;;OAEG;IACH,OAAO,CAAC,SAAS;IAoBjB,OAAO,CAAC,SAAS;IAKjB;;OAEG;IACH,QAAQ,IAAI,aAAa;IAczB;;OAEG;IACH,MAAM,IAAI;QACR,IAAI,EAAE,WAAW,CAAC;QAClB,UAAU,EAAE,eAAe,EAAE,CAAC;QAC9B,KAAK,EAAE,aAAa,CAAC;KACtB;IAQD;;OAEG;IACH,MAAM,CAAC,IAAI,EAAE;QAAE,IAAI,CAAC,EAAE,WAAW,CAAC;QAAC,UAAU,CAAC,EAAE,eAAe,EAAE,CAAA;KAAE,GAAG,IAAI;IAS1E;;OAEG;IACH,KAAK,IAAI,IAAI;IAUb;;OAEG;IACH,aAAa,IAAI;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE;CAGnD;AAQD,wBAAgB,mBAAmB,CAAC,MAAM,CAAC,EAAE,cAAc,GAAG,gBAAgB,CAK7E;AAED,wBAAsB,oBAAoB,CAAC,MAAM,CAAC,EAAE,cAAc,GAAG,OAAO,CAAC,gBAAgB,CAAC,CAI7F;AAED,eAAe,gBAAgB,CAAC"}
|
||||
838
npm/packages/ruvector/src/core/adaptive-embedder.js
Normal file
838
npm/packages/ruvector/src/core/adaptive-embedder.js
Normal file
@@ -0,0 +1,838 @@
|
||||
"use strict";
|
||||
/**
|
||||
* AdaptiveEmbedder - Micro-LoRA Style Optimization for ONNX Embeddings
|
||||
*
|
||||
* Applies continual learning techniques to frozen ONNX embeddings:
|
||||
*
|
||||
* 1. MICRO-LORA ADAPTERS
|
||||
* - Low-rank projection layers (rank 2-8) on top of frozen embeddings
|
||||
* - Domain-specific fine-tuning with minimal parameters
|
||||
* - ~0.1% of base model parameters
|
||||
*
|
||||
* 2. CONTRASTIVE LEARNING
|
||||
* - Files edited together → embeddings closer
|
||||
* - Semantic clustering from trajectories
|
||||
* - Online learning from user behavior
|
||||
*
|
||||
* 3. EWC++ (Elastic Weight Consolidation)
|
||||
* - Prevents catastrophic forgetting
|
||||
* - Consolidates important adaptations
|
||||
* - Fisher information regularization
|
||||
*
|
||||
* 4. MEMORY-AUGMENTED RETRIEVAL
|
||||
* - Episodic memory for context-aware embeddings
|
||||
* - Attention over past similar embeddings
|
||||
* - Domain prototype learning
|
||||
*
|
||||
* Architecture:
|
||||
* ONNX(text) → [frozen 384d] → LoRA_A → LoRA_B → [adapted 384d]
|
||||
* (384×r) (r×384)
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.AdaptiveEmbedder = void 0;
|
||||
exports.getAdaptiveEmbedder = getAdaptiveEmbedder;
|
||||
exports.initAdaptiveEmbedder = initAdaptiveEmbedder;
|
||||
const onnx_embedder_1 = require("./onnx-embedder");
|
||||
// ============================================================================
|
||||
// Optimized Micro-LoRA Layer with Float32Array and Caching
|
||||
// ============================================================================
|
||||
/**
|
||||
* Low-rank adaptation layer for embeddings (OPTIMIZED)
|
||||
* Implements: output = input + scale * (input @ A @ B)
|
||||
*
|
||||
* Optimizations:
|
||||
* - Float32Array for 2-3x faster math operations
|
||||
* - Flattened matrices for cache-friendly access
|
||||
* - Pre-allocated buffers to avoid GC pressure
|
||||
* - LRU embedding cache for repeated inputs
|
||||
*/
|
||||
class MicroLoRA {
|
||||
constructor(dim, rank, scale = 0.1) {
|
||||
// EWC Fisher information (importance weights)
|
||||
this.fisherA = null;
|
||||
this.fisherB = null;
|
||||
this.savedA = null;
|
||||
this.savedB = null;
|
||||
// LRU cache for repeated embeddings (key: hash, value: output)
|
||||
this.cache = new Map();
|
||||
this.cacheMaxSize = 256;
|
||||
this.dim = dim;
|
||||
this.rank = rank;
|
||||
this.scale = scale;
|
||||
// Initialize with small random values (Xavier-like)
|
||||
const stdA = Math.sqrt(2 / (dim + rank));
|
||||
const stdB = Math.sqrt(2 / (rank + dim)) * 0.01; // B starts near zero
|
||||
this.A = this.initFlatMatrix(dim, rank, stdA);
|
||||
this.B = this.initFlatMatrix(rank, dim, stdB);
|
||||
// Pre-allocate buffers
|
||||
this.hiddenBuffer = new Float32Array(rank);
|
||||
this.outputBuffer = new Float32Array(dim);
|
||||
}
|
||||
initFlatMatrix(rows, cols, std) {
|
||||
const arr = new Float32Array(rows * cols);
|
||||
for (let i = 0; i < arr.length; i++) {
|
||||
arr[i] = (Math.random() - 0.5) * 2 * std;
|
||||
}
|
||||
return arr;
|
||||
}
|
||||
/**
|
||||
* Fast hash for cache key (FNV-1a variant)
|
||||
*/
|
||||
hashInput(input) {
|
||||
let h = 2166136261;
|
||||
const len = Math.min(input.length, 32); // Sample first 32 for speed
|
||||
for (let i = 0; i < len; i++) {
|
||||
h ^= Math.floor(input[i] * 10000);
|
||||
h = Math.imul(h, 16777619);
|
||||
}
|
||||
return h.toString(36);
|
||||
}
|
||||
/**
|
||||
* Forward pass: input + scale * (input @ A @ B)
|
||||
* OPTIMIZED with Float32Array and loop unrolling
|
||||
*/
|
||||
forward(input) {
|
||||
// Check cache first
|
||||
const cacheKey = this.hashInput(input);
|
||||
const cached = this.cache.get(cacheKey);
|
||||
if (cached) {
|
||||
return Array.from(cached);
|
||||
}
|
||||
// Zero the hidden buffer
|
||||
this.hiddenBuffer.fill(0);
|
||||
// Compute input @ A (dim → rank) - SIMD-friendly loop
|
||||
// Unroll by 4 for better pipelining
|
||||
const dim4 = this.dim - (this.dim % 4);
|
||||
for (let r = 0; r < this.rank; r++) {
|
||||
let sum = 0;
|
||||
const rOffset = r;
|
||||
// Unrolled loop
|
||||
for (let d = 0; d < dim4; d += 4) {
|
||||
const aIdx = d * this.rank + rOffset;
|
||||
sum += input[d] * this.A[aIdx];
|
||||
sum += input[d + 1] * this.A[aIdx + this.rank];
|
||||
sum += input[d + 2] * this.A[aIdx + 2 * this.rank];
|
||||
sum += input[d + 3] * this.A[aIdx + 3 * this.rank];
|
||||
}
|
||||
// Remainder
|
||||
for (let d = dim4; d < this.dim; d++) {
|
||||
sum += input[d] * this.A[d * this.rank + rOffset];
|
||||
}
|
||||
this.hiddenBuffer[r] = sum;
|
||||
}
|
||||
// Compute hidden @ B (rank → dim) and add residual
|
||||
// Copy input to output buffer first
|
||||
for (let d = 0; d < this.dim; d++) {
|
||||
this.outputBuffer[d] = input[d];
|
||||
}
|
||||
// Add scaled LoRA contribution
|
||||
for (let d = 0; d < this.dim; d++) {
|
||||
let delta = 0;
|
||||
for (let r = 0; r < this.rank; r++) {
|
||||
delta += this.hiddenBuffer[r] * this.B[r * this.dim + d];
|
||||
}
|
||||
this.outputBuffer[d] += this.scale * delta;
|
||||
}
|
||||
// Cache result (LRU eviction if full)
|
||||
if (this.cache.size >= this.cacheMaxSize) {
|
||||
const firstKey = this.cache.keys().next().value;
|
||||
if (firstKey)
|
||||
this.cache.delete(firstKey);
|
||||
}
|
||||
this.cache.set(cacheKey, new Float32Array(this.outputBuffer));
|
||||
return Array.from(this.outputBuffer);
|
||||
}
|
||||
/**
|
||||
* Clear cache (call after weight updates)
|
||||
*/
|
||||
clearCache() {
|
||||
this.cache.clear();
|
||||
}
|
||||
/**
|
||||
* Backward pass with contrastive loss
|
||||
* Pulls positive pairs closer, pushes negatives apart
|
||||
* OPTIMIZED: Uses Float32Array buffers
|
||||
*/
|
||||
backward(anchor, positive, negatives, lr, ewcLambda = 0) {
|
||||
if (!positive && negatives.length === 0)
|
||||
return 0;
|
||||
// Clear cache since weights will change
|
||||
this.clearCache();
|
||||
// Compute adapted embeddings
|
||||
const anchorOut = this.forward(anchor);
|
||||
const positiveOut = positive ? this.forward(positive) : null;
|
||||
const negativeOuts = negatives.map(n => this.forward(n));
|
||||
// Contrastive loss with temperature scaling
|
||||
const temp = 0.07;
|
||||
let loss = 0;
|
||||
if (positiveOut) {
|
||||
// Positive similarity
|
||||
const posSim = this.cosineSimilarity(anchorOut, positiveOut) / temp;
|
||||
// Negative similarities
|
||||
const negSims = negativeOuts.map(n => this.cosineSimilarity(anchorOut, n) / temp);
|
||||
// InfoNCE loss
|
||||
const maxSim = Math.max(posSim, ...negSims);
|
||||
const expPos = Math.exp(posSim - maxSim);
|
||||
const expNegs = negSims.reduce((sum, s) => sum + Math.exp(s - maxSim), 0);
|
||||
loss = -Math.log(expPos / (expPos + expNegs) + 1e-8);
|
||||
// Compute gradients (simplified)
|
||||
const gradScale = lr * this.scale;
|
||||
// Update A based on gradient direction (flattened access)
|
||||
for (let d = 0; d < this.dim; d++) {
|
||||
for (let r = 0; r < this.rank; r++) {
|
||||
const idx = d * this.rank + r;
|
||||
// Gradient from positive (pull closer)
|
||||
const pOutR = r < positiveOut.length ? positiveOut[r] : 0;
|
||||
const aOutR = r < anchorOut.length ? anchorOut[r] : 0;
|
||||
const gradA = anchor[d] * (pOutR - aOutR) * gradScale;
|
||||
this.A[idx] += gradA;
|
||||
// EWC regularization
|
||||
if (ewcLambda > 0 && this.fisherA && this.savedA) {
|
||||
this.A[idx] -= ewcLambda * this.fisherA[idx] * (this.A[idx] - this.savedA[idx]);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Update B (flattened access)
|
||||
for (let r = 0; r < this.rank; r++) {
|
||||
const anchorR = r < anchor.length ? anchor[r] : 0;
|
||||
for (let d = 0; d < this.dim; d++) {
|
||||
const idx = r * this.dim + d;
|
||||
const gradB = anchorR * (positiveOut[d] - anchorOut[d]) * gradScale * 0.1;
|
||||
this.B[idx] += gradB;
|
||||
if (ewcLambda > 0 && this.fisherB && this.savedB) {
|
||||
this.B[idx] -= ewcLambda * this.fisherB[idx] * (this.B[idx] - this.savedB[idx]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return loss;
|
||||
}
|
||||
/**
|
||||
* EWC consolidation - save current weights and compute Fisher information
|
||||
* OPTIMIZED: Uses Float32Array
|
||||
*/
|
||||
consolidate(embeddings) {
|
||||
// Save current weights
|
||||
this.savedA = new Float32Array(this.A);
|
||||
this.savedB = new Float32Array(this.B);
|
||||
// Estimate Fisher information (diagonal approximation)
|
||||
this.fisherA = new Float32Array(this.dim * this.rank);
|
||||
this.fisherB = new Float32Array(this.rank * this.dim);
|
||||
const numEmb = embeddings.length;
|
||||
for (const emb of embeddings) {
|
||||
// Accumulate squared gradients as Fisher estimate
|
||||
for (let d = 0; d < this.dim; d++) {
|
||||
const embD = emb[d] * emb[d] / numEmb;
|
||||
for (let r = 0; r < this.rank; r++) {
|
||||
this.fisherA[d * this.rank + r] += embD;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Clear cache after consolidation
|
||||
this.clearCache();
|
||||
}
|
||||
/**
|
||||
* Optimized cosine similarity with early termination
|
||||
*/
|
||||
cosineSimilarity(a, b) {
|
||||
let dot = 0, normA = 0, normB = 0;
|
||||
const len = Math.min(a.length, b.length);
|
||||
// Unrolled loop for speed
|
||||
const len4 = len - (len % 4);
|
||||
for (let i = 0; i < len4; i += 4) {
|
||||
dot += a[i] * b[i] + a[i + 1] * b[i + 1] + a[i + 2] * b[i + 2] + a[i + 3] * b[i + 3];
|
||||
normA += a[i] * a[i] + a[i + 1] * a[i + 1] + a[i + 2] * a[i + 2] + a[i + 3] * a[i + 3];
|
||||
normB += b[i] * b[i] + b[i + 1] * b[i + 1] + b[i + 2] * b[i + 2] + b[i + 3] * b[i + 3];
|
||||
}
|
||||
// Remainder
|
||||
for (let i = len4; i < len; i++) {
|
||||
dot += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
return dot / (Math.sqrt(normA * normB) + 1e-8);
|
||||
}
|
||||
getParams() {
|
||||
return this.dim * this.rank + this.rank * this.dim;
|
||||
}
|
||||
getCacheStats() {
|
||||
return {
|
||||
size: this.cache.size,
|
||||
maxSize: this.cacheMaxSize,
|
||||
hitRate: 0, // Would need hit counter for accurate tracking
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Export weights as 2D arrays for serialization
|
||||
*/
|
||||
export() {
|
||||
// Convert flattened Float32Array back to 2D number[][]
|
||||
const A = [];
|
||||
for (let d = 0; d < this.dim; d++) {
|
||||
const row = [];
|
||||
for (let r = 0; r < this.rank; r++) {
|
||||
row.push(this.A[d * this.rank + r]);
|
||||
}
|
||||
A.push(row);
|
||||
}
|
||||
const B = [];
|
||||
for (let r = 0; r < this.rank; r++) {
|
||||
const row = [];
|
||||
for (let d = 0; d < this.dim; d++) {
|
||||
row.push(this.B[r * this.dim + d]);
|
||||
}
|
||||
B.push(row);
|
||||
}
|
||||
return { A, B };
|
||||
}
|
||||
/**
|
||||
* Import weights from 2D arrays
|
||||
*/
|
||||
import(weights) {
|
||||
// Convert 2D number[][] to flattened Float32Array
|
||||
for (let d = 0; d < this.dim && d < weights.A.length; d++) {
|
||||
for (let r = 0; r < this.rank && r < weights.A[d].length; r++) {
|
||||
this.A[d * this.rank + r] = weights.A[d][r];
|
||||
}
|
||||
}
|
||||
for (let r = 0; r < this.rank && r < weights.B.length; r++) {
|
||||
for (let d = 0; d < this.dim && d < weights.B[r].length; d++) {
|
||||
this.B[r * this.dim + d] = weights.B[r][d];
|
||||
}
|
||||
}
|
||||
// Clear cache after import
|
||||
this.clearCache();
|
||||
}
|
||||
}
|
||||
// ============================================================================
|
||||
// Domain Prototype Learning (OPTIMIZED with Float32Array)
|
||||
// ============================================================================
|
||||
class PrototypeMemory {
|
||||
constructor(maxPrototypes = 50, dimension = 384) {
|
||||
this.prototypes = new Map();
|
||||
this.maxPrototypes = maxPrototypes;
|
||||
this.scratchBuffer = new Float32Array(dimension);
|
||||
}
|
||||
/**
|
||||
* Update prototype with new embedding (online mean update)
|
||||
* OPTIMIZED: Uses Float32Array internally
|
||||
*/
|
||||
update(domain, embedding) {
|
||||
const existing = this.prototypes.get(domain);
|
||||
if (existing) {
|
||||
// Online mean update: new_mean = old_mean + (x - old_mean) / n
|
||||
const n = existing.count + 1;
|
||||
const invN = 1 / n;
|
||||
// Unrolled update loop
|
||||
const len = Math.min(embedding.length, existing.centroid.length);
|
||||
const len4 = len - (len % 4);
|
||||
for (let i = 0; i < len4; i += 4) {
|
||||
const d0 = embedding[i] - existing.centroid[i];
|
||||
const d1 = embedding[i + 1] - existing.centroid[i + 1];
|
||||
const d2 = embedding[i + 2] - existing.centroid[i + 2];
|
||||
const d3 = embedding[i + 3] - existing.centroid[i + 3];
|
||||
existing.centroid[i] += d0 * invN;
|
||||
existing.centroid[i + 1] += d1 * invN;
|
||||
existing.centroid[i + 2] += d2 * invN;
|
||||
existing.centroid[i + 3] += d3 * invN;
|
||||
existing.variance += d0 * (embedding[i] - existing.centroid[i]);
|
||||
existing.variance += d1 * (embedding[i + 1] - existing.centroid[i + 1]);
|
||||
existing.variance += d2 * (embedding[i + 2] - existing.centroid[i + 2]);
|
||||
existing.variance += d3 * (embedding[i + 3] - existing.centroid[i + 3]);
|
||||
}
|
||||
for (let i = len4; i < len; i++) {
|
||||
const delta = embedding[i] - existing.centroid[i];
|
||||
existing.centroid[i] += delta * invN;
|
||||
existing.variance += delta * (embedding[i] - existing.centroid[i]);
|
||||
}
|
||||
existing.count = n;
|
||||
}
|
||||
else {
|
||||
// Create new prototype
|
||||
if (this.prototypes.size >= this.maxPrototypes) {
|
||||
// Remove least used prototype
|
||||
let minCount = Infinity;
|
||||
let minKey = '';
|
||||
for (const [key, proto] of this.prototypes) {
|
||||
if (proto.count < minCount) {
|
||||
minCount = proto.count;
|
||||
minKey = key;
|
||||
}
|
||||
}
|
||||
this.prototypes.delete(minKey);
|
||||
}
|
||||
this.prototypes.set(domain, {
|
||||
domain,
|
||||
centroid: Array.from(embedding),
|
||||
count: 1,
|
||||
variance: 0,
|
||||
});
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Find closest prototype and return domain-adjusted embedding
|
||||
* OPTIMIZED: Single-pass similarity with early exit
|
||||
*/
|
||||
adjust(embedding) {
|
||||
if (this.prototypes.size === 0) {
|
||||
return { adjusted: Array.from(embedding), domain: null, confidence: 0 };
|
||||
}
|
||||
let bestSim = -Infinity;
|
||||
let bestProto = null;
|
||||
for (const proto of this.prototypes.values()) {
|
||||
const sim = this.cosineSimilarityFast(embedding, proto.centroid);
|
||||
if (sim > bestSim) {
|
||||
bestSim = sim;
|
||||
bestProto = proto;
|
||||
}
|
||||
}
|
||||
if (!bestProto || bestSim < 0.5) {
|
||||
return { adjusted: Array.from(embedding), domain: null, confidence: 0 };
|
||||
}
|
||||
// Adjust embedding toward prototype (soft assignment)
|
||||
const alpha = 0.1 * bestSim;
|
||||
const oneMinusAlpha = 1 - alpha;
|
||||
const adjusted = new Array(embedding.length);
|
||||
// Unrolled adjustment
|
||||
const len = embedding.length;
|
||||
const len4 = len - (len % 4);
|
||||
for (let i = 0; i < len4; i += 4) {
|
||||
adjusted[i] = embedding[i] * oneMinusAlpha + bestProto.centroid[i] * alpha;
|
||||
adjusted[i + 1] = embedding[i + 1] * oneMinusAlpha + bestProto.centroid[i + 1] * alpha;
|
||||
adjusted[i + 2] = embedding[i + 2] * oneMinusAlpha + bestProto.centroid[i + 2] * alpha;
|
||||
adjusted[i + 3] = embedding[i + 3] * oneMinusAlpha + bestProto.centroid[i + 3] * alpha;
|
||||
}
|
||||
for (let i = len4; i < len; i++) {
|
||||
adjusted[i] = embedding[i] * oneMinusAlpha + bestProto.centroid[i] * alpha;
|
||||
}
|
||||
return {
|
||||
adjusted,
|
||||
domain: bestProto.domain,
|
||||
confidence: bestSim,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Fast cosine similarity with loop unrolling
|
||||
*/
|
||||
cosineSimilarityFast(a, b) {
|
||||
let dot = 0, normA = 0, normB = 0;
|
||||
const len = Math.min(a.length, b.length);
|
||||
const len4 = len - (len % 4);
|
||||
for (let i = 0; i < len4; i += 4) {
|
||||
dot += a[i] * b[i] + a[i + 1] * b[i + 1] + a[i + 2] * b[i + 2] + a[i + 3] * b[i + 3];
|
||||
normA += a[i] * a[i] + a[i + 1] * a[i + 1] + a[i + 2] * a[i + 2] + a[i + 3] * a[i + 3];
|
||||
normB += b[i] * b[i] + b[i + 1] * b[i + 1] + b[i + 2] * b[i + 2] + b[i + 3] * b[i + 3];
|
||||
}
|
||||
for (let i = len4; i < len; i++) {
|
||||
dot += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
return dot / (Math.sqrt(normA * normB) + 1e-8);
|
||||
}
|
||||
getPrototypes() {
|
||||
return Array.from(this.prototypes.values());
|
||||
}
|
||||
export() {
|
||||
return this.getPrototypes();
|
||||
}
|
||||
import(prototypes) {
|
||||
this.prototypes.clear();
|
||||
for (const p of prototypes) {
|
||||
this.prototypes.set(p.domain, p);
|
||||
}
|
||||
}
|
||||
}
|
||||
class EpisodicMemory {
|
||||
constructor(capacity = 1000, dimension = 384) {
|
||||
this.entries = [];
|
||||
this.capacity = capacity;
|
||||
this.dimension = dimension;
|
||||
this.augmentBuffer = new Float32Array(dimension);
|
||||
this.weightsBuffer = new Float32Array(Math.min(capacity, 16)); // Max k
|
||||
}
|
||||
add(embedding, context) {
|
||||
if (this.entries.length >= this.capacity) {
|
||||
// Find and remove least used entry (O(n) but infrequent)
|
||||
let minIdx = 0;
|
||||
let minCount = this.entries[0].useCount;
|
||||
for (let i = 1; i < this.entries.length; i++) {
|
||||
if (this.entries[i].useCount < minCount) {
|
||||
minCount = this.entries[i].useCount;
|
||||
minIdx = i;
|
||||
}
|
||||
}
|
||||
this.entries.splice(minIdx, 1);
|
||||
}
|
||||
// Convert to Float32Array and pre-compute norm
|
||||
const emb = embedding instanceof Float32Array
|
||||
? new Float32Array(embedding)
|
||||
: new Float32Array(embedding);
|
||||
let normSq = 0;
|
||||
for (let i = 0; i < emb.length; i++) {
|
||||
normSq += emb[i] * emb[i];
|
||||
}
|
||||
this.entries.push({
|
||||
embedding: emb,
|
||||
context,
|
||||
timestamp: Date.now(),
|
||||
useCount: 0,
|
||||
normSquared: normSq,
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Retrieve similar past embeddings for context augmentation
|
||||
* OPTIMIZED: Uses pre-computed norms for fast similarity
|
||||
*/
|
||||
retrieve(query, k = 5) {
|
||||
if (this.entries.length === 0)
|
||||
return [];
|
||||
// Pre-compute query norm
|
||||
let queryNormSq = 0;
|
||||
for (let i = 0; i < query.length; i++) {
|
||||
queryNormSq += query[i] * query[i];
|
||||
}
|
||||
const queryNorm = Math.sqrt(queryNormSq);
|
||||
// Score all entries
|
||||
const scored = [];
|
||||
for (const entry of this.entries) {
|
||||
// Fast dot product with loop unrolling
|
||||
let dot = 0;
|
||||
const len = Math.min(query.length, entry.embedding.length);
|
||||
const len4 = len - (len % 4);
|
||||
for (let i = 0; i < len4; i += 4) {
|
||||
dot += query[i] * entry.embedding[i];
|
||||
dot += query[i + 1] * entry.embedding[i + 1];
|
||||
dot += query[i + 2] * entry.embedding[i + 2];
|
||||
dot += query[i + 3] * entry.embedding[i + 3];
|
||||
}
|
||||
for (let i = len4; i < len; i++) {
|
||||
dot += query[i] * entry.embedding[i];
|
||||
}
|
||||
const similarity = dot / (queryNorm * Math.sqrt(entry.normSquared) + 1e-8);
|
||||
scored.push({ entry, similarity });
|
||||
}
|
||||
// Partial sort for top-k (faster than full sort for large arrays)
|
||||
if (scored.length <= k) {
|
||||
scored.sort((a, b) => b.similarity - a.similarity);
|
||||
for (const s of scored)
|
||||
s.entry.useCount++;
|
||||
return scored.map(s => s.entry);
|
||||
}
|
||||
// Quick select for top-k
|
||||
scored.sort((a, b) => b.similarity - a.similarity);
|
||||
const topK = scored.slice(0, k);
|
||||
for (const s of topK)
|
||||
s.entry.useCount++;
|
||||
return topK.map(s => s.entry);
|
||||
}
|
||||
/**
|
||||
* Augment embedding with episodic memory (attention-like)
|
||||
* OPTIMIZED: Uses pre-allocated buffers
|
||||
*/
|
||||
augment(embedding, k = 3) {
|
||||
const similar = this.retrieve(embedding, k);
|
||||
if (similar.length === 0)
|
||||
return Array.from(embedding);
|
||||
// Pre-compute query norm
|
||||
let queryNormSq = 0;
|
||||
for (let i = 0; i < embedding.length; i++) {
|
||||
queryNormSq += embedding[i] * embedding[i];
|
||||
}
|
||||
const queryNorm = Math.sqrt(queryNormSq);
|
||||
// Compute weights
|
||||
let sumWeights = 1; // Start with 1 for query
|
||||
for (let j = 0; j < similar.length; j++) {
|
||||
// Fast dot product for similarity
|
||||
let dot = 0;
|
||||
const emb = similar[j].embedding;
|
||||
const len = Math.min(embedding.length, emb.length);
|
||||
for (let i = 0; i < len; i++) {
|
||||
dot += embedding[i] * emb[i];
|
||||
}
|
||||
const sim = dot / (queryNorm * Math.sqrt(similar[j].normSquared) + 1e-8);
|
||||
const weight = Math.exp(sim / 0.1);
|
||||
this.weightsBuffer[j] = weight;
|
||||
sumWeights += weight;
|
||||
}
|
||||
const invSumWeights = 1 / sumWeights;
|
||||
// Weighted average
|
||||
const dim = embedding.length;
|
||||
for (let i = 0; i < dim; i++) {
|
||||
let sum = embedding[i]; // Query contribution
|
||||
for (let j = 0; j < similar.length; j++) {
|
||||
sum += this.weightsBuffer[j] * similar[j].embedding[i];
|
||||
}
|
||||
this.augmentBuffer[i] = sum * invSumWeights;
|
||||
}
|
||||
return Array.from(this.augmentBuffer.subarray(0, dim));
|
||||
}
|
||||
size() {
|
||||
return this.entries.length;
|
||||
}
|
||||
clear() {
|
||||
this.entries = [];
|
||||
}
|
||||
}
|
||||
// ============================================================================
|
||||
// Adaptive Embedder (Main Class)
|
||||
// ============================================================================
|
||||
class AdaptiveEmbedder {
|
||||
constructor(config = {}) {
|
||||
this.onnxReady = false;
|
||||
this.dimension = 384;
|
||||
// Stats
|
||||
this.adaptationCount = 0;
|
||||
this.ewcCount = 0;
|
||||
this.contrastiveCount = 0;
|
||||
// Co-edit buffer for contrastive learning
|
||||
this.coEditBuffer = [];
|
||||
this.config = {
|
||||
loraRank: config.loraRank ?? 4,
|
||||
learningRate: config.learningRate ?? 0.01,
|
||||
ewcLambda: config.ewcLambda ?? 0.1,
|
||||
numPrototypes: config.numPrototypes ?? 50,
|
||||
contrastiveLearning: config.contrastiveLearning ?? true,
|
||||
contrastiveTemp: config.contrastiveTemp ?? 0.07,
|
||||
memoryCapacity: config.memoryCapacity ?? 1000,
|
||||
};
|
||||
// Pass dimension for pre-allocation of Float32Array buffers
|
||||
this.lora = new MicroLoRA(this.dimension, this.config.loraRank);
|
||||
this.prototypes = new PrototypeMemory(this.config.numPrototypes, this.dimension);
|
||||
this.episodic = new EpisodicMemory(this.config.memoryCapacity, this.dimension);
|
||||
}
|
||||
/**
|
||||
* Initialize ONNX backend
|
||||
*/
|
||||
async init() {
|
||||
if ((0, onnx_embedder_1.isOnnxAvailable)()) {
|
||||
await (0, onnx_embedder_1.initOnnxEmbedder)();
|
||||
this.onnxReady = true;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Generate adaptive embedding
|
||||
* Pipeline: ONNX → LoRA → Prototype Adjustment → Episodic Augmentation
|
||||
*/
|
||||
async embed(text, options) {
|
||||
// Step 1: Get base ONNX embedding
|
||||
let baseEmb;
|
||||
if (this.onnxReady) {
|
||||
const result = await (0, onnx_embedder_1.embed)(text);
|
||||
baseEmb = result.embedding;
|
||||
}
|
||||
else {
|
||||
// Fallback to hash embedding
|
||||
baseEmb = this.hashEmbed(text);
|
||||
}
|
||||
// Step 2: Apply LoRA adaptation
|
||||
let adapted = this.lora.forward(baseEmb);
|
||||
// Step 3: Prototype adjustment (if domain specified)
|
||||
if (options?.domain) {
|
||||
this.prototypes.update(options.domain, adapted);
|
||||
}
|
||||
const { adjusted, domain } = this.prototypes.adjust(adapted);
|
||||
adapted = adjusted;
|
||||
// Step 4: Episodic memory augmentation
|
||||
if (options?.useEpisodic !== false) {
|
||||
adapted = this.episodic.augment(adapted);
|
||||
}
|
||||
// Step 5: Store in episodic memory
|
||||
if (options?.storeInMemory !== false) {
|
||||
this.episodic.add(adapted, text.slice(0, 100));
|
||||
}
|
||||
// Normalize
|
||||
return this.normalize(adapted);
|
||||
}
|
||||
/**
|
||||
* Batch embed with adaptation
|
||||
*/
|
||||
async embedBatch(texts, options) {
|
||||
const results = [];
|
||||
if (this.onnxReady) {
|
||||
const baseResults = await (0, onnx_embedder_1.embedBatch)(texts);
|
||||
for (let i = 0; i < baseResults.length; i++) {
|
||||
let adapted = this.lora.forward(baseResults[i].embedding);
|
||||
if (options?.domain) {
|
||||
this.prototypes.update(options.domain, adapted);
|
||||
}
|
||||
const { adjusted } = this.prototypes.adjust(adapted);
|
||||
results.push(this.normalize(adjusted));
|
||||
}
|
||||
}
|
||||
else {
|
||||
for (const text of texts) {
|
||||
results.push(await this.embed(text, options));
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
/**
|
||||
* Learn from co-edit pattern (contrastive learning)
|
||||
* Files edited together should have similar embeddings
|
||||
*/
|
||||
async learnCoEdit(file1, content1, file2, content2) {
|
||||
if (!this.config.contrastiveLearning)
|
||||
return 0;
|
||||
// Get embeddings
|
||||
const emb1 = await this.embed(content1.slice(0, 512), { storeInMemory: false });
|
||||
const emb2 = await this.embed(content2.slice(0, 512), { storeInMemory: false });
|
||||
// Store in buffer for batch learning
|
||||
this.coEditBuffer.push({ file1, emb1, file2, emb2 });
|
||||
// Process batch when buffer is full
|
||||
if (this.coEditBuffer.length >= 16) {
|
||||
return this.processCoEditBatch();
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
/**
|
||||
* Process co-edit batch with contrastive loss
|
||||
*/
|
||||
processCoEditBatch() {
|
||||
if (this.coEditBuffer.length < 2)
|
||||
return 0;
|
||||
let totalLoss = 0;
|
||||
for (const { emb1, emb2 } of this.coEditBuffer) {
|
||||
// Use other pairs as negatives
|
||||
const negatives = this.coEditBuffer
|
||||
.filter(p => p.emb1 !== emb1)
|
||||
.slice(0, 4)
|
||||
.map(p => p.emb1);
|
||||
// Backward pass with contrastive loss
|
||||
const loss = this.lora.backward(emb1, emb2, negatives, this.config.learningRate, this.config.ewcLambda);
|
||||
totalLoss += loss;
|
||||
this.contrastiveCount++;
|
||||
}
|
||||
this.coEditBuffer = [];
|
||||
this.adaptationCount++;
|
||||
return totalLoss / this.coEditBuffer.length;
|
||||
}
|
||||
/**
|
||||
* Learn from trajectory outcome (reinforcement-like)
|
||||
*/
|
||||
async learnFromOutcome(context, action, success, quality = 0.5) {
|
||||
const contextEmb = await this.embed(context, { storeInMemory: false });
|
||||
const actionEmb = await this.embed(action, { storeInMemory: false });
|
||||
if (success && quality > 0.7) {
|
||||
// Positive outcome - pull embeddings closer
|
||||
this.lora.backward(contextEmb, actionEmb, [], this.config.learningRate * quality, this.config.ewcLambda);
|
||||
this.adaptationCount++;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* EWC consolidation - prevent forgetting important adaptations
|
||||
* OPTIMIZED: Works with Float32Array episodic entries
|
||||
*/
|
||||
async consolidate() {
|
||||
// Collect current episodic memories for Fisher estimation
|
||||
const embeddings = [];
|
||||
const entries = this.episodic.entries || [];
|
||||
// Get last 100 entries for Fisher estimation
|
||||
const recentEntries = entries.slice(-100);
|
||||
for (const entry of recentEntries) {
|
||||
if (entry.embedding instanceof Float32Array) {
|
||||
embeddings.push(entry.embedding);
|
||||
}
|
||||
}
|
||||
if (embeddings.length > 10) {
|
||||
this.lora.consolidate(embeddings);
|
||||
this.ewcCount++;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Fallback hash embedding
|
||||
*/
|
||||
hashEmbed(text) {
|
||||
const embedding = new Array(this.dimension).fill(0);
|
||||
const tokens = text.toLowerCase().split(/\s+/);
|
||||
for (let t = 0; t < tokens.length; t++) {
|
||||
const token = tokens[t];
|
||||
const posWeight = 1 / (1 + t * 0.1);
|
||||
for (let i = 0; i < token.length; i++) {
|
||||
const code = token.charCodeAt(i);
|
||||
const h1 = (code * 31 + i * 17 + t * 7) % this.dimension;
|
||||
const h2 = (code * 37 + i * 23 + t * 11) % this.dimension;
|
||||
embedding[h1] += posWeight;
|
||||
embedding[h2] += posWeight * 0.5;
|
||||
}
|
||||
}
|
||||
return this.normalize(embedding);
|
||||
}
|
||||
normalize(v) {
|
||||
const norm = Math.sqrt(v.reduce((a, b) => a + b * b, 0));
|
||||
return norm > 0 ? v.map(x => x / norm) : v;
|
||||
}
|
||||
/**
|
||||
* Get statistics
|
||||
*/
|
||||
getStats() {
|
||||
return {
|
||||
baseModel: 'all-MiniLM-L6-v2',
|
||||
dimension: this.dimension,
|
||||
loraRank: this.config.loraRank,
|
||||
loraParams: this.lora.getParams(),
|
||||
adaptations: this.adaptationCount,
|
||||
prototypes: this.prototypes.getPrototypes().length,
|
||||
memorySize: this.episodic.size(),
|
||||
ewcConsolidations: this.ewcCount,
|
||||
contrastiveUpdates: this.contrastiveCount,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Export learned weights
|
||||
*/
|
||||
export() {
|
||||
return {
|
||||
lora: this.lora.export(),
|
||||
prototypes: this.prototypes.export(),
|
||||
stats: this.getStats(),
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Import learned weights
|
||||
*/
|
||||
import(data) {
|
||||
if (data.lora) {
|
||||
this.lora.import(data.lora);
|
||||
}
|
||||
if (data.prototypes) {
|
||||
this.prototypes.import(data.prototypes);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Reset adaptations
|
||||
*/
|
||||
reset() {
|
||||
this.lora = new MicroLoRA(this.dimension, this.config.loraRank);
|
||||
this.prototypes = new PrototypeMemory(this.config.numPrototypes, this.dimension);
|
||||
this.episodic.clear();
|
||||
this.adaptationCount = 0;
|
||||
this.ewcCount = 0;
|
||||
this.contrastiveCount = 0;
|
||||
this.coEditBuffer = [];
|
||||
}
|
||||
/**
|
||||
* Get LoRA cache statistics
|
||||
*/
|
||||
getCacheStats() {
|
||||
return this.lora.getCacheStats?.() ?? { size: 0, maxSize: 256 };
|
||||
}
|
||||
}
|
||||
exports.AdaptiveEmbedder = AdaptiveEmbedder;
|
||||
// ============================================================================
|
||||
// Factory & Singleton
|
||||
// ============================================================================
|
||||
let instance = null;
|
||||
function getAdaptiveEmbedder(config) {
|
||||
if (!instance) {
|
||||
instance = new AdaptiveEmbedder(config);
|
||||
}
|
||||
return instance;
|
||||
}
|
||||
async function initAdaptiveEmbedder(config) {
|
||||
const embedder = getAdaptiveEmbedder(config);
|
||||
await embedder.init();
|
||||
return embedder;
|
||||
}
|
||||
exports.default = AdaptiveEmbedder;
|
||||
//# sourceMappingURL=adaptive-embedder.js.map
|
||||
1
npm/packages/ruvector/src/core/adaptive-embedder.js.map
Normal file
1
npm/packages/ruvector/src/core/adaptive-embedder.js.map
Normal file
File diff suppressed because one or more lines are too long
1076
npm/packages/ruvector/src/core/adaptive-embedder.ts
Normal file
1076
npm/packages/ruvector/src/core/adaptive-embedder.ts
Normal file
File diff suppressed because it is too large
Load Diff
149
npm/packages/ruvector/src/core/agentdb-fast.d.ts
vendored
Normal file
149
npm/packages/ruvector/src/core/agentdb-fast.d.ts
vendored
Normal file
@@ -0,0 +1,149 @@
|
||||
/**
|
||||
* AgentDB Fast - High-performance in-process alternative to AgentDB CLI
|
||||
*
|
||||
* The AgentDB CLI has ~2.3s startup overhead due to npx initialization.
|
||||
* This module provides 50-200x faster operations by using in-process calls.
|
||||
*
|
||||
* Features:
|
||||
* - In-memory episode storage with LRU eviction
|
||||
* - Vector similarity search using @ruvector/core
|
||||
* - Compatible API with AgentDB's episode/trajectory interfaces
|
||||
*/
|
||||
/**
|
||||
* Episode entry for trajectory storage
|
||||
*/
|
||||
export interface Episode {
|
||||
id: string;
|
||||
state: number[];
|
||||
action: string | number;
|
||||
reward: number;
|
||||
nextState: number[];
|
||||
done: boolean;
|
||||
metadata?: Record<string, any>;
|
||||
timestamp?: number;
|
||||
}
|
||||
/**
|
||||
* Trajectory (sequence of episodes)
|
||||
*/
|
||||
export interface Trajectory {
|
||||
id: string;
|
||||
episodes: Episode[];
|
||||
totalReward: number;
|
||||
metadata?: Record<string, any>;
|
||||
}
|
||||
/**
|
||||
* Search result for episode queries
|
||||
*/
|
||||
export interface EpisodeSearchResult {
|
||||
episode: Episode;
|
||||
similarity: number;
|
||||
trajectoryId?: string;
|
||||
}
|
||||
/**
|
||||
* Fast in-memory AgentDB implementation
|
||||
*/
|
||||
export declare class FastAgentDB {
|
||||
private episodes;
|
||||
private trajectories;
|
||||
private vectorDb;
|
||||
private dimensions;
|
||||
private maxEpisodes;
|
||||
private episodeOrder;
|
||||
/**
|
||||
* Create a new FastAgentDB instance
|
||||
*
|
||||
* @param dimensions - Vector dimensions for state embeddings
|
||||
* @param maxEpisodes - Maximum episodes to store (LRU eviction)
|
||||
*/
|
||||
constructor(dimensions?: number, maxEpisodes?: number);
|
||||
/**
|
||||
* Initialize the vector database
|
||||
*/
|
||||
private initVectorDb;
|
||||
/**
|
||||
* Store an episode
|
||||
*
|
||||
* @param episode - Episode to store
|
||||
* @returns Episode ID
|
||||
*/
|
||||
storeEpisode(episode: Omit<Episode, 'id'> & {
|
||||
id?: string;
|
||||
}): Promise<string>;
|
||||
/**
|
||||
* Store multiple episodes in batch
|
||||
*/
|
||||
storeEpisodes(episodes: (Omit<Episode, 'id'> & {
|
||||
id?: string;
|
||||
})[]): Promise<string[]>;
|
||||
/**
|
||||
* Retrieve an episode by ID
|
||||
*/
|
||||
getEpisode(id: string): Promise<Episode | null>;
|
||||
/**
|
||||
* Search for similar episodes by state
|
||||
*
|
||||
* @param queryState - State vector to search for
|
||||
* @param k - Number of results to return
|
||||
* @returns Similar episodes sorted by similarity
|
||||
*/
|
||||
searchByState(queryState: number[] | Float32Array, k?: number): Promise<EpisodeSearchResult[]>;
|
||||
/**
|
||||
* Fallback similarity search using brute-force cosine similarity
|
||||
*/
|
||||
private fallbackSearch;
|
||||
/**
|
||||
* Compute cosine similarity between two vectors
|
||||
*/
|
||||
private cosineSimilarity;
|
||||
/**
|
||||
* Store a trajectory (sequence of episodes)
|
||||
*/
|
||||
storeTrajectory(episodes: (Omit<Episode, 'id'> & {
|
||||
id?: string;
|
||||
})[], metadata?: Record<string, any>): Promise<string>;
|
||||
/**
|
||||
* Get a trajectory by ID
|
||||
*/
|
||||
getTrajectory(id: string): Promise<Trajectory | null>;
|
||||
/**
|
||||
* Get top trajectories by total reward
|
||||
*/
|
||||
getTopTrajectories(k?: number): Promise<Trajectory[]>;
|
||||
/**
|
||||
* Sample random episodes (for experience replay)
|
||||
*/
|
||||
sampleEpisodes(n: number): Promise<Episode[]>;
|
||||
/**
|
||||
* Get database statistics
|
||||
*/
|
||||
getStats(): {
|
||||
episodeCount: number;
|
||||
trajectoryCount: number;
|
||||
dimensions: number;
|
||||
maxEpisodes: number;
|
||||
vectorDbAvailable: boolean;
|
||||
};
|
||||
/**
|
||||
* Clear all data
|
||||
*/
|
||||
clear(): void;
|
||||
/**
|
||||
* Generate a unique ID
|
||||
*/
|
||||
private generateId;
|
||||
}
|
||||
/**
|
||||
* Create a fast AgentDB instance
|
||||
*/
|
||||
export declare function createFastAgentDB(dimensions?: number, maxEpisodes?: number): FastAgentDB;
|
||||
/**
|
||||
* Get the default FastAgentDB instance
|
||||
*/
|
||||
export declare function getDefaultAgentDB(): FastAgentDB;
|
||||
declare const _default: {
|
||||
FastAgentDB: typeof FastAgentDB;
|
||||
createFastAgentDB: typeof createFastAgentDB;
|
||||
getDefaultAgentDB: typeof getDefaultAgentDB;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=agentdb-fast.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/agentdb-fast.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/agentdb-fast.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"agentdb-fast.d.ts","sourceRoot":"","sources":["agentdb-fast.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;GAUG;AA6BH;;GAEG;AACH,MAAM,WAAW,OAAO;IACtB,EAAE,EAAE,MAAM,CAAC;IACX,KAAK,EAAE,MAAM,EAAE,CAAC;IAChB,MAAM,EAAE,MAAM,GAAG,MAAM,CAAC;IACxB,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,IAAI,EAAE,OAAO,CAAC;IACd,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC;IAC/B,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED;;GAEG;AACH,MAAM,WAAW,UAAU;IACzB,EAAE,EAAE,MAAM,CAAC;IACX,QAAQ,EAAE,OAAO,EAAE,CAAC;IACpB,WAAW,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC;CAChC;AAED;;GAEG;AACH,MAAM,WAAW,mBAAmB;IAClC,OAAO,EAAE,OAAO,CAAC;IACjB,UAAU,EAAE,MAAM,CAAC;IACnB,YAAY,CAAC,EAAE,MAAM,CAAC;CACvB;AAED;;GAEG;AACH,qBAAa,WAAW;IACtB,OAAO,CAAC,QAAQ,CAAmC;IACnD,OAAO,CAAC,YAAY,CAAsC;IAC1D,OAAO,CAAC,QAAQ,CAAa;IAC7B,OAAO,CAAC,UAAU,CAAS;IAC3B,OAAO,CAAC,WAAW,CAAS;IAC5B,OAAO,CAAC,YAAY,CAAgB;IAEpC;;;;;OAKG;gBACS,UAAU,GAAE,MAAY,EAAE,WAAW,GAAE,MAAe;IAKlE;;OAEG;YACW,YAAY;IAe1B;;;;;OAKG;IACG,YAAY,CAAC,OAAO,EAAE,IAAI,CAAC,OAAO,EAAE,IAAI,CAAC,GAAG;QAAE,EAAE,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,OAAO,CAAC,MAAM,CAAC;IAoCnF;;OAEG;IACG,aAAa,CAAC,QAAQ,EAAE,CAAC,IAAI,CAAC,OAAO,EAAE,IAAI,CAAC,GAAG;QAAE,EAAE,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC,EAAE,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IAS3F;;OAEG;IACG,UAAU,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,GAAG,IAAI,CAAC;IAarD;;;;;;OAMG;IACG,aAAa,CACjB,UAAU,EAAE,MAAM,EAAE,GAAG,YAAY,EACnC,CAAC,GAAE,MAAW,GACb,OAAO,CAAC,mBAAmB,EAAE,CAAC;IAgCjC;;OAEG;IACH,OAAO,CAAC,cAAc;IAetB;;OAEG;IACH,OAAO,CAAC,gBAAgB;IAexB;;OAEG;IACG,eAAe,CACnB,QAAQ,EAAE,CAAC,IAAI,CAAC,OAAO,EAAE,IAAI,CAAC,GAAG;QAAE,EAAE,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC,EAAE,EACnD,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,GAC7B,OAAO,CAAC,MAAM,CAAC;IAyBlB;;OAEG;IACG,aAAa,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO,CAAC,UAAU,GAAG,IAAI,CAAC;IAI3D;;OAEG;IACG,kBAAkB,CAAC,CAAC,GAAE,MAAW,GAAG,OAAO,CAAC,UAAU,EAAE,CAAC;IAM/D;;OAEG;IACG,cAAc,CAAC,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,EAAE,CAAC;IAYnD;;OAEG;IACH,QAAQ,IAAI;QACV,YAAY,EAAE,MAAM,CAAC;QACrB,eAAe,EAAE,MAAM,CAAC;QACxB,UAAU,EAAE,MAAM,CAAC;QACnB,WAAW,EAAE,MAAM,CAAC;QACpB,iBAAiB,EAAE,OAAO,CAAC;KAC5B;IAUD;;OAEG;IACH,KAAK,IAAI,IAAI;IAMb;;OAEG;IACH,OAAO,CAAC,UAAU;CAGnB;AAED;;GAEG;AACH,wBAAgB,iBAAiB,CAC/B,UAAU,GAAE,MAAY,EACxB,WAAW,GAAE,MAAe,GAC3B,WAAW,CAEb;AAKD;;GAEG;AACH,wBAAgB,iBAAiB,IAAI,WAAW,CAK/C;;;;;;AAED,wBAIE"}
|
||||
302
npm/packages/ruvector/src/core/agentdb-fast.js
Normal file
302
npm/packages/ruvector/src/core/agentdb-fast.js
Normal file
@@ -0,0 +1,302 @@
|
||||
"use strict";
|
||||
/**
|
||||
* AgentDB Fast - High-performance in-process alternative to AgentDB CLI
|
||||
*
|
||||
* The AgentDB CLI has ~2.3s startup overhead due to npx initialization.
|
||||
* This module provides 50-200x faster operations by using in-process calls.
|
||||
*
|
||||
* Features:
|
||||
* - In-memory episode storage with LRU eviction
|
||||
* - Vector similarity search using @ruvector/core
|
||||
* - Compatible API with AgentDB's episode/trajectory interfaces
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.FastAgentDB = void 0;
|
||||
exports.createFastAgentDB = createFastAgentDB;
|
||||
exports.getDefaultAgentDB = getDefaultAgentDB;
|
||||
// Lazy load ruvector core
|
||||
let coreModule = null;
|
||||
function getCoreModule() {
|
||||
if (coreModule)
|
||||
return coreModule;
|
||||
try {
|
||||
coreModule = require('@ruvector/core');
|
||||
return coreModule;
|
||||
}
|
||||
catch {
|
||||
// Fallback to ruvector if core not available
|
||||
try {
|
||||
coreModule = require('ruvector');
|
||||
return coreModule;
|
||||
}
|
||||
catch (e) {
|
||||
throw new Error(`Neither @ruvector/core nor ruvector is available: ${e.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Fast in-memory AgentDB implementation
|
||||
*/
|
||||
class FastAgentDB {
|
||||
/**
|
||||
* Create a new FastAgentDB instance
|
||||
*
|
||||
* @param dimensions - Vector dimensions for state embeddings
|
||||
* @param maxEpisodes - Maximum episodes to store (LRU eviction)
|
||||
*/
|
||||
constructor(dimensions = 128, maxEpisodes = 100000) {
|
||||
this.episodes = new Map();
|
||||
this.trajectories = new Map();
|
||||
this.vectorDb = null;
|
||||
this.episodeOrder = []; // For LRU eviction
|
||||
this.dimensions = dimensions;
|
||||
this.maxEpisodes = maxEpisodes;
|
||||
}
|
||||
/**
|
||||
* Initialize the vector database
|
||||
*/
|
||||
async initVectorDb() {
|
||||
if (this.vectorDb)
|
||||
return;
|
||||
try {
|
||||
const core = getCoreModule();
|
||||
this.vectorDb = new core.VectorDB({
|
||||
dimensions: this.dimensions,
|
||||
distanceMetric: 'Cosine',
|
||||
});
|
||||
}
|
||||
catch (e) {
|
||||
// Vector DB not available, use fallback similarity
|
||||
console.warn(`VectorDB not available, using fallback similarity: ${e.message}`);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Store an episode
|
||||
*
|
||||
* @param episode - Episode to store
|
||||
* @returns Episode ID
|
||||
*/
|
||||
async storeEpisode(episode) {
|
||||
await this.initVectorDb();
|
||||
const id = episode.id ?? this.generateId();
|
||||
const fullEpisode = {
|
||||
...episode,
|
||||
id,
|
||||
timestamp: episode.timestamp ?? Date.now(),
|
||||
};
|
||||
// LRU eviction if needed
|
||||
if (this.episodes.size >= this.maxEpisodes) {
|
||||
const oldestId = this.episodeOrder.shift();
|
||||
if (oldestId) {
|
||||
this.episodes.delete(oldestId);
|
||||
}
|
||||
}
|
||||
this.episodes.set(id, fullEpisode);
|
||||
this.episodeOrder.push(id);
|
||||
// Index in vector DB if available
|
||||
if (this.vectorDb && fullEpisode.state.length === this.dimensions) {
|
||||
try {
|
||||
await this.vectorDb.insert({
|
||||
id,
|
||||
vector: new Float32Array(fullEpisode.state),
|
||||
});
|
||||
}
|
||||
catch {
|
||||
// Ignore indexing errors
|
||||
}
|
||||
}
|
||||
return id;
|
||||
}
|
||||
/**
|
||||
* Store multiple episodes in batch
|
||||
*/
|
||||
async storeEpisodes(episodes) {
|
||||
const ids = [];
|
||||
for (const episode of episodes) {
|
||||
const id = await this.storeEpisode(episode);
|
||||
ids.push(id);
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
/**
|
||||
* Retrieve an episode by ID
|
||||
*/
|
||||
async getEpisode(id) {
|
||||
const episode = this.episodes.get(id);
|
||||
if (episode) {
|
||||
// Update LRU order
|
||||
const idx = this.episodeOrder.indexOf(id);
|
||||
if (idx > -1) {
|
||||
this.episodeOrder.splice(idx, 1);
|
||||
this.episodeOrder.push(id);
|
||||
}
|
||||
}
|
||||
return episode ?? null;
|
||||
}
|
||||
/**
|
||||
* Search for similar episodes by state
|
||||
*
|
||||
* @param queryState - State vector to search for
|
||||
* @param k - Number of results to return
|
||||
* @returns Similar episodes sorted by similarity
|
||||
*/
|
||||
async searchByState(queryState, k = 10) {
|
||||
await this.initVectorDb();
|
||||
const query = Array.isArray(queryState) ? queryState : Array.from(queryState);
|
||||
// Use vector DB if available
|
||||
if (this.vectorDb && query.length === this.dimensions) {
|
||||
try {
|
||||
const results = await this.vectorDb.search({
|
||||
vector: new Float32Array(query),
|
||||
k,
|
||||
});
|
||||
return results
|
||||
.map((r) => {
|
||||
const episode = this.episodes.get(r.id);
|
||||
if (!episode)
|
||||
return null;
|
||||
return {
|
||||
episode,
|
||||
similarity: 1 - r.score, // Convert distance to similarity
|
||||
};
|
||||
})
|
||||
.filter((r) => r !== null);
|
||||
}
|
||||
catch {
|
||||
// Fall through to fallback
|
||||
}
|
||||
}
|
||||
// Fallback: brute-force cosine similarity
|
||||
return this.fallbackSearch(query, k);
|
||||
}
|
||||
/**
|
||||
* Fallback similarity search using brute-force cosine similarity
|
||||
*/
|
||||
fallbackSearch(query, k) {
|
||||
const results = [];
|
||||
for (const episode of this.episodes.values()) {
|
||||
if (episode.state.length !== query.length)
|
||||
continue;
|
||||
const similarity = this.cosineSimilarity(query, episode.state);
|
||||
results.push({ episode, similarity });
|
||||
}
|
||||
return results
|
||||
.sort((a, b) => b.similarity - a.similarity)
|
||||
.slice(0, k);
|
||||
}
|
||||
/**
|
||||
* Compute cosine similarity between two vectors
|
||||
*/
|
||||
cosineSimilarity(a, b) {
|
||||
let dotProduct = 0;
|
||||
let normA = 0;
|
||||
let normB = 0;
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dotProduct += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
const denom = Math.sqrt(normA) * Math.sqrt(normB);
|
||||
return denom === 0 ? 0 : dotProduct / denom;
|
||||
}
|
||||
/**
|
||||
* Store a trajectory (sequence of episodes)
|
||||
*/
|
||||
async storeTrajectory(episodes, metadata) {
|
||||
const trajectoryId = this.generateId();
|
||||
const storedEpisodes = [];
|
||||
let totalReward = 0;
|
||||
for (const episode of episodes) {
|
||||
const id = await this.storeEpisode(episode);
|
||||
const stored = await this.getEpisode(id);
|
||||
if (stored) {
|
||||
storedEpisodes.push(stored);
|
||||
totalReward += stored.reward;
|
||||
}
|
||||
}
|
||||
const trajectory = {
|
||||
id: trajectoryId,
|
||||
episodes: storedEpisodes,
|
||||
totalReward,
|
||||
metadata,
|
||||
};
|
||||
this.trajectories.set(trajectoryId, trajectory);
|
||||
return trajectoryId;
|
||||
}
|
||||
/**
|
||||
* Get a trajectory by ID
|
||||
*/
|
||||
async getTrajectory(id) {
|
||||
return this.trajectories.get(id) ?? null;
|
||||
}
|
||||
/**
|
||||
* Get top trajectories by total reward
|
||||
*/
|
||||
async getTopTrajectories(k = 10) {
|
||||
return Array.from(this.trajectories.values())
|
||||
.sort((a, b) => b.totalReward - a.totalReward)
|
||||
.slice(0, k);
|
||||
}
|
||||
/**
|
||||
* Sample random episodes (for experience replay)
|
||||
*/
|
||||
async sampleEpisodes(n) {
|
||||
const allEpisodes = Array.from(this.episodes.values());
|
||||
const sampled = [];
|
||||
for (let i = 0; i < Math.min(n, allEpisodes.length); i++) {
|
||||
const idx = Math.floor(Math.random() * allEpisodes.length);
|
||||
sampled.push(allEpisodes[idx]);
|
||||
}
|
||||
return sampled;
|
||||
}
|
||||
/**
|
||||
* Get database statistics
|
||||
*/
|
||||
getStats() {
|
||||
return {
|
||||
episodeCount: this.episodes.size,
|
||||
trajectoryCount: this.trajectories.size,
|
||||
dimensions: this.dimensions,
|
||||
maxEpisodes: this.maxEpisodes,
|
||||
vectorDbAvailable: this.vectorDb !== null,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Clear all data
|
||||
*/
|
||||
clear() {
|
||||
this.episodes.clear();
|
||||
this.trajectories.clear();
|
||||
this.episodeOrder = [];
|
||||
}
|
||||
/**
|
||||
* Generate a unique ID
|
||||
*/
|
||||
generateId() {
|
||||
return `${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
|
||||
}
|
||||
}
|
||||
exports.FastAgentDB = FastAgentDB;
|
||||
/**
|
||||
* Create a fast AgentDB instance
|
||||
*/
|
||||
function createFastAgentDB(dimensions = 128, maxEpisodes = 100000) {
|
||||
return new FastAgentDB(dimensions, maxEpisodes);
|
||||
}
|
||||
// Singleton instance for convenience
|
||||
let defaultInstance = null;
|
||||
/**
|
||||
* Get the default FastAgentDB instance
|
||||
*/
|
||||
function getDefaultAgentDB() {
|
||||
if (!defaultInstance) {
|
||||
defaultInstance = new FastAgentDB();
|
||||
}
|
||||
return defaultInstance;
|
||||
}
|
||||
exports.default = {
|
||||
FastAgentDB,
|
||||
createFastAgentDB,
|
||||
getDefaultAgentDB,
|
||||
};
|
||||
//# sourceMappingURL=agentdb-fast.js.map
|
||||
1
npm/packages/ruvector/src/core/agentdb-fast.js.map
Normal file
1
npm/packages/ruvector/src/core/agentdb-fast.js.map
Normal file
File diff suppressed because one or more lines are too long
386
npm/packages/ruvector/src/core/agentdb-fast.ts
Normal file
386
npm/packages/ruvector/src/core/agentdb-fast.ts
Normal file
@@ -0,0 +1,386 @@
|
||||
/**
|
||||
* AgentDB Fast - High-performance in-process alternative to AgentDB CLI
|
||||
*
|
||||
* The AgentDB CLI has ~2.3s startup overhead due to npx initialization.
|
||||
* This module provides 50-200x faster operations by using in-process calls.
|
||||
*
|
||||
* Features:
|
||||
* - In-memory episode storage with LRU eviction
|
||||
* - Vector similarity search using @ruvector/core
|
||||
* - Compatible API with AgentDB's episode/trajectory interfaces
|
||||
*/
|
||||
|
||||
import type {
|
||||
VectorEntry,
|
||||
SearchResult,
|
||||
SearchQuery,
|
||||
} from '../types';
|
||||
|
||||
// Lazy load ruvector core
|
||||
let coreModule: any = null;
|
||||
|
||||
function getCoreModule() {
|
||||
if (coreModule) return coreModule;
|
||||
try {
|
||||
coreModule = require('@ruvector/core');
|
||||
return coreModule;
|
||||
} catch {
|
||||
// Fallback to ruvector if core not available
|
||||
try {
|
||||
coreModule = require('ruvector');
|
||||
return coreModule;
|
||||
} catch (e: any) {
|
||||
throw new Error(
|
||||
`Neither @ruvector/core nor ruvector is available: ${e.message}`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Episode entry for trajectory storage
|
||||
*/
|
||||
export interface Episode {
|
||||
id: string;
|
||||
state: number[];
|
||||
action: string | number;
|
||||
reward: number;
|
||||
nextState: number[];
|
||||
done: boolean;
|
||||
metadata?: Record<string, any>;
|
||||
timestamp?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Trajectory (sequence of episodes)
|
||||
*/
|
||||
export interface Trajectory {
|
||||
id: string;
|
||||
episodes: Episode[];
|
||||
totalReward: number;
|
||||
metadata?: Record<string, any>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Search result for episode queries
|
||||
*/
|
||||
export interface EpisodeSearchResult {
|
||||
episode: Episode;
|
||||
similarity: number;
|
||||
trajectoryId?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fast in-memory AgentDB implementation
|
||||
*/
|
||||
export class FastAgentDB {
|
||||
private episodes: Map<string, Episode> = new Map();
|
||||
private trajectories: Map<string, Trajectory> = new Map();
|
||||
private vectorDb: any = null;
|
||||
private dimensions: number;
|
||||
private maxEpisodes: number;
|
||||
private episodeOrder: string[] = []; // For LRU eviction
|
||||
|
||||
/**
|
||||
* Create a new FastAgentDB instance
|
||||
*
|
||||
* @param dimensions - Vector dimensions for state embeddings
|
||||
* @param maxEpisodes - Maximum episodes to store (LRU eviction)
|
||||
*/
|
||||
constructor(dimensions: number = 128, maxEpisodes: number = 100000) {
|
||||
this.dimensions = dimensions;
|
||||
this.maxEpisodes = maxEpisodes;
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize the vector database
|
||||
*/
|
||||
private async initVectorDb(): Promise<void> {
|
||||
if (this.vectorDb) return;
|
||||
|
||||
try {
|
||||
const core = getCoreModule();
|
||||
this.vectorDb = new core.VectorDB({
|
||||
dimensions: this.dimensions,
|
||||
distanceMetric: 'Cosine',
|
||||
});
|
||||
} catch (e: any) {
|
||||
// Vector DB not available, use fallback similarity
|
||||
console.warn(`VectorDB not available, using fallback similarity: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Store an episode
|
||||
*
|
||||
* @param episode - Episode to store
|
||||
* @returns Episode ID
|
||||
*/
|
||||
async storeEpisode(episode: Omit<Episode, 'id'> & { id?: string }): Promise<string> {
|
||||
await this.initVectorDb();
|
||||
|
||||
const id = episode.id ?? this.generateId();
|
||||
const fullEpisode: Episode = {
|
||||
...episode,
|
||||
id,
|
||||
timestamp: episode.timestamp ?? Date.now(),
|
||||
};
|
||||
|
||||
// LRU eviction if needed
|
||||
if (this.episodes.size >= this.maxEpisodes) {
|
||||
const oldestId = this.episodeOrder.shift();
|
||||
if (oldestId) {
|
||||
this.episodes.delete(oldestId);
|
||||
}
|
||||
}
|
||||
|
||||
this.episodes.set(id, fullEpisode);
|
||||
this.episodeOrder.push(id);
|
||||
|
||||
// Index in vector DB if available
|
||||
if (this.vectorDb && fullEpisode.state.length === this.dimensions) {
|
||||
try {
|
||||
await this.vectorDb.insert({
|
||||
id,
|
||||
vector: new Float32Array(fullEpisode.state),
|
||||
});
|
||||
} catch {
|
||||
// Ignore indexing errors
|
||||
}
|
||||
}
|
||||
|
||||
return id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Store multiple episodes in batch
|
||||
*/
|
||||
async storeEpisodes(episodes: (Omit<Episode, 'id'> & { id?: string })[]): Promise<string[]> {
|
||||
const ids: string[] = [];
|
||||
for (const episode of episodes) {
|
||||
const id = await this.storeEpisode(episode);
|
||||
ids.push(id);
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieve an episode by ID
|
||||
*/
|
||||
async getEpisode(id: string): Promise<Episode | null> {
|
||||
const episode = this.episodes.get(id);
|
||||
if (episode) {
|
||||
// Update LRU order
|
||||
const idx = this.episodeOrder.indexOf(id);
|
||||
if (idx > -1) {
|
||||
this.episodeOrder.splice(idx, 1);
|
||||
this.episodeOrder.push(id);
|
||||
}
|
||||
}
|
||||
return episode ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Search for similar episodes by state
|
||||
*
|
||||
* @param queryState - State vector to search for
|
||||
* @param k - Number of results to return
|
||||
* @returns Similar episodes sorted by similarity
|
||||
*/
|
||||
async searchByState(
|
||||
queryState: number[] | Float32Array,
|
||||
k: number = 10
|
||||
): Promise<EpisodeSearchResult[]> {
|
||||
await this.initVectorDb();
|
||||
|
||||
const query = Array.isArray(queryState) ? queryState : Array.from(queryState);
|
||||
|
||||
// Use vector DB if available
|
||||
if (this.vectorDb && query.length === this.dimensions) {
|
||||
try {
|
||||
const results: SearchResult[] = await this.vectorDb.search({
|
||||
vector: new Float32Array(query),
|
||||
k,
|
||||
});
|
||||
|
||||
return results
|
||||
.map((r) => {
|
||||
const episode = this.episodes.get(r.id);
|
||||
if (!episode) return null;
|
||||
return {
|
||||
episode,
|
||||
similarity: 1 - r.score, // Convert distance to similarity
|
||||
};
|
||||
})
|
||||
.filter((r): r is EpisodeSearchResult => r !== null);
|
||||
} catch {
|
||||
// Fall through to fallback
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: brute-force cosine similarity
|
||||
return this.fallbackSearch(query, k);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback similarity search using brute-force cosine similarity
|
||||
*/
|
||||
private fallbackSearch(query: number[], k: number): EpisodeSearchResult[] {
|
||||
const results: EpisodeSearchResult[] = [];
|
||||
|
||||
for (const episode of this.episodes.values()) {
|
||||
if (episode.state.length !== query.length) continue;
|
||||
|
||||
const similarity = this.cosineSimilarity(query, episode.state);
|
||||
results.push({ episode, similarity });
|
||||
}
|
||||
|
||||
return results
|
||||
.sort((a, b) => b.similarity - a.similarity)
|
||||
.slice(0, k);
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute cosine similarity between two vectors
|
||||
*/
|
||||
private cosineSimilarity(a: number[], b: number[]): number {
|
||||
let dotProduct = 0;
|
||||
let normA = 0;
|
||||
let normB = 0;
|
||||
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dotProduct += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
|
||||
const denom = Math.sqrt(normA) * Math.sqrt(normB);
|
||||
return denom === 0 ? 0 : dotProduct / denom;
|
||||
}
|
||||
|
||||
/**
|
||||
* Store a trajectory (sequence of episodes)
|
||||
*/
|
||||
async storeTrajectory(
|
||||
episodes: (Omit<Episode, 'id'> & { id?: string })[],
|
||||
metadata?: Record<string, any>
|
||||
): Promise<string> {
|
||||
const trajectoryId = this.generateId();
|
||||
const storedEpisodes: Episode[] = [];
|
||||
let totalReward = 0;
|
||||
|
||||
for (const episode of episodes) {
|
||||
const id = await this.storeEpisode(episode);
|
||||
const stored = await this.getEpisode(id);
|
||||
if (stored) {
|
||||
storedEpisodes.push(stored);
|
||||
totalReward += stored.reward;
|
||||
}
|
||||
}
|
||||
|
||||
const trajectory: Trajectory = {
|
||||
id: trajectoryId,
|
||||
episodes: storedEpisodes,
|
||||
totalReward,
|
||||
metadata,
|
||||
};
|
||||
|
||||
this.trajectories.set(trajectoryId, trajectory);
|
||||
return trajectoryId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a trajectory by ID
|
||||
*/
|
||||
async getTrajectory(id: string): Promise<Trajectory | null> {
|
||||
return this.trajectories.get(id) ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get top trajectories by total reward
|
||||
*/
|
||||
async getTopTrajectories(k: number = 10): Promise<Trajectory[]> {
|
||||
return Array.from(this.trajectories.values())
|
||||
.sort((a, b) => b.totalReward - a.totalReward)
|
||||
.slice(0, k);
|
||||
}
|
||||
|
||||
/**
|
||||
* Sample random episodes (for experience replay)
|
||||
*/
|
||||
async sampleEpisodes(n: number): Promise<Episode[]> {
|
||||
const allEpisodes = Array.from(this.episodes.values());
|
||||
const sampled: Episode[] = [];
|
||||
|
||||
for (let i = 0; i < Math.min(n, allEpisodes.length); i++) {
|
||||
const idx = Math.floor(Math.random() * allEpisodes.length);
|
||||
sampled.push(allEpisodes[idx]);
|
||||
}
|
||||
|
||||
return sampled;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get database statistics
|
||||
*/
|
||||
getStats(): {
|
||||
episodeCount: number;
|
||||
trajectoryCount: number;
|
||||
dimensions: number;
|
||||
maxEpisodes: number;
|
||||
vectorDbAvailable: boolean;
|
||||
} {
|
||||
return {
|
||||
episodeCount: this.episodes.size,
|
||||
trajectoryCount: this.trajectories.size,
|
||||
dimensions: this.dimensions,
|
||||
maxEpisodes: this.maxEpisodes,
|
||||
vectorDbAvailable: this.vectorDb !== null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all data
|
||||
*/
|
||||
clear(): void {
|
||||
this.episodes.clear();
|
||||
this.trajectories.clear();
|
||||
this.episodeOrder = [];
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a unique ID
|
||||
*/
|
||||
private generateId(): string {
|
||||
return `${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a fast AgentDB instance
|
||||
*/
|
||||
export function createFastAgentDB(
|
||||
dimensions: number = 128,
|
||||
maxEpisodes: number = 100000
|
||||
): FastAgentDB {
|
||||
return new FastAgentDB(dimensions, maxEpisodes);
|
||||
}
|
||||
|
||||
// Singleton instance for convenience
|
||||
let defaultInstance: FastAgentDB | null = null;
|
||||
|
||||
/**
|
||||
* Get the default FastAgentDB instance
|
||||
*/
|
||||
export function getDefaultAgentDB(): FastAgentDB {
|
||||
if (!defaultInstance) {
|
||||
defaultInstance = new FastAgentDB();
|
||||
}
|
||||
return defaultInstance;
|
||||
}
|
||||
|
||||
export default {
|
||||
FastAgentDB,
|
||||
createFastAgentDB,
|
||||
getDefaultAgentDB,
|
||||
};
|
||||
108
npm/packages/ruvector/src/core/ast-parser.d.ts
vendored
Normal file
108
npm/packages/ruvector/src/core/ast-parser.d.ts
vendored
Normal file
@@ -0,0 +1,108 @@
|
||||
/**
|
||||
* AST Parser - Tree-sitter based code parsing
|
||||
*
|
||||
* Provides real AST parsing for accurate code analysis,
|
||||
* replacing regex-based heuristics with proper parsing.
|
||||
*
|
||||
* Supports: TypeScript, JavaScript, Python, Rust, Go, Java, C/C++
|
||||
*/
|
||||
export declare function isTreeSitterAvailable(): boolean;
|
||||
export interface ASTNode {
|
||||
type: string;
|
||||
text: string;
|
||||
startPosition: {
|
||||
row: number;
|
||||
column: number;
|
||||
};
|
||||
endPosition: {
|
||||
row: number;
|
||||
column: number;
|
||||
};
|
||||
children: ASTNode[];
|
||||
parent?: string;
|
||||
}
|
||||
export interface FunctionInfo {
|
||||
name: string;
|
||||
params: string[];
|
||||
returnType?: string;
|
||||
async: boolean;
|
||||
exported: boolean;
|
||||
startLine: number;
|
||||
endLine: number;
|
||||
complexity: number;
|
||||
calls: string[];
|
||||
}
|
||||
export interface ClassInfo {
|
||||
name: string;
|
||||
extends?: string;
|
||||
implements: string[];
|
||||
methods: FunctionInfo[];
|
||||
properties: string[];
|
||||
exported: boolean;
|
||||
startLine: number;
|
||||
endLine: number;
|
||||
}
|
||||
export interface ImportInfo {
|
||||
source: string;
|
||||
default?: string;
|
||||
named: string[];
|
||||
namespace?: string;
|
||||
type: 'esm' | 'commonjs' | 'dynamic';
|
||||
}
|
||||
export interface ExportInfo {
|
||||
name: string;
|
||||
type: 'default' | 'named' | 'all';
|
||||
source?: string;
|
||||
}
|
||||
export interface FileAnalysis {
|
||||
file: string;
|
||||
language: string;
|
||||
imports: ImportInfo[];
|
||||
exports: ExportInfo[];
|
||||
functions: FunctionInfo[];
|
||||
classes: ClassInfo[];
|
||||
variables: string[];
|
||||
types: string[];
|
||||
complexity: number;
|
||||
lines: number;
|
||||
parseTime: number;
|
||||
}
|
||||
export declare class CodeParser {
|
||||
private parser;
|
||||
private initialized;
|
||||
init(): Promise<boolean>;
|
||||
/**
|
||||
* Detect language from file extension
|
||||
*/
|
||||
detectLanguage(file: string): string;
|
||||
/**
|
||||
* Parse a file and return the AST
|
||||
*/
|
||||
parse(file: string, content?: string): Promise<ASTNode | null>;
|
||||
private convertNode;
|
||||
/**
|
||||
* Analyze a file for functions, classes, imports, etc.
|
||||
*/
|
||||
analyze(file: string, content?: string): Promise<FileAnalysis>;
|
||||
private analyzeTree;
|
||||
private parseImport;
|
||||
private parseExport;
|
||||
private parseFunction;
|
||||
private parseClass;
|
||||
private findChild;
|
||||
private getIdentifierName;
|
||||
private calculateComplexity;
|
||||
private analyzeWithRegex;
|
||||
/**
|
||||
* Get all symbols (functions, classes, types) in a file
|
||||
*/
|
||||
getSymbols(file: string): Promise<string[]>;
|
||||
/**
|
||||
* Get the call graph for a file
|
||||
*/
|
||||
getCallGraph(file: string): Promise<Map<string, string[]>>;
|
||||
}
|
||||
export declare function getCodeParser(): CodeParser;
|
||||
export declare function initCodeParser(): Promise<CodeParser>;
|
||||
export default CodeParser;
|
||||
//# sourceMappingURL=ast-parser.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/ast-parser.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/ast-parser.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"ast-parser.d.ts","sourceRoot":"","sources":["ast-parser.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAgEH,wBAAgB,qBAAqB,IAAI,OAAO,CAO/C;AAMD,MAAM,WAAW,OAAO;IACtB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,aAAa,EAAE;QAAE,GAAG,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;IAC/C,WAAW,EAAE;QAAE,GAAG,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7C,QAAQ,EAAE,OAAO,EAAE,CAAC;IACpB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,KAAK,EAAE,OAAO,CAAC;IACf,QAAQ,EAAE,OAAO,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;IACnB,KAAK,EAAE,MAAM,EAAE,CAAC;CACjB;AAED,MAAM,WAAW,SAAS;IACxB,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,OAAO,EAAE,YAAY,EAAE,CAAC;IACxB,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,QAAQ,EAAE,OAAO,CAAC;IAClB,SAAS,EAAE,MAAM,CAAC;IAClB,OAAO,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,KAAK,EAAE,MAAM,EAAE,CAAC;IAChB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,IAAI,EAAE,KAAK,GAAG,UAAU,GAAG,SAAS,CAAC;CACtC;AAED,MAAM,WAAW,UAAU;IACzB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,SAAS,GAAG,OAAO,GAAG,KAAK,CAAC;IAClC,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,UAAU,EAAE,CAAC;IACtB,OAAO,EAAE,UAAU,EAAE,CAAC;IACtB,SAAS,EAAE,YAAY,EAAE,CAAC;IAC1B,OAAO,EAAE,SAAS,EAAE,CAAC;IACrB,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,KAAK,EAAE,MAAM,EAAE,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;IACnB,KAAK,EAAE,MAAM,CAAC;IACd,SAAS,EAAE,MAAM,CAAC;CACnB;AAMD,qBAAa,UAAU;IACrB,OAAO,CAAC,MAAM,CAAa;IAC3B,OAAO,CAAC,WAAW,CAAS;IAEtB,IAAI,IAAI,OAAO,CAAC,OAAO,CAAC;IAW9B;;OAEG;IACH,cAAc,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM;IAyBpC;;OAEG;IACG,KAAK,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,GAAG,IAAI,CAAC;IAkBpE,OAAO,CAAC,WAAW;IAUnB;;OAEG;IACG,OAAO,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,YAAY,CAAC;IAmBpE,OAAO,CAAC,WAAW;IAuEnB,OAAO,CAAC,WAAW;IAmCnB,OAAO,CAAC,WAAW;IAenB,OAAO,CAAC,aAAa;IAsDrB,OAAO,CAAC,UAAU;IA8ClB,OAAO,CAAC,SAAS;IAUjB,OAAO,CAAC,iBAAiB;IAWzB,OAAO,CAAC,mBAAmB;IAoB3B,OAAO,CAAC,gBAAgB;IA8GxB;;OAEG;IACG,UAAU,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IAUjD;;OAEG;IACG,YAAY,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC;CAgBjE;AAQD,wBAAgB,aAAa,IAAI,UAAU,CAK1C;AAED,wBAAsB,cAAc,IAAI,OAAO,CAAC,UAAU,CAAC,CAI1D;AAED,eAAe,UAAU,CAAC"}
|
||||
603
npm/packages/ruvector/src/core/ast-parser.js
Normal file
603
npm/packages/ruvector/src/core/ast-parser.js
Normal file
@@ -0,0 +1,603 @@
|
||||
"use strict";
|
||||
/**
|
||||
* AST Parser - Tree-sitter based code parsing
|
||||
*
|
||||
* Provides real AST parsing for accurate code analysis,
|
||||
* replacing regex-based heuristics with proper parsing.
|
||||
*
|
||||
* Supports: TypeScript, JavaScript, Python, Rust, Go, Java, C/C++
|
||||
*/
|
||||
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
var desc = Object.getOwnPropertyDescriptor(m, k);
|
||||
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
||||
desc = { enumerable: true, get: function() { return m[k]; } };
|
||||
}
|
||||
Object.defineProperty(o, k2, desc);
|
||||
}) : (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
o[k2] = m[k];
|
||||
}));
|
||||
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
||||
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
||||
}) : function(o, v) {
|
||||
o["default"] = v;
|
||||
});
|
||||
var __importStar = (this && this.__importStar) || (function () {
|
||||
var ownKeys = function(o) {
|
||||
ownKeys = Object.getOwnPropertyNames || function (o) {
|
||||
var ar = [];
|
||||
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
||||
return ar;
|
||||
};
|
||||
return ownKeys(o);
|
||||
};
|
||||
return function (mod) {
|
||||
if (mod && mod.__esModule) return mod;
|
||||
var result = {};
|
||||
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
||||
__setModuleDefault(result, mod);
|
||||
return result;
|
||||
};
|
||||
})();
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.CodeParser = void 0;
|
||||
exports.isTreeSitterAvailable = isTreeSitterAvailable;
|
||||
exports.getCodeParser = getCodeParser;
|
||||
exports.initCodeParser = initCodeParser;
|
||||
const fs = __importStar(require("fs"));
|
||||
const path = __importStar(require("path"));
|
||||
// Try to load tree-sitter
|
||||
let Parser = null;
|
||||
let languages = new Map();
|
||||
let parserError = null;
|
||||
async function loadTreeSitter() {
|
||||
if (Parser)
|
||||
return true;
|
||||
if (parserError)
|
||||
return false;
|
||||
try {
|
||||
// Dynamic require to avoid TypeScript errors
|
||||
Parser = require('tree-sitter');
|
||||
return true;
|
||||
}
|
||||
catch (e) {
|
||||
parserError = new Error(`tree-sitter not installed: ${e.message}\n` +
|
||||
`Install with: npm install tree-sitter tree-sitter-typescript tree-sitter-javascript tree-sitter-python`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
async function loadLanguage(lang) {
|
||||
if (languages.has(lang))
|
||||
return languages.get(lang);
|
||||
const langPackages = {
|
||||
typescript: 'tree-sitter-typescript',
|
||||
javascript: 'tree-sitter-javascript',
|
||||
python: 'tree-sitter-python',
|
||||
rust: 'tree-sitter-rust',
|
||||
go: 'tree-sitter-go',
|
||||
java: 'tree-sitter-java',
|
||||
c: 'tree-sitter-c',
|
||||
cpp: 'tree-sitter-cpp',
|
||||
ruby: 'tree-sitter-ruby',
|
||||
php: 'tree-sitter-php',
|
||||
};
|
||||
const pkg = langPackages[lang];
|
||||
if (!pkg)
|
||||
return null;
|
||||
try {
|
||||
const langModule = await Promise.resolve(`${pkg}`).then(s => __importStar(require(s)));
|
||||
const language = langModule.default || langModule;
|
||||
// Handle TypeScript which exports tsx and typescript
|
||||
if (lang === 'typescript' && language.typescript) {
|
||||
languages.set(lang, language.typescript);
|
||||
languages.set('tsx', language.tsx);
|
||||
return language.typescript;
|
||||
}
|
||||
languages.set(lang, language);
|
||||
return language;
|
||||
}
|
||||
catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
function isTreeSitterAvailable() {
|
||||
try {
|
||||
require.resolve('tree-sitter');
|
||||
return true;
|
||||
}
|
||||
catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
// ============================================================================
|
||||
// Parser
|
||||
// ============================================================================
|
||||
class CodeParser {
|
||||
constructor() {
|
||||
this.parser = null;
|
||||
this.initialized = false;
|
||||
}
|
||||
async init() {
|
||||
if (this.initialized)
|
||||
return true;
|
||||
const loaded = await loadTreeSitter();
|
||||
if (!loaded)
|
||||
return false;
|
||||
this.parser = new Parser();
|
||||
this.initialized = true;
|
||||
return true;
|
||||
}
|
||||
/**
|
||||
* Detect language from file extension
|
||||
*/
|
||||
detectLanguage(file) {
|
||||
const ext = path.extname(file).toLowerCase();
|
||||
const langMap = {
|
||||
'.ts': 'typescript',
|
||||
'.tsx': 'tsx',
|
||||
'.js': 'javascript',
|
||||
'.jsx': 'javascript',
|
||||
'.mjs': 'javascript',
|
||||
'.cjs': 'javascript',
|
||||
'.py': 'python',
|
||||
'.rs': 'rust',
|
||||
'.go': 'go',
|
||||
'.java': 'java',
|
||||
'.c': 'c',
|
||||
'.h': 'c',
|
||||
'.cpp': 'cpp',
|
||||
'.cc': 'cpp',
|
||||
'.cxx': 'cpp',
|
||||
'.hpp': 'cpp',
|
||||
'.rb': 'ruby',
|
||||
'.php': 'php',
|
||||
};
|
||||
return langMap[ext] || 'unknown';
|
||||
}
|
||||
/**
|
||||
* Parse a file and return the AST
|
||||
*/
|
||||
async parse(file, content) {
|
||||
if (!this.initialized) {
|
||||
await this.init();
|
||||
}
|
||||
if (!this.parser)
|
||||
return null;
|
||||
const lang = this.detectLanguage(file);
|
||||
const language = await loadLanguage(lang);
|
||||
if (!language)
|
||||
return null;
|
||||
this.parser.setLanguage(language);
|
||||
const code = content ?? (fs.existsSync(file) ? fs.readFileSync(file, 'utf8') : '');
|
||||
const tree = this.parser.parse(code);
|
||||
return this.convertNode(tree.rootNode);
|
||||
}
|
||||
convertNode(node) {
|
||||
return {
|
||||
type: node.type,
|
||||
text: node.text,
|
||||
startPosition: node.startPosition,
|
||||
endPosition: node.endPosition,
|
||||
children: node.children?.map((c) => this.convertNode(c)) || [],
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Analyze a file for functions, classes, imports, etc.
|
||||
*/
|
||||
async analyze(file, content) {
|
||||
const start = performance.now();
|
||||
const lang = this.detectLanguage(file);
|
||||
const code = content ?? (fs.existsSync(file) ? fs.readFileSync(file, 'utf8') : '');
|
||||
// Try tree-sitter first, fall back to regex
|
||||
if (this.initialized && this.parser) {
|
||||
const language = await loadLanguage(lang);
|
||||
if (language) {
|
||||
this.parser.setLanguage(language);
|
||||
const tree = this.parser.parse(code);
|
||||
return this.analyzeTree(file, lang, tree.rootNode, code, start);
|
||||
}
|
||||
}
|
||||
// Regex fallback
|
||||
return this.analyzeWithRegex(file, lang, code, start);
|
||||
}
|
||||
analyzeTree(file, lang, root, code, start) {
|
||||
const imports = [];
|
||||
const exports = [];
|
||||
const functions = [];
|
||||
const classes = [];
|
||||
const variables = [];
|
||||
const types = [];
|
||||
const visit = (node) => {
|
||||
// Imports
|
||||
if (node.type === 'import_statement' || node.type === 'import_declaration') {
|
||||
const imp = this.parseImport(node, lang);
|
||||
if (imp)
|
||||
imports.push(imp);
|
||||
}
|
||||
// Exports
|
||||
if (node.type.includes('export')) {
|
||||
const exp = this.parseExport(node, lang);
|
||||
if (exp)
|
||||
exports.push(exp);
|
||||
}
|
||||
// Functions
|
||||
if (node.type.includes('function') || node.type === 'method_definition' || node.type === 'arrow_function') {
|
||||
const fn = this.parseFunction(node, code, lang);
|
||||
if (fn)
|
||||
functions.push(fn);
|
||||
}
|
||||
// Classes
|
||||
if (node.type === 'class_declaration' || node.type === 'class') {
|
||||
const cls = this.parseClass(node, code, lang);
|
||||
if (cls)
|
||||
classes.push(cls);
|
||||
}
|
||||
// Variables
|
||||
if (node.type === 'variable_declarator' || node.type === 'assignment') {
|
||||
const name = this.getIdentifierName(node);
|
||||
if (name)
|
||||
variables.push(name);
|
||||
}
|
||||
// Type definitions
|
||||
if (node.type === 'type_alias_declaration' || node.type === 'interface_declaration') {
|
||||
const name = this.getIdentifierName(node);
|
||||
if (name)
|
||||
types.push(name);
|
||||
}
|
||||
// Recurse
|
||||
for (const child of node.children || []) {
|
||||
visit(child);
|
||||
}
|
||||
};
|
||||
visit(root);
|
||||
const lines = code.split('\n').length;
|
||||
const complexity = this.calculateComplexity(code);
|
||||
return {
|
||||
file,
|
||||
language: lang,
|
||||
imports,
|
||||
exports,
|
||||
functions,
|
||||
classes,
|
||||
variables,
|
||||
types,
|
||||
complexity,
|
||||
lines,
|
||||
parseTime: performance.now() - start,
|
||||
};
|
||||
}
|
||||
parseImport(node, lang) {
|
||||
try {
|
||||
const source = this.findChild(node, 'string')?.text?.replace(/['"]/g, '') || '';
|
||||
const named = [];
|
||||
let defaultImport;
|
||||
let namespace;
|
||||
// Find import specifiers
|
||||
const specifiers = this.findChild(node, 'import_clause') || node;
|
||||
for (const child of specifiers.children || []) {
|
||||
if (child.type === 'identifier') {
|
||||
defaultImport = child.text;
|
||||
}
|
||||
else if (child.type === 'namespace_import') {
|
||||
namespace = this.getIdentifierName(child) || undefined;
|
||||
}
|
||||
else if (child.type === 'named_imports') {
|
||||
for (const spec of child.children || []) {
|
||||
if (spec.type === 'import_specifier') {
|
||||
named.push(this.getIdentifierName(spec) || '');
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return {
|
||||
source,
|
||||
default: defaultImport,
|
||||
named: named.filter(Boolean),
|
||||
namespace,
|
||||
type: 'esm',
|
||||
};
|
||||
}
|
||||
catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
parseExport(node, lang) {
|
||||
try {
|
||||
if (node.type === 'export_statement') {
|
||||
const declaration = this.findChild(node, 'declaration');
|
||||
if (declaration) {
|
||||
const name = this.getIdentifierName(declaration);
|
||||
return { name: name || 'default', type: node.text.includes('default') ? 'default' : 'named' };
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
parseFunction(node, code, lang) {
|
||||
try {
|
||||
const name = this.getIdentifierName(node) || '<anonymous>';
|
||||
const params = [];
|
||||
let returnType;
|
||||
const isAsync = node.text.includes('async');
|
||||
const isExported = node.parent?.type?.includes('export');
|
||||
// Get parameters
|
||||
const paramsNode = this.findChild(node, 'formal_parameters') || this.findChild(node, 'parameters');
|
||||
if (paramsNode) {
|
||||
for (const param of paramsNode.children || []) {
|
||||
if (param.type === 'identifier' || param.type === 'required_parameter') {
|
||||
params.push(this.getIdentifierName(param) || '');
|
||||
}
|
||||
}
|
||||
}
|
||||
// Get return type
|
||||
const returnNode = this.findChild(node, 'type_annotation');
|
||||
if (returnNode) {
|
||||
returnType = returnNode.text.replace(/^:\s*/, '');
|
||||
}
|
||||
// Calculate complexity
|
||||
const bodyText = this.findChild(node, 'statement_block')?.text || '';
|
||||
const complexity = this.calculateComplexity(bodyText);
|
||||
// Find function calls
|
||||
const calls = [];
|
||||
const callRegex = /(\w+)\s*\(/g;
|
||||
let match;
|
||||
while ((match = callRegex.exec(bodyText)) !== null) {
|
||||
if (!['if', 'for', 'while', 'switch', 'catch', 'function'].includes(match[1])) {
|
||||
calls.push(match[1]);
|
||||
}
|
||||
}
|
||||
return {
|
||||
name,
|
||||
params: params.filter(Boolean),
|
||||
returnType,
|
||||
async: isAsync,
|
||||
exported: isExported,
|
||||
startLine: node.startPosition.row + 1,
|
||||
endLine: node.endPosition.row + 1,
|
||||
complexity,
|
||||
calls: [...new Set(calls)],
|
||||
};
|
||||
}
|
||||
catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
parseClass(node, code, lang) {
|
||||
try {
|
||||
const name = this.getIdentifierName(node) || '<anonymous>';
|
||||
let extendsClass;
|
||||
const implementsList = [];
|
||||
const methods = [];
|
||||
const properties = [];
|
||||
// Get extends/implements
|
||||
const heritage = this.findChild(node, 'class_heritage');
|
||||
if (heritage) {
|
||||
const extendsNode = this.findChild(heritage, 'extends_clause');
|
||||
if (extendsNode) {
|
||||
extendsClass = this.getIdentifierName(extendsNode) || undefined;
|
||||
}
|
||||
}
|
||||
// Get methods and properties
|
||||
const body = this.findChild(node, 'class_body');
|
||||
if (body) {
|
||||
for (const member of body.children || []) {
|
||||
if (member.type === 'method_definition') {
|
||||
const method = this.parseFunction(member, code, lang);
|
||||
if (method)
|
||||
methods.push(method);
|
||||
}
|
||||
else if (member.type === 'field_definition' || member.type === 'public_field_definition') {
|
||||
const propName = this.getIdentifierName(member);
|
||||
if (propName)
|
||||
properties.push(propName);
|
||||
}
|
||||
}
|
||||
}
|
||||
return {
|
||||
name,
|
||||
extends: extendsClass,
|
||||
implements: implementsList,
|
||||
methods,
|
||||
properties,
|
||||
exported: node.parent?.type?.includes('export'),
|
||||
startLine: node.startPosition.row + 1,
|
||||
endLine: node.endPosition.row + 1,
|
||||
};
|
||||
}
|
||||
catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
findChild(node, type) {
|
||||
if (!node.children)
|
||||
return null;
|
||||
for (const child of node.children) {
|
||||
if (child.type === type)
|
||||
return child;
|
||||
const found = this.findChild(child, type);
|
||||
if (found)
|
||||
return found;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
getIdentifierName(node) {
|
||||
if (node.type === 'identifier')
|
||||
return node.text;
|
||||
if (!node.children)
|
||||
return null;
|
||||
for (const child of node.children) {
|
||||
if (child.type === 'identifier' || child.type === 'property_identifier') {
|
||||
return child.text;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
calculateComplexity(code) {
|
||||
const patterns = [
|
||||
/\bif\b/g,
|
||||
/\belse\b/g,
|
||||
/\bfor\b/g,
|
||||
/\bwhile\b/g,
|
||||
/\bcase\b/g,
|
||||
/\bcatch\b/g,
|
||||
/\?\s*[^:]/g, // ternary
|
||||
/&&/g,
|
||||
/\|\|/g,
|
||||
];
|
||||
let complexity = 1;
|
||||
for (const pattern of patterns) {
|
||||
complexity += (code.match(pattern) || []).length;
|
||||
}
|
||||
return complexity;
|
||||
}
|
||||
analyzeWithRegex(file, lang, code, start) {
|
||||
const lines = code.split('\n');
|
||||
const imports = [];
|
||||
const exports = [];
|
||||
const functions = [];
|
||||
const classes = [];
|
||||
const variables = [];
|
||||
const types = [];
|
||||
// Regex patterns
|
||||
const importRegex = /import\s+(?:(\w+)\s*,?\s*)?(?:\{([^}]+)\}\s*)?(?:\*\s+as\s+(\w+)\s*)?from\s+['"]([^'"]+)['"]/g;
|
||||
const requireRegex = /(?:const|let|var)\s+(?:(\w+)|\{([^}]+)\})\s*=\s*require\s*\(['"]([^'"]+)['"]\)/g;
|
||||
const exportRegex = /export\s+(?:(default)\s+)?(?:(class|function|const|let|var|interface|type)\s+)?(\w+)?/g;
|
||||
const functionRegex = /(?:export\s+)?(?:async\s+)?function\s+(\w+)\s*\(([^)]*)\)/g;
|
||||
const arrowRegex = /(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>/g;
|
||||
const classRegex = /(?:export\s+)?class\s+(\w+)(?:\s+extends\s+(\w+))?/g;
|
||||
const typeRegex = /(?:export\s+)?(?:type|interface)\s+(\w+)/g;
|
||||
// Parse imports
|
||||
let match;
|
||||
while ((match = importRegex.exec(code)) !== null) {
|
||||
imports.push({
|
||||
source: match[4],
|
||||
default: match[1],
|
||||
named: match[2] ? match[2].split(',').map(s => s.trim().split(/\s+as\s+/)[0]) : [],
|
||||
namespace: match[3],
|
||||
type: 'esm',
|
||||
});
|
||||
}
|
||||
while ((match = requireRegex.exec(code)) !== null) {
|
||||
imports.push({
|
||||
source: match[3],
|
||||
default: match[1],
|
||||
named: match[2] ? match[2].split(',').map(s => s.trim()) : [],
|
||||
type: 'commonjs',
|
||||
});
|
||||
}
|
||||
// Parse exports
|
||||
while ((match = exportRegex.exec(code)) !== null) {
|
||||
if (match[3]) {
|
||||
exports.push({
|
||||
name: match[3],
|
||||
type: match[1] === 'default' ? 'default' : 'named',
|
||||
});
|
||||
}
|
||||
}
|
||||
// Parse functions
|
||||
while ((match = functionRegex.exec(code)) !== null) {
|
||||
functions.push({
|
||||
name: match[1],
|
||||
params: match[2].split(',').map(p => p.trim().split(/[:\s]/)[0]).filter(Boolean),
|
||||
async: code.substring(match.index - 10, match.index).includes('async'),
|
||||
exported: code.substring(match.index - 10, match.index).includes('export'),
|
||||
startLine: code.substring(0, match.index).split('\n').length,
|
||||
endLine: 0,
|
||||
complexity: 1,
|
||||
calls: [],
|
||||
});
|
||||
}
|
||||
while ((match = arrowRegex.exec(code)) !== null) {
|
||||
functions.push({
|
||||
name: match[1],
|
||||
params: [],
|
||||
async: code.substring(match.index, match.index + 50).includes('async'),
|
||||
exported: false,
|
||||
startLine: code.substring(0, match.index).split('\n').length,
|
||||
endLine: 0,
|
||||
complexity: 1,
|
||||
calls: [],
|
||||
});
|
||||
}
|
||||
// Parse classes
|
||||
while ((match = classRegex.exec(code)) !== null) {
|
||||
classes.push({
|
||||
name: match[1],
|
||||
extends: match[2],
|
||||
implements: [],
|
||||
methods: [],
|
||||
properties: [],
|
||||
exported: code.substring(match.index - 10, match.index).includes('export'),
|
||||
startLine: code.substring(0, match.index).split('\n').length,
|
||||
endLine: 0,
|
||||
});
|
||||
}
|
||||
// Parse types
|
||||
while ((match = typeRegex.exec(code)) !== null) {
|
||||
types.push(match[1]);
|
||||
}
|
||||
return {
|
||||
file,
|
||||
language: lang,
|
||||
imports,
|
||||
exports,
|
||||
functions,
|
||||
classes,
|
||||
variables,
|
||||
types,
|
||||
complexity: this.calculateComplexity(code),
|
||||
lines: lines.length,
|
||||
parseTime: performance.now() - start,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Get all symbols (functions, classes, types) in a file
|
||||
*/
|
||||
async getSymbols(file) {
|
||||
const analysis = await this.analyze(file);
|
||||
return [
|
||||
...analysis.functions.map(f => f.name),
|
||||
...analysis.classes.map(c => c.name),
|
||||
...analysis.types,
|
||||
...analysis.variables,
|
||||
];
|
||||
}
|
||||
/**
|
||||
* Get the call graph for a file
|
||||
*/
|
||||
async getCallGraph(file) {
|
||||
const analysis = await this.analyze(file);
|
||||
const graph = new Map();
|
||||
for (const fn of analysis.functions) {
|
||||
graph.set(fn.name, fn.calls);
|
||||
}
|
||||
for (const cls of analysis.classes) {
|
||||
for (const method of cls.methods) {
|
||||
graph.set(`${cls.name}.${method.name}`, method.calls);
|
||||
}
|
||||
}
|
||||
return graph;
|
||||
}
|
||||
}
|
||||
exports.CodeParser = CodeParser;
|
||||
// ============================================================================
|
||||
// Singleton
|
||||
// ============================================================================
|
||||
let parserInstance = null;
|
||||
function getCodeParser() {
|
||||
if (!parserInstance) {
|
||||
parserInstance = new CodeParser();
|
||||
}
|
||||
return parserInstance;
|
||||
}
|
||||
async function initCodeParser() {
|
||||
const parser = getCodeParser();
|
||||
await parser.init();
|
||||
return parser;
|
||||
}
|
||||
exports.default = CodeParser;
|
||||
//# sourceMappingURL=ast-parser.js.map
|
||||
1
npm/packages/ruvector/src/core/ast-parser.js.map
Normal file
1
npm/packages/ruvector/src/core/ast-parser.js.map
Normal file
File diff suppressed because one or more lines are too long
670
npm/packages/ruvector/src/core/ast-parser.ts
Normal file
670
npm/packages/ruvector/src/core/ast-parser.ts
Normal file
@@ -0,0 +1,670 @@
|
||||
/**
|
||||
* AST Parser - Tree-sitter based code parsing
|
||||
*
|
||||
* Provides real AST parsing for accurate code analysis,
|
||||
* replacing regex-based heuristics with proper parsing.
|
||||
*
|
||||
* Supports: TypeScript, JavaScript, Python, Rust, Go, Java, C/C++
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
// Try to load tree-sitter
|
||||
let Parser: any = null;
|
||||
let languages: Map<string, any> = new Map();
|
||||
let parserError: Error | null = null;
|
||||
|
||||
async function loadTreeSitter(): Promise<boolean> {
|
||||
if (Parser) return true;
|
||||
if (parserError) return false;
|
||||
|
||||
try {
|
||||
// Dynamic require to avoid TypeScript errors
|
||||
Parser = require('tree-sitter');
|
||||
return true;
|
||||
} catch (e: any) {
|
||||
parserError = new Error(
|
||||
`tree-sitter not installed: ${e.message}\n` +
|
||||
`Install with: npm install tree-sitter tree-sitter-typescript tree-sitter-javascript tree-sitter-python`
|
||||
);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function loadLanguage(lang: string): Promise<any> {
|
||||
if (languages.has(lang)) return languages.get(lang);
|
||||
|
||||
const langPackages: Record<string, string> = {
|
||||
typescript: 'tree-sitter-typescript',
|
||||
javascript: 'tree-sitter-javascript',
|
||||
python: 'tree-sitter-python',
|
||||
rust: 'tree-sitter-rust',
|
||||
go: 'tree-sitter-go',
|
||||
java: 'tree-sitter-java',
|
||||
c: 'tree-sitter-c',
|
||||
cpp: 'tree-sitter-cpp',
|
||||
ruby: 'tree-sitter-ruby',
|
||||
php: 'tree-sitter-php',
|
||||
};
|
||||
|
||||
const pkg = langPackages[lang];
|
||||
if (!pkg) return null;
|
||||
|
||||
try {
|
||||
const langModule = await import(pkg);
|
||||
const language = langModule.default || langModule;
|
||||
|
||||
// Handle TypeScript which exports tsx and typescript
|
||||
if (lang === 'typescript' && language.typescript) {
|
||||
languages.set(lang, language.typescript);
|
||||
languages.set('tsx', language.tsx);
|
||||
return language.typescript;
|
||||
}
|
||||
|
||||
languages.set(lang, language);
|
||||
return language;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function isTreeSitterAvailable(): boolean {
|
||||
try {
|
||||
require.resolve('tree-sitter');
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Types
|
||||
// ============================================================================
|
||||
|
||||
export interface ASTNode {
|
||||
type: string;
|
||||
text: string;
|
||||
startPosition: { row: number; column: number };
|
||||
endPosition: { row: number; column: number };
|
||||
children: ASTNode[];
|
||||
parent?: string;
|
||||
}
|
||||
|
||||
export interface FunctionInfo {
|
||||
name: string;
|
||||
params: string[];
|
||||
returnType?: string;
|
||||
async: boolean;
|
||||
exported: boolean;
|
||||
startLine: number;
|
||||
endLine: number;
|
||||
complexity: number;
|
||||
calls: string[];
|
||||
}
|
||||
|
||||
export interface ClassInfo {
|
||||
name: string;
|
||||
extends?: string;
|
||||
implements: string[];
|
||||
methods: FunctionInfo[];
|
||||
properties: string[];
|
||||
exported: boolean;
|
||||
startLine: number;
|
||||
endLine: number;
|
||||
}
|
||||
|
||||
export interface ImportInfo {
|
||||
source: string;
|
||||
default?: string;
|
||||
named: string[];
|
||||
namespace?: string;
|
||||
type: 'esm' | 'commonjs' | 'dynamic';
|
||||
}
|
||||
|
||||
export interface ExportInfo {
|
||||
name: string;
|
||||
type: 'default' | 'named' | 'all';
|
||||
source?: string;
|
||||
}
|
||||
|
||||
export interface FileAnalysis {
|
||||
file: string;
|
||||
language: string;
|
||||
imports: ImportInfo[];
|
||||
exports: ExportInfo[];
|
||||
functions: FunctionInfo[];
|
||||
classes: ClassInfo[];
|
||||
variables: string[];
|
||||
types: string[];
|
||||
complexity: number;
|
||||
lines: number;
|
||||
parseTime: number;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Parser
|
||||
// ============================================================================
|
||||
|
||||
export class CodeParser {
|
||||
private parser: any = null;
|
||||
private initialized = false;
|
||||
|
||||
async init(): Promise<boolean> {
|
||||
if (this.initialized) return true;
|
||||
|
||||
const loaded = await loadTreeSitter();
|
||||
if (!loaded) return false;
|
||||
|
||||
this.parser = new Parser();
|
||||
this.initialized = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect language from file extension
|
||||
*/
|
||||
detectLanguage(file: string): string {
|
||||
const ext = path.extname(file).toLowerCase();
|
||||
const langMap: Record<string, string> = {
|
||||
'.ts': 'typescript',
|
||||
'.tsx': 'tsx',
|
||||
'.js': 'javascript',
|
||||
'.jsx': 'javascript',
|
||||
'.mjs': 'javascript',
|
||||
'.cjs': 'javascript',
|
||||
'.py': 'python',
|
||||
'.rs': 'rust',
|
||||
'.go': 'go',
|
||||
'.java': 'java',
|
||||
'.c': 'c',
|
||||
'.h': 'c',
|
||||
'.cpp': 'cpp',
|
||||
'.cc': 'cpp',
|
||||
'.cxx': 'cpp',
|
||||
'.hpp': 'cpp',
|
||||
'.rb': 'ruby',
|
||||
'.php': 'php',
|
||||
};
|
||||
return langMap[ext] || 'unknown';
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a file and return the AST
|
||||
*/
|
||||
async parse(file: string, content?: string): Promise<ASTNode | null> {
|
||||
if (!this.initialized) {
|
||||
await this.init();
|
||||
}
|
||||
if (!this.parser) return null;
|
||||
|
||||
const lang = this.detectLanguage(file);
|
||||
const language = await loadLanguage(lang);
|
||||
if (!language) return null;
|
||||
|
||||
this.parser.setLanguage(language);
|
||||
|
||||
const code = content ?? (fs.existsSync(file) ? fs.readFileSync(file, 'utf8') : '');
|
||||
const tree = this.parser.parse(code);
|
||||
|
||||
return this.convertNode(tree.rootNode);
|
||||
}
|
||||
|
||||
private convertNode(node: any): ASTNode {
|
||||
return {
|
||||
type: node.type,
|
||||
text: node.text,
|
||||
startPosition: node.startPosition,
|
||||
endPosition: node.endPosition,
|
||||
children: node.children?.map((c: any) => this.convertNode(c)) || [],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Analyze a file for functions, classes, imports, etc.
|
||||
*/
|
||||
async analyze(file: string, content?: string): Promise<FileAnalysis> {
|
||||
const start = performance.now();
|
||||
const lang = this.detectLanguage(file);
|
||||
const code = content ?? (fs.existsSync(file) ? fs.readFileSync(file, 'utf8') : '');
|
||||
|
||||
// Try tree-sitter first, fall back to regex
|
||||
if (this.initialized && this.parser) {
|
||||
const language = await loadLanguage(lang);
|
||||
if (language) {
|
||||
this.parser.setLanguage(language);
|
||||
const tree = this.parser.parse(code);
|
||||
return this.analyzeTree(file, lang, tree.rootNode, code, start);
|
||||
}
|
||||
}
|
||||
|
||||
// Regex fallback
|
||||
return this.analyzeWithRegex(file, lang, code, start);
|
||||
}
|
||||
|
||||
private analyzeTree(file: string, lang: string, root: any, code: string, start: number): FileAnalysis {
|
||||
const imports: ImportInfo[] = [];
|
||||
const exports: ExportInfo[] = [];
|
||||
const functions: FunctionInfo[] = [];
|
||||
const classes: ClassInfo[] = [];
|
||||
const variables: string[] = [];
|
||||
const types: string[] = [];
|
||||
|
||||
const visit = (node: any) => {
|
||||
// Imports
|
||||
if (node.type === 'import_statement' || node.type === 'import_declaration') {
|
||||
const imp = this.parseImport(node, lang);
|
||||
if (imp) imports.push(imp);
|
||||
}
|
||||
|
||||
// Exports
|
||||
if (node.type.includes('export')) {
|
||||
const exp = this.parseExport(node, lang);
|
||||
if (exp) exports.push(exp);
|
||||
}
|
||||
|
||||
// Functions
|
||||
if (node.type.includes('function') || node.type === 'method_definition' || node.type === 'arrow_function') {
|
||||
const fn = this.parseFunction(node, code, lang);
|
||||
if (fn) functions.push(fn);
|
||||
}
|
||||
|
||||
// Classes
|
||||
if (node.type === 'class_declaration' || node.type === 'class') {
|
||||
const cls = this.parseClass(node, code, lang);
|
||||
if (cls) classes.push(cls);
|
||||
}
|
||||
|
||||
// Variables
|
||||
if (node.type === 'variable_declarator' || node.type === 'assignment') {
|
||||
const name = this.getIdentifierName(node);
|
||||
if (name) variables.push(name);
|
||||
}
|
||||
|
||||
// Type definitions
|
||||
if (node.type === 'type_alias_declaration' || node.type === 'interface_declaration') {
|
||||
const name = this.getIdentifierName(node);
|
||||
if (name) types.push(name);
|
||||
}
|
||||
|
||||
// Recurse
|
||||
for (const child of node.children || []) {
|
||||
visit(child);
|
||||
}
|
||||
};
|
||||
|
||||
visit(root);
|
||||
|
||||
const lines = code.split('\n').length;
|
||||
const complexity = this.calculateComplexity(code);
|
||||
|
||||
return {
|
||||
file,
|
||||
language: lang,
|
||||
imports,
|
||||
exports,
|
||||
functions,
|
||||
classes,
|
||||
variables,
|
||||
types,
|
||||
complexity,
|
||||
lines,
|
||||
parseTime: performance.now() - start,
|
||||
};
|
||||
}
|
||||
|
||||
private parseImport(node: any, lang: string): ImportInfo | null {
|
||||
try {
|
||||
const source = this.findChild(node, 'string')?.text?.replace(/['"]/g, '') || '';
|
||||
const named: string[] = [];
|
||||
let defaultImport: string | undefined;
|
||||
let namespace: string | undefined;
|
||||
|
||||
// Find import specifiers
|
||||
const specifiers = this.findChild(node, 'import_clause') || node;
|
||||
for (const child of specifiers.children || []) {
|
||||
if (child.type === 'identifier') {
|
||||
defaultImport = child.text;
|
||||
} else if (child.type === 'namespace_import') {
|
||||
namespace = this.getIdentifierName(child) || undefined;
|
||||
} else if (child.type === 'named_imports') {
|
||||
for (const spec of child.children || []) {
|
||||
if (spec.type === 'import_specifier') {
|
||||
named.push(this.getIdentifierName(spec) || '');
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
source,
|
||||
default: defaultImport,
|
||||
named: named.filter(Boolean),
|
||||
namespace,
|
||||
type: 'esm',
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private parseExport(node: any, lang: string): ExportInfo | null {
|
||||
try {
|
||||
if (node.type === 'export_statement') {
|
||||
const declaration = this.findChild(node, 'declaration');
|
||||
if (declaration) {
|
||||
const name = this.getIdentifierName(declaration);
|
||||
return { name: name || 'default', type: node.text.includes('default') ? 'default' : 'named' };
|
||||
}
|
||||
}
|
||||
return null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private parseFunction(node: any, code: string, lang: string): FunctionInfo | null {
|
||||
try {
|
||||
const name = this.getIdentifierName(node) || '<anonymous>';
|
||||
const params: string[] = [];
|
||||
let returnType: string | undefined;
|
||||
const isAsync = node.text.includes('async');
|
||||
const isExported = node.parent?.type?.includes('export');
|
||||
|
||||
// Get parameters
|
||||
const paramsNode = this.findChild(node, 'formal_parameters') || this.findChild(node, 'parameters');
|
||||
if (paramsNode) {
|
||||
for (const param of paramsNode.children || []) {
|
||||
if (param.type === 'identifier' || param.type === 'required_parameter') {
|
||||
params.push(this.getIdentifierName(param) || '');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Get return type
|
||||
const returnNode = this.findChild(node, 'type_annotation');
|
||||
if (returnNode) {
|
||||
returnType = returnNode.text.replace(/^:\s*/, '');
|
||||
}
|
||||
|
||||
// Calculate complexity
|
||||
const bodyText = this.findChild(node, 'statement_block')?.text || '';
|
||||
const complexity = this.calculateComplexity(bodyText);
|
||||
|
||||
// Find function calls
|
||||
const calls: string[] = [];
|
||||
const callRegex = /(\w+)\s*\(/g;
|
||||
let match;
|
||||
while ((match = callRegex.exec(bodyText)) !== null) {
|
||||
if (!['if', 'for', 'while', 'switch', 'catch', 'function'].includes(match[1])) {
|
||||
calls.push(match[1]);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
name,
|
||||
params: params.filter(Boolean),
|
||||
returnType,
|
||||
async: isAsync,
|
||||
exported: isExported,
|
||||
startLine: node.startPosition.row + 1,
|
||||
endLine: node.endPosition.row + 1,
|
||||
complexity,
|
||||
calls: [...new Set(calls)],
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private parseClass(node: any, code: string, lang: string): ClassInfo | null {
|
||||
try {
|
||||
const name = this.getIdentifierName(node) || '<anonymous>';
|
||||
let extendsClass: string | undefined;
|
||||
const implementsList: string[] = [];
|
||||
const methods: FunctionInfo[] = [];
|
||||
const properties: string[] = [];
|
||||
|
||||
// Get extends/implements
|
||||
const heritage = this.findChild(node, 'class_heritage');
|
||||
if (heritage) {
|
||||
const extendsNode = this.findChild(heritage, 'extends_clause');
|
||||
if (extendsNode) {
|
||||
extendsClass = this.getIdentifierName(extendsNode) || undefined;
|
||||
}
|
||||
}
|
||||
|
||||
// Get methods and properties
|
||||
const body = this.findChild(node, 'class_body');
|
||||
if (body) {
|
||||
for (const member of body.children || []) {
|
||||
if (member.type === 'method_definition') {
|
||||
const method = this.parseFunction(member, code, lang);
|
||||
if (method) methods.push(method);
|
||||
} else if (member.type === 'field_definition' || member.type === 'public_field_definition') {
|
||||
const propName = this.getIdentifierName(member);
|
||||
if (propName) properties.push(propName);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
name,
|
||||
extends: extendsClass,
|
||||
implements: implementsList,
|
||||
methods,
|
||||
properties,
|
||||
exported: node.parent?.type?.includes('export'),
|
||||
startLine: node.startPosition.row + 1,
|
||||
endLine: node.endPosition.row + 1,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private findChild(node: any, type: string): any {
|
||||
if (!node.children) return null;
|
||||
for (const child of node.children) {
|
||||
if (child.type === type) return child;
|
||||
const found = this.findChild(child, type);
|
||||
if (found) return found;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private getIdentifierName(node: any): string | null {
|
||||
if (node.type === 'identifier') return node.text;
|
||||
if (!node.children) return null;
|
||||
for (const child of node.children) {
|
||||
if (child.type === 'identifier' || child.type === 'property_identifier') {
|
||||
return child.text;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private calculateComplexity(code: string): number {
|
||||
const patterns = [
|
||||
/\bif\b/g,
|
||||
/\belse\b/g,
|
||||
/\bfor\b/g,
|
||||
/\bwhile\b/g,
|
||||
/\bcase\b/g,
|
||||
/\bcatch\b/g,
|
||||
/\?\s*[^:]/g, // ternary
|
||||
/&&/g,
|
||||
/\|\|/g,
|
||||
];
|
||||
|
||||
let complexity = 1;
|
||||
for (const pattern of patterns) {
|
||||
complexity += (code.match(pattern) || []).length;
|
||||
}
|
||||
return complexity;
|
||||
}
|
||||
|
||||
private analyzeWithRegex(file: string, lang: string, code: string, start: number): FileAnalysis {
|
||||
const lines = code.split('\n');
|
||||
const imports: ImportInfo[] = [];
|
||||
const exports: ExportInfo[] = [];
|
||||
const functions: FunctionInfo[] = [];
|
||||
const classes: ClassInfo[] = [];
|
||||
const variables: string[] = [];
|
||||
const types: string[] = [];
|
||||
|
||||
// Regex patterns
|
||||
const importRegex = /import\s+(?:(\w+)\s*,?\s*)?(?:\{([^}]+)\}\s*)?(?:\*\s+as\s+(\w+)\s*)?from\s+['"]([^'"]+)['"]/g;
|
||||
const requireRegex = /(?:const|let|var)\s+(?:(\w+)|\{([^}]+)\})\s*=\s*require\s*\(['"]([^'"]+)['"]\)/g;
|
||||
const exportRegex = /export\s+(?:(default)\s+)?(?:(class|function|const|let|var|interface|type)\s+)?(\w+)?/g;
|
||||
const functionRegex = /(?:export\s+)?(?:async\s+)?function\s+(\w+)\s*\(([^)]*)\)/g;
|
||||
const arrowRegex = /(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>/g;
|
||||
const classRegex = /(?:export\s+)?class\s+(\w+)(?:\s+extends\s+(\w+))?/g;
|
||||
const typeRegex = /(?:export\s+)?(?:type|interface)\s+(\w+)/g;
|
||||
|
||||
// Parse imports
|
||||
let match;
|
||||
while ((match = importRegex.exec(code)) !== null) {
|
||||
imports.push({
|
||||
source: match[4],
|
||||
default: match[1],
|
||||
named: match[2] ? match[2].split(',').map(s => s.trim().split(/\s+as\s+/)[0]) : [],
|
||||
namespace: match[3],
|
||||
type: 'esm',
|
||||
});
|
||||
}
|
||||
|
||||
while ((match = requireRegex.exec(code)) !== null) {
|
||||
imports.push({
|
||||
source: match[3],
|
||||
default: match[1],
|
||||
named: match[2] ? match[2].split(',').map(s => s.trim()) : [],
|
||||
type: 'commonjs',
|
||||
});
|
||||
}
|
||||
|
||||
// Parse exports
|
||||
while ((match = exportRegex.exec(code)) !== null) {
|
||||
if (match[3]) {
|
||||
exports.push({
|
||||
name: match[3],
|
||||
type: match[1] === 'default' ? 'default' : 'named',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Parse functions
|
||||
while ((match = functionRegex.exec(code)) !== null) {
|
||||
functions.push({
|
||||
name: match[1],
|
||||
params: match[2].split(',').map(p => p.trim().split(/[:\s]/)[0]).filter(Boolean),
|
||||
async: code.substring(match.index - 10, match.index).includes('async'),
|
||||
exported: code.substring(match.index - 10, match.index).includes('export'),
|
||||
startLine: code.substring(0, match.index).split('\n').length,
|
||||
endLine: 0,
|
||||
complexity: 1,
|
||||
calls: [],
|
||||
});
|
||||
}
|
||||
|
||||
while ((match = arrowRegex.exec(code)) !== null) {
|
||||
functions.push({
|
||||
name: match[1],
|
||||
params: [],
|
||||
async: code.substring(match.index, match.index + 50).includes('async'),
|
||||
exported: false,
|
||||
startLine: code.substring(0, match.index).split('\n').length,
|
||||
endLine: 0,
|
||||
complexity: 1,
|
||||
calls: [],
|
||||
});
|
||||
}
|
||||
|
||||
// Parse classes
|
||||
while ((match = classRegex.exec(code)) !== null) {
|
||||
classes.push({
|
||||
name: match[1],
|
||||
extends: match[2],
|
||||
implements: [],
|
||||
methods: [],
|
||||
properties: [],
|
||||
exported: code.substring(match.index - 10, match.index).includes('export'),
|
||||
startLine: code.substring(0, match.index).split('\n').length,
|
||||
endLine: 0,
|
||||
});
|
||||
}
|
||||
|
||||
// Parse types
|
||||
while ((match = typeRegex.exec(code)) !== null) {
|
||||
types.push(match[1]);
|
||||
}
|
||||
|
||||
return {
|
||||
file,
|
||||
language: lang,
|
||||
imports,
|
||||
exports,
|
||||
functions,
|
||||
classes,
|
||||
variables,
|
||||
types,
|
||||
complexity: this.calculateComplexity(code),
|
||||
lines: lines.length,
|
||||
parseTime: performance.now() - start,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all symbols (functions, classes, types) in a file
|
||||
*/
|
||||
async getSymbols(file: string): Promise<string[]> {
|
||||
const analysis = await this.analyze(file);
|
||||
return [
|
||||
...analysis.functions.map(f => f.name),
|
||||
...analysis.classes.map(c => c.name),
|
||||
...analysis.types,
|
||||
...analysis.variables,
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the call graph for a file
|
||||
*/
|
||||
async getCallGraph(file: string): Promise<Map<string, string[]>> {
|
||||
const analysis = await this.analyze(file);
|
||||
const graph = new Map<string, string[]>();
|
||||
|
||||
for (const fn of analysis.functions) {
|
||||
graph.set(fn.name, fn.calls);
|
||||
}
|
||||
|
||||
for (const cls of analysis.classes) {
|
||||
for (const method of cls.methods) {
|
||||
graph.set(`${cls.name}.${method.name}`, method.calls);
|
||||
}
|
||||
}
|
||||
|
||||
return graph;
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Singleton
|
||||
// ============================================================================
|
||||
|
||||
let parserInstance: CodeParser | null = null;
|
||||
|
||||
export function getCodeParser(): CodeParser {
|
||||
if (!parserInstance) {
|
||||
parserInstance = new CodeParser();
|
||||
}
|
||||
return parserInstance;
|
||||
}
|
||||
|
||||
export async function initCodeParser(): Promise<CodeParser> {
|
||||
const parser = getCodeParser();
|
||||
await parser.init();
|
||||
return parser;
|
||||
}
|
||||
|
||||
export default CodeParser;
|
||||
321
npm/packages/ruvector/src/core/attention-fallbacks.d.ts
vendored
Normal file
321
npm/packages/ruvector/src/core/attention-fallbacks.d.ts
vendored
Normal file
@@ -0,0 +1,321 @@
|
||||
/**
|
||||
* Attention Fallbacks - Safe wrapper around @ruvector/attention with automatic array conversion
|
||||
*
|
||||
* This wrapper handles the array type conversion automatically, allowing users
|
||||
* to pass either regular arrays or Float32Arrays.
|
||||
*
|
||||
* @ruvector/attention requires Float32Array inputs.
|
||||
* This wrapper handles the conversion automatically.
|
||||
*/
|
||||
/**
|
||||
* Attention output interface
|
||||
*/
|
||||
export interface AttentionOutput {
|
||||
/** Output vector as regular array */
|
||||
values: number[];
|
||||
/** Output as Float32Array for performance-critical code */
|
||||
raw: Float32Array;
|
||||
}
|
||||
/**
|
||||
* Multi-head attention mechanism
|
||||
*
|
||||
* This wrapper automatically converts array inputs to Float32Array.
|
||||
*/
|
||||
export declare class MultiHeadAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly numHeads: number;
|
||||
/**
|
||||
* Create a new multi-head attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension (must be divisible by numHeads)
|
||||
* @param numHeads - Number of attention heads
|
||||
*/
|
||||
constructor(dim: number, numHeads: number);
|
||||
/**
|
||||
* Compute multi-head attention
|
||||
*
|
||||
* @param query - Query vector
|
||||
* @param keys - Array of key vectors
|
||||
* @param values - Array of value vectors
|
||||
* @returns Attention output
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const mha = new MultiHeadAttention(64, 4);
|
||||
*
|
||||
* // Works with regular arrays
|
||||
* const result1 = mha.compute([...64 values], [[...64], [...64]], [[...64], [...64]]);
|
||||
*
|
||||
* // Also works with Float32Array
|
||||
* const q = new Float32Array(64);
|
||||
* const k = [new Float32Array(64)];
|
||||
* const v = [new Float32Array(64)];
|
||||
* const result2 = mha.compute(q, k, v);
|
||||
* ```
|
||||
*/
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
/**
|
||||
* Compute and return raw Float32Array (faster, no conversion)
|
||||
*/
|
||||
computeRaw(query: Float32Array, keys: Float32Array[], values: Float32Array[]): Float32Array;
|
||||
get headDim(): number;
|
||||
}
|
||||
/**
|
||||
* Flash attention with tiled computation
|
||||
*/
|
||||
export declare class FlashAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly blockSize: number;
|
||||
/**
|
||||
* Create a new flash attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param blockSize - Block size for tiled computation (default: 512)
|
||||
*/
|
||||
constructor(dim: number, blockSize?: number);
|
||||
/**
|
||||
* Compute flash attention
|
||||
*/
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
computeRaw(query: Float32Array, keys: Float32Array[], values: Float32Array[]): Float32Array;
|
||||
}
|
||||
/**
|
||||
* Hyperbolic attention in Poincare ball model
|
||||
*/
|
||||
export declare class HyperbolicAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly curvature: number;
|
||||
/**
|
||||
* Create a new hyperbolic attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param curvature - Hyperbolic curvature (typically 1.0)
|
||||
*/
|
||||
constructor(dim: number, curvature?: number);
|
||||
/**
|
||||
* Compute hyperbolic attention
|
||||
*/
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
computeRaw(query: Float32Array, keys: Float32Array[], values: Float32Array[]): Float32Array;
|
||||
}
|
||||
/**
|
||||
* Linear attention (Performer-style) with O(n) complexity
|
||||
*/
|
||||
export declare class LinearAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly numFeatures: number;
|
||||
/**
|
||||
* Create a new linear attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param numFeatures - Number of random features
|
||||
*/
|
||||
constructor(dim: number, numFeatures: number);
|
||||
/**
|
||||
* Compute linear attention
|
||||
*/
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
computeRaw(query: Float32Array, keys: Float32Array[], values: Float32Array[]): Float32Array;
|
||||
}
|
||||
/**
|
||||
* Local-global attention (Longformer-style)
|
||||
*/
|
||||
export declare class LocalGlobalAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly localWindow: number;
|
||||
readonly globalTokens: number;
|
||||
/**
|
||||
* Create a new local-global attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param localWindow - Size of local attention window
|
||||
* @param globalTokens - Number of global attention tokens
|
||||
*/
|
||||
constructor(dim: number, localWindow: number, globalTokens: number);
|
||||
/**
|
||||
* Compute local-global attention
|
||||
*/
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
computeRaw(query: Float32Array, keys: Float32Array[], values: Float32Array[]): Float32Array;
|
||||
}
|
||||
/**
|
||||
* MoE configuration
|
||||
*/
|
||||
export interface MoEConfig {
|
||||
dim: number;
|
||||
numExperts: number;
|
||||
topK: number;
|
||||
expertCapacity?: number;
|
||||
}
|
||||
/**
|
||||
* Mixture of Experts attention
|
||||
*/
|
||||
export declare class MoEAttention {
|
||||
private inner;
|
||||
readonly config: MoEConfig;
|
||||
/**
|
||||
* Create a new MoE attention instance
|
||||
*
|
||||
* @param config - MoE configuration
|
||||
*/
|
||||
constructor(config: MoEConfig);
|
||||
/**
|
||||
* Create with simple parameters
|
||||
*/
|
||||
static simple(dim: number, numExperts: number, topK: number): MoEAttention;
|
||||
/**
|
||||
* Compute MoE attention
|
||||
*/
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
computeRaw(query: Float32Array, keys: Float32Array[], values: Float32Array[]): Float32Array;
|
||||
}
|
||||
/**
|
||||
* Project a vector into the Poincare ball
|
||||
*/
|
||||
export declare function projectToPoincareBall(vector: number[] | Float32Array, curvature?: number): number[];
|
||||
/**
|
||||
* Compute hyperbolic (Poincare) distance between two points
|
||||
*/
|
||||
export declare function poincareDistance(a: number[] | Float32Array, b: number[] | Float32Array, curvature?: number): number;
|
||||
/**
|
||||
* Mobius addition in hyperbolic space
|
||||
*/
|
||||
export declare function mobiusAddition(a: number[] | Float32Array, b: number[] | Float32Array, curvature?: number): number[];
|
||||
/**
|
||||
* Exponential map from tangent space to hyperbolic space
|
||||
*/
|
||||
export declare function expMap(base: number[] | Float32Array, tangent: number[] | Float32Array, curvature?: number): number[];
|
||||
/**
|
||||
* Logarithmic map from hyperbolic space to tangent space
|
||||
*/
|
||||
export declare function logMap(base: number[] | Float32Array, point: number[] | Float32Array, curvature?: number): number[];
|
||||
/**
|
||||
* Check if attention module is available
|
||||
*/
|
||||
export declare function isAttentionAvailable(): boolean;
|
||||
/**
|
||||
* Get attention module version
|
||||
*/
|
||||
export declare function getAttentionVersion(): string | null;
|
||||
/**
|
||||
* Graph attention with Rotary Position Embeddings
|
||||
* Excellent for code AST and dependency graphs
|
||||
*/
|
||||
export declare class GraphRoPeAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly numHeads: number;
|
||||
readonly maxSeqLen: number;
|
||||
constructor(dim: number, numHeads?: number, maxSeqLen?: number);
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[], positions?: number[]): AttentionOutput;
|
||||
}
|
||||
/**
|
||||
* Edge-featured attention for graphs with edge attributes
|
||||
* Useful for weighted dependency graphs
|
||||
*/
|
||||
export declare class EdgeFeaturedAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly edgeDim: number;
|
||||
constructor(dim: number, edgeDim?: number);
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[], edgeFeatures?: (number[] | Float32Array)[]): AttentionOutput;
|
||||
}
|
||||
/**
|
||||
* Dual-space attention (Euclidean + Hyperbolic)
|
||||
* Best of both worlds for hierarchical + semantic similarity
|
||||
*/
|
||||
export declare class DualSpaceAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
readonly curvature: number;
|
||||
readonly alpha: number;
|
||||
constructor(dim: number, curvature?: number, alpha?: number);
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
}
|
||||
/**
|
||||
* Basic dot-product attention
|
||||
*/
|
||||
export declare class DotProductAttention {
|
||||
private inner;
|
||||
readonly dim: number;
|
||||
constructor(dim: number);
|
||||
compute(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): AttentionOutput;
|
||||
}
|
||||
/**
|
||||
* Compute attention in parallel across multiple queries
|
||||
*/
|
||||
export declare function parallelAttentionCompute(queries: (number[] | Float32Array)[], keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[], attentionType?: 'dot' | 'multi-head' | 'flash' | 'hyperbolic' | 'linear'): Promise<number[][]>;
|
||||
/**
|
||||
* Batch attention compute for multiple query-key-value sets
|
||||
*/
|
||||
export declare function batchAttentionCompute(batches: Array<{
|
||||
query: number[] | Float32Array;
|
||||
keys: (number[] | Float32Array)[];
|
||||
values: (number[] | Float32Array)[];
|
||||
}>, attentionType?: 'dot' | 'multi-head' | 'flash' | 'hyperbolic' | 'linear'): Promise<number[][]>;
|
||||
/**
|
||||
* Async flash attention with callback
|
||||
*/
|
||||
export declare function computeFlashAttentionAsync(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[]): Promise<number[]>;
|
||||
/**
|
||||
* Async hyperbolic attention
|
||||
*/
|
||||
export declare function computeHyperbolicAttentionAsync(query: number[] | Float32Array, keys: (number[] | Float32Array)[], values: (number[] | Float32Array)[], curvature?: number): Promise<number[]>;
|
||||
/**
|
||||
* Adam optimizer for attention training
|
||||
*/
|
||||
export declare class AdamOptimizer {
|
||||
private inner;
|
||||
constructor(learningRate?: number, beta1?: number, beta2?: number);
|
||||
step(gradients: number[] | Float32Array, params: number[] | Float32Array): number[];
|
||||
}
|
||||
/**
|
||||
* InfoNCE contrastive loss
|
||||
*/
|
||||
export declare function infoNceLoss(anchor: number[] | Float32Array, positive: number[] | Float32Array, negatives: (number[] | Float32Array)[], temperature?: number): number;
|
||||
/**
|
||||
* Hard negative mining for contrastive learning
|
||||
*/
|
||||
export declare function mineHardNegatives(anchor: number[] | Float32Array, candidates: (number[] | Float32Array)[], topK?: number): number[][];
|
||||
/**
|
||||
* Benchmark attention implementations
|
||||
*/
|
||||
export declare function benchmarkAttention(dim: number, seqLen: number, iterations?: number): Promise<Record<string, {
|
||||
avgMs: number;
|
||||
minMs: number;
|
||||
maxMs: number;
|
||||
}>>;
|
||||
declare const _default: {
|
||||
DotProductAttention: typeof DotProductAttention;
|
||||
MultiHeadAttention: typeof MultiHeadAttention;
|
||||
FlashAttention: typeof FlashAttention;
|
||||
HyperbolicAttention: typeof HyperbolicAttention;
|
||||
LinearAttention: typeof LinearAttention;
|
||||
LocalGlobalAttention: typeof LocalGlobalAttention;
|
||||
MoEAttention: typeof MoEAttention;
|
||||
GraphRoPeAttention: typeof GraphRoPeAttention;
|
||||
EdgeFeaturedAttention: typeof EdgeFeaturedAttention;
|
||||
DualSpaceAttention: typeof DualSpaceAttention;
|
||||
parallelAttentionCompute: typeof parallelAttentionCompute;
|
||||
batchAttentionCompute: typeof batchAttentionCompute;
|
||||
computeFlashAttentionAsync: typeof computeFlashAttentionAsync;
|
||||
computeHyperbolicAttentionAsync: typeof computeHyperbolicAttentionAsync;
|
||||
AdamOptimizer: typeof AdamOptimizer;
|
||||
infoNceLoss: typeof infoNceLoss;
|
||||
mineHardNegatives: typeof mineHardNegatives;
|
||||
projectToPoincareBall: typeof projectToPoincareBall;
|
||||
poincareDistance: typeof poincareDistance;
|
||||
mobiusAddition: typeof mobiusAddition;
|
||||
expMap: typeof expMap;
|
||||
logMap: typeof logMap;
|
||||
isAttentionAvailable: typeof isAttentionAvailable;
|
||||
getAttentionVersion: typeof getAttentionVersion;
|
||||
benchmarkAttention: typeof benchmarkAttention;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=attention-fallbacks.d.ts.map
|
||||
File diff suppressed because one or more lines are too long
553
npm/packages/ruvector/src/core/attention-fallbacks.js
Normal file
553
npm/packages/ruvector/src/core/attention-fallbacks.js
Normal file
@@ -0,0 +1,553 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Attention Fallbacks - Safe wrapper around @ruvector/attention with automatic array conversion
|
||||
*
|
||||
* This wrapper handles the array type conversion automatically, allowing users
|
||||
* to pass either regular arrays or Float32Arrays.
|
||||
*
|
||||
* @ruvector/attention requires Float32Array inputs.
|
||||
* This wrapper handles the conversion automatically.
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.AdamOptimizer = exports.DotProductAttention = exports.DualSpaceAttention = exports.EdgeFeaturedAttention = exports.GraphRoPeAttention = exports.MoEAttention = exports.LocalGlobalAttention = exports.LinearAttention = exports.HyperbolicAttention = exports.FlashAttention = exports.MultiHeadAttention = void 0;
|
||||
exports.projectToPoincareBall = projectToPoincareBall;
|
||||
exports.poincareDistance = poincareDistance;
|
||||
exports.mobiusAddition = mobiusAddition;
|
||||
exports.expMap = expMap;
|
||||
exports.logMap = logMap;
|
||||
exports.isAttentionAvailable = isAttentionAvailable;
|
||||
exports.getAttentionVersion = getAttentionVersion;
|
||||
exports.parallelAttentionCompute = parallelAttentionCompute;
|
||||
exports.batchAttentionCompute = batchAttentionCompute;
|
||||
exports.computeFlashAttentionAsync = computeFlashAttentionAsync;
|
||||
exports.computeHyperbolicAttentionAsync = computeHyperbolicAttentionAsync;
|
||||
exports.infoNceLoss = infoNceLoss;
|
||||
exports.mineHardNegatives = mineHardNegatives;
|
||||
exports.benchmarkAttention = benchmarkAttention;
|
||||
// Lazy load to avoid import errors if not installed
|
||||
let attentionModule = null;
|
||||
let loadError = null;
|
||||
function getAttentionModule() {
|
||||
if (attentionModule)
|
||||
return attentionModule;
|
||||
if (loadError)
|
||||
throw loadError;
|
||||
try {
|
||||
attentionModule = require('@ruvector/attention');
|
||||
return attentionModule;
|
||||
}
|
||||
catch (e) {
|
||||
loadError = new Error(`@ruvector/attention is not installed or failed to load: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/attention`);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Convert any array-like input to Float32Array
|
||||
*/
|
||||
function toFloat32Array(input) {
|
||||
if (input instanceof Float32Array) {
|
||||
return input;
|
||||
}
|
||||
return new Float32Array(input);
|
||||
}
|
||||
/**
|
||||
* Convert nested arrays to Float32Arrays
|
||||
*/
|
||||
function toFloat32Arrays(inputs) {
|
||||
return inputs.map(arr => toFloat32Array(arr));
|
||||
}
|
||||
/**
|
||||
* Convert Float32Array result back to regular array if needed
|
||||
*/
|
||||
function fromFloat32Array(input) {
|
||||
return Array.from(input);
|
||||
}
|
||||
/**
|
||||
* Multi-head attention mechanism
|
||||
*
|
||||
* This wrapper automatically converts array inputs to Float32Array.
|
||||
*/
|
||||
class MultiHeadAttention {
|
||||
/**
|
||||
* Create a new multi-head attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension (must be divisible by numHeads)
|
||||
* @param numHeads - Number of attention heads
|
||||
*/
|
||||
constructor(dim, numHeads) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.MultiHeadAttention(dim, numHeads);
|
||||
this.dim = dim;
|
||||
this.numHeads = numHeads;
|
||||
}
|
||||
/**
|
||||
* Compute multi-head attention
|
||||
*
|
||||
* @param query - Query vector
|
||||
* @param keys - Array of key vectors
|
||||
* @param values - Array of value vectors
|
||||
* @returns Attention output
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const mha = new MultiHeadAttention(64, 4);
|
||||
*
|
||||
* // Works with regular arrays
|
||||
* const result1 = mha.compute([...64 values], [[...64], [...64]], [[...64], [...64]]);
|
||||
*
|
||||
* // Also works with Float32Array
|
||||
* const q = new Float32Array(64);
|
||||
* const k = [new Float32Array(64)];
|
||||
* const v = [new Float32Array(64)];
|
||||
* const result2 = mha.compute(q, k, v);
|
||||
* ```
|
||||
*/
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Compute and return raw Float32Array (faster, no conversion)
|
||||
*/
|
||||
computeRaw(query, keys, values) {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
get headDim() {
|
||||
return this.dim / this.numHeads;
|
||||
}
|
||||
}
|
||||
exports.MultiHeadAttention = MultiHeadAttention;
|
||||
/**
|
||||
* Flash attention with tiled computation
|
||||
*/
|
||||
class FlashAttention {
|
||||
/**
|
||||
* Create a new flash attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param blockSize - Block size for tiled computation (default: 512)
|
||||
*/
|
||||
constructor(dim, blockSize = 512) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.FlashAttention(dim, blockSize);
|
||||
this.dim = dim;
|
||||
this.blockSize = blockSize;
|
||||
}
|
||||
/**
|
||||
* Compute flash attention
|
||||
*/
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
computeRaw(query, keys, values) {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
exports.FlashAttention = FlashAttention;
|
||||
/**
|
||||
* Hyperbolic attention in Poincare ball model
|
||||
*/
|
||||
class HyperbolicAttention {
|
||||
/**
|
||||
* Create a new hyperbolic attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param curvature - Hyperbolic curvature (typically 1.0)
|
||||
*/
|
||||
constructor(dim, curvature = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.HyperbolicAttention(dim, curvature);
|
||||
this.dim = dim;
|
||||
this.curvature = curvature;
|
||||
}
|
||||
/**
|
||||
* Compute hyperbolic attention
|
||||
*/
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
computeRaw(query, keys, values) {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
exports.HyperbolicAttention = HyperbolicAttention;
|
||||
/**
|
||||
* Linear attention (Performer-style) with O(n) complexity
|
||||
*/
|
||||
class LinearAttention {
|
||||
/**
|
||||
* Create a new linear attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param numFeatures - Number of random features
|
||||
*/
|
||||
constructor(dim, numFeatures) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.LinearAttention(dim, numFeatures);
|
||||
this.dim = dim;
|
||||
this.numFeatures = numFeatures;
|
||||
}
|
||||
/**
|
||||
* Compute linear attention
|
||||
*/
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
computeRaw(query, keys, values) {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
exports.LinearAttention = LinearAttention;
|
||||
/**
|
||||
* Local-global attention (Longformer-style)
|
||||
*/
|
||||
class LocalGlobalAttention {
|
||||
/**
|
||||
* Create a new local-global attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param localWindow - Size of local attention window
|
||||
* @param globalTokens - Number of global attention tokens
|
||||
*/
|
||||
constructor(dim, localWindow, globalTokens) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.LocalGlobalAttention(dim, localWindow, globalTokens);
|
||||
this.dim = dim;
|
||||
this.localWindow = localWindow;
|
||||
this.globalTokens = globalTokens;
|
||||
}
|
||||
/**
|
||||
* Compute local-global attention
|
||||
*/
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
computeRaw(query, keys, values) {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
exports.LocalGlobalAttention = LocalGlobalAttention;
|
||||
/**
|
||||
* Mixture of Experts attention
|
||||
*/
|
||||
class MoEAttention {
|
||||
/**
|
||||
* Create a new MoE attention instance
|
||||
*
|
||||
* @param config - MoE configuration
|
||||
*/
|
||||
constructor(config) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.MoEAttention({
|
||||
dim: config.dim,
|
||||
num_experts: config.numExperts,
|
||||
top_k: config.topK,
|
||||
expert_capacity: config.expertCapacity ?? 1.25,
|
||||
});
|
||||
this.config = config;
|
||||
}
|
||||
/**
|
||||
* Create with simple parameters
|
||||
*/
|
||||
static simple(dim, numExperts, topK) {
|
||||
return new MoEAttention({ dim, numExperts, topK });
|
||||
}
|
||||
/**
|
||||
* Compute MoE attention
|
||||
*/
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
computeRaw(query, keys, values) {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
exports.MoEAttention = MoEAttention;
|
||||
// Hyperbolic math utilities
|
||||
/**
|
||||
* Project a vector into the Poincare ball
|
||||
*/
|
||||
function projectToPoincareBall(vector, curvature = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.projectToPoincareBall(toFloat32Array(vector), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
/**
|
||||
* Compute hyperbolic (Poincare) distance between two points
|
||||
*/
|
||||
function poincareDistance(a, b, curvature = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
return attention.poincareDistance(toFloat32Array(a), toFloat32Array(b), curvature);
|
||||
}
|
||||
/**
|
||||
* Mobius addition in hyperbolic space
|
||||
*/
|
||||
function mobiusAddition(a, b, curvature = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.mobiusAddition(toFloat32Array(a), toFloat32Array(b), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
/**
|
||||
* Exponential map from tangent space to hyperbolic space
|
||||
*/
|
||||
function expMap(base, tangent, curvature = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.expMap(toFloat32Array(base), toFloat32Array(tangent), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
/**
|
||||
* Logarithmic map from hyperbolic space to tangent space
|
||||
*/
|
||||
function logMap(base, point, curvature = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.logMap(toFloat32Array(base), toFloat32Array(point), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
/**
|
||||
* Check if attention module is available
|
||||
*/
|
||||
function isAttentionAvailable() {
|
||||
try {
|
||||
getAttentionModule();
|
||||
return true;
|
||||
}
|
||||
catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Get attention module version
|
||||
*/
|
||||
function getAttentionVersion() {
|
||||
try {
|
||||
const attention = getAttentionModule();
|
||||
return attention.version?.() ?? null;
|
||||
}
|
||||
catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
// ============================================================================
|
||||
// Graph-based Attention (for code structure)
|
||||
// ============================================================================
|
||||
/**
|
||||
* Graph attention with Rotary Position Embeddings
|
||||
* Excellent for code AST and dependency graphs
|
||||
*/
|
||||
class GraphRoPeAttention {
|
||||
constructor(dim, numHeads = 4, maxSeqLen = 4096) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.GraphRoPeAttention(dim, numHeads, maxSeqLen);
|
||||
this.dim = dim;
|
||||
this.numHeads = numHeads;
|
||||
this.maxSeqLen = maxSeqLen;
|
||||
}
|
||||
compute(query, keys, values, positions) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values), positions ? new Int32Array(positions) : undefined);
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
exports.GraphRoPeAttention = GraphRoPeAttention;
|
||||
/**
|
||||
* Edge-featured attention for graphs with edge attributes
|
||||
* Useful for weighted dependency graphs
|
||||
*/
|
||||
class EdgeFeaturedAttention {
|
||||
constructor(dim, edgeDim = 16) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.EdgeFeaturedAttention(dim, edgeDim);
|
||||
this.dim = dim;
|
||||
this.edgeDim = edgeDim;
|
||||
}
|
||||
compute(query, keys, values, edgeFeatures) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values), edgeFeatures ? toFloat32Arrays(edgeFeatures) : undefined);
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
exports.EdgeFeaturedAttention = EdgeFeaturedAttention;
|
||||
/**
|
||||
* Dual-space attention (Euclidean + Hyperbolic)
|
||||
* Best of both worlds for hierarchical + semantic similarity
|
||||
*/
|
||||
class DualSpaceAttention {
|
||||
constructor(dim, curvature = 1.0, alpha = 0.5) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.DualSpaceAttention(dim, curvature, alpha);
|
||||
this.dim = dim;
|
||||
this.curvature = curvature;
|
||||
this.alpha = alpha;
|
||||
}
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
exports.DualSpaceAttention = DualSpaceAttention;
|
||||
/**
|
||||
* Basic dot-product attention
|
||||
*/
|
||||
class DotProductAttention {
|
||||
constructor(dim) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.DotProductAttention(dim);
|
||||
this.dim = dim;
|
||||
}
|
||||
compute(query, keys, values) {
|
||||
const raw = this.inner.compute(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values));
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
exports.DotProductAttention = DotProductAttention;
|
||||
// ============================================================================
|
||||
// Parallel/Batch Attention Compute
|
||||
// ============================================================================
|
||||
/**
|
||||
* Compute attention in parallel across multiple queries
|
||||
*/
|
||||
async function parallelAttentionCompute(queries, keys, values, attentionType = 'multi-head') {
|
||||
const attention = getAttentionModule();
|
||||
const results = await attention.parallelAttentionCompute(toFloat32Arrays(queries), toFloat32Arrays(keys), toFloat32Arrays(values), attentionType);
|
||||
return results.map((r) => fromFloat32Array(r));
|
||||
}
|
||||
/**
|
||||
* Batch attention compute for multiple query-key-value sets
|
||||
*/
|
||||
async function batchAttentionCompute(batches, attentionType = 'multi-head') {
|
||||
const attention = getAttentionModule();
|
||||
const nativeBatches = batches.map(b => ({
|
||||
query: toFloat32Array(b.query),
|
||||
keys: toFloat32Arrays(b.keys),
|
||||
values: toFloat32Arrays(b.values),
|
||||
}));
|
||||
const results = await attention.batchAttentionCompute(nativeBatches, attentionType);
|
||||
return results.map((r) => fromFloat32Array(r));
|
||||
}
|
||||
/**
|
||||
* Async flash attention with callback
|
||||
*/
|
||||
function computeFlashAttentionAsync(query, keys, values) {
|
||||
const attention = getAttentionModule();
|
||||
return new Promise((resolve, reject) => {
|
||||
attention.computeFlashAttentionAsync(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values), (err, result) => {
|
||||
if (err)
|
||||
reject(err);
|
||||
else
|
||||
resolve(fromFloat32Array(result));
|
||||
});
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Async hyperbolic attention
|
||||
*/
|
||||
function computeHyperbolicAttentionAsync(query, keys, values, curvature = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
return new Promise((resolve, reject) => {
|
||||
attention.computeHyperbolicAttentionAsync(toFloat32Array(query), toFloat32Arrays(keys), toFloat32Arrays(values), curvature, (err, result) => {
|
||||
if (err)
|
||||
reject(err);
|
||||
else
|
||||
resolve(fromFloat32Array(result));
|
||||
});
|
||||
});
|
||||
}
|
||||
// ============================================================================
|
||||
// Training Utilities (for SONA integration)
|
||||
// ============================================================================
|
||||
/**
|
||||
* Adam optimizer for attention training
|
||||
*/
|
||||
class AdamOptimizer {
|
||||
constructor(learningRate = 0.001, beta1 = 0.9, beta2 = 0.999) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.AdamOptimizer(learningRate, beta1, beta2);
|
||||
}
|
||||
step(gradients, params) {
|
||||
const result = this.inner.step(toFloat32Array(gradients), toFloat32Array(params));
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
}
|
||||
exports.AdamOptimizer = AdamOptimizer;
|
||||
/**
|
||||
* InfoNCE contrastive loss
|
||||
*/
|
||||
function infoNceLoss(anchor, positive, negatives, temperature = 0.07) {
|
||||
const attention = getAttentionModule();
|
||||
return attention.InfoNceLoss.compute(toFloat32Array(anchor), toFloat32Array(positive), toFloat32Arrays(negatives), temperature);
|
||||
}
|
||||
/**
|
||||
* Hard negative mining for contrastive learning
|
||||
*/
|
||||
function mineHardNegatives(anchor, candidates, topK = 5) {
|
||||
const attention = getAttentionModule();
|
||||
const miner = new attention.HardNegativeMiner(topK);
|
||||
const results = miner.mine(toFloat32Array(anchor), toFloat32Arrays(candidates));
|
||||
return results.map((r) => fromFloat32Array(r));
|
||||
}
|
||||
// ============================================================================
|
||||
// Benchmarking
|
||||
// ============================================================================
|
||||
/**
|
||||
* Benchmark attention implementations
|
||||
*/
|
||||
async function benchmarkAttention(dim, seqLen, iterations = 100) {
|
||||
const attention = getAttentionModule();
|
||||
return attention.benchmarkAttention(dim, seqLen, iterations);
|
||||
}
|
||||
exports.default = {
|
||||
// Core attention types
|
||||
DotProductAttention,
|
||||
MultiHeadAttention,
|
||||
FlashAttention,
|
||||
HyperbolicAttention,
|
||||
LinearAttention,
|
||||
LocalGlobalAttention,
|
||||
MoEAttention,
|
||||
// Graph attention types
|
||||
GraphRoPeAttention,
|
||||
EdgeFeaturedAttention,
|
||||
DualSpaceAttention,
|
||||
// Parallel/batch compute
|
||||
parallelAttentionCompute,
|
||||
batchAttentionCompute,
|
||||
computeFlashAttentionAsync,
|
||||
computeHyperbolicAttentionAsync,
|
||||
// Training utilities
|
||||
AdamOptimizer,
|
||||
infoNceLoss,
|
||||
mineHardNegatives,
|
||||
// Hyperbolic math
|
||||
projectToPoincareBall,
|
||||
poincareDistance,
|
||||
mobiusAddition,
|
||||
expMap,
|
||||
logMap,
|
||||
// Utilities
|
||||
isAttentionAvailable,
|
||||
getAttentionVersion,
|
||||
benchmarkAttention,
|
||||
};
|
||||
//# sourceMappingURL=attention-fallbacks.js.map
|
||||
File diff suppressed because one or more lines are too long
823
npm/packages/ruvector/src/core/attention-fallbacks.ts
Normal file
823
npm/packages/ruvector/src/core/attention-fallbacks.ts
Normal file
@@ -0,0 +1,823 @@
|
||||
/**
|
||||
* Attention Fallbacks - Safe wrapper around @ruvector/attention with automatic array conversion
|
||||
*
|
||||
* This wrapper handles the array type conversion automatically, allowing users
|
||||
* to pass either regular arrays or Float32Arrays.
|
||||
*
|
||||
* @ruvector/attention requires Float32Array inputs.
|
||||
* This wrapper handles the conversion automatically.
|
||||
*/
|
||||
|
||||
// Lazy load to avoid import errors if not installed
|
||||
let attentionModule: any = null;
|
||||
let loadError: Error | null = null;
|
||||
|
||||
function getAttentionModule() {
|
||||
if (attentionModule) return attentionModule;
|
||||
if (loadError) throw loadError;
|
||||
|
||||
try {
|
||||
attentionModule = require('@ruvector/attention');
|
||||
return attentionModule;
|
||||
} catch (e: any) {
|
||||
loadError = new Error(
|
||||
`@ruvector/attention is not installed or failed to load: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/attention`
|
||||
);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert any array-like input to Float32Array
|
||||
*/
|
||||
function toFloat32Array(input: number[] | Float32Array | Float64Array): Float32Array {
|
||||
if (input instanceof Float32Array) {
|
||||
return input;
|
||||
}
|
||||
return new Float32Array(input);
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert nested arrays to Float32Arrays
|
||||
*/
|
||||
function toFloat32Arrays(inputs: (number[] | Float32Array | Float64Array)[]): Float32Array[] {
|
||||
return inputs.map(arr => toFloat32Array(arr));
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert Float32Array result back to regular array if needed
|
||||
*/
|
||||
function fromFloat32Array(input: Float32Array): number[] {
|
||||
return Array.from(input);
|
||||
}
|
||||
|
||||
/**
|
||||
* Attention output interface
|
||||
*/
|
||||
export interface AttentionOutput {
|
||||
/** Output vector as regular array */
|
||||
values: number[];
|
||||
/** Output as Float32Array for performance-critical code */
|
||||
raw: Float32Array;
|
||||
}
|
||||
|
||||
/**
|
||||
* Multi-head attention mechanism
|
||||
*
|
||||
* This wrapper automatically converts array inputs to Float32Array.
|
||||
*/
|
||||
export class MultiHeadAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly numHeads: number;
|
||||
|
||||
/**
|
||||
* Create a new multi-head attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension (must be divisible by numHeads)
|
||||
* @param numHeads - Number of attention heads
|
||||
*/
|
||||
constructor(dim: number, numHeads: number) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.MultiHeadAttention(dim, numHeads);
|
||||
this.dim = dim;
|
||||
this.numHeads = numHeads;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute multi-head attention
|
||||
*
|
||||
* @param query - Query vector
|
||||
* @param keys - Array of key vectors
|
||||
* @param values - Array of value vectors
|
||||
* @returns Attention output
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* const mha = new MultiHeadAttention(64, 4);
|
||||
*
|
||||
* // Works with regular arrays
|
||||
* const result1 = mha.compute([...64 values], [[...64], [...64]], [[...64], [...64]]);
|
||||
*
|
||||
* // Also works with Float32Array
|
||||
* const q = new Float32Array(64);
|
||||
* const k = [new Float32Array(64)];
|
||||
* const v = [new Float32Array(64)];
|
||||
* const result2 = mha.compute(q, k, v);
|
||||
* ```
|
||||
*/
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute and return raw Float32Array (faster, no conversion)
|
||||
*/
|
||||
computeRaw(
|
||||
query: Float32Array,
|
||||
keys: Float32Array[],
|
||||
values: Float32Array[]
|
||||
): Float32Array {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
|
||||
get headDim(): number {
|
||||
return this.dim / this.numHeads;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Flash attention with tiled computation
|
||||
*/
|
||||
export class FlashAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly blockSize: number;
|
||||
|
||||
/**
|
||||
* Create a new flash attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param blockSize - Block size for tiled computation (default: 512)
|
||||
*/
|
||||
constructor(dim: number, blockSize: number = 512) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.FlashAttention(dim, blockSize);
|
||||
this.dim = dim;
|
||||
this.blockSize = blockSize;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute flash attention
|
||||
*/
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
|
||||
computeRaw(
|
||||
query: Float32Array,
|
||||
keys: Float32Array[],
|
||||
values: Float32Array[]
|
||||
): Float32Array {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hyperbolic attention in Poincare ball model
|
||||
*/
|
||||
export class HyperbolicAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly curvature: number;
|
||||
|
||||
/**
|
||||
* Create a new hyperbolic attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param curvature - Hyperbolic curvature (typically 1.0)
|
||||
*/
|
||||
constructor(dim: number, curvature: number = 1.0) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.HyperbolicAttention(dim, curvature);
|
||||
this.dim = dim;
|
||||
this.curvature = curvature;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute hyperbolic attention
|
||||
*/
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
|
||||
computeRaw(
|
||||
query: Float32Array,
|
||||
keys: Float32Array[],
|
||||
values: Float32Array[]
|
||||
): Float32Array {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Linear attention (Performer-style) with O(n) complexity
|
||||
*/
|
||||
export class LinearAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly numFeatures: number;
|
||||
|
||||
/**
|
||||
* Create a new linear attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param numFeatures - Number of random features
|
||||
*/
|
||||
constructor(dim: number, numFeatures: number) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.LinearAttention(dim, numFeatures);
|
||||
this.dim = dim;
|
||||
this.numFeatures = numFeatures;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute linear attention
|
||||
*/
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
|
||||
computeRaw(
|
||||
query: Float32Array,
|
||||
keys: Float32Array[],
|
||||
values: Float32Array[]
|
||||
): Float32Array {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Local-global attention (Longformer-style)
|
||||
*/
|
||||
export class LocalGlobalAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly localWindow: number;
|
||||
public readonly globalTokens: number;
|
||||
|
||||
/**
|
||||
* Create a new local-global attention instance
|
||||
*
|
||||
* @param dim - Embedding dimension
|
||||
* @param localWindow - Size of local attention window
|
||||
* @param globalTokens - Number of global attention tokens
|
||||
*/
|
||||
constructor(dim: number, localWindow: number, globalTokens: number) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.LocalGlobalAttention(dim, localWindow, globalTokens);
|
||||
this.dim = dim;
|
||||
this.localWindow = localWindow;
|
||||
this.globalTokens = globalTokens;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute local-global attention
|
||||
*/
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
|
||||
computeRaw(
|
||||
query: Float32Array,
|
||||
keys: Float32Array[],
|
||||
values: Float32Array[]
|
||||
): Float32Array {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* MoE configuration
|
||||
*/
|
||||
export interface MoEConfig {
|
||||
dim: number;
|
||||
numExperts: number;
|
||||
topK: number;
|
||||
expertCapacity?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mixture of Experts attention
|
||||
*/
|
||||
export class MoEAttention {
|
||||
private inner: any;
|
||||
public readonly config: MoEConfig;
|
||||
|
||||
/**
|
||||
* Create a new MoE attention instance
|
||||
*
|
||||
* @param config - MoE configuration
|
||||
*/
|
||||
constructor(config: MoEConfig) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.MoEAttention({
|
||||
dim: config.dim,
|
||||
num_experts: config.numExperts,
|
||||
top_k: config.topK,
|
||||
expert_capacity: config.expertCapacity ?? 1.25,
|
||||
});
|
||||
this.config = config;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create with simple parameters
|
||||
*/
|
||||
static simple(dim: number, numExperts: number, topK: number): MoEAttention {
|
||||
return new MoEAttention({ dim, numExperts, topK });
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute MoE attention
|
||||
*/
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return {
|
||||
values: fromFloat32Array(raw),
|
||||
raw
|
||||
};
|
||||
}
|
||||
|
||||
computeRaw(
|
||||
query: Float32Array,
|
||||
keys: Float32Array[],
|
||||
values: Float32Array[]
|
||||
): Float32Array {
|
||||
return this.inner.compute(query, keys, values);
|
||||
}
|
||||
}
|
||||
|
||||
// Hyperbolic math utilities
|
||||
|
||||
/**
|
||||
* Project a vector into the Poincare ball
|
||||
*/
|
||||
export function projectToPoincareBall(
|
||||
vector: number[] | Float32Array,
|
||||
curvature: number = 1.0
|
||||
): number[] {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.projectToPoincareBall(toFloat32Array(vector), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute hyperbolic (Poincare) distance between two points
|
||||
*/
|
||||
export function poincareDistance(
|
||||
a: number[] | Float32Array,
|
||||
b: number[] | Float32Array,
|
||||
curvature: number = 1.0
|
||||
): number {
|
||||
const attention = getAttentionModule();
|
||||
return attention.poincareDistance(toFloat32Array(a), toFloat32Array(b), curvature);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mobius addition in hyperbolic space
|
||||
*/
|
||||
export function mobiusAddition(
|
||||
a: number[] | Float32Array,
|
||||
b: number[] | Float32Array,
|
||||
curvature: number = 1.0
|
||||
): number[] {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.mobiusAddition(toFloat32Array(a), toFloat32Array(b), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
|
||||
/**
|
||||
* Exponential map from tangent space to hyperbolic space
|
||||
*/
|
||||
export function expMap(
|
||||
base: number[] | Float32Array,
|
||||
tangent: number[] | Float32Array,
|
||||
curvature: number = 1.0
|
||||
): number[] {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.expMap(toFloat32Array(base), toFloat32Array(tangent), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
|
||||
/**
|
||||
* Logarithmic map from hyperbolic space to tangent space
|
||||
*/
|
||||
export function logMap(
|
||||
base: number[] | Float32Array,
|
||||
point: number[] | Float32Array,
|
||||
curvature: number = 1.0
|
||||
): number[] {
|
||||
const attention = getAttentionModule();
|
||||
const result = attention.logMap(toFloat32Array(base), toFloat32Array(point), curvature);
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if attention module is available
|
||||
*/
|
||||
export function isAttentionAvailable(): boolean {
|
||||
try {
|
||||
getAttentionModule();
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get attention module version
|
||||
*/
|
||||
export function getAttentionVersion(): string | null {
|
||||
try {
|
||||
const attention = getAttentionModule();
|
||||
return attention.version?.() ?? null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Graph-based Attention (for code structure)
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Graph attention with Rotary Position Embeddings
|
||||
* Excellent for code AST and dependency graphs
|
||||
*/
|
||||
export class GraphRoPeAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly numHeads: number;
|
||||
public readonly maxSeqLen: number;
|
||||
|
||||
constructor(dim: number, numHeads: number = 4, maxSeqLen: number = 4096) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.GraphRoPeAttention(dim, numHeads, maxSeqLen);
|
||||
this.dim = dim;
|
||||
this.numHeads = numHeads;
|
||||
this.maxSeqLen = maxSeqLen;
|
||||
}
|
||||
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[],
|
||||
positions?: number[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values),
|
||||
positions ? new Int32Array(positions) : undefined
|
||||
);
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Edge-featured attention for graphs with edge attributes
|
||||
* Useful for weighted dependency graphs
|
||||
*/
|
||||
export class EdgeFeaturedAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly edgeDim: number;
|
||||
|
||||
constructor(dim: number, edgeDim: number = 16) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.EdgeFeaturedAttention(dim, edgeDim);
|
||||
this.dim = dim;
|
||||
this.edgeDim = edgeDim;
|
||||
}
|
||||
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[],
|
||||
edgeFeatures?: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values),
|
||||
edgeFeatures ? toFloat32Arrays(edgeFeatures) : undefined
|
||||
);
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Dual-space attention (Euclidean + Hyperbolic)
|
||||
* Best of both worlds for hierarchical + semantic similarity
|
||||
*/
|
||||
export class DualSpaceAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
public readonly curvature: number;
|
||||
public readonly alpha: number;
|
||||
|
||||
constructor(dim: number, curvature: number = 1.0, alpha: number = 0.5) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.DualSpaceAttention(dim, curvature, alpha);
|
||||
this.dim = dim;
|
||||
this.curvature = curvature;
|
||||
this.alpha = alpha;
|
||||
}
|
||||
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Basic dot-product attention
|
||||
*/
|
||||
export class DotProductAttention {
|
||||
private inner: any;
|
||||
public readonly dim: number;
|
||||
|
||||
constructor(dim: number) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.DotProductAttention(dim);
|
||||
this.dim = dim;
|
||||
}
|
||||
|
||||
compute(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): AttentionOutput {
|
||||
const raw = this.inner.compute(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values)
|
||||
);
|
||||
return { values: fromFloat32Array(raw), raw };
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Parallel/Batch Attention Compute
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Compute attention in parallel across multiple queries
|
||||
*/
|
||||
export async function parallelAttentionCompute(
|
||||
queries: (number[] | Float32Array)[],
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[],
|
||||
attentionType: 'dot' | 'multi-head' | 'flash' | 'hyperbolic' | 'linear' = 'multi-head'
|
||||
): Promise<number[][]> {
|
||||
const attention = getAttentionModule();
|
||||
const results = await attention.parallelAttentionCompute(
|
||||
toFloat32Arrays(queries),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values),
|
||||
attentionType
|
||||
);
|
||||
return results.map((r: Float32Array) => fromFloat32Array(r));
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch attention compute for multiple query-key-value sets
|
||||
*/
|
||||
export async function batchAttentionCompute(
|
||||
batches: Array<{
|
||||
query: number[] | Float32Array;
|
||||
keys: (number[] | Float32Array)[];
|
||||
values: (number[] | Float32Array)[];
|
||||
}>,
|
||||
attentionType: 'dot' | 'multi-head' | 'flash' | 'hyperbolic' | 'linear' = 'multi-head'
|
||||
): Promise<number[][]> {
|
||||
const attention = getAttentionModule();
|
||||
const nativeBatches = batches.map(b => ({
|
||||
query: toFloat32Array(b.query),
|
||||
keys: toFloat32Arrays(b.keys),
|
||||
values: toFloat32Arrays(b.values),
|
||||
}));
|
||||
const results = await attention.batchAttentionCompute(nativeBatches, attentionType);
|
||||
return results.map((r: Float32Array) => fromFloat32Array(r));
|
||||
}
|
||||
|
||||
/**
|
||||
* Async flash attention with callback
|
||||
*/
|
||||
export function computeFlashAttentionAsync(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[]
|
||||
): Promise<number[]> {
|
||||
const attention = getAttentionModule();
|
||||
return new Promise((resolve, reject) => {
|
||||
attention.computeFlashAttentionAsync(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values),
|
||||
(err: Error | null, result: Float32Array) => {
|
||||
if (err) reject(err);
|
||||
else resolve(fromFloat32Array(result));
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Async hyperbolic attention
|
||||
*/
|
||||
export function computeHyperbolicAttentionAsync(
|
||||
query: number[] | Float32Array,
|
||||
keys: (number[] | Float32Array)[],
|
||||
values: (number[] | Float32Array)[],
|
||||
curvature: number = 1.0
|
||||
): Promise<number[]> {
|
||||
const attention = getAttentionModule();
|
||||
return new Promise((resolve, reject) => {
|
||||
attention.computeHyperbolicAttentionAsync(
|
||||
toFloat32Array(query),
|
||||
toFloat32Arrays(keys),
|
||||
toFloat32Arrays(values),
|
||||
curvature,
|
||||
(err: Error | null, result: Float32Array) => {
|
||||
if (err) reject(err);
|
||||
else resolve(fromFloat32Array(result));
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Training Utilities (for SONA integration)
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Adam optimizer for attention training
|
||||
*/
|
||||
export class AdamOptimizer {
|
||||
private inner: any;
|
||||
|
||||
constructor(learningRate: number = 0.001, beta1: number = 0.9, beta2: number = 0.999) {
|
||||
const attention = getAttentionModule();
|
||||
this.inner = new attention.AdamOptimizer(learningRate, beta1, beta2);
|
||||
}
|
||||
|
||||
step(gradients: number[] | Float32Array, params: number[] | Float32Array): number[] {
|
||||
const result = this.inner.step(toFloat32Array(gradients), toFloat32Array(params));
|
||||
return fromFloat32Array(result);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* InfoNCE contrastive loss
|
||||
*/
|
||||
export function infoNceLoss(
|
||||
anchor: number[] | Float32Array,
|
||||
positive: number[] | Float32Array,
|
||||
negatives: (number[] | Float32Array)[],
|
||||
temperature: number = 0.07
|
||||
): number {
|
||||
const attention = getAttentionModule();
|
||||
return attention.InfoNceLoss.compute(
|
||||
toFloat32Array(anchor),
|
||||
toFloat32Array(positive),
|
||||
toFloat32Arrays(negatives),
|
||||
temperature
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Hard negative mining for contrastive learning
|
||||
*/
|
||||
export function mineHardNegatives(
|
||||
anchor: number[] | Float32Array,
|
||||
candidates: (number[] | Float32Array)[],
|
||||
topK: number = 5
|
||||
): number[][] {
|
||||
const attention = getAttentionModule();
|
||||
const miner = new attention.HardNegativeMiner(topK);
|
||||
const results = miner.mine(toFloat32Array(anchor), toFloat32Arrays(candidates));
|
||||
return results.map((r: Float32Array) => fromFloat32Array(r));
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Benchmarking
|
||||
// ============================================================================
|
||||
|
||||
/**
|
||||
* Benchmark attention implementations
|
||||
*/
|
||||
export async function benchmarkAttention(
|
||||
dim: number,
|
||||
seqLen: number,
|
||||
iterations: number = 100
|
||||
): Promise<Record<string, { avgMs: number; minMs: number; maxMs: number }>> {
|
||||
const attention = getAttentionModule();
|
||||
return attention.benchmarkAttention(dim, seqLen, iterations);
|
||||
}
|
||||
|
||||
export default {
|
||||
// Core attention types
|
||||
DotProductAttention,
|
||||
MultiHeadAttention,
|
||||
FlashAttention,
|
||||
HyperbolicAttention,
|
||||
LinearAttention,
|
||||
LocalGlobalAttention,
|
||||
MoEAttention,
|
||||
|
||||
// Graph attention types
|
||||
GraphRoPeAttention,
|
||||
EdgeFeaturedAttention,
|
||||
DualSpaceAttention,
|
||||
|
||||
// Parallel/batch compute
|
||||
parallelAttentionCompute,
|
||||
batchAttentionCompute,
|
||||
computeFlashAttentionAsync,
|
||||
computeHyperbolicAttentionAsync,
|
||||
|
||||
// Training utilities
|
||||
AdamOptimizer,
|
||||
infoNceLoss,
|
||||
mineHardNegatives,
|
||||
|
||||
// Hyperbolic math
|
||||
projectToPoincareBall,
|
||||
poincareDistance,
|
||||
mobiusAddition,
|
||||
expMap,
|
||||
logMap,
|
||||
|
||||
// Utilities
|
||||
isAttentionAvailable,
|
||||
getAttentionVersion,
|
||||
benchmarkAttention,
|
||||
};
|
||||
148
npm/packages/ruvector/src/core/cluster-wrapper.d.ts
vendored
Normal file
148
npm/packages/ruvector/src/core/cluster-wrapper.d.ts
vendored
Normal file
@@ -0,0 +1,148 @@
|
||||
/**
|
||||
* Cluster Wrapper - Distributed coordination for multi-agent systems
|
||||
*
|
||||
* Wraps @ruvector/cluster for Raft consensus, auto-sharding,
|
||||
* and distributed memory across agents.
|
||||
*/
|
||||
export declare function isClusterAvailable(): boolean;
|
||||
export interface ClusterNode {
|
||||
id: string;
|
||||
address: string;
|
||||
role: 'leader' | 'follower' | 'candidate';
|
||||
status: 'healthy' | 'unhealthy' | 'unknown';
|
||||
lastHeartbeat: number;
|
||||
}
|
||||
export interface ShardInfo {
|
||||
id: number;
|
||||
range: [number, number];
|
||||
node: string;
|
||||
size: number;
|
||||
status: 'active' | 'migrating' | 'offline';
|
||||
}
|
||||
export interface ClusterConfig {
|
||||
nodeId: string;
|
||||
address: string;
|
||||
peers?: string[];
|
||||
shards?: number;
|
||||
replicationFactor?: number;
|
||||
}
|
||||
/**
|
||||
* Distributed cluster for multi-agent coordination
|
||||
*/
|
||||
export declare class RuvectorCluster {
|
||||
private inner;
|
||||
private nodeId;
|
||||
private isLeader;
|
||||
constructor(config: ClusterConfig);
|
||||
/**
|
||||
* Start the cluster node
|
||||
*/
|
||||
start(): Promise<void>;
|
||||
/**
|
||||
* Stop the cluster node gracefully
|
||||
*/
|
||||
stop(): Promise<void>;
|
||||
/**
|
||||
* Join an existing cluster
|
||||
*/
|
||||
join(peerAddress: string): Promise<boolean>;
|
||||
/**
|
||||
* Leave the cluster
|
||||
*/
|
||||
leave(): Promise<void>;
|
||||
/**
|
||||
* Get current node info
|
||||
*/
|
||||
getNodeInfo(): ClusterNode;
|
||||
/**
|
||||
* Get all cluster nodes
|
||||
*/
|
||||
getNodes(): ClusterNode[];
|
||||
/**
|
||||
* Check if this node is the leader
|
||||
*/
|
||||
isClusterLeader(): boolean;
|
||||
/**
|
||||
* Get the current leader
|
||||
*/
|
||||
getLeader(): ClusterNode | null;
|
||||
/**
|
||||
* Put a value in distributed storage
|
||||
*/
|
||||
put(key: string, value: any): Promise<boolean>;
|
||||
/**
|
||||
* Get a value from distributed storage
|
||||
*/
|
||||
get(key: string): Promise<any | null>;
|
||||
/**
|
||||
* Delete a value from distributed storage
|
||||
*/
|
||||
delete(key: string): Promise<boolean>;
|
||||
/**
|
||||
* Atomic compare-and-swap
|
||||
*/
|
||||
compareAndSwap(key: string, expected: any, newValue: any): Promise<boolean>;
|
||||
/**
|
||||
* Get shard information
|
||||
*/
|
||||
getShards(): ShardInfo[];
|
||||
/**
|
||||
* Get the shard for a key
|
||||
*/
|
||||
getShardForKey(key: string): ShardInfo;
|
||||
/**
|
||||
* Trigger shard rebalancing
|
||||
*/
|
||||
rebalance(): Promise<void>;
|
||||
/**
|
||||
* Acquire a distributed lock
|
||||
*/
|
||||
lock(name: string, timeout?: number): Promise<string | null>;
|
||||
/**
|
||||
* Release a distributed lock
|
||||
*/
|
||||
unlock(name: string, token: string): Promise<boolean>;
|
||||
/**
|
||||
* Extend a lock's TTL
|
||||
*/
|
||||
extendLock(name: string, token: string, extension?: number): Promise<boolean>;
|
||||
/**
|
||||
* Subscribe to a channel
|
||||
*/
|
||||
subscribe(channel: string, callback: (message: any) => void): () => void;
|
||||
/**
|
||||
* Publish to a channel
|
||||
*/
|
||||
publish(channel: string, message: any): Promise<number>;
|
||||
/**
|
||||
* Register an agent with the cluster
|
||||
*/
|
||||
registerAgent(agentId: string, capabilities: string[]): Promise<boolean>;
|
||||
/**
|
||||
* Find agents with a capability
|
||||
*/
|
||||
findAgents(capability: string): Promise<string[]>;
|
||||
/**
|
||||
* Assign a task to an agent
|
||||
*/
|
||||
assignTask(taskId: string, agentId: string, task: any): Promise<boolean>;
|
||||
/**
|
||||
* Complete a task
|
||||
*/
|
||||
completeTask(taskId: string, result: any): Promise<boolean>;
|
||||
/**
|
||||
* Get cluster statistics
|
||||
*/
|
||||
stats(): {
|
||||
nodes: number;
|
||||
shards: number;
|
||||
leader: string | null;
|
||||
healthy: boolean;
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Create a cluster node for agent coordination
|
||||
*/
|
||||
export declare function createCluster(config: ClusterConfig): RuvectorCluster;
|
||||
export default RuvectorCluster;
|
||||
//# sourceMappingURL=cluster-wrapper.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/cluster-wrapper.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/cluster-wrapper.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"cluster-wrapper.d.ts","sourceRoot":"","sources":["cluster-wrapper.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAqBH,wBAAgB,kBAAkB,IAAI,OAAO,CAO5C;AAED,MAAM,WAAW,WAAW;IAC1B,EAAE,EAAE,MAAM,CAAC;IACX,OAAO,EAAE,MAAM,CAAC;IAChB,IAAI,EAAE,QAAQ,GAAG,UAAU,GAAG,WAAW,CAAC;IAC1C,MAAM,EAAE,SAAS,GAAG,WAAW,GAAG,SAAS,CAAC;IAC5C,aAAa,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,SAAS;IACxB,EAAE,EAAE,MAAM,CAAC;IACX,KAAK,EAAE,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACxB,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,QAAQ,GAAG,WAAW,GAAG,SAAS,CAAC;CAC5C;AAED,MAAM,WAAW,aAAa;IAC5B,MAAM,EAAE,MAAM,CAAC;IACf,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,CAAC,EAAE,MAAM,EAAE,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,iBAAiB,CAAC,EAAE,MAAM,CAAC;CAC5B;AAED;;GAEG;AACH,qBAAa,eAAe;IAC1B,OAAO,CAAC,KAAK,CAAM;IACnB,OAAO,CAAC,MAAM,CAAS;IACvB,OAAO,CAAC,QAAQ,CAAkB;gBAEtB,MAAM,EAAE,aAAa;IAgBjC;;OAEG;IACG,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;IAI5B;;OAEG;IACG,IAAI,IAAI,OAAO,CAAC,IAAI,CAAC;IAI3B;;OAEG;IACG,IAAI,CAAC,WAAW,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAIjD;;OAEG;IACG,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC;IAQ5B;;OAEG;IACH,WAAW,IAAI,WAAW;IAI1B;;OAEG;IACH,QAAQ,IAAI,WAAW,EAAE;IAIzB;;OAEG;IACH,eAAe,IAAI,OAAO;IAK1B;;OAEG;IACH,SAAS,IAAI,WAAW,GAAG,IAAI;IAQ/B;;OAEG;IACG,GAAG,CAAC,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,GAAG,GAAG,OAAO,CAAC,OAAO,CAAC;IAIpD;;OAEG;IACG,GAAG,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,GAAG,GAAG,IAAI,CAAC;IAK3C;;OAEG;IACG,MAAM,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAI3C;;OAEG;IACG,cAAc,CAAC,GAAG,EAAE,MAAM,EAAE,QAAQ,EAAE,GAAG,EAAE,QAAQ,EAAE,GAAG,GAAG,OAAO,CAAC,OAAO,CAAC;IAYjF;;OAEG;IACH,SAAS,IAAI,SAAS,EAAE;IAIxB;;OAEG;IACH,cAAc,CAAC,GAAG,EAAE,MAAM,GAAG,SAAS;IAItC;;OAEG;IACG,SAAS,IAAI,OAAO,CAAC,IAAI,CAAC;IAQhC;;OAEG;IACG,IAAI,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,GAAE,MAAc,GAAG,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IAIzE;;OAEG;IACG,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAI3D;;OAEG;IACG,UAAU,CAAC,IAAI,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,SAAS,GAAE,MAAc,GAAG,OAAO,CAAC,OAAO,CAAC;IAQ1F;;OAEG;IACH,SAAS,CAAC,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,CAAC,OAAO,EAAE,GAAG,KAAK,IAAI,GAAG,MAAM,IAAI;IAMxE;;OAEG;IACG,OAAO,CAAC,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE,GAAG,GAAG,OAAO,CAAC,MAAM,CAAC;IAQ7D;;OAEG;IACG,aAAa,CAAC,OAAO,EAAE,MAAM,EAAE,YAAY,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC,OAAO,CAAC;IAS9E;;OAEG;IACG,UAAU,CAAC,UAAU,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IAcvD;;OAEG;IACG,UAAU,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,IAAI,EAAE,GAAG,GAAG,OAAO,CAAC,OAAO,CAAC;IAgB9E;;OAEG;IACG,YAAY,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,GAAG,OAAO,CAAC,OAAO,CAAC;IAgBjE;;OAEG;IACH,KAAK,IAAI;QACP,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;QACf,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;QACtB,OAAO,EAAE,OAAO,CAAC;KAClB;CAGF;AAED;;GAEG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,aAAa,GAAG,eAAe,CAEpE;AAED,eAAe,eAAe,CAAC"}
|
||||
272
npm/packages/ruvector/src/core/cluster-wrapper.js
Normal file
272
npm/packages/ruvector/src/core/cluster-wrapper.js
Normal file
@@ -0,0 +1,272 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Cluster Wrapper - Distributed coordination for multi-agent systems
|
||||
*
|
||||
* Wraps @ruvector/cluster for Raft consensus, auto-sharding,
|
||||
* and distributed memory across agents.
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.RuvectorCluster = void 0;
|
||||
exports.isClusterAvailable = isClusterAvailable;
|
||||
exports.createCluster = createCluster;
|
||||
let clusterModule = null;
|
||||
let loadError = null;
|
||||
function getClusterModule() {
|
||||
if (clusterModule)
|
||||
return clusterModule;
|
||||
if (loadError)
|
||||
throw loadError;
|
||||
try {
|
||||
clusterModule = require('@ruvector/cluster');
|
||||
return clusterModule;
|
||||
}
|
||||
catch (e) {
|
||||
loadError = new Error(`@ruvector/cluster not installed: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/cluster`);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
function isClusterAvailable() {
|
||||
try {
|
||||
getClusterModule();
|
||||
return true;
|
||||
}
|
||||
catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Distributed cluster for multi-agent coordination
|
||||
*/
|
||||
class RuvectorCluster {
|
||||
constructor(config) {
|
||||
this.isLeader = false;
|
||||
const cluster = getClusterModule();
|
||||
this.nodeId = config.nodeId;
|
||||
this.inner = new cluster.Cluster({
|
||||
nodeId: config.nodeId,
|
||||
address: config.address,
|
||||
peers: config.peers ?? [],
|
||||
shards: config.shards ?? 16,
|
||||
replicationFactor: config.replicationFactor ?? 2,
|
||||
});
|
||||
}
|
||||
// ===========================================================================
|
||||
// Cluster Lifecycle
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Start the cluster node
|
||||
*/
|
||||
async start() {
|
||||
await this.inner.start();
|
||||
}
|
||||
/**
|
||||
* Stop the cluster node gracefully
|
||||
*/
|
||||
async stop() {
|
||||
await this.inner.stop();
|
||||
}
|
||||
/**
|
||||
* Join an existing cluster
|
||||
*/
|
||||
async join(peerAddress) {
|
||||
return this.inner.join(peerAddress);
|
||||
}
|
||||
/**
|
||||
* Leave the cluster
|
||||
*/
|
||||
async leave() {
|
||||
await this.inner.leave();
|
||||
}
|
||||
// ===========================================================================
|
||||
// Node Management
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Get current node info
|
||||
*/
|
||||
getNodeInfo() {
|
||||
return this.inner.getNodeInfo();
|
||||
}
|
||||
/**
|
||||
* Get all cluster nodes
|
||||
*/
|
||||
getNodes() {
|
||||
return this.inner.getNodes();
|
||||
}
|
||||
/**
|
||||
* Check if this node is the leader
|
||||
*/
|
||||
isClusterLeader() {
|
||||
this.isLeader = this.inner.isLeader();
|
||||
return this.isLeader;
|
||||
}
|
||||
/**
|
||||
* Get the current leader
|
||||
*/
|
||||
getLeader() {
|
||||
return this.inner.getLeader();
|
||||
}
|
||||
// ===========================================================================
|
||||
// Distributed Operations
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Put a value in distributed storage
|
||||
*/
|
||||
async put(key, value) {
|
||||
return this.inner.put(key, JSON.stringify(value));
|
||||
}
|
||||
/**
|
||||
* Get a value from distributed storage
|
||||
*/
|
||||
async get(key) {
|
||||
const result = await this.inner.get(key);
|
||||
return result ? JSON.parse(result) : null;
|
||||
}
|
||||
/**
|
||||
* Delete a value from distributed storage
|
||||
*/
|
||||
async delete(key) {
|
||||
return this.inner.delete(key);
|
||||
}
|
||||
/**
|
||||
* Atomic compare-and-swap
|
||||
*/
|
||||
async compareAndSwap(key, expected, newValue) {
|
||||
return this.inner.compareAndSwap(key, JSON.stringify(expected), JSON.stringify(newValue));
|
||||
}
|
||||
// ===========================================================================
|
||||
// Sharding
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Get shard information
|
||||
*/
|
||||
getShards() {
|
||||
return this.inner.getShards();
|
||||
}
|
||||
/**
|
||||
* Get the shard for a key
|
||||
*/
|
||||
getShardForKey(key) {
|
||||
return this.inner.getShardForKey(key);
|
||||
}
|
||||
/**
|
||||
* Trigger shard rebalancing
|
||||
*/
|
||||
async rebalance() {
|
||||
await this.inner.rebalance();
|
||||
}
|
||||
// ===========================================================================
|
||||
// Distributed Locks
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Acquire a distributed lock
|
||||
*/
|
||||
async lock(name, timeout = 30000) {
|
||||
return this.inner.lock(name, timeout);
|
||||
}
|
||||
/**
|
||||
* Release a distributed lock
|
||||
*/
|
||||
async unlock(name, token) {
|
||||
return this.inner.unlock(name, token);
|
||||
}
|
||||
/**
|
||||
* Extend a lock's TTL
|
||||
*/
|
||||
async extendLock(name, token, extension = 30000) {
|
||||
return this.inner.extendLock(name, token, extension);
|
||||
}
|
||||
// ===========================================================================
|
||||
// Pub/Sub
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Subscribe to a channel
|
||||
*/
|
||||
subscribe(channel, callback) {
|
||||
return this.inner.subscribe(channel, (msg) => {
|
||||
callback(JSON.parse(msg));
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Publish to a channel
|
||||
*/
|
||||
async publish(channel, message) {
|
||||
return this.inner.publish(channel, JSON.stringify(message));
|
||||
}
|
||||
// ===========================================================================
|
||||
// Agent Coordination
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Register an agent with the cluster
|
||||
*/
|
||||
async registerAgent(agentId, capabilities) {
|
||||
return this.put(`agent:${agentId}`, {
|
||||
id: agentId,
|
||||
capabilities,
|
||||
node: this.nodeId,
|
||||
registeredAt: Date.now(),
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Find agents with a capability
|
||||
*/
|
||||
async findAgents(capability) {
|
||||
const agents = await this.inner.scan('agent:*');
|
||||
const matching = [];
|
||||
for (const key of agents) {
|
||||
const agent = await this.get(key);
|
||||
if (agent?.capabilities?.includes(capability)) {
|
||||
matching.push(agent.id);
|
||||
}
|
||||
}
|
||||
return matching;
|
||||
}
|
||||
/**
|
||||
* Assign a task to an agent
|
||||
*/
|
||||
async assignTask(taskId, agentId, task) {
|
||||
const assigned = await this.put(`task:${taskId}`, {
|
||||
id: taskId,
|
||||
agent: agentId,
|
||||
task,
|
||||
status: 'assigned',
|
||||
assignedAt: Date.now(),
|
||||
});
|
||||
if (assigned) {
|
||||
await this.publish(`agent:${agentId}:tasks`, { type: 'new_task', taskId });
|
||||
}
|
||||
return assigned;
|
||||
}
|
||||
/**
|
||||
* Complete a task
|
||||
*/
|
||||
async completeTask(taskId, result) {
|
||||
const task = await this.get(`task:${taskId}`);
|
||||
if (!task)
|
||||
return false;
|
||||
return this.put(`task:${taskId}`, {
|
||||
...task,
|
||||
status: 'completed',
|
||||
result,
|
||||
completedAt: Date.now(),
|
||||
});
|
||||
}
|
||||
// ===========================================================================
|
||||
// Stats
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Get cluster statistics
|
||||
*/
|
||||
stats() {
|
||||
return this.inner.stats();
|
||||
}
|
||||
}
|
||||
exports.RuvectorCluster = RuvectorCluster;
|
||||
/**
|
||||
* Create a cluster node for agent coordination
|
||||
*/
|
||||
function createCluster(config) {
|
||||
return new RuvectorCluster(config);
|
||||
}
|
||||
exports.default = RuvectorCluster;
|
||||
//# sourceMappingURL=cluster-wrapper.js.map
|
||||
1
npm/packages/ruvector/src/core/cluster-wrapper.js.map
Normal file
1
npm/packages/ruvector/src/core/cluster-wrapper.js.map
Normal file
File diff suppressed because one or more lines are too long
343
npm/packages/ruvector/src/core/cluster-wrapper.ts
Normal file
343
npm/packages/ruvector/src/core/cluster-wrapper.ts
Normal file
@@ -0,0 +1,343 @@
|
||||
/**
|
||||
* Cluster Wrapper - Distributed coordination for multi-agent systems
|
||||
*
|
||||
* Wraps @ruvector/cluster for Raft consensus, auto-sharding,
|
||||
* and distributed memory across agents.
|
||||
*/
|
||||
|
||||
let clusterModule: any = null;
|
||||
let loadError: Error | null = null;
|
||||
|
||||
function getClusterModule() {
|
||||
if (clusterModule) return clusterModule;
|
||||
if (loadError) throw loadError;
|
||||
|
||||
try {
|
||||
clusterModule = require('@ruvector/cluster');
|
||||
return clusterModule;
|
||||
} catch (e: any) {
|
||||
loadError = new Error(
|
||||
`@ruvector/cluster not installed: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/cluster`
|
||||
);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
|
||||
export function isClusterAvailable(): boolean {
|
||||
try {
|
||||
getClusterModule();
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export interface ClusterNode {
|
||||
id: string;
|
||||
address: string;
|
||||
role: 'leader' | 'follower' | 'candidate';
|
||||
status: 'healthy' | 'unhealthy' | 'unknown';
|
||||
lastHeartbeat: number;
|
||||
}
|
||||
|
||||
export interface ShardInfo {
|
||||
id: number;
|
||||
range: [number, number];
|
||||
node: string;
|
||||
size: number;
|
||||
status: 'active' | 'migrating' | 'offline';
|
||||
}
|
||||
|
||||
export interface ClusterConfig {
|
||||
nodeId: string;
|
||||
address: string;
|
||||
peers?: string[];
|
||||
shards?: number;
|
||||
replicationFactor?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Distributed cluster for multi-agent coordination
|
||||
*/
|
||||
export class RuvectorCluster {
|
||||
private inner: any;
|
||||
private nodeId: string;
|
||||
private isLeader: boolean = false;
|
||||
|
||||
constructor(config: ClusterConfig) {
|
||||
const cluster = getClusterModule();
|
||||
this.nodeId = config.nodeId;
|
||||
this.inner = new cluster.Cluster({
|
||||
nodeId: config.nodeId,
|
||||
address: config.address,
|
||||
peers: config.peers ?? [],
|
||||
shards: config.shards ?? 16,
|
||||
replicationFactor: config.replicationFactor ?? 2,
|
||||
});
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Cluster Lifecycle
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Start the cluster node
|
||||
*/
|
||||
async start(): Promise<void> {
|
||||
await this.inner.start();
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop the cluster node gracefully
|
||||
*/
|
||||
async stop(): Promise<void> {
|
||||
await this.inner.stop();
|
||||
}
|
||||
|
||||
/**
|
||||
* Join an existing cluster
|
||||
*/
|
||||
async join(peerAddress: string): Promise<boolean> {
|
||||
return this.inner.join(peerAddress);
|
||||
}
|
||||
|
||||
/**
|
||||
* Leave the cluster
|
||||
*/
|
||||
async leave(): Promise<void> {
|
||||
await this.inner.leave();
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Node Management
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Get current node info
|
||||
*/
|
||||
getNodeInfo(): ClusterNode {
|
||||
return this.inner.getNodeInfo();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all cluster nodes
|
||||
*/
|
||||
getNodes(): ClusterNode[] {
|
||||
return this.inner.getNodes();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if this node is the leader
|
||||
*/
|
||||
isClusterLeader(): boolean {
|
||||
this.isLeader = this.inner.isLeader();
|
||||
return this.isLeader;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the current leader
|
||||
*/
|
||||
getLeader(): ClusterNode | null {
|
||||
return this.inner.getLeader();
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Distributed Operations
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Put a value in distributed storage
|
||||
*/
|
||||
async put(key: string, value: any): Promise<boolean> {
|
||||
return this.inner.put(key, JSON.stringify(value));
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a value from distributed storage
|
||||
*/
|
||||
async get(key: string): Promise<any | null> {
|
||||
const result = await this.inner.get(key);
|
||||
return result ? JSON.parse(result) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a value from distributed storage
|
||||
*/
|
||||
async delete(key: string): Promise<boolean> {
|
||||
return this.inner.delete(key);
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomic compare-and-swap
|
||||
*/
|
||||
async compareAndSwap(key: string, expected: any, newValue: any): Promise<boolean> {
|
||||
return this.inner.compareAndSwap(
|
||||
key,
|
||||
JSON.stringify(expected),
|
||||
JSON.stringify(newValue)
|
||||
);
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Sharding
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Get shard information
|
||||
*/
|
||||
getShards(): ShardInfo[] {
|
||||
return this.inner.getShards();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the shard for a key
|
||||
*/
|
||||
getShardForKey(key: string): ShardInfo {
|
||||
return this.inner.getShardForKey(key);
|
||||
}
|
||||
|
||||
/**
|
||||
* Trigger shard rebalancing
|
||||
*/
|
||||
async rebalance(): Promise<void> {
|
||||
await this.inner.rebalance();
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Distributed Locks
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Acquire a distributed lock
|
||||
*/
|
||||
async lock(name: string, timeout: number = 30000): Promise<string | null> {
|
||||
return this.inner.lock(name, timeout);
|
||||
}
|
||||
|
||||
/**
|
||||
* Release a distributed lock
|
||||
*/
|
||||
async unlock(name: string, token: string): Promise<boolean> {
|
||||
return this.inner.unlock(name, token);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extend a lock's TTL
|
||||
*/
|
||||
async extendLock(name: string, token: string, extension: number = 30000): Promise<boolean> {
|
||||
return this.inner.extendLock(name, token, extension);
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Pub/Sub
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Subscribe to a channel
|
||||
*/
|
||||
subscribe(channel: string, callback: (message: any) => void): () => void {
|
||||
return this.inner.subscribe(channel, (msg: string) => {
|
||||
callback(JSON.parse(msg));
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Publish to a channel
|
||||
*/
|
||||
async publish(channel: string, message: any): Promise<number> {
|
||||
return this.inner.publish(channel, JSON.stringify(message));
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Agent Coordination
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Register an agent with the cluster
|
||||
*/
|
||||
async registerAgent(agentId: string, capabilities: string[]): Promise<boolean> {
|
||||
return this.put(`agent:${agentId}`, {
|
||||
id: agentId,
|
||||
capabilities,
|
||||
node: this.nodeId,
|
||||
registeredAt: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Find agents with a capability
|
||||
*/
|
||||
async findAgents(capability: string): Promise<string[]> {
|
||||
const agents = await this.inner.scan('agent:*');
|
||||
const matching: string[] = [];
|
||||
|
||||
for (const key of agents) {
|
||||
const agent = await this.get(key);
|
||||
if (agent?.capabilities?.includes(capability)) {
|
||||
matching.push(agent.id);
|
||||
}
|
||||
}
|
||||
|
||||
return matching;
|
||||
}
|
||||
|
||||
/**
|
||||
* Assign a task to an agent
|
||||
*/
|
||||
async assignTask(taskId: string, agentId: string, task: any): Promise<boolean> {
|
||||
const assigned = await this.put(`task:${taskId}`, {
|
||||
id: taskId,
|
||||
agent: agentId,
|
||||
task,
|
||||
status: 'assigned',
|
||||
assignedAt: Date.now(),
|
||||
});
|
||||
|
||||
if (assigned) {
|
||||
await this.publish(`agent:${agentId}:tasks`, { type: 'new_task', taskId });
|
||||
}
|
||||
|
||||
return assigned;
|
||||
}
|
||||
|
||||
/**
|
||||
* Complete a task
|
||||
*/
|
||||
async completeTask(taskId: string, result: any): Promise<boolean> {
|
||||
const task = await this.get(`task:${taskId}`);
|
||||
if (!task) return false;
|
||||
|
||||
return this.put(`task:${taskId}`, {
|
||||
...task,
|
||||
status: 'completed',
|
||||
result,
|
||||
completedAt: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Stats
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Get cluster statistics
|
||||
*/
|
||||
stats(): {
|
||||
nodes: number;
|
||||
shards: number;
|
||||
leader: string | null;
|
||||
healthy: boolean;
|
||||
} {
|
||||
return this.inner.stats();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a cluster node for agent coordination
|
||||
*/
|
||||
export function createCluster(config: ClusterConfig): RuvectorCluster {
|
||||
return new RuvectorCluster(config);
|
||||
}
|
||||
|
||||
export default RuvectorCluster;
|
||||
88
npm/packages/ruvector/src/core/coverage-router.d.ts
vendored
Normal file
88
npm/packages/ruvector/src/core/coverage-router.d.ts
vendored
Normal file
@@ -0,0 +1,88 @@
|
||||
/**
|
||||
* Coverage Router - Test coverage-aware agent routing
|
||||
*
|
||||
* Uses test coverage data to make smarter routing decisions:
|
||||
* - Prioritize testing for uncovered code
|
||||
* - Route to tester agent for low-coverage files
|
||||
* - Suggest test files for modified code
|
||||
*/
|
||||
export interface CoverageData {
|
||||
file: string;
|
||||
lines: {
|
||||
total: number;
|
||||
covered: number;
|
||||
percentage: number;
|
||||
};
|
||||
functions: {
|
||||
total: number;
|
||||
covered: number;
|
||||
percentage: number;
|
||||
};
|
||||
branches: {
|
||||
total: number;
|
||||
covered: number;
|
||||
percentage: number;
|
||||
};
|
||||
uncoveredLines: number[];
|
||||
uncoveredFunctions: string[];
|
||||
}
|
||||
export interface CoverageSummary {
|
||||
files: Map<string, CoverageData>;
|
||||
overall: {
|
||||
lines: number;
|
||||
functions: number;
|
||||
branches: number;
|
||||
};
|
||||
lowCoverageFiles: string[];
|
||||
uncoveredFiles: string[];
|
||||
}
|
||||
export interface TestSuggestion {
|
||||
file: string;
|
||||
testFile: string;
|
||||
reason: string;
|
||||
priority: 'high' | 'medium' | 'low';
|
||||
coverage: number;
|
||||
uncoveredFunctions: string[];
|
||||
}
|
||||
/**
|
||||
* Parse Istanbul/NYC JSON coverage report
|
||||
*/
|
||||
export declare function parseIstanbulCoverage(coveragePath: string): CoverageSummary;
|
||||
/**
|
||||
* Find coverage report in project
|
||||
*/
|
||||
export declare function findCoverageReport(projectRoot?: string): string | null;
|
||||
/**
|
||||
* Get coverage data for a specific file
|
||||
*/
|
||||
export declare function getFileCoverage(file: string, summary?: CoverageSummary): CoverageData | null;
|
||||
/**
|
||||
* Suggest tests for files based on coverage
|
||||
*/
|
||||
export declare function suggestTests(files: string[], summary?: CoverageSummary): TestSuggestion[];
|
||||
/**
|
||||
* Determine if a file needs the tester agent based on coverage
|
||||
*/
|
||||
export declare function shouldRouteToTester(file: string, summary?: CoverageSummary): {
|
||||
route: boolean;
|
||||
reason: string;
|
||||
coverage: number;
|
||||
};
|
||||
/**
|
||||
* Get coverage-aware routing weight for agent selection
|
||||
*/
|
||||
export declare function getCoverageRoutingWeight(file: string, summary?: CoverageSummary): {
|
||||
coder: number;
|
||||
tester: number;
|
||||
reviewer: number;
|
||||
};
|
||||
declare const _default: {
|
||||
parseIstanbulCoverage: typeof parseIstanbulCoverage;
|
||||
findCoverageReport: typeof findCoverageReport;
|
||||
getFileCoverage: typeof getFileCoverage;
|
||||
suggestTests: typeof suggestTests;
|
||||
shouldRouteToTester: typeof shouldRouteToTester;
|
||||
getCoverageRoutingWeight: typeof getCoverageRoutingWeight;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=coverage-router.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/coverage-router.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/coverage-router.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"coverage-router.d.ts","sourceRoot":"","sources":["coverage-router.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAKH,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE;QACL,KAAK,EAAE,MAAM,CAAC;QACd,OAAO,EAAE,MAAM,CAAC;QAChB,UAAU,EAAE,MAAM,CAAC;KACpB,CAAC;IACF,SAAS,EAAE;QACT,KAAK,EAAE,MAAM,CAAC;QACd,OAAO,EAAE,MAAM,CAAC;QAChB,UAAU,EAAE,MAAM,CAAC;KACpB,CAAC;IACF,QAAQ,EAAE;QACR,KAAK,EAAE,MAAM,CAAC;QACd,OAAO,EAAE,MAAM,CAAC;QAChB,UAAU,EAAE,MAAM,CAAC;KACpB,CAAC;IACF,cAAc,EAAE,MAAM,EAAE,CAAC;IACzB,kBAAkB,EAAE,MAAM,EAAE,CAAC;CAC9B;AAED,MAAM,WAAW,eAAe;IAC9B,KAAK,EAAE,GAAG,CAAC,MAAM,EAAE,YAAY,CAAC,CAAC;IACjC,OAAO,EAAE;QACP,KAAK,EAAE,MAAM,CAAC;QACd,SAAS,EAAE,MAAM,CAAC;QAClB,QAAQ,EAAE,MAAM,CAAC;KAClB,CAAC;IACF,gBAAgB,EAAE,MAAM,EAAE,CAAC;IAC3B,cAAc,EAAE,MAAM,EAAE,CAAC;CAC1B;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,GAAG,QAAQ,GAAG,KAAK,CAAC;IACpC,QAAQ,EAAE,MAAM,CAAC;IACjB,kBAAkB,EAAE,MAAM,EAAE,CAAC;CAC9B;AAED;;GAEG;AACH,wBAAgB,qBAAqB,CAAC,YAAY,EAAE,MAAM,GAAG,eAAe,CA0F3E;AAED;;GAEG;AACH,wBAAgB,kBAAkB,CAAC,WAAW,GAAE,MAAsB,GAAG,MAAM,GAAG,IAAI,CAiBrF;AAED;;GAEG;AACH,wBAAgB,eAAe,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,YAAY,GAAG,IAAI,CAqB5F;AAED;;GAEG;AACH,wBAAgB,YAAY,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG,cAAc,EAAE,CAwEzF;AAED;;GAEG;AACH,wBAAgB,mBAAmB,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG;IAC5E,KAAK,EAAE,OAAO,CAAC;IACf,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;CAClB,CAgCA;AAED;;GAEG;AACH,wBAAgB,wBAAwB,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,eAAe,GAAG;IACjF,KAAK,EAAE,MAAM,CAAC;IACd,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;CAClB,CAuBA;;;;;;;;;AAED,wBAOE"}
|
||||
316
npm/packages/ruvector/src/core/coverage-router.js
Normal file
316
npm/packages/ruvector/src/core/coverage-router.js
Normal file
@@ -0,0 +1,316 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Coverage Router - Test coverage-aware agent routing
|
||||
*
|
||||
* Uses test coverage data to make smarter routing decisions:
|
||||
* - Prioritize testing for uncovered code
|
||||
* - Route to tester agent for low-coverage files
|
||||
* - Suggest test files for modified code
|
||||
*/
|
||||
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
var desc = Object.getOwnPropertyDescriptor(m, k);
|
||||
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
||||
desc = { enumerable: true, get: function() { return m[k]; } };
|
||||
}
|
||||
Object.defineProperty(o, k2, desc);
|
||||
}) : (function(o, m, k, k2) {
|
||||
if (k2 === undefined) k2 = k;
|
||||
o[k2] = m[k];
|
||||
}));
|
||||
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
||||
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
||||
}) : function(o, v) {
|
||||
o["default"] = v;
|
||||
});
|
||||
var __importStar = (this && this.__importStar) || (function () {
|
||||
var ownKeys = function(o) {
|
||||
ownKeys = Object.getOwnPropertyNames || function (o) {
|
||||
var ar = [];
|
||||
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
||||
return ar;
|
||||
};
|
||||
return ownKeys(o);
|
||||
};
|
||||
return function (mod) {
|
||||
if (mod && mod.__esModule) return mod;
|
||||
var result = {};
|
||||
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
||||
__setModuleDefault(result, mod);
|
||||
return result;
|
||||
};
|
||||
})();
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.parseIstanbulCoverage = parseIstanbulCoverage;
|
||||
exports.findCoverageReport = findCoverageReport;
|
||||
exports.getFileCoverage = getFileCoverage;
|
||||
exports.suggestTests = suggestTests;
|
||||
exports.shouldRouteToTester = shouldRouteToTester;
|
||||
exports.getCoverageRoutingWeight = getCoverageRoutingWeight;
|
||||
const fs = __importStar(require("fs"));
|
||||
const path = __importStar(require("path"));
|
||||
/**
|
||||
* Parse Istanbul/NYC JSON coverage report
|
||||
*/
|
||||
function parseIstanbulCoverage(coveragePath) {
|
||||
const files = new Map();
|
||||
const lowCoverageFiles = [];
|
||||
const uncoveredFiles = [];
|
||||
let totalLines = 0, coveredLines = 0;
|
||||
let totalFunctions = 0, coveredFunctions = 0;
|
||||
let totalBranches = 0, coveredBranches = 0;
|
||||
try {
|
||||
const coverage = JSON.parse(fs.readFileSync(coveragePath, 'utf8'));
|
||||
for (const [file, data] of Object.entries(coverage)) {
|
||||
// Skip test files
|
||||
if (file.includes('.test.') || file.includes('.spec.') || file.includes('__tests__')) {
|
||||
continue;
|
||||
}
|
||||
// Parse statement coverage
|
||||
const statements = Object.values(data.s || {});
|
||||
const linesCovered = statements.filter(n => n > 0).length;
|
||||
const linesTotal = statements.length;
|
||||
// Parse function coverage
|
||||
const functions = Object.values(data.f || {});
|
||||
const fnCovered = functions.filter(n => n > 0).length;
|
||||
const fnTotal = functions.length;
|
||||
// Parse branch coverage
|
||||
const branches = Object.values(data.b || {}).flat();
|
||||
const brCovered = branches.filter(n => n > 0).length;
|
||||
const brTotal = branches.length;
|
||||
// Find uncovered lines
|
||||
const uncoveredLines = [];
|
||||
for (const [line, count] of Object.entries(data.s || {})) {
|
||||
if (count === 0) {
|
||||
uncoveredLines.push(parseInt(line));
|
||||
}
|
||||
}
|
||||
// Find uncovered functions
|
||||
const uncoveredFunctions = [];
|
||||
const fnMap = data.fnMap || {};
|
||||
for (const [fnId, count] of Object.entries(data.f || {})) {
|
||||
if (count === 0 && fnMap[fnId]) {
|
||||
uncoveredFunctions.push(fnMap[fnId].name || `function_${fnId}`);
|
||||
}
|
||||
}
|
||||
const linePercentage = linesTotal > 0 ? (linesCovered / linesTotal) * 100 : 100;
|
||||
const fnPercentage = fnTotal > 0 ? (fnCovered / fnTotal) * 100 : 100;
|
||||
const brPercentage = brTotal > 0 ? (brCovered / brTotal) * 100 : 100;
|
||||
files.set(file, {
|
||||
file,
|
||||
lines: { total: linesTotal, covered: linesCovered, percentage: linePercentage },
|
||||
functions: { total: fnTotal, covered: fnCovered, percentage: fnPercentage },
|
||||
branches: { total: brTotal, covered: brCovered, percentage: brPercentage },
|
||||
uncoveredLines,
|
||||
uncoveredFunctions,
|
||||
});
|
||||
totalLines += linesTotal;
|
||||
coveredLines += linesCovered;
|
||||
totalFunctions += fnTotal;
|
||||
coveredFunctions += fnCovered;
|
||||
totalBranches += brTotal;
|
||||
coveredBranches += brCovered;
|
||||
if (linePercentage < 50) {
|
||||
lowCoverageFiles.push(file);
|
||||
}
|
||||
if (linePercentage === 0 && linesTotal > 0) {
|
||||
uncoveredFiles.push(file);
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (e) {
|
||||
// Return empty summary on error
|
||||
}
|
||||
return {
|
||||
files,
|
||||
overall: {
|
||||
lines: totalLines > 0 ? (coveredLines / totalLines) * 100 : 0,
|
||||
functions: totalFunctions > 0 ? (coveredFunctions / totalFunctions) * 100 : 0,
|
||||
branches: totalBranches > 0 ? (coveredBranches / totalBranches) * 100 : 0,
|
||||
},
|
||||
lowCoverageFiles,
|
||||
uncoveredFiles,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Find coverage report in project
|
||||
*/
|
||||
function findCoverageReport(projectRoot = process.cwd()) {
|
||||
const possiblePaths = [
|
||||
'coverage/coverage-final.json',
|
||||
'coverage/coverage-summary.json',
|
||||
'.nyc_output/coverage.json',
|
||||
'coverage.json',
|
||||
'coverage/lcov.info',
|
||||
];
|
||||
for (const p of possiblePaths) {
|
||||
const fullPath = path.join(projectRoot, p);
|
||||
if (fs.existsSync(fullPath)) {
|
||||
return fullPath;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
/**
|
||||
* Get coverage data for a specific file
|
||||
*/
|
||||
function getFileCoverage(file, summary) {
|
||||
if (!summary) {
|
||||
const reportPath = findCoverageReport();
|
||||
if (!reportPath)
|
||||
return null;
|
||||
summary = parseIstanbulCoverage(reportPath);
|
||||
}
|
||||
// Try exact match first
|
||||
if (summary.files.has(file)) {
|
||||
return summary.files.get(file);
|
||||
}
|
||||
// Try matching by basename
|
||||
const basename = path.basename(file);
|
||||
for (const [key, data] of summary.files) {
|
||||
if (key.endsWith(file) || key.endsWith(basename)) {
|
||||
return data;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
/**
|
||||
* Suggest tests for files based on coverage
|
||||
*/
|
||||
function suggestTests(files, summary) {
|
||||
if (!summary) {
|
||||
const reportPath = findCoverageReport();
|
||||
if (reportPath) {
|
||||
summary = parseIstanbulCoverage(reportPath);
|
||||
}
|
||||
}
|
||||
const suggestions = [];
|
||||
for (const file of files) {
|
||||
const coverage = summary ? getFileCoverage(file, summary) : null;
|
||||
// Determine test file path
|
||||
const ext = path.extname(file);
|
||||
const base = path.basename(file, ext);
|
||||
const dir = path.dirname(file);
|
||||
const possibleTestFiles = [
|
||||
path.join(dir, `${base}.test${ext}`),
|
||||
path.join(dir, `${base}.spec${ext}`),
|
||||
path.join(dir, '__tests__', `${base}.test${ext}`),
|
||||
path.join('test', `${base}.test${ext}`),
|
||||
path.join('tests', `${base}.test${ext}`),
|
||||
];
|
||||
const existingTestFile = possibleTestFiles.find(t => fs.existsSync(t));
|
||||
const testFile = existingTestFile || possibleTestFiles[0];
|
||||
if (!coverage) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: 'No coverage data - needs test file',
|
||||
priority: 'high',
|
||||
coverage: 0,
|
||||
uncoveredFunctions: [],
|
||||
});
|
||||
}
|
||||
else if (coverage.lines.percentage < 30) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: `Very low coverage (${coverage.lines.percentage.toFixed(1)}%)`,
|
||||
priority: 'high',
|
||||
coverage: coverage.lines.percentage,
|
||||
uncoveredFunctions: coverage.uncoveredFunctions,
|
||||
});
|
||||
}
|
||||
else if (coverage.lines.percentage < 70) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: `Low coverage (${coverage.lines.percentage.toFixed(1)}%)`,
|
||||
priority: 'medium',
|
||||
coverage: coverage.lines.percentage,
|
||||
uncoveredFunctions: coverage.uncoveredFunctions,
|
||||
});
|
||||
}
|
||||
else if (coverage.uncoveredFunctions.length > 0) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: `${coverage.uncoveredFunctions.length} untested functions`,
|
||||
priority: 'low',
|
||||
coverage: coverage.lines.percentage,
|
||||
uncoveredFunctions: coverage.uncoveredFunctions,
|
||||
});
|
||||
}
|
||||
}
|
||||
return suggestions.sort((a, b) => {
|
||||
const priorityOrder = { high: 0, medium: 1, low: 2 };
|
||||
return priorityOrder[a.priority] - priorityOrder[b.priority];
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Determine if a file needs the tester agent based on coverage
|
||||
*/
|
||||
function shouldRouteToTester(file, summary) {
|
||||
const coverage = getFileCoverage(file, summary);
|
||||
if (!coverage) {
|
||||
return {
|
||||
route: true,
|
||||
reason: 'No test coverage data available',
|
||||
coverage: 0,
|
||||
};
|
||||
}
|
||||
if (coverage.lines.percentage < 50) {
|
||||
return {
|
||||
route: true,
|
||||
reason: `Low coverage: ${coverage.lines.percentage.toFixed(1)}%`,
|
||||
coverage: coverage.lines.percentage,
|
||||
};
|
||||
}
|
||||
if (coverage.uncoveredFunctions.length > 3) {
|
||||
return {
|
||||
route: true,
|
||||
reason: `${coverage.uncoveredFunctions.length} untested functions`,
|
||||
coverage: coverage.lines.percentage,
|
||||
};
|
||||
}
|
||||
return {
|
||||
route: false,
|
||||
reason: `Adequate coverage: ${coverage.lines.percentage.toFixed(1)}%`,
|
||||
coverage: coverage.lines.percentage,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Get coverage-aware routing weight for agent selection
|
||||
*/
|
||||
function getCoverageRoutingWeight(file, summary) {
|
||||
const coverage = getFileCoverage(file, summary);
|
||||
if (!coverage) {
|
||||
// No coverage = prioritize testing
|
||||
return { coder: 0.3, tester: 0.5, reviewer: 0.2 };
|
||||
}
|
||||
const pct = coverage.lines.percentage;
|
||||
if (pct < 30) {
|
||||
// Very low - strongly prioritize testing
|
||||
return { coder: 0.2, tester: 0.6, reviewer: 0.2 };
|
||||
}
|
||||
else if (pct < 60) {
|
||||
// Low - moderate testing priority
|
||||
return { coder: 0.4, tester: 0.4, reviewer: 0.2 };
|
||||
}
|
||||
else if (pct < 80) {
|
||||
// Okay - balanced
|
||||
return { coder: 0.5, tester: 0.3, reviewer: 0.2 };
|
||||
}
|
||||
else {
|
||||
// Good - focus on code quality
|
||||
return { coder: 0.5, tester: 0.2, reviewer: 0.3 };
|
||||
}
|
||||
}
|
||||
exports.default = {
|
||||
parseIstanbulCoverage,
|
||||
findCoverageReport,
|
||||
getFileCoverage,
|
||||
suggestTests,
|
||||
shouldRouteToTester,
|
||||
getCoverageRoutingWeight,
|
||||
};
|
||||
//# sourceMappingURL=coverage-router.js.map
|
||||
1
npm/packages/ruvector/src/core/coverage-router.js.map
Normal file
1
npm/packages/ruvector/src/core/coverage-router.js.map
Normal file
File diff suppressed because one or more lines are too long
354
npm/packages/ruvector/src/core/coverage-router.ts
Normal file
354
npm/packages/ruvector/src/core/coverage-router.ts
Normal file
@@ -0,0 +1,354 @@
|
||||
/**
|
||||
* Coverage Router - Test coverage-aware agent routing
|
||||
*
|
||||
* Uses test coverage data to make smarter routing decisions:
|
||||
* - Prioritize testing for uncovered code
|
||||
* - Route to tester agent for low-coverage files
|
||||
* - Suggest test files for modified code
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
export interface CoverageData {
|
||||
file: string;
|
||||
lines: {
|
||||
total: number;
|
||||
covered: number;
|
||||
percentage: number;
|
||||
};
|
||||
functions: {
|
||||
total: number;
|
||||
covered: number;
|
||||
percentage: number;
|
||||
};
|
||||
branches: {
|
||||
total: number;
|
||||
covered: number;
|
||||
percentage: number;
|
||||
};
|
||||
uncoveredLines: number[];
|
||||
uncoveredFunctions: string[];
|
||||
}
|
||||
|
||||
export interface CoverageSummary {
|
||||
files: Map<string, CoverageData>;
|
||||
overall: {
|
||||
lines: number;
|
||||
functions: number;
|
||||
branches: number;
|
||||
};
|
||||
lowCoverageFiles: string[];
|
||||
uncoveredFiles: string[];
|
||||
}
|
||||
|
||||
export interface TestSuggestion {
|
||||
file: string;
|
||||
testFile: string;
|
||||
reason: string;
|
||||
priority: 'high' | 'medium' | 'low';
|
||||
coverage: number;
|
||||
uncoveredFunctions: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse Istanbul/NYC JSON coverage report
|
||||
*/
|
||||
export function parseIstanbulCoverage(coveragePath: string): CoverageSummary {
|
||||
const files = new Map<string, CoverageData>();
|
||||
const lowCoverageFiles: string[] = [];
|
||||
const uncoveredFiles: string[] = [];
|
||||
let totalLines = 0, coveredLines = 0;
|
||||
let totalFunctions = 0, coveredFunctions = 0;
|
||||
let totalBranches = 0, coveredBranches = 0;
|
||||
|
||||
try {
|
||||
const coverage = JSON.parse(fs.readFileSync(coveragePath, 'utf8'));
|
||||
|
||||
for (const [file, data] of Object.entries(coverage) as [string, any][]) {
|
||||
// Skip test files
|
||||
if (file.includes('.test.') || file.includes('.spec.') || file.includes('__tests__')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Parse statement coverage
|
||||
const statements = Object.values(data.s || {}) as number[];
|
||||
const linesCovered = statements.filter(n => n > 0).length;
|
||||
const linesTotal = statements.length;
|
||||
|
||||
// Parse function coverage
|
||||
const functions = Object.values(data.f || {}) as number[];
|
||||
const fnCovered = functions.filter(n => n > 0).length;
|
||||
const fnTotal = functions.length;
|
||||
|
||||
// Parse branch coverage
|
||||
const branches = Object.values(data.b || {}).flat() as number[];
|
||||
const brCovered = branches.filter(n => n > 0).length;
|
||||
const brTotal = branches.length;
|
||||
|
||||
// Find uncovered lines
|
||||
const uncoveredLines: number[] = [];
|
||||
for (const [line, count] of Object.entries(data.s || {})) {
|
||||
if (count === 0) {
|
||||
uncoveredLines.push(parseInt(line));
|
||||
}
|
||||
}
|
||||
|
||||
// Find uncovered functions
|
||||
const uncoveredFunctions: string[] = [];
|
||||
const fnMap = data.fnMap || {};
|
||||
for (const [fnId, count] of Object.entries(data.f || {})) {
|
||||
if (count === 0 && fnMap[fnId]) {
|
||||
uncoveredFunctions.push(fnMap[fnId].name || `function_${fnId}`);
|
||||
}
|
||||
}
|
||||
|
||||
const linePercentage = linesTotal > 0 ? (linesCovered / linesTotal) * 100 : 100;
|
||||
const fnPercentage = fnTotal > 0 ? (fnCovered / fnTotal) * 100 : 100;
|
||||
const brPercentage = brTotal > 0 ? (brCovered / brTotal) * 100 : 100;
|
||||
|
||||
files.set(file, {
|
||||
file,
|
||||
lines: { total: linesTotal, covered: linesCovered, percentage: linePercentage },
|
||||
functions: { total: fnTotal, covered: fnCovered, percentage: fnPercentage },
|
||||
branches: { total: brTotal, covered: brCovered, percentage: brPercentage },
|
||||
uncoveredLines,
|
||||
uncoveredFunctions,
|
||||
});
|
||||
|
||||
totalLines += linesTotal;
|
||||
coveredLines += linesCovered;
|
||||
totalFunctions += fnTotal;
|
||||
coveredFunctions += fnCovered;
|
||||
totalBranches += brTotal;
|
||||
coveredBranches += brCovered;
|
||||
|
||||
if (linePercentage < 50) {
|
||||
lowCoverageFiles.push(file);
|
||||
}
|
||||
if (linePercentage === 0 && linesTotal > 0) {
|
||||
uncoveredFiles.push(file);
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
// Return empty summary on error
|
||||
}
|
||||
|
||||
return {
|
||||
files,
|
||||
overall: {
|
||||
lines: totalLines > 0 ? (coveredLines / totalLines) * 100 : 0,
|
||||
functions: totalFunctions > 0 ? (coveredFunctions / totalFunctions) * 100 : 0,
|
||||
branches: totalBranches > 0 ? (coveredBranches / totalBranches) * 100 : 0,
|
||||
},
|
||||
lowCoverageFiles,
|
||||
uncoveredFiles,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find coverage report in project
|
||||
*/
|
||||
export function findCoverageReport(projectRoot: string = process.cwd()): string | null {
|
||||
const possiblePaths = [
|
||||
'coverage/coverage-final.json',
|
||||
'coverage/coverage-summary.json',
|
||||
'.nyc_output/coverage.json',
|
||||
'coverage.json',
|
||||
'coverage/lcov.info',
|
||||
];
|
||||
|
||||
for (const p of possiblePaths) {
|
||||
const fullPath = path.join(projectRoot, p);
|
||||
if (fs.existsSync(fullPath)) {
|
||||
return fullPath;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get coverage data for a specific file
|
||||
*/
|
||||
export function getFileCoverage(file: string, summary?: CoverageSummary): CoverageData | null {
|
||||
if (!summary) {
|
||||
const reportPath = findCoverageReport();
|
||||
if (!reportPath) return null;
|
||||
summary = parseIstanbulCoverage(reportPath);
|
||||
}
|
||||
|
||||
// Try exact match first
|
||||
if (summary.files.has(file)) {
|
||||
return summary.files.get(file)!;
|
||||
}
|
||||
|
||||
// Try matching by basename
|
||||
const basename = path.basename(file);
|
||||
for (const [key, data] of summary.files) {
|
||||
if (key.endsWith(file) || key.endsWith(basename)) {
|
||||
return data;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Suggest tests for files based on coverage
|
||||
*/
|
||||
export function suggestTests(files: string[], summary?: CoverageSummary): TestSuggestion[] {
|
||||
if (!summary) {
|
||||
const reportPath = findCoverageReport();
|
||||
if (reportPath) {
|
||||
summary = parseIstanbulCoverage(reportPath);
|
||||
}
|
||||
}
|
||||
|
||||
const suggestions: TestSuggestion[] = [];
|
||||
|
||||
for (const file of files) {
|
||||
const coverage = summary ? getFileCoverage(file, summary) : null;
|
||||
|
||||
// Determine test file path
|
||||
const ext = path.extname(file);
|
||||
const base = path.basename(file, ext);
|
||||
const dir = path.dirname(file);
|
||||
|
||||
const possibleTestFiles = [
|
||||
path.join(dir, `${base}.test${ext}`),
|
||||
path.join(dir, `${base}.spec${ext}`),
|
||||
path.join(dir, '__tests__', `${base}.test${ext}`),
|
||||
path.join('test', `${base}.test${ext}`),
|
||||
path.join('tests', `${base}.test${ext}`),
|
||||
];
|
||||
|
||||
const existingTestFile = possibleTestFiles.find(t => fs.existsSync(t));
|
||||
const testFile = existingTestFile || possibleTestFiles[0];
|
||||
|
||||
if (!coverage) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: 'No coverage data - needs test file',
|
||||
priority: 'high',
|
||||
coverage: 0,
|
||||
uncoveredFunctions: [],
|
||||
});
|
||||
} else if (coverage.lines.percentage < 30) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: `Very low coverage (${coverage.lines.percentage.toFixed(1)}%)`,
|
||||
priority: 'high',
|
||||
coverage: coverage.lines.percentage,
|
||||
uncoveredFunctions: coverage.uncoveredFunctions,
|
||||
});
|
||||
} else if (coverage.lines.percentage < 70) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: `Low coverage (${coverage.lines.percentage.toFixed(1)}%)`,
|
||||
priority: 'medium',
|
||||
coverage: coverage.lines.percentage,
|
||||
uncoveredFunctions: coverage.uncoveredFunctions,
|
||||
});
|
||||
} else if (coverage.uncoveredFunctions.length > 0) {
|
||||
suggestions.push({
|
||||
file,
|
||||
testFile,
|
||||
reason: `${coverage.uncoveredFunctions.length} untested functions`,
|
||||
priority: 'low',
|
||||
coverage: coverage.lines.percentage,
|
||||
uncoveredFunctions: coverage.uncoveredFunctions,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return suggestions.sort((a, b) => {
|
||||
const priorityOrder = { high: 0, medium: 1, low: 2 };
|
||||
return priorityOrder[a.priority] - priorityOrder[b.priority];
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Determine if a file needs the tester agent based on coverage
|
||||
*/
|
||||
export function shouldRouteToTester(file: string, summary?: CoverageSummary): {
|
||||
route: boolean;
|
||||
reason: string;
|
||||
coverage: number;
|
||||
} {
|
||||
const coverage = getFileCoverage(file, summary);
|
||||
|
||||
if (!coverage) {
|
||||
return {
|
||||
route: true,
|
||||
reason: 'No test coverage data available',
|
||||
coverage: 0,
|
||||
};
|
||||
}
|
||||
|
||||
if (coverage.lines.percentage < 50) {
|
||||
return {
|
||||
route: true,
|
||||
reason: `Low coverage: ${coverage.lines.percentage.toFixed(1)}%`,
|
||||
coverage: coverage.lines.percentage,
|
||||
};
|
||||
}
|
||||
|
||||
if (coverage.uncoveredFunctions.length > 3) {
|
||||
return {
|
||||
route: true,
|
||||
reason: `${coverage.uncoveredFunctions.length} untested functions`,
|
||||
coverage: coverage.lines.percentage,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
route: false,
|
||||
reason: `Adequate coverage: ${coverage.lines.percentage.toFixed(1)}%`,
|
||||
coverage: coverage.lines.percentage,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get coverage-aware routing weight for agent selection
|
||||
*/
|
||||
export function getCoverageRoutingWeight(file: string, summary?: CoverageSummary): {
|
||||
coder: number;
|
||||
tester: number;
|
||||
reviewer: number;
|
||||
} {
|
||||
const coverage = getFileCoverage(file, summary);
|
||||
|
||||
if (!coverage) {
|
||||
// No coverage = prioritize testing
|
||||
return { coder: 0.3, tester: 0.5, reviewer: 0.2 };
|
||||
}
|
||||
|
||||
const pct = coverage.lines.percentage;
|
||||
|
||||
if (pct < 30) {
|
||||
// Very low - strongly prioritize testing
|
||||
return { coder: 0.2, tester: 0.6, reviewer: 0.2 };
|
||||
} else if (pct < 60) {
|
||||
// Low - moderate testing priority
|
||||
return { coder: 0.4, tester: 0.4, reviewer: 0.2 };
|
||||
} else if (pct < 80) {
|
||||
// Okay - balanced
|
||||
return { coder: 0.5, tester: 0.3, reviewer: 0.2 };
|
||||
} else {
|
||||
// Good - focus on code quality
|
||||
return { coder: 0.5, tester: 0.2, reviewer: 0.3 };
|
||||
}
|
||||
}
|
||||
|
||||
export default {
|
||||
parseIstanbulCoverage,
|
||||
findCoverageReport,
|
||||
getFileCoverage,
|
||||
suggestTests,
|
||||
shouldRouteToTester,
|
||||
getCoverageRoutingWeight,
|
||||
};
|
||||
93
npm/packages/ruvector/src/core/diff-embeddings.d.ts
vendored
Normal file
93
npm/packages/ruvector/src/core/diff-embeddings.d.ts
vendored
Normal file
@@ -0,0 +1,93 @@
|
||||
/**
|
||||
* Diff Embeddings - Semantic encoding of git diffs
|
||||
*
|
||||
* Generates embeddings for code changes to enable:
|
||||
* - Change classification (feature, bugfix, refactor)
|
||||
* - Similar change detection
|
||||
* - Risk assessment
|
||||
* - Review prioritization
|
||||
*/
|
||||
export interface DiffHunk {
|
||||
file: string;
|
||||
oldStart: number;
|
||||
oldLines: number;
|
||||
newStart: number;
|
||||
newLines: number;
|
||||
content: string;
|
||||
additions: string[];
|
||||
deletions: string[];
|
||||
}
|
||||
export interface DiffAnalysis {
|
||||
file: string;
|
||||
hunks: DiffHunk[];
|
||||
totalAdditions: number;
|
||||
totalDeletions: number;
|
||||
complexity: number;
|
||||
riskScore: number;
|
||||
category: 'feature' | 'bugfix' | 'refactor' | 'docs' | 'test' | 'config' | 'unknown';
|
||||
embedding?: number[];
|
||||
}
|
||||
export interface CommitAnalysis {
|
||||
hash: string;
|
||||
message: string;
|
||||
author: string;
|
||||
date: string;
|
||||
files: DiffAnalysis[];
|
||||
totalAdditions: number;
|
||||
totalDeletions: number;
|
||||
riskScore: number;
|
||||
embedding?: number[];
|
||||
}
|
||||
/**
|
||||
* Parse a unified diff into hunks
|
||||
*/
|
||||
export declare function parseDiff(diff: string): DiffHunk[];
|
||||
/**
|
||||
* Classify a change based on patterns
|
||||
*/
|
||||
export declare function classifyChange(diff: string, message?: string): 'feature' | 'bugfix' | 'refactor' | 'docs' | 'test' | 'config' | 'unknown';
|
||||
/**
|
||||
* Calculate risk score for a diff
|
||||
*/
|
||||
export declare function calculateRiskScore(analysis: DiffAnalysis): number;
|
||||
/**
|
||||
* Analyze a single file diff
|
||||
*/
|
||||
export declare function analyzeFileDiff(file: string, diff: string, message?: string): Promise<DiffAnalysis>;
|
||||
/**
|
||||
* Get diff for a commit
|
||||
*/
|
||||
export declare function getCommitDiff(commitHash?: string): string;
|
||||
/**
|
||||
* Get diff for staged changes
|
||||
*/
|
||||
export declare function getStagedDiff(): string;
|
||||
/**
|
||||
* Get diff for unstaged changes
|
||||
*/
|
||||
export declare function getUnstagedDiff(): string;
|
||||
/**
|
||||
* Analyze a commit
|
||||
*/
|
||||
export declare function analyzeCommit(commitHash?: string): Promise<CommitAnalysis>;
|
||||
/**
|
||||
* Find similar past commits based on diff embeddings
|
||||
*/
|
||||
export declare function findSimilarCommits(currentDiff: string, recentCommits?: number, topK?: number): Promise<Array<{
|
||||
hash: string;
|
||||
similarity: number;
|
||||
message: string;
|
||||
}>>;
|
||||
declare const _default: {
|
||||
parseDiff: typeof parseDiff;
|
||||
classifyChange: typeof classifyChange;
|
||||
calculateRiskScore: typeof calculateRiskScore;
|
||||
analyzeFileDiff: typeof analyzeFileDiff;
|
||||
analyzeCommit: typeof analyzeCommit;
|
||||
getCommitDiff: typeof getCommitDiff;
|
||||
getStagedDiff: typeof getStagedDiff;
|
||||
getUnstagedDiff: typeof getUnstagedDiff;
|
||||
findSimilarCommits: typeof findSimilarCommits;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=diff-embeddings.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/diff-embeddings.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/diff-embeddings.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"diff-embeddings.d.ts","sourceRoot":"","sources":["diff-embeddings.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAKH,MAAM,WAAW,QAAQ;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,OAAO,EAAE,MAAM,CAAC;IAChB,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,SAAS,EAAE,MAAM,EAAE,CAAC;CACrB;AAED,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,QAAQ,EAAE,CAAC;IAClB,cAAc,EAAE,MAAM,CAAC;IACvB,cAAc,EAAE,MAAM,CAAC;IACvB,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,SAAS,GAAG,QAAQ,GAAG,UAAU,GAAG,MAAM,GAAG,MAAM,GAAG,QAAQ,GAAG,SAAS,CAAC;IACrF,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC;CACtB;AAED,MAAM,WAAW,cAAc;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,EAAE,MAAM,CAAC;IAChB,MAAM,EAAE,MAAM,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,YAAY,EAAE,CAAC;IACtB,cAAc,EAAE,MAAM,CAAC;IACvB,cAAc,EAAE,MAAM,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,CAAC,EAAE,MAAM,EAAE,CAAC;CACtB;AAED;;GAEG;AACH,wBAAgB,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,QAAQ,EAAE,CAsDlD;AAED;;GAEG;AACH,wBAAgB,cAAc,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,GAAE,MAAW,GAAG,SAAS,GAAG,QAAQ,GAAG,UAAU,GAAG,MAAM,GAAG,MAAM,GAAG,QAAQ,GAAG,SAAS,CAsB7I;AAED;;GAEG;AACH,wBAAgB,kBAAkB,CAAC,QAAQ,EAAE,YAAY,GAAG,MAAM,CA2BjE;AAED;;GAEG;AACH,wBAAsB,eAAe,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,GAAE,MAAW,GAAG,OAAO,CAAC,YAAY,CAAC,CAsC7G;AAED;;GAEG;AACH,wBAAgB,aAAa,CAAC,UAAU,GAAE,MAAe,GAAG,MAAM,CASjE;AAED;;GAEG;AACH,wBAAgB,aAAa,IAAI,MAAM,CAStC;AAED;;GAEG;AACH,wBAAgB,eAAe,IAAI,MAAM,CASxC;AAED;;GAEG;AACH,wBAAsB,aAAa,CAAC,UAAU,GAAE,MAAe,GAAG,OAAO,CAAC,cAAc,CAAC,CAwDxF;AAED;;GAEG;AACH,wBAAsB,kBAAkB,CACtC,WAAW,EAAE,MAAM,EACnB,aAAa,GAAE,MAAW,EAC1B,IAAI,GAAE,MAAU,GACf,OAAO,CAAC,KAAK,CAAC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,UAAU,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAA;CAAE,CAAC,CAAC,CAgCvE;;;;;;;;;;;;AAmBD,wBAUE"}
|
||||
335
npm/packages/ruvector/src/core/diff-embeddings.js
Normal file
335
npm/packages/ruvector/src/core/diff-embeddings.js
Normal file
@@ -0,0 +1,335 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Diff Embeddings - Semantic encoding of git diffs
|
||||
*
|
||||
* Generates embeddings for code changes to enable:
|
||||
* - Change classification (feature, bugfix, refactor)
|
||||
* - Similar change detection
|
||||
* - Risk assessment
|
||||
* - Review prioritization
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.parseDiff = parseDiff;
|
||||
exports.classifyChange = classifyChange;
|
||||
exports.calculateRiskScore = calculateRiskScore;
|
||||
exports.analyzeFileDiff = analyzeFileDiff;
|
||||
exports.getCommitDiff = getCommitDiff;
|
||||
exports.getStagedDiff = getStagedDiff;
|
||||
exports.getUnstagedDiff = getUnstagedDiff;
|
||||
exports.analyzeCommit = analyzeCommit;
|
||||
exports.findSimilarCommits = findSimilarCommits;
|
||||
const child_process_1 = require("child_process");
|
||||
const onnx_embedder_1 = require("./onnx-embedder");
|
||||
/**
|
||||
* Parse a unified diff into hunks
|
||||
*/
|
||||
function parseDiff(diff) {
|
||||
const hunks = [];
|
||||
const lines = diff.split('\n');
|
||||
let currentFile = '';
|
||||
let currentHunk = null;
|
||||
for (const line of lines) {
|
||||
// File header
|
||||
if (line.startsWith('diff --git')) {
|
||||
const match = line.match(/diff --git a\/(.+) b\/(.+)/);
|
||||
if (match) {
|
||||
currentFile = match[2];
|
||||
}
|
||||
}
|
||||
// Hunk header
|
||||
if (line.startsWith('@@')) {
|
||||
if (currentHunk) {
|
||||
hunks.push(currentHunk);
|
||||
}
|
||||
const match = line.match(/@@ -(\d+),?(\d*) \+(\d+),?(\d*) @@/);
|
||||
if (match) {
|
||||
currentHunk = {
|
||||
file: currentFile,
|
||||
oldStart: parseInt(match[1]),
|
||||
oldLines: parseInt(match[2] || '1'),
|
||||
newStart: parseInt(match[3]),
|
||||
newLines: parseInt(match[4] || '1'),
|
||||
content: '',
|
||||
additions: [],
|
||||
deletions: [],
|
||||
};
|
||||
}
|
||||
}
|
||||
else if (currentHunk) {
|
||||
// Content lines
|
||||
if (line.startsWith('+') && !line.startsWith('+++')) {
|
||||
currentHunk.additions.push(line.substring(1));
|
||||
currentHunk.content += line + '\n';
|
||||
}
|
||||
else if (line.startsWith('-') && !line.startsWith('---')) {
|
||||
currentHunk.deletions.push(line.substring(1));
|
||||
currentHunk.content += line + '\n';
|
||||
}
|
||||
else if (line.startsWith(' ')) {
|
||||
currentHunk.content += line + '\n';
|
||||
}
|
||||
}
|
||||
}
|
||||
if (currentHunk) {
|
||||
hunks.push(currentHunk);
|
||||
}
|
||||
return hunks;
|
||||
}
|
||||
/**
|
||||
* Classify a change based on patterns
|
||||
*/
|
||||
function classifyChange(diff, message = '') {
|
||||
const lowerMessage = message.toLowerCase();
|
||||
const lowerDiff = diff.toLowerCase();
|
||||
// Check message patterns
|
||||
if (/\b(fix|bug|issue|error|crash|patch)\b/.test(lowerMessage))
|
||||
return 'bugfix';
|
||||
if (/\b(feat|feature|add|new|implement)\b/.test(lowerMessage))
|
||||
return 'feature';
|
||||
if (/\b(refactor|clean|improve|optimize)\b/.test(lowerMessage))
|
||||
return 'refactor';
|
||||
if (/\b(doc|readme|comment|jsdoc)\b/.test(lowerMessage))
|
||||
return 'docs';
|
||||
if (/\b(test|spec|coverage)\b/.test(lowerMessage))
|
||||
return 'test';
|
||||
if (/\b(config|ci|cd|build|deps)\b/.test(lowerMessage))
|
||||
return 'config';
|
||||
// Check diff patterns
|
||||
if (/\.(md|txt|rst)$/.test(diff))
|
||||
return 'docs';
|
||||
if (/\.(test|spec)\.[jt]sx?/.test(diff))
|
||||
return 'test';
|
||||
if (/\.(json|ya?ml|toml|ini)$/.test(diff))
|
||||
return 'config';
|
||||
// Check content patterns
|
||||
if (/\bcatch\b|\btry\b|\berror\b/.test(lowerDiff) && /\bfix\b/.test(lowerDiff))
|
||||
return 'bugfix';
|
||||
if (/\bfunction\b|\bclass\b|\bexport\b/.test(lowerDiff))
|
||||
return 'feature';
|
||||
return 'unknown';
|
||||
}
|
||||
/**
|
||||
* Calculate risk score for a diff
|
||||
*/
|
||||
function calculateRiskScore(analysis) {
|
||||
let risk = 0;
|
||||
// Size risk
|
||||
const totalChanges = analysis.totalAdditions + analysis.totalDeletions;
|
||||
if (totalChanges > 500)
|
||||
risk += 0.3;
|
||||
else if (totalChanges > 200)
|
||||
risk += 0.2;
|
||||
else if (totalChanges > 50)
|
||||
risk += 0.1;
|
||||
// Complexity risk
|
||||
if (analysis.complexity > 20)
|
||||
risk += 0.2;
|
||||
else if (analysis.complexity > 10)
|
||||
risk += 0.1;
|
||||
// File type risk
|
||||
if (analysis.file.includes('auth') || analysis.file.includes('security'))
|
||||
risk += 0.2;
|
||||
if (analysis.file.includes('database') || analysis.file.includes('migration'))
|
||||
risk += 0.15;
|
||||
if (analysis.file.includes('api') || analysis.file.includes('endpoint'))
|
||||
risk += 0.1;
|
||||
// Pattern risk (deletions of error handling, etc.)
|
||||
for (const hunk of analysis.hunks) {
|
||||
for (const del of hunk.deletions) {
|
||||
if (/\bcatch\b|\berror\b|\bvalidat/.test(del))
|
||||
risk += 0.1;
|
||||
if (/\bif\b.*\bnull\b|\bundefined\b/.test(del))
|
||||
risk += 0.05;
|
||||
}
|
||||
}
|
||||
return Math.min(1, risk);
|
||||
}
|
||||
/**
|
||||
* Analyze a single file diff
|
||||
*/
|
||||
async function analyzeFileDiff(file, diff, message = '') {
|
||||
const hunks = parseDiff(diff).filter(h => h.file === file || h.file === '');
|
||||
const totalAdditions = hunks.reduce((sum, h) => sum + h.additions.length, 0);
|
||||
const totalDeletions = hunks.reduce((sum, h) => sum + h.deletions.length, 0);
|
||||
// Calculate complexity (branch keywords in additions)
|
||||
let complexity = 0;
|
||||
for (const hunk of hunks) {
|
||||
for (const add of hunk.additions) {
|
||||
if (/\bif\b|\belse\b|\bfor\b|\bwhile\b|\bswitch\b|\bcatch\b|\?/.test(add)) {
|
||||
complexity++;
|
||||
}
|
||||
}
|
||||
}
|
||||
const category = classifyChange(diff, message);
|
||||
const analysis = {
|
||||
file,
|
||||
hunks,
|
||||
totalAdditions,
|
||||
totalDeletions,
|
||||
complexity,
|
||||
riskScore: 0,
|
||||
category,
|
||||
};
|
||||
analysis.riskScore = calculateRiskScore(analysis);
|
||||
// Generate embedding for the diff
|
||||
if ((0, onnx_embedder_1.isReady)()) {
|
||||
const diffText = hunks.map(h => h.content).join('\n');
|
||||
const result = await (0, onnx_embedder_1.embed)(`${category} change in ${file}: ${diffText.substring(0, 500)}`);
|
||||
analysis.embedding = result.embedding;
|
||||
}
|
||||
return analysis;
|
||||
}
|
||||
/**
|
||||
* Get diff for a commit
|
||||
*/
|
||||
function getCommitDiff(commitHash = 'HEAD') {
|
||||
try {
|
||||
return (0, child_process_1.execSync)(`git show ${commitHash} --format="" 2>/dev/null`, {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
}
|
||||
catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Get diff for staged changes
|
||||
*/
|
||||
function getStagedDiff() {
|
||||
try {
|
||||
return (0, child_process_1.execSync)('git diff --cached 2>/dev/null', {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
}
|
||||
catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Get diff for unstaged changes
|
||||
*/
|
||||
function getUnstagedDiff() {
|
||||
try {
|
||||
return (0, child_process_1.execSync)('git diff 2>/dev/null', {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
}
|
||||
catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Analyze a commit
|
||||
*/
|
||||
async function analyzeCommit(commitHash = 'HEAD') {
|
||||
const diff = getCommitDiff(commitHash);
|
||||
// Get commit metadata
|
||||
let message = '', author = '', date = '';
|
||||
try {
|
||||
const info = (0, child_process_1.execSync)(`git log -1 --format="%s|%an|%aI" ${commitHash} 2>/dev/null`, {
|
||||
encoding: 'utf8',
|
||||
}).trim();
|
||||
[message, author, date] = info.split('|');
|
||||
}
|
||||
catch { }
|
||||
// Parse hunks and group by file
|
||||
const hunks = parseDiff(diff);
|
||||
const fileHunks = new Map();
|
||||
for (const hunk of hunks) {
|
||||
if (!fileHunks.has(hunk.file)) {
|
||||
fileHunks.set(hunk.file, []);
|
||||
}
|
||||
fileHunks.get(hunk.file).push(hunk);
|
||||
}
|
||||
// Analyze each file
|
||||
const files = [];
|
||||
for (const [file, fileHunkList] of fileHunks) {
|
||||
const fileDiff = fileHunkList.map(h => h.content).join('\n');
|
||||
const analysis = await analyzeFileDiff(file, diff, message);
|
||||
files.push(analysis);
|
||||
}
|
||||
const totalAdditions = files.reduce((sum, f) => sum + f.totalAdditions, 0);
|
||||
const totalDeletions = files.reduce((sum, f) => sum + f.totalDeletions, 0);
|
||||
const riskScore = files.length > 0
|
||||
? files.reduce((sum, f) => sum + f.riskScore, 0) / files.length
|
||||
: 0;
|
||||
// Generate commit embedding
|
||||
let embedding;
|
||||
if ((0, onnx_embedder_1.isReady)()) {
|
||||
const commitText = `${message}\n\nFiles changed: ${files.map(f => f.file).join(', ')}\n+${totalAdditions} -${totalDeletions}`;
|
||||
const result = await (0, onnx_embedder_1.embed)(commitText);
|
||||
embedding = result.embedding;
|
||||
}
|
||||
return {
|
||||
hash: commitHash,
|
||||
message,
|
||||
author,
|
||||
date,
|
||||
files,
|
||||
totalAdditions,
|
||||
totalDeletions,
|
||||
riskScore,
|
||||
embedding,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Find similar past commits based on diff embeddings
|
||||
*/
|
||||
async function findSimilarCommits(currentDiff, recentCommits = 50, topK = 5) {
|
||||
if (!(0, onnx_embedder_1.isReady)()) {
|
||||
await (0, onnx_embedder_1.initOnnxEmbedder)();
|
||||
}
|
||||
// Get current diff embedding
|
||||
const currentEmbedding = (await (0, onnx_embedder_1.embed)(currentDiff.substring(0, 1000))).embedding;
|
||||
// Get recent commits
|
||||
let commits = [];
|
||||
try {
|
||||
commits = (0, child_process_1.execSync)(`git log -${recentCommits} --format="%H" 2>/dev/null`, {
|
||||
encoding: 'utf8',
|
||||
}).trim().split('\n');
|
||||
}
|
||||
catch {
|
||||
return [];
|
||||
}
|
||||
// Analyze and compare
|
||||
const results = [];
|
||||
for (const hash of commits.slice(0, Math.min(commits.length, recentCommits))) {
|
||||
const analysis = await analyzeCommit(hash);
|
||||
if (analysis.embedding) {
|
||||
const similarity = cosineSimilarity(currentEmbedding, analysis.embedding);
|
||||
results.push({ hash, similarity, message: analysis.message });
|
||||
}
|
||||
}
|
||||
return results
|
||||
.sort((a, b) => b.similarity - a.similarity)
|
||||
.slice(0, topK);
|
||||
}
|
||||
function cosineSimilarity(a, b) {
|
||||
if (a.length !== b.length)
|
||||
return 0;
|
||||
let dotProduct = 0;
|
||||
let normA = 0;
|
||||
let normB = 0;
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dotProduct += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
const magnitude = Math.sqrt(normA) * Math.sqrt(normB);
|
||||
return magnitude === 0 ? 0 : dotProduct / magnitude;
|
||||
}
|
||||
exports.default = {
|
||||
parseDiff,
|
||||
classifyChange,
|
||||
calculateRiskScore,
|
||||
analyzeFileDiff,
|
||||
analyzeCommit,
|
||||
getCommitDiff,
|
||||
getStagedDiff,
|
||||
getUnstagedDiff,
|
||||
findSimilarCommits,
|
||||
};
|
||||
//# sourceMappingURL=diff-embeddings.js.map
|
||||
1
npm/packages/ruvector/src/core/diff-embeddings.js.map
Normal file
1
npm/packages/ruvector/src/core/diff-embeddings.js.map
Normal file
File diff suppressed because one or more lines are too long
380
npm/packages/ruvector/src/core/diff-embeddings.ts
Normal file
380
npm/packages/ruvector/src/core/diff-embeddings.ts
Normal file
@@ -0,0 +1,380 @@
|
||||
/**
|
||||
* Diff Embeddings - Semantic encoding of git diffs
|
||||
*
|
||||
* Generates embeddings for code changes to enable:
|
||||
* - Change classification (feature, bugfix, refactor)
|
||||
* - Similar change detection
|
||||
* - Risk assessment
|
||||
* - Review prioritization
|
||||
*/
|
||||
|
||||
import { execSync } from 'child_process';
|
||||
import { embed, embedBatch, isReady, initOnnxEmbedder } from './onnx-embedder';
|
||||
|
||||
export interface DiffHunk {
|
||||
file: string;
|
||||
oldStart: number;
|
||||
oldLines: number;
|
||||
newStart: number;
|
||||
newLines: number;
|
||||
content: string;
|
||||
additions: string[];
|
||||
deletions: string[];
|
||||
}
|
||||
|
||||
export interface DiffAnalysis {
|
||||
file: string;
|
||||
hunks: DiffHunk[];
|
||||
totalAdditions: number;
|
||||
totalDeletions: number;
|
||||
complexity: number;
|
||||
riskScore: number;
|
||||
category: 'feature' | 'bugfix' | 'refactor' | 'docs' | 'test' | 'config' | 'unknown';
|
||||
embedding?: number[];
|
||||
}
|
||||
|
||||
export interface CommitAnalysis {
|
||||
hash: string;
|
||||
message: string;
|
||||
author: string;
|
||||
date: string;
|
||||
files: DiffAnalysis[];
|
||||
totalAdditions: number;
|
||||
totalDeletions: number;
|
||||
riskScore: number;
|
||||
embedding?: number[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a unified diff into hunks
|
||||
*/
|
||||
export function parseDiff(diff: string): DiffHunk[] {
|
||||
const hunks: DiffHunk[] = [];
|
||||
const lines = diff.split('\n');
|
||||
|
||||
let currentFile = '';
|
||||
let currentHunk: DiffHunk | null = null;
|
||||
|
||||
for (const line of lines) {
|
||||
// File header
|
||||
if (line.startsWith('diff --git')) {
|
||||
const match = line.match(/diff --git a\/(.+) b\/(.+)/);
|
||||
if (match) {
|
||||
currentFile = match[2];
|
||||
}
|
||||
}
|
||||
|
||||
// Hunk header
|
||||
if (line.startsWith('@@')) {
|
||||
if (currentHunk) {
|
||||
hunks.push(currentHunk);
|
||||
}
|
||||
|
||||
const match = line.match(/@@ -(\d+),?(\d*) \+(\d+),?(\d*) @@/);
|
||||
if (match) {
|
||||
currentHunk = {
|
||||
file: currentFile,
|
||||
oldStart: parseInt(match[1]),
|
||||
oldLines: parseInt(match[2] || '1'),
|
||||
newStart: parseInt(match[3]),
|
||||
newLines: parseInt(match[4] || '1'),
|
||||
content: '',
|
||||
additions: [],
|
||||
deletions: [],
|
||||
};
|
||||
}
|
||||
} else if (currentHunk) {
|
||||
// Content lines
|
||||
if (line.startsWith('+') && !line.startsWith('+++')) {
|
||||
currentHunk.additions.push(line.substring(1));
|
||||
currentHunk.content += line + '\n';
|
||||
} else if (line.startsWith('-') && !line.startsWith('---')) {
|
||||
currentHunk.deletions.push(line.substring(1));
|
||||
currentHunk.content += line + '\n';
|
||||
} else if (line.startsWith(' ')) {
|
||||
currentHunk.content += line + '\n';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (currentHunk) {
|
||||
hunks.push(currentHunk);
|
||||
}
|
||||
|
||||
return hunks;
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify a change based on patterns
|
||||
*/
|
||||
export function classifyChange(diff: string, message: string = ''): 'feature' | 'bugfix' | 'refactor' | 'docs' | 'test' | 'config' | 'unknown' {
|
||||
const lowerMessage = message.toLowerCase();
|
||||
const lowerDiff = diff.toLowerCase();
|
||||
|
||||
// Check message patterns
|
||||
if (/\b(fix|bug|issue|error|crash|patch)\b/.test(lowerMessage)) return 'bugfix';
|
||||
if (/\b(feat|feature|add|new|implement)\b/.test(lowerMessage)) return 'feature';
|
||||
if (/\b(refactor|clean|improve|optimize)\b/.test(lowerMessage)) return 'refactor';
|
||||
if (/\b(doc|readme|comment|jsdoc)\b/.test(lowerMessage)) return 'docs';
|
||||
if (/\b(test|spec|coverage)\b/.test(lowerMessage)) return 'test';
|
||||
if (/\b(config|ci|cd|build|deps)\b/.test(lowerMessage)) return 'config';
|
||||
|
||||
// Check diff patterns
|
||||
if (/\.(md|txt|rst)$/.test(diff)) return 'docs';
|
||||
if (/\.(test|spec)\.[jt]sx?/.test(diff)) return 'test';
|
||||
if (/\.(json|ya?ml|toml|ini)$/.test(diff)) return 'config';
|
||||
|
||||
// Check content patterns
|
||||
if (/\bcatch\b|\btry\b|\berror\b/.test(lowerDiff) && /\bfix\b/.test(lowerDiff)) return 'bugfix';
|
||||
if (/\bfunction\b|\bclass\b|\bexport\b/.test(lowerDiff)) return 'feature';
|
||||
|
||||
return 'unknown';
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate risk score for a diff
|
||||
*/
|
||||
export function calculateRiskScore(analysis: DiffAnalysis): number {
|
||||
let risk = 0;
|
||||
|
||||
// Size risk
|
||||
const totalChanges = analysis.totalAdditions + analysis.totalDeletions;
|
||||
if (totalChanges > 500) risk += 0.3;
|
||||
else if (totalChanges > 200) risk += 0.2;
|
||||
else if (totalChanges > 50) risk += 0.1;
|
||||
|
||||
// Complexity risk
|
||||
if (analysis.complexity > 20) risk += 0.2;
|
||||
else if (analysis.complexity > 10) risk += 0.1;
|
||||
|
||||
// File type risk
|
||||
if (analysis.file.includes('auth') || analysis.file.includes('security')) risk += 0.2;
|
||||
if (analysis.file.includes('database') || analysis.file.includes('migration')) risk += 0.15;
|
||||
if (analysis.file.includes('api') || analysis.file.includes('endpoint')) risk += 0.1;
|
||||
|
||||
// Pattern risk (deletions of error handling, etc.)
|
||||
for (const hunk of analysis.hunks) {
|
||||
for (const del of hunk.deletions) {
|
||||
if (/\bcatch\b|\berror\b|\bvalidat/.test(del)) risk += 0.1;
|
||||
if (/\bif\b.*\bnull\b|\bundefined\b/.test(del)) risk += 0.05;
|
||||
}
|
||||
}
|
||||
|
||||
return Math.min(1, risk);
|
||||
}
|
||||
|
||||
/**
|
||||
* Analyze a single file diff
|
||||
*/
|
||||
export async function analyzeFileDiff(file: string, diff: string, message: string = ''): Promise<DiffAnalysis> {
|
||||
const hunks = parseDiff(diff).filter(h => h.file === file || h.file === '');
|
||||
|
||||
const totalAdditions = hunks.reduce((sum, h) => sum + h.additions.length, 0);
|
||||
const totalDeletions = hunks.reduce((sum, h) => sum + h.deletions.length, 0);
|
||||
|
||||
// Calculate complexity (branch keywords in additions)
|
||||
let complexity = 0;
|
||||
for (const hunk of hunks) {
|
||||
for (const add of hunk.additions) {
|
||||
if (/\bif\b|\belse\b|\bfor\b|\bwhile\b|\bswitch\b|\bcatch\b|\?/.test(add)) {
|
||||
complexity++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const category = classifyChange(diff, message);
|
||||
|
||||
const analysis: DiffAnalysis = {
|
||||
file,
|
||||
hunks,
|
||||
totalAdditions,
|
||||
totalDeletions,
|
||||
complexity,
|
||||
riskScore: 0,
|
||||
category,
|
||||
};
|
||||
|
||||
analysis.riskScore = calculateRiskScore(analysis);
|
||||
|
||||
// Generate embedding for the diff
|
||||
if (isReady()) {
|
||||
const diffText = hunks.map(h => h.content).join('\n');
|
||||
const result = await embed(`${category} change in ${file}: ${diffText.substring(0, 500)}`);
|
||||
analysis.embedding = result.embedding;
|
||||
}
|
||||
|
||||
return analysis;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get diff for a commit
|
||||
*/
|
||||
export function getCommitDiff(commitHash: string = 'HEAD'): string {
|
||||
try {
|
||||
return execSync(`git show ${commitHash} --format="" 2>/dev/null`, {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get diff for staged changes
|
||||
*/
|
||||
export function getStagedDiff(): string {
|
||||
try {
|
||||
return execSync('git diff --cached 2>/dev/null', {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get diff for unstaged changes
|
||||
*/
|
||||
export function getUnstagedDiff(): string {
|
||||
try {
|
||||
return execSync('git diff 2>/dev/null', {
|
||||
encoding: 'utf8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Analyze a commit
|
||||
*/
|
||||
export async function analyzeCommit(commitHash: string = 'HEAD'): Promise<CommitAnalysis> {
|
||||
const diff = getCommitDiff(commitHash);
|
||||
|
||||
// Get commit metadata
|
||||
let message = '', author = '', date = '';
|
||||
try {
|
||||
const info = execSync(`git log -1 --format="%s|%an|%aI" ${commitHash} 2>/dev/null`, {
|
||||
encoding: 'utf8',
|
||||
}).trim();
|
||||
[message, author, date] = info.split('|');
|
||||
} catch {}
|
||||
|
||||
// Parse hunks and group by file
|
||||
const hunks = parseDiff(diff);
|
||||
const fileHunks = new Map<string, DiffHunk[]>();
|
||||
|
||||
for (const hunk of hunks) {
|
||||
if (!fileHunks.has(hunk.file)) {
|
||||
fileHunks.set(hunk.file, []);
|
||||
}
|
||||
fileHunks.get(hunk.file)!.push(hunk);
|
||||
}
|
||||
|
||||
// Analyze each file
|
||||
const files: DiffAnalysis[] = [];
|
||||
for (const [file, fileHunkList] of fileHunks) {
|
||||
const fileDiff = fileHunkList.map(h => h.content).join('\n');
|
||||
const analysis = await analyzeFileDiff(file, diff, message);
|
||||
files.push(analysis);
|
||||
}
|
||||
|
||||
const totalAdditions = files.reduce((sum, f) => sum + f.totalAdditions, 0);
|
||||
const totalDeletions = files.reduce((sum, f) => sum + f.totalDeletions, 0);
|
||||
const riskScore = files.length > 0
|
||||
? files.reduce((sum, f) => sum + f.riskScore, 0) / files.length
|
||||
: 0;
|
||||
|
||||
// Generate commit embedding
|
||||
let embedding: number[] | undefined;
|
||||
if (isReady()) {
|
||||
const commitText = `${message}\n\nFiles changed: ${files.map(f => f.file).join(', ')}\n+${totalAdditions} -${totalDeletions}`;
|
||||
const result = await embed(commitText);
|
||||
embedding = result.embedding;
|
||||
}
|
||||
|
||||
return {
|
||||
hash: commitHash,
|
||||
message,
|
||||
author,
|
||||
date,
|
||||
files,
|
||||
totalAdditions,
|
||||
totalDeletions,
|
||||
riskScore,
|
||||
embedding,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find similar past commits based on diff embeddings
|
||||
*/
|
||||
export async function findSimilarCommits(
|
||||
currentDiff: string,
|
||||
recentCommits: number = 50,
|
||||
topK: number = 5
|
||||
): Promise<Array<{ hash: string; similarity: number; message: string }>> {
|
||||
if (!isReady()) {
|
||||
await initOnnxEmbedder();
|
||||
}
|
||||
|
||||
// Get current diff embedding
|
||||
const currentEmbedding = (await embed(currentDiff.substring(0, 1000))).embedding;
|
||||
|
||||
// Get recent commits
|
||||
let commits: string[] = [];
|
||||
try {
|
||||
commits = execSync(`git log -${recentCommits} --format="%H" 2>/dev/null`, {
|
||||
encoding: 'utf8',
|
||||
}).trim().split('\n');
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
|
||||
// Analyze and compare
|
||||
const results: Array<{ hash: string; similarity: number; message: string }> = [];
|
||||
|
||||
for (const hash of commits.slice(0, Math.min(commits.length, recentCommits))) {
|
||||
const analysis = await analyzeCommit(hash);
|
||||
if (analysis.embedding) {
|
||||
const similarity = cosineSimilarity(currentEmbedding, analysis.embedding);
|
||||
results.push({ hash, similarity, message: analysis.message });
|
||||
}
|
||||
}
|
||||
|
||||
return results
|
||||
.sort((a, b) => b.similarity - a.similarity)
|
||||
.slice(0, topK);
|
||||
}
|
||||
|
||||
function cosineSimilarity(a: number[], b: number[]): number {
|
||||
if (a.length !== b.length) return 0;
|
||||
|
||||
let dotProduct = 0;
|
||||
let normA = 0;
|
||||
let normB = 0;
|
||||
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dotProduct += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
|
||||
const magnitude = Math.sqrt(normA) * Math.sqrt(normB);
|
||||
return magnitude === 0 ? 0 : dotProduct / magnitude;
|
||||
}
|
||||
|
||||
export default {
|
||||
parseDiff,
|
||||
classifyChange,
|
||||
calculateRiskScore,
|
||||
analyzeFileDiff,
|
||||
analyzeCommit,
|
||||
getCommitDiff,
|
||||
getStagedDiff,
|
||||
getUnstagedDiff,
|
||||
findSimilarCommits,
|
||||
};
|
||||
143
npm/packages/ruvector/src/core/gnn-wrapper.d.ts
vendored
Normal file
143
npm/packages/ruvector/src/core/gnn-wrapper.d.ts
vendored
Normal file
@@ -0,0 +1,143 @@
|
||||
/**
|
||||
* GNN Wrapper - Safe wrapper around @ruvector/gnn with automatic array conversion
|
||||
*
|
||||
* This wrapper handles the array type conversion automatically, allowing users
|
||||
* to pass either regular arrays or Float32Arrays.
|
||||
*
|
||||
* The native @ruvector/gnn requires Float32Array for maximum performance.
|
||||
* This wrapper converts any input type to Float32Array automatically.
|
||||
*
|
||||
* Performance Tips:
|
||||
* - Pass Float32Array directly for zero-copy performance
|
||||
* - Use toFloat32Array/toFloat32ArrayBatch for pre-conversion
|
||||
* - Avoid repeated conversions in hot paths
|
||||
*/
|
||||
/**
|
||||
* Convert any array-like input to Float32Array (native requires Float32Array)
|
||||
* Optimized paths:
|
||||
* - Float32Array: zero-copy return
|
||||
* - Float64Array: efficient typed array copy
|
||||
* - Array: direct Float32Array construction
|
||||
*/
|
||||
export declare function toFloat32Array(input: number[] | Float32Array | Float64Array): Float32Array;
|
||||
/**
|
||||
* Convert array of arrays to array of Float32Arrays
|
||||
*/
|
||||
export declare function toFloat32ArrayBatch(input: (number[] | Float32Array | Float64Array)[]): Float32Array[];
|
||||
/**
|
||||
* Search result from differentiable search
|
||||
*/
|
||||
export interface DifferentiableSearchResult {
|
||||
/** Indices of top-k candidates */
|
||||
indices: number[];
|
||||
/** Soft weights for top-k candidates */
|
||||
weights: number[];
|
||||
}
|
||||
/**
|
||||
* Differentiable search using soft attention mechanism
|
||||
*
|
||||
* This wrapper automatically converts Float32Array inputs to regular arrays.
|
||||
*
|
||||
* @param query - Query vector (array or Float32Array)
|
||||
* @param candidates - List of candidate vectors (arrays or Float32Arrays)
|
||||
* @param k - Number of top results to return
|
||||
* @param temperature - Temperature for softmax (lower = sharper, higher = smoother)
|
||||
* @returns Search result with indices and soft weights
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { differentiableSearch } from 'ruvector/core/gnn-wrapper';
|
||||
*
|
||||
* // Works with regular arrays (auto-converted to Float32Array)
|
||||
* const result1 = differentiableSearch([1, 0, 0], [[1, 0, 0], [0, 1, 0]], 2, 1.0);
|
||||
*
|
||||
* // For best performance, use Float32Array directly (zero-copy)
|
||||
* const query = new Float32Array([1, 0, 0]);
|
||||
* const candidates = [new Float32Array([1, 0, 0]), new Float32Array([0, 1, 0])];
|
||||
* const result2 = differentiableSearch(query, candidates, 2, 1.0);
|
||||
* ```
|
||||
*/
|
||||
export declare function differentiableSearch(query: number[] | Float32Array | Float64Array, candidates: (number[] | Float32Array | Float64Array)[], k: number, temperature?: number): DifferentiableSearchResult;
|
||||
/**
|
||||
* GNN Layer for HNSW topology
|
||||
*/
|
||||
export declare class RuvectorLayer {
|
||||
private inner;
|
||||
/**
|
||||
* Create a new Ruvector GNN layer
|
||||
*
|
||||
* @param inputDim - Dimension of input node embeddings
|
||||
* @param hiddenDim - Dimension of hidden representations
|
||||
* @param heads - Number of attention heads
|
||||
* @param dropout - Dropout rate (0.0 to 1.0)
|
||||
*/
|
||||
constructor(inputDim: number, hiddenDim: number, heads: number, dropout?: number);
|
||||
/**
|
||||
* Forward pass through the GNN layer
|
||||
*
|
||||
* @param nodeEmbedding - Current node's embedding
|
||||
* @param neighborEmbeddings - Embeddings of neighbor nodes
|
||||
* @param edgeWeights - Weights of edges to neighbors
|
||||
* @returns Updated node embedding as Float32Array
|
||||
*/
|
||||
forward(nodeEmbedding: number[] | Float32Array, neighborEmbeddings: (number[] | Float32Array)[], edgeWeights: number[] | Float32Array): Float32Array;
|
||||
/**
|
||||
* Serialize the layer to JSON
|
||||
*/
|
||||
toJson(): string;
|
||||
/**
|
||||
* Deserialize the layer from JSON
|
||||
*/
|
||||
static fromJson(json: string): RuvectorLayer;
|
||||
}
|
||||
/**
|
||||
* Tensor compressor with adaptive level selection
|
||||
*/
|
||||
export declare class TensorCompress {
|
||||
private inner;
|
||||
constructor();
|
||||
/**
|
||||
* Compress an embedding based on access frequency
|
||||
*
|
||||
* @param embedding - Input embedding vector
|
||||
* @param accessFreq - Access frequency (0.0 to 1.0)
|
||||
* @returns Compressed tensor as JSON string
|
||||
*/
|
||||
compress(embedding: number[] | Float32Array, accessFreq: number): string;
|
||||
/**
|
||||
* Decompress a compressed tensor
|
||||
*
|
||||
* @param compressedJson - Compressed tensor JSON
|
||||
* @returns Decompressed embedding
|
||||
*/
|
||||
decompress(compressedJson: string): number[];
|
||||
}
|
||||
/**
|
||||
* Hierarchical forward pass through GNN layers
|
||||
*
|
||||
* @param query - Query vector
|
||||
* @param layerEmbeddings - Embeddings organized by layer
|
||||
* @param gnnLayersJson - JSON array of serialized GNN layers
|
||||
* @returns Final embedding after hierarchical processing as Float32Array
|
||||
*/
|
||||
export declare function hierarchicalForward(query: number[] | Float32Array, layerEmbeddings: (number[] | Float32Array)[][], gnnLayersJson: string[]): Float32Array;
|
||||
/**
|
||||
* Get compression level for a given access frequency
|
||||
*/
|
||||
export declare function getCompressionLevel(accessFreq: number): string;
|
||||
/**
|
||||
* Check if GNN module is available
|
||||
*/
|
||||
export declare function isGnnAvailable(): boolean;
|
||||
declare const _default: {
|
||||
differentiableSearch: typeof differentiableSearch;
|
||||
RuvectorLayer: typeof RuvectorLayer;
|
||||
TensorCompress: typeof TensorCompress;
|
||||
hierarchicalForward: typeof hierarchicalForward;
|
||||
getCompressionLevel: typeof getCompressionLevel;
|
||||
isGnnAvailable: typeof isGnnAvailable;
|
||||
toFloat32Array: typeof toFloat32Array;
|
||||
toFloat32ArrayBatch: typeof toFloat32ArrayBatch;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=gnn-wrapper.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/gnn-wrapper.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/gnn-wrapper.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"gnn-wrapper.d.ts","sourceRoot":"","sources":["gnn-wrapper.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;GAaG;AAsBH;;;;;;GAMG;AACH,wBAAgB,cAAc,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,YAAY,GAAG,YAAY,GAAG,YAAY,CAK1F;AAED;;GAEG;AACH,wBAAgB,mBAAmB,CAAC,KAAK,EAAE,CAAC,MAAM,EAAE,GAAG,YAAY,GAAG,YAAY,CAAC,EAAE,GAAG,YAAY,EAAE,CAMrG;AAED;;GAEG;AACH,MAAM,WAAW,0BAA0B;IACzC,kCAAkC;IAClC,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,wCAAwC;IACxC,OAAO,EAAE,MAAM,EAAE,CAAC;CACnB;AAED;;;;;;;;;;;;;;;;;;;;;;;GAuBG;AACH,wBAAgB,oBAAoB,CAClC,KAAK,EAAE,MAAM,EAAE,GAAG,YAAY,GAAG,YAAY,EAC7C,UAAU,EAAE,CAAC,MAAM,EAAE,GAAG,YAAY,GAAG,YAAY,CAAC,EAAE,EACtD,CAAC,EAAE,MAAM,EACT,WAAW,GAAE,MAAY,GACxB,0BAA0B,CAQ5B;AAED;;GAEG;AACH,qBAAa,aAAa;IACxB,OAAO,CAAC,KAAK,CAAM;IAEnB;;;;;;;OAOG;gBACS,QAAQ,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,OAAO,GAAE,MAAY;IAKrF;;;;;;;OAOG;IACH,OAAO,CACL,aAAa,EAAE,MAAM,EAAE,GAAG,YAAY,EACtC,kBAAkB,EAAE,CAAC,MAAM,EAAE,GAAG,YAAY,CAAC,EAAE,EAC/C,WAAW,EAAE,MAAM,EAAE,GAAG,YAAY,GACnC,YAAY;IAQf;;OAEG;IACH,MAAM,IAAI,MAAM;IAIhB;;OAEG;IACH,MAAM,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,aAAa;CAM7C;AAED;;GAEG;AACH,qBAAa,cAAc;IACzB,OAAO,CAAC,KAAK,CAAM;;IAOnB;;;;;;OAMG;IACH,QAAQ,CAAC,SAAS,EAAE,MAAM,EAAE,GAAG,YAAY,EAAE,UAAU,EAAE,MAAM,GAAG,MAAM;IAIxE;;;;;OAKG;IACH,UAAU,CAAC,cAAc,EAAE,MAAM,GAAG,MAAM,EAAE;CAG7C;AAED;;;;;;;GAOG;AACH,wBAAgB,mBAAmB,CACjC,KAAK,EAAE,MAAM,EAAE,GAAG,YAAY,EAC9B,eAAe,EAAE,CAAC,MAAM,EAAE,GAAG,YAAY,CAAC,EAAE,EAAE,EAC9C,aAAa,EAAE,MAAM,EAAE,GACtB,YAAY,CAOd;AAED;;GAEG;AACH,wBAAgB,mBAAmB,CAAC,UAAU,EAAE,MAAM,GAAG,MAAM,CAG9D;AAED;;GAEG;AACH,wBAAgB,cAAc,IAAI,OAAO,CAOxC;;;;;;;;;;;AAED,wBAUE"}
|
||||
214
npm/packages/ruvector/src/core/gnn-wrapper.js
Normal file
214
npm/packages/ruvector/src/core/gnn-wrapper.js
Normal file
@@ -0,0 +1,214 @@
|
||||
"use strict";
|
||||
/**
|
||||
* GNN Wrapper - Safe wrapper around @ruvector/gnn with automatic array conversion
|
||||
*
|
||||
* This wrapper handles the array type conversion automatically, allowing users
|
||||
* to pass either regular arrays or Float32Arrays.
|
||||
*
|
||||
* The native @ruvector/gnn requires Float32Array for maximum performance.
|
||||
* This wrapper converts any input type to Float32Array automatically.
|
||||
*
|
||||
* Performance Tips:
|
||||
* - Pass Float32Array directly for zero-copy performance
|
||||
* - Use toFloat32Array/toFloat32ArrayBatch for pre-conversion
|
||||
* - Avoid repeated conversions in hot paths
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.TensorCompress = exports.RuvectorLayer = void 0;
|
||||
exports.toFloat32Array = toFloat32Array;
|
||||
exports.toFloat32ArrayBatch = toFloat32ArrayBatch;
|
||||
exports.differentiableSearch = differentiableSearch;
|
||||
exports.hierarchicalForward = hierarchicalForward;
|
||||
exports.getCompressionLevel = getCompressionLevel;
|
||||
exports.isGnnAvailable = isGnnAvailable;
|
||||
// Lazy load to avoid import errors if not installed
|
||||
let gnnModule = null;
|
||||
let loadError = null;
|
||||
function getGnnModule() {
|
||||
if (gnnModule)
|
||||
return gnnModule;
|
||||
if (loadError)
|
||||
throw loadError;
|
||||
try {
|
||||
gnnModule = require('@ruvector/gnn');
|
||||
return gnnModule;
|
||||
}
|
||||
catch (e) {
|
||||
loadError = new Error(`@ruvector/gnn is not installed or failed to load: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/gnn`);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Convert any array-like input to Float32Array (native requires Float32Array)
|
||||
* Optimized paths:
|
||||
* - Float32Array: zero-copy return
|
||||
* - Float64Array: efficient typed array copy
|
||||
* - Array: direct Float32Array construction
|
||||
*/
|
||||
function toFloat32Array(input) {
|
||||
if (input instanceof Float32Array)
|
||||
return input;
|
||||
if (input instanceof Float64Array)
|
||||
return new Float32Array(input);
|
||||
if (Array.isArray(input))
|
||||
return new Float32Array(input);
|
||||
return new Float32Array(Array.from(input));
|
||||
}
|
||||
/**
|
||||
* Convert array of arrays to array of Float32Arrays
|
||||
*/
|
||||
function toFloat32ArrayBatch(input) {
|
||||
const result = new Array(input.length);
|
||||
for (let i = 0; i < input.length; i++) {
|
||||
result[i] = toFloat32Array(input[i]);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
/**
|
||||
* Differentiable search using soft attention mechanism
|
||||
*
|
||||
* This wrapper automatically converts Float32Array inputs to regular arrays.
|
||||
*
|
||||
* @param query - Query vector (array or Float32Array)
|
||||
* @param candidates - List of candidate vectors (arrays or Float32Arrays)
|
||||
* @param k - Number of top results to return
|
||||
* @param temperature - Temperature for softmax (lower = sharper, higher = smoother)
|
||||
* @returns Search result with indices and soft weights
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { differentiableSearch } from 'ruvector/core/gnn-wrapper';
|
||||
*
|
||||
* // Works with regular arrays (auto-converted to Float32Array)
|
||||
* const result1 = differentiableSearch([1, 0, 0], [[1, 0, 0], [0, 1, 0]], 2, 1.0);
|
||||
*
|
||||
* // For best performance, use Float32Array directly (zero-copy)
|
||||
* const query = new Float32Array([1, 0, 0]);
|
||||
* const candidates = [new Float32Array([1, 0, 0]), new Float32Array([0, 1, 0])];
|
||||
* const result2 = differentiableSearch(query, candidates, 2, 1.0);
|
||||
* ```
|
||||
*/
|
||||
function differentiableSearch(query, candidates, k, temperature = 1.0) {
|
||||
const gnn = getGnnModule();
|
||||
// Convert to Float32Array (native Rust expects Float32Array for performance)
|
||||
const queryFloat32 = toFloat32Array(query);
|
||||
const candidatesFloat32 = toFloat32ArrayBatch(candidates);
|
||||
return gnn.differentiableSearch(queryFloat32, candidatesFloat32, k, temperature);
|
||||
}
|
||||
/**
|
||||
* GNN Layer for HNSW topology
|
||||
*/
|
||||
class RuvectorLayer {
|
||||
/**
|
||||
* Create a new Ruvector GNN layer
|
||||
*
|
||||
* @param inputDim - Dimension of input node embeddings
|
||||
* @param hiddenDim - Dimension of hidden representations
|
||||
* @param heads - Number of attention heads
|
||||
* @param dropout - Dropout rate (0.0 to 1.0)
|
||||
*/
|
||||
constructor(inputDim, hiddenDim, heads, dropout = 0.1) {
|
||||
const gnn = getGnnModule();
|
||||
this.inner = new gnn.RuvectorLayer(inputDim, hiddenDim, heads, dropout);
|
||||
}
|
||||
/**
|
||||
* Forward pass through the GNN layer
|
||||
*
|
||||
* @param nodeEmbedding - Current node's embedding
|
||||
* @param neighborEmbeddings - Embeddings of neighbor nodes
|
||||
* @param edgeWeights - Weights of edges to neighbors
|
||||
* @returns Updated node embedding as Float32Array
|
||||
*/
|
||||
forward(nodeEmbedding, neighborEmbeddings, edgeWeights) {
|
||||
return this.inner.forward(toFloat32Array(nodeEmbedding), toFloat32ArrayBatch(neighborEmbeddings), toFloat32Array(edgeWeights));
|
||||
}
|
||||
/**
|
||||
* Serialize the layer to JSON
|
||||
*/
|
||||
toJson() {
|
||||
return this.inner.toJson();
|
||||
}
|
||||
/**
|
||||
* Deserialize the layer from JSON
|
||||
*/
|
||||
static fromJson(json) {
|
||||
const gnn = getGnnModule();
|
||||
const layer = new RuvectorLayer(1, 1, 1, 0); // Dummy constructor
|
||||
layer.inner = gnn.RuvectorLayer.fromJson(json);
|
||||
return layer;
|
||||
}
|
||||
}
|
||||
exports.RuvectorLayer = RuvectorLayer;
|
||||
/**
|
||||
* Tensor compressor with adaptive level selection
|
||||
*/
|
||||
class TensorCompress {
|
||||
constructor() {
|
||||
const gnn = getGnnModule();
|
||||
this.inner = new gnn.TensorCompress();
|
||||
}
|
||||
/**
|
||||
* Compress an embedding based on access frequency
|
||||
*
|
||||
* @param embedding - Input embedding vector
|
||||
* @param accessFreq - Access frequency (0.0 to 1.0)
|
||||
* @returns Compressed tensor as JSON string
|
||||
*/
|
||||
compress(embedding, accessFreq) {
|
||||
return this.inner.compress(toFloat32Array(embedding), accessFreq);
|
||||
}
|
||||
/**
|
||||
* Decompress a compressed tensor
|
||||
*
|
||||
* @param compressedJson - Compressed tensor JSON
|
||||
* @returns Decompressed embedding
|
||||
*/
|
||||
decompress(compressedJson) {
|
||||
return this.inner.decompress(compressedJson);
|
||||
}
|
||||
}
|
||||
exports.TensorCompress = TensorCompress;
|
||||
/**
|
||||
* Hierarchical forward pass through GNN layers
|
||||
*
|
||||
* @param query - Query vector
|
||||
* @param layerEmbeddings - Embeddings organized by layer
|
||||
* @param gnnLayersJson - JSON array of serialized GNN layers
|
||||
* @returns Final embedding after hierarchical processing as Float32Array
|
||||
*/
|
||||
function hierarchicalForward(query, layerEmbeddings, gnnLayersJson) {
|
||||
const gnn = getGnnModule();
|
||||
return gnn.hierarchicalForward(toFloat32Array(query), layerEmbeddings.map(layer => toFloat32ArrayBatch(layer)), gnnLayersJson);
|
||||
}
|
||||
/**
|
||||
* Get compression level for a given access frequency
|
||||
*/
|
||||
function getCompressionLevel(accessFreq) {
|
||||
const gnn = getGnnModule();
|
||||
return gnn.getCompressionLevel(accessFreq);
|
||||
}
|
||||
/**
|
||||
* Check if GNN module is available
|
||||
*/
|
||||
function isGnnAvailable() {
|
||||
try {
|
||||
getGnnModule();
|
||||
return true;
|
||||
}
|
||||
catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
exports.default = {
|
||||
differentiableSearch,
|
||||
RuvectorLayer,
|
||||
TensorCompress,
|
||||
hierarchicalForward,
|
||||
getCompressionLevel,
|
||||
isGnnAvailable,
|
||||
// Export conversion helpers for performance optimization
|
||||
toFloat32Array,
|
||||
toFloat32ArrayBatch,
|
||||
};
|
||||
//# sourceMappingURL=gnn-wrapper.js.map
|
||||
1
npm/packages/ruvector/src/core/gnn-wrapper.js.map
Normal file
1
npm/packages/ruvector/src/core/gnn-wrapper.js.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"gnn-wrapper.js","sourceRoot":"","sources":["gnn-wrapper.ts"],"names":[],"mappings":";AAAA;;;;;;;;;;;;;GAaG;;;AA6BH,wCAKC;AAKD,kDAMC;AAoCD,oDAaC;AAoGD,kDAWC;AAKD,kDAGC;AAKD,wCAOC;AA/ND,oDAAoD;AACpD,IAAI,SAAS,GAAQ,IAAI,CAAC;AAC1B,IAAI,SAAS,GAAiB,IAAI,CAAC;AAEnC,SAAS,YAAY;IACnB,IAAI,SAAS;QAAE,OAAO,SAAS,CAAC;IAChC,IAAI,SAAS;QAAE,MAAM,SAAS,CAAC;IAE/B,IAAI,CAAC;QACH,SAAS,GAAG,OAAO,CAAC,eAAe,CAAC,CAAC;QACrC,OAAO,SAAS,CAAC;IACnB,CAAC;IAAC,OAAO,CAAM,EAAE,CAAC;QAChB,SAAS,GAAG,IAAI,KAAK,CACnB,qDAAqD,CAAC,CAAC,OAAO,IAAI;YAClE,yCAAyC,CAC1C,CAAC;QACF,MAAM,SAAS,CAAC;IAClB,CAAC;AACH,CAAC;AAED;;;;;;GAMG;AACH,SAAgB,cAAc,CAAC,KAA6C;IAC1E,IAAI,KAAK,YAAY,YAAY;QAAE,OAAO,KAAK,CAAC;IAChD,IAAI,KAAK,YAAY,YAAY;QAAE,OAAO,IAAI,YAAY,CAAC,KAAK,CAAC,CAAC;IAClE,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC;QAAE,OAAO,IAAI,YAAY,CAAC,KAAK,CAAC,CAAC;IACzD,OAAO,IAAI,YAAY,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC;AAC7C,CAAC;AAED;;GAEG;AACH,SAAgB,mBAAmB,CAAC,KAAiD;IACnF,MAAM,MAAM,GAAG,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IACvC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACtC,MAAM,CAAC,CAAC,CAAC,GAAG,cAAc,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IACvC,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC;AAYD;;;;;;;;;;;;;;;;;;;;;;;GAuBG;AACH,SAAgB,oBAAoB,CAClC,KAA6C,EAC7C,UAAsD,EACtD,CAAS,EACT,cAAsB,GAAG;IAEzB,MAAM,GAAG,GAAG,YAAY,EAAE,CAAC;IAE3B,6EAA6E;IAC7E,MAAM,YAAY,GAAG,cAAc,CAAC,KAAK,CAAC,CAAC;IAC3C,MAAM,iBAAiB,GAAG,mBAAmB,CAAC,UAAU,CAAC,CAAC;IAE1D,OAAO,GAAG,CAAC,oBAAoB,CAAC,YAAY,EAAE,iBAAiB,EAAE,CAAC,EAAE,WAAW,CAAC,CAAC;AACnF,CAAC;AAED;;GAEG;AACH,MAAa,aAAa;IAGxB;;;;;;;OAOG;IACH,YAAY,QAAgB,EAAE,SAAiB,EAAE,KAAa,EAAE,UAAkB,GAAG;QACnF,MAAM,GAAG,GAAG,YAAY,EAAE,CAAC;QAC3B,IAAI,CAAC,KAAK,GAAG,IAAI,GAAG,CAAC,aAAa,CAAC,QAAQ,EAAE,SAAS,EAAE,KAAK,EAAE,OAAO,CAAC,CAAC;IAC1E,CAAC;IAED;;;;;;;OAOG;IACH,OAAO,CACL,aAAsC,EACtC,kBAA+C,EAC/C,WAAoC;QAEpC,OAAO,IAAI,CAAC,KAAK,CAAC,OAAO,CACvB,cAAc,CAAC,aAAa,CAAC,EAC7B,mBAAmB,CAAC,kBAAkB,CAAC,EACvC,cAAc,CAAC,WAAW,CAAC,CAC5B,CAAC;IACJ,CAAC;IAED;;OAEG;IACH,MAAM;QACJ,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC;IAC7B,CAAC;IAED;;OAEG;IACH,MAAM,CAAC,QAAQ,CAAC,IAAY;QAC1B,MAAM,GAAG,GAAG,YAAY,EAAE,CAAC;QAC3B,MAAM,KAAK,GAAG,IAAI,aAAa,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,oBAAoB;QACjE,KAAK,CAAC,KAAK,GAAG,GAAG,CAAC,aAAa,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;QAC/C,OAAO,KAAK,CAAC;IACf,CAAC;CACF;AApDD,sCAoDC;AAED;;GAEG;AACH,MAAa,cAAc;IAGzB;QACE,MAAM,GAAG,GAAG,YAAY,EAAE,CAAC;QAC3B,IAAI,CAAC,KAAK,GAAG,IAAI,GAAG,CAAC,cAAc,EAAE,CAAC;IACxC,CAAC;IAED;;;;;;OAMG;IACH,QAAQ,CAAC,SAAkC,EAAE,UAAkB;QAC7D,OAAO,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,cAAc,CAAC,SAAS,CAAC,EAAE,UAAU,CAAC,CAAC;IACpE,CAAC;IAED;;;;;OAKG;IACH,UAAU,CAAC,cAAsB;QAC/B,OAAO,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,cAAc,CAAC,CAAC;IAC/C,CAAC;CACF;AA5BD,wCA4BC;AAED;;;;;;;GAOG;AACH,SAAgB,mBAAmB,CACjC,KAA8B,EAC9B,eAA8C,EAC9C,aAAuB;IAEvB,MAAM,GAAG,GAAG,YAAY,EAAE,CAAC;IAC3B,OAAO,GAAG,CAAC,mBAAmB,CAC5B,cAAc,CAAC,KAAK,CAAC,EACrB,eAAe,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,CAAC,mBAAmB,CAAC,KAAK,CAAC,CAAC,EACxD,aAAa,CACd,CAAC;AACJ,CAAC;AAED;;GAEG;AACH,SAAgB,mBAAmB,CAAC,UAAkB;IACpD,MAAM,GAAG,GAAG,YAAY,EAAE,CAAC;IAC3B,OAAO,GAAG,CAAC,mBAAmB,CAAC,UAAU,CAAC,CAAC;AAC7C,CAAC;AAED;;GAEG;AACH,SAAgB,cAAc;IAC5B,IAAI,CAAC;QACH,YAAY,EAAE,CAAC;QACf,OAAO,IAAI,CAAC;IACd,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC;AAED,kBAAe;IACb,oBAAoB;IACpB,aAAa;IACb,cAAc;IACd,mBAAmB;IACnB,mBAAmB;IACnB,cAAc;IACd,yDAAyD;IACzD,cAAc;IACd,mBAAmB;CACpB,CAAC"}
|
||||
251
npm/packages/ruvector/src/core/gnn-wrapper.ts
Normal file
251
npm/packages/ruvector/src/core/gnn-wrapper.ts
Normal file
@@ -0,0 +1,251 @@
|
||||
/**
|
||||
* GNN Wrapper - Safe wrapper around @ruvector/gnn with automatic array conversion
|
||||
*
|
||||
* This wrapper handles the array type conversion automatically, allowing users
|
||||
* to pass either regular arrays or Float32Arrays.
|
||||
*
|
||||
* The native @ruvector/gnn requires Float32Array for maximum performance.
|
||||
* This wrapper converts any input type to Float32Array automatically.
|
||||
*
|
||||
* Performance Tips:
|
||||
* - Pass Float32Array directly for zero-copy performance
|
||||
* - Use toFloat32Array/toFloat32ArrayBatch for pre-conversion
|
||||
* - Avoid repeated conversions in hot paths
|
||||
*/
|
||||
|
||||
// Lazy load to avoid import errors if not installed
|
||||
let gnnModule: any = null;
|
||||
let loadError: Error | null = null;
|
||||
|
||||
function getGnnModule() {
|
||||
if (gnnModule) return gnnModule;
|
||||
if (loadError) throw loadError;
|
||||
|
||||
try {
|
||||
gnnModule = require('@ruvector/gnn');
|
||||
return gnnModule;
|
||||
} catch (e: any) {
|
||||
loadError = new Error(
|
||||
`@ruvector/gnn is not installed or failed to load: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/gnn`
|
||||
);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert any array-like input to Float32Array (native requires Float32Array)
|
||||
* Optimized paths:
|
||||
* - Float32Array: zero-copy return
|
||||
* - Float64Array: efficient typed array copy
|
||||
* - Array: direct Float32Array construction
|
||||
*/
|
||||
export function toFloat32Array(input: number[] | Float32Array | Float64Array): Float32Array {
|
||||
if (input instanceof Float32Array) return input;
|
||||
if (input instanceof Float64Array) return new Float32Array(input);
|
||||
if (Array.isArray(input)) return new Float32Array(input);
|
||||
return new Float32Array(Array.from(input));
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert array of arrays to array of Float32Arrays
|
||||
*/
|
||||
export function toFloat32ArrayBatch(input: (number[] | Float32Array | Float64Array)[]): Float32Array[] {
|
||||
const result = new Array(input.length);
|
||||
for (let i = 0; i < input.length; i++) {
|
||||
result[i] = toFloat32Array(input[i]);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Search result from differentiable search
|
||||
*/
|
||||
export interface DifferentiableSearchResult {
|
||||
/** Indices of top-k candidates */
|
||||
indices: number[];
|
||||
/** Soft weights for top-k candidates */
|
||||
weights: number[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Differentiable search using soft attention mechanism
|
||||
*
|
||||
* This wrapper automatically converts Float32Array inputs to regular arrays.
|
||||
*
|
||||
* @param query - Query vector (array or Float32Array)
|
||||
* @param candidates - List of candidate vectors (arrays or Float32Arrays)
|
||||
* @param k - Number of top results to return
|
||||
* @param temperature - Temperature for softmax (lower = sharper, higher = smoother)
|
||||
* @returns Search result with indices and soft weights
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { differentiableSearch } from 'ruvector/core/gnn-wrapper';
|
||||
*
|
||||
* // Works with regular arrays (auto-converted to Float32Array)
|
||||
* const result1 = differentiableSearch([1, 0, 0], [[1, 0, 0], [0, 1, 0]], 2, 1.0);
|
||||
*
|
||||
* // For best performance, use Float32Array directly (zero-copy)
|
||||
* const query = new Float32Array([1, 0, 0]);
|
||||
* const candidates = [new Float32Array([1, 0, 0]), new Float32Array([0, 1, 0])];
|
||||
* const result2 = differentiableSearch(query, candidates, 2, 1.0);
|
||||
* ```
|
||||
*/
|
||||
export function differentiableSearch(
|
||||
query: number[] | Float32Array | Float64Array,
|
||||
candidates: (number[] | Float32Array | Float64Array)[],
|
||||
k: number,
|
||||
temperature: number = 1.0
|
||||
): DifferentiableSearchResult {
|
||||
const gnn = getGnnModule();
|
||||
|
||||
// Convert to Float32Array (native Rust expects Float32Array for performance)
|
||||
const queryFloat32 = toFloat32Array(query);
|
||||
const candidatesFloat32 = toFloat32ArrayBatch(candidates);
|
||||
|
||||
return gnn.differentiableSearch(queryFloat32, candidatesFloat32, k, temperature);
|
||||
}
|
||||
|
||||
/**
|
||||
* GNN Layer for HNSW topology
|
||||
*/
|
||||
export class RuvectorLayer {
|
||||
private inner: any;
|
||||
|
||||
/**
|
||||
* Create a new Ruvector GNN layer
|
||||
*
|
||||
* @param inputDim - Dimension of input node embeddings
|
||||
* @param hiddenDim - Dimension of hidden representations
|
||||
* @param heads - Number of attention heads
|
||||
* @param dropout - Dropout rate (0.0 to 1.0)
|
||||
*/
|
||||
constructor(inputDim: number, hiddenDim: number, heads: number, dropout: number = 0.1) {
|
||||
const gnn = getGnnModule();
|
||||
this.inner = new gnn.RuvectorLayer(inputDim, hiddenDim, heads, dropout);
|
||||
}
|
||||
|
||||
/**
|
||||
* Forward pass through the GNN layer
|
||||
*
|
||||
* @param nodeEmbedding - Current node's embedding
|
||||
* @param neighborEmbeddings - Embeddings of neighbor nodes
|
||||
* @param edgeWeights - Weights of edges to neighbors
|
||||
* @returns Updated node embedding as Float32Array
|
||||
*/
|
||||
forward(
|
||||
nodeEmbedding: number[] | Float32Array,
|
||||
neighborEmbeddings: (number[] | Float32Array)[],
|
||||
edgeWeights: number[] | Float32Array
|
||||
): Float32Array {
|
||||
return this.inner.forward(
|
||||
toFloat32Array(nodeEmbedding),
|
||||
toFloat32ArrayBatch(neighborEmbeddings),
|
||||
toFloat32Array(edgeWeights)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize the layer to JSON
|
||||
*/
|
||||
toJson(): string {
|
||||
return this.inner.toJson();
|
||||
}
|
||||
|
||||
/**
|
||||
* Deserialize the layer from JSON
|
||||
*/
|
||||
static fromJson(json: string): RuvectorLayer {
|
||||
const gnn = getGnnModule();
|
||||
const layer = new RuvectorLayer(1, 1, 1, 0); // Dummy constructor
|
||||
layer.inner = gnn.RuvectorLayer.fromJson(json);
|
||||
return layer;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Tensor compressor with adaptive level selection
|
||||
*/
|
||||
export class TensorCompress {
|
||||
private inner: any;
|
||||
|
||||
constructor() {
|
||||
const gnn = getGnnModule();
|
||||
this.inner = new gnn.TensorCompress();
|
||||
}
|
||||
|
||||
/**
|
||||
* Compress an embedding based on access frequency
|
||||
*
|
||||
* @param embedding - Input embedding vector
|
||||
* @param accessFreq - Access frequency (0.0 to 1.0)
|
||||
* @returns Compressed tensor as JSON string
|
||||
*/
|
||||
compress(embedding: number[] | Float32Array, accessFreq: number): string {
|
||||
return this.inner.compress(toFloat32Array(embedding), accessFreq);
|
||||
}
|
||||
|
||||
/**
|
||||
* Decompress a compressed tensor
|
||||
*
|
||||
* @param compressedJson - Compressed tensor JSON
|
||||
* @returns Decompressed embedding
|
||||
*/
|
||||
decompress(compressedJson: string): number[] {
|
||||
return this.inner.decompress(compressedJson);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hierarchical forward pass through GNN layers
|
||||
*
|
||||
* @param query - Query vector
|
||||
* @param layerEmbeddings - Embeddings organized by layer
|
||||
* @param gnnLayersJson - JSON array of serialized GNN layers
|
||||
* @returns Final embedding after hierarchical processing as Float32Array
|
||||
*/
|
||||
export function hierarchicalForward(
|
||||
query: number[] | Float32Array,
|
||||
layerEmbeddings: (number[] | Float32Array)[][],
|
||||
gnnLayersJson: string[]
|
||||
): Float32Array {
|
||||
const gnn = getGnnModule();
|
||||
return gnn.hierarchicalForward(
|
||||
toFloat32Array(query),
|
||||
layerEmbeddings.map(layer => toFloat32ArrayBatch(layer)),
|
||||
gnnLayersJson
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get compression level for a given access frequency
|
||||
*/
|
||||
export function getCompressionLevel(accessFreq: number): string {
|
||||
const gnn = getGnnModule();
|
||||
return gnn.getCompressionLevel(accessFreq);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if GNN module is available
|
||||
*/
|
||||
export function isGnnAvailable(): boolean {
|
||||
try {
|
||||
getGnnModule();
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export default {
|
||||
differentiableSearch,
|
||||
RuvectorLayer,
|
||||
TensorCompress,
|
||||
hierarchicalForward,
|
||||
getCompressionLevel,
|
||||
isGnnAvailable,
|
||||
// Export conversion helpers for performance optimization
|
||||
toFloat32Array,
|
||||
toFloat32ArrayBatch,
|
||||
};
|
||||
83
npm/packages/ruvector/src/core/graph-algorithms.d.ts
vendored
Normal file
83
npm/packages/ruvector/src/core/graph-algorithms.d.ts
vendored
Normal file
@@ -0,0 +1,83 @@
|
||||
/**
|
||||
* Graph Algorithms - MinCut, Spectral Clustering, Community Detection
|
||||
*
|
||||
* Provides graph partitioning and clustering algorithms for:
|
||||
* - Code module detection
|
||||
* - Dependency clustering
|
||||
* - Architecture analysis
|
||||
* - Refactoring suggestions
|
||||
*/
|
||||
export interface Graph {
|
||||
nodes: string[];
|
||||
edges: Array<{
|
||||
from: string;
|
||||
to: string;
|
||||
weight?: number;
|
||||
}>;
|
||||
adjacency: Map<string, Map<string, number>>;
|
||||
}
|
||||
export interface Partition {
|
||||
groups: string[][];
|
||||
cutWeight: number;
|
||||
modularity: number;
|
||||
}
|
||||
export interface SpectralResult {
|
||||
clusters: Map<string, number>;
|
||||
eigenvalues: number[];
|
||||
coordinates: Map<string, number[]>;
|
||||
}
|
||||
/**
|
||||
* Build adjacency representation from edges
|
||||
*/
|
||||
export declare function buildGraph(nodes: string[], edges: Array<{
|
||||
from: string;
|
||||
to: string;
|
||||
weight?: number;
|
||||
}>): Graph;
|
||||
/**
|
||||
* Minimum Cut (Stoer-Wagner algorithm)
|
||||
*
|
||||
* Finds the minimum weight cut that partitions the graph into two parts.
|
||||
* Useful for finding loosely coupled module boundaries.
|
||||
*/
|
||||
export declare function minCut(graph: Graph): Partition;
|
||||
/**
|
||||
* Spectral Clustering (using power iteration)
|
||||
*
|
||||
* Uses graph Laplacian eigenvectors for clustering.
|
||||
* Good for finding natural clusters in code dependencies.
|
||||
*/
|
||||
export declare function spectralClustering(graph: Graph, k?: number): SpectralResult;
|
||||
/**
|
||||
* Louvain Community Detection
|
||||
*
|
||||
* Greedy modularity optimization for finding communities.
|
||||
* Good for detecting natural module boundaries.
|
||||
*/
|
||||
export declare function louvainCommunities(graph: Graph): Map<string, number>;
|
||||
/**
|
||||
* Calculate modularity of a partition
|
||||
*/
|
||||
export declare function calculateModularity(graph: Graph, partition: string[][]): number;
|
||||
/**
|
||||
* Find bridges (edges whose removal disconnects components)
|
||||
*/
|
||||
export declare function findBridges(graph: Graph): Array<{
|
||||
from: string;
|
||||
to: string;
|
||||
}>;
|
||||
/**
|
||||
* Find articulation points (nodes whose removal disconnects components)
|
||||
*/
|
||||
export declare function findArticulationPoints(graph: Graph): string[];
|
||||
declare const _default: {
|
||||
buildGraph: typeof buildGraph;
|
||||
minCut: typeof minCut;
|
||||
spectralClustering: typeof spectralClustering;
|
||||
louvainCommunities: typeof louvainCommunities;
|
||||
calculateModularity: typeof calculateModularity;
|
||||
findBridges: typeof findBridges;
|
||||
findArticulationPoints: typeof findArticulationPoints;
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=graph-algorithms.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/graph-algorithms.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/graph-algorithms.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"graph-algorithms.d.ts","sourceRoot":"","sources":["graph-algorithms.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,MAAM,WAAW,KAAK;IACpB,KAAK,EAAE,MAAM,EAAE,CAAC;IAChB,KAAK,EAAE,KAAK,CAAC;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,EAAE,EAAE,MAAM,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;IAC5D,SAAS,EAAE,GAAG,CAAC,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAC;CAC7C;AAED,MAAM,WAAW,SAAS;IACxB,MAAM,EAAE,MAAM,EAAE,EAAE,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC7B,QAAQ,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC9B,WAAW,EAAE,MAAM,EAAE,CAAC;IACtB,WAAW,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC;CACpC;AAED;;GAEG;AACH,wBAAgB,UAAU,CACxB,KAAK,EAAE,MAAM,EAAE,EACf,KAAK,EAAE,KAAK,CAAC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,EAAE,EAAE,MAAM,CAAC;IAAC,MAAM,CAAC,EAAE,MAAM,CAAA;CAAE,CAAC,GAC1D,KAAK,CAiBP;AAED;;;;;GAKG;AACH,wBAAgB,MAAM,CAAC,KAAK,EAAE,KAAK,GAAG,SAAS,CAwG9C;AAED;;;;;GAKG;AACH,wBAAgB,kBAAkB,CAAC,KAAK,EAAE,KAAK,EAAE,CAAC,GAAE,MAAU,GAAG,cAAc,CA0G9E;AAED;;;;;GAKG;AACH,wBAAgB,kBAAkB,CAAC,KAAK,EAAE,KAAK,GAAG,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,CAiGpE;AAED;;GAEG;AACH,wBAAgB,mBAAmB,CAAC,KAAK,EAAE,KAAK,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,GAAG,MAAM,CAgC/E;AAED;;GAEG;AACH,wBAAgB,WAAW,CAAC,KAAK,EAAE,KAAK,GAAG,KAAK,CAAC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,EAAE,EAAE,MAAM,CAAA;CAAE,CAAC,CAsC7E;AAED;;GAEG;AACH,wBAAgB,sBAAsB,CAAC,KAAK,EAAE,KAAK,GAAG,MAAM,EAAE,CA+C7D;;;;;;;;;;AA+FD,wBAQE"}
|
||||
515
npm/packages/ruvector/src/core/graph-algorithms.js
Normal file
515
npm/packages/ruvector/src/core/graph-algorithms.js
Normal file
@@ -0,0 +1,515 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Graph Algorithms - MinCut, Spectral Clustering, Community Detection
|
||||
*
|
||||
* Provides graph partitioning and clustering algorithms for:
|
||||
* - Code module detection
|
||||
* - Dependency clustering
|
||||
* - Architecture analysis
|
||||
* - Refactoring suggestions
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.buildGraph = buildGraph;
|
||||
exports.minCut = minCut;
|
||||
exports.spectralClustering = spectralClustering;
|
||||
exports.louvainCommunities = louvainCommunities;
|
||||
exports.calculateModularity = calculateModularity;
|
||||
exports.findBridges = findBridges;
|
||||
exports.findArticulationPoints = findArticulationPoints;
|
||||
/**
|
||||
* Build adjacency representation from edges
|
||||
*/
|
||||
function buildGraph(nodes, edges) {
|
||||
const adjacency = new Map();
|
||||
for (const node of nodes) {
|
||||
adjacency.set(node, new Map());
|
||||
}
|
||||
for (const { from, to, weight = 1 } of edges) {
|
||||
if (!adjacency.has(from))
|
||||
adjacency.set(from, new Map());
|
||||
if (!adjacency.has(to))
|
||||
adjacency.set(to, new Map());
|
||||
// Undirected graph - add both directions
|
||||
adjacency.get(from).set(to, weight);
|
||||
adjacency.get(to).set(from, weight);
|
||||
}
|
||||
return { nodes, edges, adjacency };
|
||||
}
|
||||
/**
|
||||
* Minimum Cut (Stoer-Wagner algorithm)
|
||||
*
|
||||
* Finds the minimum weight cut that partitions the graph into two parts.
|
||||
* Useful for finding loosely coupled module boundaries.
|
||||
*/
|
||||
function minCut(graph) {
|
||||
const n = graph.nodes.length;
|
||||
if (n < 2) {
|
||||
return { groups: [graph.nodes], cutWeight: 0, modularity: 0 };
|
||||
}
|
||||
// Copy adjacency for modification
|
||||
const adj = new Map();
|
||||
for (const [node, neighbors] of graph.adjacency) {
|
||||
adj.set(node, new Map(neighbors));
|
||||
}
|
||||
let minCutWeight = Infinity;
|
||||
let bestPartition = [];
|
||||
const merged = new Map(); // Track merged nodes
|
||||
for (const node of graph.nodes) {
|
||||
merged.set(node, [node]);
|
||||
}
|
||||
let remaining = [...graph.nodes];
|
||||
// Stoer-Wagner phases
|
||||
while (remaining.length > 1) {
|
||||
// Maximum adjacency search
|
||||
const inA = new Set([remaining[0]]);
|
||||
const weights = new Map();
|
||||
for (const node of remaining) {
|
||||
if (!inA.has(node)) {
|
||||
weights.set(node, adj.get(remaining[0])?.get(node) || 0);
|
||||
}
|
||||
}
|
||||
let lastAdded = remaining[0];
|
||||
let beforeLast = remaining[0];
|
||||
while (inA.size < remaining.length) {
|
||||
// Find node with maximum weight to A
|
||||
let maxWeight = -Infinity;
|
||||
let maxNode = '';
|
||||
for (const [node, weight] of weights) {
|
||||
if (!inA.has(node) && weight > maxWeight) {
|
||||
maxWeight = weight;
|
||||
maxNode = node;
|
||||
}
|
||||
}
|
||||
if (!maxNode)
|
||||
break;
|
||||
beforeLast = lastAdded;
|
||||
lastAdded = maxNode;
|
||||
inA.add(maxNode);
|
||||
// Update weights
|
||||
for (const [neighbor, w] of adj.get(maxNode) || []) {
|
||||
if (!inA.has(neighbor)) {
|
||||
weights.set(neighbor, (weights.get(neighbor) || 0) + w);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Cut of the phase
|
||||
const cutWeight = weights.get(lastAdded) || 0;
|
||||
if (cutWeight < minCutWeight) {
|
||||
minCutWeight = cutWeight;
|
||||
const lastGroup = merged.get(lastAdded) || [lastAdded];
|
||||
const otherNodes = remaining.filter(n => n !== lastAdded).flatMap(n => merged.get(n) || [n]);
|
||||
bestPartition = [lastGroup, otherNodes];
|
||||
}
|
||||
// Merge last two nodes
|
||||
if (remaining.length > 1) {
|
||||
// Merge lastAdded into beforeLast
|
||||
const mergedNodes = [...(merged.get(beforeLast) || []), ...(merged.get(lastAdded) || [])];
|
||||
merged.set(beforeLast, mergedNodes);
|
||||
// Update adjacency
|
||||
for (const [neighbor, w] of adj.get(lastAdded) || []) {
|
||||
if (neighbor !== beforeLast) {
|
||||
const current = adj.get(beforeLast)?.get(neighbor) || 0;
|
||||
adj.get(beforeLast)?.set(neighbor, current + w);
|
||||
adj.get(neighbor)?.set(beforeLast, current + w);
|
||||
}
|
||||
}
|
||||
// Remove lastAdded
|
||||
remaining = remaining.filter(n => n !== lastAdded);
|
||||
adj.delete(lastAdded);
|
||||
for (const [, neighbors] of adj) {
|
||||
neighbors.delete(lastAdded);
|
||||
}
|
||||
}
|
||||
}
|
||||
const modularity = calculateModularity(graph, bestPartition);
|
||||
return {
|
||||
groups: bestPartition.filter(g => g.length > 0),
|
||||
cutWeight: minCutWeight,
|
||||
modularity,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Spectral Clustering (using power iteration)
|
||||
*
|
||||
* Uses graph Laplacian eigenvectors for clustering.
|
||||
* Good for finding natural clusters in code dependencies.
|
||||
*/
|
||||
function spectralClustering(graph, k = 2) {
|
||||
const n = graph.nodes.length;
|
||||
const nodeIndex = new Map(graph.nodes.map((node, i) => [node, i]));
|
||||
const clusters = new Map();
|
||||
if (n === 0) {
|
||||
return { clusters, eigenvalues: [], coordinates: new Map() };
|
||||
}
|
||||
// Build Laplacian matrix (D - A)
|
||||
const degree = new Float64Array(n);
|
||||
const laplacian = Array(n).fill(null).map(() => Array(n).fill(0));
|
||||
for (const [node, neighbors] of graph.adjacency) {
|
||||
const i = nodeIndex.get(node);
|
||||
let d = 0;
|
||||
for (const [neighbor, weight] of neighbors) {
|
||||
const j = nodeIndex.get(neighbor);
|
||||
laplacian[i][j] = -weight;
|
||||
d += weight;
|
||||
}
|
||||
degree[i] = d;
|
||||
laplacian[i][i] = d;
|
||||
}
|
||||
// Normalized Laplacian: D^(-1/2) L D^(-1/2)
|
||||
for (let i = 0; i < n; i++) {
|
||||
for (let j = 0; j < n; j++) {
|
||||
if (degree[i] > 0 && degree[j] > 0) {
|
||||
laplacian[i][j] /= Math.sqrt(degree[i] * degree[j]);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Power iteration to find eigenvectors
|
||||
const eigenvectors = [];
|
||||
const eigenvalues = [];
|
||||
for (let ev = 0; ev < Math.min(k, n); ev++) {
|
||||
let vector = new Float64Array(n);
|
||||
for (let i = 0; i < n; i++) {
|
||||
vector[i] = Math.random();
|
||||
}
|
||||
normalize(vector);
|
||||
// Deflation: orthogonalize against previous eigenvectors
|
||||
for (const prev of eigenvectors) {
|
||||
const dot = dotProduct(vector, new Float64Array(prev));
|
||||
for (let i = 0; i < n; i++) {
|
||||
vector[i] -= dot * prev[i];
|
||||
}
|
||||
}
|
||||
normalize(vector);
|
||||
// Power iteration
|
||||
for (let iter = 0; iter < 100; iter++) {
|
||||
const newVector = new Float64Array(n);
|
||||
for (let i = 0; i < n; i++) {
|
||||
for (let j = 0; j < n; j++) {
|
||||
newVector[i] += laplacian[i][j] * vector[j];
|
||||
}
|
||||
}
|
||||
// Deflation
|
||||
for (const prev of eigenvectors) {
|
||||
const dot = dotProduct(newVector, new Float64Array(prev));
|
||||
for (let i = 0; i < n; i++) {
|
||||
newVector[i] -= dot * prev[i];
|
||||
}
|
||||
}
|
||||
normalize(newVector);
|
||||
vector = newVector;
|
||||
}
|
||||
// Compute eigenvalue
|
||||
let eigenvalue = 0;
|
||||
for (let i = 0; i < n; i++) {
|
||||
let sum = 0;
|
||||
for (let j = 0; j < n; j++) {
|
||||
sum += laplacian[i][j] * vector[j];
|
||||
}
|
||||
eigenvalue += vector[i] * sum;
|
||||
}
|
||||
eigenvectors.push(Array.from(vector));
|
||||
eigenvalues.push(eigenvalue);
|
||||
}
|
||||
// K-means clustering on eigenvector coordinates
|
||||
const coordinates = new Map();
|
||||
for (let i = 0; i < n; i++) {
|
||||
coordinates.set(graph.nodes[i], eigenvectors.map(ev => ev[i]));
|
||||
}
|
||||
// Simple k-means
|
||||
const clusterAssignment = kMeans(graph.nodes.map(node => coordinates.get(node)), k);
|
||||
for (let i = 0; i < n; i++) {
|
||||
clusters.set(graph.nodes[i], clusterAssignment[i]);
|
||||
}
|
||||
return { clusters, eigenvalues, coordinates };
|
||||
}
|
||||
/**
|
||||
* Louvain Community Detection
|
||||
*
|
||||
* Greedy modularity optimization for finding communities.
|
||||
* Good for detecting natural module boundaries.
|
||||
*/
|
||||
function louvainCommunities(graph) {
|
||||
const communities = new Map();
|
||||
let communityId = 0;
|
||||
// Initialize: each node in its own community
|
||||
for (const node of graph.nodes) {
|
||||
communities.set(node, communityId++);
|
||||
}
|
||||
// Total edge weight
|
||||
let m = 0;
|
||||
for (const { weight = 1 } of graph.edges) {
|
||||
m += weight;
|
||||
}
|
||||
m /= 2; // Undirected
|
||||
if (m === 0)
|
||||
return communities;
|
||||
// Node weights (sum of edge weights)
|
||||
const nodeWeight = new Map();
|
||||
for (const node of graph.nodes) {
|
||||
let w = 0;
|
||||
for (const [, weight] of graph.adjacency.get(node) || []) {
|
||||
w += weight;
|
||||
}
|
||||
nodeWeight.set(node, w);
|
||||
}
|
||||
// Community weights
|
||||
const communityWeight = new Map();
|
||||
for (const node of graph.nodes) {
|
||||
const c = communities.get(node);
|
||||
communityWeight.set(c, (communityWeight.get(c) || 0) + (nodeWeight.get(node) || 0));
|
||||
}
|
||||
// Iterate until no improvement
|
||||
let improved = true;
|
||||
while (improved) {
|
||||
improved = false;
|
||||
for (const node of graph.nodes) {
|
||||
const currentCommunity = communities.get(node);
|
||||
const ki = nodeWeight.get(node) || 0;
|
||||
// Calculate modularity gain for moving to neighbor communities
|
||||
let bestCommunity = currentCommunity;
|
||||
let bestGain = 0;
|
||||
const neighborCommunities = new Set();
|
||||
for (const [neighbor] of graph.adjacency.get(node) || []) {
|
||||
neighborCommunities.add(communities.get(neighbor));
|
||||
}
|
||||
for (const targetCommunity of neighborCommunities) {
|
||||
if (targetCommunity === currentCommunity)
|
||||
continue;
|
||||
// Calculate edge weight to target community
|
||||
let ki_in = 0;
|
||||
for (const [neighbor, weight] of graph.adjacency.get(node) || []) {
|
||||
if (communities.get(neighbor) === targetCommunity) {
|
||||
ki_in += weight;
|
||||
}
|
||||
}
|
||||
const sumTot = communityWeight.get(targetCommunity) || 0;
|
||||
const gain = ki_in / m - (ki * sumTot) / (2 * m * m);
|
||||
if (gain > bestGain) {
|
||||
bestGain = gain;
|
||||
bestCommunity = targetCommunity;
|
||||
}
|
||||
}
|
||||
// Move node if beneficial
|
||||
if (bestCommunity !== currentCommunity) {
|
||||
communities.set(node, bestCommunity);
|
||||
// Update community weights
|
||||
communityWeight.set(currentCommunity, (communityWeight.get(currentCommunity) || 0) - ki);
|
||||
communityWeight.set(bestCommunity, (communityWeight.get(bestCommunity) || 0) + ki);
|
||||
improved = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Renumber communities to be contiguous
|
||||
const renumber = new Map();
|
||||
let newId = 0;
|
||||
for (const [node, c] of communities) {
|
||||
if (!renumber.has(c)) {
|
||||
renumber.set(c, newId++);
|
||||
}
|
||||
communities.set(node, renumber.get(c));
|
||||
}
|
||||
return communities;
|
||||
}
|
||||
/**
|
||||
* Calculate modularity of a partition
|
||||
*/
|
||||
function calculateModularity(graph, partition) {
|
||||
let m = 0;
|
||||
for (const { weight = 1 } of graph.edges) {
|
||||
m += weight;
|
||||
}
|
||||
m /= 2;
|
||||
if (m === 0)
|
||||
return 0;
|
||||
let modularity = 0;
|
||||
for (const group of partition) {
|
||||
const groupSet = new Set(group);
|
||||
// Edges within group
|
||||
let inGroup = 0;
|
||||
let degreeSum = 0;
|
||||
for (const node of group) {
|
||||
for (const [neighbor, weight] of graph.adjacency.get(node) || []) {
|
||||
if (groupSet.has(neighbor)) {
|
||||
inGroup += weight;
|
||||
}
|
||||
degreeSum += weight;
|
||||
}
|
||||
}
|
||||
inGroup /= 2; // Count each edge once
|
||||
modularity += inGroup / m - Math.pow(degreeSum / (2 * m), 2);
|
||||
}
|
||||
return modularity;
|
||||
}
|
||||
/**
|
||||
* Find bridges (edges whose removal disconnects components)
|
||||
*/
|
||||
function findBridges(graph) {
|
||||
const bridges = [];
|
||||
const visited = new Set();
|
||||
const discovery = new Map();
|
||||
const low = new Map();
|
||||
const parent = new Map();
|
||||
let time = 0;
|
||||
function dfs(node) {
|
||||
visited.add(node);
|
||||
discovery.set(node, time);
|
||||
low.set(node, time);
|
||||
time++;
|
||||
for (const [neighbor] of graph.adjacency.get(node) || []) {
|
||||
if (!visited.has(neighbor)) {
|
||||
parent.set(neighbor, node);
|
||||
dfs(neighbor);
|
||||
low.set(node, Math.min(low.get(node), low.get(neighbor)));
|
||||
if (low.get(neighbor) > discovery.get(node)) {
|
||||
bridges.push({ from: node, to: neighbor });
|
||||
}
|
||||
}
|
||||
else if (neighbor !== parent.get(node)) {
|
||||
low.set(node, Math.min(low.get(node), discovery.get(neighbor)));
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const node of graph.nodes) {
|
||||
if (!visited.has(node)) {
|
||||
parent.set(node, null);
|
||||
dfs(node);
|
||||
}
|
||||
}
|
||||
return bridges;
|
||||
}
|
||||
/**
|
||||
* Find articulation points (nodes whose removal disconnects components)
|
||||
*/
|
||||
function findArticulationPoints(graph) {
|
||||
const points = [];
|
||||
const visited = new Set();
|
||||
const discovery = new Map();
|
||||
const low = new Map();
|
||||
const parent = new Map();
|
||||
let time = 0;
|
||||
function dfs(node) {
|
||||
visited.add(node);
|
||||
discovery.set(node, time);
|
||||
low.set(node, time);
|
||||
time++;
|
||||
let children = 0;
|
||||
for (const [neighbor] of graph.adjacency.get(node) || []) {
|
||||
if (!visited.has(neighbor)) {
|
||||
children++;
|
||||
parent.set(neighbor, node);
|
||||
dfs(neighbor);
|
||||
low.set(node, Math.min(low.get(node), low.get(neighbor)));
|
||||
// Root with 2+ children or non-root with low[v] >= disc[u]
|
||||
if ((parent.get(node) === null && children > 1) ||
|
||||
(parent.get(node) !== null && low.get(neighbor) >= discovery.get(node))) {
|
||||
if (!points.includes(node)) {
|
||||
points.push(node);
|
||||
}
|
||||
}
|
||||
}
|
||||
else if (neighbor !== parent.get(node)) {
|
||||
low.set(node, Math.min(low.get(node), discovery.get(neighbor)));
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const node of graph.nodes) {
|
||||
if (!visited.has(node)) {
|
||||
parent.set(node, null);
|
||||
dfs(node);
|
||||
}
|
||||
}
|
||||
return points;
|
||||
}
|
||||
// Helper functions
|
||||
function normalize(v) {
|
||||
let sum = 0;
|
||||
for (let i = 0; i < v.length; i++) {
|
||||
sum += v[i] * v[i];
|
||||
}
|
||||
const norm = Math.sqrt(sum);
|
||||
if (norm > 0) {
|
||||
for (let i = 0; i < v.length; i++) {
|
||||
v[i] /= norm;
|
||||
}
|
||||
}
|
||||
}
|
||||
function dotProduct(a, b) {
|
||||
let sum = 0;
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
sum += a[i] * b[i];
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
function kMeans(points, k, maxIter = 100) {
|
||||
const n = points.length;
|
||||
if (n === 0 || k === 0)
|
||||
return [];
|
||||
const dim = points[0].length;
|
||||
// Random initialization
|
||||
const centroids = [];
|
||||
const used = new Set();
|
||||
while (centroids.length < Math.min(k, n)) {
|
||||
const idx = Math.floor(Math.random() * n);
|
||||
if (!used.has(idx)) {
|
||||
used.add(idx);
|
||||
centroids.push([...points[idx]]);
|
||||
}
|
||||
}
|
||||
const assignment = new Array(n).fill(0);
|
||||
for (let iter = 0; iter < maxIter; iter++) {
|
||||
// Assign points to nearest centroid
|
||||
let changed = false;
|
||||
for (let i = 0; i < n; i++) {
|
||||
let minDist = Infinity;
|
||||
let minC = 0;
|
||||
for (let c = 0; c < centroids.length; c++) {
|
||||
let dist = 0;
|
||||
for (let d = 0; d < dim; d++) {
|
||||
dist += Math.pow(points[i][d] - centroids[c][d], 2);
|
||||
}
|
||||
if (dist < minDist) {
|
||||
minDist = dist;
|
||||
minC = c;
|
||||
}
|
||||
}
|
||||
if (assignment[i] !== minC) {
|
||||
assignment[i] = minC;
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
if (!changed)
|
||||
break;
|
||||
// Update centroids
|
||||
const counts = new Array(k).fill(0);
|
||||
for (let c = 0; c < centroids.length; c++) {
|
||||
for (let d = 0; d < dim; d++) {
|
||||
centroids[c][d] = 0;
|
||||
}
|
||||
}
|
||||
for (let i = 0; i < n; i++) {
|
||||
const c = assignment[i];
|
||||
counts[c]++;
|
||||
for (let d = 0; d < dim; d++) {
|
||||
centroids[c][d] += points[i][d];
|
||||
}
|
||||
}
|
||||
for (let c = 0; c < centroids.length; c++) {
|
||||
if (counts[c] > 0) {
|
||||
for (let d = 0; d < dim; d++) {
|
||||
centroids[c][d] /= counts[c];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return assignment;
|
||||
}
|
||||
exports.default = {
|
||||
buildGraph,
|
||||
minCut,
|
||||
spectralClustering,
|
||||
louvainCommunities,
|
||||
calculateModularity,
|
||||
findBridges,
|
||||
findArticulationPoints,
|
||||
};
|
||||
//# sourceMappingURL=graph-algorithms.js.map
|
||||
1
npm/packages/ruvector/src/core/graph-algorithms.js.map
Normal file
1
npm/packages/ruvector/src/core/graph-algorithms.js.map
Normal file
File diff suppressed because one or more lines are too long
618
npm/packages/ruvector/src/core/graph-algorithms.ts
Normal file
618
npm/packages/ruvector/src/core/graph-algorithms.ts
Normal file
@@ -0,0 +1,618 @@
|
||||
/**
|
||||
* Graph Algorithms - MinCut, Spectral Clustering, Community Detection
|
||||
*
|
||||
* Provides graph partitioning and clustering algorithms for:
|
||||
* - Code module detection
|
||||
* - Dependency clustering
|
||||
* - Architecture analysis
|
||||
* - Refactoring suggestions
|
||||
*/
|
||||
|
||||
export interface Graph {
|
||||
nodes: string[];
|
||||
edges: Array<{ from: string; to: string; weight?: number }>;
|
||||
adjacency: Map<string, Map<string, number>>;
|
||||
}
|
||||
|
||||
export interface Partition {
|
||||
groups: string[][];
|
||||
cutWeight: number;
|
||||
modularity: number;
|
||||
}
|
||||
|
||||
export interface SpectralResult {
|
||||
clusters: Map<string, number>;
|
||||
eigenvalues: number[];
|
||||
coordinates: Map<string, number[]>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build adjacency representation from edges
|
||||
*/
|
||||
export function buildGraph(
|
||||
nodes: string[],
|
||||
edges: Array<{ from: string; to: string; weight?: number }>
|
||||
): Graph {
|
||||
const adjacency = new Map<string, Map<string, number>>();
|
||||
|
||||
for (const node of nodes) {
|
||||
adjacency.set(node, new Map());
|
||||
}
|
||||
|
||||
for (const { from, to, weight = 1 } of edges) {
|
||||
if (!adjacency.has(from)) adjacency.set(from, new Map());
|
||||
if (!adjacency.has(to)) adjacency.set(to, new Map());
|
||||
|
||||
// Undirected graph - add both directions
|
||||
adjacency.get(from)!.set(to, weight);
|
||||
adjacency.get(to)!.set(from, weight);
|
||||
}
|
||||
|
||||
return { nodes, edges, adjacency };
|
||||
}
|
||||
|
||||
/**
|
||||
* Minimum Cut (Stoer-Wagner algorithm)
|
||||
*
|
||||
* Finds the minimum weight cut that partitions the graph into two parts.
|
||||
* Useful for finding loosely coupled module boundaries.
|
||||
*/
|
||||
export function minCut(graph: Graph): Partition {
|
||||
const n = graph.nodes.length;
|
||||
if (n < 2) {
|
||||
return { groups: [graph.nodes], cutWeight: 0, modularity: 0 };
|
||||
}
|
||||
|
||||
// Copy adjacency for modification
|
||||
const adj = new Map<string, Map<string, number>>();
|
||||
for (const [node, neighbors] of graph.adjacency) {
|
||||
adj.set(node, new Map(neighbors));
|
||||
}
|
||||
|
||||
let minCutWeight = Infinity;
|
||||
let bestPartition: string[][] = [];
|
||||
const merged = new Map<string, string[]>(); // Track merged nodes
|
||||
|
||||
for (const node of graph.nodes) {
|
||||
merged.set(node, [node]);
|
||||
}
|
||||
|
||||
let remaining = [...graph.nodes];
|
||||
|
||||
// Stoer-Wagner phases
|
||||
while (remaining.length > 1) {
|
||||
// Maximum adjacency search
|
||||
const inA = new Set<string>([remaining[0]]);
|
||||
const weights = new Map<string, number>();
|
||||
|
||||
for (const node of remaining) {
|
||||
if (!inA.has(node)) {
|
||||
weights.set(node, adj.get(remaining[0])?.get(node) || 0);
|
||||
}
|
||||
}
|
||||
|
||||
let lastAdded = remaining[0];
|
||||
let beforeLast = remaining[0];
|
||||
|
||||
while (inA.size < remaining.length) {
|
||||
// Find node with maximum weight to A
|
||||
let maxWeight = -Infinity;
|
||||
let maxNode = '';
|
||||
|
||||
for (const [node, weight] of weights) {
|
||||
if (!inA.has(node) && weight > maxWeight) {
|
||||
maxWeight = weight;
|
||||
maxNode = node;
|
||||
}
|
||||
}
|
||||
|
||||
if (!maxNode) break;
|
||||
|
||||
beforeLast = lastAdded;
|
||||
lastAdded = maxNode;
|
||||
inA.add(maxNode);
|
||||
|
||||
// Update weights
|
||||
for (const [neighbor, w] of adj.get(maxNode) || []) {
|
||||
if (!inA.has(neighbor)) {
|
||||
weights.set(neighbor, (weights.get(neighbor) || 0) + w);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Cut of the phase
|
||||
const cutWeight = weights.get(lastAdded) || 0;
|
||||
|
||||
if (cutWeight < minCutWeight) {
|
||||
minCutWeight = cutWeight;
|
||||
const lastGroup = merged.get(lastAdded) || [lastAdded];
|
||||
const otherNodes = remaining.filter(n => n !== lastAdded).flatMap(n => merged.get(n) || [n]);
|
||||
bestPartition = [lastGroup, otherNodes];
|
||||
}
|
||||
|
||||
// Merge last two nodes
|
||||
if (remaining.length > 1) {
|
||||
// Merge lastAdded into beforeLast
|
||||
const mergedNodes = [...(merged.get(beforeLast) || []), ...(merged.get(lastAdded) || [])];
|
||||
merged.set(beforeLast, mergedNodes);
|
||||
|
||||
// Update adjacency
|
||||
for (const [neighbor, w] of adj.get(lastAdded) || []) {
|
||||
if (neighbor !== beforeLast) {
|
||||
const current = adj.get(beforeLast)?.get(neighbor) || 0;
|
||||
adj.get(beforeLast)?.set(neighbor, current + w);
|
||||
adj.get(neighbor)?.set(beforeLast, current + w);
|
||||
}
|
||||
}
|
||||
|
||||
// Remove lastAdded
|
||||
remaining = remaining.filter(n => n !== lastAdded);
|
||||
adj.delete(lastAdded);
|
||||
for (const [, neighbors] of adj) {
|
||||
neighbors.delete(lastAdded);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const modularity = calculateModularity(graph, bestPartition);
|
||||
|
||||
return {
|
||||
groups: bestPartition.filter(g => g.length > 0),
|
||||
cutWeight: minCutWeight,
|
||||
modularity,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Spectral Clustering (using power iteration)
|
||||
*
|
||||
* Uses graph Laplacian eigenvectors for clustering.
|
||||
* Good for finding natural clusters in code dependencies.
|
||||
*/
|
||||
export function spectralClustering(graph: Graph, k: number = 2): SpectralResult {
|
||||
const n = graph.nodes.length;
|
||||
const nodeIndex = new Map(graph.nodes.map((node, i) => [node, i]));
|
||||
const clusters = new Map<string, number>();
|
||||
|
||||
if (n === 0) {
|
||||
return { clusters, eigenvalues: [], coordinates: new Map() };
|
||||
}
|
||||
|
||||
// Build Laplacian matrix (D - A)
|
||||
const degree = new Float64Array(n);
|
||||
const laplacian: number[][] = Array(n).fill(null).map(() => Array(n).fill(0));
|
||||
|
||||
for (const [node, neighbors] of graph.adjacency) {
|
||||
const i = nodeIndex.get(node)!;
|
||||
let d = 0;
|
||||
for (const [neighbor, weight] of neighbors) {
|
||||
const j = nodeIndex.get(neighbor)!;
|
||||
laplacian[i][j] = -weight;
|
||||
d += weight;
|
||||
}
|
||||
degree[i] = d;
|
||||
laplacian[i][i] = d;
|
||||
}
|
||||
|
||||
// Normalized Laplacian: D^(-1/2) L D^(-1/2)
|
||||
for (let i = 0; i < n; i++) {
|
||||
for (let j = 0; j < n; j++) {
|
||||
if (degree[i] > 0 && degree[j] > 0) {
|
||||
laplacian[i][j] /= Math.sqrt(degree[i] * degree[j]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Power iteration to find eigenvectors
|
||||
const eigenvectors: number[][] = [];
|
||||
const eigenvalues: number[] = [];
|
||||
|
||||
for (let ev = 0; ev < Math.min(k, n); ev++) {
|
||||
let vector = new Float64Array(n);
|
||||
for (let i = 0; i < n; i++) {
|
||||
vector[i] = Math.random();
|
||||
}
|
||||
normalize(vector);
|
||||
|
||||
// Deflation: orthogonalize against previous eigenvectors
|
||||
for (const prev of eigenvectors) {
|
||||
const dot = dotProduct(vector, new Float64Array(prev));
|
||||
for (let i = 0; i < n; i++) {
|
||||
vector[i] -= dot * prev[i];
|
||||
}
|
||||
}
|
||||
normalize(vector);
|
||||
|
||||
// Power iteration
|
||||
for (let iter = 0; iter < 100; iter++) {
|
||||
const newVector = new Float64Array(n);
|
||||
for (let i = 0; i < n; i++) {
|
||||
for (let j = 0; j < n; j++) {
|
||||
newVector[i] += laplacian[i][j] * vector[j];
|
||||
}
|
||||
}
|
||||
|
||||
// Deflation
|
||||
for (const prev of eigenvectors) {
|
||||
const dot = dotProduct(newVector, new Float64Array(prev));
|
||||
for (let i = 0; i < n; i++) {
|
||||
newVector[i] -= dot * prev[i];
|
||||
}
|
||||
}
|
||||
|
||||
normalize(newVector);
|
||||
vector = newVector;
|
||||
}
|
||||
|
||||
// Compute eigenvalue
|
||||
let eigenvalue = 0;
|
||||
for (let i = 0; i < n; i++) {
|
||||
let sum = 0;
|
||||
for (let j = 0; j < n; j++) {
|
||||
sum += laplacian[i][j] * vector[j];
|
||||
}
|
||||
eigenvalue += vector[i] * sum;
|
||||
}
|
||||
|
||||
eigenvectors.push(Array.from(vector));
|
||||
eigenvalues.push(eigenvalue);
|
||||
}
|
||||
|
||||
// K-means clustering on eigenvector coordinates
|
||||
const coordinates = new Map<string, number[]>();
|
||||
for (let i = 0; i < n; i++) {
|
||||
coordinates.set(graph.nodes[i], eigenvectors.map(ev => ev[i]));
|
||||
}
|
||||
|
||||
// Simple k-means
|
||||
const clusterAssignment = kMeans(
|
||||
graph.nodes.map(node => coordinates.get(node)!),
|
||||
k
|
||||
);
|
||||
|
||||
for (let i = 0; i < n; i++) {
|
||||
clusters.set(graph.nodes[i], clusterAssignment[i]);
|
||||
}
|
||||
|
||||
return { clusters, eigenvalues, coordinates };
|
||||
}
|
||||
|
||||
/**
|
||||
* Louvain Community Detection
|
||||
*
|
||||
* Greedy modularity optimization for finding communities.
|
||||
* Good for detecting natural module boundaries.
|
||||
*/
|
||||
export function louvainCommunities(graph: Graph): Map<string, number> {
|
||||
const communities = new Map<string, number>();
|
||||
let communityId = 0;
|
||||
|
||||
// Initialize: each node in its own community
|
||||
for (const node of graph.nodes) {
|
||||
communities.set(node, communityId++);
|
||||
}
|
||||
|
||||
// Total edge weight
|
||||
let m = 0;
|
||||
for (const { weight = 1 } of graph.edges) {
|
||||
m += weight;
|
||||
}
|
||||
m /= 2; // Undirected
|
||||
|
||||
if (m === 0) return communities;
|
||||
|
||||
// Node weights (sum of edge weights)
|
||||
const nodeWeight = new Map<string, number>();
|
||||
for (const node of graph.nodes) {
|
||||
let w = 0;
|
||||
for (const [, weight] of graph.adjacency.get(node) || []) {
|
||||
w += weight;
|
||||
}
|
||||
nodeWeight.set(node, w);
|
||||
}
|
||||
|
||||
// Community weights
|
||||
const communityWeight = new Map<number, number>();
|
||||
for (const node of graph.nodes) {
|
||||
const c = communities.get(node)!;
|
||||
communityWeight.set(c, (communityWeight.get(c) || 0) + (nodeWeight.get(node) || 0));
|
||||
}
|
||||
|
||||
// Iterate until no improvement
|
||||
let improved = true;
|
||||
while (improved) {
|
||||
improved = false;
|
||||
|
||||
for (const node of graph.nodes) {
|
||||
const currentCommunity = communities.get(node)!;
|
||||
const ki = nodeWeight.get(node) || 0;
|
||||
|
||||
// Calculate modularity gain for moving to neighbor communities
|
||||
let bestCommunity = currentCommunity;
|
||||
let bestGain = 0;
|
||||
|
||||
const neighborCommunities = new Set<number>();
|
||||
for (const [neighbor] of graph.adjacency.get(node) || []) {
|
||||
neighborCommunities.add(communities.get(neighbor)!);
|
||||
}
|
||||
|
||||
for (const targetCommunity of neighborCommunities) {
|
||||
if (targetCommunity === currentCommunity) continue;
|
||||
|
||||
// Calculate edge weight to target community
|
||||
let ki_in = 0;
|
||||
for (const [neighbor, weight] of graph.adjacency.get(node) || []) {
|
||||
if (communities.get(neighbor) === targetCommunity) {
|
||||
ki_in += weight;
|
||||
}
|
||||
}
|
||||
|
||||
const sumTot = communityWeight.get(targetCommunity) || 0;
|
||||
const gain = ki_in / m - (ki * sumTot) / (2 * m * m);
|
||||
|
||||
if (gain > bestGain) {
|
||||
bestGain = gain;
|
||||
bestCommunity = targetCommunity;
|
||||
}
|
||||
}
|
||||
|
||||
// Move node if beneficial
|
||||
if (bestCommunity !== currentCommunity) {
|
||||
communities.set(node, bestCommunity);
|
||||
|
||||
// Update community weights
|
||||
communityWeight.set(currentCommunity, (communityWeight.get(currentCommunity) || 0) - ki);
|
||||
communityWeight.set(bestCommunity, (communityWeight.get(bestCommunity) || 0) + ki);
|
||||
|
||||
improved = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Renumber communities to be contiguous
|
||||
const renumber = new Map<number, number>();
|
||||
let newId = 0;
|
||||
for (const [node, c] of communities) {
|
||||
if (!renumber.has(c)) {
|
||||
renumber.set(c, newId++);
|
||||
}
|
||||
communities.set(node, renumber.get(c)!);
|
||||
}
|
||||
|
||||
return communities;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate modularity of a partition
|
||||
*/
|
||||
export function calculateModularity(graph: Graph, partition: string[][]): number {
|
||||
let m = 0;
|
||||
for (const { weight = 1 } of graph.edges) {
|
||||
m += weight;
|
||||
}
|
||||
m /= 2;
|
||||
|
||||
if (m === 0) return 0;
|
||||
|
||||
let modularity = 0;
|
||||
|
||||
for (const group of partition) {
|
||||
const groupSet = new Set(group);
|
||||
|
||||
// Edges within group
|
||||
let inGroup = 0;
|
||||
let degreeSum = 0;
|
||||
|
||||
for (const node of group) {
|
||||
for (const [neighbor, weight] of graph.adjacency.get(node) || []) {
|
||||
if (groupSet.has(neighbor)) {
|
||||
inGroup += weight;
|
||||
}
|
||||
degreeSum += weight;
|
||||
}
|
||||
}
|
||||
|
||||
inGroup /= 2; // Count each edge once
|
||||
modularity += inGroup / m - Math.pow(degreeSum / (2 * m), 2);
|
||||
}
|
||||
|
||||
return modularity;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find bridges (edges whose removal disconnects components)
|
||||
*/
|
||||
export function findBridges(graph: Graph): Array<{ from: string; to: string }> {
|
||||
const bridges: Array<{ from: string; to: string }> = [];
|
||||
const visited = new Set<string>();
|
||||
const discovery = new Map<string, number>();
|
||||
const low = new Map<string, number>();
|
||||
const parent = new Map<string, string | null>();
|
||||
let time = 0;
|
||||
|
||||
function dfs(node: string) {
|
||||
visited.add(node);
|
||||
discovery.set(node, time);
|
||||
low.set(node, time);
|
||||
time++;
|
||||
|
||||
for (const [neighbor] of graph.adjacency.get(node) || []) {
|
||||
if (!visited.has(neighbor)) {
|
||||
parent.set(neighbor, node);
|
||||
dfs(neighbor);
|
||||
|
||||
low.set(node, Math.min(low.get(node)!, low.get(neighbor)!));
|
||||
|
||||
if (low.get(neighbor)! > discovery.get(node)!) {
|
||||
bridges.push({ from: node, to: neighbor });
|
||||
}
|
||||
} else if (neighbor !== parent.get(node)) {
|
||||
low.set(node, Math.min(low.get(node)!, discovery.get(neighbor)!));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const node of graph.nodes) {
|
||||
if (!visited.has(node)) {
|
||||
parent.set(node, null);
|
||||
dfs(node);
|
||||
}
|
||||
}
|
||||
|
||||
return bridges;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find articulation points (nodes whose removal disconnects components)
|
||||
*/
|
||||
export function findArticulationPoints(graph: Graph): string[] {
|
||||
const points: string[] = [];
|
||||
const visited = new Set<string>();
|
||||
const discovery = new Map<string, number>();
|
||||
const low = new Map<string, number>();
|
||||
const parent = new Map<string, string | null>();
|
||||
let time = 0;
|
||||
|
||||
function dfs(node: string) {
|
||||
visited.add(node);
|
||||
discovery.set(node, time);
|
||||
low.set(node, time);
|
||||
time++;
|
||||
|
||||
let children = 0;
|
||||
|
||||
for (const [neighbor] of graph.adjacency.get(node) || []) {
|
||||
if (!visited.has(neighbor)) {
|
||||
children++;
|
||||
parent.set(neighbor, node);
|
||||
dfs(neighbor);
|
||||
|
||||
low.set(node, Math.min(low.get(node)!, low.get(neighbor)!));
|
||||
|
||||
// Root with 2+ children or non-root with low[v] >= disc[u]
|
||||
if (
|
||||
(parent.get(node) === null && children > 1) ||
|
||||
(parent.get(node) !== null && low.get(neighbor)! >= discovery.get(node)!)
|
||||
) {
|
||||
if (!points.includes(node)) {
|
||||
points.push(node);
|
||||
}
|
||||
}
|
||||
} else if (neighbor !== parent.get(node)) {
|
||||
low.set(node, Math.min(low.get(node)!, discovery.get(neighbor)!));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const node of graph.nodes) {
|
||||
if (!visited.has(node)) {
|
||||
parent.set(node, null);
|
||||
dfs(node);
|
||||
}
|
||||
}
|
||||
|
||||
return points;
|
||||
}
|
||||
|
||||
// Helper functions
|
||||
function normalize(v: Float64Array) {
|
||||
let sum = 0;
|
||||
for (let i = 0; i < v.length; i++) {
|
||||
sum += v[i] * v[i];
|
||||
}
|
||||
const norm = Math.sqrt(sum);
|
||||
if (norm > 0) {
|
||||
for (let i = 0; i < v.length; i++) {
|
||||
v[i] /= norm;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function dotProduct(a: Float64Array, b: Float64Array): number {
|
||||
let sum = 0;
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
sum += a[i] * b[i];
|
||||
}
|
||||
return sum;
|
||||
}
|
||||
|
||||
function kMeans(points: number[][], k: number, maxIter: number = 100): number[] {
|
||||
const n = points.length;
|
||||
if (n === 0 || k === 0) return [];
|
||||
|
||||
const dim = points[0].length;
|
||||
|
||||
// Random initialization
|
||||
const centroids: number[][] = [];
|
||||
const used = new Set<number>();
|
||||
while (centroids.length < Math.min(k, n)) {
|
||||
const idx = Math.floor(Math.random() * n);
|
||||
if (!used.has(idx)) {
|
||||
used.add(idx);
|
||||
centroids.push([...points[idx]]);
|
||||
}
|
||||
}
|
||||
|
||||
const assignment = new Array(n).fill(0);
|
||||
|
||||
for (let iter = 0; iter < maxIter; iter++) {
|
||||
// Assign points to nearest centroid
|
||||
let changed = false;
|
||||
for (let i = 0; i < n; i++) {
|
||||
let minDist = Infinity;
|
||||
let minC = 0;
|
||||
for (let c = 0; c < centroids.length; c++) {
|
||||
let dist = 0;
|
||||
for (let d = 0; d < dim; d++) {
|
||||
dist += Math.pow(points[i][d] - centroids[c][d], 2);
|
||||
}
|
||||
if (dist < minDist) {
|
||||
minDist = dist;
|
||||
minC = c;
|
||||
}
|
||||
}
|
||||
if (assignment[i] !== minC) {
|
||||
assignment[i] = minC;
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (!changed) break;
|
||||
|
||||
// Update centroids
|
||||
const counts = new Array(k).fill(0);
|
||||
for (let c = 0; c < centroids.length; c++) {
|
||||
for (let d = 0; d < dim; d++) {
|
||||
centroids[c][d] = 0;
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < n; i++) {
|
||||
const c = assignment[i];
|
||||
counts[c]++;
|
||||
for (let d = 0; d < dim; d++) {
|
||||
centroids[c][d] += points[i][d];
|
||||
}
|
||||
}
|
||||
|
||||
for (let c = 0; c < centroids.length; c++) {
|
||||
if (counts[c] > 0) {
|
||||
for (let d = 0; d < dim; d++) {
|
||||
centroids[c][d] /= counts[c];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return assignment;
|
||||
}
|
||||
|
||||
export default {
|
||||
buildGraph,
|
||||
minCut,
|
||||
spectralClustering,
|
||||
louvainCommunities,
|
||||
calculateModularity,
|
||||
findBridges,
|
||||
findArticulationPoints,
|
||||
};
|
||||
147
npm/packages/ruvector/src/core/graph-wrapper.d.ts
vendored
Normal file
147
npm/packages/ruvector/src/core/graph-wrapper.d.ts
vendored
Normal file
@@ -0,0 +1,147 @@
|
||||
/**
|
||||
* Graph Wrapper - Hypergraph database for code relationships
|
||||
*
|
||||
* Wraps @ruvector/graph-node for dependency analysis, co-edit patterns,
|
||||
* and code structure understanding.
|
||||
*/
|
||||
export declare function isGraphAvailable(): boolean;
|
||||
export interface Node {
|
||||
id: string;
|
||||
labels: string[];
|
||||
properties: Record<string, any>;
|
||||
}
|
||||
export interface Edge {
|
||||
id?: string;
|
||||
from: string;
|
||||
to: string;
|
||||
type: string;
|
||||
properties?: Record<string, any>;
|
||||
}
|
||||
export interface Hyperedge {
|
||||
id?: string;
|
||||
nodes: string[];
|
||||
type: string;
|
||||
properties?: Record<string, any>;
|
||||
}
|
||||
export interface CypherResult {
|
||||
columns: string[];
|
||||
rows: any[][];
|
||||
}
|
||||
export interface PathResult {
|
||||
nodes: Node[];
|
||||
edges: Edge[];
|
||||
length: number;
|
||||
}
|
||||
/**
|
||||
* Graph Database for code relationships
|
||||
*/
|
||||
export declare class CodeGraph {
|
||||
private inner;
|
||||
private storagePath?;
|
||||
constructor(options?: {
|
||||
storagePath?: string;
|
||||
inMemory?: boolean;
|
||||
});
|
||||
/**
|
||||
* Create a node (file, function, class, etc.)
|
||||
*/
|
||||
createNode(id: string, labels: string[], properties?: Record<string, any>): Node;
|
||||
/**
|
||||
* Get a node by ID
|
||||
*/
|
||||
getNode(id: string): Node | null;
|
||||
/**
|
||||
* Update node properties
|
||||
*/
|
||||
updateNode(id: string, properties: Record<string, any>): boolean;
|
||||
/**
|
||||
* Delete a node
|
||||
*/
|
||||
deleteNode(id: string): boolean;
|
||||
/**
|
||||
* Find nodes by label
|
||||
*/
|
||||
findNodesByLabel(label: string): Node[];
|
||||
/**
|
||||
* Create an edge (import, call, reference, etc.)
|
||||
*/
|
||||
createEdge(from: string, to: string, type: string, properties?: Record<string, any>): Edge;
|
||||
/**
|
||||
* Get edges from a node
|
||||
*/
|
||||
getOutgoingEdges(nodeId: string, type?: string): Edge[];
|
||||
/**
|
||||
* Get edges to a node
|
||||
*/
|
||||
getIncomingEdges(nodeId: string, type?: string): Edge[];
|
||||
/**
|
||||
* Delete an edge
|
||||
*/
|
||||
deleteEdge(edgeId: string): boolean;
|
||||
/**
|
||||
* Create a hyperedge connecting multiple nodes
|
||||
*/
|
||||
createHyperedge(nodes: string[], type: string, properties?: Record<string, any>): Hyperedge;
|
||||
/**
|
||||
* Get hyperedges containing a node
|
||||
*/
|
||||
getHyperedges(nodeId: string, type?: string): Hyperedge[];
|
||||
/**
|
||||
* Execute a Cypher query
|
||||
*/
|
||||
cypher(query: string, params?: Record<string, any>): CypherResult;
|
||||
/**
|
||||
* Find shortest path between nodes
|
||||
*/
|
||||
shortestPath(from: string, to: string, maxDepth?: number): PathResult | null;
|
||||
/**
|
||||
* Get all paths between nodes (up to maxPaths)
|
||||
*/
|
||||
allPaths(from: string, to: string, maxDepth?: number, maxPaths?: number): PathResult[];
|
||||
/**
|
||||
* Get neighbors of a node
|
||||
*/
|
||||
neighbors(nodeId: string, depth?: number): Node[];
|
||||
/**
|
||||
* Calculate PageRank for nodes
|
||||
*/
|
||||
pageRank(iterations?: number, dampingFactor?: number): Map<string, number>;
|
||||
/**
|
||||
* Find connected components
|
||||
*/
|
||||
connectedComponents(): string[][];
|
||||
/**
|
||||
* Detect communities (Louvain algorithm)
|
||||
*/
|
||||
communities(): Map<string, number>;
|
||||
/**
|
||||
* Calculate betweenness centrality
|
||||
*/
|
||||
betweennessCentrality(): Map<string, number>;
|
||||
/**
|
||||
* Save graph to storage
|
||||
*/
|
||||
save(): void;
|
||||
/**
|
||||
* Load graph from storage
|
||||
*/
|
||||
load(): void;
|
||||
/**
|
||||
* Clear all data
|
||||
*/
|
||||
clear(): void;
|
||||
/**
|
||||
* Get graph statistics
|
||||
*/
|
||||
stats(): {
|
||||
nodes: number;
|
||||
edges: number;
|
||||
hyperedges: number;
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Create a code dependency graph from file analysis
|
||||
*/
|
||||
export declare function createCodeDependencyGraph(storagePath?: string): CodeGraph;
|
||||
export default CodeGraph;
|
||||
//# sourceMappingURL=graph-wrapper.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/graph-wrapper.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/graph-wrapper.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"graph-wrapper.d.ts","sourceRoot":"","sources":["graph-wrapper.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAqBH,wBAAgB,gBAAgB,IAAI,OAAO,CAO1C;AAED,MAAM,WAAW,IAAI;IACnB,EAAE,EAAE,MAAM,CAAC;IACX,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC;CACjC;AAED,MAAM,WAAW,IAAI;IACnB,EAAE,CAAC,EAAE,MAAM,CAAC;IACZ,IAAI,EAAE,MAAM,CAAC;IACb,EAAE,EAAE,MAAM,CAAC;IACX,IAAI,EAAE,MAAM,CAAC;IACb,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC;CAClC;AAED,MAAM,WAAW,SAAS;IACxB,EAAE,CAAC,EAAE,MAAM,CAAC;IACZ,KAAK,EAAE,MAAM,EAAE,CAAC;IAChB,IAAI,EAAE,MAAM,CAAC;IACb,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC;CAClC;AAED,MAAM,WAAW,YAAY;IAC3B,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,IAAI,EAAE,GAAG,EAAE,EAAE,CAAC;CACf;AAED,MAAM,WAAW,UAAU;IACzB,KAAK,EAAE,IAAI,EAAE,CAAC;IACd,KAAK,EAAE,IAAI,EAAE,CAAC;IACd,MAAM,EAAE,MAAM,CAAC;CAChB;AAED;;GAEG;AACH,qBAAa,SAAS;IACpB,OAAO,CAAC,KAAK,CAAM;IACnB,OAAO,CAAC,WAAW,CAAC,CAAS;gBAEjB,OAAO,GAAE;QAAE,WAAW,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,EAAE,OAAO,CAAA;KAAO;IAatE;;OAEG;IACH,UAAU,CAAC,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,UAAU,GAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAM,GAAG,IAAI;IAKpF;;OAEG;IACH,OAAO,CAAC,EAAE,EAAE,MAAM,GAAG,IAAI,GAAG,IAAI;IAUhC;;OAEG;IACH,UAAU,CAAC,EAAE,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,GAAG,OAAO;IAIhE;;OAEG;IACH,UAAU,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO;IAI/B;;OAEG;IACH,gBAAgB,CAAC,KAAK,EAAE,MAAM,GAAG,IAAI,EAAE;IAavC;;OAEG;IACH,UAAU,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,GAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAM,GAAG,IAAI;IAK9F;;OAEG;IACH,gBAAgB,CAAC,MAAM,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,MAAM,GAAG,IAAI,EAAE;IAWvD;;OAEG;IACH,gBAAgB,CAAC,MAAM,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,MAAM,GAAG,IAAI,EAAE;IAWvD;;OAEG;IACH,UAAU,CAAC,MAAM,EAAE,MAAM,GAAG,OAAO;IAQnC;;OAEG;IACH,eAAe,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,MAAM,EAAE,UAAU,GAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAM,GAAG,SAAS;IAK/F;;OAEG;IACH,aAAa,CAAC,MAAM,EAAE,MAAM,EAAE,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,EAAE;IAczD;;OAEG;IACH,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,GAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAM,GAAG,YAAY;IAQrE;;OAEG;IACH,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,EAAE,QAAQ,GAAE,MAAW,GAAG,UAAU,GAAG,IAAI;IAoBhF;;OAEG;IACH,QAAQ,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,EAAE,QAAQ,GAAE,MAAU,EAAE,QAAQ,GAAE,MAAW,GAAG,UAAU,EAAE;IAmB7F;;OAEG;IACH,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,KAAK,GAAE,MAAU,GAAG,IAAI,EAAE;IAapD;;OAEG;IACH,QAAQ,CAAC,UAAU,GAAE,MAAW,EAAE,aAAa,GAAE,MAAa,GAAG,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC;IAKpF;;OAEG;IACH,mBAAmB,IAAI,MAAM,EAAE,EAAE;IAIjC;;OAEG;IACH,WAAW,IAAI,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC;IAKlC;;OAEG;IACH,qBAAqB,IAAI,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC;IAS5C;;OAEG;IACH,IAAI,IAAI,IAAI;IAOZ;;OAEG;IACH,IAAI,IAAI,IAAI;IAOZ;;OAEG;IACH,KAAK,IAAI,IAAI;IAIb;;OAEG;IACH,KAAK,IAAI;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAC;QAAC,UAAU,EAAE,MAAM,CAAA;KAAE;CAG9D;AAED;;GAEG;AACH,wBAAgB,yBAAyB,CAAC,WAAW,CAAC,EAAE,MAAM,GAAG,SAAS,CAEzE;AAED,eAAe,SAAS,CAAC"}
|
||||
300
npm/packages/ruvector/src/core/graph-wrapper.js
Normal file
300
npm/packages/ruvector/src/core/graph-wrapper.js
Normal file
@@ -0,0 +1,300 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Graph Wrapper - Hypergraph database for code relationships
|
||||
*
|
||||
* Wraps @ruvector/graph-node for dependency analysis, co-edit patterns,
|
||||
* and code structure understanding.
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.CodeGraph = void 0;
|
||||
exports.isGraphAvailable = isGraphAvailable;
|
||||
exports.createCodeDependencyGraph = createCodeDependencyGraph;
|
||||
let graphModule = null;
|
||||
let loadError = null;
|
||||
function getGraphModule() {
|
||||
if (graphModule)
|
||||
return graphModule;
|
||||
if (loadError)
|
||||
throw loadError;
|
||||
try {
|
||||
graphModule = require('@ruvector/graph-node');
|
||||
return graphModule;
|
||||
}
|
||||
catch (e) {
|
||||
loadError = new Error(`@ruvector/graph-node not installed: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/graph-node`);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
function isGraphAvailable() {
|
||||
try {
|
||||
getGraphModule();
|
||||
return true;
|
||||
}
|
||||
catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Graph Database for code relationships
|
||||
*/
|
||||
class CodeGraph {
|
||||
constructor(options = {}) {
|
||||
const graph = getGraphModule();
|
||||
this.storagePath = options.storagePath;
|
||||
this.inner = new graph.GraphDatabase({
|
||||
storagePath: options.storagePath,
|
||||
inMemory: options.inMemory ?? true,
|
||||
});
|
||||
}
|
||||
// ===========================================================================
|
||||
// Node Operations
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Create a node (file, function, class, etc.)
|
||||
*/
|
||||
createNode(id, labels, properties = {}) {
|
||||
this.inner.createNode(id, labels, JSON.stringify(properties));
|
||||
return { id, labels, properties };
|
||||
}
|
||||
/**
|
||||
* Get a node by ID
|
||||
*/
|
||||
getNode(id) {
|
||||
const result = this.inner.getNode(id);
|
||||
if (!result)
|
||||
return null;
|
||||
return {
|
||||
id: result.id,
|
||||
labels: result.labels,
|
||||
properties: result.properties ? JSON.parse(result.properties) : {},
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Update node properties
|
||||
*/
|
||||
updateNode(id, properties) {
|
||||
return this.inner.updateNode(id, JSON.stringify(properties));
|
||||
}
|
||||
/**
|
||||
* Delete a node
|
||||
*/
|
||||
deleteNode(id) {
|
||||
return this.inner.deleteNode(id);
|
||||
}
|
||||
/**
|
||||
* Find nodes by label
|
||||
*/
|
||||
findNodesByLabel(label) {
|
||||
const results = this.inner.findNodesByLabel(label);
|
||||
return results.map((r) => ({
|
||||
id: r.id,
|
||||
labels: r.labels,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
// ===========================================================================
|
||||
// Edge Operations
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Create an edge (import, call, reference, etc.)
|
||||
*/
|
||||
createEdge(from, to, type, properties = {}) {
|
||||
const id = this.inner.createEdge(from, to, type, JSON.stringify(properties));
|
||||
return { id, from, to, type, properties };
|
||||
}
|
||||
/**
|
||||
* Get edges from a node
|
||||
*/
|
||||
getOutgoingEdges(nodeId, type) {
|
||||
const results = this.inner.getOutgoingEdges(nodeId, type);
|
||||
return results.map((r) => ({
|
||||
id: r.id,
|
||||
from: r.from,
|
||||
to: r.to,
|
||||
type: r.type,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
/**
|
||||
* Get edges to a node
|
||||
*/
|
||||
getIncomingEdges(nodeId, type) {
|
||||
const results = this.inner.getIncomingEdges(nodeId, type);
|
||||
return results.map((r) => ({
|
||||
id: r.id,
|
||||
from: r.from,
|
||||
to: r.to,
|
||||
type: r.type,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
/**
|
||||
* Delete an edge
|
||||
*/
|
||||
deleteEdge(edgeId) {
|
||||
return this.inner.deleteEdge(edgeId);
|
||||
}
|
||||
// ===========================================================================
|
||||
// Hyperedge Operations (for co-edit patterns)
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Create a hyperedge connecting multiple nodes
|
||||
*/
|
||||
createHyperedge(nodes, type, properties = {}) {
|
||||
const id = this.inner.createHyperedge(nodes, type, JSON.stringify(properties));
|
||||
return { id, nodes, type, properties };
|
||||
}
|
||||
/**
|
||||
* Get hyperedges containing a node
|
||||
*/
|
||||
getHyperedges(nodeId, type) {
|
||||
const results = this.inner.getHyperedges(nodeId, type);
|
||||
return results.map((r) => ({
|
||||
id: r.id,
|
||||
nodes: r.nodes,
|
||||
type: r.type,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
// ===========================================================================
|
||||
// Query Operations
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Execute a Cypher query
|
||||
*/
|
||||
cypher(query, params = {}) {
|
||||
const result = this.inner.cypher(query, JSON.stringify(params));
|
||||
return {
|
||||
columns: result.columns,
|
||||
rows: result.rows,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Find shortest path between nodes
|
||||
*/
|
||||
shortestPath(from, to, maxDepth = 10) {
|
||||
const result = this.inner.shortestPath(from, to, maxDepth);
|
||||
if (!result)
|
||||
return null;
|
||||
return {
|
||||
nodes: result.nodes.map((n) => ({
|
||||
id: n.id,
|
||||
labels: n.labels,
|
||||
properties: n.properties ? JSON.parse(n.properties) : {},
|
||||
})),
|
||||
edges: result.edges.map((e) => ({
|
||||
id: e.id,
|
||||
from: e.from,
|
||||
to: e.to,
|
||||
type: e.type,
|
||||
properties: e.properties ? JSON.parse(e.properties) : {},
|
||||
})),
|
||||
length: result.length,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Get all paths between nodes (up to maxPaths)
|
||||
*/
|
||||
allPaths(from, to, maxDepth = 5, maxPaths = 10) {
|
||||
const results = this.inner.allPaths(from, to, maxDepth, maxPaths);
|
||||
return results.map((r) => ({
|
||||
nodes: r.nodes.map((n) => ({
|
||||
id: n.id,
|
||||
labels: n.labels,
|
||||
properties: n.properties ? JSON.parse(n.properties) : {},
|
||||
})),
|
||||
edges: r.edges.map((e) => ({
|
||||
id: e.id,
|
||||
from: e.from,
|
||||
to: e.to,
|
||||
type: e.type,
|
||||
properties: e.properties ? JSON.parse(e.properties) : {},
|
||||
})),
|
||||
length: r.length,
|
||||
}));
|
||||
}
|
||||
/**
|
||||
* Get neighbors of a node
|
||||
*/
|
||||
neighbors(nodeId, depth = 1) {
|
||||
const results = this.inner.neighbors(nodeId, depth);
|
||||
return results.map((n) => ({
|
||||
id: n.id,
|
||||
labels: n.labels,
|
||||
properties: n.properties ? JSON.parse(n.properties) : {},
|
||||
}));
|
||||
}
|
||||
// ===========================================================================
|
||||
// Graph Algorithms
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Calculate PageRank for nodes
|
||||
*/
|
||||
pageRank(iterations = 20, dampingFactor = 0.85) {
|
||||
const result = this.inner.pageRank(iterations, dampingFactor);
|
||||
return new Map(Object.entries(result));
|
||||
}
|
||||
/**
|
||||
* Find connected components
|
||||
*/
|
||||
connectedComponents() {
|
||||
return this.inner.connectedComponents();
|
||||
}
|
||||
/**
|
||||
* Detect communities (Louvain algorithm)
|
||||
*/
|
||||
communities() {
|
||||
const result = this.inner.communities();
|
||||
return new Map(Object.entries(result));
|
||||
}
|
||||
/**
|
||||
* Calculate betweenness centrality
|
||||
*/
|
||||
betweennessCentrality() {
|
||||
const result = this.inner.betweennessCentrality();
|
||||
return new Map(Object.entries(result));
|
||||
}
|
||||
// ===========================================================================
|
||||
// Persistence
|
||||
// ===========================================================================
|
||||
/**
|
||||
* Save graph to storage
|
||||
*/
|
||||
save() {
|
||||
if (!this.storagePath) {
|
||||
throw new Error('No storage path configured');
|
||||
}
|
||||
this.inner.save();
|
||||
}
|
||||
/**
|
||||
* Load graph from storage
|
||||
*/
|
||||
load() {
|
||||
if (!this.storagePath) {
|
||||
throw new Error('No storage path configured');
|
||||
}
|
||||
this.inner.load();
|
||||
}
|
||||
/**
|
||||
* Clear all data
|
||||
*/
|
||||
clear() {
|
||||
this.inner.clear();
|
||||
}
|
||||
/**
|
||||
* Get graph statistics
|
||||
*/
|
||||
stats() {
|
||||
return this.inner.stats();
|
||||
}
|
||||
}
|
||||
exports.CodeGraph = CodeGraph;
|
||||
/**
|
||||
* Create a code dependency graph from file analysis
|
||||
*/
|
||||
function createCodeDependencyGraph(storagePath) {
|
||||
return new CodeGraph({ storagePath, inMemory: !storagePath });
|
||||
}
|
||||
exports.default = CodeGraph;
|
||||
//# sourceMappingURL=graph-wrapper.js.map
|
||||
1
npm/packages/ruvector/src/core/graph-wrapper.js.map
Normal file
1
npm/packages/ruvector/src/core/graph-wrapper.js.map
Normal file
File diff suppressed because one or more lines are too long
360
npm/packages/ruvector/src/core/graph-wrapper.ts
Normal file
360
npm/packages/ruvector/src/core/graph-wrapper.ts
Normal file
@@ -0,0 +1,360 @@
|
||||
/**
|
||||
* Graph Wrapper - Hypergraph database for code relationships
|
||||
*
|
||||
* Wraps @ruvector/graph-node for dependency analysis, co-edit patterns,
|
||||
* and code structure understanding.
|
||||
*/
|
||||
|
||||
let graphModule: any = null;
|
||||
let loadError: Error | null = null;
|
||||
|
||||
function getGraphModule() {
|
||||
if (graphModule) return graphModule;
|
||||
if (loadError) throw loadError;
|
||||
|
||||
try {
|
||||
graphModule = require('@ruvector/graph-node');
|
||||
return graphModule;
|
||||
} catch (e: any) {
|
||||
loadError = new Error(
|
||||
`@ruvector/graph-node not installed: ${e.message}\n` +
|
||||
`Install with: npm install @ruvector/graph-node`
|
||||
);
|
||||
throw loadError;
|
||||
}
|
||||
}
|
||||
|
||||
export function isGraphAvailable(): boolean {
|
||||
try {
|
||||
getGraphModule();
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export interface Node {
|
||||
id: string;
|
||||
labels: string[];
|
||||
properties: Record<string, any>;
|
||||
}
|
||||
|
||||
export interface Edge {
|
||||
id?: string;
|
||||
from: string;
|
||||
to: string;
|
||||
type: string;
|
||||
properties?: Record<string, any>;
|
||||
}
|
||||
|
||||
export interface Hyperedge {
|
||||
id?: string;
|
||||
nodes: string[];
|
||||
type: string;
|
||||
properties?: Record<string, any>;
|
||||
}
|
||||
|
||||
export interface CypherResult {
|
||||
columns: string[];
|
||||
rows: any[][];
|
||||
}
|
||||
|
||||
export interface PathResult {
|
||||
nodes: Node[];
|
||||
edges: Edge[];
|
||||
length: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Graph Database for code relationships
|
||||
*/
|
||||
export class CodeGraph {
|
||||
private inner: any;
|
||||
private storagePath?: string;
|
||||
|
||||
constructor(options: { storagePath?: string; inMemory?: boolean } = {}) {
|
||||
const graph = getGraphModule();
|
||||
this.storagePath = options.storagePath;
|
||||
this.inner = new graph.GraphDatabase({
|
||||
storagePath: options.storagePath,
|
||||
inMemory: options.inMemory ?? true,
|
||||
});
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Node Operations
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Create a node (file, function, class, etc.)
|
||||
*/
|
||||
createNode(id: string, labels: string[], properties: Record<string, any> = {}): Node {
|
||||
this.inner.createNode(id, labels, JSON.stringify(properties));
|
||||
return { id, labels, properties };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a node by ID
|
||||
*/
|
||||
getNode(id: string): Node | null {
|
||||
const result = this.inner.getNode(id);
|
||||
if (!result) return null;
|
||||
return {
|
||||
id: result.id,
|
||||
labels: result.labels,
|
||||
properties: result.properties ? JSON.parse(result.properties) : {},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Update node properties
|
||||
*/
|
||||
updateNode(id: string, properties: Record<string, any>): boolean {
|
||||
return this.inner.updateNode(id, JSON.stringify(properties));
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a node
|
||||
*/
|
||||
deleteNode(id: string): boolean {
|
||||
return this.inner.deleteNode(id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Find nodes by label
|
||||
*/
|
||||
findNodesByLabel(label: string): Node[] {
|
||||
const results = this.inner.findNodesByLabel(label);
|
||||
return results.map((r: any) => ({
|
||||
id: r.id,
|
||||
labels: r.labels,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Edge Operations
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Create an edge (import, call, reference, etc.)
|
||||
*/
|
||||
createEdge(from: string, to: string, type: string, properties: Record<string, any> = {}): Edge {
|
||||
const id = this.inner.createEdge(from, to, type, JSON.stringify(properties));
|
||||
return { id, from, to, type, properties };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get edges from a node
|
||||
*/
|
||||
getOutgoingEdges(nodeId: string, type?: string): Edge[] {
|
||||
const results = this.inner.getOutgoingEdges(nodeId, type);
|
||||
return results.map((r: any) => ({
|
||||
id: r.id,
|
||||
from: r.from,
|
||||
to: r.to,
|
||||
type: r.type,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Get edges to a node
|
||||
*/
|
||||
getIncomingEdges(nodeId: string, type?: string): Edge[] {
|
||||
const results = this.inner.getIncomingEdges(nodeId, type);
|
||||
return results.map((r: any) => ({
|
||||
id: r.id,
|
||||
from: r.from,
|
||||
to: r.to,
|
||||
type: r.type,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete an edge
|
||||
*/
|
||||
deleteEdge(edgeId: string): boolean {
|
||||
return this.inner.deleteEdge(edgeId);
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Hyperedge Operations (for co-edit patterns)
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Create a hyperedge connecting multiple nodes
|
||||
*/
|
||||
createHyperedge(nodes: string[], type: string, properties: Record<string, any> = {}): Hyperedge {
|
||||
const id = this.inner.createHyperedge(nodes, type, JSON.stringify(properties));
|
||||
return { id, nodes, type, properties };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get hyperedges containing a node
|
||||
*/
|
||||
getHyperedges(nodeId: string, type?: string): Hyperedge[] {
|
||||
const results = this.inner.getHyperedges(nodeId, type);
|
||||
return results.map((r: any) => ({
|
||||
id: r.id,
|
||||
nodes: r.nodes,
|
||||
type: r.type,
|
||||
properties: r.properties ? JSON.parse(r.properties) : {},
|
||||
}));
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Query Operations
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Execute a Cypher query
|
||||
*/
|
||||
cypher(query: string, params: Record<string, any> = {}): CypherResult {
|
||||
const result = this.inner.cypher(query, JSON.stringify(params));
|
||||
return {
|
||||
columns: result.columns,
|
||||
rows: result.rows,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find shortest path between nodes
|
||||
*/
|
||||
shortestPath(from: string, to: string, maxDepth: number = 10): PathResult | null {
|
||||
const result = this.inner.shortestPath(from, to, maxDepth);
|
||||
if (!result) return null;
|
||||
return {
|
||||
nodes: result.nodes.map((n: any) => ({
|
||||
id: n.id,
|
||||
labels: n.labels,
|
||||
properties: n.properties ? JSON.parse(n.properties) : {},
|
||||
})),
|
||||
edges: result.edges.map((e: any) => ({
|
||||
id: e.id,
|
||||
from: e.from,
|
||||
to: e.to,
|
||||
type: e.type,
|
||||
properties: e.properties ? JSON.parse(e.properties) : {},
|
||||
})),
|
||||
length: result.length,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all paths between nodes (up to maxPaths)
|
||||
*/
|
||||
allPaths(from: string, to: string, maxDepth: number = 5, maxPaths: number = 10): PathResult[] {
|
||||
const results = this.inner.allPaths(from, to, maxDepth, maxPaths);
|
||||
return results.map((r: any) => ({
|
||||
nodes: r.nodes.map((n: any) => ({
|
||||
id: n.id,
|
||||
labels: n.labels,
|
||||
properties: n.properties ? JSON.parse(n.properties) : {},
|
||||
})),
|
||||
edges: r.edges.map((e: any) => ({
|
||||
id: e.id,
|
||||
from: e.from,
|
||||
to: e.to,
|
||||
type: e.type,
|
||||
properties: e.properties ? JSON.parse(e.properties) : {},
|
||||
})),
|
||||
length: r.length,
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Get neighbors of a node
|
||||
*/
|
||||
neighbors(nodeId: string, depth: number = 1): Node[] {
|
||||
const results = this.inner.neighbors(nodeId, depth);
|
||||
return results.map((n: any) => ({
|
||||
id: n.id,
|
||||
labels: n.labels,
|
||||
properties: n.properties ? JSON.parse(n.properties) : {},
|
||||
}));
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Graph Algorithms
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Calculate PageRank for nodes
|
||||
*/
|
||||
pageRank(iterations: number = 20, dampingFactor: number = 0.85): Map<string, number> {
|
||||
const result = this.inner.pageRank(iterations, dampingFactor);
|
||||
return new Map(Object.entries(result));
|
||||
}
|
||||
|
||||
/**
|
||||
* Find connected components
|
||||
*/
|
||||
connectedComponents(): string[][] {
|
||||
return this.inner.connectedComponents();
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect communities (Louvain algorithm)
|
||||
*/
|
||||
communities(): Map<string, number> {
|
||||
const result = this.inner.communities();
|
||||
return new Map(Object.entries(result));
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate betweenness centrality
|
||||
*/
|
||||
betweennessCentrality(): Map<string, number> {
|
||||
const result = this.inner.betweennessCentrality();
|
||||
return new Map(Object.entries(result));
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Persistence
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Save graph to storage
|
||||
*/
|
||||
save(): void {
|
||||
if (!this.storagePath) {
|
||||
throw new Error('No storage path configured');
|
||||
}
|
||||
this.inner.save();
|
||||
}
|
||||
|
||||
/**
|
||||
* Load graph from storage
|
||||
*/
|
||||
load(): void {
|
||||
if (!this.storagePath) {
|
||||
throw new Error('No storage path configured');
|
||||
}
|
||||
this.inner.load();
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all data
|
||||
*/
|
||||
clear(): void {
|
||||
this.inner.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get graph statistics
|
||||
*/
|
||||
stats(): { nodes: number; edges: number; hyperedges: number } {
|
||||
return this.inner.stats();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a code dependency graph from file analysis
|
||||
*/
|
||||
export function createCodeDependencyGraph(storagePath?: string): CodeGraph {
|
||||
return new CodeGraph({ storagePath, inMemory: !storagePath });
|
||||
}
|
||||
|
||||
export default CodeGraph;
|
||||
1
npm/packages/ruvector/src/core/index.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/index.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,cAAc,eAAe,CAAC;AAC9B,cAAc,uBAAuB,CAAC;AACtC,cAAc,gBAAgB,CAAC;AAC/B,cAAc,gBAAgB,CAAC;AAC/B,cAAc,uBAAuB,CAAC;AACtC,cAAc,iBAAiB,CAAC;AAChC,cAAc,kBAAkB,CAAC;AACjC,cAAc,yBAAyB,CAAC;AACxC,cAAc,oBAAoB,CAAC;AACnC,cAAc,kBAAkB,CAAC;AACjC,cAAc,iBAAiB,CAAC;AAChC,cAAc,mBAAmB,CAAC;AAClC,cAAc,cAAc,CAAC;AAC7B,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,oBAAoB,CAAC;AACnC,cAAc,mBAAmB,CAAC;AAClC,cAAc,mBAAmB,CAAC;AAClC,cAAc,qBAAqB,CAAC;AACpC,cAAc,qBAAqB,CAAC;AACpC,cAAc,eAAe,CAAC;AAC9B,cAAc,eAAe,CAAC;AAG9B,cAAc,aAAa,CAAC;AAG5B,OAAO,EAAE,OAAO,IAAI,UAAU,EAAE,MAAM,eAAe,CAAC;AACtD,OAAO,EAAE,OAAO,IAAI,kBAAkB,EAAE,MAAM,uBAAuB,CAAC;AACtE,OAAO,EAAE,OAAO,IAAI,WAAW,EAAE,MAAM,gBAAgB,CAAC;AACxD,OAAO,EAAE,OAAO,IAAI,IAAI,EAAE,MAAM,gBAAgB,CAAC;AACjD,OAAO,EAAE,OAAO,IAAI,kBAAkB,EAAE,MAAM,uBAAuB,CAAC;AACtE,OAAO,EAAE,OAAO,IAAI,YAAY,EAAE,MAAM,iBAAiB,CAAC;AAC1D,OAAO,EAAE,OAAO,IAAI,qBAAqB,EAAE,MAAM,kBAAkB,CAAC;AACpE,OAAO,EAAE,OAAO,IAAI,oBAAoB,EAAE,MAAM,yBAAyB,CAAC;AAC1E,OAAO,EAAE,OAAO,IAAI,kBAAkB,EAAE,MAAM,oBAAoB,CAAC;AACnE,OAAO,EAAE,OAAO,IAAI,cAAc,EAAE,MAAM,kBAAkB,CAAC;AAC7D,OAAO,EAAE,OAAO,IAAI,SAAS,EAAE,MAAM,iBAAiB,CAAC;AACvD,OAAO,EAAE,OAAO,IAAI,eAAe,EAAE,MAAM,mBAAmB,CAAC;AAC/D,OAAO,EAAE,OAAO,IAAI,UAAU,EAAE,MAAM,cAAc,CAAC;AAGrD,OAAO,EAAE,UAAU,IAAI,SAAS,EAAE,MAAM,cAAc,CAAC;AAGvD,OAAO,EAAE,OAAO,IAAI,cAAc,EAAE,MAAM,mBAAmB,CAAC;AAC9D,OAAO,EAAE,OAAO,IAAI,cAAc,EAAE,MAAM,mBAAmB,CAAC;AAC9D,OAAO,EAAE,OAAO,IAAI,gBAAgB,EAAE,MAAM,qBAAqB,CAAC;AAClE,OAAO,EAAE,OAAO,IAAI,eAAe,EAAE,MAAM,qBAAqB,CAAC"}
|
||||
1
npm/packages/ruvector/src/core/index.js.map
Normal file
1
npm/packages/ruvector/src/core/index.js.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"index.js","sourceRoot":"","sources":["index.ts"],"names":[],"mappings":";AAAA;;;;;GAKG;;;;;;;;;;;;;;;;;;;;AAEH,gDAA8B;AAC9B,wDAAsC;AACtC,iDAA+B;AAC/B,iDAA+B;AAC/B,wDAAsC;AACtC,kDAAgC;AAChC,mDAAiC;AACjC,0DAAwC;AACxC,qDAAmC;AACnC,mDAAiC;AACjC,kDAAgC;AAChC,oDAAkC;AAClC,+CAA6B;AAC7B,oDAAkC;AAClC,oDAAkC;AAClC,qDAAmC;AACnC,oDAAkC;AAClC,oDAAkC;AAClC,sDAAoC;AACpC,sDAAoC;AACpC,gDAA8B;AAC9B,gDAA8B;AAE9B,gEAAgE;AAChE,8CAA4B;AAE5B,4CAA4C;AAC5C,6CAAsD;AAA7C,0HAAA,OAAO,OAAc;AAC9B,6DAAsE;AAA7D,0IAAA,OAAO,OAAsB;AACtC,+CAAwD;AAA/C,4HAAA,OAAO,OAAe;AAC/B,+CAAiD;AAAxC,qHAAA,OAAO,OAAQ;AACxB,6DAAsE;AAA7D,0IAAA,OAAO,OAAsB;AACtC,iDAA0D;AAAjD,8HAAA,OAAO,OAAgB;AAChC,mDAAoE;AAA3D,wIAAA,OAAO,OAAyB;AACzC,iEAA0E;AAAjE,8IAAA,OAAO,OAAwB;AACxC,uDAAmE;AAA1D,uIAAA,OAAO,OAAsB;AACtC,mDAA6D;AAApD,iIAAA,OAAO,OAAkB;AAClC,iDAAuD;AAA9C,2HAAA,OAAO,OAAa;AAC7B,qDAA+D;AAAtD,mIAAA,OAAO,OAAmB;AACnC,2CAAqD;AAA5C,yHAAA,OAAO,OAAc;AAE9B,mCAAmC;AACnC,2CAAuD;AAA9C,uGAAA,UAAU,OAAa;AAEhC,mBAAmB;AACnB,qDAA8D;AAArD,kIAAA,OAAO,OAAkB;AAClC,qDAA8D;AAArD,kIAAA,OAAO,OAAkB;AAClC,yDAAkE;AAAzD,sIAAA,OAAO,OAAoB;AACpC,yDAAiE;AAAxD,qIAAA,OAAO,OAAmB"}
|
||||
56
npm/packages/ruvector/src/core/index.ts
Normal file
56
npm/packages/ruvector/src/core/index.ts
Normal file
@@ -0,0 +1,56 @@
|
||||
/**
|
||||
* Core module exports
|
||||
*
|
||||
* These wrappers provide safe, type-flexible interfaces to the underlying
|
||||
* native packages, handling array type conversions automatically.
|
||||
*/
|
||||
|
||||
export * from './gnn-wrapper';
|
||||
export * from './attention-fallbacks';
|
||||
export * from './agentdb-fast';
|
||||
export * from './sona-wrapper';
|
||||
export * from './intelligence-engine';
|
||||
export * from './onnx-embedder';
|
||||
export * from './onnx-optimized';
|
||||
export * from './parallel-intelligence';
|
||||
export * from './parallel-workers';
|
||||
export * from './router-wrapper';
|
||||
export * from './graph-wrapper';
|
||||
export * from './cluster-wrapper';
|
||||
export * from './ast-parser';
|
||||
export * from './diff-embeddings';
|
||||
export * from './coverage-router';
|
||||
export * from './graph-algorithms';
|
||||
export * from './tensor-compress';
|
||||
export * from './learning-engine';
|
||||
export * from './adaptive-embedder';
|
||||
export * from './neural-embeddings';
|
||||
export * from './neural-perf';
|
||||
export * from './rvf-wrapper';
|
||||
|
||||
// Analysis module (consolidated security, complexity, patterns)
|
||||
export * from '../analysis';
|
||||
|
||||
// Re-export default objects for convenience
|
||||
export { default as gnnWrapper } from './gnn-wrapper';
|
||||
export { default as attentionFallbacks } from './attention-fallbacks';
|
||||
export { default as agentdbFast } from './agentdb-fast';
|
||||
export { default as Sona } from './sona-wrapper';
|
||||
export { default as IntelligenceEngine } from './intelligence-engine';
|
||||
export { default as OnnxEmbedder } from './onnx-embedder';
|
||||
export { default as OptimizedOnnxEmbedder } from './onnx-optimized';
|
||||
export { default as ParallelIntelligence } from './parallel-intelligence';
|
||||
export { default as ExtendedWorkerPool } from './parallel-workers';
|
||||
export { default as SemanticRouter } from './router-wrapper';
|
||||
export { default as CodeGraph } from './graph-wrapper';
|
||||
export { default as RuvectorCluster } from './cluster-wrapper';
|
||||
export { default as CodeParser } from './ast-parser';
|
||||
|
||||
// Alias for backward compatibility
|
||||
export { CodeParser as ASTParser } from './ast-parser';
|
||||
|
||||
// New v2.1 modules
|
||||
export { default as TensorCompress } from './tensor-compress';
|
||||
export { default as LearningEngine } from './learning-engine';
|
||||
export { default as AdaptiveEmbedder } from './adaptive-embedder';
|
||||
export { default as NeuralSubstrate } from './neural-embeddings';
|
||||
258
npm/packages/ruvector/src/core/intelligence-engine.d.ts
vendored
Normal file
258
npm/packages/ruvector/src/core/intelligence-engine.d.ts
vendored
Normal file
@@ -0,0 +1,258 @@
|
||||
/**
|
||||
* IntelligenceEngine - Full RuVector Intelligence Stack
|
||||
*
|
||||
* Integrates all RuVector capabilities for self-learning hooks:
|
||||
* - VectorDB with HNSW for semantic memory (150x faster)
|
||||
* - SONA for continual learning (Micro-LoRA, EWC++)
|
||||
* - FastAgentDB for episode/trajectory storage
|
||||
* - Attention mechanisms for pattern recognition
|
||||
* - ReasoningBank for pattern clustering
|
||||
*
|
||||
* Replaces the simple Q-learning approach with real ML-powered intelligence.
|
||||
*/
|
||||
import { EpisodeSearchResult } from './agentdb-fast';
|
||||
import { SonaConfig, LearnedPattern } from './sona-wrapper';
|
||||
import { ParallelConfig, BatchEpisode } from './parallel-intelligence';
|
||||
export interface MemoryEntry {
|
||||
id: string;
|
||||
content: string;
|
||||
type: string;
|
||||
embedding: number[];
|
||||
created: string;
|
||||
accessed: number;
|
||||
score?: number;
|
||||
}
|
||||
export interface AgentRoute {
|
||||
agent: string;
|
||||
confidence: number;
|
||||
reason: string;
|
||||
patterns?: LearnedPattern[];
|
||||
alternates?: Array<{
|
||||
agent: string;
|
||||
confidence: number;
|
||||
}>;
|
||||
}
|
||||
export interface LearningStats {
|
||||
totalMemories: number;
|
||||
memoryDimensions: number;
|
||||
totalEpisodes: number;
|
||||
totalTrajectories: number;
|
||||
avgReward: number;
|
||||
sonaEnabled: boolean;
|
||||
trajectoriesRecorded: number;
|
||||
patternsLearned: number;
|
||||
microLoraUpdates: number;
|
||||
baseLoraUpdates: number;
|
||||
ewcConsolidations: number;
|
||||
routingPatterns: number;
|
||||
errorPatterns: number;
|
||||
coEditPatterns: number;
|
||||
workerTriggers: number;
|
||||
attentionEnabled: boolean;
|
||||
onnxEnabled: boolean;
|
||||
parallelEnabled: boolean;
|
||||
parallelWorkers: number;
|
||||
parallelBusy: number;
|
||||
parallelQueued: number;
|
||||
}
|
||||
export interface IntelligenceConfig {
|
||||
/** Embedding dimension for vectors (default: 256, 384 for ONNX) */
|
||||
embeddingDim?: number;
|
||||
/** Maximum memories to store (default: 100000) */
|
||||
maxMemories?: number;
|
||||
/** Maximum episodes for trajectory storage (default: 50000) */
|
||||
maxEpisodes?: number;
|
||||
/** Enable SONA continual learning (default: true if available) */
|
||||
enableSona?: boolean;
|
||||
/** Enable attention mechanisms (default: true if available) */
|
||||
enableAttention?: boolean;
|
||||
/** Enable ONNX semantic embeddings (default: false, opt-in for quality) */
|
||||
enableOnnx?: boolean;
|
||||
/** SONA configuration */
|
||||
sonaConfig?: Partial<SonaConfig>;
|
||||
/** Storage path for persistence */
|
||||
storagePath?: string;
|
||||
/** Learning rate for pattern updates (default: 0.1) */
|
||||
learningRate?: number;
|
||||
/**
|
||||
* Enable parallel workers for batch operations
|
||||
* Auto-enabled for MCP servers, disabled for CLI hooks
|
||||
*/
|
||||
parallelConfig?: Partial<ParallelConfig>;
|
||||
}
|
||||
/**
|
||||
* Full-stack intelligence engine using all RuVector capabilities
|
||||
*/
|
||||
export declare class IntelligenceEngine {
|
||||
private config;
|
||||
private vectorDb;
|
||||
private agentDb;
|
||||
private sona;
|
||||
private attention;
|
||||
private onnxEmbedder;
|
||||
private onnxReady;
|
||||
private parallel;
|
||||
private memories;
|
||||
private routingPatterns;
|
||||
private errorPatterns;
|
||||
private coEditPatterns;
|
||||
private agentMappings;
|
||||
private workerTriggerMappings;
|
||||
private currentTrajectoryId;
|
||||
private sessionStart;
|
||||
private learningEnabled;
|
||||
private episodeBatchQueue;
|
||||
constructor(config?: IntelligenceConfig);
|
||||
private initOnnx;
|
||||
private initVectorDb;
|
||||
private initParallel;
|
||||
/**
|
||||
* Generate embedding using ONNX, attention, or hash (in order of preference)
|
||||
*/
|
||||
embed(text: string): number[];
|
||||
/**
|
||||
* Async embedding with ONNX support (recommended for semantic quality)
|
||||
*/
|
||||
embedAsync(text: string): Promise<number[]>;
|
||||
/**
|
||||
* Attention-based embedding using Flash or Multi-head attention
|
||||
*/
|
||||
private attentionEmbed;
|
||||
/**
|
||||
* Improved hash-based embedding with positional encoding
|
||||
*/
|
||||
private hashEmbed;
|
||||
private tokenize;
|
||||
private tokenEmbed;
|
||||
private meanPool;
|
||||
/**
|
||||
* Store content in vector memory (uses ONNX if available)
|
||||
*/
|
||||
remember(content: string, type?: string): Promise<MemoryEntry>;
|
||||
/**
|
||||
* Semantic search of memories (uses ONNX if available)
|
||||
*/
|
||||
recall(query: string, topK?: number): Promise<MemoryEntry[]>;
|
||||
private cosineSimilarity;
|
||||
/**
|
||||
* Route a task to the best agent using learned patterns
|
||||
*/
|
||||
route(task: string, file?: string): Promise<AgentRoute>;
|
||||
private getExtension;
|
||||
private getState;
|
||||
private getAlternates;
|
||||
/**
|
||||
* Begin recording a trajectory (before edit/command)
|
||||
*/
|
||||
beginTrajectory(context: string, file?: string): void;
|
||||
/**
|
||||
* Add a step to the current trajectory
|
||||
*/
|
||||
addTrajectoryStep(activations: number[], reward: number): void;
|
||||
/**
|
||||
* End the current trajectory with a quality score
|
||||
*/
|
||||
endTrajectory(success: boolean, quality?: number): void;
|
||||
/**
|
||||
* Set the agent route for current trajectory
|
||||
*/
|
||||
setTrajectoryRoute(agent: string): void;
|
||||
/**
|
||||
* Record an episode for learning
|
||||
*/
|
||||
recordEpisode(state: string, action: string, reward: number, nextState: string, done: boolean, metadata?: Record<string, any>): Promise<void>;
|
||||
/**
|
||||
* Queue episode for batch processing (3-4x faster with workers)
|
||||
*/
|
||||
queueEpisode(episode: BatchEpisode): void;
|
||||
/**
|
||||
* Process queued episodes in parallel batch
|
||||
*/
|
||||
flushEpisodeBatch(): Promise<number>;
|
||||
/**
|
||||
* Learn from similar past episodes
|
||||
*/
|
||||
learnFromSimilar(state: string, k?: number): Promise<EpisodeSearchResult[]>;
|
||||
/**
|
||||
* Register worker trigger to agent mappings
|
||||
*/
|
||||
registerWorkerTrigger(trigger: string, priority: string, agents: string[]): void;
|
||||
/**
|
||||
* Get agents for a worker trigger
|
||||
*/
|
||||
getAgentsForTrigger(trigger: string): {
|
||||
priority: string;
|
||||
agents: string[];
|
||||
} | undefined;
|
||||
/**
|
||||
* Route a task using worker trigger patterns first, then fall back to regular routing
|
||||
*/
|
||||
routeWithWorkers(task: string, file?: string): Promise<AgentRoute>;
|
||||
/**
|
||||
* Initialize default worker trigger mappings
|
||||
*/
|
||||
initDefaultWorkerMappings(): void;
|
||||
/**
|
||||
* Record a co-edit pattern
|
||||
*/
|
||||
recordCoEdit(file1: string, file2: string): void;
|
||||
/**
|
||||
* Get likely next files to edit
|
||||
*/
|
||||
getLikelyNextFiles(file: string, topK?: number): Array<{
|
||||
file: string;
|
||||
count: number;
|
||||
}>;
|
||||
/**
|
||||
* Record an error pattern with fixes
|
||||
*/
|
||||
recordErrorFix(errorPattern: string, fix: string): void;
|
||||
/**
|
||||
* Get suggested fixes for an error
|
||||
*/
|
||||
getSuggestedFixes(error: string): string[];
|
||||
/**
|
||||
* Run background learning cycle
|
||||
*/
|
||||
tick(): string | null;
|
||||
/**
|
||||
* Force immediate learning
|
||||
*/
|
||||
forceLearn(): string | null;
|
||||
/**
|
||||
* Get comprehensive learning statistics
|
||||
*/
|
||||
getStats(): LearningStats;
|
||||
/**
|
||||
* Export all data for persistence
|
||||
*/
|
||||
export(): Record<string, any>;
|
||||
/**
|
||||
* Import data from persistence
|
||||
*/
|
||||
import(data: Record<string, any>, merge?: boolean): void;
|
||||
/**
|
||||
* Clear all data
|
||||
*/
|
||||
clear(): void;
|
||||
/** Legacy: patterns object */
|
||||
get patterns(): Record<string, Record<string, number>>;
|
||||
/** Legacy: file_sequences array */
|
||||
get file_sequences(): string[][];
|
||||
/** Legacy: errors object */
|
||||
get errors(): Record<string, string[]>;
|
||||
}
|
||||
/**
|
||||
* Create a new IntelligenceEngine with default settings
|
||||
*/
|
||||
export declare function createIntelligenceEngine(config?: IntelligenceConfig): IntelligenceEngine;
|
||||
/**
|
||||
* Create a high-performance engine with all features enabled
|
||||
*/
|
||||
export declare function createHighPerformanceEngine(): IntelligenceEngine;
|
||||
/**
|
||||
* Create a lightweight engine for fast startup
|
||||
*/
|
||||
export declare function createLightweightEngine(): IntelligenceEngine;
|
||||
export default IntelligenceEngine;
|
||||
//# sourceMappingURL=intelligence-engine.d.ts.map
|
||||
File diff suppressed because one or more lines are too long
1030
npm/packages/ruvector/src/core/intelligence-engine.js
Normal file
1030
npm/packages/ruvector/src/core/intelligence-engine.js
Normal file
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
1233
npm/packages/ruvector/src/core/intelligence-engine.ts
Normal file
1233
npm/packages/ruvector/src/core/intelligence-engine.ts
Normal file
File diff suppressed because it is too large
Load Diff
160
npm/packages/ruvector/src/core/learning-engine.d.ts
vendored
Normal file
160
npm/packages/ruvector/src/core/learning-engine.d.ts
vendored
Normal file
@@ -0,0 +1,160 @@
|
||||
/**
|
||||
* Multi-Algorithm Learning Engine
|
||||
* Supports 9 RL algorithms for intelligent hooks optimization
|
||||
*/
|
||||
export type LearningAlgorithm = 'q-learning' | 'sarsa' | 'double-q' | 'actor-critic' | 'ppo' | 'decision-transformer' | 'monte-carlo' | 'td-lambda' | 'dqn';
|
||||
export type TaskType = 'agent-routing' | 'error-avoidance' | 'confidence-scoring' | 'trajectory-learning' | 'context-ranking' | 'memory-recall';
|
||||
export interface LearningConfig {
|
||||
algorithm: LearningAlgorithm;
|
||||
learningRate: number;
|
||||
discountFactor: number;
|
||||
epsilon: number;
|
||||
lambda?: number;
|
||||
clipRange?: number;
|
||||
entropyCoef?: number;
|
||||
sequenceLength?: number;
|
||||
}
|
||||
export interface Experience {
|
||||
state: string;
|
||||
action: string;
|
||||
reward: number;
|
||||
nextState: string;
|
||||
done: boolean;
|
||||
timestamp?: number;
|
||||
}
|
||||
export interface LearningTrajectory {
|
||||
experiences: Experience[];
|
||||
totalReward: number;
|
||||
completed: boolean;
|
||||
}
|
||||
export interface AlgorithmStats {
|
||||
algorithm: LearningAlgorithm;
|
||||
updates: number;
|
||||
avgReward: number;
|
||||
convergenceScore: number;
|
||||
lastUpdate: number;
|
||||
}
|
||||
export declare class LearningEngine {
|
||||
private configs;
|
||||
private qTables;
|
||||
private qTables2;
|
||||
private eligibilityTraces;
|
||||
private actorWeights;
|
||||
private criticValues;
|
||||
private trajectories;
|
||||
private stats;
|
||||
private rewardHistory;
|
||||
constructor();
|
||||
/**
|
||||
* Configure algorithm for a specific task type
|
||||
*/
|
||||
configure(task: TaskType, config: Partial<LearningConfig>): void;
|
||||
/**
|
||||
* Get current configuration for a task
|
||||
*/
|
||||
getConfig(task: TaskType): LearningConfig;
|
||||
/**
|
||||
* Update Q-value using the appropriate algorithm
|
||||
*/
|
||||
update(task: TaskType, experience: Experience): number;
|
||||
/**
|
||||
* Get best action for a state
|
||||
*/
|
||||
getBestAction(task: TaskType, state: string, actions: string[]): {
|
||||
action: string;
|
||||
confidence: number;
|
||||
};
|
||||
/**
|
||||
* Get action probabilities (for Actor-Critic and PPO)
|
||||
*/
|
||||
getActionProbabilities(state: string, actions: string[]): Map<string, number>;
|
||||
/**
|
||||
* Standard Q-Learning: Q(s,a) += α * (r + γ * max_a' Q(s',a') - Q(s,a))
|
||||
*/
|
||||
private qLearningUpdate;
|
||||
/**
|
||||
* SARSA: On-policy, more conservative
|
||||
* Q(s,a) += α * (r + γ * Q(s',a') - Q(s,a))
|
||||
*/
|
||||
private sarsaUpdate;
|
||||
/**
|
||||
* Double Q-Learning: Reduces overestimation bias
|
||||
* Uses two Q-tables, randomly updates one using the other for target
|
||||
*/
|
||||
private doubleQUpdate;
|
||||
/**
|
||||
* Actor-Critic: Policy gradient with value baseline
|
||||
*/
|
||||
private actorCriticUpdate;
|
||||
/**
|
||||
* PPO: Clipped policy gradient for stable training
|
||||
*/
|
||||
private ppoUpdate;
|
||||
/**
|
||||
* TD(λ): Temporal difference with eligibility traces
|
||||
*/
|
||||
private tdLambdaUpdate;
|
||||
/**
|
||||
* Monte Carlo: Full episode learning
|
||||
*/
|
||||
private monteCarloUpdate;
|
||||
/**
|
||||
* Decision Transformer: Sequence modeling for trajectories
|
||||
*/
|
||||
private decisionTransformerUpdate;
|
||||
/**
|
||||
* DQN: Deep Q-Network (simplified without actual neural network)
|
||||
* Uses experience replay and target network concepts
|
||||
*/
|
||||
private dqnUpdate;
|
||||
private getQTable;
|
||||
private getQTable2;
|
||||
private getEligibilityTraces;
|
||||
private softmaxConfidence;
|
||||
private addToCurrentTrajectory;
|
||||
private sampleFromReplay;
|
||||
private updateStats;
|
||||
/**
|
||||
* Get statistics for all algorithms
|
||||
*/
|
||||
getStats(): Map<LearningAlgorithm, AlgorithmStats>;
|
||||
/**
|
||||
* Get statistics summary
|
||||
*/
|
||||
getStatsSummary(): {
|
||||
bestAlgorithm: LearningAlgorithm;
|
||||
totalUpdates: number;
|
||||
avgReward: number;
|
||||
algorithms: AlgorithmStats[];
|
||||
};
|
||||
/**
|
||||
* Export state for persistence
|
||||
*/
|
||||
export(): {
|
||||
qTables: Record<string, Record<string, number>>;
|
||||
qTables2: Record<string, Record<string, number>>;
|
||||
criticValues: Record<string, number>;
|
||||
trajectories: LearningTrajectory[];
|
||||
stats: Record<string, AlgorithmStats>;
|
||||
configs: Record<string, LearningConfig>;
|
||||
rewardHistory: number[];
|
||||
};
|
||||
/**
|
||||
* Import state from persistence
|
||||
*/
|
||||
import(data: ReturnType<LearningEngine['export']>): void;
|
||||
/**
|
||||
* Clear all learning data
|
||||
*/
|
||||
clear(): void;
|
||||
/**
|
||||
* Get available algorithms
|
||||
*/
|
||||
static getAlgorithms(): {
|
||||
algorithm: LearningAlgorithm;
|
||||
description: string;
|
||||
bestFor: string;
|
||||
}[];
|
||||
}
|
||||
export default LearningEngine;
|
||||
//# sourceMappingURL=learning-engine.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/learning-engine.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/learning-engine.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"learning-engine.d.ts","sourceRoot":"","sources":["learning-engine.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,MAAM,MAAM,iBAAiB,GACzB,YAAY,GACZ,OAAO,GACP,UAAU,GACV,cAAc,GACd,KAAK,GACL,sBAAsB,GACtB,aAAa,GACb,WAAW,GACX,KAAK,CAAC;AAEV,MAAM,MAAM,QAAQ,GAChB,eAAe,GACf,iBAAiB,GACjB,oBAAoB,GACpB,qBAAqB,GACrB,iBAAiB,GACjB,eAAe,CAAC;AAEpB,MAAM,WAAW,cAAc;IAC7B,SAAS,EAAE,iBAAiB,CAAC;IAC7B,YAAY,EAAE,MAAM,CAAC;IACrB,cAAc,EAAE,MAAM,CAAC;IACvB,OAAO,EAAE,MAAM,CAAC;IAChB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,cAAc,CAAC,EAAE,MAAM,CAAC;CACzB;AAED,MAAM,WAAW,UAAU;IACzB,KAAK,EAAE,MAAM,CAAC;IACd,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,OAAO,CAAC;IACd,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,kBAAkB;IACjC,WAAW,EAAE,UAAU,EAAE,CAAC;IAC1B,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,OAAO,CAAC;CACpB;AAED,MAAM,WAAW,cAAc;IAC7B,SAAS,EAAE,iBAAiB,CAAC;IAC7B,OAAO,EAAE,MAAM,CAAC;IAChB,SAAS,EAAE,MAAM,CAAC;IAClB,gBAAgB,EAAE,MAAM,CAAC;IACzB,UAAU,EAAE,MAAM,CAAC;CACpB;AA+CD,qBAAa,cAAc;IACzB,OAAO,CAAC,OAAO,CAA4C;IAC3D,OAAO,CAAC,OAAO,CAA+C;IAC9D,OAAO,CAAC,QAAQ,CAA+C;IAC/D,OAAO,CAAC,iBAAiB,CAA+C;IACxE,OAAO,CAAC,YAAY,CAAoC;IACxD,OAAO,CAAC,YAAY,CAAkC;IACtD,OAAO,CAAC,YAAY,CAA4B;IAChD,OAAO,CAAC,KAAK,CAAqD;IAClE,OAAO,CAAC,aAAa,CAAgB;;IAwBrC;;OAEG;IACH,SAAS,CAAC,IAAI,EAAE,QAAQ,EAAE,MAAM,EAAE,OAAO,CAAC,cAAc,CAAC,GAAG,IAAI;IAKhE;;OAEG;IACH,SAAS,CAAC,IAAI,EAAE,QAAQ,GAAG,cAAc;IAIzC;;OAEG;IACH,MAAM,CAAC,IAAI,EAAE,QAAQ,EAAE,UAAU,EAAE,UAAU,GAAG,MAAM;IA+CtD;;OAEG;IACH,aAAa,CAAC,IAAI,EAAE,QAAQ,EAAE,KAAK,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG;QAAE,MAAM,EAAE,MAAM,CAAC;QAAC,UAAU,EAAE,MAAM,CAAA;KAAE;IA8BvG;;OAEG;IACH,sBAAsB,CAAC,KAAK,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC;IAkB7E;;OAEG;IACH,OAAO,CAAC,eAAe;IAkBvB;;;OAGG;IACH,OAAO,CAAC,WAAW;IA6BnB;;;OAGG;IACH,OAAO,CAAC,aAAa;IAoCrB;;OAEG;IACH,OAAO,CAAC,iBAAiB;IAoBzB;;OAEG;IACH,OAAO,CAAC,SAAS;IA0BjB;;OAEG;IACH,OAAO,CAAC,cAAc;IA8BtB;;OAEG;IACH,OAAO,CAAC,gBAAgB;IA0BxB;;OAEG;IACH,OAAO,CAAC,yBAAyB;IAwCjC;;;OAGG;IACH,OAAO,CAAC,SAAS;IAcjB,OAAO,CAAC,SAAS;IAOjB,OAAO,CAAC,UAAU;IAOlB,OAAO,CAAC,oBAAoB;IAO5B,OAAO,CAAC,iBAAiB;IAQzB,OAAO,CAAC,sBAAsB;IAW9B,OAAO,CAAC,gBAAgB;IASxB,OAAO,CAAC,WAAW;IAkBnB;;OAEG;IACH,QAAQ,IAAI,GAAG,CAAC,iBAAiB,EAAE,cAAc,CAAC;IAIlD;;OAEG;IACH,eAAe,IAAI;QACjB,aAAa,EAAE,iBAAiB,CAAC;QACjC,YAAY,EAAE,MAAM,CAAC;QACrB,SAAS,EAAE,MAAM,CAAC;QAClB,UAAU,EAAE,cAAc,EAAE,CAAC;KAC9B;IA4BD;;OAEG;IACH,MAAM,IAAI;QACR,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAC;QAChD,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAC;QACjD,YAAY,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QACrC,YAAY,EAAE,kBAAkB,EAAE,CAAC;QACnC,KAAK,EAAE,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC;QACtC,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC;QACxC,aAAa,EAAE,MAAM,EAAE,CAAC;KACzB;IAiCD;;OAEG;IACH,MAAM,CAAC,IAAI,EAAE,UAAU,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC,GAAG,IAAI;IAgCxD;;OAEG;IACH,KAAK,IAAI,IAAI;IAiBb;;OAEG;IACH,MAAM,CAAC,aAAa,IAAI;QAAE,SAAS,EAAE,iBAAiB,CAAC;QAAC,WAAW,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE,EAAE;CAajG;AAED,eAAe,cAAc,CAAC"}
|
||||
590
npm/packages/ruvector/src/core/learning-engine.js
Normal file
590
npm/packages/ruvector/src/core/learning-engine.js
Normal file
@@ -0,0 +1,590 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Multi-Algorithm Learning Engine
|
||||
* Supports 9 RL algorithms for intelligent hooks optimization
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.LearningEngine = void 0;
|
||||
// Default configs for each task type
|
||||
const TASK_ALGORITHM_MAP = {
|
||||
'agent-routing': {
|
||||
algorithm: 'double-q',
|
||||
learningRate: 0.1,
|
||||
discountFactor: 0.95,
|
||||
epsilon: 0.1,
|
||||
},
|
||||
'error-avoidance': {
|
||||
algorithm: 'sarsa',
|
||||
learningRate: 0.05,
|
||||
discountFactor: 0.99,
|
||||
epsilon: 0.05,
|
||||
},
|
||||
'confidence-scoring': {
|
||||
algorithm: 'actor-critic',
|
||||
learningRate: 0.01,
|
||||
discountFactor: 0.95,
|
||||
epsilon: 0.1,
|
||||
entropyCoef: 0.01,
|
||||
},
|
||||
'trajectory-learning': {
|
||||
algorithm: 'decision-transformer',
|
||||
learningRate: 0.001,
|
||||
discountFactor: 0.99,
|
||||
epsilon: 0,
|
||||
sequenceLength: 20,
|
||||
},
|
||||
'context-ranking': {
|
||||
algorithm: 'ppo',
|
||||
learningRate: 0.0003,
|
||||
discountFactor: 0.99,
|
||||
epsilon: 0.2,
|
||||
clipRange: 0.2,
|
||||
entropyCoef: 0.01,
|
||||
},
|
||||
'memory-recall': {
|
||||
algorithm: 'td-lambda',
|
||||
learningRate: 0.1,
|
||||
discountFactor: 0.9,
|
||||
epsilon: 0.1,
|
||||
lambda: 0.8,
|
||||
},
|
||||
};
|
||||
class LearningEngine {
|
||||
constructor() {
|
||||
this.configs = new Map();
|
||||
this.qTables = new Map();
|
||||
this.qTables2 = new Map(); // For Double-Q
|
||||
this.eligibilityTraces = new Map();
|
||||
this.actorWeights = new Map();
|
||||
this.criticValues = new Map();
|
||||
this.trajectories = [];
|
||||
this.stats = new Map();
|
||||
this.rewardHistory = [];
|
||||
// Initialize with default configs
|
||||
for (const [task, config] of Object.entries(TASK_ALGORITHM_MAP)) {
|
||||
this.configs.set(task, { ...config });
|
||||
}
|
||||
// Initialize stats for all algorithms
|
||||
const algorithms = [
|
||||
'q-learning', 'sarsa', 'double-q', 'actor-critic',
|
||||
'ppo', 'decision-transformer', 'monte-carlo', 'td-lambda', 'dqn'
|
||||
];
|
||||
for (const alg of algorithms) {
|
||||
this.stats.set(alg, {
|
||||
algorithm: alg,
|
||||
updates: 0,
|
||||
avgReward: 0,
|
||||
convergenceScore: 0,
|
||||
lastUpdate: Date.now(),
|
||||
});
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Configure algorithm for a specific task type
|
||||
*/
|
||||
configure(task, config) {
|
||||
const existing = this.configs.get(task) || TASK_ALGORITHM_MAP[task];
|
||||
this.configs.set(task, { ...existing, ...config });
|
||||
}
|
||||
/**
|
||||
* Get current configuration for a task
|
||||
*/
|
||||
getConfig(task) {
|
||||
return this.configs.get(task) || TASK_ALGORITHM_MAP[task];
|
||||
}
|
||||
/**
|
||||
* Update Q-value using the appropriate algorithm
|
||||
*/
|
||||
update(task, experience) {
|
||||
const config = this.getConfig(task);
|
||||
let delta = 0;
|
||||
switch (config.algorithm) {
|
||||
case 'q-learning':
|
||||
delta = this.qLearningUpdate(experience, config);
|
||||
break;
|
||||
case 'sarsa':
|
||||
delta = this.sarsaUpdate(experience, config);
|
||||
break;
|
||||
case 'double-q':
|
||||
delta = this.doubleQUpdate(experience, config);
|
||||
break;
|
||||
case 'actor-critic':
|
||||
delta = this.actorCriticUpdate(experience, config);
|
||||
break;
|
||||
case 'ppo':
|
||||
delta = this.ppoUpdate(experience, config);
|
||||
break;
|
||||
case 'td-lambda':
|
||||
delta = this.tdLambdaUpdate(experience, config);
|
||||
break;
|
||||
case 'monte-carlo':
|
||||
// Monte Carlo needs full episodes
|
||||
this.addToCurrentTrajectory(experience);
|
||||
if (experience.done) {
|
||||
delta = this.monteCarloUpdate(config);
|
||||
}
|
||||
break;
|
||||
case 'decision-transformer':
|
||||
this.addToCurrentTrajectory(experience);
|
||||
if (experience.done) {
|
||||
delta = this.decisionTransformerUpdate(config);
|
||||
}
|
||||
break;
|
||||
case 'dqn':
|
||||
delta = this.dqnUpdate(experience, config);
|
||||
break;
|
||||
}
|
||||
// Update stats
|
||||
this.updateStats(config.algorithm, experience.reward, Math.abs(delta));
|
||||
return delta;
|
||||
}
|
||||
/**
|
||||
* Get best action for a state
|
||||
*/
|
||||
getBestAction(task, state, actions) {
|
||||
const config = this.getConfig(task);
|
||||
// Epsilon-greedy exploration
|
||||
if (Math.random() < config.epsilon) {
|
||||
const randomAction = actions[Math.floor(Math.random() * actions.length)];
|
||||
return { action: randomAction, confidence: 0.5 };
|
||||
}
|
||||
let bestAction = actions[0];
|
||||
let bestValue = -Infinity;
|
||||
let values = [];
|
||||
const qTable = this.getQTable(state);
|
||||
for (const action of actions) {
|
||||
const value = qTable.get(action) || 0;
|
||||
values.push(value);
|
||||
if (value > bestValue) {
|
||||
bestValue = value;
|
||||
bestAction = action;
|
||||
}
|
||||
}
|
||||
// Calculate confidence using softmax
|
||||
const confidence = this.softmaxConfidence(values, actions.indexOf(bestAction));
|
||||
return { action: bestAction, confidence };
|
||||
}
|
||||
/**
|
||||
* Get action probabilities (for Actor-Critic and PPO)
|
||||
*/
|
||||
getActionProbabilities(state, actions) {
|
||||
const probs = new Map();
|
||||
const qTable = this.getQTable(state);
|
||||
const values = actions.map(a => qTable.get(a) || 0);
|
||||
const maxVal = Math.max(...values);
|
||||
const expValues = values.map(v => Math.exp(v - maxVal));
|
||||
const sumExp = expValues.reduce((a, b) => a + b, 0);
|
||||
for (let i = 0; i < actions.length; i++) {
|
||||
probs.set(actions[i], expValues[i] / sumExp);
|
||||
}
|
||||
return probs;
|
||||
}
|
||||
// ============ Algorithm Implementations ============
|
||||
/**
|
||||
* Standard Q-Learning: Q(s,a) += α * (r + γ * max_a' Q(s',a') - Q(s,a))
|
||||
*/
|
||||
qLearningUpdate(exp, config) {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
const qTable = this.getQTable(state);
|
||||
const nextQTable = this.getQTable(nextState);
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
const maxNextQ = done ? 0 : Math.max(0, ...Array.from(nextQTable.values()));
|
||||
const target = reward + γ * maxNextQ;
|
||||
const delta = target - currentQ;
|
||||
const newQ = currentQ + α * delta;
|
||||
qTable.set(action, newQ);
|
||||
return delta;
|
||||
}
|
||||
/**
|
||||
* SARSA: On-policy, more conservative
|
||||
* Q(s,a) += α * (r + γ * Q(s',a') - Q(s,a))
|
||||
*/
|
||||
sarsaUpdate(exp, config) {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ, epsilon } = config;
|
||||
const qTable = this.getQTable(state);
|
||||
const nextQTable = this.getQTable(nextState);
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
// On-policy: use expected value under current policy (ε-greedy)
|
||||
let nextQ = 0;
|
||||
if (!done) {
|
||||
const nextActions = Array.from(nextQTable.keys());
|
||||
if (nextActions.length > 0) {
|
||||
const maxQ = Math.max(...Array.from(nextQTable.values()));
|
||||
const avgQ = Array.from(nextQTable.values()).reduce((a, b) => a + b, 0) / nextActions.length;
|
||||
// Expected value under ε-greedy
|
||||
nextQ = (1 - epsilon) * maxQ + epsilon * avgQ;
|
||||
}
|
||||
}
|
||||
const target = reward + γ * nextQ;
|
||||
const delta = target - currentQ;
|
||||
const newQ = currentQ + α * delta;
|
||||
qTable.set(action, newQ);
|
||||
return delta;
|
||||
}
|
||||
/**
|
||||
* Double Q-Learning: Reduces overestimation bias
|
||||
* Uses two Q-tables, randomly updates one using the other for target
|
||||
*/
|
||||
doubleQUpdate(exp, config) {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
const useFirst = Math.random() < 0.5;
|
||||
const qTable = useFirst ? this.getQTable(state) : this.getQTable2(state);
|
||||
const otherQTable = useFirst ? this.getQTable2(nextState) : this.getQTable(nextState);
|
||||
const nextQTable = useFirst ? this.getQTable(nextState) : this.getQTable2(nextState);
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
let nextQ = 0;
|
||||
if (!done) {
|
||||
// Find best action in next state using one table
|
||||
let bestAction = '';
|
||||
let bestValue = -Infinity;
|
||||
for (const [a, v] of nextQTable) {
|
||||
if (v > bestValue) {
|
||||
bestValue = v;
|
||||
bestAction = a;
|
||||
}
|
||||
}
|
||||
// Evaluate using other table
|
||||
if (bestAction) {
|
||||
nextQ = otherQTable.get(bestAction) || 0;
|
||||
}
|
||||
}
|
||||
const target = reward + γ * nextQ;
|
||||
const delta = target - currentQ;
|
||||
const newQ = currentQ + α * delta;
|
||||
qTable.set(action, newQ);
|
||||
return delta;
|
||||
}
|
||||
/**
|
||||
* Actor-Critic: Policy gradient with value baseline
|
||||
*/
|
||||
actorCriticUpdate(exp, config) {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
// Critic update (TD error)
|
||||
const V = this.criticValues.get(state) || 0;
|
||||
const V_next = done ? 0 : (this.criticValues.get(nextState) || 0);
|
||||
const tdError = reward + γ * V_next - V;
|
||||
this.criticValues.set(state, V + α * tdError);
|
||||
// Actor update (policy gradient)
|
||||
const qTable = this.getQTable(state);
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
// Use TD error as advantage estimate
|
||||
const newQ = currentQ + α * tdError;
|
||||
qTable.set(action, newQ);
|
||||
return tdError;
|
||||
}
|
||||
/**
|
||||
* PPO: Clipped policy gradient for stable training
|
||||
*/
|
||||
ppoUpdate(exp, config) {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ, clipRange = 0.2 } = config;
|
||||
// Critic update
|
||||
const V = this.criticValues.get(state) || 0;
|
||||
const V_next = done ? 0 : (this.criticValues.get(nextState) || 0);
|
||||
const advantage = reward + γ * V_next - V;
|
||||
this.criticValues.set(state, V + α * advantage);
|
||||
// Actor update with clipping
|
||||
const qTable = this.getQTable(state);
|
||||
const oldQ = qTable.get(action) || 0;
|
||||
// Compute probability ratio (simplified)
|
||||
const ratio = Math.exp(α * advantage);
|
||||
const clippedRatio = Math.max(1 - clipRange, Math.min(1 + clipRange, ratio));
|
||||
// PPO objective: min(ratio * A, clip(ratio) * A)
|
||||
const update = Math.min(ratio * advantage, clippedRatio * advantage);
|
||||
const newQ = oldQ + α * update;
|
||||
qTable.set(action, newQ);
|
||||
return advantage;
|
||||
}
|
||||
/**
|
||||
* TD(λ): Temporal difference with eligibility traces
|
||||
*/
|
||||
tdLambdaUpdate(exp, config) {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ, lambda = 0.8 } = config;
|
||||
const qTable = this.getQTable(state);
|
||||
const nextQTable = this.getQTable(nextState);
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
const maxNextQ = done ? 0 : Math.max(0, ...Array.from(nextQTable.values()));
|
||||
const tdError = reward + γ * maxNextQ - currentQ;
|
||||
// Update eligibility trace for current state-action
|
||||
const traces = this.getEligibilityTraces(state);
|
||||
traces.set(action, (traces.get(action) || 0) + 1);
|
||||
// Update all state-actions with eligibility traces
|
||||
for (const [s, sTraces] of this.eligibilityTraces) {
|
||||
const sQTable = this.getQTable(s);
|
||||
for (const [a, trace] of sTraces) {
|
||||
const q = sQTable.get(a) || 0;
|
||||
sQTable.set(a, q + α * tdError * trace);
|
||||
// Decay trace
|
||||
sTraces.set(a, γ * lambda * trace);
|
||||
}
|
||||
}
|
||||
return tdError;
|
||||
}
|
||||
/**
|
||||
* Monte Carlo: Full episode learning
|
||||
*/
|
||||
monteCarloUpdate(config) {
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
const trajectory = this.trajectories[this.trajectories.length - 1];
|
||||
if (!trajectory || trajectory.experiences.length === 0)
|
||||
return 0;
|
||||
let G = 0; // Return
|
||||
let totalDelta = 0;
|
||||
// Work backwards through episode
|
||||
for (let t = trajectory.experiences.length - 1; t >= 0; t--) {
|
||||
const exp = trajectory.experiences[t];
|
||||
G = exp.reward + γ * G;
|
||||
const qTable = this.getQTable(exp.state);
|
||||
const currentQ = qTable.get(exp.action) || 0;
|
||||
const delta = G - currentQ;
|
||||
qTable.set(exp.action, currentQ + α * delta);
|
||||
totalDelta += Math.abs(delta);
|
||||
}
|
||||
trajectory.completed = true;
|
||||
trajectory.totalReward = G;
|
||||
return totalDelta / trajectory.experiences.length;
|
||||
}
|
||||
/**
|
||||
* Decision Transformer: Sequence modeling for trajectories
|
||||
*/
|
||||
decisionTransformerUpdate(config) {
|
||||
const { learningRate: α, sequenceLength = 20 } = config;
|
||||
const trajectory = this.trajectories[this.trajectories.length - 1];
|
||||
if (!trajectory || trajectory.experiences.length === 0)
|
||||
return 0;
|
||||
// Decision Transformer learns to predict actions given (return, state, action) sequences
|
||||
// Here we use a simplified version that learns state-action patterns
|
||||
let totalDelta = 0;
|
||||
const experiences = trajectory.experiences.slice(-sequenceLength);
|
||||
// Calculate returns-to-go
|
||||
const returns = [];
|
||||
let R = 0;
|
||||
for (let i = experiences.length - 1; i >= 0; i--) {
|
||||
R += experiences[i].reward;
|
||||
returns.unshift(R);
|
||||
}
|
||||
// Update Q-values weighted by return-to-go
|
||||
for (let i = 0; i < experiences.length; i++) {
|
||||
const exp = experiences[i];
|
||||
const qTable = this.getQTable(exp.state);
|
||||
const currentQ = qTable.get(exp.action) || 0;
|
||||
// Weight by normalized return
|
||||
const normalizedReturn = returns[i] / (Math.abs(returns[0]) + 1);
|
||||
const target = currentQ + α * normalizedReturn * exp.reward;
|
||||
const delta = target - currentQ;
|
||||
qTable.set(exp.action, target);
|
||||
totalDelta += Math.abs(delta);
|
||||
}
|
||||
trajectory.completed = true;
|
||||
trajectory.totalReward = returns[0];
|
||||
return totalDelta / experiences.length;
|
||||
}
|
||||
/**
|
||||
* DQN: Deep Q-Network (simplified without actual neural network)
|
||||
* Uses experience replay and target network concepts
|
||||
*/
|
||||
dqnUpdate(exp, config) {
|
||||
// Add to replay buffer (trajectory)
|
||||
this.addToCurrentTrajectory(exp);
|
||||
// Sample from replay buffer
|
||||
const replayExp = this.sampleFromReplay();
|
||||
if (!replayExp)
|
||||
return this.qLearningUpdate(exp, config);
|
||||
// Use sampled experience for update (breaks correlation)
|
||||
return this.qLearningUpdate(replayExp, config);
|
||||
}
|
||||
// ============ Helper Methods ============
|
||||
getQTable(state) {
|
||||
if (!this.qTables.has(state)) {
|
||||
this.qTables.set(state, new Map());
|
||||
}
|
||||
return this.qTables.get(state);
|
||||
}
|
||||
getQTable2(state) {
|
||||
if (!this.qTables2.has(state)) {
|
||||
this.qTables2.set(state, new Map());
|
||||
}
|
||||
return this.qTables2.get(state);
|
||||
}
|
||||
getEligibilityTraces(state) {
|
||||
if (!this.eligibilityTraces.has(state)) {
|
||||
this.eligibilityTraces.set(state, new Map());
|
||||
}
|
||||
return this.eligibilityTraces.get(state);
|
||||
}
|
||||
softmaxConfidence(values, selectedIdx) {
|
||||
if (values.length === 0)
|
||||
return 0.5;
|
||||
const maxVal = Math.max(...values);
|
||||
const expValues = values.map(v => Math.exp(v - maxVal));
|
||||
const sumExp = expValues.reduce((a, b) => a + b, 0);
|
||||
return expValues[selectedIdx] / sumExp;
|
||||
}
|
||||
addToCurrentTrajectory(exp) {
|
||||
if (this.trajectories.length === 0 || this.trajectories[this.trajectories.length - 1].completed) {
|
||||
this.trajectories.push({
|
||||
experiences: [],
|
||||
totalReward: 0,
|
||||
completed: false,
|
||||
});
|
||||
}
|
||||
this.trajectories[this.trajectories.length - 1].experiences.push(exp);
|
||||
}
|
||||
sampleFromReplay() {
|
||||
const allExperiences = [];
|
||||
for (const traj of this.trajectories) {
|
||||
allExperiences.push(...traj.experiences);
|
||||
}
|
||||
if (allExperiences.length === 0)
|
||||
return null;
|
||||
return allExperiences[Math.floor(Math.random() * allExperiences.length)];
|
||||
}
|
||||
updateStats(algorithm, reward, delta) {
|
||||
const stats = this.stats.get(algorithm);
|
||||
if (!stats)
|
||||
return;
|
||||
stats.updates++;
|
||||
stats.lastUpdate = Date.now();
|
||||
// Running average reward
|
||||
this.rewardHistory.push(reward);
|
||||
if (this.rewardHistory.length > 1000) {
|
||||
this.rewardHistory.shift();
|
||||
}
|
||||
stats.avgReward = this.rewardHistory.reduce((a, b) => a + b, 0) / this.rewardHistory.length;
|
||||
// Convergence score (inverse of recent delta magnitude)
|
||||
stats.convergenceScore = 1 / (1 + delta);
|
||||
}
|
||||
/**
|
||||
* Get statistics for all algorithms
|
||||
*/
|
||||
getStats() {
|
||||
return new Map(this.stats);
|
||||
}
|
||||
/**
|
||||
* Get statistics summary
|
||||
*/
|
||||
getStatsSummary() {
|
||||
let bestAlgorithm = 'q-learning';
|
||||
let bestScore = -Infinity;
|
||||
let totalUpdates = 0;
|
||||
const algorithms = [];
|
||||
for (const [alg, stats] of this.stats) {
|
||||
algorithms.push(stats);
|
||||
totalUpdates += stats.updates;
|
||||
const score = stats.avgReward * stats.convergenceScore;
|
||||
if (score > bestScore && stats.updates > 0) {
|
||||
bestScore = score;
|
||||
bestAlgorithm = alg;
|
||||
}
|
||||
}
|
||||
return {
|
||||
bestAlgorithm,
|
||||
totalUpdates,
|
||||
avgReward: this.rewardHistory.length > 0
|
||||
? this.rewardHistory.reduce((a, b) => a + b, 0) / this.rewardHistory.length
|
||||
: 0,
|
||||
algorithms: algorithms.filter(a => a.updates > 0),
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Export state for persistence
|
||||
*/
|
||||
export() {
|
||||
const qTables = {};
|
||||
for (const [state, actions] of this.qTables) {
|
||||
qTables[state] = Object.fromEntries(actions);
|
||||
}
|
||||
const qTables2 = {};
|
||||
for (const [state, actions] of this.qTables2) {
|
||||
qTables2[state] = Object.fromEntries(actions);
|
||||
}
|
||||
const criticValues = Object.fromEntries(this.criticValues);
|
||||
const stats = {};
|
||||
for (const [alg, s] of this.stats) {
|
||||
stats[alg] = s;
|
||||
}
|
||||
const configs = {};
|
||||
for (const [task, config] of this.configs) {
|
||||
configs[task] = config;
|
||||
}
|
||||
return {
|
||||
qTables,
|
||||
qTables2,
|
||||
criticValues,
|
||||
trajectories: this.trajectories.slice(-100), // Keep last 100 trajectories
|
||||
stats,
|
||||
configs,
|
||||
rewardHistory: this.rewardHistory.slice(-1000),
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Import state from persistence
|
||||
*/
|
||||
import(data) {
|
||||
// Q-tables
|
||||
this.qTables.clear();
|
||||
for (const [state, actions] of Object.entries(data.qTables || {})) {
|
||||
this.qTables.set(state, new Map(Object.entries(actions)));
|
||||
}
|
||||
this.qTables2.clear();
|
||||
for (const [state, actions] of Object.entries(data.qTables2 || {})) {
|
||||
this.qTables2.set(state, new Map(Object.entries(actions)));
|
||||
}
|
||||
// Critic values
|
||||
this.criticValues = new Map(Object.entries(data.criticValues || {}));
|
||||
// Trajectories
|
||||
this.trajectories = data.trajectories || [];
|
||||
// Stats
|
||||
for (const [alg, s] of Object.entries(data.stats || {})) {
|
||||
this.stats.set(alg, s);
|
||||
}
|
||||
// Configs
|
||||
for (const [task, config] of Object.entries(data.configs || {})) {
|
||||
this.configs.set(task, config);
|
||||
}
|
||||
// Reward history
|
||||
this.rewardHistory = data.rewardHistory || [];
|
||||
}
|
||||
/**
|
||||
* Clear all learning data
|
||||
*/
|
||||
clear() {
|
||||
this.qTables.clear();
|
||||
this.qTables2.clear();
|
||||
this.eligibilityTraces.clear();
|
||||
this.actorWeights.clear();
|
||||
this.criticValues.clear();
|
||||
this.trajectories = [];
|
||||
this.rewardHistory = [];
|
||||
// Reset stats
|
||||
for (const stats of this.stats.values()) {
|
||||
stats.updates = 0;
|
||||
stats.avgReward = 0;
|
||||
stats.convergenceScore = 0;
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Get available algorithms
|
||||
*/
|
||||
static getAlgorithms() {
|
||||
return [
|
||||
{ algorithm: 'q-learning', description: 'Simple off-policy learning', bestFor: 'General routing' },
|
||||
{ algorithm: 'sarsa', description: 'On-policy, conservative', bestFor: 'Error avoidance' },
|
||||
{ algorithm: 'double-q', description: 'Reduces overestimation', bestFor: 'Precise routing' },
|
||||
{ algorithm: 'actor-critic', description: 'Policy gradient + value', bestFor: 'Confidence scoring' },
|
||||
{ algorithm: 'ppo', description: 'Stable policy updates', bestFor: 'Preference learning' },
|
||||
{ algorithm: 'decision-transformer', description: 'Sequence modeling', bestFor: 'Trajectory patterns' },
|
||||
{ algorithm: 'monte-carlo', description: 'Full episode learning', bestFor: 'Unbiased estimates' },
|
||||
{ algorithm: 'td-lambda', description: 'Eligibility traces', bestFor: 'Credit assignment' },
|
||||
{ algorithm: 'dqn', description: 'Experience replay', bestFor: 'High-dim states' },
|
||||
];
|
||||
}
|
||||
}
|
||||
exports.LearningEngine = LearningEngine;
|
||||
exports.default = LearningEngine;
|
||||
//# sourceMappingURL=learning-engine.js.map
|
||||
1
npm/packages/ruvector/src/core/learning-engine.js.map
Normal file
1
npm/packages/ruvector/src/core/learning-engine.js.map
Normal file
File diff suppressed because one or more lines are too long
752
npm/packages/ruvector/src/core/learning-engine.ts
Normal file
752
npm/packages/ruvector/src/core/learning-engine.ts
Normal file
@@ -0,0 +1,752 @@
|
||||
/**
|
||||
* Multi-Algorithm Learning Engine
|
||||
* Supports 9 RL algorithms for intelligent hooks optimization
|
||||
*/
|
||||
|
||||
export type LearningAlgorithm =
|
||||
| 'q-learning'
|
||||
| 'sarsa'
|
||||
| 'double-q'
|
||||
| 'actor-critic'
|
||||
| 'ppo'
|
||||
| 'decision-transformer'
|
||||
| 'monte-carlo'
|
||||
| 'td-lambda'
|
||||
| 'dqn';
|
||||
|
||||
export type TaskType =
|
||||
| 'agent-routing'
|
||||
| 'error-avoidance'
|
||||
| 'confidence-scoring'
|
||||
| 'trajectory-learning'
|
||||
| 'context-ranking'
|
||||
| 'memory-recall';
|
||||
|
||||
export interface LearningConfig {
|
||||
algorithm: LearningAlgorithm;
|
||||
learningRate: number;
|
||||
discountFactor: number;
|
||||
epsilon: number; // Exploration rate
|
||||
lambda?: number; // For TD(λ)
|
||||
clipRange?: number; // For PPO
|
||||
entropyCoef?: number; // For Actor-Critic/PPO
|
||||
sequenceLength?: number; // For Decision Transformer
|
||||
}
|
||||
|
||||
export interface Experience {
|
||||
state: string;
|
||||
action: string;
|
||||
reward: number;
|
||||
nextState: string;
|
||||
done: boolean;
|
||||
timestamp?: number;
|
||||
}
|
||||
|
||||
export interface LearningTrajectory {
|
||||
experiences: Experience[];
|
||||
totalReward: number;
|
||||
completed: boolean;
|
||||
}
|
||||
|
||||
export interface AlgorithmStats {
|
||||
algorithm: LearningAlgorithm;
|
||||
updates: number;
|
||||
avgReward: number;
|
||||
convergenceScore: number;
|
||||
lastUpdate: number;
|
||||
}
|
||||
|
||||
// Default configs for each task type
|
||||
const TASK_ALGORITHM_MAP: Record<TaskType, LearningConfig> = {
|
||||
'agent-routing': {
|
||||
algorithm: 'double-q',
|
||||
learningRate: 0.1,
|
||||
discountFactor: 0.95,
|
||||
epsilon: 0.1,
|
||||
},
|
||||
'error-avoidance': {
|
||||
algorithm: 'sarsa',
|
||||
learningRate: 0.05,
|
||||
discountFactor: 0.99,
|
||||
epsilon: 0.05,
|
||||
},
|
||||
'confidence-scoring': {
|
||||
algorithm: 'actor-critic',
|
||||
learningRate: 0.01,
|
||||
discountFactor: 0.95,
|
||||
epsilon: 0.1,
|
||||
entropyCoef: 0.01,
|
||||
},
|
||||
'trajectory-learning': {
|
||||
algorithm: 'decision-transformer',
|
||||
learningRate: 0.001,
|
||||
discountFactor: 0.99,
|
||||
epsilon: 0,
|
||||
sequenceLength: 20,
|
||||
},
|
||||
'context-ranking': {
|
||||
algorithm: 'ppo',
|
||||
learningRate: 0.0003,
|
||||
discountFactor: 0.99,
|
||||
epsilon: 0.2,
|
||||
clipRange: 0.2,
|
||||
entropyCoef: 0.01,
|
||||
},
|
||||
'memory-recall': {
|
||||
algorithm: 'td-lambda',
|
||||
learningRate: 0.1,
|
||||
discountFactor: 0.9,
|
||||
epsilon: 0.1,
|
||||
lambda: 0.8,
|
||||
},
|
||||
};
|
||||
|
||||
export class LearningEngine {
|
||||
private configs: Map<TaskType, LearningConfig> = new Map();
|
||||
private qTables: Map<string, Map<string, number>> = new Map();
|
||||
private qTables2: Map<string, Map<string, number>> = new Map(); // For Double-Q
|
||||
private eligibilityTraces: Map<string, Map<string, number>> = new Map();
|
||||
private actorWeights: Map<string, number[]> = new Map();
|
||||
private criticValues: Map<string, number> = new Map();
|
||||
private trajectories: LearningTrajectory[] = [];
|
||||
private stats: Map<LearningAlgorithm, AlgorithmStats> = new Map();
|
||||
private rewardHistory: number[] = [];
|
||||
|
||||
constructor() {
|
||||
// Initialize with default configs
|
||||
for (const [task, config] of Object.entries(TASK_ALGORITHM_MAP)) {
|
||||
this.configs.set(task as TaskType, { ...config });
|
||||
}
|
||||
|
||||
// Initialize stats for all algorithms
|
||||
const algorithms: LearningAlgorithm[] = [
|
||||
'q-learning', 'sarsa', 'double-q', 'actor-critic',
|
||||
'ppo', 'decision-transformer', 'monte-carlo', 'td-lambda', 'dqn'
|
||||
];
|
||||
for (const alg of algorithms) {
|
||||
this.stats.set(alg, {
|
||||
algorithm: alg,
|
||||
updates: 0,
|
||||
avgReward: 0,
|
||||
convergenceScore: 0,
|
||||
lastUpdate: Date.now(),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Configure algorithm for a specific task type
|
||||
*/
|
||||
configure(task: TaskType, config: Partial<LearningConfig>): void {
|
||||
const existing = this.configs.get(task) || TASK_ALGORITHM_MAP[task];
|
||||
this.configs.set(task, { ...existing, ...config });
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current configuration for a task
|
||||
*/
|
||||
getConfig(task: TaskType): LearningConfig {
|
||||
return this.configs.get(task) || TASK_ALGORITHM_MAP[task];
|
||||
}
|
||||
|
||||
/**
|
||||
* Update Q-value using the appropriate algorithm
|
||||
*/
|
||||
update(task: TaskType, experience: Experience): number {
|
||||
const config = this.getConfig(task);
|
||||
let delta = 0;
|
||||
|
||||
switch (config.algorithm) {
|
||||
case 'q-learning':
|
||||
delta = this.qLearningUpdate(experience, config);
|
||||
break;
|
||||
case 'sarsa':
|
||||
delta = this.sarsaUpdate(experience, config);
|
||||
break;
|
||||
case 'double-q':
|
||||
delta = this.doubleQUpdate(experience, config);
|
||||
break;
|
||||
case 'actor-critic':
|
||||
delta = this.actorCriticUpdate(experience, config);
|
||||
break;
|
||||
case 'ppo':
|
||||
delta = this.ppoUpdate(experience, config);
|
||||
break;
|
||||
case 'td-lambda':
|
||||
delta = this.tdLambdaUpdate(experience, config);
|
||||
break;
|
||||
case 'monte-carlo':
|
||||
// Monte Carlo needs full episodes
|
||||
this.addToCurrentTrajectory(experience);
|
||||
if (experience.done) {
|
||||
delta = this.monteCarloUpdate(config);
|
||||
}
|
||||
break;
|
||||
case 'decision-transformer':
|
||||
this.addToCurrentTrajectory(experience);
|
||||
if (experience.done) {
|
||||
delta = this.decisionTransformerUpdate(config);
|
||||
}
|
||||
break;
|
||||
case 'dqn':
|
||||
delta = this.dqnUpdate(experience, config);
|
||||
break;
|
||||
}
|
||||
|
||||
// Update stats
|
||||
this.updateStats(config.algorithm, experience.reward, Math.abs(delta));
|
||||
|
||||
return delta;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get best action for a state
|
||||
*/
|
||||
getBestAction(task: TaskType, state: string, actions: string[]): { action: string; confidence: number } {
|
||||
const config = this.getConfig(task);
|
||||
|
||||
// Epsilon-greedy exploration
|
||||
if (Math.random() < config.epsilon) {
|
||||
const randomAction = actions[Math.floor(Math.random() * actions.length)];
|
||||
return { action: randomAction, confidence: 0.5 };
|
||||
}
|
||||
|
||||
let bestAction = actions[0];
|
||||
let bestValue = -Infinity;
|
||||
let values: number[] = [];
|
||||
|
||||
const qTable = this.getQTable(state);
|
||||
|
||||
for (const action of actions) {
|
||||
const value = qTable.get(action) || 0;
|
||||
values.push(value);
|
||||
if (value > bestValue) {
|
||||
bestValue = value;
|
||||
bestAction = action;
|
||||
}
|
||||
}
|
||||
|
||||
// Calculate confidence using softmax
|
||||
const confidence = this.softmaxConfidence(values, actions.indexOf(bestAction));
|
||||
|
||||
return { action: bestAction, confidence };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get action probabilities (for Actor-Critic and PPO)
|
||||
*/
|
||||
getActionProbabilities(state: string, actions: string[]): Map<string, number> {
|
||||
const probs = new Map<string, number>();
|
||||
const qTable = this.getQTable(state);
|
||||
|
||||
const values = actions.map(a => qTable.get(a) || 0);
|
||||
const maxVal = Math.max(...values);
|
||||
const expValues = values.map(v => Math.exp(v - maxVal));
|
||||
const sumExp = expValues.reduce((a, b) => a + b, 0);
|
||||
|
||||
for (let i = 0; i < actions.length; i++) {
|
||||
probs.set(actions[i], expValues[i] / sumExp);
|
||||
}
|
||||
|
||||
return probs;
|
||||
}
|
||||
|
||||
// ============ Algorithm Implementations ============
|
||||
|
||||
/**
|
||||
* Standard Q-Learning: Q(s,a) += α * (r + γ * max_a' Q(s',a') - Q(s,a))
|
||||
*/
|
||||
private qLearningUpdate(exp: Experience, config: LearningConfig): number {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
|
||||
const qTable = this.getQTable(state);
|
||||
const nextQTable = this.getQTable(nextState);
|
||||
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
const maxNextQ = done ? 0 : Math.max(0, ...Array.from(nextQTable.values()));
|
||||
|
||||
const target = reward + γ * maxNextQ;
|
||||
const delta = target - currentQ;
|
||||
const newQ = currentQ + α * delta;
|
||||
|
||||
qTable.set(action, newQ);
|
||||
return delta;
|
||||
}
|
||||
|
||||
/**
|
||||
* SARSA: On-policy, more conservative
|
||||
* Q(s,a) += α * (r + γ * Q(s',a') - Q(s,a))
|
||||
*/
|
||||
private sarsaUpdate(exp: Experience, config: LearningConfig): number {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ, epsilon } = config;
|
||||
|
||||
const qTable = this.getQTable(state);
|
||||
const nextQTable = this.getQTable(nextState);
|
||||
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
|
||||
// On-policy: use expected value under current policy (ε-greedy)
|
||||
let nextQ = 0;
|
||||
if (!done) {
|
||||
const nextActions = Array.from(nextQTable.keys());
|
||||
if (nextActions.length > 0) {
|
||||
const maxQ = Math.max(...Array.from(nextQTable.values()));
|
||||
const avgQ = Array.from(nextQTable.values()).reduce((a, b) => a + b, 0) / nextActions.length;
|
||||
// Expected value under ε-greedy
|
||||
nextQ = (1 - epsilon) * maxQ + epsilon * avgQ;
|
||||
}
|
||||
}
|
||||
|
||||
const target = reward + γ * nextQ;
|
||||
const delta = target - currentQ;
|
||||
const newQ = currentQ + α * delta;
|
||||
|
||||
qTable.set(action, newQ);
|
||||
return delta;
|
||||
}
|
||||
|
||||
/**
|
||||
* Double Q-Learning: Reduces overestimation bias
|
||||
* Uses two Q-tables, randomly updates one using the other for target
|
||||
*/
|
||||
private doubleQUpdate(exp: Experience, config: LearningConfig): number {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
|
||||
const useFirst = Math.random() < 0.5;
|
||||
const qTable = useFirst ? this.getQTable(state) : this.getQTable2(state);
|
||||
const otherQTable = useFirst ? this.getQTable2(nextState) : this.getQTable(nextState);
|
||||
const nextQTable = useFirst ? this.getQTable(nextState) : this.getQTable2(nextState);
|
||||
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
|
||||
let nextQ = 0;
|
||||
if (!done) {
|
||||
// Find best action in next state using one table
|
||||
let bestAction = '';
|
||||
let bestValue = -Infinity;
|
||||
for (const [a, v] of nextQTable) {
|
||||
if (v > bestValue) {
|
||||
bestValue = v;
|
||||
bestAction = a;
|
||||
}
|
||||
}
|
||||
// Evaluate using other table
|
||||
if (bestAction) {
|
||||
nextQ = otherQTable.get(bestAction) || 0;
|
||||
}
|
||||
}
|
||||
|
||||
const target = reward + γ * nextQ;
|
||||
const delta = target - currentQ;
|
||||
const newQ = currentQ + α * delta;
|
||||
|
||||
qTable.set(action, newQ);
|
||||
return delta;
|
||||
}
|
||||
|
||||
/**
|
||||
* Actor-Critic: Policy gradient with value baseline
|
||||
*/
|
||||
private actorCriticUpdate(exp: Experience, config: LearningConfig): number {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
|
||||
// Critic update (TD error)
|
||||
const V = this.criticValues.get(state) || 0;
|
||||
const V_next = done ? 0 : (this.criticValues.get(nextState) || 0);
|
||||
const tdError = reward + γ * V_next - V;
|
||||
this.criticValues.set(state, V + α * tdError);
|
||||
|
||||
// Actor update (policy gradient)
|
||||
const qTable = this.getQTable(state);
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
// Use TD error as advantage estimate
|
||||
const newQ = currentQ + α * tdError;
|
||||
qTable.set(action, newQ);
|
||||
|
||||
return tdError;
|
||||
}
|
||||
|
||||
/**
|
||||
* PPO: Clipped policy gradient for stable training
|
||||
*/
|
||||
private ppoUpdate(exp: Experience, config: LearningConfig): number {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ, clipRange = 0.2 } = config;
|
||||
|
||||
// Critic update
|
||||
const V = this.criticValues.get(state) || 0;
|
||||
const V_next = done ? 0 : (this.criticValues.get(nextState) || 0);
|
||||
const advantage = reward + γ * V_next - V;
|
||||
this.criticValues.set(state, V + α * advantage);
|
||||
|
||||
// Actor update with clipping
|
||||
const qTable = this.getQTable(state);
|
||||
const oldQ = qTable.get(action) || 0;
|
||||
|
||||
// Compute probability ratio (simplified)
|
||||
const ratio = Math.exp(α * advantage);
|
||||
const clippedRatio = Math.max(1 - clipRange, Math.min(1 + clipRange, ratio));
|
||||
|
||||
// PPO objective: min(ratio * A, clip(ratio) * A)
|
||||
const update = Math.min(ratio * advantage, clippedRatio * advantage);
|
||||
const newQ = oldQ + α * update;
|
||||
|
||||
qTable.set(action, newQ);
|
||||
return advantage;
|
||||
}
|
||||
|
||||
/**
|
||||
* TD(λ): Temporal difference with eligibility traces
|
||||
*/
|
||||
private tdLambdaUpdate(exp: Experience, config: LearningConfig): number {
|
||||
const { state, action, reward, nextState, done } = exp;
|
||||
const { learningRate: α, discountFactor: γ, lambda = 0.8 } = config;
|
||||
|
||||
const qTable = this.getQTable(state);
|
||||
const nextQTable = this.getQTable(nextState);
|
||||
|
||||
const currentQ = qTable.get(action) || 0;
|
||||
const maxNextQ = done ? 0 : Math.max(0, ...Array.from(nextQTable.values()));
|
||||
|
||||
const tdError = reward + γ * maxNextQ - currentQ;
|
||||
|
||||
// Update eligibility trace for current state-action
|
||||
const traces = this.getEligibilityTraces(state);
|
||||
traces.set(action, (traces.get(action) || 0) + 1);
|
||||
|
||||
// Update all state-actions with eligibility traces
|
||||
for (const [s, sTraces] of this.eligibilityTraces) {
|
||||
const sQTable = this.getQTable(s);
|
||||
for (const [a, trace] of sTraces) {
|
||||
const q = sQTable.get(a) || 0;
|
||||
sQTable.set(a, q + α * tdError * trace);
|
||||
// Decay trace
|
||||
sTraces.set(a, γ * lambda * trace);
|
||||
}
|
||||
}
|
||||
|
||||
return tdError;
|
||||
}
|
||||
|
||||
/**
|
||||
* Monte Carlo: Full episode learning
|
||||
*/
|
||||
private monteCarloUpdate(config: LearningConfig): number {
|
||||
const { learningRate: α, discountFactor: γ } = config;
|
||||
const trajectory = this.trajectories[this.trajectories.length - 1];
|
||||
if (!trajectory || trajectory.experiences.length === 0) return 0;
|
||||
|
||||
let G = 0; // Return
|
||||
let totalDelta = 0;
|
||||
|
||||
// Work backwards through episode
|
||||
for (let t = trajectory.experiences.length - 1; t >= 0; t--) {
|
||||
const exp = trajectory.experiences[t];
|
||||
G = exp.reward + γ * G;
|
||||
|
||||
const qTable = this.getQTable(exp.state);
|
||||
const currentQ = qTable.get(exp.action) || 0;
|
||||
const delta = G - currentQ;
|
||||
qTable.set(exp.action, currentQ + α * delta);
|
||||
totalDelta += Math.abs(delta);
|
||||
}
|
||||
|
||||
trajectory.completed = true;
|
||||
trajectory.totalReward = G;
|
||||
|
||||
return totalDelta / trajectory.experiences.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision Transformer: Sequence modeling for trajectories
|
||||
*/
|
||||
private decisionTransformerUpdate(config: LearningConfig): number {
|
||||
const { learningRate: α, sequenceLength = 20 } = config;
|
||||
const trajectory = this.trajectories[this.trajectories.length - 1];
|
||||
if (!trajectory || trajectory.experiences.length === 0) return 0;
|
||||
|
||||
// Decision Transformer learns to predict actions given (return, state, action) sequences
|
||||
// Here we use a simplified version that learns state-action patterns
|
||||
|
||||
let totalDelta = 0;
|
||||
const experiences = trajectory.experiences.slice(-sequenceLength);
|
||||
|
||||
// Calculate returns-to-go
|
||||
const returns: number[] = [];
|
||||
let R = 0;
|
||||
for (let i = experiences.length - 1; i >= 0; i--) {
|
||||
R += experiences[i].reward;
|
||||
returns.unshift(R);
|
||||
}
|
||||
|
||||
// Update Q-values weighted by return-to-go
|
||||
for (let i = 0; i < experiences.length; i++) {
|
||||
const exp = experiences[i];
|
||||
const qTable = this.getQTable(exp.state);
|
||||
const currentQ = qTable.get(exp.action) || 0;
|
||||
|
||||
// Weight by normalized return
|
||||
const normalizedReturn = returns[i] / (Math.abs(returns[0]) + 1);
|
||||
const target = currentQ + α * normalizedReturn * exp.reward;
|
||||
const delta = target - currentQ;
|
||||
|
||||
qTable.set(exp.action, target);
|
||||
totalDelta += Math.abs(delta);
|
||||
}
|
||||
|
||||
trajectory.completed = true;
|
||||
trajectory.totalReward = returns[0];
|
||||
|
||||
return totalDelta / experiences.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* DQN: Deep Q-Network (simplified without actual neural network)
|
||||
* Uses experience replay and target network concepts
|
||||
*/
|
||||
private dqnUpdate(exp: Experience, config: LearningConfig): number {
|
||||
// Add to replay buffer (trajectory)
|
||||
this.addToCurrentTrajectory(exp);
|
||||
|
||||
// Sample from replay buffer
|
||||
const replayExp = this.sampleFromReplay();
|
||||
if (!replayExp) return this.qLearningUpdate(exp, config);
|
||||
|
||||
// Use sampled experience for update (breaks correlation)
|
||||
return this.qLearningUpdate(replayExp, config);
|
||||
}
|
||||
|
||||
// ============ Helper Methods ============
|
||||
|
||||
private getQTable(state: string): Map<string, number> {
|
||||
if (!this.qTables.has(state)) {
|
||||
this.qTables.set(state, new Map());
|
||||
}
|
||||
return this.qTables.get(state)!;
|
||||
}
|
||||
|
||||
private getQTable2(state: string): Map<string, number> {
|
||||
if (!this.qTables2.has(state)) {
|
||||
this.qTables2.set(state, new Map());
|
||||
}
|
||||
return this.qTables2.get(state)!;
|
||||
}
|
||||
|
||||
private getEligibilityTraces(state: string): Map<string, number> {
|
||||
if (!this.eligibilityTraces.has(state)) {
|
||||
this.eligibilityTraces.set(state, new Map());
|
||||
}
|
||||
return this.eligibilityTraces.get(state)!;
|
||||
}
|
||||
|
||||
private softmaxConfidence(values: number[], selectedIdx: number): number {
|
||||
if (values.length === 0) return 0.5;
|
||||
const maxVal = Math.max(...values);
|
||||
const expValues = values.map(v => Math.exp(v - maxVal));
|
||||
const sumExp = expValues.reduce((a, b) => a + b, 0);
|
||||
return expValues[selectedIdx] / sumExp;
|
||||
}
|
||||
|
||||
private addToCurrentTrajectory(exp: Experience): void {
|
||||
if (this.trajectories.length === 0 || this.trajectories[this.trajectories.length - 1].completed) {
|
||||
this.trajectories.push({
|
||||
experiences: [],
|
||||
totalReward: 0,
|
||||
completed: false,
|
||||
});
|
||||
}
|
||||
this.trajectories[this.trajectories.length - 1].experiences.push(exp);
|
||||
}
|
||||
|
||||
private sampleFromReplay(): Experience | null {
|
||||
const allExperiences: Experience[] = [];
|
||||
for (const traj of this.trajectories) {
|
||||
allExperiences.push(...traj.experiences);
|
||||
}
|
||||
if (allExperiences.length === 0) return null;
|
||||
return allExperiences[Math.floor(Math.random() * allExperiences.length)];
|
||||
}
|
||||
|
||||
private updateStats(algorithm: LearningAlgorithm, reward: number, delta: number): void {
|
||||
const stats = this.stats.get(algorithm);
|
||||
if (!stats) return;
|
||||
|
||||
stats.updates++;
|
||||
stats.lastUpdate = Date.now();
|
||||
|
||||
// Running average reward
|
||||
this.rewardHistory.push(reward);
|
||||
if (this.rewardHistory.length > 1000) {
|
||||
this.rewardHistory.shift();
|
||||
}
|
||||
stats.avgReward = this.rewardHistory.reduce((a, b) => a + b, 0) / this.rewardHistory.length;
|
||||
|
||||
// Convergence score (inverse of recent delta magnitude)
|
||||
stats.convergenceScore = 1 / (1 + delta);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get statistics for all algorithms
|
||||
*/
|
||||
getStats(): Map<LearningAlgorithm, AlgorithmStats> {
|
||||
return new Map(this.stats);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get statistics summary
|
||||
*/
|
||||
getStatsSummary(): {
|
||||
bestAlgorithm: LearningAlgorithm;
|
||||
totalUpdates: number;
|
||||
avgReward: number;
|
||||
algorithms: AlgorithmStats[];
|
||||
} {
|
||||
let bestAlgorithm: LearningAlgorithm = 'q-learning';
|
||||
let bestScore = -Infinity;
|
||||
let totalUpdates = 0;
|
||||
|
||||
const algorithms: AlgorithmStats[] = [];
|
||||
|
||||
for (const [alg, stats] of this.stats) {
|
||||
algorithms.push(stats);
|
||||
totalUpdates += stats.updates;
|
||||
|
||||
const score = stats.avgReward * stats.convergenceScore;
|
||||
if (score > bestScore && stats.updates > 0) {
|
||||
bestScore = score;
|
||||
bestAlgorithm = alg;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
bestAlgorithm,
|
||||
totalUpdates,
|
||||
avgReward: this.rewardHistory.length > 0
|
||||
? this.rewardHistory.reduce((a, b) => a + b, 0) / this.rewardHistory.length
|
||||
: 0,
|
||||
algorithms: algorithms.filter(a => a.updates > 0),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Export state for persistence
|
||||
*/
|
||||
export(): {
|
||||
qTables: Record<string, Record<string, number>>;
|
||||
qTables2: Record<string, Record<string, number>>;
|
||||
criticValues: Record<string, number>;
|
||||
trajectories: LearningTrajectory[];
|
||||
stats: Record<string, AlgorithmStats>;
|
||||
configs: Record<string, LearningConfig>;
|
||||
rewardHistory: number[];
|
||||
} {
|
||||
const qTables: Record<string, Record<string, number>> = {};
|
||||
for (const [state, actions] of this.qTables) {
|
||||
qTables[state] = Object.fromEntries(actions);
|
||||
}
|
||||
|
||||
const qTables2: Record<string, Record<string, number>> = {};
|
||||
for (const [state, actions] of this.qTables2) {
|
||||
qTables2[state] = Object.fromEntries(actions);
|
||||
}
|
||||
|
||||
const criticValues = Object.fromEntries(this.criticValues);
|
||||
const stats: Record<string, AlgorithmStats> = {};
|
||||
for (const [alg, s] of this.stats) {
|
||||
stats[alg] = s;
|
||||
}
|
||||
|
||||
const configs: Record<string, LearningConfig> = {};
|
||||
for (const [task, config] of this.configs) {
|
||||
configs[task] = config;
|
||||
}
|
||||
|
||||
return {
|
||||
qTables,
|
||||
qTables2,
|
||||
criticValues,
|
||||
trajectories: this.trajectories.slice(-100), // Keep last 100 trajectories
|
||||
stats,
|
||||
configs,
|
||||
rewardHistory: this.rewardHistory.slice(-1000),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Import state from persistence
|
||||
*/
|
||||
import(data: ReturnType<LearningEngine['export']>): void {
|
||||
// Q-tables
|
||||
this.qTables.clear();
|
||||
for (const [state, actions] of Object.entries(data.qTables || {})) {
|
||||
this.qTables.set(state, new Map(Object.entries(actions)));
|
||||
}
|
||||
|
||||
this.qTables2.clear();
|
||||
for (const [state, actions] of Object.entries(data.qTables2 || {})) {
|
||||
this.qTables2.set(state, new Map(Object.entries(actions)));
|
||||
}
|
||||
|
||||
// Critic values
|
||||
this.criticValues = new Map(Object.entries(data.criticValues || {}));
|
||||
|
||||
// Trajectories
|
||||
this.trajectories = data.trajectories || [];
|
||||
|
||||
// Stats
|
||||
for (const [alg, s] of Object.entries(data.stats || {})) {
|
||||
this.stats.set(alg as LearningAlgorithm, s as AlgorithmStats);
|
||||
}
|
||||
|
||||
// Configs
|
||||
for (const [task, config] of Object.entries(data.configs || {})) {
|
||||
this.configs.set(task as TaskType, config as LearningConfig);
|
||||
}
|
||||
|
||||
// Reward history
|
||||
this.rewardHistory = data.rewardHistory || [];
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all learning data
|
||||
*/
|
||||
clear(): void {
|
||||
this.qTables.clear();
|
||||
this.qTables2.clear();
|
||||
this.eligibilityTraces.clear();
|
||||
this.actorWeights.clear();
|
||||
this.criticValues.clear();
|
||||
this.trajectories = [];
|
||||
this.rewardHistory = [];
|
||||
|
||||
// Reset stats
|
||||
for (const stats of this.stats.values()) {
|
||||
stats.updates = 0;
|
||||
stats.avgReward = 0;
|
||||
stats.convergenceScore = 0;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get available algorithms
|
||||
*/
|
||||
static getAlgorithms(): { algorithm: LearningAlgorithm; description: string; bestFor: string }[] {
|
||||
return [
|
||||
{ algorithm: 'q-learning', description: 'Simple off-policy learning', bestFor: 'General routing' },
|
||||
{ algorithm: 'sarsa', description: 'On-policy, conservative', bestFor: 'Error avoidance' },
|
||||
{ algorithm: 'double-q', description: 'Reduces overestimation', bestFor: 'Precise routing' },
|
||||
{ algorithm: 'actor-critic', description: 'Policy gradient + value', bestFor: 'Confidence scoring' },
|
||||
{ algorithm: 'ppo', description: 'Stable policy updates', bestFor: 'Preference learning' },
|
||||
{ algorithm: 'decision-transformer', description: 'Sequence modeling', bestFor: 'Trajectory patterns' },
|
||||
{ algorithm: 'monte-carlo', description: 'Full episode learning', bestFor: 'Unbiased estimates' },
|
||||
{ algorithm: 'td-lambda', description: 'Eligibility traces', bestFor: 'Credit assignment' },
|
||||
{ algorithm: 'dqn', description: 'Experience replay', bestFor: 'High-dim states' },
|
||||
];
|
||||
}
|
||||
}
|
||||
|
||||
export default LearningEngine;
|
||||
393
npm/packages/ruvector/src/core/neural-embeddings.d.ts
vendored
Normal file
393
npm/packages/ruvector/src/core/neural-embeddings.d.ts
vendored
Normal file
@@ -0,0 +1,393 @@
|
||||
/**
|
||||
* Neural Embedding System - Frontier Embedding Intelligence
|
||||
*
|
||||
* Implements late-2025 research concepts treating embeddings as:
|
||||
* 1. CONTROL SIGNALS - Semantic drift detection, reflex triggers
|
||||
* 2. MEMORY PHYSICS - Forgetting curves, interference, consolidation
|
||||
* 3. PROGRAM STATE - Agent state management via geometry
|
||||
* 4. COORDINATION PRIMITIVES - Multi-agent swarm alignment
|
||||
* 5. SAFETY MONITORS - Coherence detection, misalignment alerts
|
||||
* 6. NEURAL SUBSTRATE - Synthetic nervous system layer
|
||||
*
|
||||
* Based on:
|
||||
* - TinyTE (EMNLP 2025): Embedding-layer steering
|
||||
* - DoRA (ICML 2024): Magnitude-direction decomposition
|
||||
* - S-LoRA/Punica: Multi-adapter serving patterns
|
||||
* - MMTEB: Multilingual embedding benchmarks
|
||||
*/
|
||||
export declare const NEURAL_CONSTANTS: {
|
||||
readonly MAX_DRIFT_EVENTS: 1000;
|
||||
readonly MAX_HISTORY_SIZE: 500;
|
||||
readonly DEFAULT_DRIFT_THRESHOLD: 0.15;
|
||||
readonly DEFAULT_DRIFT_WINDOW_MS: 60000;
|
||||
readonly DRIFT_CRITICAL_MULTIPLIER: 2;
|
||||
readonly VELOCITY_WINDOW_SIZE: 10;
|
||||
readonly MAX_MEMORIES: 10000;
|
||||
readonly MAX_CONTENT_LENGTH: 10000;
|
||||
readonly MAX_ID_LENGTH: 256;
|
||||
readonly DEFAULT_MEMORY_DECAY_RATE: 0.01;
|
||||
readonly DEFAULT_INTERFERENCE_THRESHOLD: 0.8;
|
||||
readonly DEFAULT_CONSOLIDATION_RATE: 0.1;
|
||||
readonly MEMORY_FORGET_THRESHOLD: 0.01;
|
||||
readonly CONSOLIDATION_SCORE_THRESHOLD: 0.5;
|
||||
readonly MEMORY_CLEANUP_PERCENT: 0.1;
|
||||
readonly RECALL_STRENGTH_BOOST: 0.1;
|
||||
readonly MAX_TIME_JUMP_MINUTES: 1440;
|
||||
readonly MAX_AGENTS: 1000;
|
||||
readonly MAX_SPECIALTY_LENGTH: 100;
|
||||
readonly AGENT_TIMEOUT_MS: 3600000;
|
||||
readonly DEFAULT_AGENT_ENERGY: 1;
|
||||
readonly TRAJECTORY_DAMPING: 0.1;
|
||||
readonly MAX_TRAJECTORY_STEPS: 100;
|
||||
readonly MAX_CLUSTER_AGENTS: 500;
|
||||
readonly DEFAULT_CLUSTER_THRESHOLD: 0.7;
|
||||
readonly DEFAULT_WINDOW_SIZE: 100;
|
||||
readonly MIN_CALIBRATION_OBSERVATIONS: 10;
|
||||
readonly STABILITY_WINDOW_SIZE: 10;
|
||||
readonly ALIGNMENT_WINDOW_SIZE: 50;
|
||||
readonly RECENT_OBSERVATIONS_SIZE: 20;
|
||||
readonly DRIFT_WARNING_THRESHOLD: 0.3;
|
||||
readonly STABILITY_WARNING_THRESHOLD: 0.5;
|
||||
readonly ALIGNMENT_WARNING_THRESHOLD: 0.6;
|
||||
readonly COHERENCE_WARNING_THRESHOLD: 0.5;
|
||||
readonly EPSILON: 1e-8;
|
||||
readonly ZERO_VECTOR_THRESHOLD: 1e-10;
|
||||
readonly DEFAULT_DIMENSION: 384;
|
||||
readonly DEFAULT_REFLEX_LATENCY_MS: 10;
|
||||
};
|
||||
export type LogLevel = 'debug' | 'info' | 'warn' | 'error';
|
||||
export interface NeuralLogger {
|
||||
log(level: LogLevel, message: string, data?: Record<string, unknown>): void;
|
||||
}
|
||||
/** Default console logger */
|
||||
export declare const defaultLogger: NeuralLogger;
|
||||
/** Silent logger for suppressing output */
|
||||
export declare const silentLogger: NeuralLogger;
|
||||
export interface DriftEvent {
|
||||
readonly timestamp: number;
|
||||
readonly magnitude: number;
|
||||
readonly direction: Float32Array;
|
||||
readonly category: 'normal' | 'warning' | 'critical';
|
||||
readonly source?: string;
|
||||
}
|
||||
export interface NeuralMemoryEntry {
|
||||
readonly id: string;
|
||||
readonly embedding: Float32Array;
|
||||
readonly content: string;
|
||||
strength: number;
|
||||
lastAccess: number;
|
||||
accessCount: number;
|
||||
consolidationLevel: number;
|
||||
interference: number;
|
||||
}
|
||||
export interface AgentState {
|
||||
readonly id: string;
|
||||
position: Float32Array;
|
||||
velocity: Float32Array;
|
||||
attention: Float32Array;
|
||||
energy: number;
|
||||
mode: string;
|
||||
lastUpdate: number;
|
||||
}
|
||||
export interface CoherenceReport {
|
||||
readonly timestamp: number;
|
||||
readonly overallScore: number;
|
||||
readonly driftScore: number;
|
||||
readonly stabilityScore: number;
|
||||
readonly alignmentScore: number;
|
||||
readonly anomalies: ReadonlyArray<{
|
||||
readonly type: string;
|
||||
readonly severity: number;
|
||||
readonly description: string;
|
||||
}>;
|
||||
}
|
||||
export interface NeuralConfig {
|
||||
readonly dimension?: number;
|
||||
readonly driftThreshold?: number;
|
||||
readonly driftWindowMs?: number;
|
||||
readonly memoryDecayRate?: number;
|
||||
readonly interferenceThreshold?: number;
|
||||
readonly consolidationRate?: number;
|
||||
readonly reflexLatencyMs?: number;
|
||||
readonly logger?: NeuralLogger;
|
||||
}
|
||||
/**
|
||||
* Detects semantic drift and triggers reflexes based on embedding movement.
|
||||
* Instead of asking "what is similar", asks "how far did we move".
|
||||
*/
|
||||
export declare class SemanticDriftDetector {
|
||||
private baseline;
|
||||
private history;
|
||||
private driftEvents;
|
||||
private config;
|
||||
private logger;
|
||||
private reflexes;
|
||||
constructor(config?: NeuralConfig);
|
||||
/**
|
||||
* Set the baseline embedding (reference point)
|
||||
*/
|
||||
setBaseline(embedding: number[] | Float32Array): void;
|
||||
/**
|
||||
* Observe a new embedding and detect drift
|
||||
*/
|
||||
observe(embedding: number[] | Float32Array, source?: string): DriftEvent | null;
|
||||
/**
|
||||
* Calculate drift between two embeddings
|
||||
*/
|
||||
private calculateDrift;
|
||||
/**
|
||||
* Register a reflex callback for drift events
|
||||
*/
|
||||
registerReflex(name: string, callback: (event: DriftEvent) => void): void;
|
||||
/**
|
||||
* Trigger registered reflexes
|
||||
*/
|
||||
private triggerReflexes;
|
||||
/**
|
||||
* Get recent drift velocity (rate of change)
|
||||
*/
|
||||
getVelocity(): number;
|
||||
/**
|
||||
* Get drift statistics
|
||||
*/
|
||||
getStats(): {
|
||||
currentDrift: number;
|
||||
velocity: number;
|
||||
criticalEvents: number;
|
||||
warningEvents: number;
|
||||
historySize: number;
|
||||
};
|
||||
/**
|
||||
* Reset baseline to current position
|
||||
*/
|
||||
recenter(): void;
|
||||
}
|
||||
/**
|
||||
* Implements hippocampal-like memory dynamics in embedding space.
|
||||
* Memory strength decays, similar memories interfere, consolidation strengthens.
|
||||
*/
|
||||
export declare class MemoryPhysics {
|
||||
private memories;
|
||||
private config;
|
||||
private lastUpdate;
|
||||
private logger;
|
||||
constructor(config?: NeuralConfig);
|
||||
/**
|
||||
* Encode a new memory
|
||||
*/
|
||||
encode(id: string, embedding: number[] | Float32Array, content: string): NeuralMemoryEntry;
|
||||
/**
|
||||
* Recall memories similar to a query (strengthens accessed memories)
|
||||
*/
|
||||
recall(query: number[] | Float32Array, k?: number): NeuralMemoryEntry[];
|
||||
/**
|
||||
* Apply time-based decay to all memories
|
||||
*/
|
||||
private applyDecay;
|
||||
/**
|
||||
* Consolidate memories (like sleep consolidation)
|
||||
* Strengthens frequently accessed, weakly interfered memories
|
||||
*/
|
||||
consolidate(): {
|
||||
consolidated: number;
|
||||
forgotten: number;
|
||||
};
|
||||
/**
|
||||
* Get memory statistics
|
||||
*/
|
||||
getStats(): {
|
||||
totalMemories: number;
|
||||
avgStrength: number;
|
||||
avgConsolidation: number;
|
||||
avgInterference: number;
|
||||
};
|
||||
private cosineSimilarity;
|
||||
/**
|
||||
* Force cleanup of weak memories when limit reached
|
||||
*/
|
||||
private forceCleanup;
|
||||
}
|
||||
/**
|
||||
* Manages agent state as movement through embedding space.
|
||||
* Decisions become geometric - no explicit state machine.
|
||||
*/
|
||||
export declare class EmbeddingStateMachine {
|
||||
private agents;
|
||||
private modeRegions;
|
||||
private config;
|
||||
private logger;
|
||||
private lastCleanup;
|
||||
constructor(config?: NeuralConfig);
|
||||
/**
|
||||
* Create or update an agent
|
||||
*/
|
||||
updateAgent(id: string, embedding: number[] | Float32Array): AgentState;
|
||||
/**
|
||||
* Remove stale agents that haven't been updated recently
|
||||
*/
|
||||
private cleanupStaleAgents;
|
||||
/**
|
||||
* Manually remove an agent
|
||||
*/
|
||||
removeAgent(id: string): boolean;
|
||||
/**
|
||||
* Define a mode region in embedding space
|
||||
*/
|
||||
defineMode(name: string, centroid: number[] | Float32Array, radius?: number): void;
|
||||
/**
|
||||
* Determine which mode an agent is in based on position
|
||||
*/
|
||||
private determineMode;
|
||||
/**
|
||||
* Get agent trajectory prediction
|
||||
*/
|
||||
predictTrajectory(id: string, steps?: number): Float32Array[];
|
||||
/**
|
||||
* Apply attention to agent state
|
||||
*/
|
||||
attendTo(agentId: string, focusEmbedding: number[] | Float32Array): void;
|
||||
/**
|
||||
* Get all agents in a specific mode
|
||||
*/
|
||||
getAgentsInMode(mode: string): AgentState[];
|
||||
private euclideanDistance;
|
||||
}
|
||||
/**
|
||||
* Enables multi-agent coordination through shared embedding space.
|
||||
* Swarm behavior emerges from geometry, not protocol.
|
||||
*/
|
||||
export declare class SwarmCoordinator {
|
||||
private agents;
|
||||
private sharedContext;
|
||||
private config;
|
||||
private logger;
|
||||
constructor(config?: NeuralConfig);
|
||||
/**
|
||||
* Register an agent with the swarm
|
||||
*/
|
||||
register(id: string, embedding: number[] | Float32Array, specialty?: string): void;
|
||||
/**
|
||||
* Update agent position (from their work/observations)
|
||||
*/
|
||||
update(id: string, embedding: number[] | Float32Array): void;
|
||||
/**
|
||||
* Update shared context (centroid of all agents)
|
||||
*/
|
||||
private updateSharedContext;
|
||||
/**
|
||||
* Get coordination signal for an agent (how to align with swarm)
|
||||
*/
|
||||
getCoordinationSignal(id: string): Float32Array;
|
||||
/**
|
||||
* Find agents working on similar things (for collaboration)
|
||||
*/
|
||||
findCollaborators(id: string, k?: number): Array<{
|
||||
id: string;
|
||||
similarity: number;
|
||||
specialty: string;
|
||||
}>;
|
||||
/**
|
||||
* Detect emergent clusters (specialization)
|
||||
*/
|
||||
detectClusters(threshold?: number): Map<string, string[]>;
|
||||
/**
|
||||
* Get swarm coherence (how aligned are agents)
|
||||
*/
|
||||
getCoherence(): number;
|
||||
private cosineSimilarity;
|
||||
/**
|
||||
* Remove an agent from the swarm
|
||||
*/
|
||||
removeAgent(id: string): boolean;
|
||||
}
|
||||
/**
|
||||
* Monitors system coherence via embedding patterns.
|
||||
* Detects degradation, poisoning, misalignment before explicit failures.
|
||||
*/
|
||||
export declare class CoherenceMonitor {
|
||||
private history;
|
||||
private baselineDistribution;
|
||||
private config;
|
||||
private logger;
|
||||
constructor(config?: NeuralConfig & {
|
||||
windowSize?: number;
|
||||
});
|
||||
/**
|
||||
* Record an observation
|
||||
*/
|
||||
observe(embedding: number[] | Float32Array, source?: string): void;
|
||||
/**
|
||||
* Establish baseline distribution
|
||||
*/
|
||||
calibrate(): void;
|
||||
/**
|
||||
* Generate coherence report
|
||||
*/
|
||||
report(): CoherenceReport;
|
||||
private calculateDriftScore;
|
||||
private calculateStabilityScore;
|
||||
private calculateAlignmentScore;
|
||||
private cosineSimilarity;
|
||||
}
|
||||
/**
|
||||
* Unified neural embedding substrate combining all components.
|
||||
* Acts like a synthetic nervous system with reflexes, memory, and coordination.
|
||||
*/
|
||||
export declare class NeuralSubstrate {
|
||||
readonly drift: SemanticDriftDetector;
|
||||
readonly memory: MemoryPhysics;
|
||||
readonly state: EmbeddingStateMachine;
|
||||
readonly swarm: SwarmCoordinator;
|
||||
readonly coherence: CoherenceMonitor;
|
||||
private config;
|
||||
private logger;
|
||||
private reflexLatency;
|
||||
constructor(config?: NeuralConfig);
|
||||
/**
|
||||
* Process an embedding through the entire substrate
|
||||
*/
|
||||
process(embedding: number[] | Float32Array, options?: {
|
||||
agentId?: string;
|
||||
memoryId?: string;
|
||||
content?: string;
|
||||
source?: string;
|
||||
}): {
|
||||
drift: DriftEvent | null;
|
||||
memory: NeuralMemoryEntry | null;
|
||||
state: AgentState | null;
|
||||
};
|
||||
/**
|
||||
* Query the substrate
|
||||
*/
|
||||
query(embedding: number[] | Float32Array, k?: number): {
|
||||
memories: NeuralMemoryEntry[];
|
||||
collaborators: Array<{
|
||||
id: string;
|
||||
similarity: number;
|
||||
specialty: string;
|
||||
}>;
|
||||
coherence: CoherenceReport;
|
||||
};
|
||||
/**
|
||||
* Get overall system health
|
||||
*/
|
||||
health(): {
|
||||
driftStats: ReturnType<SemanticDriftDetector['getStats']>;
|
||||
memoryStats: ReturnType<MemoryPhysics['getStats']>;
|
||||
swarmCoherence: number;
|
||||
coherenceReport: CoherenceReport;
|
||||
};
|
||||
/**
|
||||
* Run consolidation (like "sleep")
|
||||
*/
|
||||
consolidate(): {
|
||||
consolidated: number;
|
||||
forgotten: number;
|
||||
};
|
||||
/**
|
||||
* Calibrate coherence baseline
|
||||
*/
|
||||
calibrate(): void;
|
||||
}
|
||||
export default NeuralSubstrate;
|
||||
//# sourceMappingURL=neural-embeddings.d.ts.map
|
||||
File diff suppressed because one or more lines are too long
1092
npm/packages/ruvector/src/core/neural-embeddings.js
Normal file
1092
npm/packages/ruvector/src/core/neural-embeddings.js
Normal file
File diff suppressed because it is too large
Load Diff
1
npm/packages/ruvector/src/core/neural-embeddings.js.map
Normal file
1
npm/packages/ruvector/src/core/neural-embeddings.js.map
Normal file
File diff suppressed because one or more lines are too long
1383
npm/packages/ruvector/src/core/neural-embeddings.ts
Normal file
1383
npm/packages/ruvector/src/core/neural-embeddings.ts
Normal file
File diff suppressed because it is too large
Load Diff
331
npm/packages/ruvector/src/core/neural-perf.d.ts
vendored
Normal file
331
npm/packages/ruvector/src/core/neural-perf.d.ts
vendored
Normal file
@@ -0,0 +1,331 @@
|
||||
/**
|
||||
* Neural Performance Optimizations
|
||||
*
|
||||
* High-performance utilities for neural embedding operations:
|
||||
* - O(1) LRU Cache with doubly-linked list + hash map
|
||||
* - Parallel batch processing
|
||||
* - Pre-allocated Float32Array buffer pools
|
||||
* - Tensor buffer reuse
|
||||
* - 8x loop unrolling for vector operations
|
||||
*/
|
||||
export declare const PERF_CONSTANTS: {
|
||||
readonly DEFAULT_CACHE_SIZE: 1000;
|
||||
readonly DEFAULT_BUFFER_POOL_SIZE: 64;
|
||||
readonly DEFAULT_BATCH_SIZE: 32;
|
||||
readonly MIN_PARALLEL_BATCH_SIZE: 8;
|
||||
readonly UNROLL_THRESHOLD: 32;
|
||||
};
|
||||
/**
|
||||
* High-performance LRU Cache with O(1) get, set, and eviction.
|
||||
* Uses doubly-linked list for ordering + hash map for O(1) lookup.
|
||||
*/
|
||||
export declare class LRUCache<K, V> {
|
||||
private capacity;
|
||||
private map;
|
||||
private head;
|
||||
private tail;
|
||||
private hits;
|
||||
private misses;
|
||||
constructor(capacity?: number);
|
||||
/**
|
||||
* Get value from cache - O(1)
|
||||
*/
|
||||
get(key: K): V | undefined;
|
||||
/**
|
||||
* Set value in cache - O(1)
|
||||
*/
|
||||
set(key: K, value: V): void;
|
||||
/**
|
||||
* Check if key exists - O(1)
|
||||
*/
|
||||
has(key: K): boolean;
|
||||
/**
|
||||
* Delete key from cache - O(1)
|
||||
*/
|
||||
delete(key: K): boolean;
|
||||
/**
|
||||
* Clear entire cache - O(1)
|
||||
*/
|
||||
clear(): void;
|
||||
/**
|
||||
* Get cache size
|
||||
*/
|
||||
get size(): number;
|
||||
/**
|
||||
* Get cache statistics
|
||||
*/
|
||||
getStats(): {
|
||||
size: number;
|
||||
capacity: number;
|
||||
hits: number;
|
||||
misses: number;
|
||||
hitRate: number;
|
||||
};
|
||||
/**
|
||||
* Reset statistics
|
||||
*/
|
||||
resetStats(): void;
|
||||
private moveToHead;
|
||||
private addToHead;
|
||||
private removeNode;
|
||||
private evictLRU;
|
||||
/**
|
||||
* Iterate over entries (most recent first)
|
||||
*/
|
||||
entries(): Generator<[K, V]>;
|
||||
}
|
||||
/**
|
||||
* High-performance buffer pool for Float32Arrays.
|
||||
* Eliminates GC pressure by reusing pre-allocated buffers.
|
||||
*/
|
||||
export declare class Float32BufferPool {
|
||||
private pools;
|
||||
private maxPoolSize;
|
||||
private allocations;
|
||||
private reuses;
|
||||
constructor(maxPoolSize?: number);
|
||||
/**
|
||||
* Acquire a buffer of specified size - O(1) amortized
|
||||
*/
|
||||
acquire(size: number): Float32Array;
|
||||
/**
|
||||
* Release a buffer back to the pool - O(1)
|
||||
*/
|
||||
release(buffer: Float32Array): void;
|
||||
/**
|
||||
* Pre-warm the pool with buffers of specific sizes
|
||||
*/
|
||||
prewarm(sizes: number[], count?: number): void;
|
||||
/**
|
||||
* Clear all pools
|
||||
*/
|
||||
clear(): void;
|
||||
/**
|
||||
* Get pool statistics
|
||||
*/
|
||||
getStats(): {
|
||||
allocations: number;
|
||||
reuses: number;
|
||||
reuseRate: number;
|
||||
pooledBuffers: number;
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Manages reusable tensor buffers for intermediate computations.
|
||||
* Reduces allocations in hot paths.
|
||||
*/
|
||||
export declare class TensorBufferManager {
|
||||
private bufferPool;
|
||||
private workingBuffers;
|
||||
constructor(pool?: Float32BufferPool);
|
||||
/**
|
||||
* Get or create a named working buffer
|
||||
*/
|
||||
getWorking(name: string, size: number): Float32Array;
|
||||
/**
|
||||
* Get a temporary buffer (caller must release)
|
||||
*/
|
||||
getTemp(size: number): Float32Array;
|
||||
/**
|
||||
* Release a temporary buffer
|
||||
*/
|
||||
releaseTemp(buffer: Float32Array): void;
|
||||
/**
|
||||
* Release all working buffers
|
||||
*/
|
||||
releaseAll(): void;
|
||||
/**
|
||||
* Get underlying pool for stats
|
||||
*/
|
||||
getPool(): Float32BufferPool;
|
||||
}
|
||||
/**
|
||||
* High-performance vector operations with 8x loop unrolling.
|
||||
* Provides 15-30% speedup on large vectors.
|
||||
*/
|
||||
export declare const VectorOps: {
|
||||
/**
|
||||
* Dot product with 8x unrolling
|
||||
*/
|
||||
dot(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Squared L2 norm with 8x unrolling
|
||||
*/
|
||||
normSq(a: Float32Array): number;
|
||||
/**
|
||||
* L2 norm
|
||||
*/
|
||||
norm(a: Float32Array): number;
|
||||
/**
|
||||
* Cosine similarity - optimized for V8 JIT
|
||||
* Uses 4x unrolling which benchmarks faster than 8x due to register pressure
|
||||
*/
|
||||
cosine(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Euclidean distance squared with 8x unrolling
|
||||
*/
|
||||
distanceSq(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Euclidean distance
|
||||
*/
|
||||
distance(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Add vectors: out = a + b (with 8x unrolling)
|
||||
*/
|
||||
add(a: Float32Array, b: Float32Array, out: Float32Array): Float32Array;
|
||||
/**
|
||||
* Subtract vectors: out = a - b (with 8x unrolling)
|
||||
*/
|
||||
sub(a: Float32Array, b: Float32Array, out: Float32Array): Float32Array;
|
||||
/**
|
||||
* Scale vector: out = a * scalar (with 8x unrolling)
|
||||
*/
|
||||
scale(a: Float32Array, scalar: number, out: Float32Array): Float32Array;
|
||||
/**
|
||||
* Normalize vector in-place
|
||||
*/
|
||||
normalize(a: Float32Array): Float32Array;
|
||||
/**
|
||||
* Mean of multiple vectors (with buffer reuse)
|
||||
*/
|
||||
mean(vectors: Float32Array[], out: Float32Array): Float32Array;
|
||||
};
|
||||
export interface BatchResult<T> {
|
||||
results: T[];
|
||||
timing: {
|
||||
totalMs: number;
|
||||
perItemMs: number;
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Parallel batch processor for embedding operations.
|
||||
* Uses chunking and Promise.all for concurrent processing.
|
||||
*/
|
||||
export declare class ParallelBatchProcessor {
|
||||
private batchSize;
|
||||
private maxConcurrency;
|
||||
constructor(options?: {
|
||||
batchSize?: number;
|
||||
maxConcurrency?: number;
|
||||
});
|
||||
/**
|
||||
* Process items in parallel batches
|
||||
*/
|
||||
processBatch<T, R>(items: T[], processor: (item: T, index: number) => Promise<R> | R): Promise<BatchResult<R>>;
|
||||
/**
|
||||
* Process with synchronous function (uses chunking for better cache locality)
|
||||
*/
|
||||
processSync<T, R>(items: T[], processor: (item: T, index: number) => R): BatchResult<R>;
|
||||
/**
|
||||
* Batch similarity search (optimized for many queries)
|
||||
*/
|
||||
batchSimilarity(queries: Float32Array[], corpus: Float32Array[], k?: number): Array<Array<{
|
||||
index: number;
|
||||
score: number;
|
||||
}>>;
|
||||
private chunkArray;
|
||||
}
|
||||
export interface CachedMemoryEntry {
|
||||
id: string;
|
||||
embedding: Float32Array;
|
||||
content: string;
|
||||
score: number;
|
||||
}
|
||||
/**
|
||||
* High-performance memory store with O(1) LRU caching.
|
||||
*/
|
||||
export declare class OptimizedMemoryStore {
|
||||
private cache;
|
||||
private bufferPool;
|
||||
private dimension;
|
||||
constructor(options?: {
|
||||
cacheSize?: number;
|
||||
dimension?: number;
|
||||
});
|
||||
/**
|
||||
* Store embedding - O(1)
|
||||
*/
|
||||
store(id: string, embedding: Float32Array | number[], content: string): void;
|
||||
/**
|
||||
* Get by ID - O(1)
|
||||
*/
|
||||
get(id: string): CachedMemoryEntry | undefined;
|
||||
/**
|
||||
* Search by similarity - O(n) but with optimized vector ops
|
||||
*/
|
||||
search(query: Float32Array, k?: number): CachedMemoryEntry[];
|
||||
/**
|
||||
* Delete entry - O(1)
|
||||
*/
|
||||
delete(id: string): boolean;
|
||||
/**
|
||||
* Get statistics
|
||||
*/
|
||||
getStats(): {
|
||||
cache: ReturnType<LRUCache<string, CachedMemoryEntry>['getStats']>;
|
||||
buffers: ReturnType<Float32BufferPool['getStats']>;
|
||||
};
|
||||
}
|
||||
declare const _default: {
|
||||
LRUCache: typeof LRUCache;
|
||||
Float32BufferPool: typeof Float32BufferPool;
|
||||
TensorBufferManager: typeof TensorBufferManager;
|
||||
VectorOps: {
|
||||
/**
|
||||
* Dot product with 8x unrolling
|
||||
*/
|
||||
dot(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Squared L2 norm with 8x unrolling
|
||||
*/
|
||||
normSq(a: Float32Array): number;
|
||||
/**
|
||||
* L2 norm
|
||||
*/
|
||||
norm(a: Float32Array): number;
|
||||
/**
|
||||
* Cosine similarity - optimized for V8 JIT
|
||||
* Uses 4x unrolling which benchmarks faster than 8x due to register pressure
|
||||
*/
|
||||
cosine(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Euclidean distance squared with 8x unrolling
|
||||
*/
|
||||
distanceSq(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Euclidean distance
|
||||
*/
|
||||
distance(a: Float32Array, b: Float32Array): number;
|
||||
/**
|
||||
* Add vectors: out = a + b (with 8x unrolling)
|
||||
*/
|
||||
add(a: Float32Array, b: Float32Array, out: Float32Array): Float32Array;
|
||||
/**
|
||||
* Subtract vectors: out = a - b (with 8x unrolling)
|
||||
*/
|
||||
sub(a: Float32Array, b: Float32Array, out: Float32Array): Float32Array;
|
||||
/**
|
||||
* Scale vector: out = a * scalar (with 8x unrolling)
|
||||
*/
|
||||
scale(a: Float32Array, scalar: number, out: Float32Array): Float32Array;
|
||||
/**
|
||||
* Normalize vector in-place
|
||||
*/
|
||||
normalize(a: Float32Array): Float32Array;
|
||||
/**
|
||||
* Mean of multiple vectors (with buffer reuse)
|
||||
*/
|
||||
mean(vectors: Float32Array[], out: Float32Array): Float32Array;
|
||||
};
|
||||
ParallelBatchProcessor: typeof ParallelBatchProcessor;
|
||||
OptimizedMemoryStore: typeof OptimizedMemoryStore;
|
||||
PERF_CONSTANTS: {
|
||||
readonly DEFAULT_CACHE_SIZE: 1000;
|
||||
readonly DEFAULT_BUFFER_POOL_SIZE: 64;
|
||||
readonly DEFAULT_BATCH_SIZE: 32;
|
||||
readonly MIN_PARALLEL_BATCH_SIZE: 8;
|
||||
readonly UNROLL_THRESHOLD: 32;
|
||||
};
|
||||
};
|
||||
export default _default;
|
||||
//# sourceMappingURL=neural-perf.d.ts.map
|
||||
1
npm/packages/ruvector/src/core/neural-perf.d.ts.map
Normal file
1
npm/packages/ruvector/src/core/neural-perf.d.ts.map
Normal file
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"neural-perf.d.ts","sourceRoot":"","sources":["neural-perf.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAMH,eAAO,MAAM,cAAc;;;;;;CAMjB,CAAC;AAaX;;;GAGG;AACH,qBAAa,QAAQ,CAAC,CAAC,EAAE,CAAC;IACxB,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,GAAG,CAAoC;IAC/C,OAAO,CAAC,IAAI,CAA8B;IAC1C,OAAO,CAAC,IAAI,CAA8B;IAG1C,OAAO,CAAC,IAAI,CAAa;IACzB,OAAO,CAAC,MAAM,CAAa;gBAEf,QAAQ,GAAE,MAA0C;IAKhE;;OAEG;IACH,GAAG,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,GAAG,SAAS;IAa1B;;OAEG;IACH,GAAG,CAAC,GAAG,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,GAAG,IAAI;IAuB3B;;OAEG;IACH,GAAG,CAAC,GAAG,EAAE,CAAC,GAAG,OAAO;IAIpB;;OAEG;IACH,MAAM,CAAC,GAAG,EAAE,CAAC,GAAG,OAAO;IASvB;;OAEG;IACH,KAAK,IAAI,IAAI;IAMb;;OAEG;IACH,IAAI,IAAI,IAAI,MAAM,CAEjB;IAED;;OAEG;IACH,QAAQ,IAAI;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE;IAW7F;;OAEG;IACH,UAAU,IAAI,IAAI;IAMlB,OAAO,CAAC,UAAU;IAOlB,OAAO,CAAC,SAAS;IAejB,OAAO,CAAC,UAAU;IAelB,OAAO,CAAC,QAAQ;IAMhB;;OAEG;IACF,OAAO,IAAI,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;CAO9B;AAMD;;;GAGG;AACH,qBAAa,iBAAiB;IAC5B,OAAO,CAAC,KAAK,CAA0C;IACvD,OAAO,CAAC,WAAW,CAAS;IAG5B,OAAO,CAAC,WAAW,CAAa;IAChC,OAAO,CAAC,MAAM,CAAa;gBAEf,WAAW,GAAE,MAAgD;IAIzE;;OAEG;IACH,OAAO,CAAC,IAAI,EAAE,MAAM,GAAG,YAAY;IAYnC;;OAEG;IACH,OAAO,CAAC,MAAM,EAAE,YAAY,GAAG,IAAI;IAiBnC;;OAEG;IACH,OAAO,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,KAAK,GAAE,MAAU,GAAG,IAAI;IAcjD;;OAEG;IACH,KAAK,IAAI,IAAI;IAIb;;OAEG;IACH,QAAQ,IAAI;QAAE,WAAW,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,CAAC;QAAC,aAAa,EAAE,MAAM,CAAA;KAAE;CAc9F;AAMD;;;GAGG;AACH,qBAAa,mBAAmB;IAC9B,OAAO,CAAC,UAAU,CAAoB;IACtC,OAAO,CAAC,cAAc,CAAwC;gBAElD,IAAI,CAAC,EAAE,iBAAiB;IAIpC;;OAEG;IACH,UAAU,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,YAAY;IAiBpD;;OAEG;IACH,OAAO,CAAC,IAAI,EAAE,MAAM,GAAG,YAAY;IAInC;;OAEG;IACH,WAAW,CAAC,MAAM,EAAE,YAAY,GAAG,IAAI;IAIvC;;OAEG;IACH,UAAU,IAAI,IAAI;IAOlB;;OAEG;IACH,OAAO,IAAI,iBAAiB;CAG7B;AAMD;;;GAGG;AACH,eAAO,MAAM,SAAS;IACpB;;OAEG;WACI,YAAY,KAAK,YAAY,GAAG,MAAM;IA2B7C;;OAEG;cACO,YAAY,GAAG,MAAM;IAyB/B;;OAEG;YACK,YAAY,GAAG,MAAM;IAI7B;;;OAGG;cACO,YAAY,KAAK,YAAY,GAAG,MAAM;IA2BhD;;OAEG;kBACW,YAAY,KAAK,YAAY,GAAG,MAAM;IA6BpD;;OAEG;gBACS,YAAY,KAAK,YAAY,GAAG,MAAM;IAIlD;;OAEG;WACI,YAAY,KAAK,YAAY,OAAO,YAAY,GAAG,YAAY;IAuBtE;;OAEG;WACI,YAAY,KAAK,YAAY,OAAO,YAAY,GAAG,YAAY;IAuBtE;;OAEG;aACM,YAAY,UAAU,MAAM,OAAO,YAAY,GAAG,YAAY;IAuBvE;;OAEG;iBACU,YAAY,GAAG,YAAY;IAQxC;;OAEG;kBACW,YAAY,EAAE,OAAO,YAAY,GAAG,YAAY;CAoB/D,CAAC;AAMF,MAAM,WAAW,WAAW,CAAC,CAAC;IAC5B,OAAO,EAAE,CAAC,EAAE,CAAC;IACb,MAAM,EAAE;QACN,OAAO,EAAE,MAAM,CAAC;QAChB,SAAS,EAAE,MAAM,CAAC;KACnB,CAAC;CACH;AAED;;;GAGG;AACH,qBAAa,sBAAsB;IACjC,OAAO,CAAC,SAAS,CAAS;IAC1B,OAAO,CAAC,cAAc,CAAS;gBAEnB,OAAO,GAAE;QAAE,SAAS,CAAC,EAAE,MAAM,CAAC;QAAC,cAAc,CAAC,EAAE,MAAM,CAAA;KAAO;IAKzE;;OAEG;IACG,YAAY,CAAC,CAAC,EAAE,CAAC,EACrB,KAAK,EAAE,CAAC,EAAE,EACV,SAAS,EAAE,CAAC,IAAI,EAAE,CAAC,EAAE,KAAK,EAAE,MAAM,KAAK,OAAO,CAAC,CAAC,CAAC,GAAG,CAAC,GACpD,OAAO,CAAC,WAAW,CAAC,CAAC,CAAC,CAAC;IAgC1B;;OAEG;IACH,WAAW,CAAC,CAAC,EAAE,CAAC,EACd,KAAK,EAAE,CAAC,EAAE,EACV,SAAS,EAAE,CAAC,IAAI,EAAE,CAAC,EAAE,KAAK,EAAE,MAAM,KAAK,CAAC,GACvC,WAAW,CAAC,CAAC,CAAC;IAsBjB;;OAEG;IACH,eAAe,CACb,OAAO,EAAE,YAAY,EAAE,EACvB,MAAM,EAAE,YAAY,EAAE,EACtB,CAAC,GAAE,MAAU,GACZ,KAAK,CAAC,KAAK,CAAC;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;IAqBjD,OAAO,CAAC,UAAU;CAOnB;AAMD,MAAM,WAAW,iBAAiB;IAChC,EAAE,EAAE,MAAM,CAAC;IACX,SAAS,EAAE,YAAY,CAAC;IACxB,OAAO,EAAE,MAAM,CAAC;IAChB,KAAK,EAAE,MAAM,CAAC;CACf;AAED;;GAEG;AACH,qBAAa,oBAAoB;IAC/B,OAAO,CAAC,KAAK,CAAsC;IACnD,OAAO,CAAC,UAAU,CAAoB;IACtC,OAAO,CAAC,SAAS,CAAS;gBAEd,OAAO,GAAE;QACnB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,SAAS,CAAC,EAAE,MAAM,CAAC;KACf;IASN;;OAEG;IACH,KAAK,CAAC,EAAE,EAAE,MAAM,EAAE,SAAS,EAAE,YAAY,GAAG,MAAM,EAAE,EAAE,OAAO,EAAE,MAAM,GAAG,IAAI;IAgB5E;;OAEG;IACH,GAAG,CAAC,EAAE,EAAE,MAAM,GAAG,iBAAiB,GAAG,SAAS;IAI9C;;OAEG;IACH,MAAM,CAAC,KAAK,EAAE,YAAY,EAAE,CAAC,GAAE,MAAU,GAAG,iBAAiB,EAAE;IAY/D;;OAEG;IACH,MAAM,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO;IAQ3B;;OAEG;IACH,QAAQ,IAAI;QACV,KAAK,EAAE,UAAU,CAAC,QAAQ,CAAC,MAAM,EAAE,iBAAiB,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC;QACnE,OAAO,EAAE,UAAU,CAAC,iBAAiB,CAAC,UAAU,CAAC,CAAC,CAAC;KACpD;CAMF;;;;;;QAxdC;;WAEG;eACI,YAAY,KAAK,YAAY,GAAG,MAAM;QA2B7C;;WAEG;kBACO,YAAY,GAAG,MAAM;QAyB/B;;WAEG;gBACK,YAAY,GAAG,MAAM;QAI7B;;;WAGG;kBACO,YAAY,KAAK,YAAY,GAAG,MAAM;QA2BhD;;WAEG;sBACW,YAAY,KAAK,YAAY,GAAG,MAAM;QA6BpD;;WAEG;oBACS,YAAY,KAAK,YAAY,GAAG,MAAM;QAIlD;;WAEG;eACI,YAAY,KAAK,YAAY,OAAO,YAAY,GAAG,YAAY;QAuBtE;;WAEG;eACI,YAAY,KAAK,YAAY,OAAO,YAAY,GAAG,YAAY;QAuBtE;;WAEG;iBACM,YAAY,UAAU,MAAM,OAAO,YAAY,GAAG,YAAY;QAuBvE;;WAEG;qBACU,YAAY,GAAG,YAAY;QAQxC;;WAEG;sBACW,YAAY,EAAE,OAAO,YAAY,GAAG,YAAY;;;;;;;;;;;;AA2PhE,wBAQE"}
|
||||
705
npm/packages/ruvector/src/core/neural-perf.js
Normal file
705
npm/packages/ruvector/src/core/neural-perf.js
Normal file
@@ -0,0 +1,705 @@
|
||||
"use strict";
|
||||
/**
|
||||
* Neural Performance Optimizations
|
||||
*
|
||||
* High-performance utilities for neural embedding operations:
|
||||
* - O(1) LRU Cache with doubly-linked list + hash map
|
||||
* - Parallel batch processing
|
||||
* - Pre-allocated Float32Array buffer pools
|
||||
* - Tensor buffer reuse
|
||||
* - 8x loop unrolling for vector operations
|
||||
*/
|
||||
Object.defineProperty(exports, "__esModule", { value: true });
|
||||
exports.OptimizedMemoryStore = exports.ParallelBatchProcessor = exports.VectorOps = exports.TensorBufferManager = exports.Float32BufferPool = exports.LRUCache = exports.PERF_CONSTANTS = void 0;
|
||||
// ============================================================================
|
||||
// Constants
|
||||
// ============================================================================
|
||||
exports.PERF_CONSTANTS = {
|
||||
DEFAULT_CACHE_SIZE: 1000,
|
||||
DEFAULT_BUFFER_POOL_SIZE: 64,
|
||||
DEFAULT_BATCH_SIZE: 32,
|
||||
MIN_PARALLEL_BATCH_SIZE: 8,
|
||||
UNROLL_THRESHOLD: 32, // Min dimension for loop unrolling
|
||||
};
|
||||
/**
|
||||
* High-performance LRU Cache with O(1) get, set, and eviction.
|
||||
* Uses doubly-linked list for ordering + hash map for O(1) lookup.
|
||||
*/
|
||||
class LRUCache {
|
||||
constructor(capacity = exports.PERF_CONSTANTS.DEFAULT_CACHE_SIZE) {
|
||||
this.map = new Map();
|
||||
this.head = null; // Most recently used
|
||||
this.tail = null; // Least recently used
|
||||
// Stats
|
||||
this.hits = 0;
|
||||
this.misses = 0;
|
||||
if (capacity < 1)
|
||||
throw new Error('Cache capacity must be >= 1');
|
||||
this.capacity = capacity;
|
||||
}
|
||||
/**
|
||||
* Get value from cache - O(1)
|
||||
*/
|
||||
get(key) {
|
||||
const node = this.map.get(key);
|
||||
if (!node) {
|
||||
this.misses++;
|
||||
return undefined;
|
||||
}
|
||||
this.hits++;
|
||||
// Move to head (most recently used)
|
||||
this.moveToHead(node);
|
||||
return node.value;
|
||||
}
|
||||
/**
|
||||
* Set value in cache - O(1)
|
||||
*/
|
||||
set(key, value) {
|
||||
const existing = this.map.get(key);
|
||||
if (existing) {
|
||||
// Update existing node
|
||||
existing.value = value;
|
||||
this.moveToHead(existing);
|
||||
return;
|
||||
}
|
||||
// Create new node
|
||||
const node = { key, value, prev: null, next: null };
|
||||
// Evict if at capacity
|
||||
if (this.map.size >= this.capacity) {
|
||||
this.evictLRU();
|
||||
}
|
||||
// Add to map and list
|
||||
this.map.set(key, node);
|
||||
this.addToHead(node);
|
||||
}
|
||||
/**
|
||||
* Check if key exists - O(1)
|
||||
*/
|
||||
has(key) {
|
||||
return this.map.has(key);
|
||||
}
|
||||
/**
|
||||
* Delete key from cache - O(1)
|
||||
*/
|
||||
delete(key) {
|
||||
const node = this.map.get(key);
|
||||
if (!node)
|
||||
return false;
|
||||
this.removeNode(node);
|
||||
this.map.delete(key);
|
||||
return true;
|
||||
}
|
||||
/**
|
||||
* Clear entire cache - O(1)
|
||||
*/
|
||||
clear() {
|
||||
this.map.clear();
|
||||
this.head = null;
|
||||
this.tail = null;
|
||||
}
|
||||
/**
|
||||
* Get cache size
|
||||
*/
|
||||
get size() {
|
||||
return this.map.size;
|
||||
}
|
||||
/**
|
||||
* Get cache statistics
|
||||
*/
|
||||
getStats() {
|
||||
const total = this.hits + this.misses;
|
||||
return {
|
||||
size: this.map.size,
|
||||
capacity: this.capacity,
|
||||
hits: this.hits,
|
||||
misses: this.misses,
|
||||
hitRate: total > 0 ? this.hits / total : 0,
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Reset statistics
|
||||
*/
|
||||
resetStats() {
|
||||
this.hits = 0;
|
||||
this.misses = 0;
|
||||
}
|
||||
// Internal: Move existing node to head
|
||||
moveToHead(node) {
|
||||
if (node === this.head)
|
||||
return;
|
||||
this.removeNode(node);
|
||||
this.addToHead(node);
|
||||
}
|
||||
// Internal: Add new node to head
|
||||
addToHead(node) {
|
||||
node.prev = null;
|
||||
node.next = this.head;
|
||||
if (this.head) {
|
||||
this.head.prev = node;
|
||||
}
|
||||
this.head = node;
|
||||
if (!this.tail) {
|
||||
this.tail = node;
|
||||
}
|
||||
}
|
||||
// Internal: Remove node from list
|
||||
removeNode(node) {
|
||||
if (node.prev) {
|
||||
node.prev.next = node.next;
|
||||
}
|
||||
else {
|
||||
this.head = node.next;
|
||||
}
|
||||
if (node.next) {
|
||||
node.next.prev = node.prev;
|
||||
}
|
||||
else {
|
||||
this.tail = node.prev;
|
||||
}
|
||||
}
|
||||
// Internal: Evict least recently used (tail)
|
||||
evictLRU() {
|
||||
if (!this.tail)
|
||||
return;
|
||||
this.map.delete(this.tail.key);
|
||||
this.removeNode(this.tail);
|
||||
}
|
||||
/**
|
||||
* Iterate over entries (most recent first)
|
||||
*/
|
||||
*entries() {
|
||||
let current = this.head;
|
||||
while (current) {
|
||||
yield [current.key, current.value];
|
||||
current = current.next;
|
||||
}
|
||||
}
|
||||
}
|
||||
exports.LRUCache = LRUCache;
|
||||
// ============================================================================
|
||||
// P1: Pre-allocated Float32Array Buffer Pool
|
||||
// ============================================================================
|
||||
/**
|
||||
* High-performance buffer pool for Float32Arrays.
|
||||
* Eliminates GC pressure by reusing pre-allocated buffers.
|
||||
*/
|
||||
class Float32BufferPool {
|
||||
constructor(maxPoolSize = exports.PERF_CONSTANTS.DEFAULT_BUFFER_POOL_SIZE) {
|
||||
this.pools = new Map();
|
||||
// Stats
|
||||
this.allocations = 0;
|
||||
this.reuses = 0;
|
||||
this.maxPoolSize = maxPoolSize;
|
||||
}
|
||||
/**
|
||||
* Acquire a buffer of specified size - O(1) amortized
|
||||
*/
|
||||
acquire(size) {
|
||||
const pool = this.pools.get(size);
|
||||
if (pool && pool.length > 0) {
|
||||
this.reuses++;
|
||||
return pool.pop();
|
||||
}
|
||||
this.allocations++;
|
||||
return new Float32Array(size);
|
||||
}
|
||||
/**
|
||||
* Release a buffer back to the pool - O(1)
|
||||
*/
|
||||
release(buffer) {
|
||||
const size = buffer.length;
|
||||
let pool = this.pools.get(size);
|
||||
if (!pool) {
|
||||
pool = [];
|
||||
this.pools.set(size, pool);
|
||||
}
|
||||
// Only keep up to maxPoolSize buffers per size
|
||||
if (pool.length < this.maxPoolSize) {
|
||||
// Zero out for security
|
||||
buffer.fill(0);
|
||||
pool.push(buffer);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Pre-warm the pool with buffers of specific sizes
|
||||
*/
|
||||
prewarm(sizes, count = 8) {
|
||||
for (const size of sizes) {
|
||||
let pool = this.pools.get(size);
|
||||
if (!pool) {
|
||||
pool = [];
|
||||
this.pools.set(size, pool);
|
||||
}
|
||||
while (pool.length < count) {
|
||||
pool.push(new Float32Array(size));
|
||||
this.allocations++;
|
||||
}
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Clear all pools
|
||||
*/
|
||||
clear() {
|
||||
this.pools.clear();
|
||||
}
|
||||
/**
|
||||
* Get pool statistics
|
||||
*/
|
||||
getStats() {
|
||||
let pooledBuffers = 0;
|
||||
for (const pool of this.pools.values()) {
|
||||
pooledBuffers += pool.length;
|
||||
}
|
||||
const total = this.allocations + this.reuses;
|
||||
return {
|
||||
allocations: this.allocations,
|
||||
reuses: this.reuses,
|
||||
reuseRate: total > 0 ? this.reuses / total : 0,
|
||||
pooledBuffers,
|
||||
};
|
||||
}
|
||||
}
|
||||
exports.Float32BufferPool = Float32BufferPool;
|
||||
// ============================================================================
|
||||
// P1: Tensor Buffer Manager (Reusable Working Memory)
|
||||
// ============================================================================
|
||||
/**
|
||||
* Manages reusable tensor buffers for intermediate computations.
|
||||
* Reduces allocations in hot paths.
|
||||
*/
|
||||
class TensorBufferManager {
|
||||
constructor(pool) {
|
||||
this.workingBuffers = new Map();
|
||||
this.bufferPool = pool ?? new Float32BufferPool();
|
||||
}
|
||||
/**
|
||||
* Get or create a named working buffer
|
||||
*/
|
||||
getWorking(name, size) {
|
||||
const existing = this.workingBuffers.get(name);
|
||||
if (existing && existing.length === size) {
|
||||
return existing;
|
||||
}
|
||||
// Release old buffer if size changed
|
||||
if (existing) {
|
||||
this.bufferPool.release(existing);
|
||||
}
|
||||
const buffer = this.bufferPool.acquire(size);
|
||||
this.workingBuffers.set(name, buffer);
|
||||
return buffer;
|
||||
}
|
||||
/**
|
||||
* Get a temporary buffer (caller must release)
|
||||
*/
|
||||
getTemp(size) {
|
||||
return this.bufferPool.acquire(size);
|
||||
}
|
||||
/**
|
||||
* Release a temporary buffer
|
||||
*/
|
||||
releaseTemp(buffer) {
|
||||
this.bufferPool.release(buffer);
|
||||
}
|
||||
/**
|
||||
* Release all working buffers
|
||||
*/
|
||||
releaseAll() {
|
||||
for (const buffer of this.workingBuffers.values()) {
|
||||
this.bufferPool.release(buffer);
|
||||
}
|
||||
this.workingBuffers.clear();
|
||||
}
|
||||
/**
|
||||
* Get underlying pool for stats
|
||||
*/
|
||||
getPool() {
|
||||
return this.bufferPool;
|
||||
}
|
||||
}
|
||||
exports.TensorBufferManager = TensorBufferManager;
|
||||
// ============================================================================
|
||||
// P2: 8x Loop Unrolling Vector Operations
|
||||
// ============================================================================
|
||||
/**
|
||||
* High-performance vector operations with 8x loop unrolling.
|
||||
* Provides 15-30% speedup on large vectors.
|
||||
*/
|
||||
exports.VectorOps = {
|
||||
/**
|
||||
* Dot product with 8x unrolling
|
||||
*/
|
||||
dot(a, b) {
|
||||
const len = a.length;
|
||||
let sum = 0;
|
||||
// 8x unrolled loop
|
||||
const unrolled = len - (len % 8);
|
||||
let i = 0;
|
||||
for (; i < unrolled; i += 8) {
|
||||
sum += a[i] * b[i]
|
||||
+ a[i + 1] * b[i + 1]
|
||||
+ a[i + 2] * b[i + 2]
|
||||
+ a[i + 3] * b[i + 3]
|
||||
+ a[i + 4] * b[i + 4]
|
||||
+ a[i + 5] * b[i + 5]
|
||||
+ a[i + 6] * b[i + 6]
|
||||
+ a[i + 7] * b[i + 7];
|
||||
}
|
||||
// Handle remainder
|
||||
for (; i < len; i++) {
|
||||
sum += a[i] * b[i];
|
||||
}
|
||||
return sum;
|
||||
},
|
||||
/**
|
||||
* Squared L2 norm with 8x unrolling
|
||||
*/
|
||||
normSq(a) {
|
||||
const len = a.length;
|
||||
let sum = 0;
|
||||
const unrolled = len - (len % 8);
|
||||
let i = 0;
|
||||
for (; i < unrolled; i += 8) {
|
||||
sum += a[i] * a[i]
|
||||
+ a[i + 1] * a[i + 1]
|
||||
+ a[i + 2] * a[i + 2]
|
||||
+ a[i + 3] * a[i + 3]
|
||||
+ a[i + 4] * a[i + 4]
|
||||
+ a[i + 5] * a[i + 5]
|
||||
+ a[i + 6] * a[i + 6]
|
||||
+ a[i + 7] * a[i + 7];
|
||||
}
|
||||
for (; i < len; i++) {
|
||||
sum += a[i] * a[i];
|
||||
}
|
||||
return sum;
|
||||
},
|
||||
/**
|
||||
* L2 norm
|
||||
*/
|
||||
norm(a) {
|
||||
return Math.sqrt(exports.VectorOps.normSq(a));
|
||||
},
|
||||
/**
|
||||
* Cosine similarity - optimized for V8 JIT
|
||||
* Uses 4x unrolling which benchmarks faster than 8x due to register pressure
|
||||
*/
|
||||
cosine(a, b) {
|
||||
const len = a.length;
|
||||
let dot = 0, normA = 0, normB = 0;
|
||||
// 4x unroll is optimal for cosine (less register pressure)
|
||||
const unrolled = len - (len % 4);
|
||||
let i = 0;
|
||||
for (; i < unrolled; i += 4) {
|
||||
const a0 = a[i], a1 = a[i + 1], a2 = a[i + 2], a3 = a[i + 3];
|
||||
const b0 = b[i], b1 = b[i + 1], b2 = b[i + 2], b3 = b[i + 3];
|
||||
dot += a0 * b0 + a1 * b1 + a2 * b2 + a3 * b3;
|
||||
normA += a0 * a0 + a1 * a1 + a2 * a2 + a3 * a3;
|
||||
normB += b0 * b0 + b1 * b1 + b2 * b2 + b3 * b3;
|
||||
}
|
||||
for (; i < len; i++) {
|
||||
dot += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
const denom = Math.sqrt(normA * normB);
|
||||
return denom > 1e-10 ? dot / denom : 0;
|
||||
},
|
||||
/**
|
||||
* Euclidean distance squared with 8x unrolling
|
||||
*/
|
||||
distanceSq(a, b) {
|
||||
const len = a.length;
|
||||
let sum = 0;
|
||||
const unrolled = len - (len % 8);
|
||||
let i = 0;
|
||||
for (; i < unrolled; i += 8) {
|
||||
const d0 = a[i] - b[i];
|
||||
const d1 = a[i + 1] - b[i + 1];
|
||||
const d2 = a[i + 2] - b[i + 2];
|
||||
const d3 = a[i + 3] - b[i + 3];
|
||||
const d4 = a[i + 4] - b[i + 4];
|
||||
const d5 = a[i + 5] - b[i + 5];
|
||||
const d6 = a[i + 6] - b[i + 6];
|
||||
const d7 = a[i + 7] - b[i + 7];
|
||||
sum += d0 * d0 + d1 * d1 + d2 * d2 + d3 * d3
|
||||
+ d4 * d4 + d5 * d5 + d6 * d6 + d7 * d7;
|
||||
}
|
||||
for (; i < len; i++) {
|
||||
const d = a[i] - b[i];
|
||||
sum += d * d;
|
||||
}
|
||||
return sum;
|
||||
},
|
||||
/**
|
||||
* Euclidean distance
|
||||
*/
|
||||
distance(a, b) {
|
||||
return Math.sqrt(exports.VectorOps.distanceSq(a, b));
|
||||
},
|
||||
/**
|
||||
* Add vectors: out = a + b (with 8x unrolling)
|
||||
*/
|
||||
add(a, b, out) {
|
||||
const len = a.length;
|
||||
const unrolled = len - (len % 8);
|
||||
let i = 0;
|
||||
for (; i < unrolled; i += 8) {
|
||||
out[i] = a[i] + b[i];
|
||||
out[i + 1] = a[i + 1] + b[i + 1];
|
||||
out[i + 2] = a[i + 2] + b[i + 2];
|
||||
out[i + 3] = a[i + 3] + b[i + 3];
|
||||
out[i + 4] = a[i + 4] + b[i + 4];
|
||||
out[i + 5] = a[i + 5] + b[i + 5];
|
||||
out[i + 6] = a[i + 6] + b[i + 6];
|
||||
out[i + 7] = a[i + 7] + b[i + 7];
|
||||
}
|
||||
for (; i < len; i++) {
|
||||
out[i] = a[i] + b[i];
|
||||
}
|
||||
return out;
|
||||
},
|
||||
/**
|
||||
* Subtract vectors: out = a - b (with 8x unrolling)
|
||||
*/
|
||||
sub(a, b, out) {
|
||||
const len = a.length;
|
||||
const unrolled = len - (len % 8);
|
||||
let i = 0;
|
||||
for (; i < unrolled; i += 8) {
|
||||
out[i] = a[i] - b[i];
|
||||
out[i + 1] = a[i + 1] - b[i + 1];
|
||||
out[i + 2] = a[i + 2] - b[i + 2];
|
||||
out[i + 3] = a[i + 3] - b[i + 3];
|
||||
out[i + 4] = a[i + 4] - b[i + 4];
|
||||
out[i + 5] = a[i + 5] - b[i + 5];
|
||||
out[i + 6] = a[i + 6] - b[i + 6];
|
||||
out[i + 7] = a[i + 7] - b[i + 7];
|
||||
}
|
||||
for (; i < len; i++) {
|
||||
out[i] = a[i] - b[i];
|
||||
}
|
||||
return out;
|
||||
},
|
||||
/**
|
||||
* Scale vector: out = a * scalar (with 8x unrolling)
|
||||
*/
|
||||
scale(a, scalar, out) {
|
||||
const len = a.length;
|
||||
const unrolled = len - (len % 8);
|
||||
let i = 0;
|
||||
for (; i < unrolled; i += 8) {
|
||||
out[i] = a[i] * scalar;
|
||||
out[i + 1] = a[i + 1] * scalar;
|
||||
out[i + 2] = a[i + 2] * scalar;
|
||||
out[i + 3] = a[i + 3] * scalar;
|
||||
out[i + 4] = a[i + 4] * scalar;
|
||||
out[i + 5] = a[i + 5] * scalar;
|
||||
out[i + 6] = a[i + 6] * scalar;
|
||||
out[i + 7] = a[i + 7] * scalar;
|
||||
}
|
||||
for (; i < len; i++) {
|
||||
out[i] = a[i] * scalar;
|
||||
}
|
||||
return out;
|
||||
},
|
||||
/**
|
||||
* Normalize vector in-place
|
||||
*/
|
||||
normalize(a) {
|
||||
const norm = exports.VectorOps.norm(a);
|
||||
if (norm > 1e-10) {
|
||||
exports.VectorOps.scale(a, 1 / norm, a);
|
||||
}
|
||||
return a;
|
||||
},
|
||||
/**
|
||||
* Mean of multiple vectors (with buffer reuse)
|
||||
*/
|
||||
mean(vectors, out) {
|
||||
const n = vectors.length;
|
||||
if (n === 0)
|
||||
return out;
|
||||
const len = out.length;
|
||||
out.fill(0);
|
||||
// Sum all vectors
|
||||
for (const vec of vectors) {
|
||||
for (let i = 0; i < len; i++) {
|
||||
out[i] += vec[i];
|
||||
}
|
||||
}
|
||||
// Divide by count (unrolled)
|
||||
const invN = 1 / n;
|
||||
exports.VectorOps.scale(out, invN, out);
|
||||
return out;
|
||||
},
|
||||
};
|
||||
/**
|
||||
* Parallel batch processor for embedding operations.
|
||||
* Uses chunking and Promise.all for concurrent processing.
|
||||
*/
|
||||
class ParallelBatchProcessor {
|
||||
constructor(options = {}) {
|
||||
this.batchSize = options.batchSize ?? exports.PERF_CONSTANTS.DEFAULT_BATCH_SIZE;
|
||||
this.maxConcurrency = options.maxConcurrency ?? 4;
|
||||
}
|
||||
/**
|
||||
* Process items in parallel batches
|
||||
*/
|
||||
async processBatch(items, processor) {
|
||||
const start = performance.now();
|
||||
const results = new Array(items.length);
|
||||
// For small batches, process sequentially
|
||||
if (items.length < exports.PERF_CONSTANTS.MIN_PARALLEL_BATCH_SIZE) {
|
||||
for (let i = 0; i < items.length; i++) {
|
||||
results[i] = await processor(items[i], i);
|
||||
}
|
||||
}
|
||||
else {
|
||||
// Chunk into concurrent batches
|
||||
const chunks = this.chunkArray(items, Math.ceil(items.length / this.maxConcurrency));
|
||||
let offset = 0;
|
||||
await Promise.all(chunks.map(async (chunk, chunkIndex) => {
|
||||
const chunkOffset = chunkIndex * chunks[0].length;
|
||||
for (let i = 0; i < chunk.length; i++) {
|
||||
results[chunkOffset + i] = await processor(chunk[i], chunkOffset + i);
|
||||
}
|
||||
}));
|
||||
}
|
||||
const totalMs = performance.now() - start;
|
||||
return {
|
||||
results,
|
||||
timing: {
|
||||
totalMs,
|
||||
perItemMs: items.length > 0 ? totalMs / items.length : 0,
|
||||
},
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Process with synchronous function (uses chunking for better cache locality)
|
||||
*/
|
||||
processSync(items, processor) {
|
||||
const start = performance.now();
|
||||
const results = new Array(items.length);
|
||||
// Process in cache-friendly chunks
|
||||
for (let i = 0; i < items.length; i += this.batchSize) {
|
||||
const end = Math.min(i + this.batchSize, items.length);
|
||||
for (let j = i; j < end; j++) {
|
||||
results[j] = processor(items[j], j);
|
||||
}
|
||||
}
|
||||
const totalMs = performance.now() - start;
|
||||
return {
|
||||
results,
|
||||
timing: {
|
||||
totalMs,
|
||||
perItemMs: items.length > 0 ? totalMs / items.length : 0,
|
||||
},
|
||||
};
|
||||
}
|
||||
/**
|
||||
* Batch similarity search (optimized for many queries)
|
||||
*/
|
||||
batchSimilarity(queries, corpus, k = 5) {
|
||||
const results = [];
|
||||
for (const query of queries) {
|
||||
const scores = [];
|
||||
for (let i = 0; i < corpus.length; i++) {
|
||||
scores.push({
|
||||
index: i,
|
||||
score: exports.VectorOps.cosine(query, corpus[i]),
|
||||
});
|
||||
}
|
||||
// Partial sort for top-k (more efficient than full sort)
|
||||
scores.sort((a, b) => b.score - a.score);
|
||||
results.push(scores.slice(0, k));
|
||||
}
|
||||
return results;
|
||||
}
|
||||
chunkArray(arr, chunkSize) {
|
||||
const chunks = [];
|
||||
for (let i = 0; i < arr.length; i += chunkSize) {
|
||||
chunks.push(arr.slice(i, i + chunkSize));
|
||||
}
|
||||
return chunks;
|
||||
}
|
||||
}
|
||||
exports.ParallelBatchProcessor = ParallelBatchProcessor;
|
||||
/**
|
||||
* High-performance memory store with O(1) LRU caching.
|
||||
*/
|
||||
class OptimizedMemoryStore {
|
||||
constructor(options = {}) {
|
||||
this.cache = new LRUCache(options.cacheSize ?? exports.PERF_CONSTANTS.DEFAULT_CACHE_SIZE);
|
||||
this.bufferPool = new Float32BufferPool();
|
||||
this.dimension = options.dimension ?? 384;
|
||||
// Pre-warm buffer pool
|
||||
this.bufferPool.prewarm([this.dimension], 16);
|
||||
}
|
||||
/**
|
||||
* Store embedding - O(1)
|
||||
*/
|
||||
store(id, embedding, content) {
|
||||
// Acquire buffer from pool
|
||||
const buffer = this.bufferPool.acquire(this.dimension);
|
||||
// Copy embedding to pooled buffer
|
||||
const emb = embedding instanceof Float32Array ? embedding : new Float32Array(embedding);
|
||||
buffer.set(emb);
|
||||
this.cache.set(id, {
|
||||
id,
|
||||
embedding: buffer,
|
||||
content,
|
||||
score: 1.0,
|
||||
});
|
||||
}
|
||||
/**
|
||||
* Get by ID - O(1)
|
||||
*/
|
||||
get(id) {
|
||||
return this.cache.get(id);
|
||||
}
|
||||
/**
|
||||
* Search by similarity - O(n) but with optimized vector ops
|
||||
*/
|
||||
search(query, k = 5) {
|
||||
const results = [];
|
||||
for (const [, entry] of this.cache.entries()) {
|
||||
const score = exports.VectorOps.cosine(query, entry.embedding);
|
||||
results.push({ entry, score });
|
||||
}
|
||||
results.sort((a, b) => b.score - a.score);
|
||||
return results.slice(0, k).map(r => ({ ...r.entry, score: r.score }));
|
||||
}
|
||||
/**
|
||||
* Delete entry - O(1)
|
||||
*/
|
||||
delete(id) {
|
||||
const entry = this.cache.get(id);
|
||||
if (entry) {
|
||||
this.bufferPool.release(entry.embedding);
|
||||
}
|
||||
return this.cache.delete(id);
|
||||
}
|
||||
/**
|
||||
* Get statistics
|
||||
*/
|
||||
getStats() {
|
||||
return {
|
||||
cache: this.cache.getStats(),
|
||||
buffers: this.bufferPool.getStats(),
|
||||
};
|
||||
}
|
||||
}
|
||||
exports.OptimizedMemoryStore = OptimizedMemoryStore;
|
||||
// ============================================================================
|
||||
// Exports
|
||||
// ============================================================================
|
||||
exports.default = {
|
||||
LRUCache,
|
||||
Float32BufferPool,
|
||||
TensorBufferManager,
|
||||
VectorOps: exports.VectorOps,
|
||||
ParallelBatchProcessor,
|
||||
OptimizedMemoryStore,
|
||||
PERF_CONSTANTS: exports.PERF_CONSTANTS,
|
||||
};
|
||||
//# sourceMappingURL=neural-perf.js.map
|
||||
1
npm/packages/ruvector/src/core/neural-perf.js.map
Normal file
1
npm/packages/ruvector/src/core/neural-perf.js.map
Normal file
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user