Squashed 'vendor/ruvector/' content from commit b64c2172
git-subtree-dir: vendor/ruvector git-subtree-split: b64c21726f2bb37286d9ee36a7869fef60cc6900
This commit is contained in:
ruv
484
npm/packages/agentic-synth/examples/self-learning/README.md
Normal file
484
npm/packages/agentic-synth/examples/self-learning/README.md
Normal file
@@ -0,0 +1,484 @@
|
||||
# Self-Learning System Examples
|
||||
|
||||
This directory contains comprehensive examples for generating synthetic data for self-learning AI systems, including reinforcement learning, feedback loops, and continual learning scenarios.
|
||||
|
||||
## Overview
|
||||
|
||||
These examples demonstrate how to use **agentic-synth** to generate training data for adaptive AI systems that learn continuously from experience and feedback.
|
||||
|
||||
## Files
|
||||
|
||||
### 1. `reinforcement-learning.ts`
|
||||
|
||||
Generates synthetic data for reinforcement learning systems.
|
||||
|
||||
**Key Features:**
|
||||
- State-Action-Reward (SAR) tuples for Q-learning
|
||||
- Complete episodes with temporal consistency
|
||||
- Exploration vs exploitation scenarios (multi-armed bandits)
|
||||
- Reward function testing data
|
||||
- Policy gradient training data
|
||||
- Multi-agent RL scenarios
|
||||
|
||||
**Examples:**
|
||||
```typescript
|
||||
import {
|
||||
generateSARTuples,
|
||||
generateEpisodes,
|
||||
generateExplorationData,
|
||||
generatePolicyGradientData,
|
||||
generateMultiAgentData
|
||||
} from './reinforcement-learning.js';
|
||||
|
||||
// Generate SAR tuples for Q-learning
|
||||
const sarData = await generateSARTuples();
|
||||
|
||||
// Generate complete episodes
|
||||
const episodes = await generateEpisodes();
|
||||
|
||||
// Generate exploration data
|
||||
const explorationData = await generateExplorationData();
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- Training RL agents (DQN, PPO, A3C, SAC)
|
||||
- Testing reward functions
|
||||
- Evaluating exploration strategies
|
||||
- Multi-agent coordination research
|
||||
|
||||
---
|
||||
|
||||
### 2. `feedback-loop.ts`
|
||||
|
||||
Generates data for self-improving systems with feedback mechanisms.
|
||||
|
||||
**Key Features:**
|
||||
- Quality scoring and automatic regeneration
|
||||
- A/B testing data for model comparison
|
||||
- Pattern learning from production data
|
||||
- Adaptive schema evolution
|
||||
- Active learning sample selection
|
||||
- Continuous model evaluation
|
||||
|
||||
**Examples:**
|
||||
```typescript
|
||||
import {
|
||||
qualityScoringLoop,
|
||||
abTestingData,
|
||||
patternLearningLoop,
|
||||
adaptiveSchemaEvolution,
|
||||
activeLearningData
|
||||
} from './feedback-loop.js';
|
||||
|
||||
// Generate and improve low-quality samples
|
||||
await qualityScoringLoop();
|
||||
|
||||
// Generate A/B test data
|
||||
const abTests = await abTestingData();
|
||||
|
||||
// Learn patterns from production
|
||||
const syntheticData = await patternLearningLoop();
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- Model improvement iterations
|
||||
- Quality assurance pipelines
|
||||
- Production data simulation
|
||||
- Active learning systems
|
||||
- Continuous integration/deployment
|
||||
|
||||
---
|
||||
|
||||
### 3. `continual-learning.ts`
|
||||
|
||||
Generates data for continual learning systems that adapt over time.
|
||||
|
||||
**Key Features:**
|
||||
- Incremental training data (multiple phases)
|
||||
- Domain adaptation (source → target)
|
||||
- Catastrophic forgetting prevention (replay buffers)
|
||||
- Transfer learning datasets (pre-training → fine-tuning)
|
||||
- Curriculum learning (easy → hard)
|
||||
- Online learning streams with concept drift
|
||||
|
||||
**Examples:**
|
||||
```typescript
|
||||
import {
|
||||
generateIncrementalData,
|
||||
generateDomainAdaptationData,
|
||||
generateAntiCatastrophicData,
|
||||
generateTransferLearningData,
|
||||
generateCurriculumData,
|
||||
generateOnlineLearningStream
|
||||
} from './continual-learning.js';
|
||||
|
||||
// Generate incremental training phases
|
||||
const phases = await generateIncrementalData();
|
||||
|
||||
// Generate domain adaptation data
|
||||
const { source, target, labeledTarget } = await generateDomainAdaptationData();
|
||||
|
||||
// Generate anti-forgetting data
|
||||
const { task1, task2, replay } = await generateAntiCatastrophicData();
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- Lifelong learning systems
|
||||
- Domain adaptation research
|
||||
- Transfer learning pipelines
|
||||
- Curriculum learning
|
||||
- Online/streaming learning
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
Ensure you have agentic-synth installed:
|
||||
|
||||
```bash
|
||||
npm install agentic-synth
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Set up your API key:
|
||||
|
||||
```bash
|
||||
# Gemini API (recommended)
|
||||
export GEMINI_API_KEY=your_api_key_here
|
||||
|
||||
# Or OpenRouter
|
||||
export OPENROUTER_API_KEY=your_api_key_here
|
||||
```
|
||||
|
||||
## Running Examples
|
||||
|
||||
### Run Individual Examples
|
||||
|
||||
```bash
|
||||
# Reinforcement learning examples
|
||||
npx tsx examples/self-learning/reinforcement-learning.ts
|
||||
|
||||
# Feedback loop examples
|
||||
npx tsx examples/self-learning/feedback-loop.ts
|
||||
|
||||
# Continual learning examples
|
||||
npx tsx examples/self-learning/continual-learning.ts
|
||||
```
|
||||
|
||||
### Run Specific Functions
|
||||
|
||||
```typescript
|
||||
import { generateSARTuples } from './reinforcement-learning.js';
|
||||
|
||||
// Run specific example
|
||||
await generateSARTuples();
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### 1. Training Loop Integration
|
||||
|
||||
```typescript
|
||||
import { createSynth } from 'agentic-synth';
|
||||
|
||||
const synth = createSynth({
|
||||
provider: 'gemini',
|
||||
apiKey: process.env.GEMINI_API_KEY,
|
||||
cacheStrategy: 'memory', // Cache for faster iterations
|
||||
});
|
||||
|
||||
// Generate training batch
|
||||
const batch = await synth.generateStructured({
|
||||
count: 1000,
|
||||
schema: {
|
||||
features: ['array of 10 numbers (0-1)'],
|
||||
label: 'number (0-4)',
|
||||
},
|
||||
});
|
||||
|
||||
// Use in training
|
||||
for (const sample of batch.data) {
|
||||
// Train model with sample.features and sample.label
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Quality-Based Regeneration
|
||||
|
||||
```typescript
|
||||
// Generate initial data
|
||||
const data = await synth.generateStructured({ count: 100, schema });
|
||||
|
||||
// Filter low-quality samples
|
||||
const lowQuality = data.data.filter(d => d.quality_score < 0.7);
|
||||
|
||||
// Regenerate with improved constraints
|
||||
const improved = await synth.generateStructured({
|
||||
count: lowQuality.length,
|
||||
schema: improvedSchema,
|
||||
constraints: ['quality_score should be >= 0.7', 'improve coherence'],
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Incremental Learning Pipeline
|
||||
|
||||
```typescript
|
||||
const phases = [];
|
||||
|
||||
// Generate multiple phases
|
||||
for (let phase = 1; phase <= 5; phase++) {
|
||||
const phaseData = await synth.generateStructured({
|
||||
count: 200,
|
||||
schema: {
|
||||
phase: `number (${phase})`,
|
||||
features: { /* evolving features */ },
|
||||
label: 'number',
|
||||
},
|
||||
constraints: [`Bias toward new patterns in phase ${phase}`],
|
||||
});
|
||||
|
||||
phases.push(phaseData);
|
||||
|
||||
// Train model incrementally
|
||||
// model.train(phaseData.data);
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Experience Replay
|
||||
|
||||
```typescript
|
||||
// Generate new task data
|
||||
const newTask = await synth.generateStructured({
|
||||
count: 200,
|
||||
schema: newTaskSchema,
|
||||
});
|
||||
|
||||
// Generate replay buffer from old task
|
||||
const replay = await synth.generateStructured({
|
||||
count: 50, // 25% of new data
|
||||
schema: oldTaskSchema,
|
||||
constraints: ['High importance samples', 'Diverse and difficult'],
|
||||
});
|
||||
|
||||
// Interleave for training
|
||||
const mixedBatch = [...newTask.data, ...replay.data];
|
||||
// model.train(shuffle(mixedBatch));
|
||||
```
|
||||
|
||||
## ML Framework Integration
|
||||
|
||||
### TensorFlow.js
|
||||
|
||||
```typescript
|
||||
import * as tf from '@tensorflow/tfjs';
|
||||
|
||||
const trainingData = await synth.generateStructured({
|
||||
count: 1000,
|
||||
schema: {
|
||||
features: ['array of 4 numbers (0-1)'],
|
||||
label: 'number (0 or 1)',
|
||||
},
|
||||
});
|
||||
|
||||
// Convert to tensors
|
||||
const xs = tf.tensor2d(trainingData.data.map(d => d.features));
|
||||
const ys = tf.tensor2d(trainingData.data.map(d => [d.label]));
|
||||
|
||||
// Train model
|
||||
await model.fit(xs, ys, { epochs: 100 });
|
||||
```
|
||||
|
||||
### PyTorch (via data export)
|
||||
|
||||
```typescript
|
||||
import { writeFileSync } from 'fs';
|
||||
|
||||
const data = await synth.generateStructured({
|
||||
count: 10000,
|
||||
schema: pytorchSchema,
|
||||
});
|
||||
|
||||
// Export as JSON for PyTorch DataLoader
|
||||
writeFileSync('training_data.json', JSON.stringify(data.data));
|
||||
```
|
||||
|
||||
```python
|
||||
# In Python
|
||||
import json
|
||||
import torch
|
||||
from torch.utils.data import Dataset, DataLoader
|
||||
|
||||
class SyntheticDataset(Dataset):
|
||||
def __init__(self, json_path):
|
||||
with open(json_path) as f:
|
||||
self.data = json.load(f)
|
||||
|
||||
def __len__(self):
|
||||
return len(self.data)
|
||||
|
||||
def __getitem__(self, idx):
|
||||
item = self.data[idx]
|
||||
return torch.tensor(item['features']), torch.tensor(item['label'])
|
||||
|
||||
dataset = SyntheticDataset('training_data.json')
|
||||
loader = DataLoader(dataset, batch_size=32, shuffle=True)
|
||||
```
|
||||
|
||||
### scikit-learn
|
||||
|
||||
```typescript
|
||||
const data = await synth.generateStructured({
|
||||
count: 500,
|
||||
schema: {
|
||||
feature1: 'number (0-100)',
|
||||
feature2: 'number (0-100)',
|
||||
feature3: 'number (0-100)',
|
||||
label: 'number (0 or 1)',
|
||||
},
|
||||
});
|
||||
|
||||
// Export for sklearn
|
||||
const X = data.data.map(d => [d.feature1, d.feature2, d.feature3]);
|
||||
const y = data.data.map(d => d.label);
|
||||
|
||||
console.log(JSON.stringify({ X, y }));
|
||||
```
|
||||
|
||||
## Advanced Use Cases
|
||||
|
||||
### 1. Curriculum Learning
|
||||
|
||||
Start with easy examples and gradually increase difficulty:
|
||||
|
||||
```typescript
|
||||
const curriculum = ['easy', 'medium', 'hard', 'expert'];
|
||||
|
||||
for (const level of curriculum) {
|
||||
const data = await generateCurriculumData(level);
|
||||
// Train model on current difficulty level
|
||||
await trainModel(data);
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Domain Adaptation
|
||||
|
||||
Adapt from source domain to target domain:
|
||||
|
||||
```typescript
|
||||
// Pre-train on source domain
|
||||
const sourceData = await generateSourceDomain();
|
||||
await model.pretrain(sourceData);
|
||||
|
||||
// Fine-tune on small labeled target set
|
||||
const targetData = await generateLabeledTarget(count: 50);
|
||||
await model.finetune(targetData);
|
||||
```
|
||||
|
||||
### 3. Multi-Task Learning
|
||||
|
||||
Generate data for multiple related tasks:
|
||||
|
||||
```typescript
|
||||
const tasks = ['task1', 'task2', 'task3'];
|
||||
const taskData = {};
|
||||
|
||||
for (const task of tasks) {
|
||||
taskData[task] = await synth.generateStructured({
|
||||
count: 200,
|
||||
schema: taskSchemas[task],
|
||||
});
|
||||
}
|
||||
|
||||
// Train multi-task model
|
||||
await multiTaskModel.train(taskData);
|
||||
```
|
||||
|
||||
### 4. Meta-Learning (Learning to Learn)
|
||||
|
||||
Generate few-shot learning episodes:
|
||||
|
||||
```typescript
|
||||
const episodes = await synth.generateStructured({
|
||||
count: 100,
|
||||
schema: {
|
||||
support_set: [{ features: [], label: 'number' }],
|
||||
query_set: [{ features: [], label: 'number' }],
|
||||
task_id: 'UUID',
|
||||
},
|
||||
});
|
||||
|
||||
// Meta-train
|
||||
for (const episode of episodes.data) {
|
||||
await metalearner.adapt(episode.support_set);
|
||||
const loss = metalearner.evaluate(episode.query_set);
|
||||
metalearner.metaUpdate(loss);
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Use Caching**: Enable memory or disk caching for repeated generations
|
||||
```typescript
|
||||
const synth = createSynth({
|
||||
cacheStrategy: 'memory',
|
||||
cacheTTL: 3600, // 1 hour
|
||||
});
|
||||
```
|
||||
|
||||
2. **Batch Generation**: Generate multiple datasets in parallel
|
||||
```typescript
|
||||
const batches = await synth.generateBatch('structured', [
|
||||
{ count: 100, schema: schema1 },
|
||||
{ count: 100, schema: schema2 },
|
||||
{ count: 100, schema: schema3 },
|
||||
], 3); // 3 concurrent generations
|
||||
```
|
||||
|
||||
3. **Streaming**: Use streaming for large datasets
|
||||
```typescript
|
||||
for await (const sample of synth.generateStream('structured', options)) {
|
||||
// Process sample immediately
|
||||
processAndTrain(sample);
|
||||
}
|
||||
```
|
||||
|
||||
## Citation
|
||||
|
||||
If you use these examples in your research, please cite:
|
||||
|
||||
```bibtex
|
||||
@software{agentic_synth,
|
||||
title = {Agentic-Synth: AI-Powered Synthetic Data Generation},
|
||||
author = {Your Name},
|
||||
year = {2024},
|
||||
url = {https://github.com/yourusername/agentic-synth}
|
||||
}
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! Please feel free to submit pull requests with:
|
||||
- New self-learning examples
|
||||
- Improved ML framework integrations
|
||||
- Performance optimizations
|
||||
- Bug fixes
|
||||
|
||||
## License
|
||||
|
||||
MIT License - see LICENSE file for details
|
||||
|
||||
## Support
|
||||
|
||||
- Documentation: [Main README](../../README.md)
|
||||
- Issues: [GitHub Issues](https://github.com/yourusername/agentic-synth/issues)
|
||||
- Discussions: [GitHub Discussions](https://github.com/yourusername/agentic-synth/discussions)
|
||||
|
||||
## Related Examples
|
||||
|
||||
- [Basic Usage](../basic-usage.ts) - Getting started with agentic-synth
|
||||
- [Integration Examples](../integration-examples.ts) - Framework integrations
|
||||
- [Benchmark Example](../benchmark-example.ts) - Performance testing
|
||||
|
||||
---
|
||||
|
||||
**Happy Learning!** 🚀🤖📈
|
||||
Reference in New Issue
Block a user