Merge commit 'd803bfe2b1fe7f5e219e50ac20d6801a0a58ac75' as 'vendor/ruvector'
This commit is contained in:
ruv
567
vendor/ruvector/.github/workflows/RELEASE.md
vendored
Normal file
567
vendor/ruvector/.github/workflows/RELEASE.md
vendored
Normal file
@@ -0,0 +1,567 @@
|
||||
# RuVector Release Pipeline Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
The RuVector release pipeline is a comprehensive CI/CD workflow that automates the building, testing, and publishing of Rust crates and npm packages across multiple platforms.
|
||||
|
||||
## Workflow Files
|
||||
|
||||
- **`release.yml`**: Main release pipeline workflow
|
||||
- **`build-native.yml`**: Reusable workflow for building native Node.js modules
|
||||
- **`validate-lockfile.yml`**: Validates package-lock.json integrity
|
||||
|
||||
## Trigger Methods
|
||||
|
||||
### 1. Tag-Based Release (Recommended)
|
||||
|
||||
```bash
|
||||
# Create and push a version tag
|
||||
git tag v0.1.3
|
||||
git push origin v0.1.3
|
||||
```
|
||||
|
||||
This automatically triggers the full release pipeline.
|
||||
|
||||
### 2. Manual Workflow Dispatch
|
||||
|
||||
Navigate to: **Actions → Release Pipeline → Run workflow**
|
||||
|
||||
Options:
|
||||
- **Version**: Version to release (e.g., `0.1.3`)
|
||||
- **Skip Tests**: Skip validation tests (not recommended)
|
||||
- **Dry Run**: Build everything but don't publish
|
||||
|
||||
## Pipeline Stages
|
||||
|
||||
### Stage 1: Validation (`validate`)
|
||||
|
||||
**Runs on**: `ubuntu-22.04`
|
||||
|
||||
**Tasks**:
|
||||
- ✅ Check code formatting with `cargo fmt`
|
||||
- ✅ Run Clippy lints with all warnings as errors
|
||||
- ✅ Run Rust test suite across all crates
|
||||
- ✅ Run npm unit tests
|
||||
- ✅ Generate validation summary
|
||||
|
||||
**Skip condition**: Set `skip_tests: true` in manual workflow dispatch
|
||||
|
||||
### Stage 2: Build Rust Crates (`build-crates`)
|
||||
|
||||
**Runs on**: `ubuntu-22.04`
|
||||
|
||||
**Tasks**:
|
||||
- Build all workspace crates in release mode
|
||||
- Run crate-specific tests
|
||||
- Generate build summary with all crate versions
|
||||
|
||||
**Crates built** (26 total):
|
||||
- Core: `ruvector-core`, `ruvector-metrics`, `ruvector-filter`
|
||||
- Graph: `ruvector-graph`, `ruvector-gnn`
|
||||
- Distributed: `ruvector-cluster`, `ruvector-raft`, `ruvector-replication`
|
||||
- Bindings: `ruvector-node`, `ruvector-wasm`
|
||||
- And 16 more specialized crates
|
||||
|
||||
### Stage 3: Build WASM Packages (`build-wasm`)
|
||||
|
||||
**Runs on**: `ubuntu-22.04`
|
||||
|
||||
**Tasks**:
|
||||
- Install `wasm-pack` build tool
|
||||
- Build WASM packages for:
|
||||
- `ruvector-wasm` (core WASM)
|
||||
- `ruvector-gnn-wasm` (graph neural networks)
|
||||
- `ruvector-graph-wasm` (graph database)
|
||||
- `ruvector-tiny-dancer-wasm` (tiny dancer)
|
||||
- Upload WASM artifacts for later stages
|
||||
|
||||
**Caching**:
|
||||
- Rust dependencies via `Swatinem/rust-cache`
|
||||
- wasm-pack binary
|
||||
|
||||
### Stage 4: Build Native Modules (`build-native`)
|
||||
|
||||
**Runs on**: Multi-platform matrix
|
||||
|
||||
**Reuses**: `./.github/workflows/build-native.yml` as callable workflow
|
||||
|
||||
**Platforms built**:
|
||||
- Linux x64 (GNU) - `ubuntu-22.04`
|
||||
- Linux ARM64 (GNU) - `ubuntu-22.04` with cross-compilation
|
||||
- macOS x64 (Intel) - `macos-13`
|
||||
- macOS ARM64 (Apple Silicon) - `macos-14`
|
||||
- Windows x64 (MSVC) - `windows-2022`
|
||||
|
||||
**Build matrix details**:
|
||||
```yaml
|
||||
- host: ubuntu-22.04, target: x86_64-unknown-linux-gnu
|
||||
- host: ubuntu-22.04, target: aarch64-unknown-linux-gnu
|
||||
- host: macos-13, target: x86_64-apple-darwin
|
||||
- host: macos-14, target: aarch64-apple-darwin
|
||||
- host: windows-2022, target: x86_64-pc-windows-msvc
|
||||
```
|
||||
|
||||
**Output**: Binary artifacts for each platform uploaded to GitHub Actions
|
||||
|
||||
### Stage 5: Publish Rust Crates (`publish-crates`)
|
||||
|
||||
**Runs on**: `ubuntu-22.04`
|
||||
|
||||
**Requires**:
|
||||
- ✅ Validation passed
|
||||
- ✅ Build crates succeeded
|
||||
- 🔑 `CARGO_REGISTRY_TOKEN` secret configured
|
||||
- Tag starts with `v*` OR manual workflow dispatch
|
||||
- NOT in dry-run mode
|
||||
|
||||
**Publishing order** (respects dependencies):
|
||||
|
||||
```
|
||||
1. ruvector-core (foundation)
|
||||
2. ruvector-metrics, ruvector-filter, ruvector-snapshot
|
||||
3. ruvector-collections, ruvector-router-core
|
||||
4. ruvector-raft, ruvector-cluster, ruvector-replication
|
||||
5. ruvector-gnn, ruvector-graph
|
||||
6. ruvector-server, ruvector-tiny-dancer-core
|
||||
7. ruvector-router-cli, ruvector-router-ffi, ruvector-router-wasm
|
||||
8. ruvector-cli, ruvector-bench
|
||||
9. ruvector-wasm, ruvector-node
|
||||
10. ruvector-gnn-wasm, ruvector-gnn-node
|
||||
11. ruvector-graph-wasm, ruvector-graph-node
|
||||
12. ruvector-tiny-dancer-wasm, ruvector-tiny-dancer-node
|
||||
```
|
||||
|
||||
**Rate limiting**: 10 second delay between publishes to avoid crates.io rate limits
|
||||
|
||||
**Error handling**: Continues if a crate already exists (409 error)
|
||||
|
||||
### Stage 6: Publish npm Packages (`publish-npm`)
|
||||
|
||||
**Runs on**: `ubuntu-22.04`
|
||||
|
||||
**Requires**:
|
||||
- ✅ Validation passed
|
||||
- ✅ Build native succeeded
|
||||
- ✅ Build WASM succeeded
|
||||
- 🔑 `NPM_TOKEN` secret configured
|
||||
- Tag starts with `v*` OR manual workflow dispatch
|
||||
- NOT in dry-run mode
|
||||
|
||||
**Publishing order**:
|
||||
|
||||
```
|
||||
1. Platform-specific packages (@ruvector/core-*)
|
||||
- @ruvector/core-linux-x64-gnu
|
||||
- @ruvector/core-linux-arm64-gnu
|
||||
- @ruvector/core-darwin-x64
|
||||
- @ruvector/core-darwin-arm64
|
||||
- @ruvector/core-win32-x64-msvc
|
||||
|
||||
2. @ruvector/wasm (WebAssembly bindings)
|
||||
3. @ruvector/cli (Command-line interface)
|
||||
4. @ruvector/extensions (Extensions)
|
||||
5. @ruvector/core (Main package - depends on platform packages)
|
||||
```
|
||||
|
||||
**Artifact handling**:
|
||||
- Downloads native binaries from `build-native` job
|
||||
- Downloads WASM packages from `build-wasm` job
|
||||
- Copies to appropriate package directories
|
||||
- Runs `npm ci` and `npm run build`
|
||||
- Publishes with `--access public`
|
||||
|
||||
### Stage 7: Create GitHub Release (`create-release`)
|
||||
|
||||
**Runs on**: `ubuntu-22.04`
|
||||
|
||||
**Requires**:
|
||||
- ✅ All build jobs succeeded
|
||||
- Tag starts with `v*` OR manual workflow dispatch
|
||||
|
||||
**Tasks**:
|
||||
|
||||
1. **Download all artifacts**
|
||||
- Native binaries for all platforms
|
||||
- WASM packages
|
||||
|
||||
2. **Package artifacts**
|
||||
- `ruvector-native-linux-x64-gnu.tar.gz`
|
||||
- `ruvector-native-linux-arm64-gnu.tar.gz`
|
||||
- `ruvector-native-darwin-x64.tar.gz`
|
||||
- `ruvector-native-darwin-arm64.tar.gz`
|
||||
- `ruvector-native-win32-x64-msvc.tar.gz`
|
||||
- `ruvector-wasm.tar.gz`
|
||||
|
||||
3. **Generate release notes**
|
||||
- What's new section
|
||||
- Package lists (Rust crates and npm)
|
||||
- Platform support matrix
|
||||
- Installation instructions
|
||||
- Links to registries
|
||||
- Build metrics
|
||||
|
||||
4. **Create GitHub release**
|
||||
- Uses `softprops/action-gh-release@v1`
|
||||
- Attaches packaged artifacts
|
||||
- Marks as prerelease if version contains `alpha` or `beta`
|
||||
|
||||
### Stage 8: Release Summary (`release-summary`)
|
||||
|
||||
**Runs on**: `ubuntu-22.04`
|
||||
|
||||
**Always runs**: Even if previous jobs fail
|
||||
|
||||
**Tasks**:
|
||||
- Generate comprehensive status table
|
||||
- Show success/failure for each job
|
||||
- Provide next steps and verification links
|
||||
|
||||
## Required Secrets
|
||||
|
||||
### CARGO_REGISTRY_TOKEN
|
||||
|
||||
**Purpose**: Publish Rust crates to crates.io
|
||||
|
||||
**Setup**:
|
||||
1. Go to https://crates.io/settings/tokens
|
||||
2. Create new token with `publish-new` and `publish-update` scopes
|
||||
3. Add to GitHub: **Settings → Secrets → Actions → New secret**
|
||||
- Name: `CARGO_REGISTRY_TOKEN`
|
||||
- Value: Your crates.io token
|
||||
|
||||
### NPM_TOKEN
|
||||
|
||||
**Purpose**: Publish npm packages to npmjs.com
|
||||
|
||||
**Setup**:
|
||||
1. Login to npmjs.com
|
||||
2. Go to **Access Tokens → Generate New Token**
|
||||
3. Select **Automation** type
|
||||
4. Add to GitHub: **Settings → Secrets → Actions → New secret**
|
||||
- Name: `NPM_TOKEN`
|
||||
- Value: Your npm token
|
||||
|
||||
## Environments
|
||||
|
||||
The workflow uses GitHub Environments for additional security:
|
||||
|
||||
### `crates-io` Environment
|
||||
- Used for `publish-crates` job
|
||||
- Can add required reviewers
|
||||
- Can add environment-specific secrets
|
||||
|
||||
### `npm` Environment
|
||||
- Used for `publish-npm` job
|
||||
- Can add required reviewers
|
||||
- Can add environment-specific secrets
|
||||
|
||||
**Setup environments**:
|
||||
1. Go to **Settings → Environments**
|
||||
2. Create `crates-io` and `npm` environments
|
||||
3. (Optional) Add required reviewers for production releases
|
||||
|
||||
## Caching Strategy
|
||||
|
||||
### Rust Cache
|
||||
```yaml
|
||||
uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
prefix-key: 'v1-rust'
|
||||
shared-key: 'validate|build-crates|wasm'
|
||||
```
|
||||
|
||||
**Caches**:
|
||||
- `~/.cargo/registry`
|
||||
- `~/.cargo/git`
|
||||
- `target/` directory
|
||||
|
||||
**Benefits**: 2-5x faster builds
|
||||
|
||||
### Node.js Cache
|
||||
```yaml
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
cache: 'npm'
|
||||
cache-dependency-path: npm/package-lock.json
|
||||
```
|
||||
|
||||
**Caches**: `~/.npm` directory
|
||||
|
||||
## Build Matrix
|
||||
|
||||
The native build job uses a strategic matrix to cover all platforms:
|
||||
|
||||
| Platform | Host Runner | Rust Target | NAPI Platform | Cross-Compile |
|
||||
|----------|-------------|-------------|---------------|---------------|
|
||||
| Linux x64 | ubuntu-22.04 | x86_64-unknown-linux-gnu | linux-x64-gnu | No |
|
||||
| Linux ARM64 | ubuntu-22.04 | aarch64-unknown-linux-gnu | linux-arm64-gnu | Yes (gcc-aarch64) |
|
||||
| macOS Intel | macos-13 | x86_64-apple-darwin | darwin-x64 | No |
|
||||
| macOS ARM | macos-14 | aarch64-apple-darwin | darwin-arm64 | No |
|
||||
| Windows | windows-2022 | x86_64-pc-windows-msvc | win32-x64-msvc | No |
|
||||
|
||||
## Artifact Retention
|
||||
|
||||
- **Native binaries**: 7 days
|
||||
- **WASM packages**: 7 days
|
||||
- **Release packages**: Permanent (attached to GitHub release)
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
### Regular Release
|
||||
|
||||
```bash
|
||||
# 1. Update versions in Cargo.toml files
|
||||
# 2. Update npm package.json files
|
||||
# 3. Commit changes
|
||||
git add .
|
||||
git commit -m "chore: Bump version to 0.1.3"
|
||||
|
||||
# 4. Create and push tag
|
||||
git tag v0.1.3
|
||||
git push origin main
|
||||
git push origin v0.1.3
|
||||
|
||||
# 5. Monitor workflow at:
|
||||
# https://github.com/ruvnet/ruvector/actions/workflows/release.yml
|
||||
```
|
||||
|
||||
### Dry Run (Test Release)
|
||||
|
||||
1. Go to **Actions → Release Pipeline**
|
||||
2. Click **Run workflow**
|
||||
3. Set:
|
||||
- Version: `0.1.3-test`
|
||||
- Dry run: `true`
|
||||
4. Click **Run workflow**
|
||||
|
||||
This builds everything but skips publishing.
|
||||
|
||||
### Emergency Hotfix
|
||||
|
||||
```bash
|
||||
# 1. Create hotfix branch
|
||||
git checkout -b hotfix/critical-fix
|
||||
|
||||
# 2. Make fixes
|
||||
# 3. Bump patch version
|
||||
# 4. Commit and tag
|
||||
git commit -m "fix: Critical security patch"
|
||||
git tag v0.1.3-hotfix.1
|
||||
git push origin hotfix/critical-fix
|
||||
git push origin v0.1.3-hotfix.1
|
||||
|
||||
# 5. Manually trigger release workflow if needed
|
||||
```
|
||||
|
||||
### Republish Failed Package
|
||||
|
||||
If a single npm package fails to publish:
|
||||
|
||||
```bash
|
||||
# 1. Check error in workflow logs
|
||||
# 2. Fix issue locally
|
||||
# 3. Manually publish that package:
|
||||
cd npm/packages/wasm
|
||||
npm publish --access public
|
||||
|
||||
# Or trigger just the npm publishing:
|
||||
# Manually run workflow_dispatch with skip_tests: true
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Build Failures
|
||||
|
||||
**Symptom**: `build-crates` job fails
|
||||
|
||||
**Solutions**:
|
||||
1. Check Rust version compatibility
|
||||
2. Verify all dependencies are available
|
||||
3. Look for compilation errors in logs
|
||||
4. Test locally: `cargo build --workspace --release`
|
||||
|
||||
### Publishing Failures
|
||||
|
||||
**Symptom**: `publish-crates` or `publish-npm` fails
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. **Rate limiting**:
|
||||
- Wait and re-run workflow
|
||||
- Increase delay between publishes
|
||||
|
||||
2. **Already published**:
|
||||
- Bump version number
|
||||
- Or skip that package (it's already live)
|
||||
|
||||
3. **Authentication**:
|
||||
- Verify secrets are set correctly
|
||||
- Check token hasn't expired
|
||||
- Verify token has correct permissions
|
||||
|
||||
4. **Dependency issues**:
|
||||
- Check publishing order
|
||||
- Ensure dependencies are published first
|
||||
|
||||
### Cross-Compilation Issues
|
||||
|
||||
**Symptom**: Linux ARM64 build fails
|
||||
|
||||
**Solutions**:
|
||||
1. Verify cross-compilation tools installed
|
||||
2. Check linker configuration
|
||||
3. Test with: `cargo build --target aarch64-unknown-linux-gnu`
|
||||
|
||||
### WASM Build Issues
|
||||
|
||||
**Symptom**: `build-wasm` job fails
|
||||
|
||||
**Solutions**:
|
||||
1. Verify `wasm-pack` installation
|
||||
2. Check for incompatible dependencies
|
||||
3. Ensure `wasm32-unknown-unknown` target installed
|
||||
4. Test locally: `wasm-pack build --target nodejs`
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
### Parallel Builds
|
||||
|
||||
The workflow runs these jobs in parallel:
|
||||
- `build-crates`
|
||||
- `build-wasm`
|
||||
- `build-native` (5 platform builds in parallel)
|
||||
|
||||
Total time: ~15-25 minutes (vs. 60+ minutes sequential)
|
||||
|
||||
### Cache Hit Rates
|
||||
|
||||
With proper caching:
|
||||
- Rust builds: 70-90% cache hit rate
|
||||
- npm installs: 90-95% cache hit rate
|
||||
|
||||
### Build Time Breakdown
|
||||
|
||||
| Job | Uncached | Cached |
|
||||
|-----|----------|--------|
|
||||
| Validate | 8-12 min | 3-5 min |
|
||||
| Build Crates | 15-20 min | 5-8 min |
|
||||
| Build WASM | 10-15 min | 4-6 min |
|
||||
| Build Native (per platform) | 8-12 min | 3-5 min |
|
||||
| Publish Crates | 5-10 min | 5-10 min |
|
||||
| Publish npm | 3-5 min | 2-3 min |
|
||||
| Create Release | 2-3 min | 2-3 min |
|
||||
|
||||
**Total (worst case)**: ~25-30 minutes with cache
|
||||
**Total (cold start)**: ~45-60 minutes without cache
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always test locally first**
|
||||
```bash
|
||||
cargo test --workspace
|
||||
cargo build --workspace --release
|
||||
cd npm && npm run build
|
||||
```
|
||||
|
||||
2. **Use semantic versioning**
|
||||
- MAJOR.MINOR.PATCH (e.g., 0.1.3)
|
||||
- Breaking changes: bump MAJOR
|
||||
- New features: bump MINOR
|
||||
- Bug fixes: bump PATCH
|
||||
|
||||
3. **Write clear commit messages**
|
||||
```bash
|
||||
feat: Add new vector search capability
|
||||
fix: Resolve memory leak in HNSW index
|
||||
chore: Bump dependencies
|
||||
```
|
||||
|
||||
4. **Review workflow logs**
|
||||
- Check for warnings
|
||||
- Verify all tests passed
|
||||
- Confirm all packages published
|
||||
|
||||
5. **Update CHANGELOG.md**
|
||||
- Document breaking changes
|
||||
- List new features
|
||||
- Mention bug fixes
|
||||
|
||||
## Monitoring and Alerts
|
||||
|
||||
### GitHub Actions Notifications
|
||||
|
||||
1. Go to **Settings → Notifications**
|
||||
2. Enable: "Actions - Only notify for failed workflows"
|
||||
|
||||
### Slack/Discord Integration
|
||||
|
||||
Add webhook to workflow:
|
||||
|
||||
```yaml
|
||||
- name: Notify Slack
|
||||
if: failure()
|
||||
uses: slackapi/slack-github-action@v1
|
||||
with:
|
||||
webhook-url: ${{ secrets.SLACK_WEBHOOK }}
|
||||
payload: |
|
||||
{
|
||||
"text": "Release failed: ${{ github.ref }}"
|
||||
}
|
||||
```
|
||||
|
||||
## Version Management
|
||||
|
||||
### Cargo.toml Versions
|
||||
|
||||
All crates use workspace version:
|
||||
|
||||
```toml
|
||||
[workspace.package]
|
||||
version = "0.1.2"
|
||||
```
|
||||
|
||||
Update once in root `Cargo.toml`, applies to all crates.
|
||||
|
||||
### package.json Versions
|
||||
|
||||
Update independently:
|
||||
- `npm/packages/core/package.json`
|
||||
- `npm/packages/wasm/package.json`
|
||||
- `npm/packages/cli/package.json`
|
||||
|
||||
Or use `npm version`:
|
||||
```bash
|
||||
cd npm/packages/core
|
||||
npm version patch # 0.1.2 -> 0.1.3
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Secrets**: Never log or expose `CARGO_REGISTRY_TOKEN` or `NPM_TOKEN`
|
||||
2. **Branch protection**: Require reviews for version tags
|
||||
3. **Environment protection**: Add reviewers for production environments
|
||||
4. **Dependency scanning**: Enabled via GitHub security features
|
||||
5. **Code signing**: Consider GPG signing for releases
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] Add code signing for native binaries
|
||||
- [ ] Implement changelog generation from commits
|
||||
- [ ] Add performance benchmarks to release notes
|
||||
- [ ] Create Docker images as release artifacts
|
||||
- [ ] Add automatic version bumping
|
||||
- [ ] Implement release candidate (RC) workflow
|
||||
- [ ] Add rollback capabilities
|
||||
- [ ] Create platform-specific installers
|
||||
- [ ] Add integration tests for published packages
|
||||
- [ ] Implement canary releases
|
||||
|
||||
## Support
|
||||
|
||||
- **Issues**: https://github.com/ruvnet/ruvector/issues
|
||||
- **Discussions**: https://github.com/ruvnet/ruvector/discussions
|
||||
- **Documentation**: https://github.com/ruvnet/ruvector
|
||||
|
||||
## License
|
||||
|
||||
This workflow is part of the RuVector project and follows the same MIT license.
|
||||
Reference in New Issue
Block a user