wifi-densepose/vendor/ruvector/.github/workflows/RELEASE.md

RuVector Release Pipeline Documentation

Overview

The RuVector release pipeline is a comprehensive CI/CD workflow that automates the building, testing, and publishing of Rust crates and npm packages across multiple platforms.

Workflow Files

• release.yml: Main release pipeline workflow
• build-native.yml: Reusable workflow for building native Node.js modules
• validate-lockfile.yml: Validates package-lock.json integrity

Trigger Methods

1. Tag-Based Release (Recommended)

# Create and push a version tag
git tag v0.1.3
git push origin v0.1.3

This automatically triggers the full release pipeline.

2. Manual Workflow Dispatch

Navigate to: Actions → Release Pipeline → Run workflow

Options:

• Version: Version to release (e.g., 0.1.3)
• Skip Tests: Skip validation tests (not recommended)
• Dry Run: Build everything but don't publish

Pipeline Stages

Stage 1: Validation (validate)

Runs on: ubuntu-22.04

Tasks:

• ✅ Check code formatting with cargo fmt
• ✅ Run Clippy lints with all warnings as errors
• ✅ Run Rust test suite across all crates
• ✅ Run npm unit tests
• ✅ Generate validation summary

Skip condition: Set skip_tests: true in manual workflow dispatch

Stage 2: Build Rust Crates (build-crates)

Runs on: ubuntu-22.04

Tasks:

• Build all workspace crates in release mode
• Run crate-specific tests
• Generate build summary with all crate versions

Crates built (26 total):

• Core: ruvector-core, ruvector-metrics, ruvector-filter
• Graph: ruvector-graph, ruvector-gnn
• Distributed: ruvector-cluster, ruvector-raft, ruvector-replication
• Bindings: ruvector-node, ruvector-wasm
• And 16 more specialized crates

Stage 3: Build WASM Packages (build-wasm)

Runs on: ubuntu-22.04

Tasks:

• Install wasm-pack build tool
• Build WASM packages for:
  • ruvector-wasm (core WASM)
  • ruvector-gnn-wasm (graph neural networks)
  • ruvector-graph-wasm (graph database)
  • ruvector-tiny-dancer-wasm (tiny dancer)
• Upload WASM artifacts for later stages

Caching:

• Rust dependencies via Swatinem/rust-cache
• wasm-pack binary

Stage 4: Build Native Modules (build-native)

Runs on: Multi-platform matrix

Reuses: ./.github/workflows/build-native.yml as callable workflow

Platforms built:

• Linux x64 (GNU) - ubuntu-22.04
• Linux ARM64 (GNU) - ubuntu-22.04 with cross-compilation
• macOS x64 (Intel) - macos-13
• macOS ARM64 (Apple Silicon) - macos-14
• Windows x64 (MSVC) - windows-2022

Build matrix details:

- host: ubuntu-22.04, target: x86_64-unknown-linux-gnu
- host: ubuntu-22.04, target: aarch64-unknown-linux-gnu
- host: macos-13, target: x86_64-apple-darwin
- host: macos-14, target: aarch64-apple-darwin
- host: windows-2022, target: x86_64-pc-windows-msvc

Output: Binary artifacts for each platform uploaded to GitHub Actions

Stage 5: Publish Rust Crates (publish-crates)

Runs on: ubuntu-22.04

Requires:

• ✅ Validation passed
• ✅ Build crates succeeded
• 🔑 CARGO_REGISTRY_TOKEN secret configured
• Tag starts with v* OR manual workflow dispatch
• NOT in dry-run mode

Publishing order (respects dependencies):

1. ruvector-core (foundation)
2. ruvector-metrics, ruvector-filter, ruvector-snapshot
3. ruvector-collections, ruvector-router-core
4. ruvector-raft, ruvector-cluster, ruvector-replication
5. ruvector-gnn, ruvector-graph
6. ruvector-server, ruvector-tiny-dancer-core
7. ruvector-router-cli, ruvector-router-ffi, ruvector-router-wasm
8. ruvector-cli, ruvector-bench
9. ruvector-wasm, ruvector-node
10. ruvector-gnn-wasm, ruvector-gnn-node
11. ruvector-graph-wasm, ruvector-graph-node
12. ruvector-tiny-dancer-wasm, ruvector-tiny-dancer-node

Rate limiting: 10 second delay between publishes to avoid crates.io rate limits

Error handling: Continues if a crate already exists (409 error)

Stage 6: Publish npm Packages (publish-npm)

Runs on: ubuntu-22.04

Requires:

• ✅ Validation passed
• ✅ Build native succeeded
• ✅ Build WASM succeeded
• 🔑 NPM_TOKEN secret configured
• Tag starts with v* OR manual workflow dispatch
• NOT in dry-run mode

Publishing order:

1. Platform-specific packages (@ruvector/core-*)
   - @ruvector/core-linux-x64-gnu
   - @ruvector/core-linux-arm64-gnu
   - @ruvector/core-darwin-x64
   - @ruvector/core-darwin-arm64
   - @ruvector/core-win32-x64-msvc

2. @ruvector/wasm (WebAssembly bindings)
3. @ruvector/cli (Command-line interface)
4. @ruvector/extensions (Extensions)
5. @ruvector/core (Main package - depends on platform packages)

Artifact handling:

• Downloads native binaries from build-native job
• Downloads WASM packages from build-wasm job
• Copies to appropriate package directories
• Runs npm ci and npm run build
• Publishes with --access public

Stage 7: Create GitHub Release (create-release)

Runs on: ubuntu-22.04

Requires:

• ✅ All build jobs succeeded
• Tag starts with v* OR manual workflow dispatch

Tasks:

1. Download all artifacts

   • Native binaries for all platforms
   • WASM packages

2. Package artifacts

   • ruvector-native-linux-x64-gnu.tar.gz
   • ruvector-native-linux-arm64-gnu.tar.gz
   • ruvector-native-darwin-x64.tar.gz
   • ruvector-native-darwin-arm64.tar.gz
   • ruvector-native-win32-x64-msvc.tar.gz
   • ruvector-wasm.tar.gz

3. Generate release notes

   • What's new section
   • Package lists (Rust crates and npm)
   • Platform support matrix
   • Installation instructions
   • Links to registries
   • Build metrics

4. Create GitHub release

   • Uses softprops/action-gh-release@v1
   • Attaches packaged artifacts
   • Marks as prerelease if version contains alpha or beta

Stage 8: Release Summary (release-summary)

Runs on: ubuntu-22.04

Always runs: Even if previous jobs fail

Tasks:

• Generate comprehensive status table
• Show success/failure for each job
• Provide next steps and verification links

Required Secrets

CARGO_REGISTRY_TOKEN

Purpose: Publish Rust crates to crates.io

Setup:

1. Go to https://crates.io/settings/tokens
2. Create new token with publish-new and publish-update scopes
3. Add to GitHub: Settings → Secrets → Actions → New secret
   • Name: CARGO_REGISTRY_TOKEN
   • Value: Your crates.io token

NPM_TOKEN

Purpose: Publish npm packages to npmjs.com

Setup:

1. Login to npmjs.com
2. Go to Access Tokens → Generate New Token
3. Select Automation type
4. Add to GitHub: Settings → Secrets → Actions → New secret
   • Name: NPM_TOKEN
   • Value: Your npm token

Environments

The workflow uses GitHub Environments for additional security:

crates-io Environment

• Used for publish-crates job
• Can add required reviewers
• Can add environment-specific secrets

npm Environment

• Used for publish-npm job
• Can add required reviewers
• Can add environment-specific secrets

Setup environments:

1. Go to Settings → Environments
2. Create crates-io and npm environments
3. (Optional) Add required reviewers for production releases

Caching Strategy

Rust Cache

uses: Swatinem/rust-cache@v2
with:
  prefix-key: 'v1-rust'
  shared-key: 'validate|build-crates|wasm'

Caches:

• ~/.cargo/registry
• ~/.cargo/git
• target/ directory

Benefits: 2-5x faster builds

Node.js Cache

uses: actions/setup-node@v4
with:
  cache: 'npm'
  cache-dependency-path: npm/package-lock.json

Caches: ~/.npm directory

Build Matrix

The native build job uses a strategic matrix to cover all platforms:

Platform	Host Runner	Rust Target	NAPI Platform	Cross-Compile
Linux x64	ubuntu-22.04	x86_64-unknown-linux-gnu	linux-x64-gnu	No
Linux ARM64	ubuntu-22.04	aarch64-unknown-linux-gnu	linux-arm64-gnu	Yes (gcc-aarch64)
macOS Intel	macos-13	x86_64-apple-darwin	darwin-x64	No
macOS ARM	macos-14	aarch64-apple-darwin	darwin-arm64	No
Windows	windows-2022	x86_64-pc-windows-msvc	win32-x64-msvc	No

Artifact Retention

• Native binaries: 7 days
• WASM packages: 7 days
• Release packages: Permanent (attached to GitHub release)

Common Scenarios

Regular Release

# 1. Update versions in Cargo.toml files
# 2. Update npm package.json files
# 3. Commit changes
git add .
git commit -m "chore: Bump version to 0.1.3"

# 4. Create and push tag
git tag v0.1.3
git push origin main
git push origin v0.1.3

# 5. Monitor workflow at:
# https://github.com/ruvnet/ruvector/actions/workflows/release.yml

Dry Run (Test Release)

1. Go to Actions → Release Pipeline
2. Click Run workflow
3. Set:
   • Version: 0.1.3-test
   • Dry run: true
4. Click Run workflow

This builds everything but skips publishing.

Emergency Hotfix

# 1. Create hotfix branch
git checkout -b hotfix/critical-fix

# 2. Make fixes
# 3. Bump patch version
# 4. Commit and tag
git commit -m "fix: Critical security patch"
git tag v0.1.3-hotfix.1
git push origin hotfix/critical-fix
git push origin v0.1.3-hotfix.1

# 5. Manually trigger release workflow if needed

Republish Failed Package

If a single npm package fails to publish:

# 1. Check error in workflow logs
# 2. Fix issue locally
# 3. Manually publish that package:
cd npm/packages/wasm
npm publish --access public

# Or trigger just the npm publishing:
# Manually run workflow_dispatch with skip_tests: true

Troubleshooting

Build Failures

Symptom: build-crates job fails

Solutions:

1. Check Rust version compatibility
2. Verify all dependencies are available
3. Look for compilation errors in logs
4. Test locally: cargo build --workspace --release

Publishing Failures

Symptom: publish-crates or publish-npm fails

Solutions:

1. Rate limiting:

   • Wait and re-run workflow
   • Increase delay between publishes

2. Already published:

   • Bump version number
   • Or skip that package (it's already live)

3. Authentication:

   • Verify secrets are set correctly
   • Check token hasn't expired
   • Verify token has correct permissions

4. Dependency issues:

   • Check publishing order
   • Ensure dependencies are published first

Cross-Compilation Issues

Symptom: Linux ARM64 build fails

Solutions:

1. Verify cross-compilation tools installed
2. Check linker configuration
3. Test with: cargo build --target aarch64-unknown-linux-gnu

WASM Build Issues

Symptom: build-wasm job fails

Solutions:

1. Verify wasm-pack installation
2. Check for incompatible dependencies
3. Ensure wasm32-unknown-unknown target installed
4. Test locally: wasm-pack build --target nodejs

Performance Optimization

Parallel Builds

The workflow runs these jobs in parallel:

• build-crates
• build-wasm
• build-native (5 platform builds in parallel)

Total time: ~15-25 minutes (vs. 60+ minutes sequential)

Cache Hit Rates

With proper caching:

• Rust builds: 70-90% cache hit rate
• npm installs: 90-95% cache hit rate

Build Time Breakdown

Job	Uncached	Cached
Validate	8-12 min	3-5 min
Build Crates	15-20 min	5-8 min
Build WASM	10-15 min	4-6 min
Build Native (per platform)	8-12 min	3-5 min
Publish Crates	5-10 min	5-10 min
Publish npm	3-5 min	2-3 min
Create Release	2-3 min	2-3 min

Total (worst case): ~25-30 minutes with cache Total (cold start): ~45-60 minutes without cache

Best Practices

1. Always test locally first

   cargo test --workspace
   cargo build --workspace --release
   cd npm && npm run build
   

2. Use semantic versioning

   • MAJOR.MINOR.PATCH (e.g., 0.1.3)
   • Breaking changes: bump MAJOR
   • New features: bump MINOR
   • Bug fixes: bump PATCH

3. Write clear commit messages

   feat: Add new vector search capability
   fix: Resolve memory leak in HNSW index
   chore: Bump dependencies
   

4. Review workflow logs

   • Check for warnings
   • Verify all tests passed
   • Confirm all packages published

5. Update CHANGELOG.md

   • Document breaking changes
   • List new features
   • Mention bug fixes

Monitoring and Alerts

GitHub Actions Notifications

1. Go to Settings → Notifications
2. Enable: "Actions - Only notify for failed workflows"

Slack/Discord Integration

Add webhook to workflow:

- name: Notify Slack
  if: failure()
  uses: slackapi/slack-github-action@v1
  with:
    webhook-url: ${{ secrets.SLACK_WEBHOOK }}
    payload: |
      {
        "text": "Release failed: ${{ github.ref }}"
      }

Version Management

Cargo.toml Versions

All crates use workspace version:

[workspace.package]
version = "0.1.2"

Update once in root Cargo.toml, applies to all crates.

package.json Versions

Update independently:

• npm/packages/core/package.json
• npm/packages/wasm/package.json
• npm/packages/cli/package.json

Or use npm version:

cd npm/packages/core
npm version patch  # 0.1.2 -> 0.1.3

Security Considerations

1. Secrets: Never log or expose CARGO_REGISTRY_TOKEN or NPM_TOKEN
2. Branch protection: Require reviews for version tags
3. Environment protection: Add reviewers for production environments
4. Dependency scanning: Enabled via GitHub security features
5. Code signing: Consider GPG signing for releases

Future Enhancements

• Add code signing for native binaries
• Implement changelog generation from commits
• Add performance benchmarks to release notes
• Create Docker images as release artifacts
• Add automatic version bumping
• Implement release candidate (RC) workflow
• Add rollback capabilities
• Create platform-specific installers
• Add integration tests for published packages
• Implement canary releases

Support

• Issues: https://github.com/ruvnet/ruvector/issues
• Discussions: https://github.com/ruvnet/ruvector/discussions
• Documentation: https://github.com/ruvnet/ruvector

License

This workflow is part of the RuVector project and follows the same MIT license.