# ruvector-robotics User Guide ## Table of Contents 1. [Getting Started](#getting-started) 2. [Bridge Module](#bridge-module) 3. [Perception Module](#perception-module) 4. [Cognitive Module](#cognitive-module) 5. [MCP Module](#mcp-module) 6. [Integration Patterns](#integration-patterns) 7. [Advanced Usage](#advanced-usage) --- ## Getting Started ### Installation Add to your `Cargo.toml`: ```toml [dependencies] ruvector-robotics = { path = "crates/ruvector-robotics" } ``` ### Minimal Example ```rust use ruvector_robotics::bridge::{Point3D, PointCloud, SpatialIndex}; use ruvector_robotics::perception::PerceptionPipeline; fn main() { // 1. Create sensor data let points = vec![ Point3D::new(1.0, 0.0, 0.0), Point3D::new(0.0, 1.0, 0.0), Point3D::new(5.0, 5.0, 5.0), ]; let cloud = PointCloud::new(points, 1000); // 2. Index for spatial search let mut index = SpatialIndex::new(3); index.insert_point_cloud(&cloud); // 3. Find nearest obstacles let nearest = index.search_nearest(&[0.0, 0.0, 0.0], 2).unwrap(); println!("Nearest 2 points: {:?}", nearest); // 4. Detect obstacles let pipeline = PerceptionPipeline::default(); let obstacles = pipeline .detect_obstacles(&cloud, [0.0, 0.0, 0.0], 10.0) .unwrap(); println!("Detected {} obstacles", obstacles.len()); } ``` --- ## Bridge Module The bridge module provides core types shared across all robotics subsystems. ### Core Types | Type | Description | Fields | |------|-------------|--------| | `Point3D` | 3D point (f32) | x, y, z | | `PointCloud` | Collection of points | points, intensities, normals, timestamp_us | | `RobotState` | Kinematic state | position, velocity, acceleration, timestamp_us | | `Pose` | 6-DOF pose | position, orientation (Quaternion) | | `SensorFrame` | Synchronized sensor bundle | cloud, state, pose | | `OccupancyGrid` | 2D occupancy map | width, height, resolution, data | | `SceneObject` | Detected object | id, center, extent, confidence, label | | `SceneGraph` | Object relationships | objects, edges | | `Trajectory` | Predicted path | waypoints, timestamps, confidence | ### SpatialIndex A flat brute-force index for nearest-neighbor search: ```rust use ruvector_robotics::bridge::{SpatialIndex, DistanceMetric}; // Create index with cosine distance let mut index = SpatialIndex::with_metric(128, DistanceMetric::Cosine); // Insert vectors index.insert_vectors(&[vec![1.0; 128], vec![0.5; 128]]); // k-NN search let results = index.search_nearest(&vec![0.9; 128], 5).unwrap(); // Radius search let within = index.search_radius(&vec![0.9; 128], 0.5).unwrap(); ``` ### Converters Convert between robotics types and flat vectors: ```rust use ruvector_robotics::bridge::{PointCloud, Point3D, converters}; let cloud = PointCloud::new(vec![Point3D::new(1.0, 2.0, 3.0)], 0); // To vectors for indexing let vecs = converters::point_cloud_to_vectors(&cloud); // -> [[1.0, 2.0, 3.0]] // Back to point cloud let cloud2 = converters::vectors_to_point_cloud(&vecs, 0).unwrap(); ``` --- ## Perception Module ### Obstacle Detection ```rust use ruvector_robotics::perception::{ObstacleDetector, PerceptionConfig}; let config = PerceptionConfig::default(); let detector = ObstacleDetector::new(config.obstacle); // Detect from point cloud let obstacles = detector.detect(&cloud, &[0.0, 0.0, 0.0]); // Classify obstacles let classified = detector.classify_obstacles(&obstacles); for c in &classified { println!("{:?}: {:?} (confidence: {:.2})", c.class, c.obstacle.center, c.confidence); } ``` ### Scene Graph Construction ```rust use ruvector_robotics::perception::{SceneGraphBuilder, PerceptionConfig}; use ruvector_robotics::bridge::SceneObject; let config = PerceptionConfig::default(); let builder = SceneGraphBuilder::new(config.scene_graph); // From point cloud (clusters -> objects -> edges) let graph = builder.build_from_point_cloud(&cloud); // From pre-detected objects let objects = vec![ SceneObject::new(0, [0.0, 0.0, 0.0], [0.5, 0.5, 0.5]), SceneObject::new(1, [3.0, 0.0, 0.0], [0.5, 0.5, 0.5]), ]; let graph = builder.build_from_objects(&objects); ``` ### Full Perception Pipeline ```rust use ruvector_robotics::perception::PerceptionPipeline; let pipeline = PerceptionPipeline::default(); // Obstacle detection + clustering let obstacles = pipeline.detect_obstacles(&cloud, [0.0, 0.0, 0.0], 20.0).unwrap(); // Trajectory prediction let traj = pipeline.predict_trajectory([0.0, 0.0, 0.0], [1.0, 0.0, 0.0], 10, 0.1).unwrap(); // Attention focusing let focused = pipeline.focus_attention(&cloud, [5.0, 5.0, 0.0], 2.0).unwrap(); // Anomaly detection let anomalies = pipeline.detect_anomalies(&cloud).unwrap(); ``` --- ## Cognitive Module ### Behavior Trees Composable reactive control structures: ```rust use ruvector_robotics::cognitive::{BehaviorTree, BehaviorNode, BehaviorStatus}; // Build a patrol tree let root = BehaviorNode::Sequence(vec![ BehaviorNode::Condition("is_battery_ok".into()), BehaviorNode::Selector(vec![ BehaviorNode::Sequence(vec![ BehaviorNode::Condition("obstacle_detected".into()), BehaviorNode::Action("avoid_obstacle".into()), ]), BehaviorNode::Action("move_forward".into()), ]), BehaviorNode::Action("update_map".into()), ]); let mut tree = BehaviorTree::new(root); // Set condition and action states tree.set_condition("is_battery_ok", true); tree.set_condition("obstacle_detected", false); tree.set_action_result("move_forward", BehaviorStatus::Success); tree.set_action_result("update_map", BehaviorStatus::Success); let status = tree.tick(); assert_eq!(status, BehaviorStatus::Success); ``` ### Cognitive Core The central perceive-think-act-learn loop: ```rust use ruvector_robotics::cognitive::{ CognitiveCore, CognitiveConfig, CognitiveMode, Percept, Outcome, }; let config = CognitiveConfig { mode: CognitiveMode::Deliberative, attention_threshold: 0.5, learning_rate: 0.01, max_percepts: 100, }; let mut core = CognitiveCore::new(config); // 1. Perceive let percept = Percept { source: "lidar".into(), data: vec![1.0, 2.0, 3.0], confidence: 0.95, timestamp: 1000, }; core.perceive(percept); // 2. Think -> Decision if let Some(decision) = core.think() { println!("Decision: {} (utility: {:.2})", decision.reasoning, decision.utility); // 3. Act let cmd = core.act(decision); println!("Action: {:?}", cmd.action); // 4. Learn core.learn(Outcome { success: true, reward: 1.0, description: "Obstacle avoided".into(), }); } ``` ### Memory System Three-tier memory architecture: ```rust use ruvector_robotics::cognitive::{WorkingMemory, EpisodicMemory, SemanticMemory, MemoryItem, Episode}; // Working memory (bounded buffer) let mut working = WorkingMemory::new(10); working.add(MemoryItem { key: "obstacle_1".into(), data: vec![1.0, 2.0, 3.0], importance: 0.8, timestamp: 1000, access_count: 0, }); // Episodic memory (experience replay) let mut episodic = EpisodicMemory::new(100); episodic.store(Episode { percepts: vec![vec![1.0, 2.0], vec![3.0, 4.0]], actions: vec!["move".into(), "turn".into()], reward: 1.0, timestamp: 1000, }); let similar = episodic.recall_similar(&[1.0, 2.0], 3); // Semantic memory (concept storage) let mut semantic = SemanticMemory::new(); semantic.store("obstacle", vec![1.0, 0.0, 0.0]); semantic.store("goal", vec![0.0, 1.0, 0.0]); let nearest = semantic.find_similar(&[0.9, 0.1, 0.0], 1); ``` ### Swarm Coordination ```rust use ruvector_robotics::cognitive::{ SwarmCoordinator, SwarmConfig, RobotCapabilities, SwarmTask, Formation, FormationType, }; let mut swarm = SwarmCoordinator::new(SwarmConfig { max_robots: 10, communication_range: 50.0, consensus_threshold: 0.6, }); // Register robots swarm.register_robot(RobotCapabilities { id: 1, max_speed: 2.0, payload: 5.0, sensors: vec!["lidar".into(), "camera".into()], }); // Assign tasks let tasks = vec![SwarmTask { id: 1, description: "Survey area A".into(), location: [10.0, 20.0, 0.0], required_capabilities: vec!["camera".into()], priority: 5, }]; let assignments = swarm.assign_tasks(&tasks); // Compute formation let formation = Formation { formation_type: FormationType::Circle, spacing: 3.0, center: [0.0, 0.0, 0.0], }; let positions = swarm.compute_formation(&formation); ``` --- ## MCP Module ### Tool Registry ```rust use ruvector_robotics::mcp::{RoboticsToolRegistry, ToolCategory}; let registry = RoboticsToolRegistry::new(); // List all 15 tools for tool in registry.list_tools() { println!("{}: {}", tool.name, tool.description); } // Filter by category let perception_tools = registry.list_by_category(ToolCategory::Perception); println!("Perception tools: {}", perception_tools.len()); // Get MCP schema let schema = registry.to_mcp_schema(); println!("{}", serde_json::to_string_pretty(&schema).unwrap()); ``` --- ## Integration Patterns ### Sensor → Perception → Cognition → Action ```rust use ruvector_robotics::bridge::*; use ruvector_robotics::perception::*; use ruvector_robotics::cognitive::*; // 1. Sensor data arrives let cloud = PointCloud::new(/* sensor points */, timestamp); // 2. Perception processes it let pipeline = PerceptionPipeline::default(); let obstacles = pipeline.detect_obstacles(&cloud, robot_pos, 20.0).unwrap(); // 3. Cognitive core makes decisions let mut core = CognitiveCore::new(CognitiveConfig::default()); for obs in &obstacles { core.perceive(Percept { source: "perception".into(), data: obs.position.to_vec(), confidence: obs.confidence as f64, timestamp: 0, }); } // 4. Think and act if let Some(decision) = core.think() { let action = core.act(decision); // Send action to robot motors } ``` ### Multi-Robot Coordination ```rust // Each robot runs its own cognitive core // SwarmCoordinator manages task allocation across robots // ConsensusResult enables group decision-making ``` --- ## Advanced Usage ### Custom Distance Metrics The SpatialIndex supports Euclidean, Cosine, and Manhattan distances. Choose based on your data: - **Euclidean**: Best for spatial point clouds (default) - **Cosine**: Best for high-dimensional feature vectors - **Manhattan**: Best for grid-aligned environments ### Behavior Tree Patterns Common patterns: - **Patrol**: `Sequence[CheckBattery, Selector[AvoidObstacle, MoveForward], UpdateMap]` - **Explore**: `Selector[GoToFrontier, RandomWalk, ReturnToBase]` - **Emergency**: `Sequence[StopMotors, SendAlert, WaitForHelp]` ### Memory Consolidation The three-tier memory system models human memory: - **Working Memory**: Current sensor data, bounded to prevent overload - **Episodic Memory**: Past experiences for pattern matching - **Semantic Memory**: Learned concepts and relationships ### Performance Tuning Key parameters to adjust: - `SceneGraphConfig::cluster_radius` — Smaller = more objects, slower - `ObstacleConfig::safety_margin` — Larger = more conservative - `CognitiveConfig::attention_threshold` — Higher = focus on important percepts - `SwarmConfig::consensus_threshold` — Higher = more agreement required --- ## API Reference Full API documentation is generated with `cargo doc -p ruvector-robotics --open`.