# WebAssembly Architecture ## Overview The Scipix WASM module provides browser-based OCR with LaTeX support through a carefully designed architecture optimizing for performance and developer experience. ## Module Structure ``` src/wasm/ ├── mod.rs # Module entry, initialization ├── api.rs # JavaScript API surface ├── worker.rs # Web Worker support ├── canvas.rs # Canvas/ImageData handling ├── memory.rs # Memory management └── types.rs # Type definitions web/ ├── index.js # JavaScript wrapper ├── worker.js # Worker thread script ├── types.ts # TypeScript definitions ├── example.html # Demo application └── package.json # NPM configuration ``` ## Key Components ### 1. WASM Core (`mod.rs`) Initializes the WASM module with: - Panic hooks for better error messages - Custom allocator (wee_alloc) for smaller binary - Logging infrastructure ```rust #[wasm_bindgen(start)] pub fn init() { console_error_panic_hook::set_once(); tracing_wasm::set_as_global_default(); } ``` ### 2. JavaScript API (`api.rs`) Provides the main `ScipixWasm` class with methods: - Image recognition from various sources - Format configuration - Batch processing - Confidence filtering Uses `wasm-bindgen` for seamless JS interop: ```rust #[wasm_bindgen] pub struct ScipixWasm { ... } #[wasm_bindgen] impl ScipixWasm { #[wasm_bindgen(constructor)] pub async fn new() -> Result { ... } } ``` ### 3. Web Worker Support (`worker.rs`) Enables off-main-thread processing: - Message-based communication - Progress reporting - Batch processing with updates Worker flow: ``` Main Thread Worker Thread │ │ ├──── Init ──────────>│ │<──── Ready ─────────┤ │ │ ├──── Process ───────>│ │<──── Started ───────┤ │<──── Progress ──────┤ │<──── Success ───────┤ ``` ### 4. Canvas Processing (`canvas.rs`) Handles browser-specific image sources: - `HTMLCanvasElement` extraction - `ImageData` conversion - Blob URL loading - Image preprocessing ```rust pub fn extract_canvas_image(&self, canvas: &HtmlCanvasElement) -> Result ``` ### 5. Memory Management (`memory.rs`) Optimizes WASM memory usage: - Efficient buffer allocation - Memory pooling - Automatic cleanup - Shared memory support ```rust pub struct WasmBuffer { data: Vec, } impl Drop for WasmBuffer { fn drop(&mut self) { self.data.clear(); self.data.shrink_to_fit(); } } ``` ## Build Pipeline ### Compilation ```bash # Development build wasm-pack build --target web --dev # Production build wasm-pack build --target web --release ``` ### Optimizations **Cargo.toml settings:** ```toml [profile.release] opt-level = "z" # Optimize for size lto = true # Link-time optimization codegen-units = 1 # Better optimization strip = true # Remove debug symbols panic = "abort" # Smaller panic handler ``` **Result:** ~800KB gzipped bundle ## Data Flow ### Main Thread Processing ``` Image File ↓ FileReader API ↓ Uint8Array ↓ WASM Memory ↓ Image Decode ↓ Preprocessing ↓ OCR Engine ↓ Result (JsValue) ↓ JavaScript ``` ### Worker Thread Processing ``` Main Thread Worker Thread │ │ Image File │ ↓ │ Uint8Array │ ├────────────────────────>│ │ WASM Memory │ ↓ │ OCR Processing │ ↓ │<────────────────── Result ↓ Display ``` ## Memory Layout ### WASM Linear Memory ``` ┌─────────────────────┐ │ Stack │ Growing down ├─────────────────────┤ │ ... │ ├─────────────────────┤ │ Image Buffers │ Pool-allocated ├─────────────────────┤ │ Model Data │ Static ├─────────────────────┤ │ Heap │ Growing up └─────────────────────┘ ``` ### Buffer Management 1. **Acquire** buffer from pool or allocate 2. **Process** image data 3. **Release** buffer back to pool 4. **Cleanup** on drop if pool is full ## Type Safety ### Rust → JavaScript ```rust #[wasm_bindgen] pub struct OcrResult { pub text: String, pub confidence: f32, } ``` Generates: ```javascript export class OcrResult { readonly text: string; readonly confidence: number; } ``` ### TypeScript Definitions Manual definitions in `types.ts` provide: - Full API documentation - IntelliSense support - Type checking - Better DX ## Error Handling ### Rust Side ```rust pub enum ScipixError { ImageProcessing(String), Ocr(String), InvalidInput(String), } impl From for JsValue { fn from(error: ScipixError) -> Self { JsValue::from_str(&error.to_string()) } } ``` ### JavaScript Side ```javascript try { const result = await scipix.recognize(imageData); } catch (error) { console.error('OCR failed:', error.message); } ``` ## Performance Considerations ### 1. Initialization - **Lazy loading**: Only load WASM when needed - **Caching**: Reuse instances - **Singleton pattern**: One shared processor ### 2. Processing - **Streaming**: Process images as they arrive - **Workers**: Parallel processing - **Batching**: Group similar operations ### 3. Memory - **Pooling**: Reuse buffers - **Cleanup**: Explicit disposal - **Monitoring**: Track usage ### 4. Network - **Compression**: Gzip WASM module - **CDN**: Cache static assets - **Prefetch**: Load before needed ## Browser Compatibility ### Required Features - ✅ WebAssembly (97% global support) - ✅ ES6 Modules (96% global support) - ✅ Async/Await (96% global support) - ⚠️ Web Workers (optional, 97% support) - ⚠️ SharedArrayBuffer (optional, 92% support) ### Polyfills Not required for core functionality. Workers are progressive enhancement. ## Security ### Content Security Policy ```html ``` ### Sandboxing WASM runs in browser sandbox: - No file system access - No network access (from WASM) - Memory isolation ## Testing ### Unit Tests ```rust #[cfg(test)] mod tests { use wasm_bindgen_test::*; #[wasm_bindgen_test] async fn test_recognition() { // Test WASM functions } } ``` Run with: ```bash wasm-pack test --headless --firefox ``` ### Integration Tests JavaScript tests using the built module: ```javascript import { createScipix } from './index.js'; test('recognizes text', async () => { const scipix = await createScipix(); const result = await scipix.recognize(testImage); expect(result.text).toBeTruthy(); }); ``` ## Debugging ### Development Mode ```bash RUST_LOG=debug wasm-pack build --dev ``` ### Browser DevTools - Console logging via `tracing_wasm` - Memory profiling - Performance timeline - Network inspection ### Source Maps Enabled in dev builds for Rust source debugging. ## Future Enhancements 1. **Streaming OCR**: Process video frames 2. **Model loading**: Dynamic ONNX models 3. **Caching**: IndexedDB for results 4. **PWA**: Offline support 5. **SIMD**: Use WebAssembly SIMD 6. **Threads**: SharedArrayBuffer parallelism ## References - [wasm-bindgen Guide](https://rustwasm.github.io/wasm-bindgen/) - [web-sys Documentation](https://rustwasm.github.io/wasm-bindgen/api/web_sys/) - [WebAssembly Spec](https://webassembly.github.io/spec/) - [MDN WebAssembly](https://developer.mozilla.org/en-US/docs/WebAssembly)