WASM Procedures: Advanced Topics

WASM Procedures: Advanced Topics

Documentation Home > User Guides > Features > Advanced Topics

---

Overview

This guide covers advanced topics for WASM procedures in HeliosDB, including multi-language integration, custom host functions, sandboxing internals, WASM optimization, and CI/CD integration.

Related Sections:

• Language SDKs - SDK documentation
• Host Functions - Available host functions
• Performance - Performance optimization
• Security Model - Security internals

---

Advanced Topics

Multi-Language Integration

Combine multiple languages in a single workflow:

-- Rust procedure for heavy computation
CREATE PROCEDURE calculate_statistics(data BYTEA)
RETURNS JSON
LANGUAGE wasm
AS '/opt/procedures/stats.wasm';  -- Compiled from Rust

-- Python procedure for ML inference
CREATE PROCEDURE predict_churn(user_features JSON)
RETURNS FLOAT
LANGUAGE wasm
AS '/opt/procedures/churn_model.wasm';  -- Compiled from Python

-- JavaScript procedure for JSON transformation
CREATE PROCEDURE transform_data(input JSON)
RETURNS JSON
LANGUAGE wasm
AS '/opt/procedures/transform.wasm';  -- Compiled from JS

-- Orchestrate all three
CREATE PROCEDURE analyze_user_churn(user_id BIGINT)
RETURNS JSON
LANGUAGE sql
AS $$
  WITH user_data AS (
    SELECT behavior_data FROM users WHERE id = user_id
  ),
  statistics AS (
    SELECT calculate_statistics(behavior_data) AS stats
    FROM user_data
  ),
  features AS (
    SELECT transform_data(stats) AS user_features
    FROM statistics
  )
  SELECT json_build_object(
    'churn_probability', predict_churn(user_features),
    'statistics', stats,
    'features', user_features
  )
  FROM features, statistics;
$$;

Custom Host Functions

Extend HeliosDB with custom host functions:

1. Define host function in Rust:

// In HeliosDB server code
pub fn custom_hash(data: &[u8]) -> Vec<u8> {
    // Custom hash implementation
    my_proprietary_hash(data)
}

// Register with WASM linker
linker.func_wrap(
    "heliosdb",
    "custom_hash",
    |caller: Caller<'_, HostState>, data_ptr: i32, data_len: i32, output_ptr: i32| -> i32 {
        // Implementation
    }
)?;

2. Use in WASM procedure:

extern "C" {
    fn custom_hash(data_ptr: *const u8, data_len: usize, output_ptr: *mut u8) -> i32;
}

#[procedure]
pub fn use_custom_hash(input: &str) -> Result<String, String> {
    let data = input.as_bytes();
    let mut output = vec![0u8; 32];

    unsafe {
        custom_hash(data.as_ptr(), data.len(), output.as_mut_ptr());
    }

    Ok(hex::encode(output))
}

Sandboxing Deep Dive

Understanding WASM Memory Model

WASM procedures have isolated linear memory:

┌─────────────────────────────────────┐
│  WASM Linear Memory (Isolated)      │
├─────────────────────────────────────┤
│  Stack (grows down)                 │
│  ↓                                   │
│                                      │
│  Heap (grows up)                    │
│  ↑                                   │
├─────────────────────────────────────┤
│  Data Section (initialized)         │
├─────────────────────────────────────┤
│  Code Section (read-only)           │
└─────────────────────────────────────┘

Host Memory (Inaccessible from WASM)

Procedures cannot:

• Access host memory directly
• Call arbitrary system functions
• Escape sandbox

Security Boundaries

What WASM CAN do:

• Call approved host functions
• Allocate memory within limits
• Perform computation
• Read/write own linear memory

What WASM CANNOT do:

• Access files without capability
• Make network requests without capability
• Read environment variables without capability
• Fork processes
• Execute arbitrary code

Bypass Prevention

HeliosDB prevents sandbox escapes:

1. No FFI: Foreign function interface disabled
2. No dynamic loading: Cannot load external libraries
3. No inline assembly: Assembly code blocked
4. Memory bounds checking: All memory access validated
5. Fuel metering: Prevents infinite loops

WASM Module Optimization

Size Optimization

Rust:

[profile.release]
opt-level = "z"        # Optimize for size
lto = true             # Link-time optimization
codegen-units = 1      # Better optimization
strip = true           # Strip symbols
panic = "abort"        # Smaller panic handler

# Additional optimization with wasm-opt
wasm-opt -Oz input.wasm -o output.wasm

# Aggressive size reduction
wasm-opt -Oz --strip-debug --strip-producers input.wasm -o output.wasm

Results:

Original:              234 KB
Rust optimizations:    156 KB  (33% smaller)
wasm-opt -Oz:           89 KB  (62% smaller)
Gzip compressed:        31 KB  (87% smaller)

Performance Optimization

Enable SIMD:

[dependencies]
packed_simd = "0.3"

use packed_simd::*;

#[procedure]
pub fn sum_array_simd(values: &[f32]) -> f32 {
    let chunks = values.chunks_exact(4);
    let remainder = chunks.remainder();

    let sum = chunks.fold(f32x4::splat(0.0), |acc, chunk| {
        acc + f32x4::from_slice_unaligned(chunk)
    });

    sum.sum() + remainder.iter().sum::<f32>()
}

Benchmark:

scalar code:    125ms
SIMD code:       32ms  (3.9x faster)

CI/CD Integration

GitHub Actions

.github/workflows/wasm-procedures.yml:

name: Build and Deploy WASM Procedures

on:
  push:
    branches: [main]
    paths:
      - 'procedures/**'

jobs:
  build-rust:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Rust
        uses: actions-rust-lang/setup-rust-toolchain@v1
        with:
          toolchain: stable
          target: wasm32-wasi

      - name: Build procedure
        run: |
          cd procedures/discount
          cargo build --target wasm32-wasi --release

      - name: Optimize WASM
        run: |
          npm install -g wasm-opt
          wasm-opt -Oz target/wasm32-wasi/release/discount.wasm \
            -o discount_optimized.wasm

      - name: Run tests
        run: cargo test

      - name: Deploy to database
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
        run: |
          psql $DATABASE_URL <<EOF
            CREATE OR REPLACE PROCEDURE calculate_discount(
              order_total DECIMAL(10,2),
              tier VARCHAR(20)
            )
            RETURNS DECIMAL(10,2)
            LANGUAGE wasm
            SECURITY PROFILE 'standard'
            AS '\$(cat discount_optimized.wasm | base64)';
          EOF

      - name: Verify deployment
        run: |
          psql $DATABASE_URL -c "SELECT calculate_discount(1000.00, 'gold');"

Testing in CI

Rust:

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_calculate_discount() {
        assert_eq!(calculate_discount(1000.0, "gold").unwrap(), 150.0);
    }

    #[test]
    fn test_integration() {
        // Integration test with mock database
        let db = MockDatabase::new();
        let result = my_procedure_with_db(&db);
        assert!(result.is_ok());
    }
}

Run in CI:

- name: Run tests
  run: |
    cargo test
    cargo test --target wasm32-wasi

---

Key Takeaways

1. Multi-Language Workflows: Combine Rust, Python, JavaScript, and Go in complex workflows
2. Custom Host Functions: Extend HeliosDB with proprietary functionality
3. Security Isolation: WASM provides strong sandboxing guarantees
4. Module Optimization: Reduce module size by 87% with proper optimization
5. SIMD Acceleration: 4x performance improvement for data-parallel operations
6. CI/CD Integration: Automate build, test, and deployment of procedures

Advanced Patterns

• Use Rust for performance-critical computation
• Use Python for ML inference and data science
• Use JavaScript for JSON/data transformation
• Use Go for concurrent operations
• Compose procedures from multiple languages
• Implement custom host functions for proprietary algorithms

Next Steps

• Migration Guide - Migrate existing procedures
• API Reference - Complete API documentation
• Conclusion - Summary and resources

---

Navigation: ← Monitoring | Index | Migration Guide →