Skip to content
HeliosDB HeliosDB Full

WebAssembly Stored Procedures for HeliosDB

WebAssembly Stored Procedures for HeliosDB

Executive Summary

HeliosDB’s WebAssembly (WASM) Stored Procedures enable polyglot, sandboxed server-side logic written in any language that compiles to WASM (Rust, Python, JavaScript, Go, C++, etc.). This provides:

  • Language Freedom: Write procedures in your preferred language
  • Security: Sandboxed execution with capability-based security
  • Performance: Near-native execution speed with WASM
  • Portability: Compile once, run anywhere HeliosDB is deployed
  • Future-Proof: Standards-based approach (W3C WebAssembly)

Architecture

High-Level Design

┌─────────────────────────────────────────────────────────────┐
│                     HeliosDB Core                            │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌──────────────────────────────────────────────────────┐  │
│  │          WASM Procedure Manager                       │  │
│  │  - Procedure registry                                 │  │
│  │  - Version management                                 │  │
│  │  - Permission control                                 │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ▲                                   │
│                          │                                   │
│  ┌───────────────────────┴──────────────────────────────┐  │
│  │          WASM Runtime (wasmer/wasmtime)              │  │
│  │  - Module loading and caching                         │  │
│  │  - Memory management                                  │  │
│  │  - Host function imports                              │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ▲                                   │
│                          │                                   │
│  ┌───────────────────────┴──────────────────────────────┐  │
│  │          Host Interface (WASI + HeliosDB API)        │  │
│  │  - Table access (read/write)                          │  │
│  │  - Transaction control                                │  │
│  │  - Logging and metrics                                │  │
│  │  - External API calls (optional)                      │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                               │
└─────────────────────────────────────────────────────────────┘


┌─────────────────────────────────────────────────────────────┐
│                   WASM Procedure Modules                     │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐         │
│  │   Rust      │  │   Python    │  │     Go      │         │
│  │  procedure  │  │  procedure  │  │  procedure  │         │
│  │   (.wasm)   │  │   (.wasm)   │  │   (.wasm)   │         │
│  └─────────────┘  └─────────────┘  └─────────────┘         │
└─────────────────────────────────────────────────────────────┘

Key Components

  1. WASM Runtime: wasmer or wasmtime for executing WASM modules
  2. Procedure Manager: Registry, versioning, and permission control
  3. Host Interface: Safe API for WASM modules to interact with HeliosDB
  4. Language SDKs: Helper libraries for each supported language

Supported Languages

Tier 1 (Fully Supported)

  • Rust: Native WASM support, zero-overhead
  • JavaScript/TypeScript: Via AssemblyScript or QuickJS
  • Python: Via RustPython or Pyodide
  • Go: Via TinyGo

Tier 2 (Community Support)

  • C/C++: Via Emscripten or clang
  • C#/.NET: Via Blazor WebAssembly
  • Ruby: Via ruby.wasm
  • Java: Via TeaVM or JWebAssembly

HeliosDB WASM API

Core Functions

// Host functions available to WASM modules


/// Execute a SQL query
extern "C" fn heliosdb_query(
    query_ptr: *const u8,
    query_len: u32,
    result_ptr: *mut u8,
    result_len: *mut u32,
) -> i32;


/// Get parameter value by index
extern "C" fn heliosdb_get_param(
    index: u32,
    value_ptr: *mut u8,
    value_len: *mut u32,
) -> i32;


/// Return a value to the caller
extern "C" fn heliosdb_return(
    value_ptr: *const u8,
    value_len: u32,
    value_type: u32, // DataType enum
) -> i32;


/// Log a message
extern "C" fn heliosdb_log(
    level: u32, // debug=0, info=1, warn=2, error=3
    message_ptr: *const u8,
    message_len: u32,
) -> i32;


/// Begin a transaction
extern "C" fn heliosdb_begin_tx() -> i32;


/// Commit the current transaction
extern "C" fn heliosdb_commit_tx() -> i32;


/// Rollback the current transaction
extern "C" fn heliosdb_rollback_tx() -> i32;


/// Get current timestamp
extern "C" fn heliosdb_now() -> i64;


/// Raise an error
extern "C" fn heliosdb_error(
    code: i32,
    message_ptr: *const u8,
    message_len: u32,
) -> !;

Memory Management

WASM modules have their own linear memory space:

  • Export memory: WASM module exports its memory to the host
  • Allocation: Host allocates memory in WASM space for large data transfers
  • Serialization: JSON or MessagePack for complex data structures

Example Implementations

1. Rust Procedure

calculate_discount.rs
use heliosdb_wasm_sdk::*;


#[heliosdb_procedure]
fn calculate_discount(customer_id: i64, order_total: f64) -> Result<f64, Error> {
    // Query customer loyalty tier
    let query = format!(
        "SELECT loyalty_tier FROM customers WHERE id = {}",
        customer_id
    );


    let result = heliosdb_query(&query)?;
    let tier: String = result.get_string(0)?;


    // Calculate discount based on tier
    let discount_rate = match tier.as_str() {
        "platinum" => 0.20,
        "gold" => 0.15,
        "silver" => 0.10,
        _ => 0.05,
    };


    let discount = order_total * discount_rate;


    // Log for auditing
    heliosdb_log(
        LogLevel::Info,
        &format!("Applied {} discount: ${:.2}", tier, discount),
    )?;


    Ok(discount)
}


// Compile: cargo build --target wasm32-wasi --release

2. Python Procedure

validate_order.py
from heliosdb_wasm import heliosdb_query, heliosdb_log, heliosdb_error


def validate_order(order_id: int) -> bool:
    """Validate an order has all required items in stock"""


    # Get order items
    query = f"""
        SELECT product_id, quantity
        FROM order_items
        WHERE order_id = {order_id}
    """


    items = heliosdb_query(query)


    # Check inventory for each item
    for item in items:
        product_id = item['product_id']
        quantity = item['quantity']


        inventory_query = f"""
            SELECT available_quantity
            FROM inventory
            WHERE product_id = {product_id}
        """


        inventory = heliosdb_query(inventory_query)
        available = inventory[0]['available_quantity']


        if available < quantity:
            heliosdb_log('warn', f'Insufficient stock for product {product_id}')
            return False


    heliosdb_log('info', f'Order {order_id} validated successfully')
    return True


# Compile: python -m wasm_compiler validate_order.py -o validate_order.wasm

3. JavaScript Procedure

fraud_detection.js
import { heliosdbQuery, heliosdbLog } from 'heliosdb-wasm-sdk';


export function detectFraud(transactionId) {
    // Get transaction details
    const query = `
        SELECT amount, merchant_id, customer_id, timestamp
        FROM transactions
        WHERE id = ${transactionId}
    `;


    const tx = heliosdbQuery(query)[0];


    // Check recent transaction velocity
    const velocityQuery = `
        SELECT COUNT(*) as count
        FROM transactions
        WHERE customer_id = ${tx.customer_id}
        AND timestamp > NOW() - INTERVAL '1 hour'
    `;


    const velocity = heliosdbQuery(velocityQuery)[0].count;


    // Check amount anomaly
    const avgQuery = `
        SELECT AVG(amount) as avg_amount
        FROM transactions
        WHERE customer_id = ${tx.customer_id}
        AND timestamp > NOW() - INTERVAL '30 days'
    `;


    const avgAmount = heliosdbQuery(avgQuery)[0].avg_amount;


    // Fraud detection logic
    let riskScore = 0;


    if (velocity > 10) riskScore += 30;
    if (tx.amount > avgAmount * 3) riskScore += 40;
    if (tx.amount > 10000) riskScore += 20;


    const isFraud = riskScore >= 50;


    heliosdbLog('info', `Transaction ${transactionId} risk score: ${riskScore}`);


    return {
        is_fraud: isFraud,
        risk_score: riskScore,
        reason: riskScore >= 50 ? 'High risk detected' : 'Normal'
    };
}


// Compile: asc fraud_detection.js -o fraud_detection.wasm

4. Go Procedure

data_aggregation.go
package main


import (
    "github.com/heliosdb/wasm-sdk-go"
)


//export aggregate_sales
func aggregateSales(startDate string, endDate string) map[string]interface{} {
    // Query sales data
    query := fmt.Sprintf(`
        SELECT
            DATE(order_date) as date,
            SUM(total) as daily_total,
            COUNT(*) as order_count,
            AVG(total) as avg_order_value
        FROM orders
        WHERE order_date BETWEEN '%s' AND '%s'
        GROUP BY DATE(order_date)
        ORDER BY date
    `, startDate, endDate)


    results := heliosdb.Query(query)


    // Calculate summary statistics
    var totalRevenue float64
    var totalOrders int


    for _, row := range results {
        totalRevenue += row["daily_total"].(float64)
        totalOrders += row["order_count"].(int)
    }


    heliosdb.Log("info", fmt.Sprintf(
        "Aggregated %d orders totaling $%.2f",
        totalOrders, totalRevenue,
    ))


    return map[string]interface{}{
        "total_revenue": totalRevenue,
        "total_orders": totalOrders,
        "avg_order_value": totalRevenue / float64(totalOrders),
        "daily_breakdown": results,
    }
}


func main() {}


// Compile: tinygo build -o data_aggregation.wasm -target=wasi data_aggregation.go

Procedure Management

Creating a Procedure

-- Register a WASM procedure
CREATE PROCEDURE calculate_discount (
    customer_id BIGINT,
    order_total DECIMAL
)
RETURNS DECIMAL
LANGUAGE wasm
AS $wasm$
-- Base64 encoded WASM module or file path
file://procedures/calculate_discount.wasm
$wasm$;

Calling a Procedure

-- Simple call
SELECT calculate_discount(12345, 599.99);


-- In a query
SELECT
    order_id,
    total,
    calculate_discount(customer_id, total) as discount,
    total - calculate_discount(customer_id, total) as final_total
FROM orders
WHERE status = 'pending';

Updating a Procedure

-- Update procedure (creates new version)
ALTER PROCEDURE calculate_discount
VERSION 2
AS $wasm$
file://procedures/calculate_discount_v2.wasm
$wasm$;


-- Rollback to previous version
ALTER PROCEDURE calculate_discount
USE VERSION 1;

Dropping a Procedure

DROP PROCEDURE calculate_discount;

Security Model

Capability-Based Security

Each procedure has an explicit capability list:

CREATE PROCEDURE fraud_detection (transaction_id BIGINT)
RETURNS JSON
LANGUAGE wasm
CAPABILITIES (
    'read:transactions',
    'read:customers',
    'write:fraud_alerts',
    'network:fraud_api.example.com'
)
AS $wasm$
file://procedures/fraud_detection.wasm
$wasm$;

Sandbox Restrictions

  • No filesystem access (unless explicitly granted)
  • No network access (unless explicitly granted with domain whitelist)
  • Limited memory (configurable, default 16MB)
  • CPU time limits (default 10 seconds)
  • No direct system calls

Permission Levels

  1. read:<table>: SELECT access to specific table
  2. write:<table>: INSERT/UPDATE/DELETE access
  3. ddl:<table>: ALTER TABLE access
  4. network:<domain>: HTTPS requests to specific domain
  5. filesystem:<path>: Read/write to specific path
  6. crypto: Access to cryptographic functions

Performance Characteristics

Execution Speed

  • Rust procedures: ~5-10% overhead vs native
  • Python procedures: ~20-30% overhead (RustPython)
  • JavaScript procedures: ~15-25% overhead (QuickJS)
  • Go procedures: ~10-15% overhead (TinyGo)

Memory Usage

  • Base runtime: ~2MB per procedure instance
  • Module cache: Shared across calls, ~100KB per unique module
  • Instance pool: 10 instances per procedure (configurable)

Startup Time

  • First call (cold start): 5-50ms depending on module size
  • Subsequent calls (warm): <1ms (instance pool)
  • Module compilation: Cached on first load

Integration with NL2SQL

Agentic Procedure Calls

The agentic NL2SQL engine can automatically decompose queries that require stored procedure calls:

Natural Language: "Calculate discounts for all platinum customers and show their savings"


Decomposed Plan:
1. [task1] Identify platinum customers
2. [task2] For each customer, call calculate_discount procedure (parallel)
3. [task3] Aggregate and rank by savings

Code Advisor Integration

The Code Advisor can analyze procedure performance:

let advisor = CodeAdvisor::new(llm_client, schema_info);


// Analyze procedure usage
let advice = advisor.analyze_procedure("calculate_discount").await?;


// Recommendations might include:
// - "Cache loyalty tier lookups to reduce queries"
// - "Consider materializing discount rates in a table"
// - "Add index on customers.loyalty_tier"

Development Workflow

1. Write Procedure

Terminal window
# Rust example
cargo new calculate_discount --lib
cd calculate_discount


# Add heliosdb-wasm-sdk dependency
cargo add heliosdb-wasm-sdk


# Write procedure in src/lib.rs

2. Compile to WASM

Terminal window
# Rust
cargo build --target wasm32-wasi --release


# Python
python -m wasm_compiler procedure.py -o procedure.wasm


# JavaScript
asc procedure.js -o procedure.wasm


# Go
tinygo build -target=wasi -o procedure.wasm procedure.go

3. Test Locally

Terminal window
# Run WASM module with test harness
heliosdb-wasm-test procedure.wasm \
    --param customer_id=12345 \
    --param order_total=599.99 \
    --expect 119.998

4. Deploy to HeliosDB

Terminal window
# Upload procedure
heliosdb-cli procedure create \
    --name calculate_discount \
    --file procedure.wasm \
    --params "customer_id:BIGINT,order_total:DECIMAL" \
    --returns DECIMAL \
    --capabilities "read:customers"


# Verify deployment
heliosdb-cli procedure list
heliosdb-cli procedure info calculate_discount

5. Monitor in Production

Terminal window
# View procedure metrics
heliosdb-cli procedure metrics calculate_discount


# View logs
heliosdb-cli logs --procedure calculate_discount --last 1h


# Profile execution
heliosdb-cli procedure profile calculate_discount

SDK Structure

Rust SDK

heliosdb-wasm-sdk/src/lib.rs
pub use heliosdb_wasm_macros::heliosdb_procedure;


pub enum LogLevel {
    Debug,
    Info,
    Warn,
    Error,
}


pub struct QueryResult {
    rows: Vec<serde_json::Value>,
}


impl QueryResult {
    pub fn get_string(&self, index: usize) -> Result<String, Error>;
    pub fn get_i64(&self, index: usize) -> Result<i64, Error>;
    pub fn get_f64(&self, index: usize) -> Result<f64, Error>;
    // ... more type getters
}


pub fn heliosdb_query(sql: &str) -> Result<QueryResult, Error>;
pub fn heliosdb_log(level: LogLevel, message: &str) -> Result<(), Error>;
pub fn heliosdb_begin_tx() -> Result<(), Error>;
pub fn heliosdb_commit_tx() -> Result<(), Error>;
pub fn heliosdb_rollback_tx() -> Result<(), Error>;

Migration Path

Phase 1: Foundation (v5.2)

  • WASM runtime integration (wasmer)
  • Basic host interface (query, log, return)
  • Rust SDK and examples
  • Procedure registry and versioning

Phase 2: Language Support (v5.3)

  • Python SDK (RustPython)
  • JavaScript SDK (QuickJS)
  • Go SDK (TinyGo)
  • Extended host interface (transactions, crypto)

Phase 3: Production Features (v5.4)

  • Capability-based security
  • Instance pooling and caching
  • Performance monitoring
  • Procedure migration tools

Phase 4: Advanced Features (v6.0)

  • Multi-version procedures
  • A/B testing support
  • Distributed procedure execution
  • Procedure marketplace

Benefits

  1. Developer Freedom: Use your preferred language and tools
  2. Performance: Near-native speed with WASM JIT compilation
  3. Security: Sandboxed execution with fine-grained capabilities
  4. Portability: Write once, deploy anywhere HeliosDB runs
  5. Maintainability: Version control, testing, and CI/CD like regular code
  6. Innovation: Rapid development of custom business logic
  7. Ecosystem: Leverage existing libraries and frameworks

Competitive Advantage

HeliosDB would be the first production database to offer:

  • Multi-language stored procedures via WASM
  • Capability-based security model for procedures
  • Hot-swappable procedure versions
  • Agentic integration with NL2SQL

Next Steps

  1. Design document (this document)
  2. ⏳ Implement basic WASM runtime integration
  3. ⏳ Create Rust SDK
  4. ⏳ Build procedure registry and versioning
  5. ⏳ Add Python and JavaScript SDKs
  6. ⏳ Implement capability-based security
  7. ⏳ Create comprehensive examples and tutorials

Status: Design Complete Target Release: v6.0 Owner: Engineering Team Last Updated: 2025-10-27