HeliosDB Edge Computing User Guide

Version: 2.0 Last Updated: 2025-12-30 Status: Complete Target Audience: DevOps Engineers, Platform Architects, Edge Application Developers

Introduction
What is Edge Computing?
Architecture Overview
Getting Started with Edge Deployment
Multi-Tier Caching Strategy
Edge Database Synchronization
Conflict Resolution
Monitoring Edge Nodes
Common Deployment Patterns
Performance Optimization
Troubleshooting Guide
Related Documentation

Introduction

HeliosDB Edge Computing transforms your database infrastructure by bringing computation and data closer to where it’s needed—on devices, at network edges, and in remote locations. This guide enables you to deploy, manage, and optimize HeliosDB at the edge with confidence.

Whether you’re building offline-first mobile applications, managing IoT sensor networks, operating remote facilities, or optimizing global content delivery, HeliosDB Edge provides the tools for reliable, high-performance distributed data management.

Key Capabilities

100% Offline Operation: Devices work completely disconnected with local databases
Automatic Synchronization: Seamless bidirectional sync between edge and cloud with conflict resolution
Multi-Tier Caching: L1 (local), L2 (regional), and L3 (global) cache hierarchy for optimal performance
CRDT-Based Conflict Resolution: Mathematically proven conflict-free data types eliminate manual resolution
Resource-Aware Execution: Query optimization for memory-constrained devices
Intelligent Prefetching: ML-based prediction of data access patterns
Geographic Routing: Latency-optimized routing across distributed nodes

Why Deploy at the Edge?

Challenge	Edge Solution
Network Latency	<1ms local queries vs. cloud roundtrip
Connectivity Outages	Full offline operation, sync when reconnected
Bandwidth Constraints	90%+ reduction via delta sync
Privacy Requirements	Keep sensitive data local, sync metadata only
Regulatory Compliance	Data residency compliance simplified
Cost Per User	Reduce cloud compute with edge processing

What is Edge Computing?

Edge computing distributes data and computation from a centralized cloud to the network edges—where data originates and where users are located. HeliosDB brings database capabilities to the edge through a unified embedded+cloud architecture.

Traditional vs. Edge Architecture

Traditional Cloud-Only:

User Device ──(high latency)──> Cloud Database ──> Cloud Analytics
                                    ↑
                            (network roundtrip)

HeliosDB Edge Model:

User Device [Local Database] ──(periodic, optimized sync)──> Cloud Hub
    ↓                                                              ↓
  Fast, offline queries                                    Central analytics
  Work offline                                             Conflict resolution
  Bidirectional sync                                       Master catalog

Edge Computing Benefits in Practice

Instant Response: Local queries return in <1ms vs. 50-500ms cloud roundtrips
Offline Resilience: Mobile app works without internet; sync happens when connected
Bandwidth Efficiency: Transmit only changes, not entire datasets
Privacy Protection: Sensitive data stays on-device; only sync what’s needed
Scalability: 1000s of edge nodes operating independently

Architecture Overview

System Components

HeliosDB Edge consists of several integrated components:

┌─────────────────────────────────────────────────────────────┐
│                    HeliosDB Cloud (Central)                 │
│   Primary data store, conflict coordination, analytics      │
└──────────────┬──────────────────────────────────────────────┘
               │
        ┌──────┴──────┬──────────────┬──────────────┐
        │             │              │              │
   ┌────▼──────┐ ┌───▼──────┐ ┌───▼──────┐ ┌─────▼─────┐
   │  Mobile   │ │ IoT      │ │ Gateway  │ │ Regional  │
   │  Device   │ │ Sensors  │ │ Server   │ │ Hub       │
   │ (Local DB)│ │(Local DB)│ │(Local DB)│ │ (Cache)   │
   └───────────┘ └──────────┘ └──────────┘ └───────────┘

Component Stack per Edge Node

Each edge node runs a complete database stack:

Application Layer
    ↓
HeliosDB Edge SDK (Query API)
    ↓
Embedded Database Engine (DuckDB-compatible)
    ↓
Multi-Tier Cache (L1, L2, L3)
    ↓
Local Storage Layer (RocksDB / SQLite)
    ↓
Sync Manager (Bidirectional)
    ↓
Conflict Resolver (CRDT-based)
    ↓
Network Transport (TCP/TLS)
    ↓
Cloud Connection

Key Architectural Patterns

Four-Tier Cache Hierarchy:

L1 Cache: Hot data in process memory (sub-millisecond access)
L2 Cache: Edge local SSD/storage (single-digit millisecond access)
L3 Cache: Regional aggregate cache (multi-region latency)
L4 Cache: Cloud primary store (source of truth)

Sync Architecture:

Delta-based synchronization (only changed records sync)
Vector clocks track causal relationships
Automatic conflict detection via CRDT operations
Configurable sync scheduling (real-time, periodic, or manual)

Getting Started with Edge Deployment

Prerequisites

Hardware Requirements:

Minimum: 64MB RAM (embedded database core)
Recommended: 256MB+ RAM (for effective caching)
Storage: 100MB to 10GB (depends on dataset and offline queue depth)
Network: Periodic connectivity required (can work 100% offline)

Software Requirements:

Linux, macOS, Windows, iOS, Android, or custom RTOS
HeliosDB Edge SDK (Rust library or language binding)
Network connectivity for initial provisioning and periodic sync

Step 1: Configure Edge Node

Create an edge configuration for your deployment:

use heliosdb_edge::{EdgeConfig, CacheConfig};

let config = EdgeConfig {
    // Unique identifier for this edge node
    node_id: "retail-store-42".to_string(),

    // Cloud endpoint for synchronization
    cloud_endpoint: Some("https://heliosdb.company.com".to_string()),

    // Maximum data stored locally (10 GB)
    max_storage_bytes: 10 * 1024 * 1024 * 1024,

    // How frequently to attempt sync (in seconds)
    sync_interval: 60,  // Sync every minute when online

    // Enable offline mode (automatic connectivity detection)
    offline_mode: true,

    // Cache configuration
    cache_config: CacheConfig {
        max_size: 512 * 1024 * 1024,  // 512 MB L1 cache
        eviction_policy: EvictionPolicy::LRU,
        prefetch_enabled: true,
        ..Default::default()
    },
};

Step 2: Initialize Edge Engine

Start the edge database engine:

use heliosdb_edge::EdgeEngine;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create and start the edge engine
    let mut engine = EdgeEngine::new(config);
    engine.start().await?;

    println!("Edge node started: {}", config.node_id);

    // Keep running until shutdown
    tokio::signal::ctrl_c().await?;
    engine.stop().await?;

    Ok(())
}

Step 3: Execute Local Queries

Once running, your application queries the local database:

// This query runs locally with <1ms latency, no network required
let results = engine.query(
    "SELECT customer_id, total_spent FROM sales WHERE date > '2025-01-01'"
).await?;

// Insert data locally (queued for sync)
engine.insert("sales", vec![
    ("customer_id", "C123"),
    ("amount", "99.99"),
    ("timestamp", "2025-01-15T10:30:00Z"),
]).await?;

// Update operations also work offline
engine.update("inventory",
    "location = 'store-42'",
    vec![("quantity", "50")]
).await?;

Step 4: Configure Synchronization

Set up sync parameters matching your use case:

use heliosdb_edge::{SyncConfig, ConflictStrategy};

let sync_config = SyncConfig {
    // Strategy for handling conflicts
    conflict_strategy: ConflictStrategy::LastWriteWins,

    // Enable bidirectional sync (edge ↔ cloud)
    bidirectional: true,

    // Compression for bandwidth savings
    compression: Some("lz4".to_string()),

    // Maximum operations to queue offline
    max_offline_queue: 10000,

    // Sync timeout in milliseconds
    sync_timeout_ms: 5000,

    ..Default::default()
};

engine.configure_sync(sync_config).await?;

Step 5: Monitor Edge Health

Continuously monitor your edge nodes:

loop {
    // Check node health every 30 seconds
    let health = engine.health().await?;

    println!("Node: {}", health.node_id);
    println!("Status: {:?}", health.state);
    println!("Local Storage: {:.1}% full", health.storage_percent);
    println!("Sync Status: {:?}", health.last_sync);

    if health.storage_percent > 90.0 {
        eprintln!("WARNING: Storage nearly full on {}", health.node_id);
    }

    tokio::time::sleep(Duration::from_secs(30)).await;
}

Multi-Tier Caching Strategy

Effective caching is crucial for edge performance. HeliosDB implements a sophisticated four-tier hierarchy:

L1 Cache: Process Memory

Purpose: Ultra-fast access to hot data Location: Application process memory Latency: <1 millisecond Size: 64-512 MB (depends on available RAM)

// L1 cache automatically stores frequently accessed data
let config = CacheConfig {
    l1_max_size: 256 * 1024 * 1024,  // 256 MB
    l1_eviction_policy: EvictionPolicy::LRU,  // Least Recently Used
    ..Default::default()
};

Best for:

Customer profiles accessed every second
Current inventory levels
Real-time metrics and counters

L2 Cache: Local Storage (SSD/Flash)

Purpose: Medium-speed access to working dataset Location: Local device SSD or persistent storage Latency: 5-50 milliseconds Size: 1-10 GB (limited by device storage)

// L2 cache uses local SSD/storage
let config = CacheConfig {
    l2_max_size: 5 * 1024 * 1024 * 1024,  // 5 GB
    l2_location: "/var/heliosdb/edge-cache".to_string(),
    ..Default::default()
};

Best for:

Historical transactions from today
Product catalogs and reference data
Offline transaction queue

L3 Cache: Regional Aggregate

Purpose: Shared cache across regional edge nodes Location: Nearby regional hub (within same country/region) Latency: 10-100 milliseconds Size: 100 GB to 1 TB (shared infrastructure)

// L3 cache routes through regional hub
let geo_routing = GeoRouter::new();
geo_routing.register_region("us-east", "hub-nyc:6379");
geo_routing.register_region("eu-west", "hub-dublin:6379");

let cache = MultiTierCache::new(l1, l2, l3);

Best for:

Aggregated analytics (daily summaries)
Reference data shared across stores
Machine learning models for inference

L4 Cache: Cloud Primary Store

Purpose: Authoritative data source and analytics Location: Central cloud data warehouse Latency: 100-500 milliseconds Size: Unlimited (cloud-scale)

Best for:

Data that never changes (historical archives)
Centralized analytics and reporting
Compliance and audit logs

Cache Strategy Recommendations

For Retail POS Systems:

CacheConfig {
    l1_max_size: 512 * 1024 * 1024,      // Hot customers & products
    l2_max_size: 5 * 1024 * 1024 * 1024, // Daily transactions
    l3_enabled: true,                     // Share across stores
    prefetch_enabled: true,               // Predict next accessed data
    prefetch_model: "seasonal",           // Learn seasonal patterns
}

For IoT Sensor Networks:

CacheConfig {
    l1_max_size: 64 * 1024 * 1024,        // Recent readings
    l2_max_size: 500 * 1024 * 1024,       // Last 24 hours
    l3_enabled: false,                    // No inter-device sharing
    prefetch_enabled: false,              // Deterministic patterns
}

For Mobile Applications:

CacheConfig {
    l1_max_size: 128 * 1024 * 1024,       // User data + active session
    l2_max_size: 1 * 1024 * 1024 * 1024,  // Downloaded content
    compression: true,                    // Save bandwidth
    prefetch_enabled: true,               // Learn user behavior
}

Edge Database Synchronization

Synchronization is the heart of edge computing. HeliosDB automatically keeps edge nodes and the cloud in sync while handling network interruptions and conflicts.

Synchronization Architecture

Delta-Based Sync (not full dataset sync):

Only changed records transmit to cloud
Saves 90%+ bandwidth compared to full sync
Changes tracked via delta log

Vector Clocks (track causality):

Each node maintains logical clock
Detects which updates happened before others
Prevents inconsistent state

CRDT Operations (conflict-free):

Mathematically proven to resolve conflicts
No manual conflict resolution needed
Always consistent across nodes

Sync Flow

Local Write        Sync Buffer         Cloud
   │                   │                  │
   ├─> Insert         │                  │
   │   Update    ─>   Queue  ─(delta)─> Merge
   │   Delete         │                  │
   │                  │        ←─(ack)─ Process
   └─ Acknowledge    │                  │
      (immediate)     │             Broadcast back
                      ←──────────────────┘

Configuring Sync Behavior

Real-Time Sync (for critical data):

// Every change syncs immediately
let sync_config = SyncConfig {
    sync_mode: SyncMode::RealTime,
    batch_size: 1,  // One change per sync
    ..Default::default()
};

Periodic Sync (for cost optimization):

// Sync every minute (batches multiple changes)
let sync_config = SyncConfig {
    sync_mode: SyncMode::Periodic,
    sync_interval_secs: 60,
    batch_size: 100,  // Bundle up to 100 changes
    ..Default::default()
};

Bandwidth-Aware Sync (for unstable networks):

// Sync only when network is good
let sync_config = SyncConfig {
    sync_mode: SyncMode::Adaptive,
    max_sync_size_kb: 1024,  // Max 1 MB per sync
    network_quality_threshold: 0.8,  // Only on good networks
    ..Default::default()
};

Manual Sync (for control):

// Application controls when sync happens
let sync_config = SyncConfig {
    sync_mode: SyncMode::Manual,
    ..Default::default()
};

// Later, manually trigger sync
engine.sync().await?;

Offline Queue Management

When a device loses connectivity, writes queue locally:

// Application writes normally (queued locally if offline)
for item in items {
    engine.insert("orders", item_to_row(item)).await?;
}

// Check queue depth
let queue_status = engine.get_sync_queue_status().await?;
println!("Queued operations: {}", queue_status.pending_count);
println!("Queue size: {} MB", queue_status.size_bytes / 1024 / 1024);

// Later, when online, sync automatically
let result = engine.sync().await?;
println!("Synced {} operations", result.operations_synced);

Sync Compression

HeliosDB automatically compresses sync payloads to reduce bandwidth:

let sync_config = SyncConfig {
    compression: Some(CompressionAlgorithm::LZ4),  // Fast, good ratio
    // or
    compression: Some(CompressionAlgorithm::Zstd),  // High ratio, slower

    ..Default::default()
};

// Monitor compression statistics
let stats = engine.get_sync_stats().await?;
println!("Bandwidth saved: {:.1}%", stats.compression_ratio * 100.0);
println!("Bytes transmitted: {}", stats.bytes_transmitted);
println!("Bytes before compression: {}", stats.bytes_uncompressed);

Conflict Resolution

When edge nodes and cloud diverge (e.g., two stores update same inventory), conflicts must resolve. HeliosDB uses CRDTs (Conflict-free Replicated Data Types) for automatic, consistent resolution.

Understanding Conflicts

Scenario: Inventory management with two stores

Time    Store A              Store B              Cloud
─────────────────────────────────────────────────────────
0:00    Inventory: 100       Inventory: 100       Inventory: 100
        (Both online)
0:05    Lost connection
0:10                         Sale: -20            (disconnected)
        Sale: -30            (Queue locally)
        (Queue locally)
1:00    Reconnected          Reconnected
        Change: 70           Change: 80           Conflict!

Store A and B both modified the same value while offline. How does HeliosDB resolve?

Conflict Strategies

Last Write Wins (LWW) - Default:

ConflictStrategy::LastWriteWins
// Whoever synced last wins
// Example: Store B's update (80) overwrites Store A's (70)
// Final: 80 units

First Write Wins (FWW):

ConflictStrategy::FirstWriteWins
// Whoever synced first wins
// Example: Store A's update (70) is kept
// Final: 70 units

Maximum Value:

ConflictStrategy::MaxValue
// Keep the larger value (good for counters, metrics)
// Example: max(70, 80) = 80
// Final: 80 units

Minimum Value:

ConflictStrategy::MinValue
// Keep the smaller value (good for thresholds, limits)
// Example: min(70, 80) = 70
// Final: 70 units

Custom Conflict Resolver:

ConflictStrategy::Custom(Box::new(|a, b| {
    // Your business logic
    if a.source == "store_priority" {
        a
    } else {
        b
    }
}))
// Final: Application-specific resolution

CRDT Data Types

HeliosDB provides seven CRDT types for conflict-free operations:

1. G-Counter (Grow-Only Counter)

// Can only increment, never conflicts
counter.increment(5);  // Always safe, even offline
// Result: counters always sum correctly

2. PN-Counter (Positive-Negative Counter)

// Can increment and decrement, no conflicts
counter.increment(5);   // +5
counter.decrement(3);   // -3
// Result: always consistent (final: +2)

3. OR-Set (Observed-Remove Set)

// Set with add/remove semantics
set.add("item1");
set.remove("item1");
set.add("item1");       // Can re-add after remove
// Result: all operations commute, always consistent

4. LWW-Register (Last-Write-Wins Register)

// Single value with timestamp
register.write("value1");
register.write("value2");  // Overwrites value1
// Result: highest timestamp wins

5. RGA (Replicated Growable Array)

// Ordered list that handles concurrent inserts
list.insert(0, "A");
list.insert(1, "B");
// Result: always consistent even with concurrent inserts

6. LSeq (List CRDT with Tombstones)

// Ordered list with tombstone deletion
list.insert(0, "A");
list.insert(1, "B");
list.delete(0);        // Tombstone, not hard delete
// Result: concurrent insertions and deletions work

7. ORMap (Observed-Remove Map)

// Dictionary with remove semantics
map.insert("key1", value);
map.remove("key1");
map.insert("key1", new_value);  // Can re-insert after remove
// Result: consistent even with concurrent modifications

Selecting the Right Strategy

For Inventory Systems (numbers must be accurate):

// Use Max/Min depending on meaning
// Stock quantity → min (don't oversell)
// Price override → max (highest value wins)
ConflictStrategy::MinValue  // For quantities

For Customer Records (changes to name, address):

// Use Last-Write-Wins (most recent is most valid)
ConflictStrategy::LastWriteWins

For Metrics/Analytics (cumulative counts):

// Use G-Counter (grows without conflicts)
use_crdt_type: CrdtType::GCounter

For Append-Only Logs (events, audit trails):

// Use RGA or LSeq (order matters, no deletions/overwrites)
use_crdt_type: CrdtType::RGA

Monitoring Edge Nodes

Production edge deployments require comprehensive monitoring to detect issues before users are impacted.

Key Metrics to Monitor

1. Sync Health

let sync_stats = engine.get_sync_stats().await?;

println!("Last sync: {:?}", sync_stats.last_sync_time);
println!("Sync success rate: {:.1}%", sync_stats.success_rate * 100.0);
println!("Pending operations: {}", sync_stats.pending_operations);
println!("Queued size: {} MB", sync_stats.queue_size_bytes / 1024 / 1024);
println!("Conflicts resolved: {}", sync_stats.conflicts_resolved);

// Alert if queue is growing (more writes than syncs)
if sync_stats.queue_size_bytes > 500 * 1024 * 1024 {
    alert!("Queue size exceeds 500 MB on {}", node_id);
}

2. Cache Performance

let cache_stats = engine.get_cache_stats().await?;

println!("Cache hit rate: {:.1}%", cache_stats.hit_rate * 100.0);
println!("L1 utilization: {:.1}%", cache_stats.l1_percent_used);
println!("L2 utilization: {:.1}%", cache_stats.l2_percent_used);
println!("Evictions/sec: {}", cache_stats.evictions_per_second);

// Alert if hit rate drops
if cache_stats.hit_rate < 0.7 {
    alert!("Cache hit rate below 70% on {}", node_id);
}

3. Resource Usage

let resources = engine.get_resource_usage().await?;

println!("Memory: {} MB / {} MB",
    resources.memory_used_mb,
    resources.memory_total_mb);
println!("Storage: {} MB / {} MB",
    resources.storage_used_mb,
    resources.storage_total_mb);
println!("CPU: {:.1}%", resources.cpu_percent);

// Alert if storage nearing capacity
let storage_percent = (resources.storage_used_mb as f32
    / resources.storage_total_mb as f32) * 100.0;
if storage_percent > 85.0 {
    alert!("Storage {:.0}% full on {}", storage_percent, node_id);
}

4. Network Connectivity

let network = engine.get_network_status().await?;

println!("Connected: {}", network.is_connected);
println!("Latency to cloud: {} ms", network.latency_ms);
println!("Bandwidth available: {} Mbps", network.bandwidth_mbps);
println!("Last heartbeat: {:?}", network.last_heartbeat);

// Alert if disconnected for too long
if !network.is_connected {
    if network.offline_duration_secs > 3600 {
        alert!("Node {} offline for 1+ hours", node_id);
    }
}

Setting Up Alerts

use heliosdb_edge::AlertConfig;

let alerts = AlertConfig {
    // Sync alerts
    sync_failure_threshold: 3,      // 3 consecutive failures
    queue_size_mb: 500,             // Queue exceeds 500 MB

    // Cache alerts
    cache_hit_rate_threshold: 0.7,  // Below 70%

    // Resource alerts
    memory_percent_threshold: 90,   // Above 90%
    storage_percent_threshold: 85,  // Above 85%

    // Connectivity alerts
    offline_duration_secs: 3600,    // Offline for 1+ hour

    // Custom alert handler
    alert_handler: |alert| {
        eprintln!("ALERT: {:?}", alert);
        // Send to monitoring system (Prometheus, DataDog, etc.)
    },
};

engine.configure_alerts(alerts).await?;

Integration with Monitoring Systems

Prometheus Metrics Export:

// HeliosDB exposes Prometheus metrics on :9090/metrics
// Scrape with:
// scrape_configs:
//   - job_name: 'heliosdb-edge'
//     static_configs:
//       - targets: ['localhost:9090']

Health Check Endpoint:

# HTTP health check (every 30 seconds)
curl http://edge-node:8080/health
# Returns 200 OK if healthy, 503 if degraded

# Detailed health status
curl http://edge-node:8080/health/detailed
# Returns JSON with sync, cache, resource details

Common Deployment Patterns

Pattern 1: Retail POS System

Scenario: 500 retail stores, 50 registers per store, single cloud HQ

HQ Cloud (Master)
    ↓
┌───┴────┬─────────┬──────────┐
│        │         │          │
Store 1  Store 2   Store 3    Store N
├─────┐  ├─────┐  ├─────┐
Reg 1 │  Reg 1│  Reg 1│
Reg 2 │  Reg 2│  Reg 2│
Reg 3 │  Reg 3│  Reg 3│

Configuration:

EdgeConfig {
    node_id: format!("store-{}", store_number),
    cloud_endpoint: Some("https://pos.hq.com".into()),
    sync_interval: 60,          // Sync every minute
    offline_mode: true,         // Works when internet down
    cache_config: CacheConfig {
        l1_max_size: 256 * 1024 * 1024,  // Hot inventory
        l2_max_size: 2 * 1024 * 1024 * 1024,  // Today's transactions
        prefetch_enabled: true,  // Predict next lookups
        ..Default::default()
    },
}

Key Benefits:

Registers work during internet outages (critical for retail)
Instant checkout queries (<10ms vs. 200ms cloud)
90%+ bandwidth savings via delta sync
Automatic reconciliation via CRDT

Operational Runbook:

Daily: Monitor sync queue size (should stay <100 MB)
Weekly: Review cache hit rates, adjust prefetch model
Monthly: Archive old transactions to cloud, free local storage

Pattern 2: IoT Sensor Network

Scenario: 10,000 sensors across factory floor, 1 gateway per section

Cloud Analytics
    ↓
Central Gateway (High Performance)
    ├─ Region 1 Hub
    │  ├─ 100 sensors
    │  └─ 20 sensors
    └─ Region 2 Hub
       ├─ 150 sensors
       └─ 50 sensors

Configuration:

// For sensors (minimal resources)
EdgeConfig {
    node_id: format!("sensor-{}", sensor_id),
    cloud_endpoint: Some("gateway.local:5432".into()),
    sync_interval: 300,  // Sync every 5 minutes
    offline_mode: true,
    cache_config: CacheConfig {
        l1_max_size: 64 * 1024 * 1024,   // Recent readings
        l2_max_size: 256 * 1024 * 1024,  // Last 24 hours
        prefetch_enabled: false,  // Deterministic patterns
        ..Default::default()
    },
}

// For gateway hub (higher resources)
EdgeConfig {
    node_id: "gateway-region-1".to_string(),
    cloud_endpoint: Some("analytics.cloud.com".into()),
    sync_interval: 30,
    cache_config: CacheConfig {
        l1_max_size: 1 * 1024 * 1024 * 1024,   // Aggregations
        l2_max_size: 10 * 1024 * 1024 * 1024,  // Daily summaries
        prefetch_enabled: true,
        ..Default::default()
    },
}

Key Benefits:

Sensors work autonomously, report periodically
Gateway aggregates before sending to cloud (bandwidth savings)
Works with intermittent gateway connectivity
Automatic time-series data deduplication

Pattern 3: Mobile App with Offline Support

Scenario: Consumer mobile app (iOS/Android), 1M users, 5M DAU

Cloud API
    ↓
Mobile App
├─ Local SQLite
├─ L1 Cache (Hot user data)
├─ Sync Queue (Pending changes)
└─ Conflict Resolution (Automatic)

Configuration:

EdgeConfig {
    node_id: format!("user-{}-device-{}", user_id, device_id),
    cloud_endpoint: Some("api.app.com".into()),
    sync_interval: 60,
    offline_mode: true,
    max_storage_bytes: 1 * 1024 * 1024 * 1024,  // 1 GB per app
    cache_config: CacheConfig {
        l1_max_size: 128 * 1024 * 1024,  // User's data
        l2_max_size: 256 * 1024 * 1024,  // Downloaded content
        compression: true,  // Reduce app size
        prefetch_enabled: true,  // Learn user behavior
        ..Default::default()
    },
}

Key Benefits:

App works with no internet (full offline mode)
Seamless sync when connected (batched changes)
Bandwidth-aware sync (don’t use cellular for large syncs)
Conflict-free updates (concurrent edits reconcile)

Implementation Example:

// User creates todo while offline
let todo = Todo {
    id: uuid::Uuid::new_v4(),
    title: "Buy milk".to_string(),
    created_at: Utc::now(),
    ..Default::default()
};

// App writes locally immediately
engine.insert("todos", todo_to_row(&todo)).await?;
app.update_ui_with_new_todo(&todo);

// App syncs when online (user doesn't wait)
tokio::spawn(async {
    if let Ok(result) = engine.sync().await {
        // Conflicts handled automatically
        println!("Synced {} changes", result.operations_synced);
    }
});

Performance Optimization

Achieving optimal edge performance requires understanding bottlenecks and applying targeted optimizations.

Query Optimization for Edge

1. Use Indexes Effectively

// Create indexes on frequently queried columns
engine.create_index("customers", "email", IndexType::Hash).await?;
engine.create_index("orders", "customer_id", IndexType::BTree).await?;

// Queries now scan index instead of full table
// Before: 500ms (full table scan)
// After: 5ms (index lookup)

2. Limit Result Sets

// BAD: Fetch all records, filter in app
let all_orders = engine.query("SELECT * FROM orders").await?;
let today = all_orders.iter()
    .filter(|o| is_today(o.created_at))
    .collect::<Vec<_>>();

// GOOD: Filter in database
let today = engine.query(
    "SELECT * FROM orders WHERE created_at > now() - interval '1 day'"
).await?;

3. Use Aggregation Pushdown

// BAD: Fetch and aggregate in app
let records = engine.query("SELECT * FROM metrics").await?;
let sum: u64 = records.iter().map(|r| r.value).sum();

// GOOD: Database does aggregation
let result = engine.query("SELECT SUM(value) FROM metrics").await?;
let sum = result[0].get::<u64>("sum");

Cache Tuning

1. Monitor and Adjust L1 Size

let stats = engine.get_cache_stats().await?;

// If hit rate > 95%: Working set fits in L1, don't increase
// If hit rate 80-95%: Consider +20% to L1
// If hit rate < 80%: May be working set too large OR access patterns random

2. Adjust Eviction Policies

// LRU (Least Recently Used) - good for temporal locality
cache_config.l1_eviction: EvictionPolicy::LRU

// LFU (Least Frequently Used) - good for popularity
cache_config.l1_eviction: EvictionPolicy::LFU

// FIFO (First In First Out) - good for time-windowed data
cache_config.l1_eviction: EvictionPolicy::FIFO

3. Enable Prefetching Strategically

// Prefetching helps when:
// - Access patterns are predictable
// - Memory available for prefetch buffer
// - Network cost of prefetch < benefit of cache hit

cache_config.prefetch_enabled = true;
cache_config.prefetch_model = "seasonal";  // Learn seasonal patterns

// Disable prefetching when:
// - Access patterns random (mobile app usage)
// - Memory constrained
// - Network bandwidth precious

Bandwidth Optimization

1. Compress Sync Payloads

// Enable compression for outbound sync
sync_config.compression = Some(CompressionAlgorithm::LZ4);

// Before: 10 MB uncompressed
// After: 1 MB compressed (90% savings)
// Tradeoff: CPU to compress/decompress vs. bandwidth

2. Delta Sync Strategy

// Delta sync (default): Only changed records
// - 90%+ bandwidth savings
// - Requires more CPU (diff calculation)

// Full sync: All records
// - 10-100% more bandwidth
// - Less CPU overhead

// Adaptive: Choose based on network
sync_config.adaptive_sync = true;

3. Batch Changes

// Sync individual changes (not recommended)
for item in items {
    engine.insert("table", item).await?;
    engine.sync().await?;  // Sync after each insert!
    // Network overhead: 5x larger than data
}

// BETTER: Batch before sync
let mut batch = Vec::new();
for item in items {
    batch.push(item);
}
// Batch insert all at once
for item in batch {
    engine.insert("table", item).await?;
}
engine.sync().await?;  // Single sync for whole batch

Storage Optimization

1. Compression at Rest

cache_config.storage_compression = Some(CompressionAlgorithm::Zstd);

// Reduces L2 cache size: 5GB → 1.5GB
// Tradeoff: Read/write latency +5-10ms

2. Data Lifecycle Management

// Delete old data periodically
engine.delete_older_than(
    "events",
    Duration::from_secs(30 * 24 * 60 * 60)  // 30 days old
).await?;

// Archive to cloud
let archived = engine.export_range(
    "transactions",
    "2025-01-01",
    "2025-01-31"
).await?;
cloud_api.archive(archived).await?;

// Free local storage for new data
engine.delete_between("transactions", "2025-01-01", "2025-01-31").await?;

Troubleshooting Guide

Problem: Sync Queue Growing Unbounded

Symptoms:

Queue size increasing over time
Storage usage climbing
Sync operations not completing

Root Causes:

Network connectivity issues (can’t reach cloud)
Cloud endpoint misconfigured or unreachable
Firewall blocking outbound traffic
Sync process stuck or deadlocked

Diagnosis:

let sync_status = engine.get_sync_queue_status().await?;
println!("Queue size: {} MB", sync_status.size_bytes / 1024 / 1024);
println!("Pending operations: {}", sync_status.pending_count);
println!("Last successful sync: {:?}", sync_status.last_success);

let network = engine.get_network_status().await?;
println!("Connected: {}", network.is_connected);
println!("Latency: {} ms", network.latency_ms);

Solutions:

// 1. Check cloud endpoint reachability
let reachable = engine.test_cloud_connection().await?;
if !reachable {
    eprintln!("Cloud endpoint unreachable, check firewall");
    // Open firewall: cloud_endpoint_port (default 5432)
}

// 2. Force sync retry
engine.sync().await?;

// 3. Increase sync timeout
let mut sync_config = engine.get_sync_config().await?;
sync_config.timeout_ms = 30000;  // 30 seconds
engine.configure_sync(sync_config).await?;

// 4. Reduce batch size for slower networks
sync_config.batch_size = 10;  // Down from 100
engine.configure_sync(sync_config).await?;

// 5. If queue exceeds limit, emergency wipe (last resort)
// WARNING: Loses unsynced changes
engine.clear_sync_queue_emergency().await?;

Problem: Cache Hit Rate Below 50%

Symptoms:

Most queries hitting L2/L3, not L1
High latency (50+ ms vs. expected <5ms)
High CPU usage (decompression)

Root Causes:

Cache size too small for working set
Access patterns too random (no locality)
Eviction policy wrong for workload
Prefetch not working

Diagnosis:

let stats = engine.get_cache_stats().await?;
println!("Hit rate: {:.1}%", stats.hit_rate * 100.0);
println!("L1 utilization: {:.1}%", stats.l1_percent_used);
println!("L2 utilization: {:.1}%", stats.l2_percent_used);
println!("Most evicted keys: {:?}", stats.top_evicted_keys);

// Check working set size
let access_patterns = engine.analyze_access_patterns().await?;
println!("Working set size: {} MB", access_patterns.working_set_mb);
println!("Hot keys: {}", access_patterns.hot_keys_count);
println!("Cold keys: {}", access_patterns.cold_keys_count);

Solutions:

// 1. Increase L1 cache size
cache_config.l1_max_size = 512 * 1024 * 1024;  // Double from 256MB
engine.reconfigure_cache(cache_config).await?;

// 2. Try different eviction policy
cache_config.l1_eviction = EvictionPolicy::LFU;  // Was LRU
engine.reconfigure_cache(cache_config).await?;

// 3. Enable prefetching
cache_config.prefetch_enabled = true;
cache_config.prefetch_model = "frequency";  // For hot data
engine.reconfigure_cache(cache_config).await?;

// 4. If access patterns are truly random, consider indexed queries
// Bad: Full table scans (no locality)
// Good: Index-based point lookups

Problem: Storage Almost Full

Symptoms:

Storage usage at 90%+ capacity
Insert/update operations slowing
Risk of running out of space

Root Causes:

Sync queue not clearing (unsynced data accumulating)
Old data not being archived/deleted
L2 cache growing unbounded
Local transaction log not being truncated

Diagnosis:

let usage = engine.get_resource_usage().await?;
let percent_used = (usage.storage_used_mb as f32
    / usage.storage_total_mb as f32) * 100.0;
println!("Storage: {:.1}% used", percent_used);

// Break down storage by category
let breakdown = engine.get_storage_breakdown().await?;
println!("Sync queue: {} MB", breakdown.queue_size_mb);
println!("L2 cache: {} MB", breakdown.l2_cache_mb);
println!("Data tables: {} MB", breakdown.data_tables_mb);
println!("Logs: {} MB", breakdown.logs_mb);

Solutions:

// 1. Force sync to clear queue
engine.sync().await?;

// 2. Delete old data (beyond retention)
engine.delete_older_than("events", Duration::from_secs(7*24*3600)).await?;

// 3. Reduce L2 cache size
cache_config.l2_max_size = 2 * 1024 * 1024 * 1024;  // From 5GB to 2GB
engine.reconfigure_cache(cache_config).await?;

// 4. Truncate old logs
engine.truncate_logs_before("2025-01-01").await?;

// 5. Compress L2 data
cache_config.storage_compression = Some(CompressionAlgorithm::Zstd);
engine.reconfigure_cache(cache_config).await?;
// Expected: 3x reduction in storage size

Problem: Conflicts During Sync

Symptoms:

Different values on edge vs. cloud after sync
Manual conflict resolution required
Data inconsistency

Root Causes:

Wrong conflict strategy for data type
Concurrent updates to same record
Clock skew between nodes
Network reordering causing out-of-order updates

Diagnosis:

let sync_stats = engine.get_sync_stats().await?;
println!("Conflicts resolved: {}", sync_stats.conflicts_resolved);
println!("Last conflict: {:?}", sync_stats.last_conflict);

// Get conflict details
let conflicts = engine.get_recent_conflicts().await?;
for conflict in conflicts {
    println!("Table: {}", conflict.table);
    println!("Key: {:?}", conflict.key);
    println!("Edge value: {:?}", conflict.edge_value);
    println!("Cloud value: {:?}", conflict.cloud_value);
    println!("Strategy used: {:?}", conflict.strategy);
}

Solutions:

// 1. Choose appropriate conflict strategy
// For inventory (must be accurate): min/max value
sync_config.conflict_strategy = ConflictStrategy::MaxValue;

// For customer records (most recent is valid): LWW
sync_config.conflict_strategy = ConflictStrategy::LastWriteWins;

// For counters (never conflicts): G-Counter CRDT
use_crdt_type: CrdtType::GCounter;

// 2. Ensure clock synchronization
// Run NTP on all nodes
engine.sync_clock_with_cloud().await?;

// 3. Use application-level conflict resolution for complex cases
sync_config.conflict_strategy = ConflictStrategy::Custom(
    Box::new(|edge_val, cloud_val| {
        // Business logic: choose based on source, timestamp, etc.
        if edge_val.priority > cloud_val.priority {
            edge_val
        } else {
            cloud_val
        }
    })
);

Quick Start Guides

EDGE_DEPLOYMENT_GUIDE.md - Step-by-step deployment
WASM_EDGE_COMPUTING_USER_GUIDE.md - WASM edge functions

Feature Documentation

F3.2 Edge Caching - Multi-tier cache details
F3.6 Edge AI Inference - ML models at the edge
F5.2.5 Edge Sync - Sync architecture

Architecture Documents

Deployment Guides

Monitoring & Operations

API Reference

Business Use Cases

Summary

HeliosDB Edge Computing enables databases at the edge—in mobile devices, IoT networks, remote facilities, and content delivery networks. By following this guide, you can:

Deploy HeliosDB to edge devices with minimal resources
Synchronize data reliably between edge and cloud with automatic conflict resolution
Optimize performance through intelligent caching and prefetching
Monitor edge health proactively and respond to issues quickly
Scale to thousands of edge nodes with confidence

The key to successful edge deployments is understanding your workload (retail POS, IoT sensors, mobile app, etc.), selecting appropriate configurations (cache sizes, sync intervals, conflict strategies), and monitoring proactively (sync health, cache performance, resource usage).

Start small with a pilot deployment in one location, learn from the metrics and operational experience, then scale to your full edge network. The HeliosDB community is ready to support your edge computing journey.

Next Steps:

Read EDGE_DEPLOYMENT_GUIDE.md for detailed setup instructions
Choose your deployment pattern (retail, IoT, mobile, gateway) from this guide
Review Production Deployment Configuration for operational best practices
Set up monitoring per the Monitoring Operations Guide
Contact the HeliosDB support team with deployment questions

HeliosDB Edge Computing User Guide

HeliosDB Edge Computing User Guide

Table of Contents

Introduction

Key Capabilities

Why Deploy at the Edge?

What is Edge Computing?

Traditional vs. Edge Architecture

Edge Computing Benefits in Practice

Architecture Overview

System Components

Component Stack per Edge Node

Key Architectural Patterns

Getting Started with Edge Deployment

Prerequisites

Step 1: Configure Edge Node

Step 2: Initialize Edge Engine

Step 3: Execute Local Queries

Step 4: Configure Synchronization

Step 5: Monitor Edge Health

Multi-Tier Caching Strategy

L1 Cache: Process Memory

L2 Cache: Local Storage (SSD/Flash)

L3 Cache: Regional Aggregate

L4 Cache: Cloud Primary Store

Cache Strategy Recommendations

Edge Database Synchronization

Synchronization Architecture

Sync Flow

Configuring Sync Behavior

Offline Queue Management

Sync Compression

Conflict Resolution

Understanding Conflicts

Conflict Strategies

CRDT Data Types

Selecting the Right Strategy

Monitoring Edge Nodes

Key Metrics to Monitor

Setting Up Alerts

Integration with Monitoring Systems

Common Deployment Patterns

Pattern 1: Retail POS System

Pattern 2: IoT Sensor Network

Pattern 3: Mobile App with Offline Support

Performance Optimization

Query Optimization for Edge

Cache Tuning

Bandwidth Optimization

Storage Optimization

Troubleshooting Guide

Problem: Sync Queue Growing Unbounded

Problem: Cache Hit Rate Below 50%

Problem: Storage Almost Full

Problem: Conflicts During Sync

Related Documentation

Quick Start Guides

Feature Documentation

Architecture Documents

Deployment Guides

Monitoring & Operations

API Reference

Business Use Cases

Summary