Skip to content
HeliosDB HeliosDB Full

WASM Edge Computing: Caching Strategy

WASM Edge Computing: Caching Strategy

Part of: WASM Edge Computing User Guide

HeliosDB implements a sophisticated 3-tier caching architecture.

L1: Edge Local Cache (Memory)

Target Latency: <1ms Capacity: 512MB per node Hit Rate: 95% (typical)

use heliosdb_edge::MultiTierCache;


let config = MultiTierCacheConfig {
    l1_size_bytes: 512 * 1024 * 1024, // 512 MB
    promotion_threshold: 5,  // Promote after 5 accesses
    demotion_threshold: 3600, // Demote if not accessed for 1 hour
    ..Default::default()
};


let cache = MultiTierCache::new(config).await?;


// Insert (starts in L2)
cache.insert(
    "user:12345".to_string(),
    user_data,
    1024, // size in bytes
    Some(Duration::from_secs(300)), // 5 min TTL
).await?;


// Get (auto-promotes from L2 to L1 after 5 accesses)
let data = cache.get(&"user:12345".to_string()).await?;

L1 Characteristics:

  • Storage: In-process memory (RAM)
  • Eviction: LRU (Least Recently Used)
  • Promotion: Automatic from L2 based on access count
  • Persistence: None (lost on restart)
  • Latency: 0.5-1ms

L2: Regional Cache (SSD)

Target Latency: 10ms Capacity: 10GB per node Hit Rate: 85% (typical)

let config = MultiTierCacheConfig {
    l2_size_bytes: 10 * 1024 * 1024 * 1024, // 10 GB
    l2_path: PathBuf::from("/var/cache/heliosdb/edge"),
    ..Default::default()
};

L2 Characteristics:

  • Storage: Local SSD
  • Eviction: Access-frequency weighted LRU
  • Promotion: To L1 after threshold
  • Persistence: Survives restarts
  • Latency: 5-15ms

L3: Global Cloud Cache

Target Latency: 50ms Capacity: 100GB (regional) Hit Rate: 60% (typical)

let config = MultiTierCacheConfig {
    l3_size_bytes: 100 * 1024 * 1024 * 1024, // 100 GB
    l3_endpoint: Some("https://cache.heliosdb.com".to_string()),
    ..Default::default()
};

L3 Characteristics:

  • Storage: Cloud object storage (S3, GCS, etc.)
  • Eviction: TTL-based
  • Scope: Regional (shared across edge nodes)
  • Persistence: Highly durable
  • Latency: 30-80ms

Cache Operations

Smart Invalidation

// Invalidate specific key
cache.remove(&"user:12345".to_string()).await?;


// Pattern-based invalidation
cdn_manager.invalidate_pattern(
    CdnProvider::Cloudflare,
    "user:*".to_string(),
).await?;


// Purge all
cdn_manager.purge_cache(
    CdnProvider::Cloudflare,
    vec![], // empty = purge all
).await?;

Cache Warming

// Pre-populate cache with frequently accessed data
let hot_keys = vec![
    "user:12345".to_string(),
    "product:top_100".to_string(),
    "config:global".to_string(),
];


cdn_manager.warm_cache(
    CdnProvider::Cloudflare,
    hot_keys,
).await?;


// Automatic warming based on access patterns
let predictions = cache.get_prefetch_predictions(60)?; // 60 second window
for key_hash in predictions {
    // Prefetch predicted keys
}

TTL Management

// Short TTL for dynamic data
cache.insert(
    "stock:AAPL".to_string(),
    price_data,
    256,
    Some(Duration::from_secs(60)), // 1 minute
).await?;


// Long TTL for static data
cache.insert(
    "asset:logo.png".to_string(),
    logo_data,
    50_000,
    Some(Duration::from_secs(86400)), // 24 hours
).await?;


// No TTL (manual invalidation)
cache.insert(
    "config:database".to_string(),
    config_data,
    1024,
    None, // No expiration
).await?;

ML-Based Prefetching

HeliosDB uses machine learning to predict and prefetch data:

// Access pattern tracking
struct AccessPattern {
    key_hash: u64,
    access_times: VecDeque<i64>,
    access_intervals: VecDeque<i64>,
    access_count: u64,
}


impl AccessPattern {
    // Predict next access using exponentially weighted moving average
    pub fn predict_next_access(&self) -> Option<i64> {
        if self.access_intervals.len() < 2 {
            return None;
        }


        let mut weighted_sum = 0.0;
        let mut weight_sum = 0.0;


        for (i, &interval) in self.access_intervals.iter().enumerate() {
            let weight = (i + 1) as f64; // More recent = higher weight
            weighted_sum += interval as f64 * weight;
            weight_sum += weight;
        }


        let predicted_interval = (weighted_sum / weight_sum) as i64;
        Some(self.last_access + predicted_interval)
    }
}


// Get prefetch predictions
let predictions = cache.get_prefetch_predictions(60)?;
println!("Prefetching {} keys", predictions.len());


// Record accuracy
if cache_hit {
    cache.record_prefetch_hit();
} else {
    cache.record_prefetch_miss();
}


// Monitor prefetch accuracy
let stats = cache.stats();
println!("Prefetch accuracy: {:.1}%", stats.prefetch_accuracy * 100.0);

Cache Statistics

let stats = cache.stats();


println!("=== L1 Cache (Memory) ===");
println!("Hits: {}", stats.l1_stats.hits);
println!("Misses: {}", stats.l1_stats.misses);
println!("Hit rate: {:.1}%", stats.l1_stats.hit_rate * 100.0);
println!("Size: {} MB", stats.l1_stats.current_size_bytes / 1_000_000);
println!("Entries: {}", stats.l1_stats.entry_count);
println!("Avg latency: {:.2}ms", stats.l1_stats.avg_latency_ms);


println!("\n=== L2 Cache (SSD) ===");
println!("Hits: {}", stats.l2_stats.hits);
println!("Misses: {}", stats.l2_stats.misses);
println!("Hit rate: {:.1}%", stats.l2_stats.hit_rate * 100.0);
println!("Promotions: {}", stats.l2_stats.promotions);
println!("Demotions: {}", stats.l2_stats.demotions);


println!("\n=== Overall ===");
println!("Total hits: {}", stats.total_hits);
println!("Overall hit rate: {:.1}%", stats.overall_hit_rate * 100.0);
println!("Avg latency: {:.2}ms", stats.avg_latency_ms);
println!("Prefetch accuracy: {:.1}%", stats.prefetch_accuracy * 100.0);

Cold Entry Demotion

Automatically demote cold data from L1 to L2:

// Manual demotion
let demoted = cache.demote_cold_entries().await?;
println!("Demoted {} cold entries", demoted);


// Automatic demotion (background task)
tokio::spawn(async move {
    let mut ticker = tokio::time::interval(Duration::from_secs(300));
    loop {
        ticker.tick().await;
        let _ = cache.demote_cold_entries().await;
    }
});

Demotion Criteria:

  • Not accessed for > 1 hour (configurable)
  • L1 capacity pressure
  • Low access frequency