Skip to content
HeliosDB HeliosDB Full

Global Distributed Cache User Guide

Global Distributed Cache User Guide

Feature ID: F5.3.4
Version: 1.0.0
Status: Production Ready
Last Updated: November 2, 2025

Table of Contents

  1. Overview
  2. Quick Start
  3. Core Concepts
  4. Configuration
  5. Usage Examples
  6. Consistency Models
  7. Eviction Policies
  8. Hot Spot Detection
  9. Performance Tuning
  10. Monitoring
  11. Best Practices

Overview

The Global Distributed Cache provides a globally distributed caching layer with intelligent consistency, eviction, and hot spot management for HeliosDB. It enables:

  • Multi-region distribution: Deploy cache across multiple geographic regions
  • Configurable consistency: Choose from eventual, strong, causal, bounded staleness, or session consistency
  • Intelligent eviction: LRU, LFU, FIFO, TTL, ML-predicted, or hybrid policies
  • Hot spot detection: Automatic detection and migration of frequently accessed data
  • Multi-region replication: Async, sync, semi-sync, or selective replication strategies
  • Smart invalidation: Broadcast, lazy, gossip, or selective invalidation protocols

Key Features

<5ms local cache hit latency
<50ms global cache hit latency

85% cache hit rate
<100ms invalidation latency
<2 second hot spot migration
Vector clock-based causal consistency
ML-based eviction predictions

Quick Start

Basic Usage

use heliosdb_global_cache::*;


#[tokio::main]
async fn main() -> Result<()> {
    // Create default configuration
    let config = GlobalCacheConfig::default();


    // Initialize cache coordinator
    let cache = GlobalCacheCoordinator::new(config).await?;


    // Set a value
    cache.set("user:123", b"John Doe").await?;


    // Get a value
    if let Some(value) = cache.get(&CacheKey::from("user:123")).await? {
        println!("Found: {:?}", String::from_utf8_lossy(&value));
    }


    // Invalidate
    cache.invalidate(&CacheKey::from("user:123")).await?;


    Ok(())
}

Advanced Configuration

use heliosdb_global_cache::*;
use std::time::Duration;


let config = GlobalCacheConfig {
    // Consistency configuration
    consistency_mode: ConsistencyMode::Causal,


    // Eviction policy
    eviction_policy: EvictionPolicyType::MLPredicted,


    // Replication strategy
    replication_strategy: ReplicationStrategyType::SemiSync { quorum_size: 2 },


    // Invalidation strategy
    invalidation_strategy: InvalidationStrategyType::Broadcast,


    // Feature flags
    enable_hot_spot_detection: true,
    enable_ml_predictions: true,
    enable_cache_warming: true,


    // Regions
    regions: vec![
        Region::new("us-west-1", "US West")
            .with_location(37.7749, -122.4194),
        Region::new("us-east-1", "US East")
            .with_location(40.7128, -74.0060),
        Region::new("eu-west-1", "EU West")
            .with_location(51.5074, -0.1278),
    ],


    // Capacity and performance
    max_cache_size_per_region: 1 * 1024 * 1024 * 1024, // 1GB
    default_ttl: Duration::from_secs(3600),
    hot_spot_threshold: 100, // accesses/sec
    target_hit_rate: 0.85,
    max_replication_lag_ms: 100,
    max_invalidation_latency_ms: 100,
};


let cache = GlobalCacheCoordinator::new(config).await?;

Core Concepts

Cache Keys

Cache keys consist of a namespace and a key identifier:

// Simple key
let key1 = CacheKey::from("my_key");


// Namespaced key
let key2 = CacheKey::new("users", "123");


// Versioned key
let key3 = CacheKey::new("users", "123").with_version(5);

Cache Entries

Entries include metadata for consistency and eviction:

pub struct CacheEntry {
    pub key: CacheKey,
    pub value: Vec<u8>,
    pub region: String,
    pub created_at: u64,
    pub last_accessed: u64,
    pub access_count: u64,
    pub ttl_secs: u64,
    pub size: usize,
    pub version: u64,
    pub vector_clock: VectorClock,
    pub metadata: HashMap<String, String>,
}

Regions

Geographic regions where cache nodes are deployed:

let region = Region::new("us-west-1", "US West")
    .with_location(37.7749, -122.4194);

Configuration

Consistency Modes

1. Eventual Consistency

Fastest, may return stale data:

config.consistency_mode = ConsistencyMode::Eventual;

Use when: Maximum performance, stale data acceptable

2. Strong Consistency

Always returns latest data:

config.consistency_mode = ConsistencyMode::Strong;

Use when: Data accuracy critical, latency acceptable

3. Causal Consistency

Preserves causal relationships using vector clocks:

config.consistency_mode = ConsistencyMode::Causal;

Use when: Need ordering guarantees, distributed writes

4. Bounded Staleness

Guarantees freshness within time bound:

config.consistency_mode = ConsistencyMode::BoundedStaleness {
    max_staleness_ms: 1000
};

Use when: Controlled staleness acceptable

5. Session Consistency

Consistent within a user session:

config.consistency_mode = ConsistencyMode::Session;

Use when: User-specific consistency needed

Usage Examples

Basic Operations

// Set with automatic replication
cache.set("product:456", product_data).await?;


// Get with consistency checks
if let Some(data) = cache.get(&CacheKey::from("product:456")).await? {
    // Use data
}


// Invalidate single key
cache.invalidate(&CacheKey::from("product:456")).await?;


// Invalidate by table
let count = cache.invalidate_table("products").await?;
println!("Invalidated {} entries", count);

Working with Namespaces

// User cache
let user_key = CacheKey::new("users", "123");
cache.set(user_key.clone(), user_data).await?;


// Product cache
let product_key = CacheKey::new("products", "456");
cache.set(product_key.clone(), product_data).await?;


// Invalidate all users
cache.invalidate_table("users").await?;

Hot Spot Management

// Automatic detection (runs periodically)
let migrated = cache.detect_and_migrate_hotspots().await?;
println!("Migrated {} hot spots", migrated);


// Check statistics
let stats = cache.stats();
println!("Hot spots detected: {}", stats.cache.hot_spots_detected);
println!("Hot spots migrated: {}", stats.cache.hot_spots_migrated);

Statistics and Monitoring

let stats = cache.stats();


println!("Cache hit rate: {:.2}%", stats.cache.hit_rate() * 100.0);
println!("Average latency: {:.2}ms", stats.cache.avg_latency_ms);
println!("Local hit rate: {:.2}%", stats.cache.local_hit_rate() * 100.0);
println!("Total entries: {}", stats.cache.total_entries);
println!("Total bytes: {} MB", stats.cache.total_bytes / (1024 * 1024));
println!("Evictions: {}", stats.cache.evictions);
println!("Invalidations: {}", stats.cache.invalidations);

Eviction Policies

LRU (Least Recently Used)

Evicts least recently accessed entries:

config.eviction_policy = EvictionPolicyType::LRU;

Best for: General-purpose caching, temporal locality

LFU (Least Frequently Used)

Evicts least frequently accessed entries:

config.eviction_policy = EvictionPolicyType::LFU;

Best for: Popular content caching, frequency-based access

FIFO (First In First Out)

Evicts oldest entries:

config.eviction_policy = EvictionPolicyType::FIFO;

Best for: Simple time-based eviction, streaming data

TTL (Time To Live)

Evicts expired entries first:

config.eviction_policy = EvictionPolicyType::TTL;

Best for: Time-sensitive data, session caching

ML-Predicted

Uses machine learning to predict eviction candidates:

config.eviction_policy = EvictionPolicyType::MLPredicted;

Formula: score = (frequency × recency) / ln(size)

Best for: Complex access patterns, adaptive workloads

Hybrid

Combines TTL + ML predictions:

config.eviction_policy = EvictionPolicyType::Hybrid;

Best for: Production deployments, balanced performance

Performance Tuning

Cache Size

// Per-region cache size
config.max_cache_size_per_region = 2 * 1024 * 1024 * 1024; // 2GB

Recommendation: 10-30% of working set size

TTL Configuration

use std::time::Duration;


config.default_ttl = Duration::from_secs(3600); // 1 hour

Guidelines:

  • Fast-changing data: 300-900s (5-15 min)
  • Moderate data: 1800-3600s (30-60 min)
  • Slow-changing data: 7200-86400s (2-24 hours)

Hot Spot Threshold

config.hot_spot_threshold = 100; // accesses per second

Recommendation: 50-200 depending on workload

Replication Strategy

// For read-heavy workloads
config.replication_strategy = ReplicationStrategyType::Async;


// For write-consistency critical
config.replication_strategy = ReplicationStrategyType::Sync;


// Balanced approach
config.replication_strategy = ReplicationStrategyType::SemiSync { quorum_size: 2 };

Monitoring

Key Metrics

Monitor these metrics for optimal performance:

  1. Hit Rate (target: >85%)

    stats.cache.hit_rate()

  2. Latency (target: <5ms local, <50ms global)

    stats.cache.avg_latency_ms

  3. Eviction Rate

    stats.cache.evictions

  4. Hot Spot Detection

    stats.hotspot.detected_hot_spots

  5. Replication Lag (target: <100ms)

    stats.replication.avg_replication_lag_ms

Alerting

Set up alerts for:

  • Hit rate < 70%
  • Latency > 100ms
  • Eviction rate > 10/sec
  • Replication lag > 200ms
  • Cache size > 90% capacity

Best Practices

1. Key Design

// Good: Namespaced, structured keys
CacheKey::new("users:profile", "user_123")
CacheKey::new("products:details", "prod_456")


// Bad: Unstructured keys
CacheKey::from("u123")
CacheKey::from("random_key")

2. Value Size

  • Keep values <1MB for optimal performance
  • Use compression for large values
  • Consider chunking for very large objects

3. TTL Strategy

// Short TTL for volatile data
cache.set_with_ttl("session:token", data, Duration::from_secs(300)).await?;


// Long TTL for static data
cache.set_with_ttl("config:app", data, Duration::from_secs(86400)).await?;

4. Invalidation

// Prefer table-based invalidation for bulk updates
cache.invalidate_table("users").await?;


// Use dependency tracking
// (Register dependencies when setting values)

5. Multi-Region Deployment

  • Deploy regions close to users
  • Use selective replication for cost optimization
  • Monitor cross-region latency
  • Configure appropriate consistency model per use case

6. Capacity Planning

Target cache size = Working set size × 0.2
Number of regions = User distribution
TTL = Data change frequency × 2

Advanced Features

Custom Eviction Scoring

For ML-predicted eviction, the system uses:

eviction_score = (access_frequency × recency_score) / size_penalty


where:
  recency_score = 1 / (time_since_last_access + 1)
  size_penalty = ln(entry_size_bytes)

Lower scores are evicted first.

Vector Clocks

For causal consistency, vector clocks track causality:

// Automatic vector clock management
let entry = cache.get(&key).await?;
// entry.vector_clock contains causal information

Hot Spot Migration

// Manual hot spot detection
let hot_spots = cache.detect_and_migrate_hotspots().await?;


// Check recommendations
for hot_spot in hot_spots {
    println!("Key: {:?}", hot_spot.key);
    println!("Access rate: {}/sec", hot_spot.access_rate);
    println!("Recommended region: {:?}", hot_spot.recommended_region);
}

Troubleshooting

Low Hit Rate

Symptoms: Hit rate < 70%

Solutions:

  1. Increase cache size
  2. Adjust TTL (may be too short)
  3. Review eviction policy
  4. Check access patterns

High Latency

Symptoms: Latency > 50ms

Solutions:

  1. Check network connectivity
  2. Review consistency mode (strong is slower)
  3. Enable hot spot detection
  4. Add more regions

Frequent Evictions

Symptoms: Eviction rate > 10/sec

Solutions:

  1. Increase cache size
  2. Review value sizes
  3. Adjust TTL strategy
  4. Check for hot spots

API Reference

See inline documentation in source code for complete API reference:

// Main coordinator
pub struct GlobalCacheCoordinator { ... }


// Configuration
pub struct GlobalCacheConfig { ... }


// Consistency
pub enum ConsistencyMode { ... }


// Eviction
pub enum EvictionPolicyType { ... }


// Statistics
pub struct GlobalCacheStats { ... }

Summary

The Global Distributed Cache provides a production-ready, intelligent caching layer with:

Multi-region distribution for global deployments
Flexible consistency models for different use cases
Intelligent eviction with ML-based predictions
Automatic hot spot detection and migration
Comprehensive monitoring and statistics
High performance (<5ms local, <50ms global latency)

For additional support or feature requests, please consult the HeliosDB documentation or contact the development team.

Document Version: 1.0
Last Updated: November 2, 2025
Author: HeliosDB Engineering Team