Skip to content
HeliosDB Full

Flink State Backend

HeliosDB State Backend - RocksDB Alternative

Production-ready state backend for Apache Flink with superior performance.

Key Features

  • 50K+ ops/sec throughput (vs RocksDB ~30K ops/sec)
  • <30s recovery for 1GB state
  • Asynchronous snapshots - non-blocking checkpoints
  • Incremental checkpoints - faster backup/restore
  • TTL support - automatic state expiration
  • RocksDB compatibility layer - drop-in replacement

Quick Start

use heliosdb_streaming::flink::{HeliosStateBackend, StateBackendConfig, StateDescriptor, StateType};


// Create state backend
let config = StateBackendConfig {
    storage_path: "/data/state".to_string(),
    async_snapshots: true,
    compression_enabled: true,
    incremental_checkpoints: true,
    write_buffer_size: 64 * 1024 * 1024,  // 64MB
    block_cache_size: 256 * 1024 * 1024,  // 256MB
    ..Default::default()
};


let backend = HeliosStateBackend::new(config);


// Create value state
let desc = StateDescriptor {
    name: "user_balance".to_string(),
    state_type: StateType::Value,
    ttl: Some(Duration::from_secs(3600)),
};


let state = backend.create_value_state::<String, f64>(desc);
state.set("user:123".to_string(), 100.50);
assert_eq!(state.get("user:123"), Some(100.50));

Performance Comparison

MetricHeliosDBRocksDB
Write throughput50K+ ops/sec~30K ops/sec
Read latency (p99)<1ms~2ms
Snapshot time (1GB)~5s~15s
Recovery time (1GB)<30s~60s
Memory overhead20% lowerBaseline

Configuration Guide

Write Buffer

write_buffer_size: 128 * 1024 * 1024  // Increase for write-heavy workloads

Block Cache

block_cache_size: 512 * 1024 * 1024  // Increase for read-heavy workloads

TTL

state_ttl: Some(Duration::from_secs(3600))  // Auto-expire after 1 hour

See complete docs in /docs/FLINK_STATE_BACKEND_FULL.md.