Skip to content
HeliosDB HeliosDB Full

HeliosDB Key Management and Migration Guide

HeliosDB Key Management and Migration Guide

Table of Contents

  1. Overview
  2. Security Risk: InMemory MEK Provider
  3. Production-Safe Alternatives
  4. Migration Paths
  5. Configuration Examples
  6. MEK Rotation
  7. Operational Procedures
  8. Troubleshooting

Overview

HeliosDB implements a two-tier key hierarchy for Transparent Data Encryption (TDE):

  • Master Encryption Key (MEK): Top-level key that encrypts all Table Encryption Keys
  • Table Encryption Key (TEK): Per-table keys encrypted with the MEK

This document provides guidance for securely managing MEKs in production environments.

Security Risk: InMemory MEK Provider

Critical Security Issue

The InMemoryMekProvider stores Master Encryption Keys in UNPROTECTED MEMORY and is ABSOLUTELY NOT SECURE for production use.

Vulnerabilities

Keys stored in memory are vulnerable to:

  • Memory dumps and core files - Keys exposed in crash dumps
  • Process memory inspection - Tools like gdb, gcore can extract keys
  • Swap space exposure - Keys may be written to unencrypted swap
  • Container/VM snapshots - Memory snapshots capture keys
  • Debugger attachment - Attackers can attach debuggers to running process
  • Memory forensics - Cold boot attacks, RAM analysis
  • Supply chain attacks - Malicious code can read process memory

Production Mode Protection

Starting in HeliosDB v5.0, the InMemoryMekProvider is automatically rejected in production mode:

// This will FAIL in production mode (release builds)
let provider = Arc::new(InMemoryMekProvider::new());
let key_manager = KeyManager::new(provider, ...);
key_manager.validate_security()?; // ERROR: Insecure MEK provider


// Error message:
// Production mode requires a secure MEK provider. Current provider: In-Memory (INSECURE - testing only) is NOT suitable for production.
// SECURITY RISK: InMemoryMekProvider stores encryption keys in unprotected memory.

Security Mode Detection

Security mode is determined by:

  1. Environment variable: HELIOSDB_SECURITY_MODE=production|development
  2. Build type: Release builds default to Production, debug builds to Development
  3. Explicit configuration: Set security_mode in TdeConfig
Terminal window
# Force production mode (recommended for production)
export HELIOSDB_SECURITY_MODE=production


# Allow insecure providers (ONLY for development/testing)
export HELIOSDB_SECURITY_MODE=development

Production-Safe Alternatives

1. HSM (Hardware Security Module) - RECOMMENDED

Security Level: ⭐⭐⭐⭐⭐ (Highest)

Hardware security modules provide the strongest protection for encryption keys.

Advantages

  • Hardware-backed key protection - Keys never leave the HSM
  • FIPS 140-2 Level 2+ compliance - Meets regulatory requirements
  • Tamper resistance - Physical and logical tamper detection
  • Cryptographic operations in hardware - No key exposure during encryption
  • Audit logging - Complete key usage audit trail
  • Key backup and recovery - Secure key backup mechanisms

Supported HSMs

  • SoftHSM - Software HSM for testing/development
  • Thales Luna - Enterprise HSM
  • Utimaco - High-performance HSM
  • AWS CloudHSM - Cloud-based HSM
  • Azure Dedicated HSM - Azure HSM service
  • YubiHSM - Compact USB HSM

Requirements

  • PKCS#11 library for your HSM
  • Proper HSM initialization and key generation
  • Network connectivity (for network HSMs)

Cost

  • Hardware: $5,000 - $50,000+ per device
  • Cloud HSM: $1.00 - $2.00 per hour
  • SoftHSM: Free (testing only)

2. Cloud KMS (Key Management Service)

Security Level: ⭐⭐⭐⭐ (High)

Cloud providers offer managed key management services.

AWS KMS

  • Managed service with hardware-backed keys
  • Integrated with AWS services
  • Automatic key rotation
  • CloudTrail audit logging
  • Cost: $1/key/month + $0.03 per 10,000 requests

Azure Key Vault

  • Hardware-backed or software-backed keys
  • FIPS 140-2 Level 2 validated
  • Managed HSM option available
  • Cost: $0.03 per 10,000 transactions

HashiCorp Vault

  • Self-hosted or cloud-hosted
  • Multiple backend options (AWS KMS, Azure, GCP)
  • Dynamic secrets and encryption as a service
  • Cost: Free (OSS) or $0.03 per hour (Enterprise)

3. Secure File Provider

Security Level: ⭐⭐⭐ (Medium)

Encrypted files with OS-level security controls.

Advantages

  • No external dependencies
  • Simple to deploy
  • Works offline
  • Low cost

Limitations

  • Keys still in filesystem (albeit encrypted)
  • Requires secure passphrase management
  • Manual key rotation
  • No tamper detection

Best Practices

  • Use strong passphrases (25+ characters)
  • Store passphrase in secure credential manager
  • Set file permissions to 600 (owner-only)
  • Enable filesystem encryption (LUKS, BitLocker)
  • Use dedicated encrypted partition for keys
  • Regular backups to secure location

Migration Paths

Migration 1: InMemory → Secure File Provider

Complexity: ⭐ (Easy) Downtime: None Cost: Free

Steps

  1. Generate MEK passphrase
Terminal window
# Generate strong passphrase
openssl rand -base64 32 > /secure/location/mek-passphrase.txt
chmod 600 /secure/location/mek-passphrase.txt


# Set environment variable
export HELIOSDB_MEK_PASSPHRASE=$(cat /secure/location/mek-passphrase.txt)
  1. Update configuration
use heliosdb_security::tde::{TdeConfig, SecureFileConfig, EncryptionAlgorithm};
use std::path::PathBuf;


let config = TdeConfig {
    enabled: true,
    algorithm: EncryptionAlgorithm::Aes256Gcm,
    key_storage_path: PathBuf::from("/var/lib/heliosdb/keys"),
    default_mek_id: "master_key_001".to_string(),
    cache_limit: 1000,
    kms_config: None,
    secure_file_config: Some(SecureFileConfig {
        storage_path: PathBuf::from("/var/lib/heliosdb/mek"),
        passphrase_env_var: Some("HELIOSDB_MEK_PASSPHRASE".to_string()),
    }),
    security_mode: Some("production".to_string()),
};
  1. Generate new MEK
let tde = TdeSubsystem::new(config).await?;
let km = tde.key_manager();


// This will create encrypted MEK file
km.mek_provider.generate_mek("master_key_001", EncryptionAlgorithm::Aes256Gcm).await?;
  1. Re-encrypt all TEKs (if migrating existing data)
// List all existing TEKs
let keys = km.list_keys(Some(KeyType::Table)).await?;


for key_meta in keys {
    // Re-encrypt with new MEK
    km.rotate_table_key(&key_meta.key_id).await?;
}
  1. Verify
Terminal window
# Check MEK file exists with correct permissions
ls -la /var/lib/heliosdb/mek/
# Should show: -rw------- (600) for .mek files


# Test encryption/decryption
# (Run integration tests)

Migration 2: InMemory → Cloud KMS (AWS KMS)

Complexity: ⭐⭐ (Medium) Downtime: None Cost: ~$1/month + API costs

Prerequisites

  • AWS account with KMS access
  • IAM role with KMS permissions
  • AWS credentials configured

Steps

  1. Create KMS key
Terminal window
aws kms create-key \
  --description "HeliosDB Master Encryption Key" \
  --key-usage ENCRYPT_DECRYPT \
  --origin AWS_KMS


# Output: KeyId (e.g., arn:aws:kms:us-east-1:123456789:key/abc-def-123)
  1. Configure HeliosDB
use heliosdb_security::tde::{TdeConfig, KmsConfig, AwsKmsConfig, EncryptionAlgorithm};


let config = TdeConfig {
    enabled: true,
    algorithm: EncryptionAlgorithm::Aes256Gcm,
    key_storage_path: PathBuf::from("/var/lib/heliosdb/keys"),
    default_mek_id: "master_key_001".to_string(),
    cache_limit: 1000,
    kms_config: Some(KmsConfig::Aws(AwsKmsConfig {
        region: "us-east-1".to_string(),
        key_id: "arn:aws:kms:us-east-1:123456789:key/abc-def-123".to_string(),
        endpoint: None,
    })),
    security_mode: Some("production".to_string()),
};
  1. Enable feature flag
Cargo.toml
[dependencies]
heliosdb-security = { version = "*", features = ["aws-kms"] }
  1. Deploy and verify

Migration 3: InMemory → HSM (PKCS#11)

Complexity: ⭐⭐⭐⭐ (Hard) Downtime: Minimal Cost: $5,000+ (hardware) or $1-2/hour (cloud)

Prerequisites

  • HSM hardware or SoftHSM for testing
  • PKCS#11 library installed
  • HSM initialized with security officer PIN
  • Slot created and initialized

Steps

  1. Initialize HSM (example with SoftHSM)
Terminal window
# Install SoftHSM
apt-get install softhsm2


# Initialize token
softhsm2-util --init-token --slot 0 --label "heliosdb" \
  --so-pin 123456 --pin 654321
  1. Configure HeliosDB
use heliosdb_security::tde::{TdeConfig, HsmConfig, EncryptionAlgorithm};


let config = TdeConfig {
    enabled: true,
    algorithm: EncryptionAlgorithm::Aes256Gcm,
    key_storage_path: PathBuf::from("/var/lib/heliosdb/keys"),
    default_mek_id: "master_key_001".to_string(),
    cache_limit: 1000,
    kms_config: None,
    hsm_config: Some(HsmConfig {
        pkcs11_library_path: "/usr/lib/softhsm/libsofthsm2.so".to_string(),
        slot_id: 0,
        pin: "654321".to_string(), // In production, use env var
        token_label: Some("heliosdb".to_string()),
        timeout_secs: 30,
        enable_logging: true,
    }),
    security_mode: Some("production".to_string()),
};
  1. Enable feature flag
Cargo.toml
[dependencies]
heliosdb-security = { version = "*", features = ["pkcs11-hsm"] }
  1. Generate MEK in HSM
let tde = TdeSubsystem::new(config).await?;
// MEK is generated directly in HSM, never exposed to application

Configuration Examples

Development/Testing Configuration

// ONLY FOR DEVELOPMENT - Will be rejected in production mode
use heliosdb_security::tde::{TdeConfig, EncryptionAlgorithm};


let config = TdeConfig {
    enabled: true,
    algorithm: EncryptionAlgorithm::Aes256Gcm,
    key_storage_path: PathBuf::from("/tmp/heliosdb/keys"),
    default_mek_id: "test_mek".to_string(),
    cache_limit: 100,
    kms_config: None,
    secure_file_config: None,
    security_mode: Some("development".to_string()), // Explicitly allow insecure
};


// Uses InMemoryMekProvider by default
let tde = TdeSubsystem::new(config).await?;

Production Configuration (Secure File)

use heliosdb_security::tde::{TdeConfig, SecureFileConfig, EncryptionAlgorithm};
use std::path::PathBuf;


let config = TdeConfig {
    enabled: true,
    algorithm: EncryptionAlgorithm::Aes256Gcm,
    key_storage_path: PathBuf::from("/var/lib/heliosdb/keys"),
    default_mek_id: "prod_mek_001".to_string(),
    cache_limit: 10000,
    kms_config: None,
    secure_file_config: Some(SecureFileConfig {
        storage_path: PathBuf::from("/var/lib/heliosdb/mek"),
        passphrase_env_var: Some("HELIOSDB_MEK_PASSPHRASE".to_string()),
    }),
    security_mode: None, // Auto-detect (will be Production in release builds)
};


let tde = TdeSubsystem::new(config).await?;

Production Configuration (AWS KMS)

use heliosdb_security::tde::{TdeConfig, KmsConfig, AwsKmsConfig, EncryptionAlgorithm};


let config = TdeConfig {
    enabled: true,
    algorithm: EncryptionAlgorithm::Aes256Gcm,
    key_storage_path: PathBuf::from("/var/lib/heliosdb/keys"),
    default_mek_id: "prod_mek_001".to_string(),
    cache_limit: 10000,
    kms_config: Some(KmsConfig::Aws(AwsKmsConfig {
        region: std::env::var("AWS_REGION").unwrap_or("us-east-1".to_string()),
        key_id: std::env::var("HELIOSDB_KMS_KEY_ID")
            .expect("HELIOSDB_KMS_KEY_ID must be set"),
        endpoint: None,
    })),
    security_mode: Some("production".to_string()),
};


let tde = TdeSubsystem::new(config).await?;

MEK Rotation

Why Rotate MEKs?

  • Compliance requirements - Many regulations require periodic key rotation
  • Cryptographic best practices - Limit key exposure over time
  • Compromise mitigation - Reduce impact of potential key disclosure
  • Personnel changes - Rotate after employee departures

Rotation Frequency

  • High security: Every 90 days
  • Medium security: Every 180 days
  • Minimum: Annually

Rotation Process

use heliosdb_security::tde::TdeSubsystem;


async fn rotate_mek(tde: &TdeSubsystem) -> Result<(), Box<dyn std::error::Error>> {
    let km = tde.key_manager();


    // Perform rotation with dual-encryption period
    let result = km.rotate_mek_with_dual_encryption("current_mek_id").await?;


    println!("MEK Rotation Complete:");
    println!("  Old MEK: {}", result.old_mek_id);
    println!("  New MEK: {}", result.new_mek_id);
    println!("  Keys re-encrypted: {}", result.keys_reencrypted);


    if !result.failed_keys.is_empty() {
        eprintln!("WARNING: {} keys failed to re-encrypt:", result.failed_keys.len());
        for (key_id, error) in result.failed_keys {
            eprintln!("  - {}: {}", key_id, error);
        }
    }


    Ok(())
}

Dual-Encryption Period

During MEK rotation, HeliosDB maintains both old and new MEK versions:

  1. Day 0: Generate new MEK, re-encrypt all TEKs
  2. Day 0-30: Both MEKs valid, gradual re-encryption
  3. Day 30: Verify all TEKs use new MEK
  4. Day 31: Deactivate old MEK

This allows zero-downtime rotation and rollback capability.

Operational Procedures

Daily Operations

Terminal window
# Check security mode
heliosdb-cli security status


# Verify MEK provider type
heliosdb-cli security mek-info


# Monitor key usage
heliosdb-cli security key-stats

Emergency Procedures

MEK Compromise

If you suspect MEK compromise:

  1. Immediately rotate MEK
  2. Generate new TEKs for all tables
  3. Re-encrypt all data (gradual or immediate based on risk)
  4. Review audit logs for unauthorized access
  5. Investigate root cause

Lost MEK

If MEK is lost (no backup):

  1. All encrypted data is PERMANENTLY LOST
  2. Restore from backup if available
  3. Rebuild database from application data sources
  4. Implement proper backup procedures (see below)

Backup and Recovery

MEK Backup (Secure File Provider)

Terminal window
# Backup MEK files (encrypted)
tar -czf mek-backup-$(date +%Y%m%d).tar.gz /var/lib/heliosdb/mek/


# Store in secure location (separate from database backups)
aws s3 cp mek-backup-*.tar.gz s3://secure-backup-bucket/mek/ --sse AES256


# Store passphrase separately (use password manager or secret store)

MEK Recovery

Terminal window
# Restore MEK files
aws s3 cp s3://secure-backup-bucket/mek/mek-backup-20241025.tar.gz .
tar -xzf mek-backup-20241025.tar.gz -C /


# Set passphrase
export HELIOSDB_MEK_PASSPHRASE=$(cat /secure/location/mek-passphrase.txt)


# Restart HeliosDB
systemctl restart heliosdb

Troubleshooting

Error: “Insecure MEK provider in production”

Production mode requires a secure MEK provider. Current provider: In-Memory (INSECURE - testing only) is NOT suitable for production.

Solution: Configure a production-safe MEK provider (HSM, Cloud KMS, or Secure File).

Temporary workaround (NOT RECOMMENDED):

Terminal window
export HELIOSDB_SECURITY_MODE=development

Error: “Passphrase environment variable not found”

MEK provider error: Passphrase environment variable HELIOSDB_MEK_PASSPHRASE not found

Solution: Set the required environment variable:

Terminal window
export HELIOSDB_MEK_PASSPHRASE="your-strong-passphrase"

Error: “Permission denied” on MEK files

IO error: Permission denied (os error 13)

Solution: Fix file permissions:

Terminal window
chmod 700 /var/lib/heliosdb/mek
chmod 600 /var/lib/heliosdb/mek/*.mek
chown heliosdb:heliosdb /var/lib/heliosdb/mek -R

Error: HSM “CKR_PIN_INCORRECT”

HSM error: CKR_PIN_INCORRECT

Solution: Verify HSM PIN is correct and not locked:

Terminal window
# Check token status
softhsm2-util --show-slots


# If locked, reinitialize (WARNING: destroys keys)
softhsm2-util --init-token --slot 0 --label "heliosdb"

Performance: Slow key operations

If key operations are slow:

  1. Increase key cache size in TdeConfig
  2. Use local HSM instead of network HSM
  3. Batch key operations where possible
  4. Monitor HSM latency and network connectivity

Security Checklist

  • InMemoryMekProvider disabled in production
  • Production MEK provider configured (HSM, Cloud KMS, or Secure File)
  • MEK passphrase stored securely (not in code or config files)
  • File permissions set correctly (600 for MEK files, 700 for directories)
  • MEK backup procedure established
  • MEK rotation schedule defined
  • Audit logging enabled
  • Access controls configured (only authorized personnel)
  • Incident response plan documented
  • Regular security audits scheduled

References

Support

For security-related questions or incident response: