Skip to content
HeliosDB HeliosDB Full

HeliosDB Key Management and Migration Guide

HeliosDB Key Management and Migration Guide

Table of Contents

  1. Overview
  2. Security Risk: InMemory MEK Provider
  3. Production-Safe Alternatives
  4. Migration Paths
  5. MEK Rotation
  6. Operational Procedures
  7. Troubleshooting

Overview

HeliosDB implements a two-tier key hierarchy for Transparent Data Encryption (TDE):

  • Master Encryption Key (MEK): Top-level key that encrypts all Table Encryption Keys
  • Table Encryption Key (TEK): Per-table keys encrypted with the MEK

This document provides guidance for securely managing MEKs in production environments.

Security Risk: InMemory MEK Provider

Critical Security Issue

The in-memory MEK provider stores Master Encryption Keys in UNPROTECTED MEMORY and is ABSOLUTELY NOT SECURE for production use.

Vulnerabilities

Keys stored in memory are vulnerable to:

  • Memory dumps and core files - Keys exposed in crash dumps
  • Process memory inspection - Tools like gdb, gcore can extract keys
  • Swap space exposure - Keys may be written to unencrypted swap
  • Container/VM snapshots - Memory snapshots capture keys
  • Debugger attachment - Attackers can attach debuggers to running process
  • Memory forensics - Cold boot attacks, RAM analysis
  • Supply chain attacks - Malicious code can read process memory

Production Mode Protection

Starting in HeliosDB v5.0, the in-memory MEK provider is automatically rejected in production mode.

Security Mode Detection

Security mode is determined by:

  1. Environment variable: HELIOSDB_SECURITY_MODE=production|development
  2. Build type: Release builds default to Production, debug builds to Development
  3. Explicit configuration: Set security_mode in the TDE configuration
Terminal window
# Force production mode (recommended for production)
export HELIOSDB_SECURITY_MODE=production


# Allow insecure providers (ONLY for development/testing)
export HELIOSDB_SECURITY_MODE=development

Production-Safe Alternatives

1. HSM (Hardware Security Module) - RECOMMENDED

Security Level: ⭐⭐⭐⭐⭐ (Highest)

Hardware security modules provide the strongest protection for encryption keys.

Advantages

  • Hardware-backed key protection - Keys never leave the HSM
  • FIPS 140-2 Level 2+ compliance - Meets regulatory requirements
  • Tamper resistance - Physical and logical tamper detection
  • Cryptographic operations in hardware - No key exposure during encryption
  • Audit logging - Complete key usage audit trail
  • Key backup and recovery - Secure key backup mechanisms

Supported HSMs

  • SoftHSM - Software HSM for testing/development
  • Thales Luna - Enterprise HSM
  • Utimaco - High-performance HSM
  • AWS CloudHSM - Cloud-based HSM
  • Azure Dedicated HSM - Azure HSM service
  • YubiHSM - Compact USB HSM

Requirements

  • PKCS#11 library for your HSM
  • Proper HSM initialization and key generation
  • Network connectivity (for network HSMs)

Cost

  • Hardware: $5,000 - $50,000+ per device
  • Cloud HSM: $1.00 - $2.00 per hour
  • SoftHSM: Free (testing only)

2. Cloud KMS (Key Management Service)

Security Level: ⭐⭐⭐⭐ (High)

Cloud providers offer managed key management services.

AWS KMS

  • Managed service with hardware-backed keys
  • Integrated with AWS services
  • Automatic key rotation
  • CloudTrail audit logging
  • Cost: $1/key/month + $0.03 per 10,000 requests

Azure Key Vault

  • Hardware-backed or software-backed keys
  • FIPS 140-2 Level 2 validated
  • Managed HSM option available
  • Cost: $0.03 per 10,000 transactions

HashiCorp Vault

  • Self-hosted or cloud-hosted
  • Multiple backend options (AWS KMS, Azure, GCP)
  • Dynamic secrets and encryption as a service
  • Cost: Free (OSS) or $0.03 per hour (Enterprise)

3. Secure File Provider

Security Level: ⭐⭐⭐ (Medium)

Encrypted files with OS-level security controls.

Advantages

  • No external dependencies
  • Simple to deploy
  • Works offline
  • Low cost

Limitations

  • Keys still in filesystem (albeit encrypted)
  • Requires secure passphrase management
  • Manual key rotation
  • No tamper detection

Best Practices

  • Use strong passphrases (25+ characters)
  • Store passphrase in secure credential manager
  • Set file permissions to 600 (owner-only)
  • Enable filesystem encryption (LUKS, BitLocker)
  • Use dedicated encrypted partition for keys
  • Regular backups to secure location

How to Configure

TDE, the key provider (local key store, cloud KMS or HSM over PKCS#11), and the “Configure HeliosDB” and “Generate MEK” steps of the migration paths are not set through a public configuration file or environment variables. Configuration for this feature is provided during onboarding — contact support@heliosdb.com (or sales@heliosdb.com if you are not yet a customer).

The heliosdb.toml file of the HeliosDB Full server accepts only the documented storage keys (storage.data_dir, storage.memtable_size_mb, storage.compaction_strategy, storage.read_cache_mb and the storage.prefetch_* settings). The server refuses to start if the file contains any other key.

Migration Paths

Migration 1: InMemory → Secure File Provider

Complexity: ⭐ (Easy) Downtime: None Cost: Free

Steps

  1. Generate MEK passphrase
Terminal window
# Generate strong passphrase
openssl rand -base64 32 > /secure/location/mek-passphrase.txt
chmod 600 /secure/location/mek-passphrase.txt


# Set environment variable
export HELIOSDB_MEK_PASSPHRASE=$(cat /secure/location/mek-passphrase.txt)

  1. Update configuration

  2. Generate new MEK

  3. Re-encrypt all TEKs (if migrating existing data)

  4. Verify

Terminal window
# Check MEK file exists with correct permissions
ls -la /var/lib/heliosdb/mek/
# Should show: -rw------- (600) for .mek files


# Test encryption/decryption
# (Run integration tests)

Migration 2: InMemory → Cloud KMS (AWS KMS)

Complexity: ⭐⭐ (Medium) Downtime: None Cost: ~$1/month + API costs

Prerequisites

  • AWS account with KMS access
  • IAM role with KMS permissions
  • AWS credentials configured

Steps

  1. Create KMS key
Terminal window
aws kms create-key \
  --description "HeliosDB Master Encryption Key" \
  --key-usage ENCRYPT_DECRYPT \
  --origin AWS_KMS


# Output: KeyId (e.g., arn:aws:kms:us-east-1:123456789:key/abc-def-123)

  1. Configure HeliosDB

  2. Deploy and verify

Migration 3: InMemory → HSM (PKCS#11)

Complexity: ⭐⭐⭐⭐ (Hard) Downtime: Minimal Cost: $5,000+ (hardware) or $1-2/hour (cloud)

Prerequisites

  • HSM hardware or SoftHSM for testing
  • PKCS#11 library installed
  • HSM initialized with security officer PIN
  • Slot created and initialized

Steps

  1. Initialize HSM (example with SoftHSM)
Terminal window
# Install SoftHSM
apt-get install softhsm2


# Initialize token
softhsm2-util --init-token --slot 0 --label "heliosdb" \
  --so-pin 123456 --pin 654321

  1. Configure HeliosDB

  2. Generate MEK in HSM

MEK Rotation

Why Rotate MEKs?

  • Compliance requirements - Many regulations require periodic key rotation
  • Cryptographic best practices - Limit key exposure over time
  • Compromise mitigation - Reduce impact of potential key disclosure
  • Personnel changes - Rotate after employee departures

Rotation Frequency

  • High security: Every 90 days
  • Medium security: Every 180 days
  • Minimum: Annually

Dual-Encryption Period

During MEK rotation, HeliosDB maintains both old and new MEK versions:

  1. Day 0: Generate new MEK, re-encrypt all TEKs
  2. Day 0-30: Both MEKs valid, gradual re-encryption
  3. Day 30: Verify all TEKs use new MEK
  4. Day 31: Deactivate old MEK

This allows zero-downtime rotation and rollback capability.

Operational Procedures

Daily Operations

Terminal window
# Check security mode
heliosdb-cli security status


# Verify MEK provider type
heliosdb-cli security mek-info


# Monitor key usage
heliosdb-cli security key-stats

Emergency Procedures

MEK Compromise

If you suspect MEK compromise:

  1. Immediately rotate MEK
  2. Generate new TEKs for all tables
  3. Re-encrypt all data (gradual or immediate based on risk)
  4. Review audit logs for unauthorized access
  5. Investigate root cause

Lost MEK

If MEK is lost (no backup):

  1. All encrypted data is PERMANENTLY LOST
  2. Restore from backup if available
  3. Rebuild database from application data sources
  4. Implement proper backup procedures (see below)

Backup and Recovery

MEK Backup (Secure File Provider)

Terminal window
# Backup MEK files (encrypted)
tar -czf mek-backup-$(date +%Y%m%d).tar.gz /var/lib/heliosdb/mek/


# Store in secure location (separate from database backups)
aws s3 cp mek-backup-*.tar.gz s3://secure-backup-bucket/mek/ --sse AES256


# Store passphrase separately (use password manager or secret store)

MEK Recovery

Terminal window
# Restore MEK files
aws s3 cp s3://secure-backup-bucket/mek/mek-backup-20241025.tar.gz .
tar -xzf mek-backup-20241025.tar.gz -C /


# Set passphrase
export HELIOSDB_MEK_PASSPHRASE=$(cat /secure/location/mek-passphrase.txt)


# Restart HeliosDB
systemctl restart heliosdb

Troubleshooting

Error: “Insecure MEK provider in production”

Production mode requires a secure MEK provider. Current provider: In-Memory (INSECURE - testing only) is NOT suitable for production.

Solution: Configure a production-safe MEK provider (HSM, Cloud KMS, or Secure File).

Temporary workaround (NOT RECOMMENDED):

Terminal window
export HELIOSDB_SECURITY_MODE=development

Error: “Passphrase environment variable not found”

MEK provider error: Passphrase environment variable HELIOSDB_MEK_PASSPHRASE not found

Solution: Set the required environment variable:

Terminal window
export HELIOSDB_MEK_PASSPHRASE="your-strong-passphrase"

Error: “Permission denied” on MEK files

IO error: Permission denied (os error 13)

Solution: Fix file permissions:

Terminal window
chmod 700 /var/lib/heliosdb/mek
chmod 600 /var/lib/heliosdb/mek/*.mek
chown heliosdb:heliosdb /var/lib/heliosdb/mek -R

Error: HSM “CKR_PIN_INCORRECT”

HSM error: CKR_PIN_INCORRECT

Solution: Verify HSM PIN is correct and not locked:

Terminal window
# Check token status
softhsm2-util --show-slots


# If locked, reinitialize (WARNING: destroys keys)
softhsm2-util --init-token --slot 0 --label "heliosdb"

Performance: Slow key operations

If key operations are slow:

  1. Increase key cache size in the TDE configuration
  2. Use local HSM instead of network HSM
  3. Batch key operations where possible
  4. Monitor HSM latency and network connectivity

Security Checklist

  • In-memory MEK provider disabled in production
  • Production MEK provider configured (HSM, Cloud KMS, or Secure File)
  • MEK passphrase stored securely (not in code or config files)
  • File permissions set correctly (600 for MEK files, 700 for directories)
  • MEK backup procedure established
  • MEK rotation schedule defined
  • Audit logging enabled
  • Access controls configured (only authorized personnel)
  • Incident response plan documented
  • Regular security audits scheduled

References

Support

For security-related questions or incident response: