Transparent Data Encryption Tutorial
Transparent Data Encryption Tutorial
Version: 2.5.0-dev Last Updated: 2025-12-01 Estimated Time: 45 minutes Difficulty: Intermediate
Table of Contents
- Introduction
- Encryption Architecture
- Setting Up Encryption
- Key Management
- Working with Encrypted Data
- Security Best Practices
- Compliance Scenarios
- Recovery Scenarios
- Performance Considerations
- Troubleshooting
Introduction
What is Transparent Data Encryption (TDE)?
Transparent Data Encryption (TDE) is a security feature that encrypts data at rest without requiring changes to application code. With TDE enabled, HeliosDB Nano automatically:
- Encrypts all data before writing to disk
- Decrypts data when reading from disk
- Protects against unauthorized access to database files
- Maintains full query functionality with zero code changes
Why Use Encryption?
Security Benefits:
- Protects sensitive data from unauthorized file access
- Prevents data theft if storage media is stolen
- Ensures data privacy during backups
- Adds defense-in-depth security layer
Compliance Requirements:
- HIPAA: Required for healthcare PHI (Protected Health Information)
- GDPR: Recommended for EU personal data protection
- PCI-DSS: Mandatory for payment card data
- SOC 2: Required for Type II certification
- State Laws: California CCPA, New York SHIELD Act, etc.
How HeliosDB Nano Implements Encryption
HeliosDB Nano uses industry-standard AES-256-GCM (Advanced Encryption Standard with Galois/Counter Mode):
- Algorithm: AES-256 (NIST FIPS 140-2 approved)
- Mode: GCM (provides both encryption and authentication)
- Key Size: 256 bits (32 bytes)
- Authentication: Built-in integrity verification
- Performance: Hardware-accelerated on modern CPUs (AES-NI)
What Gets Encrypted:
- ✅ All table data (tuples)
- ✅ Catalog metadata (schemas, column definitions)
- ✅ Index data
- ✅ Row counters and internal state
What Doesn’t Get Encrypted:
- ❌ Database keys (for indexing efficiency)
- ❌ Configuration files
- ❌ WAL metadata structure
Encryption Architecture
AES-256-GCM Algorithm
Advanced Encryption Standard (AES-256):
- Block Cipher: Operates on 128-bit blocks
- Key Size: 256 bits (highest security level)
- Rounds: 14 encryption rounds
- NIST Approved: FIPS 140-2 compliant
Galois/Counter Mode (GCM):
- Authenticated Encryption: Combines encryption + authentication
- Authentication Tag: 128-bit tag prevents tampering
- Counter Mode: Turns block cipher into stream cipher
- Parallel Processing: Can utilize multiple CPU cores
Key Derivation with Argon2
When using password-based encryption, HeliosDB Nano uses Argon2:
- Algorithm: Argon2id (winner of Password Hashing Competition)
- Purpose: Derive 256-bit keys from passwords
- Protection: Resistant to GPU/ASIC attacks
- Configurable: Memory-hard and CPU-intensive parameters
Argon2 Parameters:
// Default parameters (balanced security/performance)let argon2 = Argon2::default(); // ~19ms on modern CPU// - Memory cost: 19 MiB// - Time cost: 2 iterations// - Parallelism: 1 threadEncryption Flow Diagram
┌─────────────────────────────────────────────────────────┐│ Application Layer ││ put(key: "users:1", value: "John Doe") │└────────────────────┬────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────┐│ Storage Engine ││ if encryption_enabled: ││ encrypted = encrypt(value, encryption_key) │└────────────────────┬────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────┐│ AES-256-GCM Encryption ││ 1. Generate random 12-byte nonce ││ 2. Encrypt plaintext → ciphertext ││ 3. Generate 16-byte authentication tag ││ 4. Return: [nonce|ciphertext|tag] │└────────────────────┬────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────┐│ RocksDB ││ Store encrypted blob to disk │└─────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────┐│ Disk Storage ││ Binary data (unintelligible without key) │└─────────────────────────────────────────────────────────┘Data Format on Disk
Each encrypted value follows this format:
┌──────────────┬──────────────────┬─────────────────┐│ Nonce │ Ciphertext │ Auth Tag ││ (12 bytes) │ (variable) │ (16 bytes) │└──────────────┴──────────────────┴─────────────────┘Example:
Plaintext: "Hello, World!" (13 bytes)Encrypted: [12-byte nonce][13-byte ciphertext][16-byte tag] = 41 bytes totalOverhead: 28 bytes (12 + 16)Setting Up Encryption
Prerequisites
Before enabling encryption, ensure you have:
- ✅ HeliosDB Nano 2.5.0 or later
- ✅ Secure method to store encryption keys
- ✅ Backup strategy for keys
- ✅ Understanding of key management policies
Method 1: Environment Variable Key (Development)
Step 1: Generate an Encryption Key
# Using HeliosDB Nano CLIheliosdb-nano keygen
# Or using OpenSSLopenssl rand -hex 32# Example output: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdefStep 2: Set Environment Variable
# Linux/macOSexport HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Windows PowerShell$env:HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Windows CMDset HELIOSDB_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdefStep 3: Configure Encrypted Database (Rust)
use heliosdb_nano::{Config, storage::StorageEngine};
fn main() -> heliosdb_nano::Result<()> { // Create configuration with encryption enabled let mut config = Config::default(); config.storage.path = Some("./encrypted_db".into());
// Enable encryption config.encryption.enabled = true; config.encryption.key_source = heliosdb_nano::config::KeySource::Environment( "HELIOSDB_ENCRYPTION_KEY".to_string() );
// Open encrypted database let storage = StorageEngine::open("./encrypted_db", &config)?;
println!("✓ Encrypted database opened successfully"); println!("✓ Encryption: {}", storage.encryption_info().unwrap_or("Disabled".to_string()));
Ok(())}Step 4: Verify Encryption
// Insert some datastorage.put(b"test_key", b"sensitive_data")?;
// Verify it's encrypted on disklet raw_data = storage.db.get(b"test_key")?.unwrap();assert_ne!(raw_data, b"sensitive_data"); // Should be encrypted
// But transparent decryption workslet decrypted = storage.get(b"test_key")?.unwrap();assert_eq!(decrypted, b"sensitive_data"); // ✓ Correct plaintextMethod 2: File-Based Key (Production)
Step 1: Generate and Store Key File
# Generate key and save to fileheliosdb-nano keygen > /secure/path/encryption.key
# Verify key file (should be 64 hex characters)cat /secure/path/encryption.key# Output: 0123456789abcdef...
# Set strict file permissionschmod 600 /secure/path/encryption.keychown dbuser:dbgroup /secure/path/encryption.keyStep 2: Configure with File-Based Key
use heliosdb_nano::{Config, storage::StorageEngine};
fn main() -> heliosdb_nano::Result<()> { let mut config = Config::default(); config.storage.path = Some("./production_db".into());
// Use file-based key config.encryption.enabled = true; config.encryption.key_source = heliosdb_nano::config::KeySource::File( "/secure/path/encryption.key".into() );
let storage = StorageEngine::open("./production_db", &config)?; println!("✓ Production database encrypted with file-based key");
Ok(())}Step 3: Configuration File (TOML)
[storage]path = "/var/lib/heliosdb/data"
[encryption]enabled = truealgorithm = "Aes256Gcm"
[encryption.key_source]File = "/etc/heliosdb/encryption.key"
# Optional: Key rotation settings (future feature)rotation_interval_days = 90Load configuration:
let config = Config::from_file("config.toml")?;let storage = StorageEngine::open("/var/lib/heliosdb/data", &config)?;Method 3: Password-Based Key (Interactive)
Use Case: User-specific databases, encrypted backups
use heliosdb_nano::crypto::KeyManager;
fn setup_password_based_encryption() -> heliosdb_nano::Result<()> { // Get password from user (use secure input in production) let password = rpassword::prompt_password("Enter encryption password: ")?;
// Generate or load salt (must be stored for key derivation) let salt = load_or_generate_salt()?; // 16+ bytes
// Derive key from password let key_manager = KeyManager::from_password(&password, &salt)?;
println!("✓ Key derived from password using Argon2"); println!("✓ Salt: {} bytes", salt.len());
// Use the derived key // Note: You'll need to pass this to StorageEngine manually // or store it temporarily in an environment variable
Ok(())}
fn load_or_generate_salt() -> heliosdb_nano::Result<Vec<u8>> { let salt_path = "./.db_salt";
if std::path::Path::new(salt_path).exists() { // Load existing salt Ok(std::fs::read(salt_path)?) } else { // Generate new salt (16 bytes minimum) let salt: [u8; 16] = rand::random(); std::fs::write(salt_path, &salt)?; println!("✓ Generated new salt: {}", salt_path); Ok(salt.to_vec()) }}Method 4: KMS Integration (Enterprise - Planned)
Future Support (Phase 2):
// AWS KMS example (planned)config.encryption.key_source = heliosdb_nano::config::KeySource::Kms { provider: "aws".to_string(), key_id: "alias/heliosdb-production-key".to_string(),};
// Azure Key Vault example (planned)config.encryption.key_source = heliosdb_nano::config::KeySource::Kms { provider: "azure".to_string(), key_id: "https://myvault.vault.azure.net/keys/heliosdb-key".to_string(),};
// Google Cloud KMS example (planned)config.encryption.key_source = heliosdb_nano::config::KeySource::Kms { provider: "gcp".to_string(), key_id: "projects/my-project/locations/us/keyRings/my-ring/cryptoKeys/heliosdb".to_string(),};Key Management
Key Storage Options Comparison
| Method | Security | Ease of Use | Production Ready | Use Case |
|---|---|---|---|---|
| Environment Variable | ⚠️ Medium | ✅ Easy | ⚠️ Dev/Test Only | Local development |
| File-Based | ✅ Good | ✅ Easy | ✅ Yes (with permissions) | Single-server deployments |
| Password-Derived | ⚠️ Medium | ⚡ Interactive | ⚠️ Limited | User-specific databases |
| KMS/HSM | ✅ Excellent | ⚡ Complex | ✅ Yes | Enterprise production |
Key Generation Best Practices
Generate Cryptographically Strong Keys:
use heliosdb_nano::crypto::KeyManager;
// Method 1: Using KeyManagerlet km = KeyManager::generate_random();let hex_key = km.export_as_hex();println!("Generated key: {}", hex_key);
// Method 2: Using OpenSSL (CLI)// openssl rand -hex 32
// Method 3: Using /dev/urandom (Linux)// dd if=/dev/urandom bs=32 count=1 2>/dev/null | xxd -p -c 32Key Properties:
- ✅ 256 bits (32 bytes) of random data
- ✅ Generated from cryptographically secure source (
rand::random()) - ✅ Encoded as 64 hexadecimal characters
- ❌ Never use predictable values (timestamps, counters, etc.)
- ❌ Never reuse keys across databases
Key Backup and Storage
Backup Strategy:
# 1. Backup key to secure locationcp /etc/heliosdb/encryption.key /secure/backup/encryption.key.backup
# 2. Store in password manager (recommended for small teams)# Example: 1Password, LastPass, Bitwarden
# 3. Use key escrow service (enterprise)# Store encrypted key with recovery mechanism
# 4. Hardware Security Module (HSM)# For highest security requirementsStorage Best Practices:
# Set minimal permissions (owner read-only)chmod 400 /etc/heliosdb/encryption.keychown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Verify permissionsls -la /etc/heliosdb/encryption.key# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.keyEnvironment Variable Security:
# For systemd services[Service]Environment="HELIOSDB_ENCRYPTION_KEY=<key>"EnvironmentFile=/etc/heliosdb/secrets.env # Preferred
# For Dockerdocker run -e HELIOSDB_ENCRYPTION_KEY=<key> heliosdb-nano# Or use Docker secrets (recommended)docker secret create heliosdb_key encryption.key
# For Kuberneteskubectl create secret generic heliosdb-encryption \ --from-file=key=/path/to/encryption.keyKey Rotation Procedure
Current Limitation: Automatic key rotation is planned for Phase 2. Manual rotation requires:
Manual Rotation Steps:
# 1. Backup databaseheliosdb-nano backup --input /var/lib/heliosdb --output /backup/db.tar.gz
# 2. Generate new keyNEW_KEY=$(heliosdb-nano keygen)echo $NEW_KEY > /tmp/new_encryption.key
# 3. Export data (decrypted)heliosdb-nano export --input /var/lib/heliosdb --output /tmp/export.sql
# 4. Create new database with new keyexport HELIOSDB_ENCRYPTION_KEY=$NEW_KEYheliosdb-nano init --path /var/lib/heliosdb-new
# 5. Import data (encrypted with new key)heliosdb-nano import --input /tmp/export.sql --output /var/lib/heliosdb-new
# 6. Verify and switchmv /var/lib/heliosdb /var/lib/heliosdb-oldmv /var/lib/heliosdb-new /var/lib/heliosdb
# 7. Update key in secure storagecp /tmp/new_encryption.key /etc/heliosdb/encryption.keychmod 400 /etc/heliosdb/encryption.keyPlanned Automatic Rotation (Phase 2):
// Future APIstorage.rotate_encryption_key(&new_key_manager)?;// - Online rotation (no downtime)// - Gradual re-encryption in background// - Dual-key support during transitionKey Recovery
Lost Key Scenarios:
| Scenario | Recovery Possible? | Solution |
|---|---|---|
| Key file deleted | ✅ Yes | Restore from backup |
| Backup exists | ✅ Yes | Use backup key |
| No backup | ❌ NO | Data is permanently lost |
| Wrong key used | ❌ No | Must use correct original key |
| Corrupted key | ❌ No | Need exact original key |
Recovery Procedure:
# 1. Locate backup keyls -la /backup/keys/
# 2. Verify key is correct length (64 hex chars)wc -c /backup/keys/encryption.key.backup# Expected: 64
# 3. Test key with small database copycp -r /var/lib/heliosdb /tmp/test_dbexport HELIOSDB_ENCRYPTION_KEY=$(cat /backup/keys/encryption.key.backup)heliosdb-nano verify --path /tmp/test_db
# 4. If successful, restore to productioncp /backup/keys/encryption.key.backup /etc/heliosdb/encryption.keysystemctl restart heliosdbWorking with Encrypted Data
Transparent Operations (No Code Changes)
Good News: Encryption is completely transparent! Your application code doesn’t change.
Without Encryption:
// Regular database operationslet storage = StorageEngine::open("./db", &config)?;storage.put(b"key1", b"value1")?;let value = storage.get(b"key1")?.unwrap();With Encryption:
// Exact same code! Encryption is automaticlet storage = StorageEngine::open("./db", &config)?; // Opens encrypted DBstorage.put(b"key1", b"value1")?; // Automatically encryptedlet value = storage.get(b"key1")?.unwrap(); // Automatically decryptedTable Operations
Create Encrypted Tables:
use heliosdb_nano::{Schema, Column, DataType, Tuple, Value};
// Schema definition (same as without encryption)let schema = Schema::new(vec![ Column::new("id", DataType::Int4).primary_key(), Column::new("ssn", DataType::Text), Column::new("credit_card", DataType::Text), Column::new("salary", DataType::Float8),]);
// Create table (metadata is encrypted)let catalog = storage.catalog();catalog.create_table("employees", schema)?;
// Insert data (automatically encrypted)storage.insert_tuple("employees", Tuple::new(vec![ Value::Int4(1), Value::String("123-45-6789".to_string()), Value::String("4111-1111-1111-1111".to_string()), Value::Float8(125000.0),]))?;
// Query data (automatically decrypted)let employees = storage.scan_table("employees")?;for emp in employees { println!("Employee: {:?}", emp); // All values are automatically decrypted}SQL Queries (Server Mode)
PostgreSQL Protocol (encryption is transparent):
-- Connect to encrypted databasepsql -h localhost -p 5432 -d encrypted_db
-- All queries work normallyCREATE TABLE patients ( id SERIAL PRIMARY KEY, name TEXT NOT NULL, diagnosis TEXT, prescription TEXT);
-- Insert sensitive medical data (encrypted automatically)INSERT INTO patients (name, diagnosis, prescription)VALUES ('John Doe', 'Type 2 Diabetes', 'Metformin 500mg');
-- Query returns decrypted dataSELECT * FROM patients WHERE id = 1;
-- Joins, aggregations, etc. all workSELECT diagnosis, COUNT(*)FROM patientsGROUP BY diagnosis;Performance Considerations
Encryption Overhead:
Based on benchmarks:
| Operation | Unencrypted | Encrypted | Overhead |
|---|---|---|---|
| Small writes (64B) | 1.2 µs | 1.23 µs | +2.5% |
| Medium writes (1KB) | 1.5 µs | 1.54 µs | +2.7% |
| Large writes (16KB) | 3.8 µs | 3.91 µs | +2.9% |
| Point reads | 0.8 µs | 0.82 µs | +2.5% |
| Table scans (1000 rows) | 850 µs | 875 µs | +2.9% |
| Indexed lookups | 1.1 µs | 1.13 µs | +2.7% |
Performance Tips:
-
Use Hardware Acceleration:
Terminal window # Verify CPU supports AES-NIlscpu | grep aes# Or on macOS:sysctl -a | grep aes# If available, encryption overhead is minimal (<3%) -
Batch Operations:
// Less efficient: Many small writesfor i in 0..1000 {storage.put(&format!("key_{}", i).as_bytes(), b"value")?;}// More efficient: Batch insertslet tuples: Vec<Tuple> = (0..1000).map(|i| Tuple::new(vec![Value::Int4(i), Value::String("data".into())])).collect();storage.insert_batch("table", tuples)?; -
Memory Considerations:
// Each encrypted value has 28-byte overhead (12-byte nonce + 16-byte tag)// For small values, this can be significant// Example:// 1-byte value → 29 bytes encrypted (29x overhead)// 100-byte value → 128 bytes encrypted (1.28x overhead)// 1KB value → 1052 bytes encrypted (1.05x overhead)// Recommendation: Use larger values when possible
Debugging Encrypted Databases
Check Encryption Status:
let storage = StorageEngine::open("./db", &config)?;
// Method 1: Check encryption flagif storage.is_encrypted() { println!("✓ Database is encrypted");} else { println!("⚠ Database is NOT encrypted");}
// Method 2: Get encryption infoif let Some(info) = storage.encryption_info() { println!("Encryption: {}", info); // Output: "Enabled: AES-256-GCM"} else { println!("Encryption: Disabled");}Verify Data is Encrypted on Disk:
// Insert test datastorage.put(b"test_key", b"plaintext_value")?;
// Check raw data in RocksDBlet raw_data = storage.db.get(b"test_key")?.unwrap();
// Raw data should NOT match plaintextassert_ne!(raw_data, b"plaintext_value", "Data is not encrypted!");
// But storage.get() should decrypt itlet decrypted = storage.get(b"test_key")?.unwrap();assert_eq!(decrypted, b"plaintext_value", "Decryption failed!");
println!("✓ Data is encrypted on disk");println!("✓ Transparent decryption works");Logging and Monitoring:
// Enable encryption audit logging (future feature)config.audit.log_encryption_events = true;
// Logged events:// - Key loaded from source// - Encryption enabled/disabled// - Encryption errors// - Key rotation (future)Security Best Practices
Key Management Security
DO:
✅ Use strong random keys:
# Good: Cryptographically secure randomopenssl rand -hex 32
# Bad: Predictable or weakecho "my_password_123" | md5sum # ❌ DON'T DO THIS✅ Separate keys from data:
# Good: Key on different volume/server/etc/heliosdb/encryption.key # Different volume from /var/lib/heliosdb
# Bad: Key in same directory as database/var/lib/heliosdb/encryption.key # ❌ Same volume✅ Use proper file permissions:
# Good: Minimal permissionschmod 400 /etc/heliosdb/encryption.keychown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Bad: World-readablechmod 644 /etc/heliosdb/encryption.key # ❌ Others can read✅ Implement key rotation:
# Rotate keys every 90 days (recommended)# Rotate immediately after:# - Security incident# - Employee departure# - Suspected compromise✅ Backup keys securely:
# Good: Encrypted backup in separate locationgpg --encrypt --recipient admin@company.com encryption.keyaws s3 cp encryption.key.gpg s3://secure-backups/keys/
# Good: Hardware security module (HSM)# Store master key in HSM, use derived keys for databasesDON’T:
❌ Never commit keys to version control:
# Add to .gitignoreecho "*.key" >> .gitignoreecho ".env" >> .gitignoreecho "config/secrets.toml" >> .gitignore❌ Never log or print keys:
// Bad: Logs the keyprintln!("Key: {}", key_manager.export_as_hex()); // ❌ DON'T
// Good: Log only metadataprintln!("Key loaded from: {}", key_manager.source()); // ✅ OK❌ Never use hardcoded keys:
// Bad: Hardcoded key in source codeconst ENCRYPTION_KEY: &str = "0123..."; // ❌ NEVER DO THIS
// Good: Load from secure sourcelet key = KeyManager::from_source(&config.encryption.key_source)?; // ✅❌ Never share keys between environments:
# Bad: Same key for dev, staging, prod# HELIOSDB_ENCRYPTION_KEY=same_key_everywhere # ❌
# Good: Different key per environment# DEV: Different key for development# STAGING: Different key for staging# PROD: Different key for productionAccess Control
Principle of Least Privilege:
# Database process useruseradd -r -s /bin/false heliosdb
# Key file permissionschown root:heliosdb /etc/heliosdb/encryption.keychmod 440 /etc/heliosdb/encryption.key
# Database fileschown heliosdb:heliosdb /var/lib/heliosdbchmod 700 /var/lib/heliosdb
# Only heliosdb user can access database and keysNetwork Security (Server Mode):
[server]host = "127.0.0.1" # Localhost only (use reverse proxy for external)port = 5432ssl_enabled = true # Always use SSL/TLS for network traffic
[server.ssl]cert_file = "/etc/heliosdb/server.crt"key_file = "/etc/heliosdb/server.key"ca_file = "/etc/heliosdb/ca.crt"Integration with Audit Logging
Enable Comprehensive Auditing:
use heliosdb_nano::Config;
let mut config = Config::default();
// Enable encryptionconfig.encryption.enabled = true;config.encryption.key_source = /* ... */;
// Enable audit loggingconfig.audit.enabled = true;config.audit.log_file = Some("/var/log/heliosdb/audit.log".into());config.audit.log_format = heliosdb_nano::audit::LogFormat::Json;
// Audit all data accessconfig.audit.log_reads = true;config.audit.log_writes = true;config.audit.log_deletes = true;
// Combined: Encrypted data + full audit traillet storage = StorageEngine::open("./db", &config)?;Audit Log Example (JSON format):
{ "timestamp": "2025-12-01T10:30:45Z", "event_type": "read", "table": "patients", "user": "doctor_smith", "query": "SELECT * FROM patients WHERE id = 123", "rows_affected": 1, "encryption_enabled": true, "source_ip": "192.168.1.100"}Compliance Reporting:
# Generate encryption compliance reportheliosdb-nano audit-report \ --input /var/log/heliosdb/audit.log \ --type encryption-compliance \ --output /reports/encryption_compliance_2025_12.pdfCompliance Scenarios
HIPAA Compliance Setup
Health Insurance Portability and Accountability Act requirements:
Required Controls:
- ✅ Encryption at rest (covered by TDE)
- ✅ Encryption in transit (use SSL/TLS)
- ✅ Access controls (OS-level permissions)
- ✅ Audit logging (enable audit logs)
- ✅ Key management (secure key storage)
Configuration:
[encryption]enabled = truealgorithm = "Aes256Gcm"
[encryption.key_source]File = "/secure/hipaa/encryption.key"
[server]ssl_enabled = true
[server.ssl]cert_file = "/etc/heliosdb/hipaa_server.crt"key_file = "/etc/heliosdb/hipaa_server.key"min_tls_version = "1.2" # HIPAA requires TLS 1.2+
[audit]enabled = truelog_file = "/var/log/heliosdb/hipaa_audit.log"log_format = "Json"log_reads = truelog_writes = truelog_deletes = truelog_schema_changes = true
[audit.retention]days = 2555 # 7 years (HIPAA requirement)HIPAA-Compliant Rust Code:
use heliosdb_nano::{Config, Schema, Column, DataType, Tuple, Value};
fn setup_hipaa_database() -> heliosdb_nano::Result<()> { // Load HIPAA configuration let config = Config::from_file("hipaa_config.toml")?;
// Verify encryption is enabled assert!(config.encryption.enabled, "HIPAA requires encryption!"); assert!(config.audit.enabled, "HIPAA requires audit logging!");
// Open database let storage = StorageEngine::open("/var/lib/heliosdb/hipaa", &config)?;
// Create PHI table (Protected Health Information) let catalog = storage.catalog(); let schema = Schema::new(vec![ Column::new("patient_id", DataType::Int4).primary_key(), Column::new("name", DataType::Text), Column::new("ssn", DataType::Text), Column::new("diagnosis", DataType::Text), Column::new("medications", DataType::Text), Column::new("created_at", DataType::Timestamp), ]);
catalog.create_table("patient_records", schema)?;
println!("✓ HIPAA-compliant database configured"); println!(" - Encryption: AES-256-GCM"); println!(" - SSL/TLS: Enabled"); println!(" - Audit logging: 7-year retention");
Ok(())}Verification Checklist:
# 1. Verify encryption is enabledheliosdb-nano verify --config hipaa_config.toml --check encryption
# 2. Verify SSL/TLS certificatesopenssl s_client -connect localhost:5432 -showcerts
# 3. Verify audit loggingtail -f /var/log/heliosdb/hipaa_audit.log
# 4. Verify file permissionsls -la /var/lib/heliosdb/hipaa# Expected: drwx------ (700)
# 5. Verify key securityls -la /secure/hipaa/encryption.key# Expected: -r-------- (400)GDPR Data Protection
General Data Protection Regulation requirements:
GDPR Principles:
- Data Protection by Design: Encryption enabled from day one
- Right to Erasure: Secure deletion of personal data
- Data Portability: Export encrypted backups
- Breach Notification: Audit logs for incident response
- Privacy by Default: Minimal data collection
GDPR Configuration:
[encryption]enabled = truealgorithm = "Aes256Gcm"key_source = { File = "/etc/heliosdb/gdpr_key" }
[audit]enabled = truelog_file = "/var/log/heliosdb/gdpr_audit.log"log_format = "Json"
# Log personal data access (GDPR Article 15)log_reads = truelog_writes = truelog_deletes = true
# Retention policy (GDPR Article 17)[audit.retention]days = 730 # 2 years minimum
[storage]# Enable secure deletion (GDPR "Right to Erasure")secure_delete = trueoverwrite_deleted_data = trueGDPR-Compliant Data Operations:
// Right to Access (GDPR Article 15)fn export_user_data(storage: &StorageEngine, user_id: i32) -> heliosdb_nano::Result<String>{ // Query all user data (automatically decrypted) let query = format!("SELECT * FROM users WHERE id = {}", user_id); let results = storage.execute_sql(&query)?;
// Export as JSON (for data portability) let json = serde_json::to_string_pretty(&results)?;
// Log access (for audit trail) log::info!("User data exported for user_id={}", user_id);
Ok(json)}
// Right to Erasure (GDPR Article 17)fn delete_user_data(storage: &StorageEngine, user_id: i32) -> heliosdb_nano::Result<()>{ // Delete user from all tables storage.execute_sql(&format!("DELETE FROM users WHERE id = {}", user_id))?; storage.execute_sql(&format!("DELETE FROM orders WHERE user_id = {}", user_id))?; storage.execute_sql(&format!("DELETE FROM preferences WHERE user_id = {}", user_id))?;
// Secure deletion (overwrite on disk) // Enabled via config.storage.secure_delete = true
// Log deletion (compliance requirement) log::warn!("User data deleted for user_id={} (GDPR erasure)", user_id);
Ok(())}
// Data Breach Notification (GDPR Article 33)fn check_encryption_breach(storage: &StorageEngine) -> heliosdb_nano::Result<()> { if !storage.is_encrypted() { log::error!("CRITICAL: Database is not encrypted! GDPR violation!"); // Trigger incident response notify_data_protection_officer("Unencrypted database detected")?; } Ok(())}PCI-DSS Requirements
Payment Card Industry Data Security Standard:
PCI-DSS Requirements for Encryption:
- Requirement 3: Protect stored cardholder data
- Requirement 4: Encrypt transmission of cardholder data
- Requirement 8: Identify and authenticate access
PCI-DSS Configuration:
[encryption]enabled = true # Requirement 3.4: Render PAN unreadablealgorithm = "Aes256Gcm" # Strong cryptography (Requirement 3.5)
[encryption.key_source]File = "/secure/pci/encryption.key"
[encryption.key_rotation]enabled = trueinterval_days = 90 # Key rotation (Requirement 3.6.4)
[audit]enabled = true # Requirement 10: Track and monitor accesslog_file = "/var/log/heliosdb/pci_audit.log"
# Log all access to cardholder datalog_reads = truelog_writes = truelog_deletes = truelog_schema_changes = true
[audit.retention]days = 365 # Minimum 1 year (Requirement 10.7)
[server.ssl]enabled = true # Requirement 4: Encrypt in transitmin_tls_version = "1.2"PCI-DSS Compliant Schema:
use heliosdb_nano::{Schema, Column, DataType};
fn create_pci_compliant_schema() -> Schema { Schema::new(vec![ Column::new("transaction_id", DataType::Int8).primary_key(),
// PAN (Primary Account Number) - MUST be encrypted Column::new("card_number", DataType::Text),
// Cardholder name Column::new("cardholder_name", DataType::Text),
// Expiration date - Can be stored with encryption Column::new("expiry_month", DataType::Int2), Column::new("expiry_year", DataType::Int4),
// NEVER store these (PCI-DSS Requirement 3.2): // - CVV/CVV2 (Card Verification Value) // - Full magnetic stripe data // - PIN or PIN block
Column::new("amount", DataType::Numeric), Column::new("timestamp", DataType::Timestamp), ])}
// Example: Storing payment card data (encrypted)fn process_payment(storage: &StorageEngine, card_data: CardData) -> heliosdb_nano::Result<()>{ // Data is automatically encrypted by TDE storage.insert_tuple("transactions", Tuple::new(vec![ Value::Int8(generate_transaction_id()), Value::String(card_data.pan), // Encrypted at rest Value::String(card_data.name), // Encrypted at rest Value::Int2(card_data.exp_month), Value::Int4(card_data.exp_year), Value::Numeric(card_data.amount), Value::Timestamp(Utc::now()), ]))?;
// Audit log automatically records access
Ok(())}PCI-DSS Compliance Verification:
# 1. Verify encryption (Requirement 3.4)heliosdb-nano verify --config pci_config.toml --check encryption# ✓ AES-256-GCM enabled
# 2. Verify key management (Requirement 3.5, 3.6)ls -la /secure/pci/encryption.key# ✓ Permissions: 400 (read-only by owner)
# 3. Verify audit logging (Requirement 10)tail /var/log/heliosdb/pci_audit.log# ✓ All access logged with timestamps
# 4. Verify SSL/TLS (Requirement 4)openssl s_client -connect localhost:5432 | grep Protocol# ✓ TLSv1.2 or higher
# 5. Check for prohibited data (Requirement 3.2)grep -i "cvv\|cvv2\|cvc" /var/lib/heliosdb/pci/# ✓ No results (good - not storing CVV)Recovery Scenarios
Lost Key Recovery
Scenario: Encryption key file was accidentally deleted.
If you have a backup:
# 1. Stop databasesystemctl stop heliosdb
# 2. Restore key from backupcp /backup/keys/encryption.key.20251201 /etc/heliosdb/encryption.key
# 3. Set correct permissionschmod 400 /etc/heliosdb/encryption.keychown heliosdb:heliosdb /etc/heliosdb/encryption.key
# 4. Verify key worksheliosdb-nano verify --config /etc/heliosdb/config.toml --check encryption
# 5. Restart databasesystemctl start heliosdbsystemctl status heliosdbIf you DON’T have a backup:
⚠️ CRITICAL: Without the encryption key, your data is PERMANENTLY LOST.
There is NO way to recover encrypted data without the original key.
Prevention:- Always maintain secure backups of encryption keys- Store backups in multiple locations- Use key management services (KMS/HSM)- Document key recovery procedures- Test recovery procedures regularlyDatabase Migration with Encryption
Scenario: Migrate encrypted database to new server.
Step-by-Step Migration:
# On old server:
# 1. Backup encrypted databasetar -czf database_backup.tar.gz /var/lib/heliosdb/
# 2. Backup encryption key SEPARATELYtar -czf encryption_key.tar.gz /etc/heliosdb/encryption.key
# 3. Transfer to new server (use secure channel)scp database_backup.tar.gz user@newserver:/tmp/scp encryption_key.tar.gz user@newserver:/tmp/
# On new server:
# 4. Extract databasemkdir -p /var/lib/heliosdbtar -xzf /tmp/database_backup.tar.gz -C /
# 5. Extract encryption keymkdir -p /etc/heliosdbtar -xzf /tmp/encryption_key.tar.gz -C /
# 6. Set permissionschmod 700 /var/lib/heliosdbchown heliosdb:heliosdb /var/lib/heliosdbchmod 400 /etc/heliosdb/encryption.keychown heliosdb:heliosdb /etc/heliosdb/encryption.key
# 7. Verify database opens correctlyheliosdb-nano verify --path /var/lib/heliosdb
# 8. Start databasesystemctl start heliosdb
# 9. Test connectivitypsql -h localhost -p 5432 -c "SELECT 1"Alternative: Export/Import Method:
# On old server:# 1. Export data in decrypted formatheliosdb-nano export --input /var/lib/heliosdb --output /tmp/export.sql
# 2. Transfer export filescp /tmp/export.sql user@newserver:/tmp/
# On new server:# 3. Generate NEW encryption key for new serverheliosdb-nano keygen > /etc/heliosdb/encryption.keychmod 400 /etc/heliosdb/encryption.key
# 4. Initialize encrypted database with NEW keyexport HELIOSDB_ENCRYPTION_KEY=$(cat /etc/heliosdb/encryption.key)heliosdb-nano init --path /var/lib/heliosdb --encrypted
# 5. Import data (will be encrypted with NEW key)heliosdb-nano import --input /tmp/export.sql --output /var/lib/heliosdb
# Result: Data re-encrypted with new key on new serverBackup and Restore
Full Encrypted Backup:
# Backup strategy: Backup encrypted database files directly# Advantage: Fast, no decryption needed# Disadvantage: Requires same encryption key for restore
# 1. Consistent snapshot (stop writes)heliosdb-nano checkpoint --path /var/lib/heliosdb
# 2. Backup database files (encrypted)tar -czf /backup/heliosdb_encrypted_20251201.tar.gz /var/lib/heliosdb/
# 3. Backup encryption key SEPARATELY and SECURELYgpg --encrypt --recipient backup@company.com /etc/heliosdb/encryption.keycp encryption.key.gpg /secure/backup/keys/
# 4. Verify backuptar -tzf /backup/heliosdb_encrypted_20251201.tar.gz | head
# 5. Test restore to temporary locationmkdir /tmp/restore_testtar -xzf /backup/heliosdb_encrypted_20251201.tar.gz -C /tmp/restore_testheliosdb-nano verify --path /tmp/restore_test/var/lib/heliosdbDecrypted Logical Backup:
# Backup strategy: Export to SQL (decrypted)# Advantage: Key-independent, can restore with different key# Disadvantage: Slower, decrypted data must be secured
# 1. Export to SQL (automatically decrypts during export)heliosdb-nano export \ --input /var/lib/heliosdb \ --output /backup/heliosdb_export_20251201.sql
# 2. Encrypt the SQL export for securitygpg --encrypt --recipient backup@company.com \ /backup/heliosdb_export_20251201.sql
# 3. Store encrypted SQL filemv heliosdb_export_20251201.sql.gpg /secure/backup/exports/
# 4. Restore proceduregpg --decrypt /secure/backup/exports/heliosdb_export_20251201.sql.gpg \ > /tmp/restore.sql
heliosdb-nano import \ --input /tmp/restore.sql \ --output /var/lib/heliosdb_restored
# 5. Secure cleanupshred -u /tmp/restore.sql # Securely delete decrypted SQLAutomated Backup Script:
#!/bin/bashset -e
DB_PATH="/var/lib/heliosdb"BACKUP_DIR="/backup/heliosdb"KEY_FILE="/etc/heliosdb/encryption.key"DATE=$(date +%Y%m%d_%H%M%S)
# 1. Create consistent snapshotheliosdb-nano checkpoint --path "$DB_PATH"
# 2. Backup database (encrypted)tar -czf "$BACKUP_DIR/db_$DATE.tar.gz" "$DB_PATH"
# 3. Backup key (encrypted with GPG)gpg --encrypt --recipient backup@company.com "$KEY_FILE" \ -o "$BACKUP_DIR/key_$DATE.gpg"
# 4. Verify backup integritytar -tzf "$BACKUP_DIR/db_$DATE.tar.gz" > /dev/null
# 5. Retention: Keep last 30 daysfind "$BACKUP_DIR" -name "db_*.tar.gz" -mtime +30 -deletefind "$BACKUP_DIR" -name "key_*.gpg" -mtime +30 -delete
# 6. Upload to cloud (optional)# aws s3 cp "$BACKUP_DIR/db_$DATE.tar.gz" s3://backups/heliosdb/# aws s3 cp "$BACKUP_DIR/key_$DATE.gpg" s3://backups/heliosdb/keys/
echo "✓ Backup completed: $DATE"Schedule with cron:
# Run daily at 2 AM0 2 * * * /usr/local/bin/backup_encrypted_db.sh >> /var/log/heliosdb_backup.log 2>&1Performance Considerations
Benchmarking Encrypted vs Unencrypted
Run Built-in Benchmarks:
# Run encryption benchmarkscargo bench --bench encryption_benchmark
# Sample output:# Encryption/put_get/64_bytes time: [1.23 µs 1.25 µs 1.27 µs]# Unencrypted/put_get/64_bytes time: [1.20 µs 1.22 µs 1.24 µs]# Overhead: ~2.5%Custom Performance Testing:
use std::time::Instant;use heliosdb_nano::{Config, storage::StorageEngine};
fn benchmark_encryption_overhead() -> heliosdb_nano::Result<()> { let iterations = 10_000;
// Test 1: Encrypted database let mut config = Config::in_memory(); config.encryption.enabled = true; config.encryption.key_source = /* ... */;
let encrypted_storage = StorageEngine::open_in_memory(&config)?;
let start = Instant::now(); for i in 0..iterations { encrypted_storage.put( &format!("key_{}", i).as_bytes(), b"test_data_payload_1234567890" )?; } let encrypted_duration = start.elapsed();
// Test 2: Unencrypted database let mut config_unenc = Config::in_memory(); config_unenc.encryption.enabled = false;
let unencrypted_storage = StorageEngine::open_in_memory(&config_unenc)?;
let start = Instant::now(); for i in 0..iterations { unencrypted_storage.put( &format!("key_{}", i).as_bytes(), b"test_data_payload_1234567890" )?; } let unencrypted_duration = start.elapsed();
// Calculate overhead let overhead = (encrypted_duration.as_micros() as f64 / unencrypted_duration.as_micros() as f64 - 1.0) * 100.0;
println!("Results ({} iterations):", iterations); println!(" Unencrypted: {:?}", unencrypted_duration); println!(" Encrypted: {:?}", encrypted_duration); println!(" Overhead: {:.2}%", overhead);
Ok(())}Optimization Tips
1. Use Appropriate Value Sizes:
// Inefficient: Many tiny encrypted values// Each has 28-byte overhead (nonce + tag)for i in 0..1000 { storage.put(&format!("counter_{}", i).as_bytes(), &[i as u8])?; // 1-byte value → 29 bytes stored (29x overhead)}
// Efficient: Batch into larger valueslet counters: Vec<u8> = (0..1000).map(|i| i as u8).collect();storage.put(b"counters", &counters)?;// 1000-byte value → 1028 bytes stored (1.028x overhead)2. Minimize Metadata Operations:
// Less efficient: Frequent catalog queries (metadata is encrypted)for table_name in table_names { let schema = catalog.get_table_schema(table_name)?; // Decrypts metadata process_schema(schema);}
// More efficient: Cache metadatalet schemas: HashMap<String, Schema> = table_names.iter() .map(|name| (name.clone(), catalog.get_table_schema(name))) .collect()?;
for (name, schema) in schemas { process_schema(schema); // No decryption needed}3. Batch Writes:
// Use transactions or batch inserts to amortize encryption overheadstorage.begin_transaction()?;for tuple in large_dataset { storage.insert_tuple("table", tuple)?;}storage.commit()?; // Single fsyncHardware Acceleration
Verify AES-NI Support:
# Linuxlscpu | grep -i aes# Look for: Flags: ... aes ...
cat /proc/cpuinfo | grep aes# Should show "aes" flag
# macOSsysctl -a | grep -i aes# macOS kern.hv_support: 1 (includes AES support)
# Check with Rustcargo build --release# Compiler automatically uses AES-NI if availablePerformance Impact of AES-NI:
| CPU | Without AES-NI | With AES-NI | Speedup |
|---|---|---|---|
| Intel Core i7 | ~15 µs | ~1.2 µs | 12x faster |
| AMD Ryzen 7 | ~14 µs | ~1.1 µs | 13x faster |
| ARM (modern) | ~10 µs | ~0.9 µs | 11x faster |
Note: HeliosDB Nano’s <3% overhead assumes AES-NI is available.
Troubleshooting
Common Issues
Issue: “Environment variable ‘HELIOSDB_ENCRYPTION_KEY’ not found”
Cause: Encryption key not set in environment.
Solution:
# Check if variable is setecho $HELIOSDB_ENCRYPTION_KEY
# Set the variableexport HELIOSDB_ENCRYPTION_KEY="<64-character-hex-key>"
# For systemd servicessudo systemctl edit heliosdb# Add:# [Service]# Environment="HELIOSDB_ENCRYPTION_KEY=<key>"
# For Dockerdocker run -e HELIOSDB_ENCRYPTION_KEY="<key>" heliosdb-nanoIssue: “Decryption failed: Authentication tag verification failed”
Causes:
- Wrong encryption key
- Corrupted data
- Key changed after data was encrypted
Solutions:
# 1. Verify you're using the correct keycat /etc/heliosdb/encryption.key# Should be 64 hex characters
# 2. Check for data corruptionheliosdb-nano verify --path /var/lib/heliosdb
# 3. Try restore from backupcp -r /var/lib/heliosdb /var/lib/heliosdb.corruptedtar -xzf /backup/heliosdb_latest.tar.gz -C /var/lib/
# 4. Restore correct keycp /backup/keys/encryption.key /etc/heliosdb/encryption.keyIssue: “Hex key must be 64 characters (32 bytes), got X”
Cause: Invalid key format.
Solution:
# Generate proper key (64 hex chars)heliosdb-nano keygen# Or:openssl rand -hex 32
# Verify lengthecho -n "<your-key>" | wc -c# Should output: 64Issue: “Failed to read key file: Permission denied”
Cause: Insufficient file permissions.
Solution:
# Check permissionsls -la /etc/heliosdb/encryption.key
# Fix ownershipsudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Fix permissionssudo chmod 400 /etc/heliosdb/encryption.key
# Verify database process userps aux | grep heliosdb# Ensure it's running as 'heliosdb' userPerformance Issues
Issue: “Database operations are slow after enabling encryption”
Diagnosis:
# 1. Check if AES-NI is availablelscpu | grep aes
# 2. Run benchmarkscargo bench --bench encryption_benchmark
# 3. Profile encryption operationscargo build --release --features profilingperf record ./target/release/heliosdb-nanoperf reportSolutions:
// 1. Batch operationsstorage.begin_transaction()?;for item in items { storage.insert_tuple("table", item)?;}storage.commit()?;
// 2. Increase cache sizeconfig.storage.cache_size = 1024 * 1024 * 1024; // 1 GB
// 3. Use in-memory mode for temporary datalet temp_config = Config::in_memory();// Encryption overhead is minimal in-memorySecurity Issues
Issue: “Security audit flagged encryption key in logs”
Immediate Action:
# 1. Rotate key immediatelyNEW_KEY=$(heliosdb-nano keygen)# Follow key rotation procedure
# 2. Scrub logssudo find /var/log -name "*.log" -exec grep -l "HELIOSDB_ENCRYPTION_KEY" {} \; \ -exec shred -u {} \;
# 3. Update code to prevent logging# Remove any println!() or log statements that include keysPrevention:
// Bad: Logs keylog::debug!("Using key: {}", key_manager.export_as_hex()); // ❌
// Good: Logs only metadatalog::debug!("Key loaded from: {:?}", key_manager.source()); // ✅Issue: “Key file has insecure permissions”
Fix:
# Set minimal permissionssudo chmod 400 /etc/heliosdb/encryption.keysudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Verifyls -la /etc/heliosdb/encryption.key# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.key
# Audit permissions recursivelyfind /etc/heliosdb -type f -not -perm 400 -ls# Should be emptyGetting Help
Diagnostic Information to Collect:
# System infouname -alscpu | grep -i aes
# Database infoheliosdb-nano --versionheliosdb-nano verify --path /var/lib/heliosdb
# Configurationcat /etc/heliosdb/config.toml | grep -A 10 encryption
# Logstail -100 /var/log/heliosdb/heliosdb.log
# Permissionsls -laR /etc/heliosdbls -laR /var/lib/heliosdbReport Issues:
- GitHub Issues: https://github.com/dimensigon/HeliosDB Nano/issues
- Security Issues: security@heliosdb.io (for sensitive issues)
- Community Forum: https://discuss.heliosdb.io
Summary
You’ve learned how to:
✅ Enable transparent data encryption with AES-256-GCM ✅ Manage encryption keys securely using multiple methods ✅ Configure encryption for various deployment scenarios ✅ Achieve compliance with HIPAA, GDPR, and PCI-DSS ✅ Maintain performance with <3% overhead ✅ Handle recovery scenarios and migrations ✅ Troubleshoot common encryption issues
Key Takeaways:
- Encryption is transparent: No application code changes needed
- Security is paramount: Protect keys with same rigor as data
- Backup keys separately: Data is unrecoverable without keys
- Monitor performance: Encryption overhead is minimal with AES-NI
- Plan for compliance: Enable audit logging with encryption
- Test recovery procedures: Verify backups and key recovery regularly
Next Steps:
- Explore Audit Logging Tutorial
- Review Security Best Practices
- Set up Automated Backups
- Configure High Availability
Version History:
- 2.5.0-dev (2025-12-01): Initial comprehensive tutorial
- 2.1.0 (2024-11-13): Encryption feature released
Related Documentation: