Skip to content

Transparent Data Encryption Tutorial

Transparent Data Encryption Tutorial

Version: 2.5.0-dev Last Updated: 2025-12-01 Estimated Time: 45 minutes Difficulty: Intermediate


Table of Contents

  1. Introduction
  2. Encryption Architecture
  3. Setting Up Encryption
  4. Key Management
  5. Working with Encrypted Data
  6. Security Best Practices
  7. Compliance Scenarios
  8. Recovery Scenarios
  9. Performance Considerations
  10. Troubleshooting

Introduction

What is Transparent Data Encryption (TDE)?

Transparent Data Encryption (TDE) is a security feature that encrypts data at rest without requiring changes to application code. With TDE enabled, HeliosDB Nano automatically:

  • Encrypts all data before writing to disk
  • Decrypts data when reading from disk
  • Protects against unauthorized access to database files
  • Maintains full query functionality with zero code changes

Why Use Encryption?

Security Benefits:

  • Protects sensitive data from unauthorized file access
  • Prevents data theft if storage media is stolen
  • Ensures data privacy during backups
  • Adds defense-in-depth security layer

Compliance Requirements:

  • HIPAA: Required for healthcare PHI (Protected Health Information)
  • GDPR: Recommended for EU personal data protection
  • PCI-DSS: Mandatory for payment card data
  • SOC 2: Required for Type II certification
  • State Laws: California CCPA, New York SHIELD Act, etc.

How HeliosDB Nano Implements Encryption

HeliosDB Nano uses industry-standard AES-256-GCM (Advanced Encryption Standard with Galois/Counter Mode):

  • Algorithm: AES-256 (NIST FIPS 140-2 approved)
  • Mode: GCM (provides both encryption and authentication)
  • Key Size: 256 bits (32 bytes)
  • Authentication: Built-in integrity verification
  • Performance: Hardware-accelerated on modern CPUs (AES-NI)

What Gets Encrypted:

  • ✅ All table data (tuples)
  • ✅ Catalog metadata (schemas, column definitions)
  • ✅ Index data
  • ✅ Row counters and internal state

What Doesn’t Get Encrypted:

  • ❌ Database keys (for indexing efficiency)
  • ❌ Configuration files
  • ❌ WAL metadata structure

Encryption Architecture

AES-256-GCM Algorithm

Advanced Encryption Standard (AES-256):

  • Block Cipher: Operates on 128-bit blocks
  • Key Size: 256 bits (highest security level)
  • Rounds: 14 encryption rounds
  • NIST Approved: FIPS 140-2 compliant

Galois/Counter Mode (GCM):

  • Authenticated Encryption: Combines encryption + authentication
  • Authentication Tag: 128-bit tag prevents tampering
  • Counter Mode: Turns block cipher into stream cipher
  • Parallel Processing: Can utilize multiple CPU cores

Key Derivation with Argon2

When using password-based encryption, HeliosDB Nano uses Argon2:

  • Algorithm: Argon2id (winner of Password Hashing Competition)
  • Purpose: Derive 256-bit keys from passwords
  • Protection: Resistant to GPU/ASIC attacks
  • Configurable: Memory-hard and CPU-intensive parameters

Argon2 Parameters:

// Default parameters (balanced security/performance)
let argon2 = Argon2::default(); // ~19ms on modern CPU
// - Memory cost: 19 MiB
// - Time cost: 2 iterations
// - Parallelism: 1 thread

Encryption Flow Diagram

┌─────────────────────────────────────────────────────────┐
│ Application Layer │
│ put(key: "users:1", value: "John Doe") │
└────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ Storage Engine │
│ if encryption_enabled: │
│ encrypted = encrypt(value, encryption_key) │
└────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ AES-256-GCM Encryption │
│ 1. Generate random 12-byte nonce │
│ 2. Encrypt plaintext → ciphertext │
│ 3. Generate 16-byte authentication tag │
│ 4. Return: [nonce|ciphertext|tag] │
└────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ RocksDB │
│ Store encrypted blob to disk │
└─────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ Disk Storage │
│ Binary data (unintelligible without key) │
└─────────────────────────────────────────────────────────┘

Data Format on Disk

Each encrypted value follows this format:

┌──────────────┬──────────────────┬─────────────────┐
│ Nonce │ Ciphertext │ Auth Tag │
│ (12 bytes) │ (variable) │ (16 bytes) │
└──────────────┴──────────────────┴─────────────────┘

Example:

Plaintext: "Hello, World!" (13 bytes)
Encrypted: [12-byte nonce][13-byte ciphertext][16-byte tag] = 41 bytes total
Overhead: 28 bytes (12 + 16)

Setting Up Encryption

Prerequisites

Before enabling encryption, ensure you have:

  1. ✅ HeliosDB Nano 2.5.0 or later
  2. ✅ Secure method to store encryption keys
  3. ✅ Backup strategy for keys
  4. ✅ Understanding of key management policies

Method 1: Environment Variable Key (Development)

Step 1: Generate an Encryption Key

Terminal window
# Using HeliosDB Nano CLI
heliosdb-nano keygen
# Or using OpenSSL
openssl rand -hex 32
# Example output: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

Step 2: Set Environment Variable

Terminal window
# Linux/macOS
export HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Windows PowerShell
$env:HELIOSDB_ENCRYPTION_KEY="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
# Windows CMD
set HELIOSDB_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

Step 3: Configure Encrypted Database (Rust)

use heliosdb_nano::{Config, storage::StorageEngine};
fn main() -> heliosdb_nano::Result<()> {
// Create configuration with encryption enabled
let mut config = Config::default();
config.storage.path = Some("./encrypted_db".into());
// Enable encryption
config.encryption.enabled = true;
config.encryption.key_source =
heliosdb_nano::config::KeySource::Environment(
"HELIOSDB_ENCRYPTION_KEY".to_string()
);
// Open encrypted database
let storage = StorageEngine::open("./encrypted_db", &config)?;
println!("✓ Encrypted database opened successfully");
println!("✓ Encryption: {}",
storage.encryption_info().unwrap_or("Disabled".to_string()));
Ok(())
}

Step 4: Verify Encryption

// Insert some data
storage.put(b"test_key", b"sensitive_data")?;
// Verify it's encrypted on disk
let raw_data = storage.db.get(b"test_key")?.unwrap();
assert_ne!(raw_data, b"sensitive_data"); // Should be encrypted
// But transparent decryption works
let decrypted = storage.get(b"test_key")?.unwrap();
assert_eq!(decrypted, b"sensitive_data"); // ✓ Correct plaintext

Method 2: File-Based Key (Production)

Step 1: Generate and Store Key File

Terminal window
# Generate key and save to file
heliosdb-nano keygen > /secure/path/encryption.key
# Verify key file (should be 64 hex characters)
cat /secure/path/encryption.key
# Output: 0123456789abcdef...
# Set strict file permissions
chmod 600 /secure/path/encryption.key
chown dbuser:dbgroup /secure/path/encryption.key

Step 2: Configure with File-Based Key

use heliosdb_nano::{Config, storage::StorageEngine};
fn main() -> heliosdb_nano::Result<()> {
let mut config = Config::default();
config.storage.path = Some("./production_db".into());
// Use file-based key
config.encryption.enabled = true;
config.encryption.key_source =
heliosdb_nano::config::KeySource::File(
"/secure/path/encryption.key".into()
);
let storage = StorageEngine::open("./production_db", &config)?;
println!("✓ Production database encrypted with file-based key");
Ok(())
}

Step 3: Configuration File (TOML)

config.toml
[storage]
path = "/var/lib/heliosdb/data"
[encryption]
enabled = true
algorithm = "Aes256Gcm"
[encryption.key_source]
File = "/etc/heliosdb/encryption.key"
# Optional: Key rotation settings (future feature)
rotation_interval_days = 90

Load configuration:

let config = Config::from_file("config.toml")?;
let storage = StorageEngine::open("/var/lib/heliosdb/data", &config)?;

Method 3: Password-Based Key (Interactive)

Use Case: User-specific databases, encrypted backups

use heliosdb_nano::crypto::KeyManager;
fn setup_password_based_encryption() -> heliosdb_nano::Result<()> {
// Get password from user (use secure input in production)
let password = rpassword::prompt_password("Enter encryption password: ")?;
// Generate or load salt (must be stored for key derivation)
let salt = load_or_generate_salt()?; // 16+ bytes
// Derive key from password
let key_manager = KeyManager::from_password(&password, &salt)?;
println!("✓ Key derived from password using Argon2");
println!("✓ Salt: {} bytes", salt.len());
// Use the derived key
// Note: You'll need to pass this to StorageEngine manually
// or store it temporarily in an environment variable
Ok(())
}
fn load_or_generate_salt() -> heliosdb_nano::Result<Vec<u8>> {
let salt_path = "./.db_salt";
if std::path::Path::new(salt_path).exists() {
// Load existing salt
Ok(std::fs::read(salt_path)?)
} else {
// Generate new salt (16 bytes minimum)
let salt: [u8; 16] = rand::random();
std::fs::write(salt_path, &salt)?;
println!("✓ Generated new salt: {}", salt_path);
Ok(salt.to_vec())
}
}

Method 4: KMS Integration (Enterprise - Planned)

Future Support (Phase 2):

// AWS KMS example (planned)
config.encryption.key_source = heliosdb_nano::config::KeySource::Kms {
provider: "aws".to_string(),
key_id: "alias/heliosdb-production-key".to_string(),
};
// Azure Key Vault example (planned)
config.encryption.key_source = heliosdb_nano::config::KeySource::Kms {
provider: "azure".to_string(),
key_id: "https://myvault.vault.azure.net/keys/heliosdb-key".to_string(),
};
// Google Cloud KMS example (planned)
config.encryption.key_source = heliosdb_nano::config::KeySource::Kms {
provider: "gcp".to_string(),
key_id: "projects/my-project/locations/us/keyRings/my-ring/cryptoKeys/heliosdb".to_string(),
};

Key Management

Key Storage Options Comparison

MethodSecurityEase of UseProduction ReadyUse Case
Environment Variable⚠️ Medium✅ Easy⚠️ Dev/Test OnlyLocal development
File-Based✅ Good✅ Easy✅ Yes (with permissions)Single-server deployments
Password-Derived⚠️ Medium⚡ Interactive⚠️ LimitedUser-specific databases
KMS/HSM✅ Excellent⚡ Complex✅ YesEnterprise production

Key Generation Best Practices

Generate Cryptographically Strong Keys:

use heliosdb_nano::crypto::KeyManager;
// Method 1: Using KeyManager
let km = KeyManager::generate_random();
let hex_key = km.export_as_hex();
println!("Generated key: {}", hex_key);
// Method 2: Using OpenSSL (CLI)
// openssl rand -hex 32
// Method 3: Using /dev/urandom (Linux)
// dd if=/dev/urandom bs=32 count=1 2>/dev/null | xxd -p -c 32

Key Properties:

  • ✅ 256 bits (32 bytes) of random data
  • ✅ Generated from cryptographically secure source (rand::random())
  • ✅ Encoded as 64 hexadecimal characters
  • Never use predictable values (timestamps, counters, etc.)
  • Never reuse keys across databases

Key Backup and Storage

Backup Strategy:

Terminal window
# 1. Backup key to secure location
cp /etc/heliosdb/encryption.key /secure/backup/encryption.key.backup
# 2. Store in password manager (recommended for small teams)
# Example: 1Password, LastPass, Bitwarden
# 3. Use key escrow service (enterprise)
# Store encrypted key with recovery mechanism
# 4. Hardware Security Module (HSM)
# For highest security requirements

Storage Best Practices:

Terminal window
# Set minimal permissions (owner read-only)
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Verify permissions
ls -la /etc/heliosdb/encryption.key
# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.key

Environment Variable Security:

Terminal window
# For systemd services
[Service]
Environment="HELIOSDB_ENCRYPTION_KEY=<key>"
EnvironmentFile=/etc/heliosdb/secrets.env # Preferred
# For Docker
docker run -e HELIOSDB_ENCRYPTION_KEY=<key> heliosdb-nano
# Or use Docker secrets (recommended)
docker secret create heliosdb_key encryption.key
# For Kubernetes
kubectl create secret generic heliosdb-encryption \
--from-file=key=/path/to/encryption.key

Key Rotation Procedure

Current Limitation: Automatic key rotation is planned for Phase 2. Manual rotation requires:

Manual Rotation Steps:

Terminal window
# 1. Backup database
heliosdb-nano backup --input /var/lib/heliosdb --output /backup/db.tar.gz
# 2. Generate new key
NEW_KEY=$(heliosdb-nano keygen)
echo $NEW_KEY > /tmp/new_encryption.key
# 3. Export data (decrypted)
heliosdb-nano export --input /var/lib/heliosdb --output /tmp/export.sql
# 4. Create new database with new key
export HELIOSDB_ENCRYPTION_KEY=$NEW_KEY
heliosdb-nano init --path /var/lib/heliosdb-new
# 5. Import data (encrypted with new key)
heliosdb-nano import --input /tmp/export.sql --output /var/lib/heliosdb-new
# 6. Verify and switch
mv /var/lib/heliosdb /var/lib/heliosdb-old
mv /var/lib/heliosdb-new /var/lib/heliosdb
# 7. Update key in secure storage
cp /tmp/new_encryption.key /etc/heliosdb/encryption.key
chmod 400 /etc/heliosdb/encryption.key

Planned Automatic Rotation (Phase 2):

// Future API
storage.rotate_encryption_key(&new_key_manager)?;
// - Online rotation (no downtime)
// - Gradual re-encryption in background
// - Dual-key support during transition

Key Recovery

Lost Key Scenarios:

ScenarioRecovery Possible?Solution
Key file deleted✅ YesRestore from backup
Backup exists✅ YesUse backup key
No backupNOData is permanently lost
Wrong key used❌ NoMust use correct original key
Corrupted key❌ NoNeed exact original key

Recovery Procedure:

Terminal window
# 1. Locate backup key
ls -la /backup/keys/
# 2. Verify key is correct length (64 hex chars)
wc -c /backup/keys/encryption.key.backup
# Expected: 64
# 3. Test key with small database copy
cp -r /var/lib/heliosdb /tmp/test_db
export HELIOSDB_ENCRYPTION_KEY=$(cat /backup/keys/encryption.key.backup)
heliosdb-nano verify --path /tmp/test_db
# 4. If successful, restore to production
cp /backup/keys/encryption.key.backup /etc/heliosdb/encryption.key
systemctl restart heliosdb

Working with Encrypted Data

Transparent Operations (No Code Changes)

Good News: Encryption is completely transparent! Your application code doesn’t change.

Without Encryption:

// Regular database operations
let storage = StorageEngine::open("./db", &config)?;
storage.put(b"key1", b"value1")?;
let value = storage.get(b"key1")?.unwrap();

With Encryption:

// Exact same code! Encryption is automatic
let storage = StorageEngine::open("./db", &config)?; // Opens encrypted DB
storage.put(b"key1", b"value1")?; // Automatically encrypted
let value = storage.get(b"key1")?.unwrap(); // Automatically decrypted

Table Operations

Create Encrypted Tables:

use heliosdb_nano::{Schema, Column, DataType, Tuple, Value};
// Schema definition (same as without encryption)
let schema = Schema::new(vec![
Column::new("id", DataType::Int4).primary_key(),
Column::new("ssn", DataType::Text),
Column::new("credit_card", DataType::Text),
Column::new("salary", DataType::Float8),
]);
// Create table (metadata is encrypted)
let catalog = storage.catalog();
catalog.create_table("employees", schema)?;
// Insert data (automatically encrypted)
storage.insert_tuple("employees", Tuple::new(vec![
Value::Int4(1),
Value::String("123-45-6789".to_string()),
Value::String("4111-1111-1111-1111".to_string()),
Value::Float8(125000.0),
]))?;
// Query data (automatically decrypted)
let employees = storage.scan_table("employees")?;
for emp in employees {
println!("Employee: {:?}", emp);
// All values are automatically decrypted
}

SQL Queries (Server Mode)

PostgreSQL Protocol (encryption is transparent):

-- Connect to encrypted database
psql -h localhost -p 5432 -d encrypted_db
-- All queries work normally
CREATE TABLE patients (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
diagnosis TEXT,
prescription TEXT
);
-- Insert sensitive medical data (encrypted automatically)
INSERT INTO patients (name, diagnosis, prescription)
VALUES ('John Doe', 'Type 2 Diabetes', 'Metformin 500mg');
-- Query returns decrypted data
SELECT * FROM patients WHERE id = 1;
-- Joins, aggregations, etc. all work
SELECT diagnosis, COUNT(*)
FROM patients
GROUP BY diagnosis;

Performance Considerations

Encryption Overhead:

Based on benchmarks:

OperationUnencryptedEncryptedOverhead
Small writes (64B)1.2 µs1.23 µs+2.5%
Medium writes (1KB)1.5 µs1.54 µs+2.7%
Large writes (16KB)3.8 µs3.91 µs+2.9%
Point reads0.8 µs0.82 µs+2.5%
Table scans (1000 rows)850 µs875 µs+2.9%
Indexed lookups1.1 µs1.13 µs+2.7%

Performance Tips:

  1. Use Hardware Acceleration:

    Terminal window
    # Verify CPU supports AES-NI
    lscpu | grep aes
    # Or on macOS:
    sysctl -a | grep aes
    # If available, encryption overhead is minimal (<3%)
  2. Batch Operations:

    // Less efficient: Many small writes
    for i in 0..1000 {
    storage.put(&format!("key_{}", i).as_bytes(), b"value")?;
    }
    // More efficient: Batch inserts
    let tuples: Vec<Tuple> = (0..1000)
    .map(|i| Tuple::new(vec![Value::Int4(i), Value::String("data".into())]))
    .collect();
    storage.insert_batch("table", tuples)?;
  3. Memory Considerations:

    // Each encrypted value has 28-byte overhead (12-byte nonce + 16-byte tag)
    // For small values, this can be significant
    // Example:
    // 1-byte value → 29 bytes encrypted (29x overhead)
    // 100-byte value → 128 bytes encrypted (1.28x overhead)
    // 1KB value → 1052 bytes encrypted (1.05x overhead)
    // Recommendation: Use larger values when possible

Debugging Encrypted Databases

Check Encryption Status:

let storage = StorageEngine::open("./db", &config)?;
// Method 1: Check encryption flag
if storage.is_encrypted() {
println!("✓ Database is encrypted");
} else {
println!("⚠ Database is NOT encrypted");
}
// Method 2: Get encryption info
if let Some(info) = storage.encryption_info() {
println!("Encryption: {}", info);
// Output: "Enabled: AES-256-GCM"
} else {
println!("Encryption: Disabled");
}

Verify Data is Encrypted on Disk:

// Insert test data
storage.put(b"test_key", b"plaintext_value")?;
// Check raw data in RocksDB
let raw_data = storage.db.get(b"test_key")?.unwrap();
// Raw data should NOT match plaintext
assert_ne!(raw_data, b"plaintext_value", "Data is not encrypted!");
// But storage.get() should decrypt it
let decrypted = storage.get(b"test_key")?.unwrap();
assert_eq!(decrypted, b"plaintext_value", "Decryption failed!");
println!("✓ Data is encrypted on disk");
println!("✓ Transparent decryption works");

Logging and Monitoring:

// Enable encryption audit logging (future feature)
config.audit.log_encryption_events = true;
// Logged events:
// - Key loaded from source
// - Encryption enabled/disabled
// - Encryption errors
// - Key rotation (future)

Security Best Practices

Key Management Security

DO:

Use strong random keys:

Terminal window
# Good: Cryptographically secure random
openssl rand -hex 32
# Bad: Predictable or weak
echo "my_password_123" | md5sum # ❌ DON'T DO THIS

Separate keys from data:

Terminal window
# Good: Key on different volume/server
/etc/heliosdb/encryption.key # Different volume from /var/lib/heliosdb
# Bad: Key in same directory as database
/var/lib/heliosdb/encryption.key # ❌ Same volume

Use proper file permissions:

Terminal window
# Good: Minimal permissions
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Bad: World-readable
chmod 644 /etc/heliosdb/encryption.key # ❌ Others can read

Implement key rotation:

Terminal window
# Rotate keys every 90 days (recommended)
# Rotate immediately after:
# - Security incident
# - Employee departure
# - Suspected compromise

Backup keys securely:

Terminal window
# Good: Encrypted backup in separate location
gpg --encrypt --recipient admin@company.com encryption.key
aws s3 cp encryption.key.gpg s3://secure-backups/keys/
# Good: Hardware security module (HSM)
# Store master key in HSM, use derived keys for databases

DON’T:

Never commit keys to version control:

Terminal window
# Add to .gitignore
echo "*.key" >> .gitignore
echo ".env" >> .gitignore
echo "config/secrets.toml" >> .gitignore

Never log or print keys:

// Bad: Logs the key
println!("Key: {}", key_manager.export_as_hex()); // ❌ DON'T
// Good: Log only metadata
println!("Key loaded from: {}", key_manager.source()); // ✅ OK

Never use hardcoded keys:

// Bad: Hardcoded key in source code
const ENCRYPTION_KEY: &str = "0123..."; // ❌ NEVER DO THIS
// Good: Load from secure source
let key = KeyManager::from_source(&config.encryption.key_source)?; // ✅

Never share keys between environments:

Terminal window
# Bad: Same key for dev, staging, prod
# HELIOSDB_ENCRYPTION_KEY=same_key_everywhere # ❌
# Good: Different key per environment
# DEV: Different key for development
# STAGING: Different key for staging
# PROD: Different key for production

Access Control

Principle of Least Privilege:

Terminal window
# Database process user
useradd -r -s /bin/false heliosdb
# Key file permissions
chown root:heliosdb /etc/heliosdb/encryption.key
chmod 440 /etc/heliosdb/encryption.key
# Database files
chown heliosdb:heliosdb /var/lib/heliosdb
chmod 700 /var/lib/heliosdb
# Only heliosdb user can access database and keys

Network Security (Server Mode):

config.toml
[server]
host = "127.0.0.1" # Localhost only (use reverse proxy for external)
port = 5432
ssl_enabled = true # Always use SSL/TLS for network traffic
[server.ssl]
cert_file = "/etc/heliosdb/server.crt"
key_file = "/etc/heliosdb/server.key"
ca_file = "/etc/heliosdb/ca.crt"

Integration with Audit Logging

Enable Comprehensive Auditing:

use heliosdb_nano::Config;
let mut config = Config::default();
// Enable encryption
config.encryption.enabled = true;
config.encryption.key_source = /* ... */;
// Enable audit logging
config.audit.enabled = true;
config.audit.log_file = Some("/var/log/heliosdb/audit.log".into());
config.audit.log_format = heliosdb_nano::audit::LogFormat::Json;
// Audit all data access
config.audit.log_reads = true;
config.audit.log_writes = true;
config.audit.log_deletes = true;
// Combined: Encrypted data + full audit trail
let storage = StorageEngine::open("./db", &config)?;

Audit Log Example (JSON format):

{
"timestamp": "2025-12-01T10:30:45Z",
"event_type": "read",
"table": "patients",
"user": "doctor_smith",
"query": "SELECT * FROM patients WHERE id = 123",
"rows_affected": 1,
"encryption_enabled": true,
"source_ip": "192.168.1.100"
}

Compliance Reporting:

Terminal window
# Generate encryption compliance report
heliosdb-nano audit-report \
--input /var/log/heliosdb/audit.log \
--type encryption-compliance \
--output /reports/encryption_compliance_2025_12.pdf

Compliance Scenarios

HIPAA Compliance Setup

Health Insurance Portability and Accountability Act requirements:

Required Controls:

  1. ✅ Encryption at rest (covered by TDE)
  2. ✅ Encryption in transit (use SSL/TLS)
  3. ✅ Access controls (OS-level permissions)
  4. ✅ Audit logging (enable audit logs)
  5. ✅ Key management (secure key storage)

Configuration:

hipaa_config.toml
[encryption]
enabled = true
algorithm = "Aes256Gcm"
[encryption.key_source]
File = "/secure/hipaa/encryption.key"
[server]
ssl_enabled = true
[server.ssl]
cert_file = "/etc/heliosdb/hipaa_server.crt"
key_file = "/etc/heliosdb/hipaa_server.key"
min_tls_version = "1.2" # HIPAA requires TLS 1.2+
[audit]
enabled = true
log_file = "/var/log/heliosdb/hipaa_audit.log"
log_format = "Json"
log_reads = true
log_writes = true
log_deletes = true
log_schema_changes = true
[audit.retention]
days = 2555 # 7 years (HIPAA requirement)

HIPAA-Compliant Rust Code:

use heliosdb_nano::{Config, Schema, Column, DataType, Tuple, Value};
fn setup_hipaa_database() -> heliosdb_nano::Result<()> {
// Load HIPAA configuration
let config = Config::from_file("hipaa_config.toml")?;
// Verify encryption is enabled
assert!(config.encryption.enabled, "HIPAA requires encryption!");
assert!(config.audit.enabled, "HIPAA requires audit logging!");
// Open database
let storage = StorageEngine::open("/var/lib/heliosdb/hipaa", &config)?;
// Create PHI table (Protected Health Information)
let catalog = storage.catalog();
let schema = Schema::new(vec![
Column::new("patient_id", DataType::Int4).primary_key(),
Column::new("name", DataType::Text),
Column::new("ssn", DataType::Text),
Column::new("diagnosis", DataType::Text),
Column::new("medications", DataType::Text),
Column::new("created_at", DataType::Timestamp),
]);
catalog.create_table("patient_records", schema)?;
println!("✓ HIPAA-compliant database configured");
println!(" - Encryption: AES-256-GCM");
println!(" - SSL/TLS: Enabled");
println!(" - Audit logging: 7-year retention");
Ok(())
}

Verification Checklist:

Terminal window
# 1. Verify encryption is enabled
heliosdb-nano verify --config hipaa_config.toml --check encryption
# 2. Verify SSL/TLS certificates
openssl s_client -connect localhost:5432 -showcerts
# 3. Verify audit logging
tail -f /var/log/heliosdb/hipaa_audit.log
# 4. Verify file permissions
ls -la /var/lib/heliosdb/hipaa
# Expected: drwx------ (700)
# 5. Verify key security
ls -la /secure/hipaa/encryption.key
# Expected: -r-------- (400)

GDPR Data Protection

General Data Protection Regulation requirements:

GDPR Principles:

  1. Data Protection by Design: Encryption enabled from day one
  2. Right to Erasure: Secure deletion of personal data
  3. Data Portability: Export encrypted backups
  4. Breach Notification: Audit logs for incident response
  5. Privacy by Default: Minimal data collection

GDPR Configuration:

gdpr_config.toml
[encryption]
enabled = true
algorithm = "Aes256Gcm"
key_source = { File = "/etc/heliosdb/gdpr_key" }
[audit]
enabled = true
log_file = "/var/log/heliosdb/gdpr_audit.log"
log_format = "Json"
# Log personal data access (GDPR Article 15)
log_reads = true
log_writes = true
log_deletes = true
# Retention policy (GDPR Article 17)
[audit.retention]
days = 730 # 2 years minimum
[storage]
# Enable secure deletion (GDPR "Right to Erasure")
secure_delete = true
overwrite_deleted_data = true

GDPR-Compliant Data Operations:

// Right to Access (GDPR Article 15)
fn export_user_data(storage: &StorageEngine, user_id: i32)
-> heliosdb_nano::Result<String>
{
// Query all user data (automatically decrypted)
let query = format!("SELECT * FROM users WHERE id = {}", user_id);
let results = storage.execute_sql(&query)?;
// Export as JSON (for data portability)
let json = serde_json::to_string_pretty(&results)?;
// Log access (for audit trail)
log::info!("User data exported for user_id={}", user_id);
Ok(json)
}
// Right to Erasure (GDPR Article 17)
fn delete_user_data(storage: &StorageEngine, user_id: i32)
-> heliosdb_nano::Result<()>
{
// Delete user from all tables
storage.execute_sql(&format!("DELETE FROM users WHERE id = {}", user_id))?;
storage.execute_sql(&format!("DELETE FROM orders WHERE user_id = {}", user_id))?;
storage.execute_sql(&format!("DELETE FROM preferences WHERE user_id = {}", user_id))?;
// Secure deletion (overwrite on disk)
// Enabled via config.storage.secure_delete = true
// Log deletion (compliance requirement)
log::warn!("User data deleted for user_id={} (GDPR erasure)", user_id);
Ok(())
}
// Data Breach Notification (GDPR Article 33)
fn check_encryption_breach(storage: &StorageEngine) -> heliosdb_nano::Result<()> {
if !storage.is_encrypted() {
log::error!("CRITICAL: Database is not encrypted! GDPR violation!");
// Trigger incident response
notify_data_protection_officer("Unencrypted database detected")?;
}
Ok(())
}

PCI-DSS Requirements

Payment Card Industry Data Security Standard:

PCI-DSS Requirements for Encryption:

  • Requirement 3: Protect stored cardholder data
  • Requirement 4: Encrypt transmission of cardholder data
  • Requirement 8: Identify and authenticate access

PCI-DSS Configuration:

pci_config.toml
[encryption]
enabled = true # Requirement 3.4: Render PAN unreadable
algorithm = "Aes256Gcm" # Strong cryptography (Requirement 3.5)
[encryption.key_source]
File = "/secure/pci/encryption.key"
[encryption.key_rotation]
enabled = true
interval_days = 90 # Key rotation (Requirement 3.6.4)
[audit]
enabled = true # Requirement 10: Track and monitor access
log_file = "/var/log/heliosdb/pci_audit.log"
# Log all access to cardholder data
log_reads = true
log_writes = true
log_deletes = true
log_schema_changes = true
[audit.retention]
days = 365 # Minimum 1 year (Requirement 10.7)
[server.ssl]
enabled = true # Requirement 4: Encrypt in transit
min_tls_version = "1.2"

PCI-DSS Compliant Schema:

use heliosdb_nano::{Schema, Column, DataType};
fn create_pci_compliant_schema() -> Schema {
Schema::new(vec![
Column::new("transaction_id", DataType::Int8).primary_key(),
// PAN (Primary Account Number) - MUST be encrypted
Column::new("card_number", DataType::Text),
// Cardholder name
Column::new("cardholder_name", DataType::Text),
// Expiration date - Can be stored with encryption
Column::new("expiry_month", DataType::Int2),
Column::new("expiry_year", DataType::Int4),
// NEVER store these (PCI-DSS Requirement 3.2):
// - CVV/CVV2 (Card Verification Value)
// - Full magnetic stripe data
// - PIN or PIN block
Column::new("amount", DataType::Numeric),
Column::new("timestamp", DataType::Timestamp),
])
}
// Example: Storing payment card data (encrypted)
fn process_payment(storage: &StorageEngine, card_data: CardData)
-> heliosdb_nano::Result<()>
{
// Data is automatically encrypted by TDE
storage.insert_tuple("transactions", Tuple::new(vec![
Value::Int8(generate_transaction_id()),
Value::String(card_data.pan), // Encrypted at rest
Value::String(card_data.name), // Encrypted at rest
Value::Int2(card_data.exp_month),
Value::Int4(card_data.exp_year),
Value::Numeric(card_data.amount),
Value::Timestamp(Utc::now()),
]))?;
// Audit log automatically records access
Ok(())
}

PCI-DSS Compliance Verification:

Terminal window
# 1. Verify encryption (Requirement 3.4)
heliosdb-nano verify --config pci_config.toml --check encryption
# ✓ AES-256-GCM enabled
# 2. Verify key management (Requirement 3.5, 3.6)
ls -la /secure/pci/encryption.key
# ✓ Permissions: 400 (read-only by owner)
# 3. Verify audit logging (Requirement 10)
tail /var/log/heliosdb/pci_audit.log
# ✓ All access logged with timestamps
# 4. Verify SSL/TLS (Requirement 4)
openssl s_client -connect localhost:5432 | grep Protocol
# ✓ TLSv1.2 or higher
# 5. Check for prohibited data (Requirement 3.2)
grep -i "cvv\|cvv2\|cvc" /var/lib/heliosdb/pci/
# ✓ No results (good - not storing CVV)

Recovery Scenarios

Lost Key Recovery

Scenario: Encryption key file was accidentally deleted.

If you have a backup:

Terminal window
# 1. Stop database
systemctl stop heliosdb
# 2. Restore key from backup
cp /backup/keys/encryption.key.20251201 /etc/heliosdb/encryption.key
# 3. Set correct permissions
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# 4. Verify key works
heliosdb-nano verify --config /etc/heliosdb/config.toml --check encryption
# 5. Restart database
systemctl start heliosdb
systemctl status heliosdb

If you DON’T have a backup:

⚠️ CRITICAL: Without the encryption key, your data is PERMANENTLY LOST.
There is NO way to recover encrypted data without the original key.
Prevention:
- Always maintain secure backups of encryption keys
- Store backups in multiple locations
- Use key management services (KMS/HSM)
- Document key recovery procedures
- Test recovery procedures regularly

Database Migration with Encryption

Scenario: Migrate encrypted database to new server.

Step-by-Step Migration:

Terminal window
# On old server:
# 1. Backup encrypted database
tar -czf database_backup.tar.gz /var/lib/heliosdb/
# 2. Backup encryption key SEPARATELY
tar -czf encryption_key.tar.gz /etc/heliosdb/encryption.key
# 3. Transfer to new server (use secure channel)
scp database_backup.tar.gz user@newserver:/tmp/
scp encryption_key.tar.gz user@newserver:/tmp/
# On new server:
# 4. Extract database
mkdir -p /var/lib/heliosdb
tar -xzf /tmp/database_backup.tar.gz -C /
# 5. Extract encryption key
mkdir -p /etc/heliosdb
tar -xzf /tmp/encryption_key.tar.gz -C /
# 6. Set permissions
chmod 700 /var/lib/heliosdb
chown heliosdb:heliosdb /var/lib/heliosdb
chmod 400 /etc/heliosdb/encryption.key
chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# 7. Verify database opens correctly
heliosdb-nano verify --path /var/lib/heliosdb
# 8. Start database
systemctl start heliosdb
# 9. Test connectivity
psql -h localhost -p 5432 -c "SELECT 1"

Alternative: Export/Import Method:

Terminal window
# On old server:
# 1. Export data in decrypted format
heliosdb-nano export --input /var/lib/heliosdb --output /tmp/export.sql
# 2. Transfer export file
scp /tmp/export.sql user@newserver:/tmp/
# On new server:
# 3. Generate NEW encryption key for new server
heliosdb-nano keygen > /etc/heliosdb/encryption.key
chmod 400 /etc/heliosdb/encryption.key
# 4. Initialize encrypted database with NEW key
export HELIOSDB_ENCRYPTION_KEY=$(cat /etc/heliosdb/encryption.key)
heliosdb-nano init --path /var/lib/heliosdb --encrypted
# 5. Import data (will be encrypted with NEW key)
heliosdb-nano import --input /tmp/export.sql --output /var/lib/heliosdb
# Result: Data re-encrypted with new key on new server

Backup and Restore

Full Encrypted Backup:

Terminal window
# Backup strategy: Backup encrypted database files directly
# Advantage: Fast, no decryption needed
# Disadvantage: Requires same encryption key for restore
# 1. Consistent snapshot (stop writes)
heliosdb-nano checkpoint --path /var/lib/heliosdb
# 2. Backup database files (encrypted)
tar -czf /backup/heliosdb_encrypted_20251201.tar.gz /var/lib/heliosdb/
# 3. Backup encryption key SEPARATELY and SECURELY
gpg --encrypt --recipient backup@company.com /etc/heliosdb/encryption.key
cp encryption.key.gpg /secure/backup/keys/
# 4. Verify backup
tar -tzf /backup/heliosdb_encrypted_20251201.tar.gz | head
# 5. Test restore to temporary location
mkdir /tmp/restore_test
tar -xzf /backup/heliosdb_encrypted_20251201.tar.gz -C /tmp/restore_test
heliosdb-nano verify --path /tmp/restore_test/var/lib/heliosdb

Decrypted Logical Backup:

Terminal window
# Backup strategy: Export to SQL (decrypted)
# Advantage: Key-independent, can restore with different key
# Disadvantage: Slower, decrypted data must be secured
# 1. Export to SQL (automatically decrypts during export)
heliosdb-nano export \
--input /var/lib/heliosdb \
--output /backup/heliosdb_export_20251201.sql
# 2. Encrypt the SQL export for security
gpg --encrypt --recipient backup@company.com \
/backup/heliosdb_export_20251201.sql
# 3. Store encrypted SQL file
mv heliosdb_export_20251201.sql.gpg /secure/backup/exports/
# 4. Restore procedure
gpg --decrypt /secure/backup/exports/heliosdb_export_20251201.sql.gpg \
> /tmp/restore.sql
heliosdb-nano import \
--input /tmp/restore.sql \
--output /var/lib/heliosdb_restored
# 5. Secure cleanup
shred -u /tmp/restore.sql # Securely delete decrypted SQL

Automated Backup Script:

backup_encrypted_db.sh
#!/bin/bash
set -e
DB_PATH="/var/lib/heliosdb"
BACKUP_DIR="/backup/heliosdb"
KEY_FILE="/etc/heliosdb/encryption.key"
DATE=$(date +%Y%m%d_%H%M%S)
# 1. Create consistent snapshot
heliosdb-nano checkpoint --path "$DB_PATH"
# 2. Backup database (encrypted)
tar -czf "$BACKUP_DIR/db_$DATE.tar.gz" "$DB_PATH"
# 3. Backup key (encrypted with GPG)
gpg --encrypt --recipient backup@company.com "$KEY_FILE" \
-o "$BACKUP_DIR/key_$DATE.gpg"
# 4. Verify backup integrity
tar -tzf "$BACKUP_DIR/db_$DATE.tar.gz" > /dev/null
# 5. Retention: Keep last 30 days
find "$BACKUP_DIR" -name "db_*.tar.gz" -mtime +30 -delete
find "$BACKUP_DIR" -name "key_*.gpg" -mtime +30 -delete
# 6. Upload to cloud (optional)
# aws s3 cp "$BACKUP_DIR/db_$DATE.tar.gz" s3://backups/heliosdb/
# aws s3 cp "$BACKUP_DIR/key_$DATE.gpg" s3://backups/heliosdb/keys/
echo "✓ Backup completed: $DATE"

Schedule with cron:

Terminal window
# Run daily at 2 AM
0 2 * * * /usr/local/bin/backup_encrypted_db.sh >> /var/log/heliosdb_backup.log 2>&1

Performance Considerations

Benchmarking Encrypted vs Unencrypted

Run Built-in Benchmarks:

Terminal window
# Run encryption benchmarks
cargo bench --bench encryption_benchmark
# Sample output:
# Encryption/put_get/64_bytes time: [1.23 µs 1.25 µs 1.27 µs]
# Unencrypted/put_get/64_bytes time: [1.20 µs 1.22 µs 1.24 µs]
# Overhead: ~2.5%

Custom Performance Testing:

use std::time::Instant;
use heliosdb_nano::{Config, storage::StorageEngine};
fn benchmark_encryption_overhead() -> heliosdb_nano::Result<()> {
let iterations = 10_000;
// Test 1: Encrypted database
let mut config = Config::in_memory();
config.encryption.enabled = true;
config.encryption.key_source = /* ... */;
let encrypted_storage = StorageEngine::open_in_memory(&config)?;
let start = Instant::now();
for i in 0..iterations {
encrypted_storage.put(
&format!("key_{}", i).as_bytes(),
b"test_data_payload_1234567890"
)?;
}
let encrypted_duration = start.elapsed();
// Test 2: Unencrypted database
let mut config_unenc = Config::in_memory();
config_unenc.encryption.enabled = false;
let unencrypted_storage = StorageEngine::open_in_memory(&config_unenc)?;
let start = Instant::now();
for i in 0..iterations {
unencrypted_storage.put(
&format!("key_{}", i).as_bytes(),
b"test_data_payload_1234567890"
)?;
}
let unencrypted_duration = start.elapsed();
// Calculate overhead
let overhead = (encrypted_duration.as_micros() as f64
/ unencrypted_duration.as_micros() as f64 - 1.0) * 100.0;
println!("Results ({} iterations):", iterations);
println!(" Unencrypted: {:?}", unencrypted_duration);
println!(" Encrypted: {:?}", encrypted_duration);
println!(" Overhead: {:.2}%", overhead);
Ok(())
}

Optimization Tips

1. Use Appropriate Value Sizes:

// Inefficient: Many tiny encrypted values
// Each has 28-byte overhead (nonce + tag)
for i in 0..1000 {
storage.put(&format!("counter_{}", i).as_bytes(), &[i as u8])?;
// 1-byte value → 29 bytes stored (29x overhead)
}
// Efficient: Batch into larger values
let counters: Vec<u8> = (0..1000).map(|i| i as u8).collect();
storage.put(b"counters", &counters)?;
// 1000-byte value → 1028 bytes stored (1.028x overhead)

2. Minimize Metadata Operations:

// Less efficient: Frequent catalog queries (metadata is encrypted)
for table_name in table_names {
let schema = catalog.get_table_schema(table_name)?; // Decrypts metadata
process_schema(schema);
}
// More efficient: Cache metadata
let schemas: HashMap<String, Schema> = table_names.iter()
.map(|name| (name.clone(), catalog.get_table_schema(name)))
.collect()?;
for (name, schema) in schemas {
process_schema(schema); // No decryption needed
}

3. Batch Writes:

// Use transactions or batch inserts to amortize encryption overhead
storage.begin_transaction()?;
for tuple in large_dataset {
storage.insert_tuple("table", tuple)?;
}
storage.commit()?; // Single fsync

Hardware Acceleration

Verify AES-NI Support:

Terminal window
# Linux
lscpu | grep -i aes
# Look for: Flags: ... aes ...
cat /proc/cpuinfo | grep aes
# Should show "aes" flag
# macOS
sysctl -a | grep -i aes
# macOS kern.hv_support: 1 (includes AES support)
# Check with Rust
cargo build --release
# Compiler automatically uses AES-NI if available

Performance Impact of AES-NI:

CPUWithout AES-NIWith AES-NISpeedup
Intel Core i7~15 µs~1.2 µs12x faster
AMD Ryzen 7~14 µs~1.1 µs13x faster
ARM (modern)~10 µs~0.9 µs11x faster

Note: HeliosDB Nano’s <3% overhead assumes AES-NI is available.


Troubleshooting

Common Issues

Issue: “Environment variable ‘HELIOSDB_ENCRYPTION_KEY’ not found”

Cause: Encryption key not set in environment.

Solution:

Terminal window
# Check if variable is set
echo $HELIOSDB_ENCRYPTION_KEY
# Set the variable
export HELIOSDB_ENCRYPTION_KEY="<64-character-hex-key>"
# For systemd services
sudo systemctl edit heliosdb
# Add:
# [Service]
# Environment="HELIOSDB_ENCRYPTION_KEY=<key>"
# For Docker
docker run -e HELIOSDB_ENCRYPTION_KEY="<key>" heliosdb-nano

Issue: “Decryption failed: Authentication tag verification failed”

Causes:

  1. Wrong encryption key
  2. Corrupted data
  3. Key changed after data was encrypted

Solutions:

Terminal window
# 1. Verify you're using the correct key
cat /etc/heliosdb/encryption.key
# Should be 64 hex characters
# 2. Check for data corruption
heliosdb-nano verify --path /var/lib/heliosdb
# 3. Try restore from backup
cp -r /var/lib/heliosdb /var/lib/heliosdb.corrupted
tar -xzf /backup/heliosdb_latest.tar.gz -C /var/lib/
# 4. Restore correct key
cp /backup/keys/encryption.key /etc/heliosdb/encryption.key

Issue: “Hex key must be 64 characters (32 bytes), got X”

Cause: Invalid key format.

Solution:

Terminal window
# Generate proper key (64 hex chars)
heliosdb-nano keygen
# Or:
openssl rand -hex 32
# Verify length
echo -n "<your-key>" | wc -c
# Should output: 64

Issue: “Failed to read key file: Permission denied”

Cause: Insufficient file permissions.

Solution:

Terminal window
# Check permissions
ls -la /etc/heliosdb/encryption.key
# Fix ownership
sudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Fix permissions
sudo chmod 400 /etc/heliosdb/encryption.key
# Verify database process user
ps aux | grep heliosdb
# Ensure it's running as 'heliosdb' user

Performance Issues

Issue: “Database operations are slow after enabling encryption”

Diagnosis:

Terminal window
# 1. Check if AES-NI is available
lscpu | grep aes
# 2. Run benchmarks
cargo bench --bench encryption_benchmark
# 3. Profile encryption operations
cargo build --release --features profiling
perf record ./target/release/heliosdb-nano
perf report

Solutions:

// 1. Batch operations
storage.begin_transaction()?;
for item in items {
storage.insert_tuple("table", item)?;
}
storage.commit()?;
// 2. Increase cache size
config.storage.cache_size = 1024 * 1024 * 1024; // 1 GB
// 3. Use in-memory mode for temporary data
let temp_config = Config::in_memory();
// Encryption overhead is minimal in-memory

Security Issues

Issue: “Security audit flagged encryption key in logs”

Immediate Action:

Terminal window
# 1. Rotate key immediately
NEW_KEY=$(heliosdb-nano keygen)
# Follow key rotation procedure
# 2. Scrub logs
sudo find /var/log -name "*.log" -exec grep -l "HELIOSDB_ENCRYPTION_KEY" {} \; \
-exec shred -u {} \;
# 3. Update code to prevent logging
# Remove any println!() or log statements that include keys

Prevention:

// Bad: Logs key
log::debug!("Using key: {}", key_manager.export_as_hex()); // ❌
// Good: Logs only metadata
log::debug!("Key loaded from: {:?}", key_manager.source()); // ✅

Issue: “Key file has insecure permissions”

Fix:

Terminal window
# Set minimal permissions
sudo chmod 400 /etc/heliosdb/encryption.key
sudo chown heliosdb:heliosdb /etc/heliosdb/encryption.key
# Verify
ls -la /etc/heliosdb/encryption.key
# Expected: -r-------- 1 heliosdb heliosdb 64 Dec 01 10:00 encryption.key
# Audit permissions recursively
find /etc/heliosdb -type f -not -perm 400 -ls
# Should be empty

Getting Help

Diagnostic Information to Collect:

Terminal window
# System info
uname -a
lscpu | grep -i aes
# Database info
heliosdb-nano --version
heliosdb-nano verify --path /var/lib/heliosdb
# Configuration
cat /etc/heliosdb/config.toml | grep -A 10 encryption
# Logs
tail -100 /var/log/heliosdb/heliosdb.log
# Permissions
ls -laR /etc/heliosdb
ls -laR /var/lib/heliosdb

Report Issues:


Summary

You’ve learned how to:

Enable transparent data encryption with AES-256-GCM ✅ Manage encryption keys securely using multiple methods ✅ Configure encryption for various deployment scenarios ✅ Achieve compliance with HIPAA, GDPR, and PCI-DSS ✅ Maintain performance with <3% overhead ✅ Handle recovery scenarios and migrations ✅ Troubleshoot common encryption issues

Key Takeaways:

  1. Encryption is transparent: No application code changes needed
  2. Security is paramount: Protect keys with same rigor as data
  3. Backup keys separately: Data is unrecoverable without keys
  4. Monitor performance: Encryption overhead is minimal with AES-NI
  5. Plan for compliance: Enable audit logging with encryption
  6. Test recovery procedures: Verify backups and key recovery regularly

Next Steps:


Version History:

  • 2.5.0-dev (2025-12-01): Initial comprehensive tutorial
  • 2.1.0 (2024-11-13): Encryption feature released

Related Documentation: