HeliosDB Implicit Storage Features

HeliosDB Implicit Storage Features

Document Version: 1.0 Date: 2025-01-25 Category: Storage Features / Performance Optimization Audience: Database Administrators, Developers, Solution Architects

---

Executive Summary

HeliosDB includes a comprehensive set of implicit (automatic) storage features that operate transparently without explicit user configuration. These features are designed to optimize performance, reduce storage costs, and maintain data integrity “out of the box.”

This document catalogs all implicit storage features, explains their automatic behaviors, and documents which ones can be explicitly configured for advanced optimization.

---

Table of Contents

1. Implicit Features Overview
2. Compression Features
3. Storage Tiering
4. MVCC and Versioning
5. Index and Filter Optimization
6. Compaction Strategies
7. Caching and Memory Management
8. Sharding and Distribution
9. Write-Ahead Logging
10. Query Optimization Features
11. Self-Tuning Features
12. Configuration Reference
13. Default Benefits Summary

---

1. Implicit Features Overview

1.1 What Are Implicit Features?

Implicit features are optimizations that HeliosDB applies automatically without requiring explicit SQL statements or configuration. They include:

Category	Features	Default Behavior
Compression	HCC Adaptive, Column-level	Auto-enabled with adaptive algorithm selection
Tiering	Hot/Warm/Cold/Archive	Auto-promotes/demotes based on access patterns
MVCC	Snapshot Isolation, Versioning	Always enabled for consistency
Indexing	Bloom Filters, Sparse Indexes	Auto-created for common query patterns
Compaction	LSM-tree, Adaptive Strategy	Auto-triggered based on write patterns
Caching	Unified Cache, Prefetching	Auto-managed with adaptive sizing
Sharding	Consistent Hashing, Rebalancing	Auto-distributed based on key patterns
WAL	Write-Ahead Logging	Always enabled for durability
Query Opt	Predicate Pushdown, SIMD	Auto-applied during query planning
Self-Tuning	Statistics, Workload Analysis	Continuous background optimization

1.2 Design Philosophy

HeliosDB follows the “Zero-Configuration Performance” principle:

• Sensible Defaults: Every feature has production-ready defaults
• Adaptive Behavior: Features self-tune based on workload characteristics
• Override Capability: Advanced users can override any automatic behavior
• Observability: All implicit actions are logged and visible in EXPLAIN output

---

2. Compression Features

2.1 HCC Adaptive Compression

Location: heliosdb-storage/crates/hybrid-columnar-compression/

HeliosDB implements Oracle-compatible Hybrid Columnar Compression (HCC) with automatic algorithm selection.

Automatic Behaviors:

• Algorithm Selection: Automatically chooses between Zstd, LZ4, Snappy, or None based on data characteristics
• Column-Level Compression: Different algorithms per column based on data type and cardinality
• Compression Ratio Monitoring: Tracks compression effectiveness and adjusts algorithms
• Dictionary Encoding: Automatically applied to low-cardinality string columns

Compression Algorithm Selection Matrix:

Data Characteristic	Default Algorithm	Compression Ratio	Speed
High cardinality numeric	LZ4	2-3x	Fast
Low cardinality string	Dictionary + Zstd	10-50x	Medium
Timestamp columns	Delta + LZ4	5-10x	Fast
Binary/BLOB data	Zstd (level 3)	3-5x	Medium
Already compressed	None	1x	Fastest

Configuration Options (SQL):

-- View current compression settings
SHOW STORAGE COMPRESSION;

-- Override compression for a table (optional)
ALTER TABLE orders SET (compression = 'zstd', compression_level = 5);

-- Disable compression for specific column
ALTER TABLE orders ALTER COLUMN raw_data SET (compression = 'none');

Source Files:

• heliosdb-storage/crates/hybrid-columnar-compression/src/compressor.rs
• heliosdb-storage/crates/hybrid-columnar-compression/src/adaptive.rs

2.2 Default Compression Levels

Tier	Default Level	Rationale
Hot	LZ4 (level 1)	Speed prioritized
Warm	Zstd (level 3)	Balanced
Cold	Zstd (level 9)	Compression prioritized
Archive	Zstd (level 19)	Maximum compression

---

3. Storage Tiering

3.1 Automatic Data Tiering

Location: heliosdb-storage/crates/intelligent-tiering/

HeliosDB automatically moves data between storage tiers based on access patterns.

Tier Definitions:

Tier	Storage Type	Access Pattern	Retention
Hot	NVMe SSD / RAM	Frequent (< 1 hour old)	Active data
Warm	SSD	Moderate (1h - 7d old)	Recent data
Cold	HDD / Object Storage	Infrequent (7d - 90d)	Historical
Archive	Object Storage / Tape	Rare (> 90d)	Compliance

Automatic Behaviors:

• Access Pattern Tracking: Monitors read/write frequency per data block
• Promotion on Read: Frequently accessed cold data promoted to warm/hot
• Demotion on Age: Data automatically demoted based on last access time
• Predictive Prefetch: ML-based prediction of data access patterns

Tiering Policies (Default):

hot_tier:
  max_age: 1h
  min_access_frequency: 10/min
  storage: nvme_ssd

warm_tier:
  max_age: 7d
  min_access_frequency: 1/hour
  storage: ssd

cold_tier:
  max_age: 90d
  min_access_frequency: 1/day
  storage: hdd

archive_tier:
  age: >90d
  storage: object_storage
  compression: zstd_19

Configuration Options (SQL):

-- View current tiering status
SELECT * FROM helios_storage_tiers;

-- Override tier retention for a table
ALTER TABLE audit_logs SET (
    hot_retention = '24 hours',
    warm_retention = '30 days',
    cold_retention = '1 year'
);

-- Force immediate tier migration
CALL helios_migrate_to_tier('orders', 'archive', '2024-01-01');

Source Files:

• heliosdb-storage/crates/intelligent-tiering/src/engine.rs
• heliosdb-storage/crates/intelligent-tiering/src/policy.rs

---

4. MVCC and Versioning

4.1 Snapshot Isolation

Location: heliosdb-storage/crates/mvcc/

HeliosDB provides MVCC-based snapshot isolation for all transactions.

Automatic Behaviors:

• Version Chain Management: Maintains version chains for concurrent access
• Read Consistency: Readers see consistent snapshot without blocking writers
• Write Conflict Detection: Automatic detection of write-write conflicts
• Timestamp Ordering: Assigns monotonic timestamps for version ordering

Isolation Levels:

Level	Default	Phantom Reads	Non-Repeatable Reads	Dirty Reads
Serializable	Yes (default)	Prevented	Prevented	Prevented
Repeatable Read	Available	Possible	Prevented	Prevented
Read Committed	Available	Possible	Possible	Prevented

Configuration Options (SQL):

-- View current isolation level
SHOW TRANSACTION ISOLATION LEVEL;

-- Set session isolation level
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;

-- Per-query snapshot
SELECT * FROM orders AS OF TIMESTAMP '2025-01-01 00:00:00';

4.2 Automatic Garbage Collection

Automatic Behaviors:

• Version Cleanup: Removes old versions no longer visible to any transaction
• Adaptive GC Policies: Adjusts cleanup frequency based on write rate
• Tombstone Compaction: Removes deleted rows after retention period

GC Policies:

Policy	Behavior	Use Case
Aggressive	GC runs every 1 minute	High-write OLTP
Balanced (default)	GC runs every 5 minutes	Mixed workloads
Conservative	GC runs every 30 minutes	Long-running analytics

Configuration Options (SQL):

-- View GC statistics
SELECT * FROM helios_gc_stats;

-- Set GC policy
ALTER SYSTEM SET gc_policy = 'balanced';

-- Set minimum version retention
ALTER SYSTEM SET mvcc_version_retention = '1 hour';

Source Files:

• heliosdb-storage/crates/mvcc/src/gc.rs
• heliosdb-storage/crates/mvcc/src/version_chain.rs

---

5. Index and Filter Optimization

5.1 Bloom Filters

Location: heliosdb-storage/crates/bloom-filter/

Bloom filters are automatically maintained for efficient negative lookups.

Automatic Behaviors:

• Auto-Creation: Bloom filters created for all columns used in WHERE clauses
• Size Optimization: Filter size automatically tuned for target false positive rate
• Hierarchical Filters: Multi-level filters for range queries

Default Configuration:

Parameter	Default Value	Description
bloom_filter_fp_rate	0.01 (1%)	Target false positive rate
bloom_filter_bits_per_key	10	Bits allocated per unique key
bloom_filter_enabled	true	Auto-creation enabled

Configuration Options (SQL):

-- View bloom filter statistics
SELECT * FROM helios_bloom_filter_stats WHERE table_name = 'orders';

-- Disable bloom filter for specific column
ALTER TABLE orders ALTER COLUMN description SET (bloom_filter = false);

-- Set custom false positive rate
ALTER TABLE orders SET (bloom_filter_fp_rate = 0.001);

5.2 Sparse Indexes

Automatic Behaviors:

• Auto-Creation: Sparse indexes created for sorted columns
• Min-Max Tracking: Block-level min/max values for range pruning
• Zone Maps: Automatic zone map maintenance for partition pruning

Source Files:

• heliosdb-storage/crates/bloom-filter/src/builder.rs
• heliosdb-storage/src/index/sparse.rs

---

6. Compaction Strategies

6.1 LSM-Tree Compaction

Location: heliosdb-storage/crates/lsm-forest/

HeliosDB uses LSM-tree storage with automatic compaction.

Compaction Strategies:

Strategy	Description	Best For
Leveled (default)	Size-tiered levels with sorted runs	Read-heavy workloads
Tiered	Time-ordered tiers	Write-heavy workloads
FIFO	First-in-first-out	Time-series data
Universal	Hybrid approach	Mixed workloads

Automatic Behaviors:

• Trigger Threshold: Compaction triggered when level exceeds size ratio
• Write Amplification Control: Limits write amplification factor
• Concurrent Compaction: Background threads for non-blocking compaction
• Compaction Priority: Prioritizes levels with most impact

Default Configuration:

Parameter	Default Value	Description
compaction_style	leveled	Compaction strategy
level_size_ratio	10	Size multiplier between levels
max_levels	7	Maximum LSM levels
max_write_amplification	20	Maximum write amplification
compaction_threads	4	Background compaction threads

Configuration Options (SQL):

-- View compaction statistics
SELECT * FROM helios_compaction_stats;

-- Change compaction strategy for table
ALTER TABLE time_series SET (compaction_style = 'fifo');

-- Trigger manual compaction
CALL helios_compact_table('orders');

-- Set compaction priority
ALTER TABLE orders SET (compaction_priority = 'high');

Source Files:

• heliosdb-storage/crates/lsm-forest/src/compaction/
• heliosdb-storage/crates/lsm-forest/src/levels.rs

---

7. Caching and Memory Management

7.1 Unified Cache

Location: heliosdb-cache/crates/unified-cache/

HeliosDB maintains a unified cache layer with intelligent eviction.

Cache Tiers:

Tier	Location	Size	Eviction Policy
L1	CPU Cache	Auto	LRU
L2	RAM (Hot)	25% of RAM	ARC (Adaptive)
L3	RAM (Warm)	50% of RAM	Clock-Pro
L4	SSD Cache	Configurable	FIFO

Automatic Behaviors:

• Adaptive Sizing: Cache sizes adjust based on workload
• Scan Resistance: Prevents large scans from evicting hot data
• Prefetching: Predictive read-ahead for sequential access
• Memory Pressure Detection: Automatic cache shrinking under pressure

Default Configuration:

Parameter	Default Value	Description
cache_size_ratio	0.75	Fraction of available RAM
prefetch_enabled	true	Predictive read-ahead
scan_resistance	true	Protect hot data from scans
adaptive_sizing	true	Dynamic size adjustment

Configuration Options (SQL):

-- View cache statistics
SELECT * FROM helios_cache_stats;

-- Set cache size limit
ALTER SYSTEM SET cache_size = '8GB';

-- Disable prefetching for specific table
ALTER TABLE large_scans SET (prefetch_enabled = false);

-- Clear cache (careful in production)
CALL helios_clear_cache();

7.2 Query Result Cache

Automatic Behaviors:

• Result Caching: Caches results of identical queries
• Invalidation: Automatic invalidation on table modifications
• TTL-Based Expiry: Configurable time-to-live for cached results

Source Files:

• heliosdb-cache/crates/unified-cache/src/adaptive.rs
• heliosdb-cache/crates/unified-cache/src/prefetch.rs

---

8. Sharding and Distribution

8.1 Automatic Sharding

Location: heliosdb-cluster/crates/sharding/

HeliosDB automatically distributes data across nodes using consistent hashing.

Sharding Strategies:

Strategy	Description	Best For
Hash (default)	Consistent hash on primary key	Uniform distribution
Range	Range-based partitioning	Time-series, ordered queries
Geo	Geographic distribution	Multi-region deployments
Composite	Hash + Range hybrid	Complex access patterns

Automatic Behaviors:

• Key Distribution: Automatically hashes keys to shards
• Rebalancing: Automatic shard rebalancing on node changes
• Hot Spot Detection: Identifies and splits hot shards
• Partition Pruning: Query optimizer prunes irrelevant shards

Default Configuration:

Parameter	Default Value	Description
sharding_strategy	hash	Default sharding method
replication_factor	3	Copies of each shard
shard_count	auto	Automatically determined
auto_rebalance	true	Automatic rebalancing

Configuration Options (SQL):

-- View shard distribution
SELECT * FROM helios_shard_info WHERE table_name = 'orders';

-- Set sharding strategy
CREATE TABLE events (
    event_id UUID,
    timestamp TIMESTAMPTZ,
    data JSONB
) PARTITION BY RANGE (timestamp)
  SHARD BY HASH (event_id);

-- Manual shard split
CALL helios_split_shard('orders', 'shard_001');

-- View rebalancing status
SELECT * FROM helios_rebalance_status;

8.2 Consistent Hashing

Implementation Details:

• Virtual Nodes: 1024 virtual nodes per physical node
• Hash Function: xxHash64 for speed
• Rebalance Threshold: Triggers when imbalance > 10%

Source Files:

• heliosdb-cluster/crates/sharding/src/consistent_hash.rs
• heliosdb-cluster/crates/sharding/src/elastic.rs

---

9. Write-Ahead Logging

9.1 WAL Configuration

Location: heliosdb-storage/crates/wal/

HeliosDB uses write-ahead logging for durability with automatic management.

Automatic Behaviors:

• Synchronous Writes: WAL entries synced before commit acknowledgment
• Log Rotation: Automatic rotation based on size/time
• Checkpoint Management: Periodic checkpointing for recovery speed
• Log Compression: Optional compression of archived WAL segments

Default Configuration:

Parameter	Default Value	Description
wal_sync_mode	fsync	Durability guarantee
wal_segment_size	64MB	Size per WAL segment
checkpoint_interval	5 minutes	Time between checkpoints
wal_retention	24 hours	Minimum WAL retention

Durability Modes:

Mode	Behavior	Durability	Performance
fsync (default)	Sync each commit	Highest	Baseline
fdatasync	Sync data only	High	10-20% faster
async	Periodic sync	Medium	2-3x faster
off	No sync	None	Fastest (dev only)

Configuration Options (SQL):

-- View WAL statistics
SELECT * FROM helios_wal_stats;

-- Change sync mode (careful!)
ALTER SYSTEM SET wal_sync_mode = 'fdatasync';

-- Set checkpoint interval
ALTER SYSTEM SET checkpoint_interval = '10 minutes';

-- Force checkpoint
CHECKPOINT;

Source Files:

• heliosdb-storage/crates/wal/src/writer.rs
• heliosdb-storage/crates/wal/src/checkpoint.rs

---

10. Query Optimization Features

10.1 Predicate Pushdown

Location: heliosdb-compute/src/optimizer/

Predicates are automatically pushed down to the storage layer.

Automatic Behaviors:

• Filter Pushdown: WHERE clauses pushed to storage scan
• Join Predicate Pushdown: Join conditions evaluated during scan
• Expression Evaluation: Compatible expressions evaluated at storage level

Example:

-- Original query
SELECT * FROM orders WHERE status = 'pending' AND amount > 100;

-- Internally optimized to:
-- 1. Bloom filter check for status = 'pending'
-- 2. Storage-level filter for amount > 100
-- 3. Only matching rows returned to query engine

10.2 Projection Pushdown

Automatic Behaviors:

• Column Pruning: Only requested columns read from storage
• Computed Column Deferral: Complex expressions deferred when beneficial

10.3 SIMD Vectorization

Location: heliosdb-compute/crates/simd-accel/

HeliosDB automatically uses SIMD instructions for supported operations.

Automatic Behaviors:

• CPU Feature Detection: Detects AVX2, AVX-512 availability
• Batch Processing: Processes data in vectorized batches
• Fallback Mode: Graceful fallback to scalar operations

Supported Operations:

• Numeric comparisons and arithmetic
• String operations (LIKE, equality)
• Aggregations (SUM, COUNT, AVG, MIN, MAX)
• Hash computations

10.4 Parallel Execution

Automatic Behaviors:

• Query Parallelization: Large queries split across workers
• Parallel Scans: Multiple threads for table scans
• Parallel Aggregation: Distributed aggregation with merge

Default Configuration:

Parameter	Default Value	Description
parallel_workers	CPU cores - 2	Max parallel workers
parallel_threshold	10000 rows	Min rows for parallel
parallel_scan_enabled	true	Enable parallel scans

Configuration Options (SQL):

-- View parallel execution stats
EXPLAIN ANALYZE SELECT COUNT(*) FROM orders;

-- Set max parallel workers
SET max_parallel_workers = 8;

-- Disable parallel for session
SET parallel_query_enabled = false;

Source Files:

• heliosdb-compute/src/optimizer/predicate_pushdown.rs
• heliosdb-compute/crates/simd-accel/src/operations.rs

---

11. Self-Tuning Features

11.1 Automatic Statistics

Location: heliosdb-compute/src/statistics/

HeliosDB automatically maintains query statistics.

Automatic Behaviors:

• Sample-Based Statistics: Automatic sampling for large tables
• Histogram Generation: Automatic histogram creation for skewed columns
• Statistics Refresh: Background refresh based on data changes

Default Configuration:

Parameter	Default Value	Description
auto_analyze_threshold	10% changes	Trigger threshold
sample_ratio	0.1 (10%)	Default sample size
histogram_buckets	100	Default bucket count

11.2 Workload Analysis

Location: heliosdb-ai/crates/workload-predictor/

HeliosDB analyzes workload patterns for optimization.

Automatic Behaviors:

• Query Pattern Detection: Identifies common query patterns
• Resource Prediction: Predicts resource needs for queries
• Adaptive Configuration: Adjusts settings based on workload

11.3 Hybrid Bayesian-Genetic Optimizer (HBGDO)

Location: heliosdb-ai/crates/automl-tuning/

Advanced automatic parameter tuning using machine learning.

Automatic Behaviors:

• Parameter Space Exploration: GA for global search, BO for refinement
• Multi-Objective Optimization: Balances latency, throughput, resources
• Safe Rollback: Automatic rollback if performance degrades

Source Files:

• heliosdb-ai/crates/workload-predictor/src/analyzer.rs
• heliosdb-ai/crates/automl-tuning/src/hybrid_optimizer.rs

---

12. Configuration Reference

12.1 All Configurable Parameters

-- View all implicit feature settings
SHOW ALL HELIOS_SETTINGS;

-- Common configuration commands
ALTER SYSTEM SET <parameter> = <value>;  -- Cluster-wide
SET <parameter> = <value>;                -- Session-level
ALTER TABLE <table> SET (<param> = <value>);  -- Table-level

12.2 Configuration Hierarchy

1. System Default - Built-in defaults
2. Cluster Configuration - ALTER SYSTEM settings
3. Database Configuration - Per-database overrides
4. Table Configuration - Per-table overrides
5. Session Configuration - Per-session overrides
6. Query Hints - Per-query overrides

12.3 Key Configuration Groups

Group	Parameters	Level
Compression	compression, compression_level	Table
Tiering	hot_retention, warm_retention, cold_retention	Table
MVCC	isolation_level, version_retention	Session
Cache	cache_size, prefetch_enabled	System
Compaction	compaction_style, compaction_threads	System/Table
Parallel	max_parallel_workers, parallel_threshold	Session
WAL	wal_sync_mode, checkpoint_interval	System

---

13. Default Benefits Summary

13.1 Out-of-Box Performance Benefits

By default, without any configuration, HeliosDB provides:

Benefit	Feature	Typical Improvement
Storage Reduction	HCC Adaptive Compression	3-10x compression
Query Speed	Predicate/Projection Pushdown	2-5x faster
Memory Efficiency	Unified Cache + ARC	30-50% hit rate improvement
Write Performance	LSM + Compaction	Consistent write latency
Read Consistency	MVCC Snapshot Isolation	Zero read locks
Durability	WAL + Checkpointing	Zero data loss
Scalability	Auto Sharding	Linear scale-out
Cost Optimization	Automatic Tiering	50-70% storage cost reduction

13.2 When to Tune

Consider manual tuning when:

• Specific workload patterns are well understood
• Extreme performance requirements (sub-millisecond)
• Compliance requirements (specific durability modes)
• Cost optimization for known data lifecycle
• Multi-tenant isolation requirements

13.3 Monitoring Implicit Features

-- View all implicit feature activity
SELECT * FROM helios_implicit_features_status;

-- View optimization decisions in query plan
EXPLAIN (FORMAT JSON, FEATURES ON) SELECT * FROM orders WHERE status = 'pending';

-- View storage feature statistics
SELECT * FROM helios_storage_stats;

-- View cache and memory statistics
SELECT * FROM helios_memory_stats;

---

Appendix A: EXPLAIN Output with Implicit Features

The enhanced EXPLAIN command shows all implicit features applied:

EXPLAIN (FORMAT TEXT, FEATURES ON, WHY_NOT ON)
SELECT * FROM orders WHERE status = 'pending' AND amount > 100;

Sample Output:

Query Plan:
  Scan: orders
    Filter: status = 'pending' AND amount > 100

Implicit Features Active:
  [X] Predicate Pushdown: status = 'pending' pushed to storage
  [X] Bloom Filter: Checked for status column (1% FP rate)
  [X] Compression: HCC Adaptive (Zstd level 3)
  [X] Cache: L2 cache hit for orders.status index
  [X] SIMD: AVX2 used for amount > 100 comparison
  [X] Parallel: 4 workers assigned

Why-Not Analysis:
  [ ] Index Scan: No index on (status, amount) - consider CREATE INDEX
  [ ] Partition Pruning: Table not partitioned

Optimization Suggestions:
  1. CREATE INDEX idx_orders_status_amount ON orders(status, amount)
     - Estimated improvement: 10x for this query pattern

---

Appendix B: Feature Matrix by Edition

Feature	Community	Enterprise	Cloud
HCC Compression	Basic	Full Adaptive	Full + S3 Integration
Storage Tiering	2 tiers	4 tiers	Unlimited + S3/Glacier
MVCC	Full	Full	Full
Bloom Filters	Full	Full	Full
Compaction	Leveled only	All strategies	All + Cloud-optimized
Caching	L2-L3	L1-L4	L1-L4 + CDN
Sharding	Manual	Auto	Auto + Cross-region
WAL	Full	Full	Full + Cross-AZ
Query Optimization	Full	Full + ML hints	Full + ML + Cost optimization
Self-Tuning	Manual stats	Auto stats	HBGDO + Workload prediction

---

---

Appendix C: Source File Reference

All implicit features are implemented in the following locations:

Compression Features

Feature	Source File	Lines
HCC Adaptive	heliosdb-storage/src/hcc/adaptive_compression.rs	1-150
Time-Series Gorilla	heliosdb-storage/src/timeseries/compression_v2.rs	1-100
Dictionary Encoding	heliosdb-storage/src/hcc/enhanced_dictionary.rs	-
Delta Encoding	heliosdb-storage/src/hcc/delta_encoding.rs	-
RLE	heliosdb-storage/src/hcc/run_length_encoding.rs	-

Tiering Features

Feature	Source File	Lines
Multi-Tier Policy	heliosdb-storage/src/cloud/tiering_policy.rs	14-95
Time-Series Tiering	heliosdb-storage/src/timeseries/tiered_storage.rs	24-100

MVCC Features

Feature	Source File	Lines
Snapshot Manager	heliosdb-storage/src/mvcc_snapshot_manager.rs	16-96
Advanced GC	heliosdb-storage/src/advanced_mvcc_gc.rs	18-35
GC Tuning	heliosdb-storage/src/gc_tuning.rs	8-56

Index Features

Feature	Source File	Lines
BRIN Index	heliosdb-query/crates/indexes/src/brin.rs	54-82
Index Maintenance	heliosdb-storage/src/index/maintenance.rs	79-100
Adaptive Selection	heliosdb-storage/src/adaptive_index_selection.rs	-

Compaction Features

Feature	Source File	Lines
Compaction Strategy	heliosdb-storage/src/compaction.rs	16-55
LSM Tuning	heliosdb-storage/src/lsm_tuning.rs	8-145

Caching Features

Feature	Source File	Lines
Unified Cache	heliosdb-cache/src/unified/mod.rs	1-83
Prefetching	heliosdb-cache/src/prefetch/mod.rs	1-77
ML Predictor	heliosdb-cache/src/ml_predictor/	-

Query Optimization

Feature	Source File	Lines
Distributed Optimizer	heliosdb-query/crates/distributed-optimizer/src/optimizer.rs	11-33
Cost Optimizer v2	heliosdb-query/crates/cost-optimizer-v2/src/auto_optimizer.rs	1-78
Statistics	heliosdb-query/crates/cost-optimizer-v2/src/statistics.rs	25-95

Sharding Features

Feature	Source File	Lines
Consistent Hash Ring	heliosdb-cluster/src/sharding/hash_ring.rs	28-84

WAL Features

Feature	Source File	Lines
WAL Writer	heliosdb-storage/src/wal.rs	1-95

---

Appendix D: Actual Default Configuration Values

Based on codebase analysis, these are the exact defaults:

Storage (heliosdb-storage/src/config.rs)

memtable_size_mb: 128,        // Optimized for 3-node cluster
flush_threshold: 0.9,         // 90% before forced flush
write_batch_size_bytes: 4MB,
bloom_filter_fp_rate: 0.01,   // 1% false positive
enable_compression: true,
compaction_threads: 4,

Compaction (heliosdb-storage/src/compaction.rs)

strategy: CompactionStrategy::SizeTiered,
min_sstables_for_compaction: 4,
level0_size_threshold: 100MB,
level_size_multiplier: 10,
gc_grace_seconds: 864000,     // 10 days
max_concurrent_compactions: 4,

MVCC GC (heliosdb-storage/src/gc_tuning.rs)

max_pause_ms: 50,
trigger_threshold_percent: 75.0,
enable_incremental: true,
incremental_slice_us: 5000,   // 5ms slices
enable_arena_allocation: true,
arena_count: num_cpus::get(),

Query Optimizer (distributed-optimizer/src/optimizer.rs)

enable_join_reordering: true,
enable_partition_pruning: true,
enable_predicate_pushdown: true,
enable_cost_based_optimization: true,
max_join_reorder_size: 8,
timeout_ms: 100,

BRIN Index (heliosdb-query/crates/indexes/src/brin.rs)

pages_per_range: 128,
enable_bloom_filters: true,
bloom_filter_items: 1000,
bloom_filter_fp_rate: 0.01,
enable_minmax: true,
track_nulls: true,

Tiering (heliosdb-storage/src/timeseries/tiered_storage.rs)

hot_tier:  { aggregation: None, compression: Level1 }
warm_tier: { aggregation: 5min, compression: Level6 }
cold_tier: { aggregation: 1hr,  compression: Level9 }
archive:   { aggregation: N/A,  compression: Level9 }

---

Document Metadata:

• Classification: Technical Reference
• Review Cycle: Quarterly
• Last Updated: 2025-01-25
• Related Documents:
  • HBGDO_VS_ORACLE_CBO_COMPARISON.md
  • PROTOCOL_FEATURE_MATRIX.md
  • USER_DOCUMENTATION_INDEX.md