HeliosDB-Lite Feature Index

HeliosDB-Lite Feature Index

Quick Navigation Guide - Find features by location and category

Directory Structure

src/
├── lib.rs                   # Main library - EmbeddedDatabase API
├── types.rs                 # Data types (20+ SQL types)
├── config.rs                # Configuration system
├── error.rs                 # Error types
├── main.rs                  # CLI entry point
│
├── sql/                     # SQL execution (37 files)
│   ├── parser.rs            # SQL parsing
│   ├── planner.rs           # Query planning
│   ├── logical_plan.rs      # Logical plan structures
│   ├── evaluator.rs         # Expression evaluation
│   ├── executor/            # Query execution
│   │   ├── scan.rs          # Table scans
│   │   ├── filter.rs        # WHERE filtering
│   │   ├── project.rs       # SELECT projection
│   │   ├── join.rs          # JOIN operations
│   │   ├── aggregate.rs     # GROUP BY, aggregates
│   │   ├── ddl.rs           # CREATE/DROP operations
│   │   └── phase3.rs        # Phase 3 feature execution
│   ├── phase3/              # Phase 3 SQL extensions
│   │   ├── branching.rs     # Database branching
│   │   ├── time_travel.rs   # AS OF queries
│   │   ├── materialized_views.rs  # MV creation
│   │   └── system_views.rs  # System view queries
│   ├── explain*.rs          # Query explanation (9 files)
│   ├── index_recommender.rs # Index recommendations
│   ├── system_tables.rs     # System table definitions
│   ├── system_views.rs      # System view registry
│   └── settings.rs          # Session settings
│
├── storage/                 # Storage layer (32 files)
│   ├── engine.rs            # RocksDB storage engine
│   ├── catalog.rs           # Table catalog
│   ├── transaction.rs       # Transaction management
│   ├── mvcc.rs              # MVCC implementation
│   ├── wal.rs               # Write-ahead logging
│   ├── time_travel.rs       # Snapshot management
│   ├── branch.rs            # Database branching
│   ├── materialized_view.rs # MV catalog
│   ├── mv_*.rs              # MV-related modules (6 files)
│   ├── compression/         # Compression codecs (10 files)
│   │   ├── fsst/            # String compression (4 files)
│   │   ├── alp/             # Numeric compression (4 files)
│   │   ├── api.rs           # Compression API
│   │   ├── integration.rs   # Codec integration
│   │   ├── tuple_compression.rs  # Row compression
│   │   └── simd_ops.rs      # SIMD operations
│   ├── vector_index.rs      # Vector index metadata
│   ├── gin_index.rs         # GIN indexes (JSONB)
│   ├── statistics.rs        # Column statistics
│   └── stats.rs             # Database statistics
│
├── vector/                  # Vector search (13 files)
│   ├── hnsw_index.rs        # HNSW indexing
│   ├── quantized_hnsw.rs    # Quantized HNSW
│   ├── quantization/        # Vector quantization (8 files)
│   │   ├── product_quantizer.rs  # Product quantization
│   │   ├── codebook.rs      # Codebook management
│   │   ├── encoder.rs       # Vector encoding
│   │   ├── decoder.rs       # Vector decoding
│   │   ├── distance.rs      # Distance computation
│   │   └── training.rs      # Codebook training
│   └── simd/                # SIMD operations
│       ├── distance.rs      # SIMD distance
│       └── quantization.rs  # SIMD quantization
│
├── network/                 # Network server (5 files)
│   ├── server.rs            # PostgreSQL server
│   ├── protocol.rs          # Wire protocol
│   ├── session.rs           # Session management
│   ├── auth.rs              # Authentication
│   └── mod.rs               # Network module
│
├── repl/                    # Interactive shell (5 files)
│   ├── shell.rs             # REPL shell
│   ├── commands.rs          # Meta commands
│   ├── completer.rs         # Auto-completion
│   ├── formatter.rs         # Result formatting
│   └── mod.rs               # REPL module
│
├── audit/                   # Audit logging (5 files)
│   ├── logger.rs            # Audit logger
│   ├── events.rs            # Audit events
│   ├── config.rs            # Audit configuration
│   ├── query.rs             # Audit queries
│   └── mod.rs               # Audit module
│
├── crypto/                  # Encryption (2 files)
│   ├── key_manager.rs       # Key management
│   └── mod.rs               # Crypto module
│
├── optimizer/               # Query optimization
├── compute/                 # Computation layer
├── tenant/                  # Multi-tenancy support
├── protocol/                # Protocol definitions
├── protocols/               # Protocol adapters
└── sync/                    # Sync protocol (experimental)

---

Feature Location Matrix

Query Execution Features

Feature	Location	Status
SELECT with projection	sql/executor/project.rs	Complete
WHERE filtering	sql/executor/filter.rs	Complete
GROUP BY / aggregates	sql/executor/aggregate.rs	Complete
ORDER BY / LIMIT	sql/executor/aggregate.rs	Complete
JOINs (all types)	sql/executor/join.rs	Complete
INSERT	lib.rs:284-381	Complete
UPDATE	lib.rs:670-726	Complete
DELETE	lib.rs:727-777	Complete
TRUNCATE	lib.rs:795-823	Complete
CREATE TABLE	sql/executor/ddl.rs	Complete
DROP TABLE	sql/executor/ddl.rs	Complete
CREATE INDEX	sql/executor/ddl.rs	Complete
DROP INDEX	sql/executor/ddl.rs	Complete
Vector search	sql/executor/scan.rs	Complete
Time-travel queries	sql/phase3/time_travel.rs	Complete
Parameterized queries	lib.rs:889-901	Complete

Data Types & Structures

Feature	Location	Types
Numeric types	types.rs	Int2, Int4, Int8, Float4, Float8, Numeric
Text types	types.rs	Varchar, Text, Char
Temporal types	types.rs	Date, Time, Timestamp, Timestamptz, Interval
Structured types	types.rs	JSON, JSONB, Array, Vector
Binary type	types.rs	Bytea
Special types	types.rs	UUID, Boolean
NULL handling	types.rs	Value::Null
Type casting	sql/evaluator.rs	Auto-cast + explicit CAST

Indexing & Performance

Feature	Location	Algorithm
Vector search (HNSW)	vector/hnsw_index.rs	HNSW graphs
Vector quantization	vector/quantization/	Product quantization
Quantized HNSW	vector/quantized_hnsw.rs	PQ + HNSW
GIN index (JSONB)	storage/gin_index.rs	Inverted index
B-tree indexes	storage/catalog.rs	Range queries
Statistics	storage/statistics.rs	Column stats
Index recommender	sql/index_recommender.rs	Cost analysis

Storage & Compression

Feature	Location	Technology
Storage engine	storage/engine.rs	RocksDB LSM-tree
Write-ahead log	storage/wal.rs	Durability
Transactions	storage/transaction.rs	ACID
MVCC	storage/mvcc.rs	Snapshot isolation
FSST compression	storage/compression/fsst/	String compression
ALP compression	storage/compression/alp/	Numeric compression
Compression API	storage/compression/api.rs	Per-column codecs
Tuple compression	storage/compression/tuple_compression.rs	Per-row compression

Self-Maintaining Filter Index (SMFI)

Feature	Location	Description
Bloom Filters	storage/bloom_filter.rs	Probabilistic membership testing
Zone Maps	storage/zone_map.rs	Block-level min/max statistics
SIMD Filtering	storage/simd_filter.rs	Vectorized predicate evaluation
Predicate Pushdown	storage/predicate_pushdown.rs	Storage-level filtering
Filter Delta Tracker	storage/filter_index_delta.rs	Self-maintaining DML tracking
Consolidation Worker	storage/filter_consolidation_worker.rs	CPU-aware background maintenance
Columnar Zone Summaries	storage/columnar_zone_summary.rs	HyperLogLog, MCV, histograms
Speculative Filters	storage/speculative_filter.rs	Auto-created indexes
Parallel Filter Engine	storage/parallel_filter.rs	rayon-based parallel filtering

Time-Travel & Versioning

Feature	Location	Capability
AS OF TIMESTAMP	sql/phase3/time_travel.rs	Point-in-time queries
AS OF TRANSACTION	sql/phase3/time_travel.rs	Transaction-level queries
AS OF SCN	sql/phase3/time_travel.rs	System change numbers
Snapshots	storage/time_travel.rs	Metadata & caching
Garbage collection	storage/time_travel.rs	Snapshot cleanup

Database Management

Feature	Location	Capability
CREATE BRANCH	sql/phase3/branching.rs	Branch creation
DROP BRANCH	sql/phase3/branching.rs	Branch deletion
MERGE BRANCH	sql/phase3/branching.rs	Branch merging
Branch metadata	storage/branch.rs	Tracking & stats
Conflict resolution	storage/branch.rs	Merge strategies
Catalog	storage/catalog.rs	Table metadata
Transactions	lib.rs:1201-1478	BEGIN/COMMIT/ROLLBACK

Network & Protocol

Feature	Location	Protocol
Server implementation	network/server.rs	PostgreSQL v3.0
Wire protocol	network/protocol.rs	Async Tokio
Session management	network/session.rs	Per-connection state
Authentication	network/auth.rs	MD5, SCRAM-SHA-256
SSL/TLS	network/auth.rs	Encrypted connections

System Features

Feature	Location	Capability
Materialized views	storage/materialized_view.rs	Create/refresh/drop
MV auto-refresh	storage/mv_auto_refresh.rs	Scheduled refresh
MV incremental	storage/mv_incremental.rs	Delta-based refresh
MV scheduler	storage/mv_scheduler.rs	CPU-aware scheduling
Delta tracking	storage/mv_delta.rs	Change tracking
System views	sql/phase3/system_views.rs	pg_* views
Audit logging	audit/	Tamper-proof logs
Encryption (TDE)	crypto/	AES-256-GCM

Multi-Tenancy & Security (v3.4)

Feature	Location	Capability
Tenant management	tenant/manager.rs	Create/delete tenants
Tenant context	tenant/context.rs	Session tenant binding
Tenant quotas	tenant/quota.rs	Storage/connection limits
Tenant plans	tenant/plan.rs	Subscription tiers
Row-Level Security	sql/rls/	Policy-based filtering
RLS policies	sql/rls/policy.rs	CREATE POLICY execution
CDC events	tenant/cdc.rs	Change data capture
Tenant isolation	tenant/isolation.rs	Shared/schema/database

Developer Tools

Feature	Location	Capability
REPL shell	repl/shell.rs	Interactive SQL
Meta commands	repl/commands.rs	\d, \branches, etc.
Auto-completion	repl/completer.rs	Table/column hints
Result formatting	repl/formatter.rs	Pretty-printing
EXPLAIN plans	sql/explain.rs	Query analysis
Index recommender	sql/index_recommender.rs	Suggestions
Query optimizer	optimizer/	Cost-based

Embedded API

Feature	Location	API
Database creation	lib.rs:411-445	new(), new_in_memory()
Configuration	config.rs	with_config()
SQL execution	lib.rs:477-547	execute(), execute_params()
Querying	lib.rs:1083-1199	query(), query_params()
Transactions	lib.rs:1201-1478	begin(), commit(), rollback()

---

Feature Dependencies

Core Dependencies

EmbeddedDatabase
├── StorageEngine
│   ├── RocksDB
│   ├── Catalog
│   ├── VectorIndexManager
│   ├── SnapshotManager
│   ├── BranchManager
│   ├── CompressionManager
│   └── WriteAheadLog
├── Executor
│   ├── Parser
│   ├── Planner
│   └── Evaluator
└── Configuration

Vector Search Dependencies

HNSW Index
├── DistanceMetric (Cosine, L2, InnerProduct)
├── SIMD acceleration (AVX2)
└── ProductQuantizer (optional)

Time-Travel Dependencies

AS OF Queries
├── SnapshotManager
│   ├── Snapshot metadata
│   ├── LRU cache
│   └── GC configuration
└── MVCC isolation

Materialized View Dependencies

Materialized Views
├── Storage engine
├── MV scheduler
│   ├── CPU monitor
│   └── Priority queue
├── Delta tracker
├── Incremental refresher
└── System views

SMFI Dependencies

Self-Maintaining Filter Index
├── Filter Delta Tracker
│   ├── Bloom filter deltas
│   ├── Zone map deltas
│   └── Delta sequence
├── Consolidation Worker
│   ├── CPU monitor (shared with MV)
│   └── Background thread
├── Speculative Filter Manager
│   ├── Query pattern tracker
│   └── Filter lifecycle
├── Parallel Filter Engine
│   ├── rayon thread pool
│   └── Adaptive chunking
└── Storage-persisted structures
    ├── RocksDB bloom keys
    └── RocksDB zone keys

---

Module Interaction Diagram

lib.rs (EmbeddedDatabase)
  ↓
sql/ (Query processing)
  ├→ Parser → Planner → Executor
  ├→ phase3/ (Branching, Time-travel, MVs)
  └→ explain/ (Query analysis)
      ↓
  storage/ (Data storage)
    ├→ StorageEngine (RocksDB)
    ├→ Catalog (Metadata)
    ├→ Transaction (ACID)
    ├→ MVCC (Isolation)
    ├→ compression/ (Codecs)
    ├→ time_travel/ (Snapshots)
    ├→ branch/ (Branching)
    ├→ materialized_view/ (MVs)
    └→ smfi/ (Self-Maintaining Filters)
        ├→ bloom_filter (Membership)
        ├→ zone_map (Block stats)
        ├→ simd_filter (Vectorized)
        ├→ filter_index_delta (DML tracking)
        ├→ speculative_filter (Auto-create)
        └→ parallel_filter (Parallel eval)
        ↓
    vector/ (Indexing)
      ├→ HNSW index
      └→ Quantization

network/ (Server)
  ├→ Server (Async)
  ├→ Protocol (PostgreSQL)
  ├→ Session (State)
  └→ Auth (Security)

repl/ (Tools)
  ├→ Shell (Interactive)
  ├→ Commands (Meta)
  └→ Formatter (Output)

audit/ (Logging)
  └→ Logger (Events)

crypto/ (Security)
  └→ KeyManager (Encryption)

---

Code Complexity Guide

Simple Modules (Read First)

• types.rs - Data type definitions
• config.rs - Configuration
• error.rs - Error handling
• repl/formatter.rs - Output formatting

Medium Complexity

• sql/parser.rs - SQL parsing
• sql/logical_plan.rs - Plan structures
• sql/evaluator.rs - Expression evaluation
• storage/catalog.rs - Metadata management

High Complexity

• sql/executor/mod.rs - Query execution engine
• storage/engine.rs - Storage engine
• storage/time_travel.rs - Snapshot management
• storage/branch.rs - Branching implementation

Very High Complexity

• sql/phase3/branching.rs - Branch merging logic
• storage/mv_scheduler.rs - MV scheduling
• vector/quantization/ - PQ training
• network/server.rs - Async networking

---

Key File Sizes

src/lib.rs                          1478 lines  Core API
src/config.rs                        520 lines  Configuration
src/types.rs                         415 lines  Data types
src/error.rs                         200 lines  Errors

sql/
  parser.rs                          450 lines
  logical_plan.rs                    500 lines
  planner.rs                         600 lines
  evaluator.rs                       700 lines
  executor/mod.rs                    300 lines
  executor/scan.rs                   400 lines

storage/
  engine.rs                          600 lines
  catalog.rs                         500 lines
  compression/                      ~2000 lines
  time_travel.rs                     400 lines
  branch.rs                          500 lines

vector/
  hnsw_index.rs                      400 lines
  quantization/                     ~1500 lines

network/
  server.rs                          400 lines
  protocol.rs                        300 lines

repl/
  shell.rs                           300 lines

---

How to Navigate

1. Want to understand SQL execution?

   • Start: src/sql/parser.rs → planner.rs → executor/mod.rs
   • Details: sql/executor/* (scan, filter, project, etc.)

2. Want to understand storage?

   • Start: src/storage/engine.rs → catalog.rs
   • Details: storage/compression/, mvcc.rs, wal.rs

3. Want to understand vector search?

   • Start: src/vector/hnsw_index.rs
   • Details: vector/quantization/, vector/simd/

4. Want to understand time-travel?

   • Start: src/storage/time_travel.rs
   • Details: sql/phase3/time_travel.rs, sql/executor/phase3.rs

5. Want to understand branching?

   • Start: src/storage/branch.rs
   • Details: sql/phase3/branching.rs, sql/executor/phase3.rs

6. Want to understand materialized views?

   • Start: src/storage/materialized_view.rs
   • Details: storage/mv_*.rs, sql/phase3/materialized_views.rs

7. Want to understand SMFI (storage-level filtering)?

   • Start: src/storage/bloom_filter.rs, src/storage/zone_map.rs
   • Delta tracking: storage/filter_index_delta.rs
   • Auto-filters: storage/speculative_filter.rs
   • Parallel: storage/parallel_filter.rs

8. Want to understand the API?

   • Start: src/lib.rs (lines 107-1478)
   • Examples: examples/ directory

9. Want to understand multi-tenancy?

   • Start: src/tenant/manager.rs
   • Details: tenant/context.rs, tenant/quota.rs, tenant/cdc.rs
   • RLS: sql/rls/ directory

10. Want to understand Row-Level Security?

    • Start: src/sql/rls/policy.rs
    • System views: pg_rls_policies() in sql/phase3/system_views.rs

---

Total Codebase: ~56,000 lines of Rust Module Count: 80+ files Feature Count: 160+ Version: v3.4.0 (December 2025)