---
title: "HeliosDB-Lite v2.2.0 Release Notes"
---

# HeliosDB-Lite v2.2.0 Release Notes

> **Note (current distribution):** HeliosDB Lite is distributed to licensed customers as a
> source distribution that builds the `heliosdb-lite` binary. Upgrade steps on this page use
> that binary, SQL and the configuration file; see
> [Installation](/docs/lite/getting-started/installation/) and the
> [CLI reference](/docs/lite/reference/cli/).

**Codename**: "Quality & Performance"

---

## Highlights

HeliosDB-Lite v2.2.0 adds full ACID compliance, query optimization and production monitoring.

**Key Achievements**:
- **Full ACID Compliance**: Snapshot isolation with MVCC guarantees data consistency
- **Crash Recovery**: Complete WAL replay ensures zero data loss after crashes
- **Intelligent Query Planning**: Real statistics drive 10-100x performance gains for complex queries
- **Production Monitoring**: Comprehensive system views for observability
- **Enterprise Compression**: FSST string compression with persistent dictionaries

---

## New Features

### Storage & Durability

#### MVCC Snapshot Isolation

Complete multi-version concurrency control implementation providing true snapshot isolation:
- Read-your-own-writes semantics maintained
- Transactions see consistent data snapshots
- Version resolution via reverse timestamp indexing (O(log N))
- Automatic fallback for non-versioned keys (metadata, catalog)

**Benefits**:
- Prevents dirty reads, non-repeatable reads, and phantom reads
- Enables long-running analytical queries without blocking writes
- Full ACID guarantee compliance

**Performance**: <2x overhead vs non-MVCC reads

#### WAL Replay for Crash Recovery

Write-ahead log replay with two-pass algorithm:

**Pass 1**: Transaction state tracking
- Process Begin/Commit/Abort operations
- Build transaction state maps
- Identify incomplete transactions

**Pass 2**: Operation application
- Skip operations from aborted transactions
- Apply committed operations to restore state
- Resilient error handling (continues on non-fatal errors)

**Supported Operations**: Insert, Update, Delete, CreateTable, DropTable, Begin/Commit/Abort

**Features**:
- Transaction-aware replay handles partial transactions
- Idempotent (safe to replay multiple times)
- Progress reporting (logs every 1000 operations)
- Comprehensive debug logging

**Impact**: Data durability guaranteed after crashes

#### Table Name Extraction

Robust key format parser for WAL context.

**Benefits**:
- Point-in-time recovery with table context
- Improved debugging (crash logs show affected tables)
- Table-specific recovery from WAL
- Better observability


---

### Query Optimization

#### Real Statistics Collection

Statistics collection and analysis system:

**Table Statistics**:
- Row count tracking
- Average row size calculation
- Total table size estimation
- Last analyzed timestamp

**Column Statistics**:
- Distinct value count (cardinality)
- NULL fraction
- Min/max value ranges
- Average column width

**Selectivity Estimation**: Selectivity is estimated for equality, inequality, range, AND, OR, and IS NULL predicates.

**Integration**: Replaces hardcoded estimates with real data

**Impact**:
- Better query plan quality
- Optimal join ordering
- Accurate row count estimates

#### Cost-Based Join Selection (v2.1.0, Enhanced)

Intelligent join algorithm selection based on actual costs:

**Algorithms Supported**:
- **Hash Join**: O(left + right) for large tables with equality predicates
- **Nested Loop Join**: O(left × right) optimal for small tables or non-equality joins

**Decision Logic**:
- With cost estimator: Choose algorithm with lower cost
- Without cost estimator: Heuristic fallback (equality → hash, non-equality → nested loop)

**Example Scenarios**:
- Small tables (100 × 100): Hash join preferred
- Large tables (100K × 100K): Hash join critical (3333x speedup)
- Non-equality predicates: Nested loop required

**Overhead**: <2μs per join planning decision

---

### Protocol & Compatibility

#### PostgreSQL Schema Derivation

Automatic schema extraction for prepared statements:

**Implementation**:
1. Parse SQL query to AST using sqlparser-rs
2. Create logical plan using query planner with catalog context
3. Extract schema from logical plan
4. Store schema in the prepared statement
5. Return schema in Describe messages (RowDescription)

**Type Mapping**:
- Automatic HeliosDB → PostgreSQL OID conversion
- Supports all standard types (INT, TEXT, FLOAT, JSON, TIMESTAMP, etc.)

**Performance**: ~160μs overhead during Parse phase (one-time cost)

**Benefits**:
- Full extended query protocol support
- Works with all PostgreSQL client libraries (psycopg2, node-postgres, etc.)
- Clients can determine result column types without executing queries
- Improved client compatibility


---

### Monitoring & Observability

#### Transaction Statistics Tracking

Thread-safe atomic counters for database activity:

**Metrics Tracked**:
- `xact_commit`: Committed transaction count
- `xact_rollback`: Rolled back transaction count
- Tuple operations: inserts, updates, deletes
- Real-time statistics capture

**Features**:
- Atomic counters (lock-free performance)
- Point-in-time snapshot capture
- Integration with `pg_stat_database` system view

#### Enhanced System Views

**Branch Parent Name Lookup**:
- `pg_branches` view displays actual parent names (not just IDs)

**Snapshot Size Calculation**:
- Enables snapshot capacity planning

**Compression Value Tracking**:
- Added value count to compression statistics
- FSST encoder tracks count as number of strings compressed
- Per-value compression efficiency analysis

**Vector Index Length**:
- Accurate vector count from underlying HNSW indexes
- Proper monitoring and query planning statistics

**Query Statistics Integration**:
- `pg_stat_optimizer` returns real statistics
- EXPLAIN plans show actual cardinality estimates

---

### Compression Enhancements

#### FSST String Compression

Fast Static Symbol Table (FSST) compression:

**Dictionary Persistence**:
- Custom binary format serialization
- Exports 255 symbols (9 bytes each: 8-byte data + 1-byte length)
- Full codec restoration from stored dictionaries
- Roundtrip validation ensures compression consistency

**Optimized Batch Processing**:
- Pre-allocated batch compression: 10-15% faster
- Adaptive batch compression: evaluates the compression ratio and skips incompressible data
- 64-item chunked processing for better cache locality

**Intelligent Sampling Strategies**:
- **Stratified**: Best for uniform data
- **BeginningWeighted**: Good for time-series
- **Diversity**: Best for high-cardinality data
- **Auto**: Selects a strategy based on the data

**Performance**:
- 2.0-2.5x compression ratio on emails
- 1.8-2.2x compression ratio on URLs
- <5% CPU overhead
- Dictionary training: <200μs for 10K items


#### Compression Configuration API

Fluent builder pattern for compression configuration:

**Features**:
- Per-column compression type selection (FSST, ALP, None)
- Configurable block sizes and dictionary limits
- Statistics reporting (compression ratios, space saved)
- System view integration (`pg_compression_stats`)

---

## Performance Improvements

### Baseline vs v2.2.0

Indicative figures; reproduce on your own hardware.

| Operation | v2.1.0 Baseline | v2.2.0 Target | Improvement |
|-----------|-----------------|---------------|-------------|
| **SELECT (in-memory)** | 50μs | 35μs (est) | 30% faster |
| **INSERT (with WAL)** | 100μs | 100μs | Maintained |
| **Transaction Commit** | 200μs | 140μs (est) | 30% faster |
| **Query Planning** | Variable | 15% faster | Better statistics |
| **Compression Throughput** | Baseline | 20% faster | SIMD + batching |

### MVCC & Time-Travel Performance

- **AS OF query overhead**: <2x vs current-time queries
- **Snapshot lookup**: O(1) in-memory metadata access
- **Version resolution**: O(log N) using reverse timestamp indexing
- **Snapshot GC**: >1000 keys/sec throughput

---

## Breaking Changes

**None** - This release maintains full backward compatibility with v2.1.x.

### API Compatibility

All v2.1.x SQL, PostgreSQL wire clients and configuration files continue to work without
modification. In-process code written against v2.1.x also works unchanged (embedded API,
licensed source distribution).

### Configuration Changes

**None required** - New features are enabled by default with zero configuration:
- MVCC snapshot isolation: Automatic
- WAL replay: Automatic on database open
- Statistics collection: no SQL step; HeliosDB Lite has no standalone `ANALYZE` statement

---

## Known Limitations

### Deferred to v2.3.0

**Sync Protocol Implementation**:
- Change log query support
- Conflict detection and resolution
- HTTP/gRPC communication layer
- Delta application logic
- Client-side conflict handlers

**Incremental Materialized Views**:
- Change tracking for base tables
- Delta computation logic
- CPU-aware refresh scheduling

### Remaining Work

**Background Worker**:
- Auto-refresh for materialized views (currently manual)
- CPU monitoring and throttling
- Staleness threshold detection

### System View Completion

**Partial Implementation**:
- Some system views return placeholder data
- Full implementation targeted for v2.3.0

---

## Upgrade Guide

### From v2.1.x to v2.2.0

#### Step 1: Backup Your Database

```bash
# Stop your application
# Copy database directory
cp -r ./mydb.helio ./mydb.helio.backup
```

#### Step 2: Build the New Binary

Build v2.2.0 from the licensed source distribution and keep the v2.1.x binary for a rollback:

```bash
cd heliosdb-lite
cargo build --release
export PATH="$PWD/target/release:$PATH"
```

#### Step 3: No Configuration Changes Required

v2.2.0 is a drop-in replacement. All new features are either automatic or opt-in:

**Automatic Features**:
- MVCC snapshot isolation (zero configuration)
- WAL replay on database open (automatic crash recovery)
- Transaction statistics tracking (automatic)

**Opt-In Features**:
```sql
-- Query statistics
SELECT * FROM pg_stat_optimizer;

-- View compression stats
SELECT * FROM pg_compression_stats;

-- Transaction statistics
SELECT * FROM pg_stat_database;
```

#### Step 4: Verify Migration

Open the database with the new binary. WAL replay runs automatically when the server or the
REPL opens the data directory, so a directory left by an unclean shutdown is recovered here:

```bash
heliosdb-lite repl --data-dir ./mydb.helio
```

```sql
-- Verify data integrity
SELECT COUNT(*) FROM users;

-- Transactions now run under snapshot isolation
BEGIN;
SELECT * FROM users;
COMMIT;
```

#### Step 5: Optimize Performance

```sql
-- Query optimizer statistics
SELECT * FROM pg_stat_optimizer;

-- Test query performance
EXPLAIN SELECT * FROM users JOIN orders ON users.id = orders.user_id;
-- Shows the chosen plan, including the join algorithm
```

### Migration Checklist

- [ ] Backup database directory
- [ ] Build the v2.2.0 binary (`cargo build --release` in the source distribution)
- [ ] Restart the server or application (WAL replay automatic)
- [ ] Verify data integrity with COUNT queries
- [ ] Test transaction isolation (MVCC active by default)
- [ ] Review system views for monitoring

### Rolling Back (If Needed)

```bash
# Stop the server or application
# Restore backup
rm -rf ./mydb.helio
cp -r ./mydb.helio.backup ./mydb.helio

# Start the v2.1.x binary you kept on the restored directory
```

---

## Compatibility

### PostgreSQL Protocol

**Compatibility Level**: 95%+

**Supported Features**:
- Extended Query Protocol (Parse, Bind, Execute, Describe)
- Simple Query Protocol
- Prepared statements with schema derivation
- Authentication (MD5, SCRAM-SHA-256)
- SSL/TLS connections
- COPY protocol

**Limitations**:
- Some advanced prepared statement features not yet supported
- Sync protocol (deferred to v2.3.0)

### SQL Standard

**Compliance**: SQL-92 + PostgreSQL extensions

**Supported**:
- SELECT, INSERT, UPDATE, DELETE
- JOINs (INNER, LEFT, RIGHT, FULL)
- Subqueries and CTEs (WITH)
- Aggregations (GROUP BY, HAVING)
- Window functions
- MVCC snapshot isolation (AS OF)

### Rust Version

**Minimum Required**: Rust 1.70+

**Recommended**: Rust 1.75+ (latest stable)

**MSRV Policy**: 6-month trailing support

### Platforms

**Tier 1** (Fully Tested):
- Linux (x86_64, ARM64)
- macOS (Intel, Apple Silicon)

**Tier 2** (Community Supported):
- Windows (x86_64)
- FreeBSD (x86_64)

**Architecture Support**:
- x86_64 (full SIMD support)
- ARM64 (full NEON support)
- WASM (experimental, limited features)

### Dependencies

**Core Dependencies**:
- RocksDB 0.22+ (storage backend)
- Arrow 52+ (columnar format)
- Tokio 1.40+ (async runtime)
- sqlparser 0.53+ (SQL parsing)

**Optional Dependencies**:
- hnsw_rs 0.3+ (vector search)
- aes-gcm 0.10+ (encryption)

---

## Summary

HeliosDB-Lite v2.2.0 adds:
- Full ACID compliance (MVCC + WAL)
- Query optimization
- Crash recovery
- Monitoring
- Better protocol compliance

---

## Support & Resources

### Community

- **Product page**: https://www.heliosdb.com/lite.html
- **Issues**: Report bugs and feature requests to support@heliosdb.com
- **Community**: [Discord](https://discord.gg/yTykuUrFXc)

### Professional Support

- Contact the HeliosDB team for enterprise support options
- Custom feature development available
- Performance tuning and optimization consulting

---

**Version**: 2.2.0
**Codename**: "Quality & Performance"