---
title: "Changelog"
---

# Changelog

All notable changes to HeliosDB Lite will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [3.1.0] - 2025-12-08 - Multi-User ACID In-Memory Mode

### Highlights

HeliosDB-Lite v3.1.0 introduces comprehensive multi-user ACID transaction support for in-memory mode with user-triggered persistence:
- **Multi-User Sessions**: Concurrent user sessions with independent transaction contexts
- **Advanced Locking**: Deadlock detection, configurable timeouts, three lock modes
- **Isolation Levels**: READ COMMITTED, REPEATABLE READ, SERIALIZABLE
- **Dump/Restore**: Manual and automated backup/restore with compression
- **Resource Quotas**: Per-user memory limits, connection limits, timeouts
- **100% Backward Compatible**: All v3.0.0 code works without changes

### Added

#### Session Management
- **Session Manager**: Multi-user session coordination with per-user contexts
- **Session Lifecycle**: Create, authenticate, timeout, and cleanup sessions
- **Session Statistics**: Track transactions, queries, memory usage per session
- **User Authentication**: Trust, password, and SCRAM-SHA-256 auth methods
- **Resource Quotas**: Per-user memory limits, connection limits, transaction timeouts
- **Session Monitoring**: `pg_stat_activity` system view for active sessions

#### Lock Management
- **Lock Manager**: Advanced lock acquisition with deadlock detection
- **Lock Modes**: Shared (S), Exclusive (X), Update (U) locks
- **Deadlock Detection**: Wait-for graph based cycle detection
- **Lock Timeout**: Configurable timeout with exponential backoff
- **Lock Statistics**: Track lock acquisition time, contention, deadlocks

#### Transaction Isolation Levels
- **READ COMMITTED**: Fresh snapshot per statement, no dirty reads
- **REPEATABLE READ**: Consistent snapshot throughout transaction
- **SERIALIZABLE**: Full serializability with conflict detection
- **Per-Session Configuration**: Set isolation level per session or transaction
- **SQL Support**: `BEGIN TRANSACTION ISOLATION LEVEL...` syntax

#### Dump/Restore System
- **Dump Manager**: Export/import database to portable dump files
- **Full Dumps**: Complete database snapshot with schema and data
- **Incremental Dumps**: Append-mode dumps with LSN tracking
- **Compression**: zstd (default), lz4, or no compression
- **Selective Dumps**: Dump specific tables only
- **Dump History**: Track dump operations with `pg_stat_dump_history` view
- **Dirty State Tracking**: Track uncommitted changes, warn on shutdown
- **Automated Scheduling**: Cron-style scheduled dumps via configuration

#### CLI Commands
- **`heliosdb-lite dump`**: Dump in-memory database to file
  - Options: `--output`, `--compress`, `--tables`, `--append`, `--verbose`
- **`heliosdb-lite restore`**: Restore from dump file
  - Options: `--input`, `--mode`, `--tables`, `--on-conflict`, `--verbose`
- **`heliosdb-lite status`**: Show database status and statistics
  - Options: `--format` (text/json)
- **Enhanced `start` command**: Multi-user session support
- **Enhanced `repl` command**: Session-aware interactive shell

#### Configuration Sections
- **`[session]`**: Session timeout, max sessions per user, default isolation level
- **`[locks]`**: Lock timeout, deadlock detection interval
- **`[dump]`**: Automated dump scheduling, compression settings, retention
- **`[resource_quotas]`**: Memory limits, connection limits, query timeouts

#### API
- **Session API**: `create_session()`, `destroy_session()`, `begin_transaction()`, `commit_transaction()`, `rollback_transaction()`
- **Dump API**: `dump()`, `restore()`, `verify_dump()`, `list_dumps()`
- **Transaction API**: `set_isolation_level()`, `get_isolation_level()`

#### System Views
- **`pg_stat_activity`**: Active session monitoring
- **`pg_locks`**: Lock information and wait states
- **`pg_stat_dump_history`**: Dump operation history

#### Documentation
- **User Guides**: In-memory mode guide, dump/restore procedures
- **Reference**: CLI v3.1 reference, configuration v3.1 reference
- **API Documentation**: Session management API, lock manager API, dump API
- **Architecture**: Multi-user transactions architecture with diagrams
- **Migration Guide**: v3.0 to v3.1 migration guide

### Changed

#### Performance Improvements
- Lock-free concurrent session management (10-100x faster)
- Optimized lock acquisition with adaptive backoff
- Batch dump operations (10,000 rows per batch)
- Streaming restore for large dumps (no full load in memory)
- Early lock release in commit path
- Read-heavy optimization with MVCC snapshots (no read locks)

#### Behavior Changes (Non-Breaking)
- In-memory mode now warns on shutdown if dirty state exists
- Sessions automatically cleaned up after timeout (default: 1 hour)
- Lock timeouts default to 30 seconds (configurable)
- Dumps default to zstd compression (3x-5x reduction)

### Fixed

None (all fixes from v3.0.0 included)

### Deprecated

None

### Removed

None

### Breaking Changes

**None.** v3.1.0 is 100% backward compatible with v3.0.0.

### Security

- Argon2 password hashing for user authentication
- Per-user resource quotas prevent resource exhaustion
- Session timeouts prevent abandoned connections
- Audit logging for session creation/destruction
- Dirty state tracking prevents data loss on shutdown

### Migration

**From v3.0.0 to v3.1.0:**
- No code changes required
- Existing applications work without modification
- New features are opt-in
- Configuration file backward compatible

### Known Limitations

1. **Session Management**: Optional feature, not required for single-user apps
2. **Dump Format**: `.heliodump` files not compatible with v3.0.0
3. **In-Memory Only**: Dump/restore primarily for in-memory mode (persistent mode has WAL)
4. **No Distributed Transactions**: Single-node transactions only

### Performance Metrics

Measured on representative fixtures; reproduce on your own hardware.

| Metric | Measured |
|--------|----------|
| Concurrent Sessions | 10,000+ |
| Transaction Latency (p99) | 1-5ms |
| Lock Acquisition | 0.05-1ms |
| Dump Throughput | 66-100MB/s |
| Restore Throughput | 40-66MB/s |

---

## [3.0.0] - 2025-12-06 - GA Release

### Highlights

HeliosDB-Lite v3.0.0 is the first General Availability release, featuring:
- **Enhanced REPL**: 19 new commands for vectors, agents, and documents
- **Comprehensive Configuration**: 8 new config sections for AI, RAG, and search
- **Feature-Based Help**: Intuitive help system organized by functionality

### Added

#### New REPL Commands
- **Session Management Commands**
  - `\sessions` - List all agent sessions
  - `\session-new <name>` - Create a new agent session
  - `\session <id>` - Show session details
  - `\session-delete <id>` - Delete an agent session
  - `\chat <id>` - Interactive chat with session
  - `\session-clear <id>` - Clear session messages

- **Vector Store Commands**
  - `\vectors` - List all vector stores
  - `\vector <name>` - Show vector store details
  - `\vector create <name> <dims> [metric]` - Create vector store
  - `\vector delete <name>` - Delete vector store
  - `\vector stats <name>` - Show vector store statistics

- **Document Collection Commands**
  - `\collections` - List all document collections
  - `\collection <name>` - Show collection details
  - `\docs <collection>` - List documents in collection
  - `\doc <collection> <id>` - Get document details
  - `\search-docs <query>` - Search across document collections

- **Performance & Utility Commands**
  - `\explain <query>` - Show query execution plan
  - `\profile <query>` - Profile query with detailed timing
  - `\telemetry` - Show database statistics and health

#### Help System Improvements
- Feature-based help organization (not version-based)
- New help categories: `basics`, `schema`, `branching`, `time-travel`, `vectors`, `documents`, `agents`, `ai`, `settings`, `examples`, `sql`
- Terminal-responsive layout (2-column for narrow, 3-column for wide terminals)
- Integrated examples with each command
- Category-specific help via `\h <category>`

#### Configuration Sections
- `[ai]` - AI feature toggle
- `[ai.llm]` - LLM provider configuration (OpenAI, Anthropic, Ollama)
- `[ai.embedding]` - Embedding model configuration
- `[rag]` - RAG chunking and retrieval settings
- `[search]` - Full-text search configuration
- `[wasm]` - WebAssembly execution limits
- `[mcp]` - Model Context Protocol server settings (planned; no MCP server ships in 3.6.0)
- `[storage.tuning]` - Advanced storage optimization

### Changed
- AI commands (`\ai infer`, `\ai generate`, `\ai optimize`) now use direct database API instead of showing REST API hints
- Improved error messages with helpful suggestions
- Schema analysis provides actionable index recommendations

### Fixed
- Help system now organizes by feature instead of version numbers
- Consistent type handling in session and vector commands

## [2.5.0] - 2025-12-01

### Added

#### REST API (v1)
- Full HTTP REST API with 14 endpoints via Axum framework
- Branch management endpoints (`GET/POST/DELETE /v1/branches`)
- Table management endpoints (`GET/POST/DELETE /v1/branches/{branch}/tables`)
- Data CRUD endpoints (`GET/POST/PUT/DELETE /v1/branches/{branch}/tables/{table}/data`)
- SQL query execution (`POST /v1/branches/{branch}/query`)
- SQL statement execution (`POST /v1/branches/{branch}/execute`)
- Vector similarity search endpoint (`POST /v1/branches/{branch}/vector/search`)
- Health check endpoint (`GET /v1/health`)
- Database info endpoint (`GET /v1/info`)

#### REST API Models
- Comprehensive request/response models for all endpoints
- `QueryRequest`/`QueryResponse` for SQL query results
- `ExecuteRequest`/`ExecuteResponse` for DDL/DML operations
- `InsertDataRequest`/`InsertDataResponse` for data insertion
- `UpdateDataRequest`/`UpdateDataResponse` for data updates
- `DeleteDataRequest`/`DeleteDataResponse` for data deletion
- `VectorSearchRequest`/`VectorSearchResponse` for similarity search
- Parameterized query support with `QueryParameter` types
- Pagination support with `PaginationInfo`

#### API Middleware
- JWT authentication middleware with configurable secret key
- API key authentication support
- Rate limiting middleware with configurable limits
- Flexible authentication strategies
- Request-scoped user information
- Customizable rate limiting rules

#### API Server Infrastructure
- Shared database connection across handlers
- Graceful shutdown support
- Configurable bind address and port
- CORS support for cross-origin requests
- Structured error responses

#### Audit Logging
- Audit logger for comprehensive operation tracking
- DDL event logging (CREATE, ALTER, DROP operations)
- DML event logging (INSERT, UPDATE, DELETE operations)
- Configurable async buffer for high-throughput logging
- Audit log table (`__audit_log`) for persistent storage

#### Documentation
- REST API endpoint documentation with examples
- Authentication guide
- Rate limiting configuration guide

### Changed

#### Storage Engine
- Improved branch manager concurrency
- Enhanced table scanning for audit logging

#### Query Handlers
- Schema inference from query results
- Tuple to JSON row conversion
- DataType string representation for all 17 types

### Fixed

- FSST dictionary array size handling (padded to 8 bytes)
- DataType match exhaustiveness in query handler
- Consistent naming of the binary value type (embedded API change, licensed source distribution)

### Security

- JWT-based authentication with HS256 algorithm
- API key authentication alternative
- Rate limiting to prevent abuse
- Query parameter validation

## [2.2.0] - 2025-11-24

### Added

#### Storage & Transactions
- MVCC snapshot isolation for transactions with proper versioning
- Complete WAL replay logic for crash recovery
- Table name extraction from keys in WAL entries for better observability
- Database statistics tracking (transaction commits, rollbacks, tuple operations)
- Thread-safe atomic counters for all transaction statistics
- Branch parent name lookup capability in branch management
- Snapshot size calculation for capacity planning and monitoring
- Enhanced transaction isolation guarantees (read-your-own-writes, snapshot reads)

#### Query Optimizer
- Cost-based join algorithm selection (hash join vs nested loop)
- Nested loop join physical operator for small tables and non-equality joins
- Query statistics collection and usage in planner decisions
- Real table statistics (row counts, column distinct values, selectivity)
- Statistics analyzer for collecting and managing table/column statistics
- Selectivity estimation for predicates (equality, range, NULL, AND, OR, NOT)
- Enhanced planner API with cost estimator support
- Verbose mode for debugging query plan decisions
- Explain plan support for all physical operators including nested loop join

#### PostgreSQL Protocol
- Prepared statement schema derivation from query AST
- Automatic type mapping from HeliosDB types to PostgreSQL OIDs
- Enhanced Parse message handling with schema extraction
- Result schema storage in prepared statements
- Improved Describe message responses with actual column types

#### Compression
- Comprehensive compression configuration API with fluent builder pattern
- Compression builder for multi-table configuration
- Compression API for statistics and management
- FSST dictionary persistence (export and restore symbol tables)
- Batch compression optimization with pre-allocation and chunking
- Adaptive compression with automatic cost-benefit analysis
- Intelligent sampling strategies (Stratified, BeginningWeighted, Diversity, Auto)
- Compression value count tracking in metadata
- Column-level compression statistics
- Enhanced FSST encoder with multiple compression modes

#### System Views
- `pg_stat_optimizer` view with real query statistics
- Enhanced `pg_stat_database` with transaction statistics
- Enhanced `pg_branches` with parent branch names
- Enhanced `pg_snapshots` with actual snapshot sizes
- `pg_compression_stats` view for compression monitoring

#### Vector Search
- Vector index length tracking for all metric types (Cosine, InnerProduct, L2)
- `len()` and `is_empty()` methods for cosine and inner-product HNSW indexes
- Accurate vector count reporting in multi-metric HNSW indexes

### Changed

#### Storage Engine
- Improved WAL replay with two-pass algorithm (transaction-aware)
- Enhanced transaction handling with snapshot manager integration
- Better error handling in WAL replay (resilient to partial failures)
- Shared storage engine for better concurrency

#### Query Optimizer
- Planner now uses real statistics instead of hardcoded estimates
- Improved join algorithm selection based on actual cardinality
- Cost estimation now considers table sizes and selectivity
- Physical planner defaults to optimal algorithms per query pattern
- Better query plan quality with accurate row count estimates

#### Compression
- FSST codec now supports persistent dictionaries
- Improved batch compression performance (10-15% faster at scale)
- Better compression quality with intelligent sampling (1-5% improvement)
- Compression statistics now include value count tracking
- Column compression metadata extended with detailed metrics

#### PostgreSQL Protocol
- Extended query protocol now returns proper result schemas
- Better client compatibility with automatic schema extraction
- Parse phase includes one-time schema derivation (~160μs overhead)

#### System Views
- All system views now return real data instead of placeholder values
- Improved accuracy across all statistics views
- Better monitoring and observability capabilities

### Fixed

#### Storage Layer
- Fixed MVCC transaction isolation incomplete implementation
- Fixed WAL replay that only counted entries without applying them
- Fixed table name extraction from storage keys (was always "unknown")
- Fixed snapshot isolation for concurrent transactions
- Fixed crash recovery

#### Query Optimizer
- Fixed build validation items in logical expression handling
- Fixed missing cost-based decision logic for join selection
- Fixed hardcoded selectivity estimates (now uses real statistics)

#### Compression
- Fixed missing compression value count in statistics
- Fixed FSST dictionary serialization (was returning empty vectors)
- Fixed compression metadata propagation

#### Vector Search
- Fixed vector index length always returning 0
- Fixed multi-metric HNSW index length calculation

#### System Views
- Fixed `pg_stat_optimizer` returning placeholder data
- Fixed `pg_stat_database` missing transaction statistics
- Fixed `pg_branches` parent lookup
- Fixed `pg_snapshots` size calculation
- Fixed EXPLAIN statistics usage

#### Protocol
- Fixed prepared statement schema derivation
- Fixed incomplete extended query protocol support

#### Type Safety
- Fixed type mismatches in various storage modules
- Improved error handling throughout codebase

### Performance

#### Query Optimizer
- 10-100x speedup for queries with small table joins (nested loop optimization)
- Better query plans with real statistics reduce execution time
- Cost-based selection prevents unnecessary hash table construction
- O(log N) version lookups in MVCC read path

#### Compression
- 10-15% faster batch compression with pre-allocation
- Better CPU cache utilization with 64-item chunks
- Dictionary serialization/deserialization in ~10-25 μs
- Compression ratios: 2.0-2.5x (emails), 1.8-2.2x (URLs), 1.5-2.0x (mixed strings)
- Adaptive compression prevents wasting resources on incompressible data

#### WAL & Recovery
- Two-pass WAL replay with progress reporting (every 1000 operations)
- Resilient error handling (continues on non-fatal errors)
- Idempotent replay (safe to replay multiple times)

#### Statistics
- Negligible overhead for join decision making (~1.6 μs)
- Efficient selectivity estimation with O(1) lookups
- Real-time statistics updates with atomic counters

### Deprecated

None

### Removed

None

### Security

- Enhanced data integrity with MVCC snapshot isolation
- Better crash recovery guarantees with complete WAL replay
- Improved audit trail with accurate table name tracking in WAL

### Migration Notes

Upgrading from v2.1.0 to v2.2.0:

1. **Statistics Collection**: The query optimizer now collects real statistics. No SQL step is needed; HeliosDB Lite has no standalone `ANALYZE` statement.

2. **Compression Configuration**: New compression API is opt-in. Existing compressed data remains compatible.

3. **WAL Format**: Enhanced WAL entries now include table names. Existing WAL files are forward-compatible.

4. **System Views**: All system views now return real data. Update any queries that relied on placeholder values.

5. **API Changes**: All changes are backward-compatible. New constructors added, existing code works unchanged.

---

## [2.1.0] - 2025-11-XX

### Added
- Physical query planner implementation
- Network protocol enhancements
- PostgreSQL wire protocol improvements

### Changed
- Improved type safety throughout codebase
- Enhanced system view accuracy

---

## [0.1.0-beta] - 2025-11-15

### First Beta Release

This is the first public beta release of HeliosDB Lite, a PostgreSQL-compatible embedded database with vector search, encryption, and multi-tenancy.


### Added

#### Core Database Features
- **Storage Layer**: RocksDB-based LSM-tree storage with MVCC snapshot isolation
- **SQL Support**: PostgreSQL-compatible SQL engine (95%+ compatibility)
  - Full CRUD operations (CREATE TABLE, INSERT, UPDATE, DELETE, SELECT, DROP TABLE)
  - WHERE clauses with complex expressions
  - Aggregations (COUNT, SUM, AVG, MIN, MAX, COUNT(DISTINCT))
  - GROUP BY, HAVING, ORDER BY, LIMIT, OFFSET, DISTINCT
  - INNER JOIN support (hash join in progress)
  - Transactions with ACID guarantees
  - Snapshot isolation for consistent reads
- **Data Types**: Complete PostgreSQL type system
  - Numeric: INT2, INT4, INT8, FLOAT4, FLOAT8, NUMERIC, DECIMAL
  - String: TEXT, VARCHAR, CHAR
  - Binary: BYTEA
  - Boolean: BOOL
  - UUID: UUID
  - JSON: JSON, JSONB (bincode-compatible)
  - Array: Array types
  - Vector: VECTOR(n) for embeddings

#### Security Features
- **Transparent Data Encryption (TDE)**: AES-256-GCM encryption
  - <3% performance overhead with AES-NI
  - Argon2 key derivation (NIST recommended)
  - Per-table encryption support
  - Secure key management (environment, file, in-memory)
  - Automatic nonce generation
  - Memory zeroization for sensitive data
- **Audit Logging**: Comprehensive audit system
  - Authentication events
  - Data access tracking
  - Schema changes
  - Security events
  - Structured JSON logs
  - Query logging with metadata

#### Advanced Features
- **Vector Search**: Built-in HNSW index for AI/ML embeddings
  - Support for 1536-dimension vectors (OpenAI ada-002 compatible)
  - Distance metrics: Cosine, L2 (Euclidean), Inner Product
  - Fast k-NN search (<50ms @ 1M vectors target)
  - Configurable index parameters (M, ef_construction)
  - Serialization support
- **JSONB Support**: Native JSON binary format
  - Bincode-compatible storage
  - Efficient binary encoding
  - Full JSONB querying (operators in progress)
- **Interactive REPL**: PostgreSQL-style interactive shell
  - Tab completion for tables, columns, keywords
  - Meta-commands (\d, \dt, \?, \q)
  - Pretty table output
  - Multi-line query support
  - Persistent command history

#### Network & Protocol
- **PostgreSQL Wire Protocol**: 85% compatible
  - TCP server with async Tokio
  - SCRAM-SHA-256 authentication
  - Message handling infrastructure
  - Session management
  - Transaction status tracking
  - (Message encoding/decoding completion in progress)
- **Server Mode**: PostgreSQL-compatible network server
- **Embedded Mode**: SQLite-style in-process usage
- **In-Memory Mode**: ACID-compliant RAM-only storage

#### Developer Experience
- **Comprehensive Documentation**
  - Rustdoc API documentation
  - Complete GETTING_STARTED.md guide
  - 8 working examples
  - Quick reference guides
  - Troubleshooting documentation

### Performance

- **Binary Size**: Optimized for embedded usage
- **Memory Footprint**: Efficient with configurable cache
- **Encryption Overhead**: <3% with AES-NI

### Dependencies

#### Core Storage
- RocksDB 0.22 (LSM-tree storage)
- Apache Arrow 53 (columnar format)
- Apache Parquet 53 (columnar compression)

#### SQL Processing
- sqlparser-rs 0.53 (PostgreSQL dialect)

#### Cryptography
- AES-GCM 0.10 (encryption)
- Argon2 0.5 (key derivation)
- Ring 0.17 (cryptographic primitives)
- RustLS 0.23 (TLS 1.3)

#### Vector Search
- hnsw_rs 0.3 (HNSW index)

#### Async Runtime
- Tokio 1.40 (async runtime)

### Breaking Changes

None (initial release)

### Security

- Transparent Data Encryption (TDE) with AES-256-GCM
- SCRAM-SHA-256 authentication
- Secure key derivation with Argon2
- Memory zeroization for sensitive data
- Comprehensive audit logging

### Known Limitations (Beta)

1. **PostgreSQL Protocol**: Message encoding 85% complete (works with basic clients)
2. **Hash Joins**: Partial implementation (INNER JOIN works, optimizations pending)
3. **JSONB Operators**: Storage works, query operators in development
4. **Multi-Tenancy**: Foundation exists, RLS not yet implemented
5. **Full-Text Search**: Planned for a future release
6. **Replication**: Single-node only (clustering planned for full HeliosDB)

### Migration Path

HeliosDB Lite provides a clear upgrade path to full HeliosDB:
- Data format compatible
- SQL compatibility maintained
- CDC-based migration
- Zero-downtime tenant migration

### System Requirements

- **Rust**: 1.75+ (for building from source)
- **Operating Systems**: Linux, macOS, Windows
- **Architecture**: x86_64, ARM64
- **Memory**: Minimum 512MB, recommended 2GB+
- **Disk**: Varies by data size (uses compression)

### Contributors

This release was made possible by the HeliosDB team and community contributors.

### Links

- **Documentation**: [HeliosDB Lite documentation](/docs/lite/)
- **Product page**: https://www.heliosdb.com/lite.html
- **Issues**: support@heliosdb.com
- **Community**: [Discord](https://discord.gg/yTykuUrFXc)

---

## Upgrade Guide

### From Source Build to v0.1.0-beta

HeliosDB Lite is distributed to licensed customers as a source distribution that builds the
`heliosdb-lite` binary; see [Installation](/docs/lite/getting-started/installation/).

If you were using a source build:

1. Stop every process that has the data directory open, then back up the directory.

2. Build the new release in the unpacked source distribution:
   ```bash
   cargo build --release
   ```

3. Data format is compatible (no migration needed): open the existing data directory with the
   new `heliosdb-lite` binary.

Applications that embed HeliosDB Lite in-process rebuild against the new release (embedded
API, licensed source distribution).

### API Stability

**Beta Stability Guarantees**:
- Core API is stable
- SQL syntax is stable (PostgreSQL compatible)
- Data format is forward-compatible
- Configuration format is stable

**May Change Before v1.0**:
- Internal interfaces of the embedded API (licensed source distribution)
- Experimental features (documented as such)
- Performance tuning parameters