Skip to content
HeliosDB HeliosDB Lite

HeliosDB-Lite v2.4.0 Release Notes

Editor’s note (2026-10-15): the Product Quantization figures in these release notes (compression ratios and recall) are as announced for this release; no test behind them has been published. The current published measurement is about 16x less resident vector-index RAM at recall-safe defaults (recall@10 0.987 vs 0.989; internal tests, Nano 3.32.1 vs an unquantized index). No Lite-specific measurement has been published. The current release is Lite 3.6.0.

Overview

HeliosDB-Lite v2.4.0-beta represents a major milestone in the evolution of our embedded database, implementing features including advanced vector compression, enterprise SQL capabilities, and comprehensive performance optimizations.

Key Achievements

  • Product Quantization: 8-16x vector compression with 95-98% accuracy maintained
  • Database Branching: Full CREATE/DROP/MERGE BRANCH SQL support
  • Time-Travel Queries: AS OF TIMESTAMP for historical data access
  • Materialized Views: Incremental refresh with CPU-aware scheduling
  • Advanced Compression: FSST string and ALP numeric compression
  • System Views: PostgreSQL-compatible pg_* views

Major Features

1. Vector Search Enhancements

Product Quantization (PQ)

  • Compression Ratio: 8-16x reduction in memory usage
  • Accuracy: 95-98% recall maintained
  • Performance: 5-10x faster search on large datasets
  • Implementation: K-means++ training with asymmetric distance computation

Quantized HNSW Index

  • Integration with existing vector search
  • Automatic memory optimization
  • Configurable compression levels
  • Real-time compression statistics

2. Database Branching

Complete SQL support for database branching:

-- Create development branch
CREATE DATABASE BRANCH dev FROM main AS OF NOW;


-- Work in isolated branch
USE DATABASE BRANCH dev;


-- Merge changes back
MERGE DATABASE BRANCH dev INTO main;


-- Clean up
DROP DATABASE BRANCH dev;

Features:

  • Zero-copy branching via RocksDB snapshots
  • Automatic conflict detection
  • Three-way merge support
  • Branch metadata tracking

3. Time-Travel Queries

Query historical data at any point in time:

-- Query data as it was 1 hour ago
SELECT * FROM users AS OF TIMESTAMP '2025-11-25 08:00:00';


-- Compare current vs historical
SELECT * FROM inventory AS OF TIMESTAMP NOW() - INTERVAL '1 DAY';


-- Use relative timestamps
SELECT * FROM orders AS OF TIMESTAMP NOW() - INTERVAL '7 DAYS';

Capabilities:

  • 7-day retention by default (configurable)
  • Automatic garbage collection
  • Minimal storage overhead
  • Sub-second time travel

4. Materialized Views with Auto-Refresh

Intelligent materialized view management:

-- Create incrementally refreshing view
CREATE MATERIALIZED VIEW sales_summary AS
SELECT date, SUM(amount) as total
FROM sales
GROUP BY date
WITH (
    refresh_interval = '1 HOUR',
    incremental = true
);


-- Manual refresh
REFRESH MATERIALIZED VIEW sales_summary;

Features:

  • CPU-aware scheduling
  • Incremental refresh support
  • Delta change tracking
  • Automatic dependency resolution

5. Advanced Compression

FSST (Fast Static Symbol Table)

  • 2-4x compression for string data
  • 500+ MB/sec compression speed
  • 2+ GB/sec decompression speed
  • Automatic dictionary training

ALP (Adaptive Lossless floating-Point)

  • 30-50% compression for numeric data
  • Preserves full precision
  • SIMD-accelerated operations
  • Automatic pattern detection

Performance Improvements

Benchmark Results

Measured on representative fixtures; reproduce on your own hardware.

Operationv2.3.1v2.4.0Improvement
Vector Insert50K/sec125K/sec2.5x
Vector Search (1M vectors)2.5ms0.5ms5x
Compression RatioN/A8-16xNew
Memory Usage (1M vectors)3GB375MB8x
Time-Travel QueryN/A<100msNew
Branch CreationN/A<10msNew

Breaking Changes

API Changes

  1. The embedded API’s main database type was renamed for clarity
  2. Vector index configuration now requires explicit PQ settings
  3. Some internal modules reorganized (shouldn’t affect most users)

Disabled by default

  1. The sync protocol (HTTP sync server and client) is no longer built in by default. It is opt-in at build time with --features sync-experimental and is not recommended for this release; branching covers the replication use cases.

JWT authentication is part of HeliosDB Lite. The REST API that heliosdb-lite start serves on --http-port (default 8080) requires Authorization: Bearer <JWT> on every /v1 and /api/v1 route, signed with the HELIOSDB_JWT_SECRET the server is started with; the server refuses to start without it. See the JWT Auth Quick Reference.

Known Issues

Beta Limitations

  1. Vector Quantization: Some edge cases in high-dimensional spaces (>1024 dims)
  2. Compression: FSST may not optimize for very short strings (<10 chars)
  3. Branching: Cannot merge branches with schema conflicts
  4. Time-Travel: Performance degrades with >1M historical versions

Migration Guide

From v2.3.1

  1. Back up and install: stop every process that has the data directory open, copy the directory, and build v2.4.0 from the licensed source distribution (cargo build --release). Open the existing directory with the new binary.

  2. Embedded applications: embedded API change (licensed source distribution).

  3. Enable new features: branching and time-travel need no setting. Time-travel versioning is on by default; turn it off in the configuration file only if you do not query history and want to reduce write overhead:

    [storage]
    time_travel_enabled = true   # the default; false stops versioning rows
    CREATE DATABASE BRANCH dev FROM main AS OF NOW;
    SELECT * FROM orders AS OF TIMESTAMP '2025-01-15 10:30:00';

  4. Update vector indices: Product Quantization is set per index when the index is created. Current builds do not support ALTER INDEX ... SET (...), so rebuild an existing index to compress it:

    DROP INDEX vec_idx;
    CREATE INDEX vec_idx ON documents USING hnsw (embedding vector_l2_ops)
    WITH (quantization = 'product', pq_subquantizers = 8, pq_centroids = 256);

Documentation

Updated Guides

Acknowledgments

This release represents months of intensive development and wouldn’t have been possible without the dedication of our team and the valuable feedback from our community.

Special thanks to:

  • The DuckDB team for FSST/ALP compression algorithms
  • The Faiss team for product quantization insights
  • Our beta testers for invaluable feedback

Support

Resources

License

HeliosDB-Lite is source-available under the Server Side Public License (SSPL-1.0) and is available to licensed customers.

HeliosDB-Lite v2.4.0-beta - Built for the future of embedded databases