Migration Guide: v3.0.0 to v3.1.0

Published: 2025-12-08 Target Audience: Developers, System Administrators, DevOps Engineers

Table of Contents

1. Overview
2. What’s New in v3.1.0
3. Breaking Changes
4. New Features Overview
5. CLI Flag Changes
6. Code Migration Examples
7. Single-User to Multi-User Transition
8. Backward Compatibility Notes
9. Troubleshooting Migration Issues
10. FAQ

Overview

HeliosDB-Lite v3.1.0 adds multi-user ACID transaction support for in-memory mode with user-triggered persistence. The release is backward compatible with v3.0.0 and adds capabilities for multi-user applications.

Migration Complexity: Low Estimated Migration Time: 30 minutes - 2 hours Breaking Changes: None Recommended Upgrade Path: Direct upgrade from v3.0.0

What’s New in v3.1.0

Major Features

1. Multi-User Session Management

   • Concurrent user sessions with independent transaction contexts
   • Per-user resource quotas and connection limits
   • Session timeout and cleanup mechanisms
   • Full PostgreSQL-compatible session management

2. Enhanced Locking and Concurrency

   • Advanced lock manager with deadlock detection
   • Three lock modes: Shared, Exclusive, Update
   • Configurable lock timeouts
   • Wait-for graph based deadlock resolution

3. Configurable Isolation Levels

   • READ COMMITTED (default)
   • REPEATABLE READ
   • SERIALIZABLE with conflict detection

4. User-Triggered Persistence

   • Manual dump/restore commands
   • Incremental dump support (append mode)
   • Automated backup scheduling
   • Dirty state tracking

5. Resource Management

   • Per-user memory quotas
   • Transaction and query timeouts
   • System-wide resource limits
   • Connection pooling support

Enhancements

• Improved CLI with new dump and restore commands
• Enhanced configuration with 4 new sections
• Dirty state warnings on shutdown
• API documentation
• Performance optimizations for high-concurrency workloads

Breaking Changes

None. HeliosDB-Lite v3.1.0 is backward compatible with v3.0.0.

Existing applications will continue to work without any code changes.

New Features Overview

1. Session Management

Before (v3.0.0): a single in-process instance with no session concept.

After (v3.1.0): each PostgreSQL wire connection runs in its own session with an independent transaction context, so concurrent clients no longer share one transaction:

-- Connection A
BEGIN;
INSERT INTO users VALUES (1, 'Alice');
COMMIT;

-- Connection B, at the same time, in its own transaction
BEGIN;
SELECT count(*) FROM users;
COMMIT;

Migration Note: Sessions are created per connection. Existing clients work without changes.

2. Dump/Restore Commands

New CLI Commands:

# Dump in-memory database to file
heliosdb-lite dump --output backup.heliodump

# Restore from dump file
heliosdb-lite restore --input backup.heliodump

# Incremental backup (append mode)
heliosdb-lite dump --output backup.heliodump --append

3. Isolation Levels

Before (v3.0.0):

• Default MVCC snapshot isolation (similar to REPEATABLE READ)

After (v3.1.0):

-- Set isolation level per transaction
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE;
-- or
BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED;
-- or
BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;

4. Resource Quotas

New Configuration:

[resource_quotas]
memory_limit_per_user_mb = 1024
max_concurrent_queries = 100
query_timeout_secs = 60

CLI Flag Changes

Start Command

New Flags (v3.1.0):

# Current releases refuse to start without a REST token-signing secret
export HELIOSDB_JWT_SECRET="$(openssl rand -hex 32)"

# Before (v3.0.0): --memory flag implied embedded mode
heliosdb-lite start --memory --port 5432

# After (v3.1.0): Same command, now supports multi-user sessions
heliosdb-lite start --memory --port 5432

# New option: Explicitly require --data-dir for persistent mode
heliosdb-lite start --data-dir /var/lib/heliosdb --port 5432

Migration: No changes required. Existing commands work identically.

REPL Command

Before (v3.0.0):

heliosdb-lite repl --memory

After (v3.1.0):

# Same command, now with session support
heliosdb-lite repl --memory

# New: Execute single query and exit
heliosdb-lite repl --memory --execute "SELECT * FROM users"

Migration: No changes required.

New Commands

Dump Command

heliosdb-lite dump [OPTIONS]

Options:
  --output <PATH>           Output file path (required)
  --compress <TYPE>         Compression: none, lz4, zstd (default: zstd)
  --compression-level <N>   Compression level (default: 3)
  --tables <TABLES>         Comma-separated table list
  --append                  Incremental dump mode
  --verbose                 Verbose output

Restore Command

heliosdb-lite restore [OPTIONS]

Options:
  --input <PATH>            Input file path (required)
  --mode <MODE>             Restore mode: clean, append (default: clean)
  --tables <TABLES>         Comma-separated table list
  --on-conflict <ACTION>    Conflict resolution: error, skip, update
  --verbose                 Verbose output

health command

heliosdb-lite status [OPTIONS]

Options:
  --format <FORMAT>         Output format: text, json (default: text)

Code Migration Examples

Example 1: Basic Application (No Changes Required)

SQL that ran on v3.0.0 runs unchanged on v3.1.0, from the REPL or over the PostgreSQL wire:

CREATE TABLE users (id INT PRIMARY KEY, name TEXT);
INSERT INTO users VALUES (1, 'Alice');
SELECT * FROM users;

Example 2: Adding a Scheduled Backup

# Daily dump at 02:00
0 2 * * * heliosdb-lite dump --output /backups/daily-$(date +\%Y\%m\%d).heliodump

Example 3: Isolation Level Configuration

-- Per session
SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL SERIALIZABLE;

-- Or per transaction
BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED;

Single-User to Multi-User Transition

Step 1: Assess Current Usage

Questions to ask:

1. Do you have concurrent database access?
2. Do you need per-user resource limits?
3. Do you require different isolation levels for different operations?
4. Do you need session-based auditing?

If NO to all: Continue using v3.1.0 as you used v3.0.0 If YES to any: Give each user its own PostgreSQL wire connection and role, and set the isolation level per session or per transaction

Step 2: Configure Limits (If Needed)

[resource_quotas]
memory_limit_per_user_mb = 1024
max_concurrent_queries = 100
query_timeout_secs = 300

Step 3: Keep Transactions in SQL

BEGIN;
INSERT INTO users VALUES (1, 'Alice');
COMMIT;

Step 4: Add Backup Strategy

# Setup cron job for daily backups
0 2 * * * heliosdb-lite dump --output /backups/daily-$(date +\%Y\%m\%d).heliodump

Backward Compatibility Notes

SQL Compatibility

All existing SQL commands work unchanged.

Configuration Compatibility

v3.0.0 config.toml works unchanged in v3.1.0

Optional new sections:

[session]  # Optional
[locks]    # Optional
[dump]     # Optional
[resource_quotas]  # Optional

Database File Compatibility

• In-memory mode: No files, always compatible
• Persistent mode: RocksDB format unchanged
• Dump format: New .heliodump format (v3.1.0+)

Wire Protocol Compatibility

• PostgreSQL wire protocol: Unchanged
• HTTP REST API: Unchanged
• All existing clients continue to work

Troubleshooting Migration Issues

Issue 1: “Session not found” Errors

Symptom:

Error: SessionNotFound(123)

Cause: The connection’s session was closed or timed out

Solution: Open a new connection; each PostgreSQL wire connection creates its own session.

Issue 2: Lock Timeout Errors

Symptom:

Error: Lock timeout exceeded

Cause: High lock contention with default 30-second timeout

Solution:

[locks]
timeout_ms = 60000  # Increase to 60 seconds

Issue 3: Serialization Failures

Symptom:

Error: SerializationFailure("Read-write conflict")

Cause: Using SERIALIZABLE isolation with concurrent updates

Solution: Retry the whole transaction in the client when it fails with SQLSTATE 40001 (serialization failure), up to a small number of attempts.

Issue 4: Out of Memory

Symptom:

Error: memory limit exceeded for user

Cause: Per-user memory quota exceeded

Solution:

[resource_quotas]
memory_limit_per_user_mb = 2048  # Increase to 2 GB

Issue 5: Dump File Not Found

Symptom:

Error: No such file or directory: /backups/db.heliodump

Cause: Dump directory doesn’t exist

Solution:

mkdir -p /backups
heliosdb-lite dump --output /backups/db.heliodump

FAQ

Q1: Do I need to change my code to upgrade to v3.1.0?

A: No. v3.1.0 is 100% backward compatible. Existing code works without changes.

Q2: Can I use v3.1.0 without session management?

A: Yes. Session management is optional. Use it only if you need multi-user features.

Q3: How do I migrate from v3.0.0 to v3.1.0?

A:

1. Download v3.1.0 binary
2. Replace v3.0.0 binary
3. No code changes required
4. Optionally adopt new features (sessions, dump/restore)

Q4: Are dump files from v3.1.0 compatible with future versions?

A: Yes. Dump format is version-stamped and designed for forward compatibility.

Q5: Can I downgrade from v3.1.0 to v3.0.0?

A: Yes, but:

• Dump files (.heliodump) won’t work in v3.0.0
• Configuration sections [session], [locks], [dump], [resource_quotas] will be ignored

Q6: Does v3.1.0 require more memory than v3.0.0?

A:

• Single-user mode: Same memory usage
• Multi-user mode: +2-4 KB per session, +32 bytes per lock
• Overall: Minimal increase (<5%) for typical workloads

Q7: How do I enable automated backups?

A:

[dump]
schedule = "0 2 * * *"  # Daily at 2 AM
dump_dir = "/var/backups/heliosdb"

Or via cron:

0 2 * * * heliosdb-lite dump --output /backups/daily.heliodump

Q8: Can I use v3.1.0 features with persistent mode?

A: Partial. Session management works in both modes. Dump/restore is primarily for in-memory mode (persistent mode already has WAL/RocksDB backups).

Q9: What isolation level should I use?

A:

• READ COMMITTED: Default, best for most web applications
• REPEATABLE READ: For reports requiring consistency
• SERIALIZABLE: For financial transactions requiring strict consistency

Q10: How do I test my migration?

A:

1. Deploy v3.1.0 to staging environment
2. Run existing test suite (should pass 100%)
3. Test new features (dump/restore, sessions) if adopting
4. Monitor for errors in logs
5. Roll out to production gradually

---

Migration Checklist

• Review what’s new in v3.1.0
• Verify no breaking changes affect your application
• Download v3.1.0 binary or update dependency
• Test in development environment
• Update configuration (optional new sections)
• Adopt new features (sessions, dump/restore) if needed
• Update documentation and runbooks
• Setup automated backups (if using in-memory mode)
• Deploy to staging and verify
• Deploy to production
• Monitor for issues

---

Support

Issues: support@heliosdb.com Community: Discord Documentation: https://www.heliosdb.com/docs/lite/

---

Version: 3.1.0