Migration Guide: v2.3.1 → v2.4.0-beta

Migration Guide: v2.3.1 → v2.4.0-beta

Source Version: v2.3.1 Target Version: v2.4.0-beta Migration Complexity: Low to Medium Estimated Time: 30 minutes to 2 hours Rollback Difficulty: Easy

Table of Contents

1. Overview
2. Pre-Migration Checklist
3. Breaking Changes
4. Step-by-Step Migration
5. Code Changes
6. Configuration Changes
7. Testing Migration
8. Rollback Procedure
9. Troubleshooting
10. FAQ

Overview

What’s Changing

v2.4.0-beta introduces critical type safety corrections and feature improvements:

• Type Safety: Fixed priority queue ordering, lifetime annotations
• API Changes: DeltaType → DeltaOperation (deprecated, not removed)
• Disabled Features: Sync protocol moved to experimental flag
• Bug Fixes: 27 bug fixes from v2.3.1
• Improvements: Better error messages, cleaner warnings

Compatibility

• Backward Compatible: Mostly yes
• Data Format: No changes (databases from v2.3.1 work in v2.4.0-beta)
• API: Deprecated APIs still work with warnings
• Breaking: Only sync protocol (now optional feature)

Should You Migrate?

✅ Migrate If:

• You want Phase 3 stability improvements
• You need bug fixes from v2.3.1
• You’re using branching or materialized views
• You’re in development/testing phase

❌ Wait If:

• You’re in production with v2.3.1 and it’s stable
• You rely heavily on sync/replication protocol
• You need 100% test pass rate
• You can’t tolerate beta software

Pre-Migration Checklist

1. Backup Your Data

# Create full backup
cp -r ./data.helio ./data.helio.backup-$(date +%Y%m%d)

# Or use branching (if on v2.3.1 with branching support)
# psql or heliosdb-nano CLI:
CREATE BRANCH migration_backup;

2. Document Your Configuration

# Save current configuration
cp heliosdb.toml heliosdb.toml.v2.3.1

# Document custom settings
cat heliosdb.toml | grep -v '^#' | grep -v '^$' > custom_config.txt

3. Run Full Test Suite (v2.3.1)

# Ensure v2.3.1 is healthy
cargo test --lib

# Document baseline
cargo test --lib 2>&1 | grep "test result:" > v2.3.1_test_results.txt

4. Check for Sync Protocol Usage

# Search for sync protocol usage
grep -r "replication\|sync_protocol" ./src

# Check config
grep "sync\|replication" heliosdb.toml

5. Verify Rust Version

# Check Rust version
rustc --version

# Should be 1.75 or higher
# If not, update:
rustup update stable

6. Review Breaking Changes

Read the Breaking Changes section carefully.

Breaking Changes

1. Sync Protocol Disabled by Default

Impact: High (if using sync protocol) Effort: Medium (requires code changes)

Old Behavior (v2.3.1)

# Sync protocol enabled by default
[dependencies]
heliosdb-nano = "2.3.1"

// Sync protocol available
db.start_replication_server()?;
db.sync_to_replica("replica-host:5433")?;

New Behavior (v2.4.0-beta)

# Must explicitly enable sync protocol
[dependencies]
heliosdb-nano = { version = "2.4.0-beta", features = ["sync-experimental"] }

// Only available with feature flag
#[cfg(feature = "sync-experimental")]
db.start_replication_server()?;

Migration Steps

Option 1: Enable experimental feature (not recommended)

[dependencies]
heliosdb-nano = { version = "2.4.0-beta", features = ["sync-experimental"] }

Option 2: Use branching for backup/restore (recommended)

-- Create periodic backups
CREATE BRANCH backup_$(date +%Y%m%d);

-- Restore from backup
MERGE BRANCH backup_20251124 INTO main;

Option 3: Stay on v2.3.1 until sync protocol stabilizes (v2.5.0)

2. Priority Queue Ordering Changed

Impact: Medium (if using MV scheduler programmatically) Effort: Low (change priority values)

Old Behavior (v2.3.1)

Lower priority number = higher priority (min-heap)

// v2.3.1: Priority 1 = highest, 100 = lowest
scheduler.schedule_refresh("important_mv", 1);
scheduler.schedule_refresh("less_important_mv", 50);

New Behavior (v2.4.0-beta)

Higher priority number = higher priority (max-heap)

// v2.4.0-beta: Priority 100 = highest, 1 = lowest
scheduler.schedule_refresh("important_mv", 100);
scheduler.schedule_refresh("less_important_mv", 50);

Migration Steps

1. Audit priority assignments:

   grep -r "schedule_refresh\|set_priority" ./src

2. Invert priority values:

   // Old
   let priority = 10;
   
   // New
   let priority = 100 - 10; // = 90

3. Update configuration:

   # Old (v2.3.1)
   [materialized_views]
   default_priority = 10
   
   # New (v2.4.0-beta)
   [materialized_views]
   default_priority = 90

3. DeltaType Deprecated (Not Removed)

Impact: Low (warnings only) Effort: Low (optional refactoring)

Old API (v2.3.1)

use heliosdb_nano::storage::DeltaType;

let delta_type = DeltaType::Insert;
match delta_type {
    DeltaType::Insert => { /* ... */ }
    DeltaType::Update => { /* ... */ }
    DeltaType::Delete => { /* ... */ }
}

New API (v2.4.0-beta)

use heliosdb_nano::storage::DeltaOperation;

let operation = DeltaOperation::Insert;
match operation {
    DeltaOperation::Insert => { /* ... */ }
    DeltaOperation::Update => { /* ... */ }
    DeltaOperation::Delete => { /* ... */ }
}

Migration Steps

Option 1: Update to new API (recommended)

# Find all usages
rg "DeltaType" --type rust

# Replace systematically
# DeltaType → DeltaOperation
# delta_type → operation

Option 2: Accept deprecation warnings (works in v2.4.0-beta)

// Old code still works, just shows warnings
#[allow(deprecated)]
use heliosdb_nano::storage::DeltaType;

Step-by-Step Migration

Step 1: Update Dependencies

Cargo.toml

[dependencies]
heliosdb-nano = "2.3.1"
heliosdb-nano = "2.4.0-beta"

# If using sync protocol (not recommended in beta):
# heliosdb-nano = { version = "2.4.0-beta", features = ["sync-experimental"] }

Step 2: Update Imports

# Run update script
cargo build 2>&1 | grep "warning: unused import" | wc -l

Most unused imports are harmless warnings. Optionally clean them up:

# Use cargo fix for automatic cleanup
cargo fix --lib --allow-dirty

Step 3: Fix Sync Protocol Usage

If you were using sync protocol, either:

Option A: Enable experimental feature

[dependencies]
heliosdb-nano = { version = "2.4.0-beta", features = ["sync-experimental"] }

Option B: Remove sync code and use branching

db.start_replication_server()?;
db.sync_to_replica("host:port")?;

// Create backup branches instead
db.execute("CREATE BRANCH backup_$(date +%Y%m%d)")?;

Step 4: Fix Priority Values

If using MV scheduler programmatically:

scheduler.schedule_refresh("mv1", 10);  // High priority
scheduler.schedule_refresh("mv2", 90);  // Low priority
scheduler.schedule_refresh("mv1", 90);  // High priority
scheduler.schedule_refresh("mv2", 10);  // Low priority

Step 5: Update DeltaType Usage (Optional)

use heliosdb_nano::storage::DeltaType;
use heliosdb_nano::storage::DeltaOperation;

let delta_type = DeltaType::Insert;
let operation = DeltaOperation::Insert;

Step 6: Build and Test

# Clean build
cargo clean
cargo build --lib

# Run tests
cargo test --lib

# Check for warnings
cargo clippy --lib

Step 7: Update Configuration

heliosdb.toml

[materialized_views]
# Update priority if needed (higher = more important)
default_priority = 90  # Was 10 in v2.3.1

[features]
# Sync protocol now requires explicit opt-in
# sync_enabled = true  # Remove or comment out

Step 8: Verify Core Functionality

# Start REPL
cargo run --bin heliosdb-nano

# Test basic operations
CREATE TABLE test (id INT, name TEXT);
INSERT INTO test VALUES (1, 'test');
SELECT * FROM test;

# Test Phase 3 features
CREATE BRANCH test_branch;
CREATE MATERIALIZED VIEW test_mv AS SELECT COUNT(*) FROM test;
SELECT * FROM pg_database_branches;

# Cleanup
DROP MATERIALIZED VIEW test_mv;
DROP BRANCH test_branch;
DROP TABLE test;

Step 9: Run Full Test Suite

# Run all tests
cargo test --lib

# Expected results:
# - 527 passed
# - 27 failed (known issues, see KNOWN_ISSUES-v2.4.0-beta.md)
# - Pass rate: 95.1%

# Save results
cargo test --lib 2>&1 | grep "test result:" > v2.4.0_test_results.txt

Step 10: Validate Production Scenarios

Test your specific use cases:

// Example: Your production code
#[cfg(test)]
mod migration_tests {
    use super::*;

    #[test]
    fn test_production_workflow() {
        let db = EmbeddedDatabase::new("./test_migration.helio").unwrap();

        // Your actual production queries
        db.execute("CREATE TABLE ...").unwrap();
        db.execute("INSERT INTO ...").unwrap();

        // Verify results
        let rows = db.query("SELECT ...").unwrap();
        assert_eq!(rows.len(), expected_count);
    }
}

Code Changes

Pattern 1: Sync Protocol Removal

Before (v2.3.1):

use heliosdb_nano::{EmbeddedDatabase, Config};

let mut db = EmbeddedDatabase::new("./data.helio")?;

// Start replication
db.start_replication_server()?;
db.configure_replica("replica1", "host:port")?;
db.sync_to_replica("replica1")?;

After (v2.4.0-beta):

use heliosdb_nano::{EmbeddedDatabase, Config};

let mut db = EmbeddedDatabase::new("./data.helio")?;

// Use branching for backups instead
db.execute("CREATE BRANCH backup_branch")?;

// Or enable experimental sync (not recommended)
#[cfg(feature = "sync-experimental")]
{
    db.start_replication_server()?;
    db.configure_replica("replica1", "host:port")?;
}

Pattern 2: Priority Inversion

Before (v2.3.1):

use heliosdb_nano::storage::MVScheduler;

let scheduler = MVScheduler::new(...);

// Lower = higher priority
scheduler.schedule_refresh("critical_mv", 1)?;
scheduler.schedule_refresh("normal_mv", 50)?;
scheduler.schedule_refresh("low_priority_mv", 99)?;

After (v2.4.0-beta):

use heliosdb_nano::storage::MVScheduler;

let scheduler = MVScheduler::new(...);

// Higher = higher priority
scheduler.schedule_refresh("critical_mv", 99)?;
scheduler.schedule_refresh("normal_mv", 50)?;
scheduler.schedule_refresh("low_priority_mv", 1)?;

Pattern 3: DeltaType Migration

Before (v2.3.1):

use heliosdb_nano::storage::{DeltaType, Delta};

fn process_delta(delta: &Delta) {
    match delta.delta_type {
        DeltaType::Insert => {
            let tuple = delta.new_tuple.as_ref().unwrap();
            // Process insert
        }
        DeltaType::Update => {
            let old = delta.old_tuple.as_ref().unwrap();
            let new = delta.new_tuple.as_ref().unwrap();
            // Process update
        }
        DeltaType::Delete => {
            let tuple = delta.old_tuple.as_ref().unwrap();
            // Process delete
        }
    }
}

After (v2.4.0-beta):

use heliosdb_nano::storage::{DeltaOperation, Delta};

fn process_delta(delta: &Delta) {
    match delta.operation {
        DeltaOperation::Insert => {
            let tuple = delta.new_tuple.as_ref().unwrap();
            // Process insert
        }
        DeltaOperation::Update => {
            let old = delta.old_tuple.as_ref().unwrap();
            let new = delta.new_tuple.as_ref().unwrap();
            // Process update
        }
        DeltaOperation::Delete => {
            let tuple = delta.old_tuple.as_ref().unwrap();
            // Process delete
        }
    }
}

Configuration Changes

heliosdb.toml

[database]
path = "./data.helio"
version = "2.3.1"
version = "2.4.0-beta"

[materialized_views]
enabled = true
auto_refresh = true
cpu_threshold = 70.0
default_priority = 10
default_priority = 90  # Inverted: higher = more important

[replication]
enabled = true
listen_addr = "0.0.0.0:5433"
replicas = ["replica1:5433", "replica2:5433"]
# Replication disabled in beta
# Use branching for backups:
# CREATE BRANCH backup_$(date +%Y%m%d)

[features]
encryption = true
vector_search = true
sync = true
# sync = false  # Disabled by default

Testing Migration

Automated Tests

Create migration test suite:

tests/migration_v2_3_1_to_v2_4_0.rs

#[cfg(test)]
mod migration_tests {
    use heliosdb_nano::*;

    #[test]
    fn test_data_compatibility() {
        // Open v2.3.1 database with v2.4.0-beta
        let db = EmbeddedDatabase::new("./v2.3.1_data.helio").unwrap();

        // Verify data is readable
        let rows = db.query("SELECT * FROM users", &[]).unwrap();
        assert!(!rows.is_empty());
    }

    #[test]
    fn test_priority_behavior() {
        let db = EmbeddedDatabase::new("./test.helio").unwrap();

        db.execute("CREATE TABLE t (id INT)").unwrap();
        db.execute("CREATE MATERIALIZED VIEW mv1 AS SELECT COUNT(*) FROM t").unwrap();

        // High priority should refresh first
        // (implementation depends on your MV scheduler access)
    }

    #[test]
    fn test_branching_features() {
        let db = EmbeddedDatabase::new("./test.helio").unwrap();

        db.execute("CREATE TABLE t (id INT)").unwrap();
        db.execute("CREATE BRANCH test_branch").unwrap();

        let rows = db.query("SELECT * FROM pg_database_branches", &[]).unwrap();
        assert_eq!(rows.len(), 2); // main + test_branch
    }
}

Manual Testing Checklist

• Database opens successfully
• Existing tables are readable
• INSERT/UPDATE/DELETE work
• Queries return correct results
• Transactions work (BEGIN/COMMIT/ROLLBACK)
• Branching works (CREATE/DROP/MERGE BRANCH)
• Materialized views work (CREATE/REFRESH/DROP)
• Time-travel queries work (AS OF)
• System views return data
• REPL commands work
• Backups/restores work
• Performance is acceptable

Rollback Procedure

If migration fails or issues are found:

Step 1: Stop Database

# Stop heliosdb-nano processes
pkill heliosdb-nano

Step 2: Restore Backup

# Remove v2.4.0-beta data
mv ./data.helio ./data.helio.v2.4.0-failed

# Restore v2.3.1 backup
cp -r ./data.helio.backup-YYYYMMDD ./data.helio

Step 3: Revert Code

[dependencies]
heliosdb-nano = "2.4.0-beta"
heliosdb-nano = "2.3.1"

cargo clean
cargo build --lib

Step 4: Restore Configuration

cp heliosdb.toml.v2.3.1 heliosdb.toml

Step 5: Verify Rollback

cargo test --lib

Step 6: Report Issues

If you had to rollback, please report:

• What failed
• Error messages
• Steps to reproduce
• Your environment (OS, Rust version, etc.)

Troubleshooting

Issue 1: Build Fails with Type Errors

Symptom:

error[E0308]: mismatched types
  expected enum `DeltaOperation`
  found enum `DeltaType`

Solution:

// Update imports
use heliosdb_nano::storage::DeltaOperation;

// Update code
let operation = DeltaOperation::Insert;

Issue 2: Tests Fail with Priority Ordering Issues

Symptom: MVs refresh in wrong order.

Solution: Invert priority values (higher = more important in v2.4.0-beta).

Issue 3: Sync Protocol Not Found

Symptom:

error[E0599]: no method named `start_replication_server` found

Solution:

Either enable experimental feature:

heliosdb-nano = { version = "2.4.0-beta", features = ["sync-experimental"] }

Or remove sync code and use branching.

Issue 4: Database Fails to Open

Symptom:

Error: Database version mismatch

Solution: This shouldn’t happen (data format is compatible). If it does:

1. Check data directory permissions
2. Verify data wasn’t corrupted
3. Restore from backup
4. Report issue on GitHub

Issue 5: Performance Degradation

Symptom: Queries are slower than v2.3.1.

Solution:

1. Check MV refresh CPU usage:

   SELECT * FROM pg_mv_staleness;

2. Lower CPU threshold if needed:

   [materialized_views]
   cpu_threshold = 50.0  # Lower = less aggressive

3. Disable auto-refresh for heavy MVs:

   ALTER MATERIALIZED VIEW heavy_mv SET (auto_refresh = false);

Issue 6: Clippy Warnings Flood

Symptom: Hundreds of warnings during build.

Solution: This is expected in beta. Filter important warnings:

cargo clippy --lib 2>&1 | grep "error:"

Or allow warnings temporarily:

#![allow(warnings)]

FAQ

Q: Will my v2.3.1 database work in v2.4.0-beta?

A: Yes, data format is fully compatible. Databases created with v2.3.1 will open and work in v2.4.0-beta without conversion.

Q: Do I need to run a migration script?

A: No, there’s no schema migration needed. Code changes may be required for sync protocol and priority values.

Q: Is v2.4.0-beta production-ready?

A: No, it’s a beta release. Use for development, testing, and non-critical applications only.

Q: When will v2.4.0 stable be released?

A: Target is January 2026, after v2.4.0-rc1 (December 2025) resolves all known issues.

Q: What if I need sync protocol?

A: Three options:

1. Enable sync-experimental feature (not recommended)
2. Stay on v2.3.1 until sync protocol stabilizes (v2.5.0)
3. Use branching for backup/restore in the meantime

Q: Will v2.4.0-rc1 break compatibility again?

A: No, v2.4.0-rc1 will be backward compatible with v2.4.0-beta. Only bug fixes, no breaking changes.

Q: How do I migrate back to v2.3.1?

A: Follow the Rollback Procedure. It’s straightforward since data format is compatible.

Q: Are there any data loss risks?

A: No data loss risks. Always backup before migrating, but data format is compatible and databases are preserved.

Q: What about my custom extensions?

A: If you have custom Rust code depending on HeliosDB Nano APIs:

• Check for DeltaType usage (deprecated but still works)
• Check for sync protocol usage (now behind feature flag)
• Check for priority value assumptions (ordering changed)

Q: Can I run v2.3.1 and v2.4.0-beta side-by-side?

A: Yes, as long as they use separate data directories:

let db_v2_3_1 = EmbeddedDatabase::new("./data_v2.3.1.helio")?;
let db_v2_4_0 = EmbeddedDatabase::new("./data_v2.4.0.helio")?;

Q: How long will v2.3.1 be supported?

A: v2.3.1 will be supported until v2.4.0 stable is released (estimated January 2026). Critical security fixes only after that.

Summary

Migration Effort

• Time: 30 minutes to 2 hours
• Complexity: Low to Medium
• Risk: Low (data format compatible, easy rollback)
• Benefit: Phase 3 stability, bug fixes, type safety

Key Changes

1. Sync Protocol: Now behind sync-experimental feature
2. Priority Ordering: Higher = more important (inverted from v2.3.1)
3. DeltaType: Deprecated in favor of DeltaOperation

Recommended Approach

1. Backup data
2. Update Cargo.toml
3. Fix sync protocol usage (if applicable)
4. Invert priority values (if applicable)
5. Build and test
6. Validate with your production scenarios
7. Monitor for issues

Support

• GitHub Issues: https://github.com/dimensigon/HDB-HeliosDB-Nano/issues
• Documentation: https://docs.heliosdb.io
• Known Issues: See KNOWN_ISSUES-v2.4.0-beta.md

---

Last Updated: 2025-11-24 Version: v2.4.0-beta