Skip to content
HeliosDB Nano

Migration Guide: Test Suite API Evolution (v2.3.x)

Migration Guide: Test Suite API Evolution (v2.3.x)

Overview

This guide documents API changes introduced in v2.3.x that affect test and example code. The compilation errors have been reduced from 222 to 71 (68% fixed).

Summary of Changes

Fixed Issues (151/222)

  • LogicalPlan::Scan missing as_of field (77 fixes)
  • StorageEngine missing fields (3 fixes)
  • EmbeddedDatabase API evolution (12+ fixes)
  • Error constructor renaming (4 fixes)
  • Method signature corrections (50+ fixes)

Remaining Issues (71/222)

  • Method signature changes requiring parameter updates
  • Module visibility changes
  • Trait import requirements
  • Private field access patterns

Detailed API Changes

1. LogicalPlan::Scan - Added as_of Field

Impact: 77 compilation errors fixed

Change: The LogicalPlan::Scan variant now requires an as_of field for time-travel queries.

Before:

LogicalPlan::Scan {
    table_name: "users".to_string(),
    schema: Arc::new(schema),
    projection: None,
}

After:

LogicalPlan::Scan {
    table_name: "users".to_string(),
    schema: Arc::new(schema),
    projection: None,
    as_of: None,  // Added for AS OF clause support
}

Fix Applied: Automated via batch script to all tests and examples.

2. StorageEngine - Added MV Delta Tracking Fields

Impact: 3 compilation errors fixed

Change: StorageEngine struct now includes materialized view delta tracking.

New Fields:

  • mv_delta_tracker: Arc<MvDeltaTracker> - Tracks table changes for incremental MV refresh

Initialization: Automatically handled in StorageEngine::open() and StorageEngine::open_in_memory().

Fix Applied: Updated engine initialization in src/storage/engine.rs.

3. EmbeddedDatabase - Added with_config() Method

Impact: 12 compilation errors fixed

Change: Added new constructor to create database with custom configuration.

New API:

impl EmbeddedDatabase {
    pub fn with_config(config: Config) -> Result<Self>
}

Usage:

let mut config = Config::in_memory();
config.compression.enabled = true;
let db = EmbeddedDatabase::with_config(config)?;

Migration:

  • EmbeddedDatabase::new_with_config(config) → EmbeddedDatabase::with_config(config)

Fix Applied: Added method to src/lib.rs and updated all test references.

4. Error Constructors - Renamed configuration to config

Impact: 4 compilation errors fixed

Change: Error constructor method renamed for consistency.

Before:

Error::configuration("Invalid configuration")

After:

Error::config("Invalid configuration")

Fix Applied: Automated replacement in all tests and examples.

5. Value Enum Changes

Impact: Identified but not yet fixed (6 errors remain)

Change: Value enum variants renamed/modified:

  • Value::Text → Value::String (in some contexts)
  • Integer dereferencing behavior changed

Migration Needed:

// Before
match value {
    Value::Int8(count) => assert_eq!(*count, 2),
    _ => panic!("Expected Int8"),
}


// After
match value {
    Value::Int8(count) => assert_eq!(count, 2),  // No dereference needed
    _ => panic!("Expected Int8"),
}

Status: Requires manual review of each case.

6. PasswordStore Trait - Import Required

Impact: 9+ compilation errors remain

Change: PasswordStore trait methods (add_user, get_credentials, etc.) now require trait import.

Error:

error[E0599]: no method named `add_user` found for struct `InMemoryPasswordStore`
help: trait `PasswordStore` which provides `add_user` is implemented but not in scope

Fix Required:

// Add to imports
use heliosdb_nano::protocol::postgres::PasswordStore;


// Then methods work:
let mut store = InMemoryPasswordStore::new();
store.add_user("alice", "password")?;

Status: Partially fixed (1 test file updated, others need same fix).

7. MvDeltaTracker API Changes

Impact: 9 compilation errors remain

Changes: Several methods removed or renamed:

  • track_table() - Method not found
  • is_tracked() - Method not found

Investigation Needed: Check new API in src/storage/mv_delta.rs.

8. StorageConfig Field Rename

Impact: 1 compilation error fixed

Change: Field renamed for clarity.

Before:

if config.storage.in_memory {

After:

if config.storage.memory_only {

Fix Applied: Updated in src/lib.rs.

9. Module Visibility Changes

Impact: 9 compilation errors remain

Changes:

  • heliosdb_nano::config module is now private
  • heliosdb_nano::time_travel module is now private
  • heliosdb_nano::Database type not exported

Migration Needed:

  • Use public APIs instead of internal modules
  • Use EmbeddedDatabase instead of Database

10. Method Signature Changes

Impact: 11+ compilation errors remain

Examples:

  • Some methods changed from 1 argument to 2 arguments
  • Parameter types may have changed

Status: Requires case-by-case investigation.

Automated Fixes Applied

1. LogicalPlan::Scan Batch Fix

Script: /tmp/fix_tests.sh Files Modified: 15 test and example files Pattern: Added as_of: None to all LogicalPlan::Scan initializations

2. Error Constructor Replacement

Command: sed -i 's/Error::configuration/Error::config/g' Files: All test and example files

3. Database Constructor Replacement

Command: sed -i 's/EmbeddedDatabase::new_with_config/EmbeddedDatabase::with_config/g' Files: All test and example files

Compilation Status

Before Fixes

  • Total Errors: 222
  • Test Files: 30+
  • Example Files: 7+

After Fixes

  • Total Errors: 71 (68% reduction)
  • Errors Fixed: 151
  • Remaining Work: 71 errors across various categories

Error Distribution (Remaining)

  1. Method signature mismatches: 11
  2. Private field access (db): 10
  3. MvDeltaTracker API: 9
  4. PasswordStore trait imports: 9
  5. Module visibility: 9
  6. Other mismatched types: 23

Next Steps for Complete Migration

High Priority (Quick Fixes)

  1. Add PasswordStore imports - Affects 9 test files

    • Add use heliosdb_nano::protocol::postgres::PasswordStore;
    • Files: postgres_scram_auth_tests.rs, postgres_ssl_tests.rs, etc.

  2. Fix remaining LogicalPlan::Scan - 4 instances

    • Run script again or manual fix

  3. Update Error::configuration - Any remaining instances

Medium Priority (API Investigation Needed)

  1. MvDeltaTracker API - 9 errors

    • Review new API in src/storage/mv_delta.rs
    • Update test code to use new methods

  2. Method signature changes - 11 errors

    • Review each call site
    • Update parameters to match new signatures

Low Priority (Refactoring)

  1. Private field access - 10 errors

    • Replace direct storage.db access with public methods
    • May require adding accessor methods

  2. Module visibility - 9 errors

    • Refactor tests to use public APIs
    • Remove internal module dependencies

Testing Strategy

Incremental Compilation

Terminal window
# Test specific modules
cargo test --lib --no-run
cargo test --test integration_test --no-run
cargo test --example embedded --no-run

Full Test Suite

Terminal window
# After all fixes
cargo test --no-run  # Ensure compilation
cargo test            # Run tests

Common Patterns

Pattern 1: LogicalPlan Construction

Always include all required fields:

LogicalPlan::Scan {
    table_name: name.to_string(),
    schema: Arc::new(schema),
    projection: None,
    as_of: None,  // Don't forget!
}

Pattern 2: Error Construction

Use correct constructor name:

Error::config(msg)      // ✓ Correct
Error::configuration(msg)  // ✗ Old API

Pattern 3: Database Initialization

// Option 1: Default config
let db = EmbeddedDatabase::new("./data")?;


// Option 2: Custom config
let config = Config::in_memory();
let db = EmbeddedDatabase::with_config(config)?;

Pattern 4: Trait Imports

When using trait methods, import the trait:

use heliosdb_nano::protocol::postgres::PasswordStore;


let mut store = InMemoryPasswordStore::new();
store.add_user("user", "pass")?;  // Trait method now available

Files Modified

Core Library

  • src/storage/engine.rs - Added mv_delta_tracker field and initialization
  • src/sql/executor/phase3.rs - Fixed moved value in CREATE MATERIALIZED VIEW
  • src/storage/mv_incremental.rs - Fixed catalog method name (get_table → get_table_schema)
  • src/sql/explain_production.rs - Fixed MemoryStats initialization
  • src/lib.rs - Added with_config() method, fixed storage config field name

Tests (Partial List)

  • tests/postgres_scram_auth_tests.rs - Added PasswordStore trait import
  • examples/concurrent_mv_refresh_demo.rs - Added as_of field to LogicalPlan::Scan
  • Plus 13+ other test files with LogicalPlan fixes

Known Issues

Issue 1: Type Inference for Arc

Error: type annotations needed for Arc<_, _> Cause: Generic type parameters cannot be inferred Fix: Add explicit type annotation

Issue 2: Unresolved Imports

Errors: heliosdb_nano::Database, heliosdb_nano::compression, etc. Cause: Types not exported at crate root Fix: Use full paths or check if types were renamed/removed

Issue 3: Private Module Access

Error: module config is private Cause: Internal modules hidden from public API Fix: Use public configuration APIs instead

Summary

This migration focused on mechanical fixes that could be automated or quickly applied. The test suite now has 68% fewer compilation errors (151 fixed, 71 remaining). The remaining errors require more careful investigation of API changes and may need upstream discussion about intended behavior.

Statistics

  • Compilation Errors: 222 → 71 (-68%)
  • Files Modified: 18+ core files, 15+ test files
  • Automated Fixes: 3 batch operations
  • Manual Fixes: 6 specific corrections
  • Time Investment: Approximately 2 hours

Recommendations

  1. Complete PasswordStore trait imports (easy wins)
  2. Document MvDeltaTracker new API
  3. Review method signature changes with team
  4. Consider adding migration tests
  5. Update CI to catch API evolution issues early