Phase 3 Test Execution Guide

Version: 1.0 Created: 2025-11-17 Purpose: Guide for executing Phase 3 test suites

Quick Start

# Run all tests
cargo test --all --features "encryption,vector-search"

# Run only Phase 3 tests
cargo test --test phase3 --features "encryption,vector-search"

# Run with coverage
cargo tarpaulin --out Html --features "encryption,vector-search"

# Run benchmarks
cargo bench --features "encryption,vector-search"

Test Organization

tests/
├── unit/                           # Unit tests (60% coverage)
│   ├── sql_wrapper/
│   │   ├── parser_tests.rs        # SQL parser tests
│   │   └── system_views_tests.rs  # System view tests
│   ├── product_quantization/
│   │   ├── kmeans_tests.rs        # K-means clustering tests
│   │   ├── codec_tests.rs         # PQ encode/decode tests
│   │   └── adc_tests.rs           # Asymmetric distance tests
│   ├── compression/
│   │   ├── fsst_tests.rs          # FSST codec tests
│   │   └── alp_tests.rs           # ALP codec tests
│   └── incremental_mv/
│       ├── delta_tracking_tests.rs # Delta tracking tests
│       └── refresh_tests.rs        # MV refresh tests
├── integration/                    # Integration tests (30% coverage)
│   └── phase3_e2e_tests.rs        # End-to-end scenarios
├── compatibility/                  # Compatibility tests (10% coverage)
│   └── phase3_compatibility_tests.rs
└── test_helpers.rs                # Shared test utilities

benches/
└── phase3_benchmarks.rs           # Performance benchmarks

Running Tests by Category

1. Unit Tests

# All unit tests
cargo test --lib --features "encryption,vector-search"

# SQL Wrapper tests
cargo test --test sql_wrapper --features "encryption,vector-search"

# Product Quantization tests
cargo test --test product_quantization --features "encryption,vector-search"

# Compression tests
cargo test --test compression --features "encryption,vector-search"

2. Integration Tests

# All integration tests
cargo test --test integration --features "encryption,vector-search"

# Specific integration test
cargo test --test phase3_e2e_tests --features "encryption,vector-search"

# Run with single thread (for database tests)
RUST_TEST_THREADS=1 cargo test --test integration --features "encryption,vector-search"

3. Compatibility Tests

# All compatibility tests
cargo test --test compatibility --features "encryption,vector-search"

# Specific compatibility scenario
cargo test test_phase3_with_existing_encryption --features "encryption,vector-search"

4. Performance Benchmarks

# All benchmarks
cargo bench --features "encryption,vector-search"

# Specific benchmark group
cargo bench --bench phase3_benchmarks pq_search

# Save baseline for comparison
cargo bench --features "encryption,vector-search" -- --save-baseline main

# Compare against baseline
cargo bench --features "encryption,vector-search" -- --baseline main

Test Execution Strategies

Development Workflow

# Quick smoke test (fast unit tests only)
cargo test --lib --features "encryption,vector-search" -- --nocapture

# Before commit (unit + integration)
cargo test --all --features "encryption,vector-search"

# Before PR (full suite + benchmarks)
cargo test --all --features "encryption,vector-search" && \
cargo bench --features "encryption,vector-search"

CI/CD Pipeline

# Stage 1: Fast feedback (unit tests)
cargo test --lib --features "encryption,vector-search"

# Stage 2: Integration
RUST_TEST_THREADS=1 cargo test --test integration --features "encryption,vector-search"

# Stage 3: Compatibility
cargo test --test compatibility --features "encryption,vector-search"

# Stage 4: Benchmarks (main branch only)
cargo bench --features "encryption,vector-search" -- --save-baseline main

Test Filtering

By Feature

# Test SQL wrapper only
cargo test sql_wrapper --features "encryption,vector-search"

# Test product quantization only
cargo test pq --features "encryption,vector-search"

# Test compression only
cargo test compression --features "encryption,vector-search"

By Pattern

# All tests with "vector" in name
cargo test vector --features "encryption,vector-search"

# All tests with "incremental" in name
cargo test incremental --features "encryption,vector-search"

# Specific test
cargo test test_pq_encoding --features "encryption,vector-search"

By Package

# Test specific module
cargo test --package heliosdb-nano --lib vector --features "encryption,vector-search"

Debugging Tests

Verbose Output

# Show println! output
cargo test --features "encryption,vector-search" -- --nocapture

# Show test names
cargo test --features "encryption,vector-search" -- --list

# Show ignored tests
cargo test --features "encryption,vector-search" -- --ignored

Running Single Test

# Run specific test with output
cargo test test_pq_encode_decode_basic --features "encryption,vector-search" -- --nocapture --exact

# Run with backtrace
RUST_BACKTRACE=1 cargo test test_name --features "encryption,vector-search"

# Run with full backtrace
RUST_BACKTRACE=full cargo test test_name --features "encryption,vector-search"

Coverage Reporting

Generate Coverage

# Install tarpaulin
cargo install cargo-tarpaulin

# Generate HTML report
cargo tarpaulin --out Html --output-dir coverage --features "encryption,vector-search"

# Generate XML for CI
cargo tarpaulin --out Xml --features "encryption,vector-search"

# Generate multiple formats
cargo tarpaulin --out Html --out Xml --output-dir coverage --features "encryption,vector-search"

View Coverage

# Open HTML report
open coverage/index.html  # macOS
xdg-open coverage/index.html  # Linux
start coverage/index.html  # Windows

Coverage by Module

# Coverage for specific module
cargo tarpaulin --packages heliosdb-nano --features "encryption,vector-search" --lib vector

Performance Testing

Benchmark Execution

# Run all benchmarks
cargo bench --features "encryption,vector-search"

# Run specific benchmark
cargo bench --bench phase3_benchmarks pq_search --features "encryption,vector-search"

# Save results
cargo bench --features "encryption,vector-search" -- --save-baseline $(git rev-parse --short HEAD)

Benchmark Comparison

# Compare against baseline
cargo bench --features "encryption,vector-search" -- --baseline main

# Generate comparison report
cargo bench --features "encryption,vector-search" -- --baseline main > benchmark_comparison.txt

Profiling

# Profile with perf (Linux)
cargo bench --no-run --features "encryption,vector-search"
perf record -g ./target/release/deps/phase3_benchmarks-* --bench

# View results
perf report

Test Data Management

Cleanup

# Clean test artifacts
cargo clean

# Remove test databases
rm -rf /tmp/heliosdb_test_*

# Remove benchmark results
rm -rf target/criterion/

Test Database

# Tests automatically create temporary databases
# Location: $TMPDIR/heliosdb_test_*

# To inspect test database
ls -la /tmp/heliosdb_test_*

Common Issues and Solutions

Issue: Tests Timeout

# Increase timeout
cargo test --features "encryption,vector-search" -- --test-threads=1 --nocapture

# Or set environment variable
RUST_TEST_TIMEOUT=300 cargo test --features "encryption,vector-search"

Issue: Flaky Tests

# Run test multiple times
for i in {1..10}; do cargo test test_name --features "encryption,vector-search"; done

# Run with specific seed
RUST_TEST_SEED=12345 cargo test --features "encryption,vector-search"

Issue: Out of Memory

# Limit parallel tests
cargo test --features "encryption,vector-search" -- --test-threads=1

# Increase stack size
RUST_MIN_STACK=8388608 cargo test --features "encryption,vector-search"

Performance Targets

Unit Tests

✅ Each test < 100ms
✅ Total unit test suite < 30 seconds

Integration Tests

✅ Each test < 5 seconds
✅ Total integration suite < 2 minutes

Benchmarks

✅ PQ encoding: < 1ms per vector
✅ PQ search (1M vectors): < 10ms for k=10
✅ FSST compression: > 500 MB/sec
✅ ALP compression: > 400 MB/sec
✅ MV incremental refresh: < 100ms for <1000 rows

Test Metrics

Expected Results

Test Results Summary:
  Unit Tests:        150 tests, 150 passed, 0 failed
  Integration Tests: 20 tests, 20 passed, 0 failed
  Compatibility:     15 tests, 15 passed, 0 failed
  Total:            185 tests, 185 passed, 0 failed

Coverage:
  Lines:   87.3%
  Branches: 82.1%
  Functions: 89.5%

Performance:
  PQ Search (1M): 8.2ms ✅
  FSST Compress:  520 MB/sec ✅
  ALP Compress:   410 MB/sec ✅
  MV Refresh:     72ms ✅

Continuous Integration

GitHub Actions

Tests run automatically on:

✅ Push to main/develop
✅ Pull requests
✅ Nightly builds

Pipeline Stages

Fast Checks (1-2 min)
- Formatting
- Linting
- Unit tests
Integration (3-5 min)
- Integration tests
- Compatibility tests
Quality (5-10 min)
- Coverage report
- Security audit
Performance (main only, 10-15 min)
- Benchmarks
- Regression detection

Best Practices

Run tests before committing

cargo test --all --features "encryption,vector-search"

Check coverage for new code

cargo tarpaulin --out Html --features "encryption,vector-search"

Benchmark performance-critical changes
Terminal window
```
cargo bench --features "encryption,vector-search"
```
Write deterministic tests
- Use fixed seeds for random data
- Clean up test resources
- Avoid time-dependent assertions
Keep tests fast
- Unit tests < 100ms
- Integration tests < 5s
- Use mocks where appropriate

Getting Help

Test Failures: Check logs in target/debug/ or target/release/
Coverage Issues: Review HTML report in coverage/index.html
Performance Issues: Check benchmark results in target/criterion/
CI Failures: Review GitHub Actions logs

Next Steps

After tests pass:

Review coverage report
Check benchmark results
Commit changes
Create pull request
Monitor CI pipeline

Status: Ready for use Last Updated: 2025-11-17