Skip to content
HeliosDB HeliosDB Full

HeliosDB Quantum Optimizer Integration Architecture

HeliosDB Quantum Optimizer Integration Architecture

Date: January 12, 2026 Version: 7.1.0 Status: Design Document - Approved for Implementation Crate: heliosdb-research/crates/quantum Feature Flag: quantum-optimizer

Executive Summary

This document specifies the architecture for integrating the Quantum Optimizer into HeliosDB’s query optimization pipeline. The integration follows a Tier 1 Safe Integration approach, designed to provide performance benefits for complex OLAP queries while maintaining zero overhead for simple queries.

Key Design Principles

  1. Zero overhead for simple queries - Quantum optimization only activates for complex queries
  2. Fallback-only mode - Quantum optimizer engages only when classical optimizer times out or fails
  3. OLAP workload gating - Feature restricted to analytical workloads, not OLTP
  4. Feature flag control - Disabled by default, opt-in for production use

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                     HeliosDB Query Optimization Pipeline v7.1               │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  Query ──► Parser ──► Analyzer ──► ┌──────────────────────────────────────┐ │
│                                    │      Optimizer Router (NEW)          │ │
│                                    │  ┌────────────────────────────────┐  │ │
│                                    │  │ Workload Classifier            │  │ │
│                                    │  │ • OLTP → Classical Only        │  │ │
│                                    │  │ • OLAP → Hybrid Path           │  │ │
│                                    │  │ • Batch → Quantum Eligible     │  │ │
│                                    │  └────────────────────────────────┘  │ │
│                                    └──────────────┬─────────────────────┬─┘ │
│                                                   │                     │   │
│                   ┌───────────────────────────────┴─────┐               │   │
│                   ▼                                     ▼               │   │
│    ┌──────────────────────────────┐   ┌────────────────────────────────┐│   │
│    │    Classical Optimizer       │   │   Quantum Optimizer (Gated)   ││   │
│    │    (Always Primary)          │   │   (Fallback Only)             ││   │
│    ├──────────────────────────────┤   ├────────────────────────────────┤│   │
│    │ • Cost-Based Optimizer v2    │   │ • Simulated Quantum Annealing ││   │
│    │ • Distributed Optimizer      │   │ • QAOA (10+ joins)            ││   │
│    │ • Timeout: 50-200ms          │   │ • Iterations: 50-100 (online) ││   │
│    │ • Max joins: 8-12            │   │ • Threshold: ≥10 joins        ││   │
│    └──────────────────────────────┘   └────────────────────────────────┘│   │
│                   │                                     │               │   │
│                   │         ┌───────────────────────────┘               │   │
│                   │         │                                           │   │
│                   ▼         ▼                                           │   │
│    ┌──────────────────────────────────────────────────────────────────┐ │   │
│    │                    Plan Comparator (NEW)                         │ │   │
│    │  • Compare estimated costs from both optimizers                  │ │   │
│    │  • Select plan with lower estimated cost                         │ │   │
│    │  • Record decision for adaptive learning                         │ │   │
│    └──────────────────────────────────────────────────────────────────┘ │   │
│                               │                                         │   │
│                               ▼                                         │   │
│    ┌──────────────────────────────────────────────────────────────────┐ │   │
│    │                    Execution Engine                              │ │   │
│    └──────────────────────────────────────────────────────────────────┘ │   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Configuration Changes

1. Quantum Optimizer Configuration (v7.1.0)

Previous (v7.0.0):

pub struct Config {
    pub enable_annealing: bool,           // true
    pub enable_grover: bool,              // true
    pub annealing_iterations: usize,      // 1000
    pub initial_temperature: f64,         // 100.0
    pub cooling_rate: f64,                // 0.95
    pub complexity_threshold: usize,      // 5
}

New (v7.1.0):

pub struct QuantumOptimizerConfig {
    // Feature control
    pub enabled: bool,                        // false (opt-in)
    pub mode: QuantumOptimizerMode,           // FallbackOnly


    // Algorithm selection
    pub enable_annealing: bool,               // true
    pub enable_grover: bool,                  // false (pending rewrite)
    pub enable_qaoa: bool,                    // true


    // Performance tuning - CHANGED
    pub annealing_iterations_online: usize,   // 50 (was 1000)
    pub annealing_iterations_batch: usize,    // 500 (new)
    pub initial_temperature: f64,             // 100.0
    pub cooling_rate: f64,                    // 0.95


    // Activation thresholds - CHANGED
    pub complexity_threshold: usize,          // 10 (was 5)
    pub min_estimated_cost: f64,              // 10000.0 (new)
    pub classical_timeout_ms: u64,            // 100 (new)


    // Workload gating - NEW
    pub allowed_workloads: Vec<WorkloadType>, // [OLAP, Batch]
    pub max_concurrent_quantum_optimizations: usize, // 4
}


#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum QuantumOptimizerMode {
    /// Never use quantum optimizer (default)
    Disabled,
    /// Only use when classical times out or fails
    FallbackOnly,
    /// Run in parallel, select best plan
    Parallel,
    /// Always try quantum first (testing only)
    QuantumFirst,
}


#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum WorkloadType {
    OLTP,
    OLAP,
    Batch,
    Mixed,
}

2. Default Configuration Profiles

impl QuantumOptimizerConfig {
    /// Production default - quantum disabled
    pub fn production() -> Self {
        Self {
            enabled: false,
            mode: QuantumOptimizerMode::Disabled,
            ..Default::default()
        }
    }


    /// OLAP workload - quantum fallback enabled
    pub fn olap_optimized() -> Self {
        Self {
            enabled: true,
            mode: QuantumOptimizerMode::FallbackOnly,
            complexity_threshold: 10,
            annealing_iterations_online: 50,
            allowed_workloads: vec![WorkloadType::OLAP, WorkloadType::Batch],
            ..Default::default()
        }
    }


    /// Aggressive optimization - parallel mode
    pub fn aggressive() -> Self {
        Self {
            enabled: true,
            mode: QuantumOptimizerMode::Parallel,
            complexity_threshold: 8,
            annealing_iterations_online: 100,
            annealing_iterations_batch: 1000,
            allowed_workloads: vec![WorkloadType::OLAP, WorkloadType::Batch, WorkloadType::Mixed],
            ..Default::default()
        }
    }
}

Component Specifications

1. Optimizer Router (NEW)

Purpose: Routes queries to appropriate optimizer based on workload classification and complexity.

Location: heliosdb-query/src/optimizer_router.rs

pub struct OptimizerRouter {
    classical_optimizer: Arc<ExecutionOptimizer>,
    quantum_optimizer: Option<Arc<QuantumOptimizer>>,
    config: QuantumOptimizerConfig,
    workload_classifier: WorkloadClassifier,
    metrics: Arc<RouterMetrics>,
}


impl OptimizerRouter {
    /// Route query to appropriate optimizer(s)
    pub async fn optimize(&self, plan: QueryPlan) -> Result<OptimizedPlan> {
        let complexity = self.calculate_complexity(&plan);
        let workload = self.workload_classifier.classify(&plan);


        // Gate 1: Check if quantum is enabled
        if !self.config.enabled {
            return self.classical_optimizer.optimize(plan).await;
        }


        // Gate 2: Check workload type
        if !self.config.allowed_workloads.contains(&workload) {
            return self.classical_optimizer.optimize(plan).await;
        }


        // Gate 3: Check complexity threshold
        if complexity.join_count < self.config.complexity_threshold {
            return self.classical_optimizer.optimize(plan).await;
        }


        // Route based on mode
        match self.config.mode {
            QuantumOptimizerMode::Disabled => {
                self.classical_optimizer.optimize(plan).await
            }
            QuantumOptimizerMode::FallbackOnly => {
                self.optimize_with_fallback(plan, complexity).await
            }
            QuantumOptimizerMode::Parallel => {
                self.optimize_parallel(plan, complexity).await
            }
            QuantumOptimizerMode::QuantumFirst => {
                self.optimize_quantum_first(plan, complexity).await
            }
        }
    }


    /// Tier 1: Safe Integration - Fallback only
    async fn optimize_with_fallback(
        &self,
        plan: QueryPlan,
        complexity: QueryComplexity,
    ) -> Result<OptimizedPlan> {
        let start = Instant::now();


        // Try classical first with timeout
        let classical_result = tokio::time::timeout(
            Duration::from_millis(self.config.classical_timeout_ms),
            self.classical_optimizer.optimize(plan.clone())
        ).await;


        match classical_result {
            Ok(Ok(optimized)) => {
                self.metrics.record_classical_success(start.elapsed());
                Ok(optimized)
            }
            Ok(Err(e)) | Err(_) => {
                // Classical failed or timed out - try quantum
                self.metrics.record_classical_timeout();


                if let Some(quantum) = &self.quantum_optimizer {
                    match quantum.optimize(plan.clone()).await {
                        Ok(quantum_plan) => {
                            self.metrics.record_quantum_fallback_success();
                            Ok(quantum_plan)
                        }
                        Err(_) => {
                            // Quantum also failed - return classical error or best effort
                            self.metrics.record_quantum_fallback_failure();
                            self.classical_optimizer.optimize_best_effort(plan).await
                        }
                    }
                } else {
                    self.classical_optimizer.optimize_best_effort(plan).await
                }
            }
        }
    }
}

2. Workload Classifier (NEW)

Purpose: Classifies queries as OLTP, OLAP, or Batch based on characteristics.

Location: heliosdb-query/src/workload_classifier.rs

pub struct WorkloadClassifier {
    config: ClassifierConfig,
}


impl WorkloadClassifier {
    pub fn classify(&self, plan: &QueryPlan) -> WorkloadType {
        let features = self.extract_features(plan);


        // OLTP indicators
        let oltp_score = self.calculate_oltp_score(&features);


        // OLAP indicators
        let olap_score = self.calculate_olap_score(&features);


        // Classification logic
        if features.has_batch_hint {
            return WorkloadType::Batch;
        }


        if oltp_score > 0.7 {
            WorkloadType::OLTP
        } else if olap_score > 0.7 {
            WorkloadType::OLAP
        } else {
            WorkloadType::Mixed
        }
    }


    fn calculate_oltp_score(&self, features: &QueryFeatures) -> f64 {
        let mut score = 0.0;


        // Point lookup patterns
        if features.has_primary_key_filter { score += 0.3; }
        if features.estimated_rows < 100 { score += 0.2; }
        if features.join_count <= 2 { score += 0.2; }
        if features.has_limit && features.limit_value < 100 { score += 0.15; }
        if !features.has_aggregation { score += 0.15; }


        score.min(1.0)
    }


    fn calculate_olap_score(&self, features: &QueryFeatures) -> f64 {
        let mut score = 0.0;


        // Analytical patterns
        if features.has_aggregation { score += 0.25; }
        if features.has_group_by { score += 0.2; }
        if features.join_count >= 3 { score += 0.2; }
        if features.estimated_rows > 10000 { score += 0.2; }
        if features.scans_large_tables { score += 0.15; }


        score.min(1.0)
    }
}

3. Plan Comparator (NEW)

Purpose: Compares plans from classical and quantum optimizers, selects best.

Location: heliosdb-query/src/plan_comparator.rs

pub struct PlanComparator {
    cost_model: Arc<CostModel>,
}


impl PlanComparator {
    pub fn select_best(
        &self,
        classical: OptimizedPlan,
        quantum: OptimizedPlan,
    ) -> (OptimizedPlan, PlanSelection) {
        let classical_cost = self.cost_model.estimate(&classical);
        let quantum_cost = self.cost_model.estimate(&quantum);


        // Prefer classical unless quantum is significantly better (>10% improvement)
        let improvement_threshold = 0.10;
        let improvement = (classical_cost - quantum_cost) / classical_cost;


        if improvement > improvement_threshold {
            (quantum, PlanSelection::Quantum { improvement })
        } else {
            (classical, PlanSelection::Classical { quantum_cost, classical_cost })
        }
    }
}

Grover’s Algorithm Rewrite - IP Analysis

Current Implementation Issues

The existing Grover’s search implementation at heliosdb-research/crates/quantum/src/optimizer.rs:104-122 has a fundamental flaw:

// PROBLEMATIC: Generates all n! permutations first
let all_orderings = self.generate_orderings(n);  // O(n!)
let optimal_ordering = self.grover.search(all_orderings, ...);

This defeats the O(√N) speedup of Grover’s algorithm.

Proper Quantum Oracle Pattern

The correct implementation uses a quantum oracle that marks solutions without enumeration:

/// Proper quantum oracle for join ordering
pub trait QuantumOracle<T> {
    /// Check if item satisfies search criteria
    /// Does NOT enumerate - uses quantum superposition
    fn evaluate(&self, state: &QuantumState) -> bool;
}


pub struct JoinOrderingOracle {
    cost_threshold: f64,
    join_graph: JoinGraph,
}


impl QuantumOracle<JoinOrder> for JoinOrderingOracle {
    fn evaluate(&self, state: &QuantumState) -> bool {
        // Quantum amplitude marking without enumeration
        let cost = self.estimate_cost_from_state(state);
        cost < self.cost_threshold
    }
}

IP Risk Assessment

Algorithm ComponentPatent StatusRisk LevelNotes
Grover’s AlgorithmPublic DomainNonePublished 1996, expired 2016
Quantum Oracle PatternOpenNoneStandard quantum computing pattern
QUBO FormulationOpenNoneAcademic/open-source
Simulated AnnealingPublic DomainNonePublished 1983
QAOAOpenLowIBM open-sourced Qiskit implementations
Join Ordering via QANovelEvaluateHeliosDB innovation - consider patent

Recommendation

  1. Grover’s Rewrite is IP-safe - The algorithm is public domain
  2. Disable Grover until rewritten - Current implementation is counterproductive
  3. Consider patenting the novel QUBO formulation for join ordering optimization

Feature Flag Implementation

Cargo Feature Definition

File: heliosdb-research/crates/quantum/Cargo.toml

[features]
default = []
quantum-optimizer = [
    "quantum-annealing",
    "qaoa",
]
quantum-annealing = []
qaoa = []
grover = []  # Disabled by default until rewrite complete
full-quantum = ["quantum-optimizer", "grover"]

Runtime Configuration

File: heliosdb-config/src/quantum.rs

/// Load quantum optimizer configuration from environment/config file
pub fn load_quantum_config() -> QuantumOptimizerConfig {
    let enabled = std::env::var("HELIOSDB_QUANTUM_OPTIMIZER")
        .map(|v| v == "true" || v == "1")
        .unwrap_or(false);


    let mode = std::env::var("HELIOSDB_QUANTUM_MODE")
        .map(|v| match v.as_str() {
            "fallback" => QuantumOptimizerMode::FallbackOnly,
            "parallel" => QuantumOptimizerMode::Parallel,
            _ => QuantumOptimizerMode::Disabled,
        })
        .unwrap_or(QuantumOptimizerMode::Disabled);


    QuantumOptimizerConfig {
        enabled,
        mode,
        complexity_threshold: std::env::var("HELIOSDB_QUANTUM_THRESHOLD")
            .ok()
            .and_then(|v| v.parse().ok())
            .unwrap_or(10),
        ..Default::default()
    }
}

Performance Expectations

Overhead Analysis

ScenarioClassical OnlyWith Quantum (Fallback)Delta
Simple query (< 5 joins)10-50μs10-50μs0%
Medium query (5-9 joins)50-200μs50-200μs0%
Complex query (10+ joins)50ms timeout50ms + 5-15ms+10-30%
Very complex (15+ joins)Skip/timeout20-50ms quantumBetter plan

Success Criteria

MetricTargetMeasurement
Zero overhead for < 10 joins< 1% latency impactP99 latency monitoring
Fallback activation rate< 5% of queriesMetrics counter
Plan improvement when activated> 10% cost reductionCost comparison logs
Quantum optimization time< 100ms for onlineP99 quantum time

Metrics and Observability

New Prometheus Metrics

// Optimizer routing metrics
pub static OPTIMIZER_ROUTE_CLASSICAL: Counter = ...;
pub static OPTIMIZER_ROUTE_QUANTUM_FALLBACK: Counter = ...;
pub static OPTIMIZER_ROUTE_QUANTUM_PARALLEL: Counter = ...;


// Quantum optimizer metrics
pub static QUANTUM_OPTIMIZATION_DURATION: Histogram = ...;
pub static QUANTUM_ANNEALING_ITERATIONS: Histogram = ...;
pub static QUANTUM_PLAN_SELECTED: Counter = ...;
pub static QUANTUM_PLAN_REJECTED: Counter = ...;


// Comparison metrics
pub static PLAN_COMPARISON_QUANTUM_BETTER: Counter = ...;
pub static PLAN_COMPARISON_CLASSICAL_BETTER: Counter = ...;
pub static PLAN_IMPROVEMENT_RATIO: Histogram = ...;

Migration Path

Phase 1: v7.1.0 (This Release)

  • Implement Tier 1 Safe Integration (fallback-only)
  • Raise complexity_threshold to 10
  • Reduce annealing_iterations to 50 (online) / 500 (batch)
  • Add feature flag (disabled by default)
  • Add workload classifier
  • Disable Grover’s search (pending rewrite)

Phase 2: v7.2.0 (Future)

  • Rewrite Grover’s search with proper oracle pattern
  • Enable parallel mode option
  • Add adaptive threshold tuning
  • ML-based plan selection

Phase 3: v8.0.0 (Future)

  • Real quantum hardware integration (IBM Qiskit, AWS Braket)
  • Hybrid quantum-classical optimization
  • Auto-tuning based on workload patterns

Testing Requirements

Unit Tests

  • Workload classifier accuracy
  • Complexity calculation
  • Plan comparator logic
  • Feature flag behavior

Integration Tests

  • End-to-end fallback path
  • Classical timeout triggering quantum
  • OLTP queries bypass quantum
  • OLAP queries eligible for quantum

Performance Tests

  • Zero overhead for simple queries
  • Fallback latency within bounds
  • Concurrent quantum optimizations

Security Considerations

  1. Resource Limits: Quantum optimization is CPU-intensive; limit concurrent optimizations
  2. Timeout Enforcement: Hard timeout on quantum operations to prevent DoS
  3. Feature Flag Audit: Log all quantum optimization activations for audit trail

References

  1. Grover, L. K. (1996). “A fast quantum mechanical algorithm for database search” - Public Domain
  2. Farhi, E., et al. (2014). “A Quantum Approximate Optimization Algorithm” - arXiv:1411.4028
  3. HeliosDB Cost Optimizer v2 Architecture - docs/architecture/COST_OPTIMIZER_ARCHITECTURE.md
  4. Distributed Query Optimization - docs/architecture/F3_5_DISTRIBUTED_QUERY_OPTIMIZATION_ARCHITECTURE.md

Document Version: 1.0 Last Updated: January 12, 2026 Next Review: Before v7.2.0 release