Vector Search / Semantic Search

Problem Being Solved

Core Problem Statement

AI/ML applications require fast, accurate vector similarity search for semantic document retrieval, recommendation systems, and RAG (Retrieval Augmented Generation) pipelines, but existing solutions force teams to choose between cloud-only vector databases with high latency and cost, or building custom solutions that lack optimization. Teams deploying to edge devices, microservices, or privacy-sensitive environments cannot tolerate external database dependencies or network round-trips, yet lack embedded vector search.

Who Suffers Most

• Edge AI Engineers: Deploying computer vision or NLP models to IoT devices, industrial equipment, or mobile apps where cloud vector databases are unavailable, forcing them to implement inefficient O(N) brute-force search or abandon similarity features entirely.

---

HeliosDB Lite Solution

Architecture Overview

┌─────────────────────────────────────────────────────────────────────┐
│                 HeliosDB-Lite Vector Search Stack                   │
├─────────────────────────────────────────────────────────────────────┤
│  SQL Layer: CREATE INDEX USING hnsw, Vector Type, Distance Operators │
├─────────────────────────────────────────────────────────────────────┤
│  HNSW Index  │  Product Quantizer  │  SIMD Distance Kernels (AVX2)  │
├─────────────────────────────────────────────────────────────────────┤
│  Graph Persistence (RocksDB LSM) │ Codebook Storage │ Vector Columns│
├─────────────────────────────────────────────────────────────────────┤
│              Embedded Storage Engine (No External Dependencies)      │
└─────────────────────────────────────────────────────────────────────┘

---

Concrete Examples with Code, Config & Architecture

Example 1: RAG Application for Document Q&A - Embedded Configuration

Architecture:

User Query
    ↓
LLM Application (LangChain/LlamaIndex)
    ↓
HeliosDB-Lite Embedded Client (in-process)
    ↓
HNSW Index (semantic search) + RocksDB Storage
    ↓
Top-K Document Retrieval → Context for LLM

Configuration (heliosdb.toml):

# HeliosDB-Lite configuration for RAG vector search
[database]
path = "/var/lib/heliosdb/rag.db"
memory_limit_mb = 256
enable_wal = true
page_size = 4096

[vector]
enabled = true
# Default HNSW parameters optimized for 384-dim embeddings
default_hnsw_m = 16              # Max connections per layer
default_hnsw_ef_construction = 200  # Candidate list size during build
default_hnsw_ef_search = 100     # Candidate list size during search

[vector.quantization]
enabled = true
num_subquantizers = 8            # 384/8 = 48 dims per subquantizer
num_centroids = 256              # Byte-sized codes
training_sample_size = 10000     # Vectors for codebook training

[monitoring]
metrics_enabled = true
verbose_logging = false

[performance]
# SIMD acceleration auto-detected
simd_enabled = true

---

Example 2: Product Recommendation Engine - Python Integration

Scenario: E-commerce platform with 2M products, each with 768-dim image+text multimodal embedding from CLIP. Filter by category/price while maintaining semantic relevance.

---

Example 3: Duplicate Detection System - Docker & Kubernetes Deployment

Scenario: Content moderation platform detecting near-duplicate images/videos at scale (10M items, 512-dim perceptual hash embeddings).

Docker Deployment (Dockerfile):

FROM rust:1.75-slim as builder

WORKDIR /app

# Install build dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    libssl-dev \
    pkg-config \
    && rm -rf /var/lib/apt/lists/*

# Copy source
COPY . .

# Build HeliosDB-Lite application with vector search
RUN cargo build --release --features vector-search,simd

# Runtime stage
FROM debian:bookworm-slim

RUN apt-get update && apt-get install -y \
    ca-certificates \
    libssl3 \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Copy binary
COPY --from=builder /app/target/release/duplicate-detector /usr/local/bin/

# Create data volume mount point
RUN mkdir -p /data && chmod 755 /data

# Expose HTTP API port
EXPOSE 8080

# Health check endpoint
HEALTHCHECK --interval=30s --timeout=3s --start-period=40s --retries=3 \
    CMD curl -f http://localhost:8080/health || exit 1

# Set data directory as volume
VOLUME ["/data"]

# Run with configuration
ENTRYPOINT ["duplicate-detector"]
CMD ["--config", "/etc/heliosdb/config.toml", "--data-dir", "/data", "--port", "8080"]

Docker Compose (docker-compose.yml):

version: '3.8'

services:
  duplicate-detector:
    build:
      context: .
      dockerfile: Dockerfile
    image: duplicate-detector:latest
    container_name: duplicate-detector-prod

    ports:
      - "8080:8080"      # HTTP API

    volumes:
      - ./data:/data                           # Persistent vector database
      - ./config/heliosdb.toml:/etc/heliosdb/config.toml:ro

    environment:
      RUST_LOG: "info,duplicate_detector=debug"
      HELIOSDB_DATA_DIR: "/data"
      HELIOSDB_MEMORY_LIMIT_MB: "2048"

    restart: unless-stopped

    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 40s

    networks:
      - app-network

    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '1'
          memory: 1G

networks:
  app-network:
    driver: bridge

volumes:
  db_data:
    driver: local

Kubernetes Deployment (k8s-deployment.yaml):

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: duplicate-detector
  namespace: content-moderation
spec:
  serviceName: duplicate-detector
  replicas: 3
  selector:
    matchLabels:
      app: duplicate-detector
  template:
    metadata:
      labels:
        app: duplicate-detector
    spec:
      containers:
      - name: duplicate-detector
        image: duplicate-detector:v1.0.0
        imagePullPolicy: Always

        ports:
        - containerPort: 8080
          name: http
          protocol: TCP

        env:
        - name: RUST_LOG
          value: "info"
        - name: HELIOSDB_DATA_DIR
          value: "/data"
        - name: HELIOSDB_MEMORY_LIMIT_MB
          value: "2048"
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name

        volumeMounts:
        - name: data
          mountPath: /data
        - name: config
          mountPath: /etc/heliosdb
          readOnly: true

        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "2000m"

        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 3
          failureThreshold: 3

        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
          timeoutSeconds: 2
          failureThreshold: 2

  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: fast-ssd
      resources:
        requests:
          storage: 20Gi

---
apiVersion: v1
kind: Service
metadata:
  name: duplicate-detector
  namespace: content-moderation
spec:
  type: ClusterIP
  selector:
    app: duplicate-detector
  ports:
  - port: 80
    targetPort: 8080
    name: http

---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: duplicate-detector-hpa
  namespace: content-moderation
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: StatefulSet
    name: duplicate-detector
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Configuration for Container (config.toml):

[server]
host = "0.0.0.0"
port = 8080
max_connections = 100

[database]
path = "/data/duplicates.db"
memory_limit_mb = 2048
enable_wal = true
page_size = 8192
cache_mb = 512

[vector]
enabled = true
default_hnsw_m = 24
default_hnsw_ef_construction = 400
default_hnsw_ef_search = 200

[vector.quantization]
enabled = true
num_subquantizers = 8
num_centroids = 256

[container]
enable_shutdown_on_signal = true
graceful_shutdown_timeout_secs = 30

[monitoring]
metrics_enabled = true
prometheus_port = 9090

---

Example 4: Semantic Search Microservice - Production Rust Service

Service Architecture:

┌─────────────────────────────────────────┐
│     HTTP Request (Axum Framework)        │
├─────────────────────────────────────────┤
│  Search Handler (Async Tokio Runtime)    │
├─────────────────────────────────────────┤
│  SQL Query Builder (Dynamic Filters)     │
├─────────────────────────────────────────┤
│  HeliosDB-Lite Embedded (Shared Arc)     │
├─────────────────────────────────────────┤
│  HNSW Index (Cosine) + B-tree (Filters)  │
├─────────────────────────────────────────┤
│  RocksDB Storage Engine (LSM Tree)       │
└─────────────────────────────────────────┘

---

Example 5: Edge AI Image Search - Embedded IoT Deployment

Edge Device Configuration (config.toml):

[database]
# Ultra-low memory footprint for edge devices
path = "/var/lib/heliosdb/camera_vectors.db"
memory_limit_mb = 256          # Constrained device
page_size = 4096               # Standard page size
enable_wal = true
cache_mb = 64                  # Minimal cache

[vector]
enabled = true
default_hnsw_m = 12            # Reduced for lower memory
default_hnsw_ef_construction = 100
default_hnsw_ef_search = 50

[vector.quantization]
enabled = true
num_subquantizers = 8          # 512/8 = 64 dims per subquantizer
num_centroids = 128            # Reduced from 256 for smaller codebook

[sync]
# Optional cloud sync for alerts
enable_remote_sync = true
sync_interval_secs = 600       # Sync every 10 minutes
sync_endpoint = "https://cloud.example.com/api/camera-sync"
batch_size = 500

[performance]
# Auto-detect ARM NEON SIMD on Jetson
simd_enabled = true

[logging]
# Minimal logging for embedded
level = "warn"
output = "syslog"

Edge Architecture:

┌───────────────────────────────────────────────┐
│    NVIDIA Jetson Nano / Raspberry Pi 4        │
├───────────────────────────────────────────────┤
│   Camera Input (30 FPS Video Stream)          │
├───────────────────────────────────────────────┤
│   ResNet Embedding Model (512-dim)            │
├───────────────────────────────────────────────┤
│   HeliosDB-Lite Vector Search (Embedded)      │
│   - Duplicate detection (HNSW)                │
│   - Anomaly flagging                          │
│   - Local persistence                         │
├───────────────────────────────────────────────┤
│   Background Sync (Every 10 min)              │
├───────────────────────────────────────────────┤
│   Network (Cellular/WiFi, Optional)           │
├───────────────────────────────────────────────┤
│   Cloud Backend (Analytics & Alerts)          │
└───────────────────────────────────────────────┘

---

Adoption Strategy

Phase 1: Proof of Concept (Weeks 1-4)

Target: Validate vector search performance in target application

Tactics:

• Week 1: Deploy HeliosDB Lite in development environment
• Replace existing vector DB client with HeliosDB Lite embedded API
• Migrate 10K-100K vectors from cloud vector DB
• Run side-by-side queries to compare latency/accuracy

• Week 2: Benchmark performance
• Measure query latency (p50, p95, p99) vs existing solution
• Test memory footprint with PQ enabled/disabled

• Week 3: Integration testing
• Test with production embedding model (OpenAI, Sentence-Transformers, etc.)
• Validate SQL integration with existing queries
• Test edge cases (high-dimensional vectors, large K values)

• Week 4: Cost analysis
• Calculate infrastructure cost reduction (cloud DB → embedded)
• Measure deployment complexity reduction (services → single binary)
• Estimate developer velocity improvement (local dev environment)

Phase 2: Pilot Deployment (Weeks 5-12)

Target: Limited production deployment with real traffic

Tactics:

• Week 5-6: Production deployment
• Configure monitoring/alerting (Prometheus metrics)
• Set up performance dashboards (Grafana)

• Week 7-8: Load testing
• Test failover scenarios (pod restarts, node failures)
• Validate data durability (RocksDB WAL recovery)

• Week 9-10: Optimization
• Tune HNSW parameters (M, ef_construction, ef_search)
• Configure PQ settings for optimal compression ratio
• Optimize query patterns based on production logs

• Week 11-12: Stakeholder review
• Present cost savings data to finance/leadership
• Document performance improvements for engineering team
• Gather developer feedback on API ergonomics

Phase 3: Full Rollout

Target: Organization-wide deployment with cloud vector DB retirement

Tactics:

• Week 13-16: Gradual migration
• Maintain read-only cloud DB as backup for 30 days

• Week 17-20: Optimization & monitoring
• Implement auto-scaling policies (Kubernetes HPA)
• Configure backup/restore procedures (RocksDB snapshots)
• Set up comprehensive monitoring (latency, recall, memory)

• Week 21-24: Cloud DB retirement
• Run final parallel query validation (HeliosDB vs cloud DB)
• Shut down cloud vector DB subscription
• Redirect saved costs to other infrastructure

• Monitor for performance regressions (latency, accuracy)
• Upgrade to newer HeliosDB Lite versions (quarterly)
• Expand to additional use cases (recommendations, image search)

---

Conclusion

HeliosDB Lite's vector search capabilities fundamentally solve the AI infrastructure trilemma of performance, cost, and compliance that has forced teams to choose between expensive cloud vector databases, complex self-hosted solutions, or abandoning semantic search features entirely.

HeliosDB Lite captures this underserved market by combining the deployment simplicity of SQLite with the AI-native capabilities of specialized vector databases, creating a new category: embedded vector search for modern AI applications.

For organizations building on LangChain, LlamaIndex, or custom LLM applications, HeliosDB Lite provides an immediate migration path from expensive cloud vector databases to cost-free embedded search with superior performance. For edge AI deployments in IoT, robotics, and autonomous systems, it unlocks semantic search capabilities on resource-constrained devices. For enterprise ML teams in regulated industries, it solves compliance blockers by keeping embeddings on-premises while maintaining cloud-grade performance.

Call to Action: Start your POC today by replacing your cloud vector database with HeliosDB Lite for a single microservice or edge deployment. Measure the latency improvement, cost reduction, and deployment simplification firsthand. Contact the HeliosDB Lite team for migration guides, production deployment best practices, and architecture consultation to accelerate your transition to embedded AI infrastructure.