Containerized Microservices with HeliosCore
HeliosDB Lite gives each containerized microservice its own database: an image built from the licensed source distribution runs the server with its data directory on a named volume, publishes the PostgreSQL wire and REST ports to the service and includes a built-in health check for the orchestrator, while HeliosProxy can run beside it as a connection pooler.
Problem Being Solved
Core Problem Statement
Containerized microservices architectures promise rapid deployment and horizontal scaling, but traditional database dependencies create operational bottlenecks that negate these benefits. Deploying a database per microservice with PostgreSQL/MySQL requires provisioning separate database containers, managing persistent volumes across node failures, coordinating service discovery for database endpoints, and implementing manual or scripted failover procedures. The alternative—a shared database cluster—reintroduces tight coupling, single points of failure, and network latency that containerization was meant to eliminate. Neither approach delivers on containers' promise of stateless, rapidly-deployable, self-healing services.
Concrete Examples with Code, Config & Architecture
Example 1: Embedded Configuration
TOML Configuration (heliosdb-container.toml):
[database]
path = "/data/app.db"
page_size = 16384 # Match NVMe physical block size
cache_size_mb = 256
[wal]
mode = "async" # Max throughput for stateless services
group_commit_delay_us = 500 # Batch commits
segment_size_mb = 64
max_segments = 4 # 256MB total WAL size
[io]
# HeliosCore Direct I/O
use_direct_io = true # O_DIRECT for deterministic performance
use_io_uring = true # Linux async I/O (kernel 5.10+)
io_uring_entries = 256
io_uring_polling = false # Use interrupts to save CPU
[checksums]
enabled = true
algorithm = "xxh3" # SIMD-accelerated hashing
verify_on_read = true
repair_on_corruption = true
[crash_recovery]
# Automatic recovery on container restart
auto_replay_wal = true
parallel_recovery = true # Multi-threaded replay
recovery_threads = 4
[snapshot]
# Zero-cost branching for backups
enable_cow_snapshots = true
snapshot_retention = 3 # Keep last 3 snapshots
snapshot_interval_minutes = 60
[performance]
worker_threads = 0 # Auto-detect container CPU limit
prefetch_pages = 16
background_writer = true
[container]
# Kubernetes-specific optimizations
graceful_shutdown_timeout_seconds = 30
health_check_port = 9090
readiness_query = "SELECT 1"
[observability]
metrics_enabled = true
metrics_port = 9090
log_level = "info"
tracing_enabled = true
Example 3: Infrastructure & Container Deployment
Dockerfile (Optimized for size):
# Build stage
FROM rust:1.75-slim AS builder
RUN apt-get update && apt-get install -y libssl-dev pkg-config
WORKDIR /build
COPY Cargo.toml Cargo.lock ./
COPY src ./src
# Build with HeliosCore optimizations
RUN cargo build --release --features "helioscore-io-uring,simd-avx2"
# Runtime stage - minimal
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y \
libssl3 \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=builder /build/target/release/user-service /app/
COPY heliosdb-container.toml /app/config.toml
# Create data directory (can be EmptyDir or PVC)
RUN mkdir -p /data && chmod 755 /data
# Health check using built-in endpoint
HEALTHCHECK --interval=10s --timeout=3s --start-period=5s \
CMD wget -q --spider http://localhost:8080/health || exit 1
EXPOSE 8080 9090
# Run as non-root
RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app /data
USER appuser
# Graceful shutdown timeout (matches Kubernetes terminationGracePeriodSeconds)
STOPSIGNAL SIGTERM
CMD ["/app/user-service", "--config", "/app/config.toml"]
Kubernetes Deployment (Standard Deployment, not StatefulSet):
apiVersion: v1
kind: ConfigMap
metadata:
name: user-service-config
namespace: default
data:
heliosdb-container.toml: |
[database]
path = "/data/users.db"
cache_size_mb = 256
[wal]
mode = "async"
group_commit_delay_us = 500
[io]
use_direct_io = true
use_io_uring = true
[checksums]
enabled = true
algorithm = "xxh3"
verify_on_read = true
[crash_recovery]
auto_replay_wal = true
parallel_recovery = true
[container]
graceful_shutdown_timeout_seconds = 30
---
apiVersion: apps/v1
kind: Deployment # NOT StatefulSet - no special handling needed
metadata:
name: user-service
namespace: default
spec:
replicas: 10
selector:
matchLabels:
app: user-service
template:
metadata:
labels:
app: user-service
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9090"
spec:
containers:
- name: user-service
image: registry.example.com/user-service:v1.0.0
ports:
- name: http
containerPort: 8080
- name: metrics
containerPort: 9090
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 1000m
memory: 512Mi
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 15
periodSeconds: 10
timeoutSeconds: 3
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 2
volumeMounts:
- name: data
mountPath: /data
- name: config
mountPath: /app/config.toml
subPath: heliosdb-container.toml
env:
- name: RUST_LOG
value: "info"
# Graceful shutdown configuration
terminationGracePeriodSeconds: 30
volumes:
- name: data
# Option 1: EmptyDir (ephemeral, fast, no provisioning)
emptyDir:
sizeLimit: 10Gi
# Option 2: PersistentVolumeClaim (durable across restarts)
# persistentVolumeClaim:
# claimName: user-service-pvc
- name: config
configMap:
name: user-service-config
---
apiVersion: v1
kind: Service
metadata:
name: user-service
namespace: default
spec:
type: ClusterIP
selector:
app: user-service
ports:
- name: http
port: 80
targetPort: 8080
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: user-service-hpa
namespace: default
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: user-service
minReplicas: 5
maxReplicas: 50
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
Example 4: Microservices Integration (Go/Rust)
Architecture:
┌─────────────────────┐ gRPC ┌─────────────────────┐
│ Order Service │ ◄──────────────► │ Inventory Service │
│ ┌───────────────┐ │ │ ┌───────────────┐ │
│ │ HeliosDB-Lite │ │ │ │ HeliosDB-Lite │ │
│ │ (orders.db) │ │ │ │ (inventory.db)│ │
│ └───────────────┘ │ │ └───────────────┘ │
│ - Automatic crash │ │ - Independent data │
│ recovery │ │ - No shared state │
│ - Sub-1s failover │ │ - Isolated scaling │
└─────────────────────┘ └─────────────────────┘
Benefits:
- Data isolation (microservices principle)
- Independent scaling (no DB bottleneck)
- Fast recovery (each service self-heals)
- No coordination overhead
Example 5: Edge Computing & IoT Deployment
Edge Gateway Deployment (K3s):
apiVersion: apps/v1
kind: DaemonSet # Deploy on every edge node
metadata:
name: edge-data-collector
namespace: edge
spec:
selector:
matchLabels:
app: edge-data-collector
template:
metadata:
labels:
app: edge-data-collector
spec:
nodeSelector:
node-type: edge-gateway
hostNetwork: true # Access to local sensors
containers:
- name: collector
image: registry.local/edge-data-collector:latest
securityContext:
privileged: true # Hardware access
resources:
requests:
cpu: 500m
memory: 256Mi
limits:
cpu: 2000m
memory: 512Mi
volumeMounts:
- name: data
mountPath: /data
- name: config
mountPath: /app/config.toml
subPath: heliosdb-container.toml
env:
- name: EDGE_NODE_ID
valueFrom:
fieldRef:
fieldPath: spec.nodeName
volumes:
- name: data
hostPath:
path: /mnt/nvme/edge-db
type: DirectoryOrCreate
- name: config
configMap:
name: edge-config
Adoption Strategy
Phase 1: Proof of Concept (Month 1)
Objective: Validate operational simplicity and crash recovery
Actions: 1. Select 2-3 stateless microservices as candidates 2. Replace external PostgreSQL with embedded HeliosDB Lite 3. Deploy with standard Deployment (not StatefulSet) 4. Chaos test: kill pods randomly, verify automatic recovery 5. Measure startup time, memory, crash recovery latency
Phase 2: Production Rollout (Months 2-4)
Actions: 1. Update CI/CD templates with HeliosDB Lite configuration 2. Migrate 10-15 high-churn services (most deployments/day) 3. Eliminate StatefulSets where applicable 4. Build monitoring dashboards (Grafana + Prometheus) 5. Document cost savings (PVC elimination, smaller instances)
Phase 3: Standardization (Months 5-12)
Objective: HeliosDB Lite as default for new services
Actions: 1. Update platform documentation and training 2. Create Helm charts with HeliosDB Lite best practices Decommission shared PostgreSQL clusters 5. Negotiate enterprise support contract
Conclusion
HeliosDB Lite with HeliosCore Direct I/O and automatic crash recovery transforms containerized microservices from operationally complex StatefulSet deployments into simple, self-healing, embedded database architectures.
The combination of io_uring async I/O, checksummed pages with SIMD acceleration, zero-cost branching for instant snapshots, and automatic WAL replay positions HeliosDB Lite as the optimal database solution for cloud-native applications where operational simplicity, resource efficiency, and self-healing are critical requirements.
Performance depends on workload and hardware. Published figures are measured on representative fixtures; see the benchmarks page and reproduce them on your own workload.