HeliosDB CI/CD Pipeline Quick Start Guide

HeliosDB CI/CD Pipeline Quick Start Guide

Quick reference for developers and DevOps engineers

---

Table of Contents

1. Local Development
2. Running CI Pipeline Locally
3. Creating a Release
4. Deploying to Production
5. Monitoring Deployments
6. Troubleshooting

---

1. Local Development

Build and Run with Docker

# Build the Docker image
docker build -t heliosdb:local -f Dockerfile.cicd .

# Run locally
docker run -p 5432:5432 -p 3306:3306 -p 8443:8443 heliosdb:local

Build and Run with Docker Compose

# Start HeliosDB with monitoring stack
docker-compose -f docker-compose.cicd.yml --profile monitoring up -d

# Start with replication
docker-compose -f docker-compose.cicd.yml --profile replication --profile monitoring up -d

# View logs
docker-compose -f docker-compose.cicd.yml logs -f heliosdb

# Stop and remove
docker-compose -f docker-compose.cicd.yml down -v

Local Testing

# Run unit tests
cargo test --workspace --all-features

# Run with coverage
cargo install cargo-llvm-cov
cargo llvm-cov --workspace --all-features --html
open target/llvm-cov/html/index.html

# Run benchmarks
cargo bench --workspace

# Security audit
cargo install cargo-audit
cargo audit

# Check formatting
cargo fmt --all -- --check

# Run clippy
cargo clippy --workspace --all-features --all-targets -- -D warnings

---

2. Running CI Pipeline Locally

Using Act (GitHub Actions locally)

# Install act
brew install act  # macOS
# or
curl https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash

# Run the CI pipeline
act push

# Run specific job
act -j build-and-test

# Run with specific event
act pull_request

Manual Smoke Tests

# Make scripts executable (if not already)
chmod +x scripts/ci/*.sh

# Run smoke tests
./scripts/ci/smoke-tests.sh localhost:8443

# Run comprehensive tests
./scripts/ci/comprehensive-smoke-tests.sh staging

# Verify connectivity
./scripts/ci/verify-connectivity.sh localhost

---

3. Creating a Release

Automatic Release (Recommended)

# 1. Update version in Cargo.toml
sed -i 's/version = "4.0.0"/version = "4.0.1"/' Cargo.toml

# 2. Commit changes
git add Cargo.toml
git commit -m "chore: bump version to 4.0.1"
git push origin main

# 3. Create and push tag
git tag v4.0.1
git push origin v4.0.1

# This triggers the release-automated.yml workflow

Manual Release via GitHub UI

1. Go to Actions → Automated Release Pipeline
2. Click Run workflow
3. Select:
   • Version: v4.0.1
   • Release type: patch (or minor/major)
4. Click Run workflow

Verify Release

# Check GitHub release page
open https://github.com/heliosdb/heliosdb/releases

# Pull Docker image
docker pull heliosdb/heliosdb:4.0.1

# Verify image
docker run --rm heliosdb/heliosdb:4.0.1 /opt/heliosdb/bin/heliosdb-server --version

---

4. Deploying to Production

Blue-Green Deployment

1. Go to Actions → Deploy to Production
2. Click Run workflow
3. Select:
   • Deployment strategy: blue-green
   • Environment: production
   • Version: v4.0.1 or latest
4. Click Run workflow
5. Monitor deployment in Actions tab

Canary Deployment

1. Go to Actions → Deploy to Production
2. Click Run workflow
3. Select:
   • Deployment strategy: canary
   • Environment: production
   • Version: v4.0.1
4. Click Run workflow

Canary Phases:

• 10% traffic → 10 minutes monitoring
• 50% traffic → 10 minutes monitoring
• 100% traffic → Final validation

Rollback

If deployment fails, rollback is automatic. Manual rollback:

# Using GitHub CLI
gh workflow run deploy-production.yml \
  -f deployment_strategy=blue-green \
  -f environment=production \
  -f version=v4.0.0  # Previous stable version

---

5. Monitoring Deployments

GitHub Actions UI

1. Go to Actions tab
2. Select running workflow
3. View real-time logs
4. Check job summaries

Prometheus Metrics

# Access Prometheus UI (if using docker-compose)
open http://localhost:9091

# Query metrics
http_requests_total
heliosdb_query_duration_seconds
process_cpu_seconds_total

Grafana Dashboards

# Access Grafana UI
open http://localhost:3000

# Default credentials
Username: admin
Password: admin

CloudWatch (AWS Deployments)

# View ECS service logs
aws logs tail /ecs/heliosdb-production --follow

# Check service health
aws ecs describe-services \
  --cluster heliosdb-production \
  --services heliosdb-service

---

6. Troubleshooting

Build Failures

Problem: Build fails with “out of memory”

Solution:

# Increase Docker memory limit
docker system prune -a
# Update Docker Desktop settings: Memory → 8GB+

# Or build with reduced parallelism
CARGO_BUILD_JOBS=2 cargo build --release

Problem: Dependency conflicts

Solution:

# Update Cargo.lock
cargo update

# Or clean and rebuild
cargo clean
cargo build --workspace --all-features

Test Failures

Problem: Flaky integration tests

Solution:

# Run with increased timeout
RUST_TEST_TIMEOUT=300 cargo test --workspace

# Run specific test with output
cargo test --package heliosdb-storage test_name -- --nocapture

Problem: Coverage below 95%

Solution:

# Identify uncovered code
cargo llvm-cov --workspace --all-features --html
open target/llvm-cov/html/index.html

# Add tests for uncovered functions

Deployment Failures

Problem: Health check fails

Solution:

# Check endpoint manually
curl http://your-endpoint:8443/health

# Check logs
docker logs heliosdb-primary

# Or AWS ECS
aws logs tail /ecs/heliosdb-production --follow

Problem: Database migration fails

Solution:

# Connect to database
psql -h your-endpoint -p 5432 -U postgres

# Check migration status
SELECT * FROM schema_migrations;

# Rollback and retry

Security Scan Failures

Problem: Vulnerability detected

Solution:

# Check advisory details
cargo audit

# Update dependencies
cargo update

# Or ignore advisory (with justification)
echo "[advisories]" >> deny.toml
echo "ignore = [\"RUSTSEC-2024-XXXX\"]" >> deny.toml

Docker Image Issues

Problem: Image too large

Solution:

# Analyze layers
docker history heliosdb:latest

# Use multi-stage build (already implemented in Dockerfile.cicd)
# Ensure build cache is working
docker build --cache-from heliosdb:latest -t heliosdb:latest -f Dockerfile.cicd .

---

Quick Commands Cheat Sheet

# Development
docker-compose up -d                          # Start locally
cargo test --workspace                        # Run tests
cargo llvm-cov --workspace --html             # Coverage report

# CI/CD
git tag v4.0.1 && git push origin v4.0.1     # Trigger release
act push                                       # Test CI locally
./scripts/ci/smoke-tests.sh localhost:8443   # Smoke test

# Deployment
gh workflow run deploy-production.yml         # Deploy via CLI
docker pull heliosdb/heliosdb:latest          # Pull latest image
docker-compose -f docker-compose.cicd.yml up  # Local deployment

# Monitoring
open http://localhost:9091                    # Prometheus
open http://localhost:3000                    # Grafana
docker-compose logs -f heliosdb               # View logs

# Troubleshooting
cargo clean && cargo build                    # Clean rebuild
docker system prune -a                        # Clean Docker
cargo audit                                   # Security check

---

Environment Variables Reference

Docker

HELIOSDB_POSTGRES_PORT=5432
HELIOSDB_MYSQL_PORT=3306
HELIOSDB_HTTP_PORT=8443
HELIOSDB_ORACLE_PORT=1521
HELIOSDB_METRICS_PORT=9090
HELIOSDB_BIND_ADDR=0.0.0.0
HELIOSDB_DATA_DIR=/var/lib/heliosdb/data
RUST_LOG=info

CI/CD

CARGO_TERM_COLOR=always
RUST_BACKTRACE=1
CARGO_INCREMENTAL=0
CARGO_NET_RETRY=10
RUSTUP_MAX_RETRIES=10

---

Useful Links

• Documentation: Full Implementation Report
• GitHub Actions: Workflows Directory
• Docker Files: Dockerfile.cicd, docker-compose.cicd.yml
• Scripts: scripts/ci/
• Configuration: cliff.toml, prometheus.yml

---

Last Updated: November 24, 2025
Version: 1.0
Maintainer: HeliosDB DevOps Team