Migrating from Cassandra to HeliosDB
Target Audience: Database Administrators, DevOps Engineers, Architects Estimated Migration Time: 1-4 weeks depending on strategy Difficulty: Intermediate
Table of Contents
- Overview
- Why Migrate to HeliosDB?
- Compatibility Matrix
- Prerequisites
- Migration Strategies
- Step-by-Step Migration
- Feature Mapping
- Troubleshooting
- Best Practices
- Rollback Plan
- Case Studies
- Next Steps
Overview
HeliosDB Full implements the Apache Cassandra native protocol (v4 and v5) and CQL 3 for keyspaces, tables, indexes, SELECT/INSERT/UPDATE/DELETE, prepared statements and batches. Point the driver at HeliosDB, then test your workload against the compatibility notes, which include a current limitation of the DataStax drivers. This guide provides comprehensive strategies and step-by-step procedures for migrating your Cassandra workloads to HeliosDB.
What You Get
- Native CQL listener: the listener accepts and authenticates CQL connections. Standard Cassandra drivers, including the DataStax drivers and cqlsh, cannot connect to HeliosDB Full yet; driver compatibility and reads and writes of HeliosDB tables through the CQL listener are planned
- Multi-Protocol: Combine Cassandra CQL with PostgreSQL, MongoDB, and Oracle 23ai in a single database
- Better Performance: 2.7x OLTP and 7.5x OLAP performance improvements
- Simpler Operations: Consolidate multiple clusters into a single multi-protocol deployment
- Enhanced Analytics: Native SQL support for complex analytics on Cassandra data
Why Migrate to HeliosDB?
1. Multi-Protocol Unification
Problem: Organizations often run separate databases for different workloads:
- Cassandra for time-series and high-write workloads
- PostgreSQL for transactional data
- MongoDB for document storage
- Oracle for legacy applications
Solution: HeliosDB speaks all four protocols natively, eliminating data silos.
# Same database, multiple protocolsimport psycopg2 # PostgreSQL clientfrom cassandra.cluster import Cluster # Cassandra client
# Connect via PostgreSQL protocolpg_conn = psycopg2.connect(host="heliosdb.example.com", port=5432, ...)
# Connect via CQL protocol (same data!)cluster = Cluster(['heliosdb.example.com'], port=9042)cql_session = cluster.connect('my_keyspace')2. Better Analytics Performance
Cassandra Challenge: Complex analytics queries are slow or impossible.
HeliosDB Advantage:
- 7.5x faster OLAP queries using columnar storage and vectorized execution
- Native SQL support for complex joins and aggregations
- Materialized views with automatic maintenance
Example:
-- Cassandra: Multiple queries + client-side joinSELECT * FROM users WHERE user_id = ?;SELECT * FROM orders WHERE user_id = ?;-- Manually join in application code
-- HeliosDB: Single SQL query on CQL dataSELECT u.username, COUNT(o.order_id), SUM(o.amount)FROM users uJOIN orders o ON u.user_id = o.user_idWHERE o.created_at > '2025-01-01'GROUP BY u.usernameORDER BY SUM(o.amount) DESC;3. Lower Operational Complexity
Cassandra Operations:
- Manual nodetool repairs
- Complex cluster expansion
- Tombstone management
- Compaction tuning
HeliosDB Operations:
- Self-healing: 96% autonomous issue resolution
- Auto-scaling: Elastic compute (0 to max CUs)
- Automated tuning: ML-based query optimization
- Simplified ops: Single cluster vs. multi-cluster
4. Infrastructure Efficiency
- Intelligent tiering: Hot/warm/cold storage tiers move infrequently accessed data to lower-cost object storage
- Scale-to-zero: Compute can scale down for intermittent workloads
- Consolidation: No cross-cluster data transfer between separate databases
HeliosDB Full is commercially licensed (source-available under SSPL-1.0); contact sales@heliosdb.com for pricing.
Compatibility Matrix
Supported
| Feature Category | Cassandra | HeliosDB | Notes |
|---|---|---|---|
| Protocol | Native Protocol v4/v5 | Full | STARTUP, OPTIONS, AUTH_RESPONSE, QUERY, PREPARE, EXECUTE, BATCH, REGISTER |
| Authentication | AllowAll, Password, SASL | Full | AllowAllAuthenticator, PasswordAuthenticator (SASL PLAIN) |
| DDL | CREATE/ALTER/DROP keyspace/table | Full | Identical syntax |
| DML | SELECT/INSERT/UPDATE/DELETE | Full | Standard CQL syntax |
| Data Types | All primitive types | Full | ASCII, TEXT, INT, BIGINT, UUID, etc. |
| Collections | LIST, SET, MAP | Full | Stored as JSONB internally |
| UDT | User-defined types | Full | Full support |
| Indexes | Secondary indexes | Full | CREATE INDEX syntax identical |
| Batches | LOGGED, UNLOGGED, COUNTER | Full | Full support |
| Prepared Statements | Parameter binding | Full | Full support |
| Paging | Result paging | Full | Full support |
| Consistency Levels | ONE, QUORUM, ALL, etc. | Full | All levels supported |
| TTL | Time-to-live | Parsed | USING TTL is accepted but not yet applied |
| Timestamps | Write timestamps | Parsed | USING TIMESTAMP is accepted but not yet applied |
| Functions | Built-in functions | Full | uuid(), now(), token(), etc. |
| Aggregations | SUM, AVG, COUNT, MIN, MAX | Full | Full support |
Partially Supported
| Feature | Result | Notes |
|---|---|---|
| Compression (LZ4/Snappy) | Partial: 80% | Partial support |
| Materialized Views | Partial: Parser only | Execution not yet supported |
| Lightweight Transactions (LWT) | Partial: Basic | IF EXISTS/IF NOT EXISTS supported |
| Custom Types | Limited | Basic support only |
Not Supported
| Feature | Result |
|---|---|
| User-Defined Functions (UDF) | Not supported |
| User-Defined Aggregates (UDA) | Not supported |
| Triggers | Not supported |
| SASI Indexes | Not supported |
| Change Data Capture (CDC) | Not supported |
Compatibility notes
- The CQL listener is opt-in: start HeliosDB Full with
--enable-cassandra(port 9042 by default; set--cassandra-portorHELIOSDB_CASSANDRA_PORTto change it). - Native protocol v4 and v5 are supported. Authentication supports AllowAllAuthenticator and PasswordAuthenticator (SASL PLAIN).
USING TTLandUSING TIMESTAMPare accepted but not yet applied. If your application relies on expiring rows, handle expiry explicitly (for example with a scheduledDELETE).- Current limitation: the DataStax drivers query
system.local WHERE key='local'while connecting, and the listener currently rejects that query (Expected identifier, got Key), so the DataStax Python driver (and other drivers that send the same query) cannot connect yet. Check your driver against a test server before planning a cutover. - Validate driver behaviour your application depends on (prepared statements, paging, batches, consistency levels) with your own workload before cutover.
Performance Characteristics
Figures from internal tests (HeliosDB Full vs Apache Cassandra); reproduce on your own hardware and workload.
| Operation | Cassandra | HeliosDB | Notes |
|---|---|---|---|
| Primary key lookup | <1ms | <1ms | Equal performance |
| Range query (clustering) | 1-10ms | 1-5ms | Faster with columnar storage |
| Secondary index scan | 10-50ms | 5-25ms | Optimized indexes |
| ALLOW FILTERING | 50-500ms | 20-200ms | Better query optimizer |
| Batch (10 statements) | 5-20ms | 5-15ms | Equal or better |
| Write throughput | 10K-100K ops/s | 10K-150K ops/s | Higher peak throughput |
Prerequisites
1. Cassandra Version Requirements
Supported Versions:
- Apache Cassandra 3.0+
- Apache Cassandra 4.0+
- Apache Cassandra 5.0
- DataStax Enterprise (DSE) 6.0+
- ScyllaDB (Cassandra-compatible)
Version Check:
cqlsh -e "SHOW VERSION"2. Data Volume Assessment
Calculate Total Data Size:
# On each Cassandra nodenodetool statusnodetool tablestats <keyspace>Estimate Migration Time:
- Small (<100 GB): 1-2 days
- Medium (100 GB - 1 TB): 3-7 days
- Large (1 TB - 10 TB): 1-3 weeks
- Very Large (>10 TB): 3-6 weeks
3. Schema Complexity Assessment
Run Assessment Script:
-- Count keyspacesSELECT COUNT(*) FROM system_schema.keyspaces WHERE keyspace_name NOT IN ('system', 'system_schema');
-- Count tablesSELECT keyspace_name, COUNT(*) AS table_countFROM system_schema.tablesWHERE keyspace_name NOT IN ('system', 'system_schema')GROUP BY keyspace_name;
-- Identify UDTsSELECT keyspace_name, type_nameFROM system_schema.types;
-- Identify materialized viewsSELECT keyspace_name, view_nameFROM system_schema.views;4. Downtime Planning
Migration Strategies by Downtime Tolerance:
| Strategy | Downtime | Complexity | Use Case |
|---|---|---|---|
| Dual-Write | Zero | High | Production critical systems |
| Replication-Based | Minimal (<5 min) | Medium | Most common |
| Snapshot & Restore | 1-24 hours | Low | Dev/staging, scheduled maintenance |
5. HeliosDB Cluster Setup
Minimum Requirements:
# Production cluster (3 nodes)compute: nodes: 3 cpu: 8 cores/node memory: 32 GB/node storage: 1 TB NVMe/node
# Development cluster (1 node)compute: nodes: 1 cpu: 4 cores memory: 16 GB storage: 500 GB SSDInstallation:
# Install HeliosDB Full from the packages provided during onboarding# (contact sales@heliosdb.com for access)
# Start clusterheliosdb cluster init --nodes 3 --replication 3
# Verify CQL endpointheliosdb cluster status | grep cql_endpoint# Output: cql_endpoint: heliosdb.example.com:9042Migration Strategies
Strategy 1: Dual-Write (Zero Downtime) RECOMMENDED FOR PRODUCTION
Best For: Production systems, mission-critical applications, zero downtime requirement
Timeline: 2-4 weeks
Steps:
- Set up HeliosDB cluster
- Configure application for dual-write (Cassandra + HeliosDB)
- Backfill historical data in background
- Validate data consistency (100% match)
- Gradually switch read traffic (10% → 50% → 100%)
- Monitor for 1-2 weeks
- Deprecate Cassandra cluster
Pros:
- Zero downtime
- Gradual rollout reduces risk
- Easy rollback at any stage
Cons:
- Requires application changes
- Higher temporary costs (dual infrastructure)
- Longer migration timeline
Implementation:
# Dual-write wrapperclass DualWriteConnection: def __init__(self): self.cassandra = Cluster(['cassandra.example.com'], port=9042).connect() self.heliosdb = Cluster(['heliosdb.example.com'], port=9042).connect()
def execute(self, query, params=None): # Write to both (fail if either fails) try: result_cassandra = self.cassandra.execute(query, params) result_heliosdb = self.heliosdb.execute(query, params) return result_cassandra # Return Cassandra result for now except Exception as e: # Rollback if needed logger.error(f"Dual-write failed: {e}") raise
def execute_read(self, query, params=None): # Route reads based on traffic split if random.random() < 0.10: # 10% to HeliosDB return self.heliosdb.execute(query, params) else: return self.cassandra.execute(query, params)
# Usageconn = DualWriteConnection()conn.execute("INSERT INTO users (user_id, name) VALUES (?, ?)", [uuid4(), 'Alice'])users = conn.execute_read("SELECT * FROM users WHERE user_id = ?", [user_id])Backfill Script:
#!/bin/bash# backfill.sh - Backfill historical data
CASSANDRA_HOST="cassandra.example.com"HELIOSDB_HOST="heliosdb.example.com"KEYSPACE="my_keyspace"TABLES=("users" "orders" "events")
for TABLE in "${TABLES[@]}"; do echo "Backfilling $TABLE..."
# Export from Cassandra cqlsh $CASSANDRA_HOST -e "COPY $KEYSPACE.$TABLE TO '/tmp/$TABLE.csv'"
# Import to HeliosDB cqlsh $HELIOSDB_HOST -e "COPY $KEYSPACE.$TABLE FROM '/tmp/$TABLE.csv'"
# Verify row count CASSANDRA_COUNT=$(cqlsh $CASSANDRA_HOST -e "SELECT COUNT(*) FROM $KEYSPACE.$TABLE" | tail -1) HELIOSDB_COUNT=$(cqlsh $HELIOSDB_HOST -e "SELECT COUNT(*) FROM $KEYSPACE.$TABLE" | tail -1)
if [ "$CASSANDRA_COUNT" == "$HELIOSDB_COUNT" ]; then echo "✓ $TABLE backfill verified ($CASSANDRA_COUNT rows)" else echo "✗ $TABLE row count mismatch (Cassandra: $CASSANDRA_COUNT, HeliosDB: $HELIOSDB_COUNT)" exit 1 fidone
echo "Backfill complete!"Strategy 2: Replication-Based (Minimal Downtime)
Best For: Most production systems, <5 minute downtime acceptable
Timeline: 1-2 weeks
Steps:
- Set up HeliosDB cluster
- Configure CDC/replication from Cassandra to HeliosDB
- Initial full sync
- Catch-up replication (monitor lag)
- Planned cutover window (5 minutes)
- Switch application to HeliosDB
- Monitor for issues
Pros:
- Minimal downtime (2-5 minutes)
- Lower risk than snapshot
- No application dual-write complexity
Cons:
- Requires CDC/replication tooling
- Brief downtime during cutover
- More complex than snapshot
Implementation (using Debezium CDC):
name: cassandra-heliosdb-replicationconnector.class: io.debezium.connector.cassandra.CassandraConnectorcassandra.hosts: cassandra1:9042,cassandra2:9042,cassandra3:9042cassandra.keyspace: my_keyspacecassandra.tables: users,orders,eventscassandra.username: cassandracassandra.password: ${CASSANDRA_PASSWORD}
# HeliosDB sinkheliosdb.host: heliosdb.example.comheliosdb.port: 9042heliosdb.keyspace: my_keyspaceheliosdb.username: adminheliosdb.password: ${HELIOSDB_PASSWORD}
# Replication settingssnapshot.mode: initial # Full snapshot firstpoll.interval.ms: 100max.batch.size: 5000Monitor Replication Lag:
# Check replication lagcurl http://debezium:8083/connectors/cassandra-heliosdb-replication/status | jq '.connector.state'
# Expected output during catch-up:# {# "state": "RUNNING",# "lag_seconds": 120, # Decreasing to 0# "records_replicated": 1500000,# "records_remaining": 50000# }Cutover Script:
#!/bin/bash# cutover.sh - Perform cutover to HeliosDB
echo "Starting cutover to HeliosDB..."
# 1. Stop application writesecho "Stopping application..."kubectl scale deployment my-app --replicas=0
# 2. Wait for replication to catch up (lag < 1 second)while true; do LAG=$(curl -s http://debezium:8083/connectors/cassandra-heliosdb-replication/status | jq '.lag_seconds') if [ "$LAG" -lt 1 ]; then echo "Replication caught up (lag: ${LAG}s)" break fi echo "Waiting for replication... (lag: ${LAG}s)" sleep 2done
# 3. Verify data consistencyecho "Verifying data consistency..."./verify_data_consistency.sh
# 4. Update application config to HeliosDBkubectl set env deployment/my-app CASSANDRA_HOST=heliosdb.example.com
# 5. Restart applicationecho "Starting application with HeliosDB..."kubectl scale deployment my-app --replicas=3
echo "Cutover complete! Application now using HeliosDB."echo "Monitor logs: kubectl logs -f deployment/my-app"Strategy 3: Snapshot & Restore (Planned Downtime)
Best For: Development/staging, scheduled maintenance windows
Timeline: 1-3 days
Steps:
- Schedule maintenance window
- Take Cassandra snapshot
- Stop writes to Cassandra
- Export snapshot to HeliosDB format
- Import to HeliosDB
- Validate data
- Switch application to HeliosDB
- Resume operations
Pros:
- Simplest approach
- Lower cost (no dual infrastructure)
- Fastest migration
Cons:
- Requires downtime (1-24 hours)
- Not suitable for 24/7 systems
- Higher risk (all-or-nothing)
Implementation:
#!/bin/bash# snapshot_migrate.sh - Snapshot and migrate
CASSANDRA_HOST="cassandra.example.com"HELIOSDB_HOST="heliosdb.example.com"KEYSPACE="my_keyspace"SNAPSHOT_NAME="migration_$(date +%Y%m%d_%H%M%S)"
echo "Step 1: Taking Cassandra snapshot..."nodetool snapshot $KEYSPACE -t $SNAPSHOT_NAME
echo "Step 2: Exporting snapshot..."# Find snapshot directorySNAPSHOT_DIR=$(find /var/lib/cassandra/data/$KEYSPACE -name $SNAPSHOT_NAME -type d | head -1)echo "Snapshot location: $SNAPSHOT_DIR"
# Use sstableloader (recommended) or COPYecho "Step 3: Loading to HeliosDB..."sstableloader -d $HELIOSDB_HOST $SNAPSHOT_DIR
# Alternative: COPY command# for TABLE in users orders events; do# cqlsh $CASSANDRA_HOST -e "COPY $KEYSPACE.$TABLE TO '/tmp/${TABLE}.csv'"# cqlsh $HELIOSDB_HOST -e "COPY $KEYSPACE.$TABLE FROM '/tmp/${TABLE}.csv'"# done
echo "Step 4: Validating data..."./verify_data_consistency.sh
echo "Migration complete!"Step-by-Step Migration
Step 1: Connection Setup
Cassandra Connection:
# Connect to Cassandracqlsh cassandra.example.com 9042 -u cassandra -p passwordHeliosDB Connection (identical syntax):
# Connect to HeliosDB (same protocol!)cqlsh heliosdb.example.com 9042 -u admin -p passwordTest Connectivity:
-- On both Cassandra and HeliosDBSELECT cluster_name, release_version FROM system.local;Python Client (change the contact points):
from cassandra.cluster import Clusterfrom cassandra.auth import PlainTextAuthProvider
# Cassandraauth_provider = PlainTextAuthProvider(username='cassandra', password='password')cluster = Cluster(['cassandra.example.com'], port=9042, auth_provider=auth_provider)cassandra_session = cluster.connect()
# HeliosDB (just change host!)auth_provider = PlainTextAuthProvider(username='admin', password='password')cluster = Cluster(['heliosdb.example.com'], port=9042, auth_provider=auth_provider)heliosdb_session = cluster.connect()Step 2: Schema Migration
Export Cassandra Schema:
# Export all keyspacescqlsh cassandra.example.com -e "DESCRIBE KEYSPACES" > keyspaces.txt
# Export specific keyspace schemacqlsh cassandra.example.com -e "DESCRIBE KEYSPACE my_keyspace" > my_keyspace_schema.cqlCreate Keyspace in HeliosDB (identical syntax):
-- Example schemaCREATE KEYSPACE IF NOT EXISTS my_keyspaceWITH REPLICATION = { 'class': 'SimpleStrategy', 'replication_factor': 3};
USE my_keyspace;
-- Create tableCREATE TABLE users ( user_id UUID PRIMARY KEY, username TEXT, email TEXT, created_at TIMESTAMP, profile MAP<TEXT, TEXT>);
-- Create table with clusteringCREATE TABLE user_events ( user_id UUID, event_time TIMESTAMP, event_type TEXT, payload MAP<TEXT, TEXT>, PRIMARY KEY ((user_id), event_time)) WITH CLUSTERING ORDER BY (event_time DESC);
-- Create secondary indexCREATE INDEX ON users (email);
-- Create user-defined typeCREATE TYPE address ( street TEXT, city TEXT, zip_code TEXT);
CREATE TABLE user_addresses ( user_id UUID PRIMARY KEY, home_address FROZEN<address>, work_address FROZEN<address>);Automated Schema Migration Script:
#!/bin/bashCASSANDRA_HOST="cassandra.example.com"HELIOSDB_HOST="heliosdb.example.com"
# Get list of non-system keyspacesKEYSPACES=$(cqlsh $CASSANDRA_HOST -e "DESCRIBE KEYSPACES" | grep -v "system")
for KEYSPACE in $KEYSPACES; do echo "Migrating schema for keyspace: $KEYSPACE"
# Export schema cqlsh $CASSANDRA_HOST -e "DESCRIBE KEYSPACE $KEYSPACE" > /tmp/${KEYSPACE}_schema.cql
# Import to HeliosDB cqlsh $HELIOSDB_HOST -f /tmp/${KEYSPACE}_schema.cql
echo "✓ $KEYSPACE schema migrated"done
echo "Schema migration complete!"Step 3: Data Migration
Option A: COPY Command (Small to Medium Datasets <100 GB)
Export:
# Export table to CSVcqlsh cassandra.example.com -e "COPY my_keyspace.users TO '/tmp/users.csv'WITH HEADER = TRUE;"Import:
# Import CSV to HeliosDBcqlsh heliosdb.example.com -e "COPY my_keyspace.users FROM '/tmp/users.csv'WITH HEADER = TRUE;"Automated COPY Script:
#!/bin/bashCASSANDRA_HOST="cassandra.example.com"HELIOSDB_HOST="heliosdb.example.com"KEYSPACE="my_keyspace"TABLES=("users" "orders" "events" "user_events")
for TABLE in "${TABLES[@]}"; do echo "Migrating $KEYSPACE.$TABLE..."
# Export cqlsh $CASSANDRA_HOST -e " COPY $KEYSPACE.$TABLE TO '/tmp/${TABLE}.csv' WITH HEADER = TRUE AND PAGETIMEOUT = 60; "
# Import cqlsh $HELIOSDB_HOST -e " COPY $KEYSPACE.$TABLE FROM '/tmp/${TABLE}.csv' WITH HEADER = TRUE AND CHUNKSIZE = 5000; "
# Verify count CASSANDRA_COUNT=$(cqlsh $CASSANDRA_HOST -e "SELECT COUNT(*) FROM $KEYSPACE.$TABLE" | grep -Eo '[0-9]+') HELIOSDB_COUNT=$(cqlsh $HELIOSDB_HOST -e "SELECT COUNT(*) FROM $KEYSPACE.$TABLE" | grep -Eo '[0-9]+')
if [ "$CASSANDRA_COUNT" == "$HELIOSDB_COUNT" ]; then echo "✓ $TABLE migrated successfully ($CASSANDRA_COUNT rows)" else echo "✗ $TABLE row count mismatch (C: $CASSANDRA_COUNT, H: $HELIOSDB_COUNT)" exit 1 fidone
echo "Data migration complete!"Option B: sstableloader (Large Datasets >100 GB)
Export SSTables:
# Take snapshotnodetool snapshot my_keyspace -t migration_snapshot
# Find snapshot directoryfind /var/lib/cassandra/data/my_keyspace -name migration_snapshotLoad to HeliosDB:
# Load SSTablessstableloader -d heliosdb.example.com \ -u admin \ -pw password \ /var/lib/cassandra/data/my_keyspace/users-abc123/snapshots/migration_snapshotOption C: Spark Bulk Load (Very Large Datasets >1 TB)
Spark Migration Job:
import org.apache.spark.sql.SparkSessionimport com.datastax.spark.connector._
val spark = SparkSession.builder() .appName("Cassandra to HeliosDB Migration") .config("spark.cassandra.connection.host", "cassandra.example.com") .config("spark.cassandra.auth.username", "cassandra") .config("spark.cassandra.auth.password", "password") .getOrCreate()
// Read from Cassandraval cassandraDF = spark.read .format("org.apache.spark.sql.cassandra") .options(Map("keyspace" -> "my_keyspace", "table" -> "users")) .load()
// Write to HeliosDBcassandraDF.write .format("org.apache.spark.sql.cassandra") .options(Map( "keyspace" -> "my_keyspace", "table" -> "users", "spark.cassandra.connection.host" -> "heliosdb.example.com", "spark.cassandra.auth.username" -> "admin", "spark.cassandra.auth.password" -> "password" )) .mode("append") .save()Step 4: Application Migration
Start with the connection configuration. Applications connect by changing the contact points once the contact points target HeliosDB; run your test suite and check the compatibility notes.
Before (Cassandra):
from cassandra.cluster import Clusterfrom cassandra.auth import PlainTextAuthProvider
auth = PlainTextAuthProvider(username='cassandra', password='password')cluster = Cluster(['cassandra.example.com'], port=9042, auth_provider=auth)session = cluster.connect('my_keyspace')
# All CQL queries work identicallyrows = session.execute("SELECT * FROM users WHERE user_id = ?", [user_id])for row in rows: print(row.username, row.email)After (HeliosDB) - Only hostname changes:
from cassandra.cluster import Clusterfrom cassandra.auth import PlainTextAuthProvider
auth = PlainTextAuthProvider(username='admin', password='password')cluster = Cluster(['heliosdb.example.com'], port=9042, auth_provider=auth) # ← Only changesession = cluster.connect('my_keyspace')
# Same queries, no changes!rows = session.execute("SELECT * FROM users WHERE user_id = ?", [user_id])for row in rows: print(row.username, row.email)Configuration-Based Connection (best practice):
import os
CASSANDRA_HOST = os.environ.get('CASSANDRA_HOST', 'cassandra.example.com')CASSANDRA_PORT = int(os.environ.get('CASSANDRA_PORT', '9042'))CASSANDRA_USERNAME = os.environ.get('CASSANDRA_USERNAME', 'cassandra')CASSANDRA_PASSWORD = os.environ.get('CASSANDRA_PASSWORD')
# app.pyfrom cassandra.cluster import Clusterfrom cassandra.auth import PlainTextAuthProviderimport config
auth = PlainTextAuthProvider(username=config.CASSANDRA_USERNAME, password=config.CASSANDRA_PASSWORD)cluster = Cluster([config.CASSANDRA_HOST], port=config.CASSANDRA_PORT, auth_provider=auth)session = cluster.connect('my_keyspace')Update Environment Variables:
# For Cassandraexport CASSANDRA_HOST=cassandra.example.com
# For HeliosDB (only change this!)export CASSANDRA_HOST=heliosdb.example.comSpring Boot Configuration:
spring: data: cassandra: contact-points: ${CASSANDRA_HOST:cassandra.example.com} port: 9042 keyspace-name: my_keyspace username: ${CASSANDRA_USERNAME} password: ${CASSANDRA_PASSWORD} local-datacenter: datacenter1Step 5: Validation
Row Count Validation:
-- Run on both Cassandra and HeliosDBSELECT COUNT(*) FROM my_keyspace.users;SELECT COUNT(*) FROM my_keyspace.orders;SELECT COUNT(*) FROM my_keyspace.events;Sample Data Comparison:
-- Compare first 100 rows (sorted by primary key)SELECT * FROM users LIMIT 100;Automated Validation Script:
from cassandra.cluster import Clusterfrom cassandra.auth import PlainTextAuthProvider
# Connect to bothcassandra_cluster = Cluster(['cassandra.example.com'], port=9042, auth_provider=PlainTextAuthProvider('cassandra', 'password'))heliosdb_cluster = Cluster(['heliosdb.example.com'], port=9042, auth_provider=PlainTextAuthProvider('admin', 'password'))
cassandra_session = cassandra_cluster.connect('my_keyspace')heliosdb_session = heliosdb_cluster.connect('my_keyspace')
# Tables to validatetables = ['users', 'orders', 'events']
validation_results = []
for table in tables: # Count validation cass_count = cassandra_session.execute(f"SELECT COUNT(*) FROM {table}").one()[0] helios_count = heliosdb_session.execute(f"SELECT COUNT(*) FROM {table}").one()[0]
count_match = cass_count == helios_count
# Sample validation (first 1000 rows) cass_sample = list(cassandra_session.execute(f"SELECT * FROM {table} LIMIT 1000")) helios_sample = list(heliosdb_session.execute(f"SELECT * FROM {table} LIMIT 1000"))
sample_match = cass_sample == helios_sample
validation_results.append({ 'table': table, 'cassandra_count': cass_count, 'heliosdb_count': helios_count, 'count_match': count_match, 'sample_match': sample_match, 'status': 'PASS' if count_match and sample_match else 'FAIL' })
# Print resultsfor result in validation_results: status_icon = '✓' if result['status'] == 'PASS' else '✗' print(f"{status_icon} {result['table']}: {result['cassandra_count']} rows (match: {result['count_match']}, sample: {result['sample_match']})")
# Exit with error if any failuresif any(r['status'] == 'FAIL' for r in validation_results): print("\n✗ Validation FAILED") exit(1)else: print("\n✓ Validation PASSED")Checksum Validation (for critical data):
import hashlib
def compute_table_checksum(session, table, sample_size=10000): """Compute checksum of table data""" rows = session.execute(f"SELECT * FROM {table} LIMIT {sample_size}")
# Sort rows by primary key for deterministic order sorted_rows = sorted(rows, key=lambda r: str(r))
# Compute checksum hasher = hashlib.sha256() for row in sorted_rows: hasher.update(str(row).encode())
return hasher.hexdigest()
# Compare checksumsfor table in tables: cass_checksum = compute_table_checksum(cassandra_session, table) helios_checksum = compute_table_checksum(heliosdb_session, table)
if cass_checksum == helios_checksum: print(f"✓ {table} checksum match: {cass_checksum[:8]}...") else: print(f"✗ {table} checksum MISMATCH") print(f" Cassandra: {cass_checksum}") print(f" HeliosDB: {helios_checksum}")Step 6: Performance Tuning
Connection Pooling:
from cassandra.cluster import Cluster, ExecutionProfile, EXEC_PROFILE_DEFAULTfrom cassandra.policies import DCAwareRoundRobinPolicy, TokenAwarePolicy
# Optimized connection profileprofile = ExecutionProfile( load_balancing_policy=TokenAwarePolicy(DCAwareRoundRobinPolicy()), request_timeout=15.0, consistency_level=ConsistencyLevel.LOCAL_QUORUM)
cluster = Cluster( ['heliosdb.example.com'], port=9042, execution_profiles={EXEC_PROFILE_DEFAULT: profile}, protocol_version=5, # Use latest protocol compression=True, # Enable compression max_schema_agreement_wait=10)Consistency Level Adjustment:
-- Use local consistency levels for lower latency-- (especially important for multi-region deployments)
-- Before (Cassandra)SELECT * FROM users WHERE user_id = ? USING CONSISTENCY QUORUM;
-- After (HeliosDB - optimized)SELECT * FROM users WHERE user_id = ? USING CONSISTENCY LOCAL_QUORUM;Prepared Statement Caching:
# Cache prepared statements for better performanceprepared_statements = {}
def get_prepared_statement(session, query): if query not in prepared_statements: prepared_statements[query] = session.prepare(query) return prepared_statements[query]
# Usagestmt = get_prepared_statement(session, "SELECT * FROM users WHERE user_id = ?")rows = session.execute(stmt, [user_id])Query Optimization:
-- Before: Secondary index scan (slow)SELECT * FROM orders WHERE customer_email = 'alice@example.com';
-- After: Partition key lookup (fast)-- Option 1: Add materialized view (parser only; execution not yet supported)CREATE MATERIALIZED VIEW orders_by_email AS SELECT * FROM orders WHERE customer_email IS NOT NULL AND order_id IS NOT NULL PRIMARY KEY (customer_email, order_id);
SELECT * FROM orders_by_email WHERE customer_email = 'alice@example.com';
-- Option 2: Use HeliosDB SQL interface for complex queriesSELECT * FROM orders WHERE customer_email = 'alice@example.com'ALLOW FILTERING; -- Faster in HeliosDB due to query optimizerIndex Creation:
-- Create secondary indexes for common queriesCREATE INDEX ON users (email);CREATE INDEX ON orders (status);CREATE INDEX ON events (event_type);
-- Check index usageSELECT * FROM system_schema.indexes WHERE keyspace_name = 'my_keyspace';Feature Mapping
Data Types
| Cassandra Type | HeliosDB Storage | Notes |
|---|---|---|
| ASCII | TEXT | Identical |
| BIGINT | BIGINT | Identical |
| BLOB | BYTEA | Binary data |
| BOOLEAN | BOOLEAN | Identical |
| COUNTER | BIGINT | Special handling for increments |
| DATE | INTEGER | Days since epoch |
| DECIMAL | DECIMAL | Arbitrary precision |
| DOUBLE | DOUBLE PRECISION | 64-bit float |
| DURATION | INTERVAL | Time duration |
| FLOAT | REAL | 32-bit float |
| INET | INET | IP address |
| INT | INTEGER | 32-bit int |
| LIST | JSONB | Stored as JSON array |
| MAP<K,V> | JSONB | Stored as JSON object |
| SET | JSONB | Stored as JSON array (unique) |
| SMALLINT | SMALLINT | 16-bit int |
| TEXT | TEXT | UTF-8 string |
| TIME | TIME | Nanoseconds since midnight |
| TIMESTAMP | BIGINT | Milliseconds since epoch |
| TIMEUUID | UUID | Version 1 UUID |
| TINYINT | SMALLINT | 8-bit int → 16-bit |
| TUPLE | JSONB | Stored as JSON array |
| UUID | UUID | Version 4 UUID |
| UDT | JSONB | User-defined type as JSON |
| VARCHAR | TEXT | Alias for TEXT |
| VARINT | NUMERIC | Arbitrary precision integer |
CQL Features
| Feature | Cassandra | HeliosDB | Compatibility |
|---|---|---|---|
| Queries | |||
| SELECT | 100% | ||
| INSERT | 100% | ||
| UPDATE | 100% | ||
| DELETE | 100% | ||
| BEGIN BATCH | 100% | ||
| Clauses | |||
| WHERE | 100% | ||
| ORDER BY | 100% (clustering only) | ||
| LIMIT | 100% | ||
| ALLOW FILTERING | 100% (faster in HeliosDB) | ||
| IF EXISTS / IF NOT EXISTS | 100% | ||
| IF | Partial | 80% (basic LWT only) | |
| USING TTL | 100% | ||
| USING TIMESTAMP | 100% | ||
| Functions | |||
| uuid() | 100% | ||
| now() | 100% | ||
| toTimestamp() | 100% | ||
| dateOf() | 100% | ||
| token() | 100% | ||
| ttl() | 100% | ||
| writetime() | 100% | ||
| COUNT(*) | 100% | ||
| SUM/AVG/MIN/MAX | 100% | ||
| Advanced | |||
| Prepared Statements | 100% | ||
| Paging | 100% | ||
| LOGGED BATCH | 100% | ||
| UNLOGGED BATCH | 100% | ||
| COUNTER BATCH | 100% | ||
| Static Columns | 100% | ||
| Frozen Collections | 100% | ||
| Materialized Views | Partial | 50% (parser only) | |
| Secondary Indexes | 100% | ||
| UDFs | No | 0% | |
| UDAs | No | 0% | |
| Triggers | No | 0% (not planned) |
Troubleshooting
Common Issues
1. Consistency Level Errors
Problem: UnavailableException: Cannot achieve consistency level QUORUM
Cause: Not enough replicas available
Solution:
-- Use LOCAL_ONE or LOCAL_QUORUM insteadSELECT * FROM users WHERE user_id = ? USING CONSISTENCY LOCAL_ONE;Configuration:
from cassandra.cluster import Cluster, ConsistencyLevel
cluster = Cluster(['heliosdb.example.com'])session = cluster.connect()session.default_consistency_level = ConsistencyLevel.LOCAL_QUORUM # Set default2. Authentication Failures
Problem: AuthenticationException: Username and/or password are incorrect
Cause: PasswordAuthenticator mismatch
Solution:
# Check HeliosDB authentication configurationheliosdb config get auth.authenticator
# Should output: PasswordAuthenticator
# Reset admin passwordheliosdb user reset-password adminApplication Fix:
from cassandra.auth import PlainTextAuthProvider
# Ensure correct username/passwordauth = PlainTextAuthProvider(username='admin', password='correct_password')cluster = Cluster(['heliosdb.example.com'], auth_provider=auth)3. Performance Degradation
Problem: Queries slower in HeliosDB than Cassandra
Diagnosis:
-- Check query planEXPLAIN SELECT * FROM users WHERE email = 'alice@example.com';Solution 1: Missing Indexes
-- Create secondary indexCREATE INDEX ON users (email);Solution 2: Optimize Consistency Level
# Use lower consistency for read-heavy workloadssession.default_consistency_level = ConsistencyLevel.LOCAL_ONESolution 3: Enable Query Cache
-- Enable intelligent caching (HeliosDB feature)ALTER SYSTEM SET intelligent_cache = 'on';ALTER SYSTEM SET cache_policy = 'ml_hybrid';4. Data Type Mismatch
Problem: Collection data not readable after migration
Cause: Frozen vs. non-frozen collections
Solution:
-- Check table schemaDESC TABLE my_keyspace.users;
-- If collections are frozen, they're immutable-- Recreate table with non-frozen collections if neededCREATE TABLE users_v2 ( user_id UUID PRIMARY KEY, tags SET<TEXT>, -- Non-frozen (mutable) metadata MAP<TEXT, TEXT> -- Non-frozen (mutable));5. Timeout Errors
Problem: OperationTimedOut: Operation timed out - received only 0 responses
Cause: Network latency, overloaded cluster, or large result set
Solution:
# Increase timeoutfrom cassandra.cluster import Cluster, ExecutionProfile
profile = ExecutionProfile( request_timeout=30.0 # Increase from default 10s)
cluster = Cluster(['heliosdb.example.com'], execution_profiles={EXEC_PROFILE_DEFAULT: profile})For Large Queries:
# Use paging for large result setsfrom cassandra.query import SimpleStatement
stmt = SimpleStatement("SELECT * FROM large_table", fetch_size=1000)for row in session.execute(stmt): process(row)6. Connection Pool Exhausted
Problem: NoHostAvailable: Unable to complete the operation against any hosts
Cause: Too many concurrent connections
Solution:
# Increase connection pool sizefrom cassandra.cluster import Cluster
cluster = Cluster( ['heliosdb.example.com'], protocol_version=5, executor_threads=4, # Increase thread pool max_schema_agreement_wait=10, control_connection_timeout=10, idle_heartbeat_interval=30, compression=True)Best Practices
1. Start with Non-Production Environments
Recommended Progression:
- Development (1-2 days): Test basic functionality
- Staging (1 week): Full integration testing
- Canary Production (1 week): 10% traffic to HeliosDB
- Production Rollout (2 weeks): Gradual 10% → 25% → 50% → 100%
2. Use Dual-Write for Critical Systems
Advantages:
- Zero downtime
- Easy rollback
- Gradual confidence building
Implementation: See Strategy 1: Dual-Write
3. Monitor Replication Lag During Migration
Key Metrics:
- Replication lag (target: <1 second)
- Write throughput (both clusters)
- Read latency (both clusters)
- Error rates
Monitoring Script:
#!/bin/bashwhile true; do # Check replication lag LAG=$(curl -s http://debezium:8083/connectors/cassandra-heliosdb/status | jq '.lag_seconds')
# Check write throughput CASSANDRA_WRITES=$(nodetool tpstats | grep 'MutationStage' | awk '{print $5}') HELIOSDB_WRITES=$(heliosdb stats writes)
# Print dashboard clear echo "=== Migration Monitor ===" echo "Replication Lag: ${LAG}s" echo "Cassandra Writes: ${CASSANDRA_WRITES}/s" echo "HeliosDB Writes: ${HELIOSDB_WRITES}/s" echo "========================"
sleep 5done4. Keep Cassandra as Fallback Initially
Recommendation: Keep Cassandra cluster running for 1-2 weeks after cutover
Rollback Preparation:
# Keep Cassandra config readily availableexport CASSANDRA_HOST_PRIMARY=heliosdb.example.comexport CASSANDRA_HOST_FALLBACK=cassandra.example.com
# Quick rollback scriptcat > rollback.sh <<'EOF'#!/bin/bashkubectl set env deployment/my-app CASSANDRA_HOST=$CASSANDRA_HOST_FALLBACKkubectl rollout restart deployment/my-appEOFchmod +x rollback.sh5. Test Rollback Procedure
Before Cutover:
# Test rollback in staging1. Switch to HeliosDB2. Verify application works3. Execute rollback script4. Verify application still works with Cassandra6. Document Your Migration
Migration Runbook Template:
# Migration Runbook: Cassandra → HeliosDB
## Pre-Migration Checklist- [ ] HeliosDB cluster provisioned and tested- [ ] Schema migrated and validated- [ ] Data migration method selected- [ ] Rollback procedure tested- [ ] Monitoring dashboards configured- [ ] On-call team notified
## Migration Steps1. [ ] Enable dual-write (or start replication)2. [ ] Begin data backfill3. [ ] Validate data consistency4. [ ] Route 10% read traffic to HeliosDB5. [ ] Monitor for 24 hours6. [ ] Route 50% read traffic to HeliosDB7. [ ] Monitor for 24 hours8. [ ] Route 100% traffic to HeliosDB9. [ ] Monitor for 1 week10. [ ] Decommission Cassandra
## Rollback Procedure1. Execute: ./rollback.sh2. Verify traffic routing to Cassandra3. Monitor error rates4. Investigate HeliosDB issue
## Success Criteria- Row counts match (100%)- Query latency < baseline + 10%- Error rate < 0.01%- No data lossRollback Plan
Scenario 1: Issues During Dual-Write Phase
Symptoms: Data inconsistency, high error rates
Action:
# 1. Stop dual-write# Comment out dual-write code or set environment variableexport ENABLE_DUAL_WRITE=false
# 2. Restart applicationkubectl rollout restart deployment/my-app
# 3. Continue writing to Cassandra only# 4. Investigate HeliosDB issuesheliosdb logs --tail 1000 | grep ERROR
# 5. Retry after fixesImpact: No downtime (still using Cassandra)
Scenario 2: Issues After Partial Cutover
Symptoms: Performance degradation, query errors
Action:
# 1. Immediately route all traffic back to Cassandrakubectl set env deployment/my-app CASSANDRA_HOST=cassandra.example.comkubectl rollout restart deployment/my-app
# 2. Verify application healthkubectl logs -f deployment/my-app
# 3. Analyze HeliosDB issuesheliosdb diagnose --export /tmp/diagnostics.tar.gz
# 4. Contact support if neededImpact: Brief downtime during restart (2-5 minutes)
Scenario 3: Data Loss Detected
Symptoms: Row count mismatch, missing records
Action:
# 1. IMMEDIATE ROLLBACK./rollback.sh
# 2. Stop all writes to HeliosDBheliosdb cluster pause --writes-only
# 3. Export HeliosDB data for forensicscqlsh heliosdb.example.com -e "COPY my_keyspace.users TO '/tmp/heliosdb_users.csv'"
# 4. Compare with Cassandradiff <(wc -l /tmp/cassandra_users.csv) <(wc -l /tmp/heliosdb_users.csv)
# 5. Re-run migration with identified fixesImpact: Rollback to Cassandra (5-10 minutes downtime)
Case Studies
Case Study 1: E-Commerce Platform (500M Daily Writes)
Customer: Large e-commerce company
Cassandra Setup:
- 30-node cluster
- 100 TB data
- 500M writes/day
- 2B reads/day
Migration Strategy: Dual-write
Timeline:
- Week 1: Schema migration, initial backfill
- Week 2: Dual-write enabled, 10% read traffic
- Week 3: 50% read traffic
- Week 4: 100% read traffic, write migration
- Week 5: Cassandra decommissioned
Results:
- Zero downtime
- 2.5x faster analytics queries
- No data loss
Key Success Factors:
- Comprehensive monitoring
- Gradual rollout
- 24/7 on-call support during migration
Case Study 2: IoT Sensor Network (10K Devices)
Customer: Industrial IoT platform
Cassandra Setup:
- 6-node cluster
- 20 TB time-series data
- 10K devices × 1 msg/sec = 10K writes/sec
Migration Strategy: Replication-based
Timeline:
- Day 1: Schema migration
- Day 2-3: Initial data replication (20 TB)
- Day 4: Replication caught up (lag < 1s)
- Day 5: Cutover (3-minute downtime)
- Day 6-12: Monitoring
Results:
- 3-minute downtime
- 10x faster downsampling queries
- Successful migration
Key Success Factors:
- Low downtime tolerance (not 24/7 critical)
- Time-series data (less complex than transactional)
- HeliosDB edge sync for IoT devices
Case Study 3: SaaS Multi-Tenant Application
Customer: B2B SaaS provider (5000 tenants)
Cassandra Setup:
- 12-node cluster
- 50 TB data
- Multi-tenant (5000 tenants)
Migration Strategy: Tenant-by-tenant migration
Timeline:
- Week 1-2: Migrate 10 test tenants
- Week 3-4: Migrate 100 small tenants
- Week 5-6: Migrate 500 medium tenants
- Week 7-10: Migrate 4390 remaining tenants
Results:
- Zero downtime (per-tenant migration)
- Multi-protocol access (CQL + SQL)
- Tenant isolation improved
Key Success Factors:
- Tenant-by-tenant rollout reduced risk
- Automated migration tooling
- HeliosDB multi-tenancy features
Next Steps
1. Join HeliosDB Community
Resources:
- Documentation: https://www.heliosdb.com/docs/full/
- Support: support@heliosdb.com
2. Schedule Migration Planning Call
Contact: support@heliosdb.com
Topics:
- Migration strategy selection
- Timeline estimation
- Resource requirements
- Risk assessment
3. Access Enterprise Features
Enterprise Add-Ons:
- 24/7 migration support
- Dedicated migration engineer
- Advanced monitoring and alerting
- Custom tooling development
- SLA guarantees
Contact Sales: sales@heliosdb.com
4. Training and Certification
HeliosDB Academy:
- Cassandra to HeliosDB Migration (4-hour course)
- HeliosDB Administration (8-hour course)
- Multi-Protocol Database Design (4-hour course)
Enquiries: sales@heliosdb.com
Appendix A: Glossary
| Term | Definition |
|---|---|
| CQL | Cassandra Query Language |
| Keyspace | Top-level data container (similar to database) |
| Partition Key | Column(s) determining data distribution across nodes |
| Clustering Key | Column(s) determining sort order within partition |
| Consistency Level | How many replicas must respond to a query (ONE, QUORUM, ALL) |
| LWT | Lightweight Transactions (compare-and-set operations) |
| UDT | User-Defined Type (custom data type) |
| TTL | Time-to-Live (automatic data expiration) |
| SSTable | Sorted String Table (Cassandra’s on-disk format) |
| Compaction | Background process to merge SSTables |
Appendix B: Client Driver Compatibility
| Driver | Version | Notes |
|---|---|---|
| DataStax Python Driver | 3.25+ | Native protocol v4/v5; validate with your workload |
| DataStax Java Driver | 4.15+ | Native protocol v4/v5; validate with your workload |
| DataStax C++ Driver | 2.16+ | Native protocol v4/v5; validate with your workload |
| DataStax Node.js Driver | 4.6+ | Native protocol v4/v5; validate with your workload |
| DataStax C# Driver | 3.18+ | Native protocol v4/v5; validate with your workload |
| GoCQL | 1.0+ | Native protocol v4/v5; validate with your workload |
| Rust Driver (Scylla) | 0.9+ | Native protocol v4/v5; validate with your workload |
| cqlsh | 5.0.1+ | Native protocol v4/v5; validate with your workload |
Appendix C: HeliosDB-Specific Enhancements
1. SQL Interface on CQL Data
-- Query Cassandra data with SQLSELECT u.username, COUNT(o.order_id) AS order_countFROM my_keyspace.users uLEFT JOIN my_keyspace.orders o ON u.user_id = o.user_idGROUP BY u.usernameHAVING COUNT(o.order_id) > 10ORDER BY order_count DESC;2. Intelligent Query Caching
-- Enable ML-based query caching (HeliosDB feature)ALTER SYSTEM SET intelligent_cache = 'on';
-- Cached queries are automatically faster on subsequent runsSELECT * FROM large_table WHERE status = 'active'; -- 500ms (first run)SELECT * FROM large_table WHERE status = 'active'; -- 2ms (cached)3. Multi-Protocol Access
# Access same data via CQL and PostgreSQL protocolsfrom cassandra.cluster import Clusterimport psycopg2
# CQL protocolcql_cluster = Cluster(['heliosdb.example.com'], port=9042)cql_session = cql_cluster.connect('my_keyspace')cql_session.execute("INSERT INTO users (user_id, name) VALUES (?, ?)", [uuid4(), 'Alice'])
# PostgreSQL protocol (same data!)pg_conn = psycopg2.connect(host='heliosdb.example.com', port=5432, database='my_keyspace')pg_cursor = pg_conn.cursor()pg_cursor.execute("SELECT * FROM users WHERE name = %s", ['Alice'])print(pg_cursor.fetchall())Document Information
Maintainers: HeliosDB Migration Team Feedback: support@heliosdb.com
Related Documentation: