--- title: "Production Deployment: Backup & Disaster Recovery" --- # Production Deployment: Backup & Disaster Recovery **Part of**: [Production Deployment Guide](https://www.heliosdb.com/docs/full/deployment/production_deployment_guide/) HeliosDB Full keeps all of its state in one data directory (`--data-dir`, environment `HELIOSDB_STORAGE_DATA_DIR`; `/var/lib/heliosdb/data` in the container image): the write-ahead log under `wal/`, the SSTables, and the catalog. A backup is therefore a consistent copy of that directory, plus the server's flags and environment file so the copy can be started the same way. This page describes the methods that work with the current server and how to restore from them. --- ## 8.1 Backup Strategies ### Durability you get without a backup With the default `--synchronous-commit on`, each acknowledged write is fsynced to the WAL before the client sees success. After a crash, power loss or `kill -9`, the server replays the WAL on start and every committed transaction is present. With `--synchronous-commit off`, the loss window after a crash is bounded by `--wal-flush-interval-ms` (default 5 ms). Neither protects against losing the disk or deleting data by mistake; that is what backups are for. ### Method 1: Cold copy (simplest, fully consistent) Stop the server, copy the directory, start it again. On `SIGTERM` the server flushes its memtable and logs `Storage flushed successfully — all data persisted` before it exits. ```bash systemctl stop heliosdb # or: docker stop --time 60 heliosdb tar -C /var/lib -czf /backup/heliosdb-$(date +%Y%m%d-%H%M).tar.gz heliosdb systemctl start heliosdb ``` Use this when a short maintenance window is acceptable. ### Method 2: Storage snapshot (online, crash-consistent) An atomic snapshot of the volume that holds the data directory captures the same state a power loss would leave, which the WAL replay recovers. Take it with the storage layer you already run: ```bash # LVM lvcreate --snapshot --size 20G --name heliosdb-snap /dev/vg0/heliosdb mount -o ro /dev/vg0/heliosdb-snap /mnt/heliosdb-snap tar -C /mnt/heliosdb-snap -czf /backup/heliosdb-$(date +%Y%m%d-%H%M).tar.gz . umount /mnt/heliosdb-snap && lvremove -y /dev/vg0/heliosdb-snap # ZFS zfs snapshot tank/heliosdb@$(date +%Y%m%d-%H%M) # Cloud block storage: an EBS / Persistent Disk / Managed Disk snapshot of the data volume # Kubernetes: a VolumeSnapshot of the data PersistentVolumeClaim ``` The snapshot must cover the whole data directory, including `wal/`, in one atomic operation. A file-by-file copy of a running server (`cp`, `rsync` without a snapshot) is not consistent; use Method 1 or 2. ### Method 3: Logical export (selected tables, portable) Export table data with any client and keep the DDL under version control: ```bash # Data, one CSV file per table psql "host=db.internal user=postgres" --csv -c "SELECT * FROM orders" > orders.csv # Re-import into HeliosDB (COPY ... FROM STDIN is supported) psql "host=db.internal user=postgres" -c "\copy orders FROM 'orders.csv' WITH (FORMAT csv, HEADER true)" ``` `COPY ... TO STDOUT` (and therefore `\copy ... TO`) is not available in the current release; use `SELECT` with `--csv` as above, or your ETL tool. ### Schedule and retention A common pattern: a nightly snapshot or cold copy kept for 7 to 30 days, a weekly copy moved to another region or account, and a logical export of the most important tables before each schema change. Encrypt backups at rest in the backup target (object-store server-side encryption, an encrypted backup volume, or `gpg`/`age` on the archive) and store the environment file with the server's secrets separately from the data. ```bash # Example: nightly cold copy to object storage (cron, 02:30) 30 2 * * * systemctl stop heliosdb && \ tar -C /var/lib -czf - heliosdb | aws s3 cp - s3://example-heliosdb-backups/$(hostname)/$(date +\%F).tar.gz && \ systemctl start heliosdb ``` --- ## 8.2 Restore and Point-in-Time Recovery ### Restore a copy ```bash systemctl stop heliosdb mv /var/lib/heliosdb /var/lib/heliosdb.old mkdir -p /var/lib/heliosdb tar -C /var/lib -xzf /backup/heliosdb-20261001-0230.tar.gz chown -R heliosdb:heliosdb /var/lib/heliosdb systemctl start heliosdb psql "host=127.0.0.1 user=postgres" -c "SELECT COUNT(*) FROM orders" ``` The restored server comes up with the data as of the copy. If you restore onto another host, start it with the same flags and the same `HELIOSDB_SUPERUSER_PASSWORD` and listener credentials. ### Point in time The recovery points are the copies you keep. The current server has no WAL archiving or `AS OF` time travel, so restoring to an arbitrary moment between two backups is not available; shorten the backup interval (snapshots are cheap) if you need a smaller recovery point objective. Statements such as `SHOW BACKUP STATUS`, `CREATE SNAPSHOT` or `ALTER SYSTEM SET wal_archiving` return SQLSTATE `0A000`. ### Test restores Restore a recent backup into a scratch directory on a regular schedule and check it: ```bash mkdir /tmp/restore-test && tar -C /tmp/restore-test -xzf /backup/latest.tar.gz HELIOSDB_SUPERUSER_PASSWORD="$(cat /run/secrets/heliosdb-superuser)" \ heliosdb-full --data-dir /tmp/restore-test/heliosdb --postgres-port 55432 --no-http & psql "host=127.0.0.1 port=55432 user=postgres" -c "SELECT COUNT(*) FROM orders" kill %1 ``` --- ## 8.3 Disaster Recovery Across Sites The `heliosdb-full` server runs as a single node: it does not stream replication or fail over to another node by itself (`SHOW REPLICATION STATUS` and `FAILOVER` return `0A000`). Build site-level recovery from the backup methods above: - **Warm standby by restore.** Ship snapshots or cold copies to the second site on a schedule. To fail over, restore the latest copy there and start the server; point clients at it through DNS or your load balancer. - **Replicated storage.** Run the data directory on storage that replicates synchronously between zones (for example a regional persistent disk). After a zone failure, start the server in the surviving zone on the same volume. Only one server may use a data directory at a time. - **Application-level dual writes or change feeds** where the application already has them. Document the recovery steps, the recovery point (backup interval) and the expected recovery time, and rehearse them. --- ## Navigation - **Previous**: [Security Hardening](https://www.heliosdb.com/docs/full/deployment/production_deployment_security/) - **Next**: [Troubleshooting](https://www.heliosdb.com/docs/full/deployment/production_deployment_troubleshooting/) - **Index**: [Production Deployment Guide](https://www.heliosdb.com/docs/full/deployment/production_deployment_guide/)