--- title: "Dump and Restore Procedures Guide" --- # Dump and Restore Procedures Guide ## Table of Contents 1. [Overview](#overview) 2. [Dump Operations](#dump-operations) 3. [Restore Operations](#restore-operations) 4. [In-Memory Databases](#in-memory-databases) 5. [Scheduling Dumps](#scheduling-dumps) 6. [Best Practices](#best-practices) 7. [Troubleshooting](#troubleshooting) ## Overview `heliosdb-lite dump` writes a database to a compressed dump file, and `heliosdb-lite restore` loads a dump file into a database. Both work in two modes: | Mode | Dump | Restore | |------|------|---------| | **Embedded** | `--data-dir` reads a data directory directly | `--target` writes a new data directory | | **Server** | `--connection` runs `BACKUP DATABASE` on a running server; the file is written on the server's filesystem | `--connection` runs `RESTORE DATABASE` on a running server; the file must be readable by the server | Dump files are compressed (Zstandard by default) but not encrypted. See the [CLI Reference](/docs/lite/reference/cli/#dump) for every flag. ## Dump Operations ### Basic Dump ```bash # Full dump of a data directory (stop the server first for a consistent copy) heliosdb-lite dump --data-dir /var/lib/heliosdb --output backup.heliodump # Choose the compression: zstd (default), gzip, brotli or none heliosdb-lite dump --data-dir /var/lib/heliosdb --output backup.heliodump --compression gzip # Selected tables only (embedded mode) heliosdb-lite dump --data-dir /var/lib/heliosdb --output users.heliodump --tables users,sessions # Progress output heliosdb-lite dump --data-dir /var/lib/heliosdb --output backup.heliodump --verbose ``` ### Dump Sizes and Throughput Figures from internal tests (HeliosDB Lite dump and restore of a 1 GB database, Zstandard vs no compression); reproduce on your own hardware and data: | Compression | Typical ratio | Speed | |-------------|---------------|-------| | `none` | 1.0x | Fastest | | `zstd` (default) | 3.0-5.0x | Medium | | Operation | Size | Time | Throughput | |-----------|------|------|------------| | Dump (no compression) | 1 GB | 15s | 66 MB/s | | Dump (zstd) | 1 GB → 300 MB | 35s | 28 MB/s | | Restore (no compression) | 1 GB | 25s | 40 MB/s | | Restore (zstd) | 300 MB → 1 GB | 40s | 25 MB/s | An uncompressed dump is about 90% of the in-memory size, a Zstandard dump 20-40% of the uncompressed one, and a daily incremental dump 5-15% of a full dump. ### Incremental Dumps ```bash # First dump (full) heliosdb-lite dump --data-dir /var/lib/heliosdb --output base.heliodump # Later: append the changes since the previous dump heliosdb-lite dump --data-dir /var/lib/heliosdb --output base.heliodump --append ``` ### Dumping a Running Server ```bash heliosdb-lite dump \ --connection "host=db.example.com port=5432 user=postgres" \ --tls --tls-ca ca.crt \ --output backup.heliodump ``` The server writes the dump on its own filesystem and reports the dump ID; copy the file from the server host. `--tables` is not supported in server mode. The same operation is available in SQL to a role with maintenance privileges: ```sql BACKUP DATABASE TO '/backups/full.hldb'; BACKUP DATABASE INCREMENTAL TO '/backups/inc.hldb'; ``` Server-mode backups contain table definitions and rows. Recreate indexes, roles, grants and row-level security policies after restoring one, and pause writes while it runs for a consistent copy. ## Restore Operations ### Basic Restore ```bash # Restore into a new data directory, checking the dump first heliosdb-lite restore --input backup.heliodump --target /var/lib/heliosdb-restored --verify # Restore selected tables heliosdb-lite restore --input backup.heliodump --target ./restored --tables users,sessions ``` Without `--target`, the restore goes to `./heliosdb-data-restored`. Start a server on the restored directory with `heliosdb-lite start --data-dir `. ### Restoring into a Running Server ```bash heliosdb-lite restore \ --connection "host=db.example.com port=5432 user=postgres" \ --tls --tls-ca ca.crt \ --input /backups/backup.heliodump ``` The path must be readable by the server process. ## In-Memory Databases In-memory servers and REPL sessions can write a dump when they shut down: ```bash # Server export HELIOSDB_JWT_SECRET="$(openssl rand -hex 32)" heliosdb-lite start --memory --dump-on-shutdown # REPL heliosdb-lite repl --memory --dump-on-shutdown --dump-file ./session.heliodump ``` ## Scheduling Dumps The `--dump-schedule` flag of `heliosdb-lite start` is planned and not yet acted on. Schedule dumps with cron or a systemd timer: ```bash # crontab: nightly dump at 02:00 from a running server 0 2 * * * heliosdb-lite dump --connection "host=127.0.0.1 port=5432 user=postgres" --output /backups/nightly.heliodump ``` ## Best Practices 1. **Consistency**: dump a stopped data directory, or pause writes during the dump. 2. **Encryption**: dump files are not encrypted; encrypt them before they leave the host (for example `gpg --symmetric --cipher-algo AES256 backup.heliodump`). 3. **Off-host copies**: keep copies on another machine or in object storage. 4. **Test restores**: restore into a scratch directory regularly and compare row counts with the source. 5. **Verify after restore**: check tables, roles, grants and row-level security policies before you rely on a restored database. ## Troubleshooting ### "Either --data-dir or --connection must be specified" Pass the data directory to dump, or a connection string for a running server. ### "--tables filtering is not supported in server mode yet" Use embedded mode (`--data-dir`) for per-table dumps. ### Restore Fails with a Checksum or Format Error Run `heliosdb-lite restore --verify` on the file. If the file is damaged, restore from an older copy. Restore dumps with a release at least as new as the one that wrote them. ## See Also - [In-Memory Mode Guide](https://www.heliosdb.com/docs/lite/guides/user/inmemory_mode/) - In-memory database operations - [CLI Reference](/docs/lite/reference/cli/) - Complete command reference - [Configuration reference](/docs/lite/user-guide/configuration/) - Configuration settings