Skip to content
HeliosDB HeliosDB Lite

Backup and Recovery

Backup and Recovery

HeliosDB-Lite supports full and incremental backups with a chain-based restore model. Backups are self-contained binary files that include data, metadata, and counters. Incremental backups build on top of a previous backup to reduce size and time.

Prerequisites

  • HeliosDB-Lite v3.5+
  • Write access to the backup directory on the filesystem
  • Enough disk space for the backup files

Concepts

  • Full backup — Copies every data key, table schema, and counter to a single file. Self-contained and can be restored independently.
  • Incremental backup — Copies only the data that exists at the time of the backup but links to the most recent previous backup as its parent. Restoring requires the full chain from the base full backup through every incremental backup in order.
  • Backup chain — A full backup followed by zero or more incremental backups. Restore walks the chain from the full backup forward, applying each layer.
  • Backup metadata — Stored in the database under meta:backup:{id} keys. Contains the backup ID, type, timestamp, parent reference, key count, size, and file path.

Step 1: Create Test Data

CREATE TABLE products (
    id INT PRIMARY KEY,
    name TEXT,
    price DECIMAL
);


INSERT INTO products VALUES (1, 'Widget', 9.99);
INSERT INTO products VALUES (2, 'Gadget', 24.99);
INSERT INTO products VALUES (3, 'Gizmo', 14.99);

Step 2: Full Backup

BACKUP DATABASE TO '/tmp/helios_backups/full_001.hldb';

Expected output:

col0col1col2col3col4col5col6
BACKUPbk_1711400000full82048/tmp/helios_backups/full_001.hldb3

The columns represent: operation, backup ID, type, key count, size in bytes, file path, and duration in milliseconds.

If you omit the TO clause, the backup file is placed in the system temp directory with an auto-generated name:

BACKUP DATABASE;

Step 3: Make Changes

INSERT INTO products VALUES (4, 'Doohickey', 39.99);
UPDATE products SET price = 11.99 WHERE id = 1;
DELETE FROM products WHERE id = 3;

Step 4: Incremental Backup

An incremental backup records the current state and links to the most recent backup as its parent:

BACKUP DATABASE INCREMENTAL TO '/tmp/helios_backups/inc_001.hldb';

Expected output:

col0col1col2col3col4col5col6
BACKUPbk_1711403600incremental71800/tmp/helios_backups/inc_001.hldb2

The incremental backup is typically smaller and faster than a full backup.

Step 5: Restore from a Full Backup

Restoring replays backup data into the active database:

RESTORE DATABASE FROM 'bk_1711400000';

Expected output:

col0col1col2col3col4col5
RESTOREbk_1711400000820485main

Columns: operation, backup ID, keys restored, bytes restored, duration, target branch.

After this restore the database reflects the state at the time of the full backup (the Widget is still $9.99, Gizmo still exists, Doohickey does not).

Step 6: Restore to a Branch

To avoid overwriting the current data, restore into a new branch:

RESTORE DATABASE FROM 'bk_1711400000' TO BRANCH 'restored_v1';

The data is written to the restored_v1 branch. You can switch to it or query it without affecting the main branch.

Step 7: Restore an Incremental Backup

When you restore an incremental backup, HeliosDB automatically resolves the full chain:

RESTORE DATABASE FROM 'bk_1711403600';

Internally, the engine:

  1. Finds the incremental backup metadata.
  2. Walks the parent_id chain back to the base full backup.
  3. Reads the full backup first, then applies each incremental layer in order.
  4. Writes the final merged state to the database.

Step 8: View Backup History

The backup metadata is stored in the database. Query it through storage keys or by listing backups:

SELECT * FROM helios_backups;
backup_idbackup_typetimestampparent_idkey_countsize_bytespath
bk_1711400000full171140000082048/tmp/helios_backups/full_001.hldb
bk_1711403600incremental1711403600bk_171140000071800/tmp/helios_backups/inc_001.hldb
  1. Daily full backup at off-peak hours.
  2. Hourly incremental backups during business hours.
  3. Retain at least 7 days of full backups and their chains.
  4. Test restores regularly by restoring to a branch and verifying data.
-- Morning: full backup
BACKUP DATABASE TO '/backups/daily/full_20260326.hldb';


-- Hourly: incremental
BACKUP DATABASE INCREMENTAL TO '/backups/hourly/inc_20260326_10.hldb';
BACKUP DATABASE INCREMENTAL TO '/backups/hourly/inc_20260326_11.hldb';

Common Pitfalls

  1. Missing backup files — The backup file path is stored in metadata. If you move or delete the file, RESTORE will fail with an IO error. Keep backup files in a stable location.

  2. Chain integrity — An incremental backup depends on its parent. If any backup in the chain is corrupted or missing, the restore fails. Always keep the full base backup accessible.

  3. Restoring overwrites data — A plain RESTORE writes into the current branch. Use TO BRANCH to isolate restored data from production.

  4. Backup IDs are timestamp-based — The auto-generated ID is bk_{timestamp}. If you create two backups in the same millisecond, the second overwrites the first metadata. In practice this is unlikely, but use distinct paths.

  5. Disk space — Full backups include all data and metadata keys. Monitor disk usage on the backup volume, especially with large databases.