---
title: "High Availability (HA)"
description: "HeliosDB Lite WAL streaming replication: primary and standby flags, sync modes, write forwarding from standbys, replication views, and what is planned."
---

# High Availability (HA)

HeliosDB Lite replicates by streaming its write-ahead log (WAL) from a primary to one
or more standbys. Clients can read from any node, and a `sync` or `semi-sync` standby
forwards the writes it receives to the primary, within the limits described in
[Write Forwarding from Standbys](#write-forwarding-from-standbys).

## What Is Available and What Is Planned

| Capability | Status |
|------------|--------|
| WAL streaming: primary and standby roles | Available (build with the `ha-tier1` feature) |
| Sync modes `async`, `semi-sync`, `sync` | Available |
| Write forwarding from `sync` and `semi-sync` standbys | Available, with the limits below |
| Replication system views, `SHOW TOPOLOGY`, `SHOW CLUSTER STATUS`, `SWITCHOVER CHECK` | Available |
| Controlled switchover (`SWITCHOVER TO`) | Planned: the statement is accepted but does not change roles |
| Automatic failover / promotion | Planned; use an external HA manager |
| Observer nodes for split-brain protection | Planned: the `observer` role starts no service |
| Multi-primary replication | Planned |
| Sharding | Planned |
| Logical replication | Planned |

## Building with Replication

Replication is a build-time feature of the source distribution:

```bash
cargo build --release --features ha-tier1
```

A binary built without it refuses to start with any `--replication-role` other than
`standalone`. The source distribution marks the replication feature as experimental, and
the stock Dockerfile builds without it.

## Architecture

```
┌─────────────┐     WAL Stream    ┌─────────────┐
│   Primary   │ ───────────────→ │   Standby   │
│   (Active)  │                   │  (Passive)  │
└─────────────┘                   └─────────────┘
       ↓                                 ↓
   Read/Write                  Reads local, writes routed
```

Standbys connect to the primary's replication port; the primary accepts up to 10
standbys and needs no list of them.

## Starting a Cluster

```bash
export HELIOSDB_JWT_SECRET="$(openssl rand -hex 32)"

# Primary
heliosdb-lite start --data-dir /var/lib/heliosdb --listen 0.0.0.0 \
  --auth scram-sha-256 --password "$HELIOSDB_ADMIN_PASSWORD" \
  --replication-role primary --replication-port 5433 --sync-mode semi-sync

# Standby
heliosdb-lite start --data-dir /var/lib/heliosdb --listen 0.0.0.0 \
  --auth scram-sha-256 --password "$HELIOSDB_ADMIN_PASSWORD" \
  --replication-role standby --primary-host primary.example.com:5433
```

With password or SCRAM authentication on the primary, send writes to the primary (or
through HeliosProxy): write forwarding from a standby needs `trust` authentication on the
primary, see [Write Forwarding from Standbys](#write-forwarding-from-standbys).

| Flag | Description | Default |
|------|-------------|---------|
| `--replication-role` | `standalone`, `primary`, `standby` or `observer` | `standalone` |
| `--replication-port` | WAL streaming port | `5433` |
| `--primary-host` | Primary a standby connects to (`host:port` of its replication port) | none |
| `--standby-hosts` | Accepted and printed at startup; standbys connect to the primary without it | none |
| `--observer-hosts` | Accepted; observer-based split-brain protection is planned | none |
| `--sync-mode` | `async`, `semi-sync` or `sync` | `async` |
| `--node-id` | Node UUID, generated when omitted | generated |
| `--replication-tls` | Encrypt WAL traffic | on when `--tls-cert` is set |

### Sync Modes

| Mode | On the primary | On a standby |
|------|----------------|--------------|
| `sync`, `semi-sync` | Each change waits until one standby confirms it has received it. If no standby confirms within 10 seconds, the change completes on the primary and a warning is logged. | Forwards writes to the primary |
| `async` | Changes do not wait for standbys | Rejects writes (SQLSTATE `25006`) |

With no standby connected, a `sync` or `semi-sync` primary waits the full 10 seconds on
every change, so writes slow down until a standby reconnects.

## Docker Compose

The [Docker Deployment guide](/docs/lite/guides/docker_deployment/) has a primary and
standby Compose file built from the source distribution.

## Write Forwarding from Standbys

A standby in `sync` or `semi-sync` mode forwards writes it receives to the primary, so a
client connected to that standby can write without a second connection to the primary.

### How It Works

```
Application → Standby → (DML/DDL forwarded) → Primary
                ↓
         (SELECT executed locally)
```

### Behavior by Sync Mode

| Sync Mode | DQL (SELECT) | DML (INSERT/UPDATE/DELETE) |
|-----------|--------------|----------------------------|
| **sync** | Execute locally on standby | Forward to primary, return result |
| **semi-sync** | Execute locally on standby | Forward to primary, return result |
| **async** | Execute locally on standby | Reject (traditional read-only) |

### Operations Subject to Routing

When connected to a standby in sync/semi-sync mode:

| Operation | Behavior |
|-----------|----------|
| `SELECT` | Execute locally (DQL) |
| `INSERT` | Forward to primary (DML) |
| `UPDATE` | Forward to primary (DML) |
| `DELETE` | Forward to primary (DML) |
| `CREATE` | Forward to primary (DDL) |
| `DROP` | Forward to primary (DDL) |
| `ALTER` | Forward to primary (DDL) |
| `TRUNCATE` | Forward to primary (DDL) |

### Limits

- Only statements sent with the simple query protocol (for example from `psql`) are
  forwarded. Statements sent with the extended query protocol (prepared statements, which
  most drivers use for parameterized queries) and REST API calls to a standby are not
  forwarded; send application writes to the primary or through HeliosProxy.
- The standby connects to the primary's PostgreSQL port without a password and runs the
  forwarded statement as the `heliosdb` role, not the client's role, so forwarding works
  only while the primary accepts `trust` authentication. The primary's PostgreSQL port is
  taken as `5432` unless the standby's environment sets `HELIOSDB_PRIMARY_PG_PORT`.

### Example

```sql
-- Connect to STANDBY and execute INSERT (forwarded to primary)
INSERT INTO users VALUES (3, 'Charlie');
-- Result: INSERT 0 1 (success - executed on primary)

-- SELECT always executes locally on the connected standby
SELECT * FROM users;
```

For applications, HeliosProxy in front of the cluster is the simpler route: it sends
writes to the primary and spreads reads across healthy nodes for every protocol; see the
[HA hands-on tutorial](/docs/lite/guides/ha_hands_on_tutorial/).

---

## Monitoring

### HA System Views

HeliosDB Lite has SQL system views for the HA configuration and replication metrics.

#### pg_replication_status

View node configuration and role:

```sql
SELECT * FROM pg_replication_status;
```

| Column | Description |
|--------|-------------|
| `node_id` | Unique identifier for this node |
| `role` | primary, standby, observer, or standalone |
| `sync_mode` | async, semi-sync, or sync |
| `is_read_only` | true on a standby |
| `current_lsn` | Current log sequence number |
| `standby_count` | Number of connected standbys (primary only) |
| `primary_host` | Primary this node replicates from (standby only) |
| `listen_addr` | Listen address |
| `replication_port` | WAL streaming port |
| `started_at` | Time the node started |

#### pg_replication_standbys (Primary Only)

View connected standbys:

```sql
SELECT * FROM pg_replication_standbys;
```

| Column | Description |
|--------|-------------|
| `node_id` | Standby's unique identifier |
| `address` | Standby's connection address |
| `sync_mode` | Replication mode for this standby |
| `state` | connecting, streaming, catching_up, synced, disconnected |
| `current_lsn` | Standby's current LSN position |
| `flush_lsn` | Flushed LSN |
| `apply_lsn` | Applied LSN |
| `lag_bytes` | Replication lag in bytes |
| `lag_ms` | Replication lag in milliseconds |
| `connected_at` | Connection timestamp |
| `last_heartbeat` | Last heartbeat received |

#### pg_replication_primary (Standby Only)

View primary connection status:

```sql
SELECT * FROM pg_replication_primary;
```

| Column | Description |
|--------|-------------|
| `primary_node_id` | Primary's unique identifier |
| `primary_address` | Primary's address |
| `connection_state` | disconnected, connecting, connected, streaming, error |
| `primary_lsn` | Primary's current LSN |
| `local_lsn` | Local LSN position |
| `lag_bytes` | Replication lag in bytes |
| `lag_ms` | Replication lag in milliseconds |
| `fencing_token` | Split-brain protection token |
| `connected_at` | Connection timestamp |
| `last_heartbeat` | Last heartbeat received |

#### pg_replication_metrics

View replication metrics, one row per metric with the columns `metric_name`,
`metric_value` and `description`:

```sql
SELECT * FROM pg_replication_metrics;
```

| Metric | Description |
|--------|-------------|
| `wal_writes` | Total WAL write operations |
| `wal_bytes_written` | Total WAL bytes written |
| `records_replicated` | Records sent to standbys |
| `bytes_replicated` | Bytes sent to standbys |
| `heartbeats_sent` | Heartbeats sent |
| `heartbeats_received` | Heartbeats received |
| `reconnect_count` | Number of reconnections |
| `current_lsn` | Current log sequence number |
| `standby_count` | Connected standbys |
| `last_wal_write_epoch` | Time of the last WAL write (Unix epoch) |
| `last_replication_epoch` | Time of the last replication (Unix epoch) |
| `ha_enabled` | Whether replication is active on this node |

### Monitoring Examples

```sql
-- Check if standbys are in sync
SELECT
    node_id,
    CASE
        WHEN lag_ms < 1000 THEN 'IN_SYNC'
        WHEN lag_ms < 60000 THEN 'CATCHING_UP'
        ELSE 'LAGGING'
    END as status,
    lag_ms
FROM pg_replication_standbys;

-- This node's role and position
SELECT node_id, role, current_lsn
FROM pg_replication_status;

-- All nodes known to this node
SHOW TOPOLOGY;
```

---

## Failover

Automatic failover is planned. Today, promotion is performed by an external HA manager
(or by an operator): stop writes to the failed primary, restart the chosen standby as
`--replication-role primary`, and point the other standbys at it. HeliosProxy can route
clients to the current primary; see the
[HeliosProxy Integration Guide](/docs/lite/guides/heliosproxy_integration_guide/).

## Best Practices

1. **Network**: Use a dedicated replication network, and `--replication-tls` (on by default when `--tls-cert` is set)
2. **Monitoring**: Alert when `lag_ms` in `pg_replication_standbys` exceeds your threshold
3. **Testing**: Rehearse the manual promotion procedure in a staging environment
4. **Backups**: Keep taking backups; replication does not replace them

## See Also

- [Configuration](https://www.heliosdb.com/docs/lite/user-guide/configuration/)
- [Deployment Modes](https://www.heliosdb.com/docs/lite/getting-started/deployment/)
- [Multi-Tenancy](https://www.heliosdb.com/docs/lite/features/multi-tenancy/)