Skip to content

HeliosProxy Configuration Reference

HeliosProxy Configuration Reference

Complete reference for HeliosProxy (heliosdb-proxy) configuration. HeliosProxy uses TOML for its configuration file. The authoritative parser is ProxyConfig (and its section structs) in src/config.rs; every key documented here is a real field of that type. Keys, sections, and defaults are kept in sync with the code — if a section is not listed here, it is not part of ProxyConfig and is silently ignored on load (see Unknown Keys).


Usage

Terminal window
# Start with a configuration file
heliosdb-proxy --config /etc/heliosproxy/config.toml
# Start with command-line arguments (no config file)
heliosdb-proxy \
--listen 0.0.0.0:6432 \
--admin 127.0.0.1:9090 \
--primary db-primary:5432 \
--standby db-standby-1:5432 \
--standby db-standby-2:5432
# Override log level
heliosdb-proxy --config config.toml --log-level debug
# Emit JSON-structured logs
heliosdb-proxy --config config.toml --json-logs

Command-Line Arguments

ArgumentDefaultDescription
--config, -c(none)Path to TOML configuration file.
--listen, -l0.0.0.0:5432Client (PostgreSQL-wire) listen address.
--admin127.0.0.1:9090Admin API listen address. Loopback by default — see Admin API Security.
--primary(none)Primary node host:port.
--standby(none)Standby node host:port (repeatable).
--trtrueEnable Transaction Replay.
--log-levelinfoLog level: trace, debug, info, warn, error.
--json-logsfalseEmit logs in JSON format.

There is also an install skills subcommand (--target claude|codex|both, --symlink, --force, --dry-run) for installing the bundled agent skills.

When --config is provided, the file drives the whole configuration. Command-line node arguments (--primary, --standby) build an in-memory config only when no config file is supplied.

Signals

  • SIGHUP — live configuration reload.
  • SIGUSR2 — graceful drain for a zero-downtime binary handoff, bounded by shutdown_drain_timeout_secs (env override HELIOS_DRAIN_TIMEOUT_SECS).
  • There is no SIGTERM / Ctrl-C handler: SIGTERM terminates the process immediately.

Environment-Variable Substitution

Before the file is parsed as TOML, HeliosProxy expands environment-variable references in the raw text (ProxyConfig::from_filesubstitute_env):

SyntaxMeaning
${NAME}Replaced with the value of environment variable NAME. If NAME is unset, startup fails fast with an error naming the variable — the literal is never left in the output.
${NAME:-default}Replaced with NAME if set, otherwise the literal default text (which may be empty; it runs up to the first }).
listen_address = "${HELIOS_LISTEN:-0.0.0.0:5432}"
admin_token = "${ADMIN_TOKEN}" # fails at startup if ADMIN_TOKEN is unset
[branch]
admin_password = "${PGPASSWORD}"
# max_connections = ${POOL_MAX:-100} # unquoted numeric substitution is valid TOML

Rules and limits:

  • NAME must match [A-Za-z_][A-Za-z0-9_]*.
  • Substitution is in place, so an unquoted ${POOL_MAX:-100} becomes the bare token 100 (valid TOML) and a quoted "${X:-y}" becomes "y".
  • Env lookup only. No shell is spawned and nothing is evaluated — only the env lookup plus the :- default operator run.
  • Comment handling is line-level. A line whose first non-whitespace byte is # (a full-line comment) is copied verbatim and never substituted, so commented ${VAR} examples in the shipped reference configs do not trigger the unset-variable error. A trailing # comment on a value line is not treated as a comment.
  • Trust boundary. Substitution is textual and runs before the TOML parse, so a substituted value containing a quote or newline can inject arbitrary TOML structure. Environment variables and the config file are operator-controlled, which makes this acceptable — but do not feed an untrusted-party-controlled environment variable into a ${...} reference.

Real environment variables

There is no generic HELIOS_PROXY_* “override any value” system. Only one runtime environment variable is consulted directly by the proxy:

VariableEffect
HELIOS_DRAIN_TIMEOUT_SECSOverrides shutdown_drain_timeout_secs at runtime (SIGUSR2 drain bound).

Everything else is configured through the file (optionally via ${VAR} substitution above) or the command-line arguments.


Unknown Keys Are Warned, Not Rejected

ProxyConfig does not use deny_unknown_fields. Any top-level TOML key that is not a recognized ProxyConfig field is parsed-and-ignored, and each one is logged once at startup as a warning:

WARN unknown config section/key '<key>' ignored (not part of ProxyConfig)

This makes silent doc-drift visible without breaking pre-existing configs. Historical example blocks such as [ha], [logging], [metrics], [routing], [distribcache], [graphql], [[tenants]], and [[schema_routes]] are not part of ProxyConfig — they will be warned and ignored. Their real counterparts are documented below (lag_routing, multi_tenancy, graphql_gateway, schema_routing, …). Note that detection is top-level only: a nested unknown key (e.g. [cache.l1]) is not reported.


Top-Level Options

listen_address = "0.0.0.0:5432"
admin_address = "127.0.0.1:9090"
# admin_token = "..." # bearer token for the admin API (see below)
# admin_allow_insecure = false
tr_enabled = true
tr_mode = "session"
tr_read_functions = []
write_timeout_secs = 30
optimize_unnamed_parse = true
shutdown_drain_timeout_secs = 60
KeyTypeDefaultDescription
listen_addressstring"0.0.0.0:5432"Address/port for PostgreSQL client connections. (Required in a config file.)
admin_addressstring"127.0.0.1:9090"Address/port for the admin HTTP API. Loopback by default. (Required in a config file.)
admin_tokenstring(none)Bearer token required on every admin endpoint except liveness probes. See Admin API Security.
admin_allow_insecureboolfalseExplicit opt-in to expose the admin API on a non-loopback address without a token.
tr_enabledbooltrueEnable Transaction Replay. (Required in a config file.)
tr_modestring"session"Transaction Replay mode: none, session, select, transaction. Drives in-session failover as of 1.6.0 — see the table below and the Transaction Replay deep dive. (Required in a config file.)
tr_read_functionsarray of string[]TR-03 read re-execution policy extension. In-session replay re-executes an interrupted read on an unknown outcome only when every function it calls is a PostgreSQL built-in known to be side-effect-free; list additional provably pure functions (unqualified names, case-insensitive) here. Reads calling anything else, quoted-identifier calls, SELECT … INTO, and sequence functions are classified as opaque and never re-executed.
write_timeout_secsu6430Seconds to buffer writes during failover before returning an error — and, since 1.7.0, the single deadline for a whole session recovery: waiting for a primary, connect/auth, session-state restore and replay all draw on it, instead of each having its own timeout.
optimize_unnamed_parsebooltrueSkip re-forwarding an identical unnamed extended-protocol Parse a backend already holds, synthesizing ParseComplete locally. A kill-switch for drivers that depend on the redundant round trip.
shutdown_drain_timeout_secsu6460How long a SIGUSR2 binary-handoff drain keeps serving in-flight connections before dropping them. Runtime override: HELIOS_DRAIN_TIMEOUT_SECS.

The sections [pool], [load_balancer], [health], and at least one [[nodes]] entry are also required in a config file (they have no serde defaults). Every other section listed below is optional and defaults to disabled/off.

Transaction Replay Modes (tr_mode)

tr_mode governs in-session failover: what a live client session experiences when its backend connection fails while a request is in flight (write error / timeout, read error, EOF, reset) or dies while idle. It is independent of tr_enabled (the write journal for the operator-driven POST /api/replay engine) and needs no cargo feature. Two fault phases are distinguished: not-delivered (the request never reached the backend — it certainly did not run) and outcome-unknown (written, then the connection failed before ReadyForQuery).

ModeBehaviour on a backend fault
noneOne ErrorResponse (SQLSTATE 57P01, naming the failed node) + ReadyForQuery, then the client connection is closed.
session (default)The client connection stays open. The proxy waits for a healthy primary (write_timeout_secs), reconnects (startup parameters re-sent), replays the session’s tracked SET/RESET statements, and re-prepares named prepared statements lazily. A not-delivered statement issued outside an explicit transaction is re-executed transparently. Anything else gets ONE error — 57P01 (not delivered inside a transaction) or 08007 transaction_resolution_unknown — the client-visible transaction is aborted (ROLLBACK and retry; other statements get 25P02 until then) and the session continues on the new primary.
selectsession, plus: an outcome-unknown read (SELECT/SHOW/VALUES/read-only WITH/COPY … TO) is re-executed transparently outside a transaction, but only when every function it calls is known side-effect-free — a PostgreSQL built-in on the allowlist or a name you listed in tr_read_functions. A read calling a user-defined function, nextval, pg_notify, set_config, an advisory lock, a quoted-identifier call, or using SELECT … INTO is opaque: it returns 08007 and is never run twice. A read-only explicit transaction is replayed from its BEGIN first and the interrupted statement re-run inside it.
transactionselect, plus: an uncommitted explicit transaction is replayed from its BEGIN on the new primary (recorded simple-protocol text / raw extended-protocol batches) and the in-flight statement is re-executed inside it. Replay is verified: each statement’s original response frames were digested as the client saw them (within [limits] tr_max_observation_bytes) and the replacement backend’s responses are digested the same way — any divergence rolls the replay back and returns 40001, so a replay that would silently continue on different rows fails loudly instead. A failed replayed statement produces the same 40001. A COMMIT/END/PREPARE TRANSACTION/COMMIT PREPARED whose outcome is unknown is never retried (08007). Still opt-in: verification catches divergent results, it does not make now() or a RETURNING serial reproduce its first value.

Hard rules in every mode: a write whose outcome is unknown is never re-executed except as part of transaction mode’s replay of an uncommitted transaction (the original died uncommitted with the old backend); a COMMIT with unknown outcome is never retried; SAVEPOINT/RELEASE/ROLLBACK TO are ordinary replayed statements; a COPY in progress at the fault → 08006 and the connection is closed. A transaction is marked non-replayable (so transaction degrades to session for it) when it exceeds [limits] tr_max_replay_statements / tr_max_replay_bytes, enters the failed state, contains a COPY, or one of its statements was transformed by query-rewrite / multi-tenancy, runs at SERIALIZABLE or REPEATABLE READ (explicitly, via SET TRANSACTION, or inherited from a tracked default_transaction_isolation — no replay can reproduce that snapshot), or produced a response too large to have been digested within [limits] tr_max_observation_bytes.

Session SET tracking is transactional: a SET inside a transaction takes effect in the restore set only when that transaction commits, ROLLBACK TO SAVEPOINT discards the ones made after the savepoint, and RESET/RESET ALL/DISCARD ALL inside a transaction are deferred to commit. Variables are keyed by name, so repeating a SET reuses its slot; the cap tr_max_session_set_statements counts distinct variables, and exceeding it refuses the failover with 08006 rather than re-homing a session with incomplete state. It covers simple-protocol statements only — extended-protocol SETs as sent by JDBC, SQL-level PREPARE, session temp tables and cursors are not restored. (Extended-protocol named prepared statements are a separate mechanism: those are re-prepared lazily on the replacement backend.)

Backend credentials. Re-homing a session means opening a fresh backend connection. In pass-through auth mode the proxy never sees the client’s password (SCRAM is designed for that), so a fresh connection can only be completed against a backend that does not challenge (trust); a challenging backend fails the recovery immediately with 08006 (“the proxy could not authenticate to the replacement primary”). To fail over onto password-protected (SCRAM-SHA-256 / MD5 / cleartext) backends, make the proxy the auth boundary: [auth] mode = "scram" with a plaintext auth_file entry for the user — the proxy then authenticates the client itself and uses the same secret as a SCRAM client towards every backend connection it opens (this also removes the former “scram mode needs a trust backend” restriction on the first connection).

Counters on /metrics and /metrics/prometheus: tr_failovers_total, tr_statements_reexecuted_total, tr_transactions_replayed_total, tr_replay_failures_total, tr_unknown_outcome_errors_total, tr_replay_cap_exceeded_total, tr_session_set_cap_exceeded_total.


Admin API Security

The admin API runs privileged operations (arbitrary SQL via /api/sql, forced failover via /api/chaos, migration cutover, branch CREATE/DROP DATABASE, replay/shadow against operator-chosen targets), so its exposure is guarded at startup:

  • Default bind is loopback (127.0.0.1:9090). A fresh install is safe.
  • If admin_address parses to a non-loopback IP and admin_token is unset and admin_allow_insecure is false, the proxy refuses to start with a descriptive error. Fix it by setting admin_token, binding to 127.0.0.1, or setting admin_allow_insecure = true (only when you front the admin port with your own authenticating proxy / network policy).
  • When admin_token is set, every admin endpoint requires Authorization: Bearer <token> except the liveness probes.

Startup validation also rejects health.check_interval_secs = 0 (a zero interval would panic the health-check timer and silently stop probing).


Pool Mode ([pool_mode])

Controls Session/Transaction/Statement pooling.

[pool_mode]
mode = "transaction"
max_pool_size = 100
min_idle = 10
idle_timeout_secs = 600
max_lifetime_secs = 3600
acquire_timeout_secs = 5
reset_query = "DISCARD ALL"
prepared_statement_mode = "track"
skip_clean_reset = false
KeyTypeDefaultDescription
modestring"session"Pooling mode: session, transaction, statement.
max_pool_sizeu32100Maximum backend connections per node.
min_idleu3210Minimum idle connections to maintain.
idle_timeout_secsu64600Close idle connections after this many seconds.
max_lifetime_secsu643600Recycle connections after this many seconds.
acquire_timeout_secsu645Max seconds to wait when acquiring from the pool.
reset_querystring"DISCARD ALL"SQL run when a connection returns to the pool.
prepared_statement_modestring"disable"Prepared-statement handling: disable, track, named.
skip_clean_resetboolfalseTransaction/Statement pooling only: park a connection that provably touched no session state (no SET/GUC, temp table, prepared statement, LISTEN, advisory lock, …) without running reset_query, saving a round-trip per clean transaction. Classification is conservative — a misclassification only ever costs an unnecessary reset, never leaks state. Intended for autocommit / simple-protocol workloads.

Pooling Modes

ModeReturns To PoolBest For
sessionWhen the client disconnects (1:1 client↔backend).Prepared statements, long-running sessions, legacy apps.
transactionAfter COMMIT/ROLLBACK.Web apps, microservices, connection-starved environments.
statementAfter each statement.Simple read-heavy workloads without multi-statement transactions.

Prepared Statement Modes

ModeBehavior
disableNot tracked. Safest for transaction/statement pooling.
trackTrack PREPARE/DEALLOCATE and recreate on a new backend connection.
namedProtocol-level named statements. Compatible with session pooling.

Connection Pool ([pool])

Core per-node connection pool. (Required section.)

[pool]
min_connections = 2
max_connections = 100
idle_timeout_secs = 300
max_lifetime_secs = 1800
acquire_timeout_secs = 30
test_on_acquire = true
KeyTypeDefaultDescription
min_connectionsusize2Minimum connections per node.
max_connectionsusize100Maximum connections per node. Must be ≥ min_connections.
idle_timeout_secsu64300Close connections idle longer than this.
max_lifetime_secsu641800Maximum connection lifetime before recycling.
acquire_timeout_secsu6430Max wait for a connection from the pool.
test_on_acquirebooltrueHealth-check a connection before handing it out.

Load Balancer ([load_balancer])

(Required section.)

[load_balancer]
read_strategy = "round_robin"
read_write_split = true
latency_threshold_ms = 100
KeyTypeDefaultDescription
read_strategystring"round_robin"Read routing strategy (see below).
read_write_splitbooltrueRoute writes to the primary, reads to standby/replica nodes.
latency_threshold_msu64100Latency above which a node is treated as unhealthy for routing.

Routing Strategies

StrategyDescription
round_robinRotate through nodes equally.
weighted_round_robinRotate proportionally to each node’s weight.
least_connectionsRoute to the node with the fewest active connections.
latency_basedRoute to the lowest-latency node.
randomPick a node at random.

Health Checks ([health])

(Required section.)

[health]
check_interval_secs = 5
check_timeout_secs = 3
failure_threshold = 3
success_threshold = 2
check_query = "SELECT 1"
KeyTypeDefaultDescription
check_interval_secsu645Interval between probes. Must be ≥ 1 (0 is rejected at startup).
check_timeout_secsu643Max wait for a probe response.
failure_thresholdu323Consecutive failures before marking a node unhealthy.
success_thresholdu322Consecutive successes before marking a node healthy again.
check_querystring"SELECT 1"Health-check query.

Nodes ([[nodes]])

One entry per backend. At least one node with role = "primary" is required.

[[nodes]]
host = "db-primary.internal"
port = 5432
http_port = 8080
role = "primary"
weight = 100
enabled = true
name = "primary-1"
[[nodes]]
host = "db-standby-1.internal"
port = 5432
role = "standby"
weight = 100
enabled = true
name = "standby-1"
KeyTypeDefaultRequiredDescription
hoststringYesBackend hostname or IP.
portu16YesPostgreSQL-protocol port.
http_portu168080NoHTTP API port on the backend node (SQL API forwarding).
rolestringYesprimary, standby, or replica.
weightu32Yes*Load-balancing weight.
enabledboolYes*Whether the node is routable. Toggleable at runtime via the admin API.
namestring(none)NoHuman-readable name for logs/metrics/admin.

* weight and enabled have no serde default — supply them explicitly per node.

Node Roles

RoleDescription
primaryRead/write node. All writes and transaction-control statements route here. At least one required.
standbyPromotable standby. Eligible for failover; receives reads when read_write_split is on.
replicaRead-only replica. Not promotable; receives reads only.

TLS ([tls])

Optional TLS termination for client connections. Omit the whole section to disable.

[tls]
enabled = true
cert_path = "/etc/heliosproxy/server.crt"
key_path = "/etc/heliosproxy/server.key"
ca_path = "/etc/heliosproxy/ca.crt"
require_client_cert = false
KeyTypeDefaultDescription
enabledboolEnable TLS for client-facing connections.
cert_pathstringPEM server certificate path.
key_pathstringPEM private key path.
ca_pathstring(none)CA cert for client-certificate verification.
require_client_certboolRequire a valid client certificate.

Query Cache ([cache])

In-process query-result cache. Only active with the query-cache feature and enabled = true.

[cache]
enabled = true
ttl_secs = 300
max_result_bytes = 1048576
max_cacheable_response_bytes = 4194304
KeyTypeDefaultDescription
enabledboolfalseServe read SELECT results from the L1/L2 cache.
ttl_secsu64300Time-to-live for cached results.
max_result_bytesusize1048576Largest single result to cache; larger results bypass.
max_cacheable_response_bytesusize4194304 (4 MiB)(1.6.0) Ceiling on the transient buffer used to capture a response while it streams. A response that grows past it is marked non-cacheable and the buffer is released; the client always receives the full, unmodified response either way. Must be >= 1 (a 0 is rejected at startup, since it would make every response uncacheable while still looking like a working cache). Sits above the 1 MiB max_result_bytes default so it never masks that knob, and it governs the shared capture used by both the query cache and the edge cache.

Lag-Aware Routing ([lag_routing])

Replica-lag-aware routing + read-your-writes. Only enforced with the lag-routing feature and enabled = true.

[lag_routing]
enabled = true
ryw_window_ms = 500
max_lag_bytes = 0
KeyTypeDefaultDescription
enabledboolfalseEnable lag-aware read routing + read-your-writes.
ryw_window_msu64500Reads within this many ms of a write in the same session pin to the primary (read-your-writes). 0 disables the window.
max_lag_bytesu640Exclude a standby when its replication lag exceeds this many bytes. 0 = no lag-based exclusion.

Routing Hints ([routing_hints])

SQL-comment routing hints (/*helios:route=primary*/). Only honored with the routing-hints feature and enabled = true.

[routing_hints]
enabled = true
strip_hints = true
KeyTypeDefaultDescription
enabledboolfalseParse and honor /*helios:...*/ hints; an applied hint overrides default verb routing (but never a plugin Block).
strip_hintsbooltrueRemove the hint comment from SQL before forwarding to the backend.

Rate Limiting ([rate_limit])

Token-bucket + concurrency limiting. Only enforced with the rate-limiting feature and enabled = true.

[rate_limit]
enabled = true
default_qps = 1000
default_burst = 2000
max_concurrent = 0
key_by = "user"
KeyTypeDefaultDescription
enabledboolfalseEnforce rate limits.
default_qpsu321000Sustained queries/sec per bucket.
default_burstu322000Token-bucket depth (burst) per bucket.
max_concurrentu320Max concurrent in-flight queries per bucket (0 = engine default).
key_bystring"user"Bucket key: user, client_ip, database, global.

Circuit Breaker ([circuit_breaker])

Per-node circuit breaker. Only enforced with the circuit-breaker feature and enabled = true.

[circuit_breaker]
enabled = true
failure_threshold = 5
open_secs = 10
success_threshold = 3
KeyTypeDefaultDescription
enabledboolfalseTrip failing backends out of rotation.
failure_thresholdu325Consecutive failures that open a node’s circuit.
open_secsu6410How long a circuit stays open before a half-open probe.
success_thresholdu323Successful probes required to close a half-open circuit.

Operational Limits ([limits])

Session/protocol safety bounds and relay timeouts. Each key was previously a compiled-in constant; the defaults reproduce those constants exactly, so an absent [limits] block is byte-for-byte unchanged. All values are resolved once at startup. validate() rejects 0 for the safety bounds (where a 0 would disable a bound rather than mean anything useful) and caps every *_secs key at one year (31536000), which would otherwise overflow the connect-time deadline. The two 1.6.0 opt-in bounds are the deliberate exceptions: max_client_connections and client_idle_timeout_secs both use 0 to mean unlimited / disabled, which is their default and reproduces the pre-1.6.0 behaviour.

[limits]
max_cancel_keys = 100000
startup_timeout_secs = 30
backend_write_timeout_secs = 30
backend_read_timeout_secs = 30
client_write_timeout_secs = 60
reprepare_timeout_secs = 15
max_prepared_statements = 8192
max_prepared_bytes = 67108864
max_pending_bytes = 67108864
max_backend_frame_bytes = 104857600
backend_response_timeout_secs = 0
tr_max_observation_bytes = 1048576
max_total_idle_backend_conns = 8192
pool_reap_interval_secs = 30
max_client_connections = 0
client_idle_timeout_secs = 0
tr_max_replay_statements = 1000
tr_max_replay_bytes = 4194304
tr_max_session_set_statements = 256
KeyTypeDefaultDescription
max_cancel_keysusize100000Capacity of the query-cancellation key map (BackendKeyData → backend address); at capacity the oldest entries are FIFO-evicted.
startup_timeout_secsu6430Deadline for the pre-auth startup exchange (TLS negotiation + startup/authentication); bounds slow-loris handshakes.
backend_write_timeout_secsu6430Timeout for a single backend write on the forward path.
backend_read_timeout_secsu6430Timeout for a single backend read on the relay path (paired with backend_write_timeout_secs; a slow-but-healthy read is not itself a fault).
client_write_timeout_secsu6460Timeout for a single client write, so a wedged client cannot pin a proxy task (and its backend connection) forever.
reprepare_timeout_secsu6415Timeout for the out-of-band re-prepare exchange performed on a backend connection switch.
max_prepared_statementsusize8192Per-session cap on distinct named prepared statements.
max_prepared_bytesusize67108864Per-session cap on aggregate bytes retained in the statement registry (64 MiB).
max_pending_bytesusize67108864Per-session cap on the un-flushed extended-protocol pending buffer (64 MiB).
max_backend_frame_bytesusize104857600Cap on the declared length of one backend response frame on every streaming relay; a header above it or below the 4-byte minimum closes the backend as malformed (100 MiB, the frontend message cap). Bounds a frame, not a result set.
backend_response_timeout_secsu640Whole-response deadline for one backend response on the streaming relays, first read to ReadyForQuery. backend_read_timeout_secs re-arms per read and cannot catch a slow drip inside one response; this can. 0 disables (prior behaviour).
tr_max_observation_bytesusize1048576In-session TR: response bytes hashed per recorded statement into an observation digest, re-verified when the transaction is replayed; divergence rolls the replay back with 40001. A response over the cap has no verifiable digest, so its transaction becomes non-replayable.
max_total_idle_backend_connsusize8192Global ceiling on idle backend-pool connections across all (node,user,db) identities. Only consumed with the pool-modes feature; parsed-and-ignored otherwise.
pool_reap_interval_secsu6430How often the idle-connection reaper runs.
max_client_connectionsusize0(1.6.0) Maximum concurrent authenticated client sessions; 0 = unlimited, reproducing the pre-cap behaviour. A connection refused because the cap is saturated is counted in /metrics as connections_rejected (cancel requests are never refused). Read once at startup — the permit pool is sized when the server is built, so a SIGHUP that changes this key is logged and ignored; restart to change it.
client_idle_timeout_secsu640(1.6.0) How long an authenticated session may sit between statements before the proxy terminates it with SQLSTATE 57P05; 0 = disabled. A session idle inside an open transaction is terminated by this timeout too — PostgreSQL’s separate idle_in_transaction_session_timeout GUC is not implemented. Read once at startup.
tr_max_replay_statementsusize1000In-session TR (tr_mode = select|transaction): cap on statements recorded per explicit transaction for failover replay. Over the cap the transaction is marked non-replayable (transaction degrades to session for it; tr_replay_cap_exceeded_total).
tr_max_replay_bytesusize4194304In-session TR: cap on bytes (statement text / raw extended-protocol frames) recorded per explicit transaction (4 MiB). Same degradation as above.
tr_max_session_set_statementsusize256In-session TR (tr_mode != none): cap on DISTINCT session variables whose latest SET is replayed onto the replacement backend (a later SET of the same variable replaces the earlier one; RESET name removes it; RESET ALL/DISCARD ALL clear all). Over the cap tracking stops (tr_session_set_cap_exceeded_total) and a subsequent failover is refused with 08006 rather than re-homing the session with incomplete state.

Query Analytics ([analytics])

Fingerprinting, per-query stats, slow-query log, pattern detection. Only active with the query-analytics feature and enabled = true.

[analytics]
enabled = true
slow_query_ms = 1000
max_fingerprints = 10000
KeyTypeDefaultDescription
enabledboolfalseRecord per-query statistics and slow-query log.
slow_query_msu641000Queries slower than this are added to the slow-query log.
max_fingerprintsu3210000Maximum distinct query fingerprints to track.

Anomaly Detection ([anomaly])

Tunables for the in-process anomaly detector (SQL-injection patterns, failed-auth bursts, per-tenant rate spikes, novel-query shapes). The section is parsed on every build for config round-tripping, but the detector is only active with the anomaly-detection feature. The defaults reproduce the prior hardcoded behavior exactly, so an absent [anomaly] block changes nothing.

The detector is built once at startup, so changing [anomaly] requires a restart — a SIGHUP config reload does not rebuild it.

[anomaly]
rate_window_secs = 60
spike_z_threshold = 3.0
auth_window_secs = 60
auth_critical_count = 10
auth_warning_count = 5
event_buffer_size = 1024
emit_novel_queries = true
max_seen_fingerprints = 100000
KeyTypeDefaultDescription
rate_window_secsu6460Rolling window for the per-tenant rate EWMA, seconds. Must be >= 1.
spike_z_thresholdf643.0Minimum z-score before a rate spike fires. Must be finite and > 0.
auth_window_secsu6460Window for failed-auth (credential-stuffing) bursts, seconds. Must be >= 1.
auth_critical_countu3210Failures inside the auth window that escalate to Critical. Must be >= 1.
auth_warning_countu325Failures inside the auth window that escalate to Warning. Must be <= auth_critical_count.
event_buffer_sizeusize1024Maximum events kept in the in-memory ring buffer. Must be >= 1.
emit_novel_queriesbooltrueEmit first-seen query fingerprints as informational events; set false on high-churn workloads.
max_seen_fingerprintsusize100000Upper bound on the novel-query fingerprint set before it is cleared (bounds memory on high-cardinality SQL). Must be >= 1.

Query Rewriting ([query_rewrite])

Rules-engine SQL rewriting. Only active with the query-rewriting feature and enabled = true.

[query_rewrite]
enabled = true
[[query_rewrite.rules]]
match_table = "orders"
append_where = "deleted_at IS NULL"
[[query_rewrite.rules]]
match_regex = "^SELECT \\* FROM events"
add_limit = 1000

[query_rewrite] keys:

KeyTypeDefaultDescription
enabledboolfalseApply the rewrite rules on the query path.
rulesarray[]Ordered rewrite rules ([[query_rewrite.rules]]).

Each [[query_rewrite.rules]] entry (first matching transformation is applied):

KeyTypeDescription
match_tablestringApply to queries referencing this table.
match_regexstringApply to queries matching this regex.
replace_table_withstringRewrite match_table → this table name.
append_wherestringAppend AND <expr> to the WHERE clause.
add_limitu32Add LIMIT n to an unbounded query.

Multi-Tenancy ([multi_tenancy])

Per-tenant row isolation via injected predicates. Only active with the multi-tenancy feature and enabled = true.

[multi_tenancy]
enabled = true
identify_by = "application_name"
tenant_column = "tenant_id"
tenant_tables = ["orders", "invoices"]
tenants = ["acme", "globex"]
KeyTypeDefaultDescription
enabledboolfalseEnforce per-tenant row isolation.
identify_bystring"application_name"Connection attribute naming the tenant: a startup parameter name (e.g. application_name, user) or the literal database.
tenant_columnstring"tenant_id"The row-level tenant column injected into queries.
tenant_tablesarray[]Tables that get the tenant filter injected; others pass through.
tenantsarray[]Known tenant ids.

Schema Routing ([schema_routing])

Route analytical (OLAP) queries to a dedicated node. Only active with the schema-routing feature and enabled = true.

[schema_routing]
enabled = true
analytics_node = "analytics-1"
KeyTypeDefaultDescription
enabledboolfalseRoute aggregations / GROUP BY / window-function queries to a node.
analytics_nodestring""name of the node analytical queries route to.

Authentication ([auth]) and HBA Rules ([[hba]])

Client-side authentication mode and pg_hba-style admission.

[auth]
mode = "scram"
auth_file = "/etc/heliosproxy/userlist.txt"
[[hba]]
action = "allow"
user = "all"
database = "all"
address = "10.0.0.0/8"
[[hba]]
action = "reject"
user = "all"
database = "all"
address = "all" # trailing rule = default-deny

[auth]:

KeyTypeDefaultDescription
modestring"passthrough"passthrough relays client auth to the backend; scram makes the proxy terminate SCRAM-SHA-256 against auth_file.
auth_filestring(none)Path to a pgbouncer-style user list (user:secret, secret = plaintext or a SCRAM-SHA-256$... verifier). Required when mode = "scram".

[[hba]] rules are evaluated in order; the first rule whose user, database, and address all match decides the outcome. If no rule matches, the connection is admitted (add a trailing reject … all/all/all for default-deny):

KeyTypeDefaultDescription
actionstringallow or reject.
userstring"all"Matching PostgreSQL user, or all.
databasestring"all"Matching database, or all.
addressstring"all"all, a bare IP, or a CIDR (e.g. 10.0.0.0/8, ::1/128).

WASM Plugins ([plugins])

Plugin subsystem (a single [plugins] table, not an array of [[plugins]]). Only consumed with the wasm-plugins feature; strictly opt-in.

[plugins]
enabled = true
plugin_dir = "/etc/heliosproxy/plugins"
hot_reload = false
memory_limit_mb = 64
timeout_ms = 100
max_plugins = 20
fuel_metering = true
fuel_limit = 1000000
kv_max_value_bytes = 65536
kv_max_keys_per_plugin = 1024
kv_max_plugins = 256
kv_max_total_bytes = 67108864
# trust_root = "/etc/heliosproxy/plugin-keys"
KeyTypeDefaultDescription
enabledboolfalseEnable the plugin subsystem.
plugin_dirstring"/etc/heliosproxy/plugins"Directory scanned for .wasm plugins at startup.
hot_reloadboolfalseWatch plugin_dir and reload plugins on change.
memory_limit_mbusize64Memory limit per plugin instance.
timeout_msu64100Execution timeout per hook call.
max_pluginsusize20Maximum concurrently-loaded plugins.
fuel_meteringbooltrueEnable per-call CPU-cycle (fuel) metering.
fuel_limitu641000000Fuel units allowed per hook call when metering is on.
kv_max_value_bytesusize65536Max bytes for a single plugin-KV key OR value (via kv_set or PUT /admin/kv/<plugin>/<key>); 0 = unlimited. A write past this cap is rejected (kv_set returns -1; the admin endpoint returns 413).
kv_max_keys_per_pluginusize1024Max distinct keys per plugin KV namespace; 0 = unlimited. Overwriting an existing key never trips the cap.
kv_max_pluginsusize256Max distinct plugin KV namespaces that may exist at once; 0 = unlimited. Bounds how many <plugin> namespaces PUT /admin/kv/<plugin>/<key> can create, so a token-holder cannot exhaust memory by writing to unboundedly-many namespace names. Writing to an already-present namespace never trips the cap; deleting a namespace’s last key frees its slot.
kv_max_total_bytesusize67108864 (64 MiB)Max TOTAL retained bytes across ALL plugin KV namespaces (each entry’s key + value bytes plus each live namespace’s name bytes); 0 = unlimited. The single backstop that bounds the whole KV footprint regardless of the per-axis product kv_max_plugins × kv_max_keys_per_plugin × kv_max_value_bytes (which can otherwise reach tens of GiB), so a token-holding PUT /admin/kv/<plugin>/<key> caller cannot drive the proxy to an OOM. A write past this cap is rejected (kv_set returns -1; the admin endpoint returns 413).
trust_rootstring(none)Ed25519 trust-root directory. When set, every .wasm requires a sidecar .sig verifying against a *.pub in this directory; when omitted, signatures are not checked.

MCP Agent Gateway ([mcp]) and Agent Contracts ([[agent_contracts]])

Native MCP server exposing query / list_tables / explain tools. Disabled by default.

[mcp]
enabled = true
listen_address = "127.0.0.1:9092"
backend_host = "127.0.0.1"
backend_port = 5432
backend_user = "postgres"
# backend_password = "..."
# backend_database = "app"
read_only = true
# contract = "reporting-agent"
auth_token = "${MCP_TOKEN}"
[[agent_contracts]]
id = "reporting-agent"
read_only = true
allowed_verbs = ["SELECT"]
allowed_tables = ["orders", "invoices"]
require_limit = true
max_rows = 1000

[mcp]:

KeyTypeDefaultDescription
enabledboolfalseServe the MCP JSON-RPC endpoint.
listen_addressstring"127.0.0.1:9092"HTTP listen address for MCP.
backend_hoststring"127.0.0.1"Backend the tool SQL runs against.
backend_portu165432Backend port.
backend_userstring"postgres"Backend user.
backend_passwordstring(none)Backend password.
backend_databasestring(none)Backend database.
read_onlybooltrueRefuse write/DDL — agents get a read-only surface.
contractstring(none)Name of an [[agent_contracts]] entry to enforce on every tool call.
auth_tokenstring(none)Bearer token required on every MCP request. Absent = open, so set this for any non-loopback deployment — MCP exposes SQL and must not be anonymous off localhost.

Each [[agent_contracts]] entry (scoped grants, referenced by id from [mcp] contract):

KeyTypeDefaultDescription
idstringIdentifier matched against the agent.
read_onlybooltrueReject write/DDL statements.
allowed_verbsarray(none)If set, only these SQL verbs are allowed (upper-case).
allowed_tablesarray(none)If set, only these tables may be referenced.
denied_tablesarray[]Tables that may never be referenced (takes precedence over allow).
require_predicate_onarray[]Predicates that must be present when a named table is touched.
require_limitboolfalseRequire a LIMIT on SELECTs.
max_rowsu64(none)Suggested/enforced row cap.

HTTP SQL Gateway ([http_gateway])

Neon-serverless-driver-compatible POST /sql endpoint. Disabled by default.

[http_gateway]
enabled = true
listen_address = "127.0.0.1:9093"
backend_host = "127.0.0.1"
backend_port = 5432
backend_user = "postgres"
# backend_password = "..."
# backend_database = "app"
auth_token = "${HTTP_GW_TOKEN}"
KeyTypeDefaultDescription
enabledboolfalseServe the HTTP SQL gateway.
listen_addressstring"127.0.0.1:9093"HTTP listen address.
backend_hoststring"127.0.0.1"Backend host.
backend_portu165432Backend port.
backend_userstring"postgres"Backend user.
backend_passwordstring(none)Backend password.
backend_databasestring(none)Backend database.
auth_tokenstring(none)Optional Bearer token required on requests.

GraphQL Gateway ([graphql_gateway])

GraphQL-to-SQL gateway on a separate HTTP listener. Only active with the graphql-gateway feature and enabled = true.

[graphql_gateway]
enabled = true
listen_address = "0.0.0.0:9091"
backend_host = "127.0.0.1"
backend_port = 5432
backend_user = "postgres"
# backend_password = "..."
# backend_database = "app"
# auth_token = "..."
[[graphql_gateway.tables]]
name = "orders"
columns = ["id", "customer_id", "total", "created_at"]

[graphql_gateway]:

KeyTypeDefaultDescription
enabledboolfalseServe the GraphQL gateway.
listen_addressstring"0.0.0.0:9091"HTTP listen address.
backend_hoststring"127.0.0.1"Backend host.
backend_portu165432Backend port.
backend_userstring"postgres"Backend user.
backend_passwordstring(none)Backend password.
backend_databasestring(none)Backend database.
auth_tokenstring(none)Optional Bearer token required on requests.
tablesarray[]Tables exposed as GraphQL types ([[graphql_gateway.tables]]).

Each [[graphql_gateway.tables]]: name (string) and columns (array of strings).


Traffic Mirror ([mirror])

Continuously mirror a sampled share of live (simple-query) writes to a secondary backend, off the client hot path. Disabled by default; the on-ramp to a PG→Nano migration mirror.

[mirror]
enabled = true
sample_rate = 1.0
writes_only = true
queue_size = 10000
backend_host = "127.0.0.1"
backend_port = 5432
backend_user = "postgres"
# backend_password = "..."
# backend_database = "app"
# source_host / source_port / source_user / source_password / source_database
KeyTypeDefaultDescription
enabledboolfalseMirror eligible statements to the secondary.
sample_ratef641.0Fraction of eligible statements to mirror (0.01.0).
writes_onlybooltrueMirror only write/DDL statements; false mirrors all simple queries.
queue_sizeusize10000Bounded queue depth; when full, statements are dropped (and counted) rather than blocking.
backend_hoststring"127.0.0.1"Mirror-target host.
backend_portu165432Mirror-target port.
backend_userstring"postgres"Mirror-target user.
backend_passwordstring(none)Mirror-target password.
backend_databasestring(none)Mirror-target database.
source_host / source_port / source_user / source_password / source_databaselocalhost / 5432 / postgres / none / noneSource (primary) connection used by POST /api/migration/snapshot to bootstrap the secondary.

Edge / Geo Proxy ([edge])

Two-region result caching. A home-role proxy is authoritative (routes writes, caches reads, broadcasts SSE invalidations); an edge-role proxy serves reads from a local cache and forwards misses/writes to the home. Disabled by default. Parsed on every build, but enabled = true requires the edge-proxy compile-time feature (validation rejects it otherwise).

[edge]
enabled = true
role = "edge" # "home" (default) or "edge"
home_url = "https://home-proxy:9090" # edge: home admin base URL
auth_token = "${EDGE_HOME_TOKEN}" # edge: home admin bearer
allow_insecure_home_url = false
default_ttl_secs = 60
max_entries = 10000
max_edges = 32
liveness_window_secs = 120
subscribe_gc_secs = 30
region = "eu-west"
edge_id = "edge-a"
KeyTypeDefaultDescription
enabledboolfalseMaster switch; requires the edge-proxy feature to enable.
rolestring"home"home (authoritative) or edge (cache-first).
home_urlstring""(edge) Home proxy admin base URL the edge subscribes to. Required for role = "edge".
auth_tokenstring""(edge) Home admin bearer for the invalidation subscription. When set, home_url must be https:// unless allow_insecure_home_url = true.
allow_insecure_home_urlboolfalse(edge) Allow presenting auth_token to a plain-http home_url (private links only).
default_ttl_secsu6460Default TTL for cache entries when the home supplies none. Must be ≥ 1 when edge is enabled.
max_entriesusize10000Cache entries before LRU eviction.
max_edgesusize32(home) Maximum simultaneously-registered edges.
liveness_window_secsu64120(home) Edges not seen within this window are GC-pruned. Keep comfortably above ~45s. Must be ≥ 1 when edge is enabled.
subscribe_gc_secsu6430(home) Registry GC sweep cadence. Must be ≥ 1 when edge is enabled.
regionstring""(edge) Region label reported when subscribing.
edge_idstring""(edge) Stable registration id (empty → edge-<pid>).

An edge-role proxy also requires at least one [[nodes]] entry pointing at the home’s PG-wire listener (its data plane), and cannot be combined with [cache] enabled = true (the query-result cache does not receive edge invalidations).


Instant Branch Databases ([branch])

Provision CREATE DATABASE <branch> TEMPLATE <base> clones through the proxy. Disabled by default.

[branch]
enabled = true
backend_host = "127.0.0.1"
backend_port = 5432
admin_user = "postgres" # a role with CREATEDB
admin_password = "${PGPASSWORD}"
admin_database = "postgres" # maintenance DB for CREATE/DROP DATABASE
base_database = "postgres" # default template when a request omits `base`
KeyTypeDefaultDescription
enabledboolfalseEnable branch-database provisioning.
backend_hoststring"127.0.0.1"Backend host.
backend_portu165432Backend port.
admin_userstring"postgres"Role with CREATEDB privilege.
admin_passwordstring(none)Password for admin_user.
admin_databasestring"postgres"Maintenance database to issue CREATE/DROP DATABASE against.
base_databasestring"postgres"Default template database when a request omits base.

Configuration Validation

At startup the proxy validates the loaded config and refuses to start if:

  1. No backend nodes are configured.
  2. No node has role = "primary".
  3. pool.max_connections < pool.min_connections.
  4. health.check_interval_secs = 0.
  5. admin_address is non-loopback but admin_token is unset and admin_allow_insecure = false (see Admin API Security).
  6. edge.enabled = true without the edge-proxy feature, or an edge role missing its home_url, or its zero-value timing knobs, or combined with [cache].

Invalid configurations produce a descriptive error and a non-zero exit code. Unknown top-level keys are warned, not rejected (see Unknown Keys).


Complete Example

Ready-to-use examples live in config/proxy.example.toml, config/proxy.full.toml, config/proxy.postgres.toml, and the working scripts/regress/*.toml files.

# HeliosProxy configuration example.
listen_address = "0.0.0.0:6432"
admin_address = "127.0.0.1:9090"
# admin_token = "${ADMIN_TOKEN}"
tr_enabled = true
tr_mode = "session"
write_timeout_secs = 30
[pool_mode]
mode = "transaction"
max_pool_size = 100
prepared_statement_mode = "track"
skip_clean_reset = true
[pool]
min_connections = 5
max_connections = 100
idle_timeout_secs = 300
max_lifetime_secs = 1800
acquire_timeout_secs = 30
test_on_acquire = true
[load_balancer]
read_strategy = "least_connections"
read_write_split = true
latency_threshold_ms = 50
[health]
check_interval_secs = 5
check_timeout_secs = 3
failure_threshold = 3
success_threshold = 2
check_query = "SELECT 1"
[[nodes]]
host = "db-primary.internal"
port = 5432
role = "primary"
weight = 100
enabled = true
name = "primary"
[[nodes]]
host = "db-standby-1.internal"
port = 5432
role = "standby"
weight = 100
enabled = true
name = "standby-1"
[tls]
enabled = false
cert_path = "/etc/heliosproxy/server.crt"
key_path = "/etc/heliosproxy/server.key"
require_client_cert = false

See Also


Transaction Replay: the re-executable read subset (TR-03)

tr_mode = select and transaction re-execute an interrupted statement on the replacement backend only when its outcome on the failed backend cannot have had a side effect. The proxy decides lexically, without a catalog round trip:

  • Statement shape: SELECT, VALUES, TABLE, SHOW, EXPLAIN (without ANALYZE), COPY … TO, or a WITH whose CTEs contain no INSERT/UPDATE/ DELETE/MERGE.
  • No INTO (creates a table).
  • Every syntactic function call — an identifier followed by ( outside strings, quoted identifiers, dollar quotes and comments — must be a PostgreSQL built-in from the proxy’s side-effect-free list (aggregates, string/math/date/JSON/array functions, pg_sleep, random, now, current_setting, size/introspection helpers) or listed in tr_read_functions. Schema qualification is ignored (pg_catalog.lower(...) is lower).
  • A call through a quoted identifier ("MyFn"(...)) is never eligible: its case is opaque to the policy.

Everything else — user-defined functions, nextval/setval/currval/lastval, pg_notify, set_config, pg_advisory_*, txid_current, setseed, large-object and replication functions — makes the statement opaque: on an unknown outcome the client receives SQLSTATE 08007 and the proxy does not run it again. Inside an explicit transaction an opaque statement also marks the transaction as containing a possible write, so select mode will not replay it. Nondeterministic but side-effect-free calls are allowed because an unknown-outcome autocommit read published nothing to the client before the fault.

Session state across a failover (TR-04)

SET/RESET issued through the simple query protocol are tracked per session and replayed, in order, onto the replacement backend when the session re-homes. The tracking is transactional, mirroring PostgreSQL: a SET inside an explicit transaction takes effect for restore only when that transaction commits; ROLLBACK discards it; ROLLBACK TO SAVEPOINT discards the SETs made after the savepoint (RELEASE keeps them); RESET name, RESET ALL and DISCARD ALL inside a transaction are likewise deferred until commit. SET LOCAL, SET TRANSACTION and SET CONSTRAINTS are transaction-scoped and never tracked. Variables are identified by name (SET TIME ZONE is timezone, SET SCHEMA is search_path, SET NAMES is client_encoding), so repeated SETs of one variable cost one slot. Extended- protocol SETs, SQL-level PREPARE, temporary tables and cursors are not restored. (Extended-protocol named prepared statements are re-prepared lazily on the replacement backend — a separate mechanism from this restore set.)

Replay verification and the recovery deadline (TR-06)

While a transaction is recorded for tr_mode = transaction replay, the proxy hashes each statement’s response frames (RowDescription, DataRow, CommandComplete, EmptyQueryResponse) up to tr_max_observation_bytes. When the transaction is replayed on the replacement backend, each response is hashed the same way; if any digest differs — the data the client already saw is not what the new backend would have returned — the replay is rolled back and the client receives 40001 for the interrupted statement. Transactions opened at SERIALIZABLE or REPEATABLE READ (explicitly, or through default_transaction_isolation) are never replayed: a replay cannot reproduce the old snapshot.

One deadline, write_timeout_secs, bounds the whole recovery: waiting for a healthy primary, connecting and authenticating, restoring session SETs, replaying the transaction and re-executing the interrupted statement.