---
title: "HeliosProxy Architecture"
---

# HeliosProxy Architecture

This document describes the internal architecture of HeliosProxy — its module map, data-flow pipeline, and topology provider abstraction. The authoritative list of optional modules is the `[features]` table in `Cargo.toml`, tracked flag-for-flag in [Feature Flags](/docs/proxy/feature-flags/).

---

## System Overview

HeliosProxy is a PostgreSQL wire-protocol proxy that sits between client applications and a cluster of PostgreSQL-compatible backends. It intercepts, inspects, routes, caches, and transforms queries without requiring changes to client code or database schemas.

```
                                   HeliosProxy
                      +-----------------------------------------+
                      |                                         |
  Client Connections  |  Protocol   Pipeline   Load Balancer    |  Backend Connections
  ───────────────────>|  Decoder ──> Engine ──> & Router ───────|──────────────────>
  (PostgreSQL wire)   |     |          |            |           |  (Primary, Standby,
                      |     v          v            v           |   Replica nodes)
                      |  Auth     Query Cache   Failover       |
                      |  Proxy    (L1/L2)       Controller     |
                      |     |          |            |           |
                      |     v          v            v           |
                      |  Rate      Analytics    Switchover     |
                      |  Limiter   & Slow Log   Buffer         |
                      |                                         |
                      |  Admin API (REST) ── Port 9090          |
                      +-----------------------------------------+
```

---

## Module Map

The modules are organized into categories. Modules marked *(core)* are always compiled; all others are gated behind Cargo feature flags and excluded from the binary when not needed.

### Core Modules (Always Available)

These modules form the minimum viable proxy. They are compiled unconditionally and provide connection management, routing, health checking, and failover.

| Module | Source | Responsibility |
|--------|--------|----------------|
| `server` | `src/server.rs` | TCP listener, client session management, PostgreSQL startup handshake |
| `protocol` | `src/protocol.rs` | PostgreSQL wire-protocol codec (frontend/backend message framing) |
| `config` | `src/config.rs` | TOML configuration loading, `${VAR}` environment substitution, and validation |
| `admin` | `src/admin.rs` | REST API server on the admin port, Prometheus metrics, SQL routing API |
| `connection_pool` | `src/connection_pool.rs` | Basic connection pool with min/max sizing, idle timeout, and health-on-acquire |
| `load_balancer` | `src/load_balancer.rs` | Read/write splitting, five routing strategies, weighted node selection |
| `health_checker` | `src/health_checker.rs` | Periodic backend health probes with configurable thresholds |
| `failover_controller` | `src/failover_controller.rs` | Failover with candidate ranking and sync-standby preference. **Library component:** `FailoverController` has no construction site in the daemon — `src/server.rs` runs its own independent failover path — so `FailoverConfig` (including `max_history`) is a library API, not a `proxy.toml` section. |
| `switchover_buffer` | `src/switchover_buffer.rs` | Query buffer during planned switchover, drains to new primary |
| `primary_tracker` | `src/primary_tracker.rs` | Pluggable topology discovery and primary-change events. The daemon runs it when `[topology] provider` is `postgres` or `patroni` (feature `postgres-topology`); with the default `static` provider it derives the primary from static `[[nodes]]` roles plus health checks. Both report through `/topology`. See [Topology Providers](/docs/proxy/topology-providers/). |
| `pipeline` | `src/pipeline.rs` | Extended query protocol pipelining (Parse/Bind/Execute batching) |
| `batch` | `src/batch.rs` | INSERT coalescing into multi-row batches. A **standalone utility**, genuinely functional (it executes the combined statement and reports the true outcome) but **not mounted on the proxy's per-query path**. |

### Connection Pooling Modes (`pool-modes`)

| Module | Source | Responsibility |
|--------|--------|----------------|
| `pool::mode` | `src/pool/mode.rs` | Session, Transaction, and Statement pooling mode logic |
| `pool::manager` | `src/pool/manager.rs` | Pool lifecycle management, lease acquisition, connection return |
| `pool::lease` | `src/pool/lease.rs` | Connection lease tracking with automatic release |
| `pool::prepared` | `src/pool/prepared.rs` | Prepared statement tracking across pooled connections |
| `pool::reset` | `src/pool/reset.rs` | Connection reset sequences (DISCARD ALL, SET defaults) |
| `pool::hardening` | `src/pool/hardening.rs` | Connection validation and hardening before reuse |
| `pool::session` | `src/pool/session.rs` | Session-mode pool backend |
| `pool::transaction` | `src/pool/transaction.rs` | Transaction-mode pool backend |
| `pool::statement` | `src/pool/statement.rs` | Statement-mode pool backend |

### Transaction Replay

The recovery path a live client takes and the journal/operator tooling are both part of the
**default build**. `tr_enabled = false` disables journaling, in-session recovery and the
`/api/replay` endpoint at runtime.

| Module | Feature Flag | Source | Responsibility |
|--------|-------------|--------|----------------|
| In-session recovery | *(core)* | `src/server.rs` | Records statements, session state and a bounded response digest inside a transaction; on a backend fault classifies delivery, re-homes the session, restores `SET` state and replays under one deadline |
| Replay SQL lexer | *(core)* | `src/replay_sql.rs` | Bounded statement scan: commit boundaries, transaction ends, function calls — decides what may be re-executed |
| `transaction_journal` | *(default)* | `src/transaction_journal.rs` | Recovery journal of committed transactions (statements, bound parameters, outcomes), in memory or durable with `[journal] dir`; feeds operator replay |
| `failover_replay` | *(default)* | `src/failover_replay.rs` | Replay coordinator for embedded/programmatic use |
| `session_migrate` | *(default)* | `src/session_migrate.rs` | Session state capture/restore library. **Not on the recovery path** — the core path does its own `SET` tracking |
| `cursor_restore` | *(default)* | `src/cursor_restore.rs` | Cursor position library. **Not on the recovery path**; cursors are not restored across a failover |

### Query Intelligence

| Module | Feature Flag | Source | Responsibility |
|--------|-------------|--------|----------------|
| Query Cache | `query-cache` | `src/cache/` | Two-tier result cache (L1 hot, L2 warm); a semantic tier is planned |
| Query Routing | `routing-hints` | `src/routing/` | SQL comment-based routing hints, automatic read/write classification |
| Lag-Aware Routing | `lag-routing` | `src/lag/` | Replication lag monitoring, read-your-writes consistency |
| Query Rewriter | `query-rewriting` | `src/rewriter/` | Rule-based SQL transformation engine |
| Query Analytics | `query-analytics` | `src/analytics/` | Query fingerprinting, slow query log, N+1 detection, intent classification |
| Schema Routing | `schema-routing` | `src/schema_routing/` | Table-level routing rules, data temperature classification |

### Security and Access Control

| Module | Feature Flag | Source | Responsibility |
|--------|-------------|--------|----------------|
| Auth Proxy | `auth-proxy` | `src/auth/` | Proxy-terminated client auth: JWT (HS256), API keys, OAuth RFC 7662 introspection. Also hosts proxy-terminated SCRAM-SHA-256 (`src/auth_scram.rs`, `[auth] mode = "scram"`). |
| LDAP Auth | `ldap-auth` | `src/auth/` | Search-then-bind directory authentication (StartTLS / ldaps). Implies `auth-proxy`; off by default because it pulls `ldap3`. |
| Rate Limiter | `rate-limiting` | `src/rate_limit/` | Token bucket, sliding window, concurrency guards |
| Circuit Breaker | `circuit-breaker` | `src/circuit_breaker/` | Adaptive circuit breaker with sliding-window failure detection |
| Anomaly Detection | `anomaly-detection` | `src/anomaly/` | In-process sliding-window heuristics: rate spikes, credential stuffing, SQL-injection patterns, novel query shapes. Tuned via `[anomaly]`; built once at startup, so changes need a restart. |

### Multi-Tenancy and Extensibility

| Module | Feature Flag | Source | Responsibility |
|--------|-------------|--------|----------------|
| Multi-Tenancy | `multi-tenancy` | `src/multi_tenancy/` | Tenant isolation, per-tenant pools, resource quotas |
| WASM Plugins | `wasm-plugins` | `src/plugins/` | Sandboxed WASM runtime with hot-reload |
| GraphQL Gateway | `graphql-gateway` | `src/graphql/` | GraphQL-to-SQL translation with DataLoader batching |

### Distributed Caching

| Module | Feature Flag | Source | Responsibility |
|--------|-------------|--------|----------------|
| DistribCache | `distribcache` | `src/distribcache/` | Multi-tier distributed cache with AI workload classification. A genuinely functional standalone subsystem, **not mounted on the per-query data path** — `query-cache` is the data-path cache. |
| Edge / Geo Proxy | `edge-proxy` | `src/edge/` | Two-region result cache with a home-authoritative clock, SSE invalidation push and a PG-wire data plane (`[edge]`). Cannot be combined with `[cache] enabled = true`. |

---

## Data-Flow Pipeline

Every client query traverses a well-defined pipeline. Each stage is optional and only executes when its corresponding feature flag is enabled.

```
Client Connection (TCP, PostgreSQL wire protocol)
        |
        v
  +-----------+
  | Protocol  |  Decode PostgreSQL frontend messages
  | Decoder   |  (Query, Parse, Bind, Execute, etc.)
  +-----------+
        |
        v
  +-----------+
  | Auth      |  [auth-proxy] JWT / OAuth / API key validation
  | Proxy     |  Skip if feature not enabled; pass-through auth to backend
  +-----------+
        |
        v
  +-----------+
  | Tenant    |  [multi-tenancy] Identify tenant from connection metadata
  | Identify  |  Apply per-tenant pool and quota limits
  +-----------+
        |
        v
  +-----------+
  | Rate      |  [rate-limiting] Token bucket / sliding window check
  | Limiter   |  Reject with error if rate exceeded
  +-----------+
        |
        v
  +-----------+
  | Circuit   |  [circuit-breaker] Check breaker state for target node
  | Breaker   |  Reject immediately if circuit is open
  +-----------+
        |
        v
  +-----------+
  | WASM      |  [wasm-plugins] Execute on_query_start hooks
  | Plugins   |  Plugins may modify, reject, or annotate the query
  +-----------+
        |
        v
  +-----------+
  | Query     |  [query-rewriting] Apply rewrite rules
  | Rewriter  |  Schema prefixing, query normalization
  +-----------+
        |
        v
  +-----------+
  | Query     |  [query-analytics] Fingerprint and classify the query
  | Analytics |  Record execution start time for latency tracking
  +-----------+
        |
        v
  +-----------+
  | Routing   |  Parse SQL hints (/*helios:route=primary*/)
  | Hints     |  [routing-hints] Override default routing decision
  +-----------+
        |
        v
  +-----------+
  | Query     |  [query-cache] Check L1 -> L2 cache tiers
  | Cache     |  Return cached result if hit; skip backend entirely
  +-----------+
        |
        v  (cache miss)
  +-----------+
  | Schema    |  [schema-routing] Route based on table metadata
  | Router    |  and data temperature classification
  +-----------+
        |
        v
  +-----------+
  | Lag-Aware |  [lag-routing] Filter replicas by replication lag
  | Router    |  Enforce read-your-writes consistency
  +-----------+
        |
        v
  +-----------+
  | Load      |  Select target node (round-robin, least-conn, latency-based)
  | Balancer  |  Read/write splitting: writes -> primary, reads -> standbys
  +-----------+
        |
        v
  +-----------+
  | Connection|  Acquire backend connection from pool
  | Pool      |  [pool-modes] Session / Transaction / Statement mode
  +-----------+
        |
        v
  +-----------+
  | Pipeline  |  Batch Parse/Bind/Execute for reduced round trips
  | Engine    |  Record statement + response digest when in a transaction (core)
  |           |  [TR] Journal statement for operator replay
  +-----------+
        |
        v
  +-----------+
  | Batch     |  Coalesce individual INSERTs into multi-row batches
  | INSERT    |
  +-----------+
        |
        v
  Backend Node (Primary / Standby / Replica)
        |
        v  (response)
  +-----------+
  | Query     |  [query-cache] Populate cache on miss
  | Cache     |  [query-analytics] Record execution time, update statistics
  | Populate  |  [wasm-plugins] Execute on_query_complete hooks
  +-----------+
        |
        v
  Client Response (PostgreSQL wire protocol)
```

---

## Topology Provider Abstraction

The `PrimaryTracker` is the central component for primary node discovery. It is decoupled from any specific database backend through the `TopologyProvider` trait.

### TopologyProvider Trait

```rust
pub trait TopologyProvider: Send + Sync + 'static {
    /// Subscribe to topology change events.
    fn subscribe(&self) -> broadcast::Receiver<TopologyEvent>;

    /// Get the current primary node, if one exists.
    fn get_primary(&self) -> Option<TopologyNodeInfo>;

    /// Look up a node by its UUID.
    fn get_node(&self, id: Uuid) -> Option<TopologyNodeInfo>;
}
```

### Topology Events

```rust
pub enum TopologyEvent {
    PrimaryChanged { old_primary: Option<Uuid>, new_primary: Uuid },
    NodeLeft { node_id: Uuid },
    HealthChanged { node_id: Uuid, is_healthy: bool },
}
```

### Provider Implementations

| Provider | Feature Flag | Discovery Method | Failover Detection |
|----------|-------------|------------------|--------------------|
| PostgreSQL | `postgres-topology` | Polls `pg_is_in_recovery()` on each node | Detects role change across polling intervals |
| Manual / Standalone | *(none)* | API calls: `set_primary()`, `clear_primary()` | External orchestration |

### PrimaryTracker Operating Modes

1. **Provider-backed** -- Created with `PrimaryTracker::with_provider()`. Subscribes to `TopologyEvent` broadcasts and runs a periodic consistency check. Fully automatic.

2. **Standalone** -- Created with `PrimaryTracker::new_standalone()`. Primary is managed through explicit `set_primary()`, `confirm_primary()`, and `clear_primary()` calls. Suitable for external failover managers (Patroni, pg_auto_failover, Stolon) that notify the proxy via the Admin API.

### Primary Lifecycle

```
  set_primary(id, address)       confirm_primary()        clear_primary()
  ───────────────────────> PENDING ─────────────────> CONFIRMED ──────────> NONE
                             |                                     ^
                             |          (node lost / unhealthy)    |
                             +─────────────────────────────────────+
```

Events are broadcast to all subscribers (load balancer, failover controller, switchover buffer) on each transition.

---

## Failover Sequence

### What a live session does (core path)

This is what a connected client experiences when its backend dies. It runs in every
build; `tr_enabled = false` disables it (`tr_mode` is forced to `none`).

```
1. Backend fault detected mid-statement
      |
2. Classify how far the statement got:
   a. Not delivered            -> safe to recover
   b. Outcome unknown          -> NEVER re-executed; client gets 08007
   c. Response already visible -> session closed rather than append a second result
      |
3. One deadline starts (write_timeout_secs) and covers steps 4-7
      |
4. Wait for a healthy primary
      |
5. Connect and authenticate the replacement backend
      |
6. Restore tracked session state (SET/RESET, savepoint-scoped, cap-checked)
      |
7. Replay the recorded transaction, verifying each statement's response digest
   against what the client already saw -- divergence => ROLLBACK + 40001
      |
8. Re-execute the interrupted statement and resume streaming to the client
```

Recovery is declined, conservatively, for a `SERIALIZABLE`/`REPEATABLE READ`
transaction, a response too large to have been digested, a read calling anything
outside the side-effect-free set, or a session whose tracked `SET` count exceeded its
cap. See [transaction-replay.md](/docs/proxy/transaction-replay/#in-session-replay-the-core-path).

### Controller sequence (embedded / library use)

`FailoverController`, `PrimaryTracker` and `SwitchoverBuffer` are library components
for embedded use; the standalone daemon does not drive them. They orchestrate:

```
1. Health checker detects primary failure (failure_threshold consecutive failures)
      |
2. Failover controller enters FailoverState::Detecting
      |
3. Switchover buffer activates -- incoming writes are buffered
      |
4. Failover controller ranks candidates by:
   a. Sync standby preference (is_sync = true ranked first)
   b. Replication lag (lowest lag_bytes preferred)
   c. Configured priority
      |
5. Best candidate selected, primary tracker updated
      |
6. [TR] Transaction journal identifies in-flight transactions
      |
7. [TR] Failover replay re-executes journaled statements on new primary
      |
8. [library, not wired] Session migrate would restore SET parameters and prepared
   statements. The core path restores SET state itself; SQL-level PREPARE and
   cursors are not restored by either path (extended-protocol named prepared
   statements are lazily re-prepared on the new backend)
      |
9. [library, not wired] Cursor restore would reposition open cursors
      |
10. Primary tracker confirmed, switchover buffer drains to new primary
      |
11. Normal operation resumes
```

---

## Connection Pool Architecture

The connection pool operates in three modes, each with different connection return semantics:

```
                    +-----------+
                    |  Client   |
                    | Session   |
                    +-----+-----+
                          |
                   acquire_connection()
                          |
                    +-----v-----+
                    |   Pool    |
                    |  Manager  |
                    +-----+-----+
                          |
              +-----------+-----------+
              |           |           |
        +-----v---+ +----v----+ +----v----+
        | Session | | Txn     | | Stmt    |
        | Mode    | | Mode    | | Mode    |
        +---------+ +---------+ +---------+
        Return on    Return on   Return on
        disconnect   COMMIT/     each
                     ROLLBACK    statement
```

Each mode passes the connection through a reset sequence (`DISCARD ALL` by default) before returning it to the pool. Prepared statement tracking (`track` or `named` mode) preserves statement definitions across connection reuse.

---

## Admin API Architecture

The admin server runs on a separate TCP listener (default port 9090) and exposes a REST API for monitoring, management, and SQL routing.

```
  Admin Port (9090)
        |
  +-----v-----------+
  |  HTTP Parser     |
  |  (built-in)      |
  +-----+-----------+
        |
  +-----v-----------+
  |  Request Router  |
  |  (method + path) |
  +-----+-----------+
        |
        +-- GET  /health         --> Liveness check
        +-- GET  /health/ready   --> Readiness (at least one healthy node)
        +-- GET  /health/live    --> Simple alive check
        +-- GET  /nodes          --> All node health status
        +-- GET  /nodes/{addr}   --> Single node health
        +-- POST /nodes/{addr}/enable  --> Enable node
        +-- POST /nodes/{addr}/disable --> Disable node
        +-- GET  /config         --> Current configuration snapshot
        +-- GET  /metrics        --> JSON metrics
        +-- GET  /metrics/prometheus --> Prometheus text format
        +-- GET  /sessions       --> Active session count
        +-- GET  /pools          --> Pool statistics
        +-- GET  /version        --> Proxy version
        +-- POST /api/sql        --> SQL execution with TWR routing
```

---

## See Also

- [Configuration Reference](/docs/proxy/configuration/)
- [Feature Flags](/docs/proxy/feature-flags/)
- [Admin API Reference](/docs/proxy/admin-api/)
- [Topology Providers](/docs/proxy/topology-providers/)