Real-Time Pub/Sub (LISTEN/NOTIFY)

Executive Summary

With thread-safe in-memory channel management, automatic subscription cleanup, and 8KB payload support, HeliosDB Lite enables event-driven applications to implement real-time dashboards, cache invalidation, activity feeds, workflow automation, and microservices coordination entirely within embedded, edge, and microservice deployments without Redis, Kafka, or message queue infrastructure.

Problem Being Solved

Core Problem Statement

Teams deploying to edge devices, embedded systems, or resource-constrained environments cannot afford external message queue dependencies, yet lack a lightweight pub/sub mechanism for event-driven patterns.

Who Suffers Most

• IoT/Edge Application Developers: Building collaborative tools or real-time monitoring on edge devices where external message brokers are unavailable, forcing them to implement inefficient polling or abandon real-time features entirely, degrading user experience significantly.

Concrete Examples with Code, Config & Architecture

Example 1: Real-Time Dashboard Updates - Embedded Configuration

Architecture:

Web Browser (WebSocket)
    ↓
Web Server (Axum/Actix)
    ↓
HeliosDB-Lite (LISTEN 'sales_updates')
    ↓
In-Process Notification Queue
    ↓
Database Trigger on INSERT → NOTIFY 'sales_updates'

Configuration (heliosdb.toml):

# HeliosDB-Lite configuration for real-time dashboard
[database]
path = "/var/lib/heliosdb/sales.db"
memory_limit_mb = 512
enable_wal = true

[pubsub]
# Enable pub/sub with optimized settings
enabled = true
max_channels = 1000
max_payload_bytes = 8000
notification_queue_size = 100

[monitoring]
metrics_enabled = true
verbose_logging = false

Example 2: Cache Invalidation - Python Integration

Scenario: Multi-instance web application with in-memory caches that must invalidate when source data changes, without polling or manual TTL management.

Architecture Pattern:

┌─────────────────────────────────────────┐
│  Application Instance 1                  │
│  ┌────────────────┐  ┌────────────────┐ │
│  │ In-Memory Cache│  │ LISTEN Thread  │ │
│  └────────────────┘  └────────────────┘ │
├─────────────────────────────────────────┤
│  Application Instance 2                  │
│  ┌────────────────┐  ┌────────────────┐ │
│  │ In-Memory Cache│  │ LISTEN Thread  │ │
│  └────────────────┘  └────────────────┘ │
├─────────────────────────────────────────┤
│         HeliosDB-Lite (Shared)           │
│  Database Trigger → NOTIFY all listeners │
└─────────────────────────────────────────┘

Flow:
1. Instance 1: UPDATE products SET price = 19.99 WHERE id = 1
2. Database Trigger: NOTIFY 'cache_invalidate', 'product:1'
3. All Instances: Receive notification, invalidate local cache
4. Next read: Cache miss, fetch fresh data

Example 3: Activity Feed - Microservices Integration (Go/Rust)

Scenario: Social platform with 10,000 users, displaying real-time activity feed updates (likes, comments, follows) without polling every user's feed.

Service Architecture:

┌─────────────────────────────────────────┐
│     Client (Browser / Mobile App)       │
│     Server-Sent Events (SSE) Stream     │
├─────────────────────────────────────────┤
│     Activity Microservice (Rust)        │
│  ┌────────────────────────────────────┐ │
│  │  POST /activities                  │ │
│  │    ↓                               │ │
│  │  INSERT INTO activities            │ │
│  │    ↓                               │ │
│  │  Trigger fires NOTIFY              │ │
│  └────────────────────────────────────┘ │
│  ┌────────────────────────────────────┐ │
│  │  GET /activities/stream/:user_id   │ │
│  │    ↓                               │ │
│  │  LISTEN 'activity_feed_123'        │ │
│  │    ↓                               │ │
│  │  Poll subscription → SSE stream    │ │
│  └────────────────────────────────────┘ │
├─────────────────────────────────────────┤
│         HeliosDB-Lite (In-Process)      │
│  PubSubManager → Notification Delivery  │
└─────────────────────────────────────────┘

Example 4: Workflow Automation - Docker/Container Deployment

Scenario: Order processing workflow where each step triggers the next automatically via database events, eliminating manual coordination.

Docker Deployment (Dockerfile):

FROM rust:latest as builder

WORKDIR /app
COPY . .
RUN cargo build --release

FROM debian:bookworm-slim

RUN apt-get update && apt-get install -y \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /app/target/release/workflow-engine /usr/local/bin/

RUN mkdir -p /data

EXPOSE 8080

VOLUME ["/data"]

ENTRYPOINT ["workflow-engine"]
CMD ["--config", "/etc/workflow/config.toml"]

Docker Compose (docker-compose.yml):

version: '3.8'

services:
  workflow-engine:
    build:
      context: .
      dockerfile: Dockerfile
    image: workflow-engine:latest

    ports:
      - "8080:8080"

    volumes:
      - ./data:/data
      - ./config/workflow.toml:/etc/workflow/config.toml:ro

    environment:
      RUST_LOG: "workflow_engine=info"
      HELIOSDB_DATA_DIR: "/data"

    restart: unless-stopped

    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 3s
      retries: 3

networks:
  default:
    driver: bridge

Workflow Configuration (config/workflow.toml):

[database]
path = "/data/workflow.db"
memory_limit_mb = 256
enable_wal = true

[pubsub]
enabled = true
max_channels = 100

[workflow]
# Workflow step definitions
[workflow.steps.payment_processing]
trigger_channel = "order_created"
action = "process_payment"
next_channel = "payment_completed"

[workflow.steps.inventory_allocation]
trigger_channel = "payment_completed"
action = "allocate_inventory"
next_channel = "inventory_allocated"

[workflow.steps.shipping]
trigger_channel = "inventory_allocated"
action = "create_shipment"
next_channel = "shipment_created"

Workflow Architecture:

Order Created (INSERT)
    ↓
Trigger fires NOTIFY 'order_created'
    ↓
Payment Processor (LISTEN 'order_created')
    ↓ (process payment)
    ↓ NOTIFY 'payment_completed'
    ↓
Inventory Allocator (LISTEN 'payment_completed')
    ↓ (allocate inventory)
    ↓ NOTIFY 'inventory_allocated'
    ↓
Shipment Creator (LISTEN 'inventory_allocated')
    ↓ (create shipment)
    ↓ Order Complete

Example 5: Microservices Event Bus - Edge/IoT Deployment

Scenario: IoT sensor network with 100 edge devices coordinating via lightweight event bus for distributed alerting without cloud connectivity.

Edge Device Configuration (edge-config.toml):

[database]
path = "/var/lib/iot/sensors.db"
memory_limit_mb = 64        # Low memory for edge devices
page_size = 512
enable_wal = true

[pubsub]
enabled = true
max_channels = 50
max_payload_bytes = 1024    # Smaller payloads for IoT

[edge]
device_id = "sensor_001"
location = "warehouse_a"

Adoption Strategy

Phase 1: Proof of Concept (Weeks 1-4)

Target: Validate LISTEN/NOTIFY for one real-time feature

Tactics:

• Replace polling-based cache invalidation with LISTEN/NOTIFY in development environment
• Measure CPU reduction and latency improvement
• Test with 100-1000 notifications per second
• Validate PostgreSQL client compatibility (psycopg2, node-postgres)

Phase 2: Pilot Deployment (Weeks 5-12)

Target: Deploy to production for one critical use case

Tactics:

• Deploy real-time dashboard or cache invalidation to production
• Monitor notification latency and throughput with metrics
• Gather user feedback on real-time update responsiveness
• Measure infrastructure cost savings (Redis elimination)

Phase 3: Full Rollout

Target: Organization-wide adoption for all event-driven features

Tactics:

• Expand to all polling-based features (activity feeds, workflow automation, etc.)
• Implement database triggers for automatic event generation
• Migrate from Redis Pub/Sub to HeliosDB Lite LISTEN/NOTIFY
• Document best practices and architectural patterns for team

Conclusion

With transaction-integrated NOTIFY delivery, automatic trigger-based events, and native database integration, HeliosDB Lite transforms event-driven architecture from a complex distributed systems problem into a simple database feature.

Start your proof of concept today: replace one polling-based cache invalidation or real-time dashboard feature with LISTEN/NOTIFY, measure the CPU and latency improvements, and experience the simplicity of event-driven architecture without external dependencies.