AI-Native at the Database Layer: Chat Proxy + Vector Store + Transcribe + SSE

HeliosDB Full v7.5 and v7.6 collapsed an entire AI-app middle tier into the database. The chat proxy, vector store, embedding service, transcribe endpoint, SSE streaming, agents/documents/quotas surface — every endpoint a modern AI app expects from a Pinecone-plus-OpenAI-proxy stack — now lives behind one Postgres-wire instance. This post walks through what shipped and how the surface composes.

The endpoints

v7.5 added Tier 1: chat proxy and vector store.

POST /api/v1/chat/completions       # OpenAI-compatible chat proxy
POST /api/v1/embeddings              # model-routed embedding generation
POST /api/v1/vectors/search/hybrid   # BM25 + HNSW hybrid retrieval

v7.6 added Tier 2 (agents, documents) and Tier 3 (embeddings as a managed service, transcribe, SSE, quotas):

GET  /api/v1/agents/:id/threads      # multi-turn agent state
POST /api/v1/agents/:id/threads/:t   # append message, get streamed response
POST /api/v1/documents               # upload + chunk + embed + store
POST /api/v1/transcribe              # audio → text via routed model
GET  /api/v1/events                  # Server-Sent Events for stream subscription
GET  /api/v1/tenants/:id/quota       # per-tenant token / request budgets

The chat proxy

The chat proxy is OpenAI-compatible: same JSON, same streaming format, same tool-use schema. Behind it, requests fan out to whichever model the tenant's policy routes them to — OpenAI, Anthropic, an on-prem Ollama, a fine-tuned model on a GPU node. The proxy maintains a token budget per tenant (the v7.6 quota module enforces it), logs every request to a CDC stream you can subscribe to, and persists conversation state to the vector-store-backed thread table.

The vector store and hybrid search

v7.6 wired HNSW into /api/v1/vectors/search/hybrid (the perf wave: see "What Changed in HeliosDB Full Perf"). The hybrid endpoint accepts a query string + optional vector, computes BM25 over the FTS index, fuses the two ranked lists with RRF, and returns the top-k. Same surface as the SQL hybrid_search() table function, but over HTTP for non-SQL clients.

SSE streaming

SSE was the obvious binding for chat completions and agent threads — it composes cleanly with HTTP/2, every browser supports it natively, and it doesn't require a websocket upgrade dance. The /api/v1/events endpoint accepts a query and streams updates as text/event-stream. Subscribers get cursor-based deltas; reconnection is handled by the standard Last-Event-Id header.

Transcribe

POST /api/v1/transcribe accepts audio bytes, routes to a configured speech-to-text model (Whisper-class), and returns text plus optional segment timestamps. The implementation is intentionally thin — it's a routed proxy, not an embedded model.

Why it lives in the database

Three reasons. State. Chat threads, agent memory, documents, and embeddings all want to live next to the relational data they refer to. Putting the proxy in the database means a chat reply can include a SELECT over the user's own rows without crossing a service boundary. Quotas. Per-tenant token budgets need to read and write authoritative state — the database is already that state. Auditability. Every prompt, every embedding, every transcription becomes a row you can SELECT over for compliance.

HeliosDB Full is source-available under SSPL-1.0. Repository: github.com/dimensigon/HDB-HeliosDB-Full.

← Back to Blog