HeliosDB vs MinIO

Choose HeliosDB when you need to query, search and transact over your data in SQL; keep MinIO when you need low-cost, erasure-coded S3 storage for large immutable objects at petabyte scale. Run both side by side: MinIO stores the files, and HeliosDB stores the rows, metadata and embeddings that describe them. Reading files in MinIO directly from HeliosDB Full through S3 foreign tables is planned.

MinIO is a widely deployed self-hosted S3-compatible object store, fast and reliable at petabyte scale. But storing objects and understanding them are two very different problems. HeliosDB is a unified database engine that combines SQL, full-text search, vector search, ACID transactions, and AI-native capabilities in a single binary. When your application needs to query, search, and reason about the data inside those objects, MinIO alone falls short -- and HeliosDB fills that gap. The two run side by side: MinIO keeps the objects, and HeliosDB keeps the queryable data that describes them.

---

Quick Comparison

---

The Architecture Problem

With MinIO: The Object Storage Stack

When you need to search, query, or analyze data stored in MinIO, you end up building a pipeline of external services:

+--------------+     +--------------+     +--------------+
|   MinIO      |---->|  ETL / Spark |---->|  Data        |
|  (storage)   |     |  (transform) |     |  Warehouse   |
+--------------+     +--------------+     +--------------+
       |                                         |
       |              +--------------+           |
       +----------->  |  OpenSearch  |<----------+
                      |  (FTS)       |
                      +--------------+
       |              +--------------+
       +----------->  |  Milvus      |
                      |  (vectors)   |
                      +--------------+

 5 services to deploy, monitor, and keep in sync
 No ACID guarantees across the pipeline
 Data freshness measured in minutes to hours

With HeliosDB: One Engine, Everything Built In

+---------------------------------------------+
|                 HeliosDB                     |
|                                             |
|  +---------+  +------+  +--------------+   |
|  |   SQL   |  |  FTS |  | Vector Search|   |
|  |  Engine |  |  @@  |  |   HNSW+PQ    |   |
|  +----+----+  +--+---+  +------+-------+   |
|       |          |              |            |
|  +----+----------+--------------+----------+ |
|  |        MVCC Storage Engine              | |
|  |    (ACID + TDE + WAL + Branching)       | |
|  +-----------------------------------------+ |
|                                             |
|  Beside it: MinIO keeps the files           |
+---------------------------------------------+

 1 binary, everything ACID-compliant
 Query, search, and analyze in one SQL statement
 Real-time: data available instantly on INSERT

---

S3 Select vs Full SQL Optimizer

MinIO supports S3 Select for basic filtering of CSV, JSON, and Parquet objects. HeliosDB provides a complete SQL engine with a cost-based optimizer, parallel execution, and 69+ plan node types.

MinIO: S3 Select

import boto3

s3 = boto3.client('s3', endpoint_url='http://minio:9000',
                  aws_access_key_id='minioadmin',
                  aws_secret_access_key='minioadmin')

# S3 Select: filter a single CSV object
result = s3.select_object_content(
    Bucket='sales-data',
    Key='2026/01/transactions.csv',
    Expression="SELECT s.amount FROM s3object s WHERE s.region = 'EU'",
    ExpressionType='SQL',
    InputSerialization={'CSV': {'FileHeaderInfo': 'USE'}},
    OutputSerialization={'CSV': {}}
)

# Limitations:
#  - Cannot JOIN across objects
#  - Cannot aggregate (no SUM/AVG/COUNT)
#  - Cannot use subqueries or CTEs
#  - One file at a time
#  - No window functions

HeliosDB: Full SQL with Optimizer

-- Connect via any PostgreSQL client (psql, pgAdmin, DBeaver, etc.)

-- Complex analytics query: JOINs, CTEs, window functions, aggregation
WITH monthly_sales AS (
    SELECT
        region,
        date_trunc('month', sale_date) AS month,
        SUM(amount) AS total,
        COUNT(*) AS txn_count
    FROM transactions
    WHERE sale_date >= '2026-01-01'
    GROUP BY region, date_trunc('month', sale_date)
),
ranked AS (
    SELECT *,
        RANK() OVER (PARTITION BY month ORDER BY total DESC) AS region_rank,
        LAG(total) OVER (PARTITION BY region ORDER BY month) AS prev_month
    FROM monthly_sales
)
SELECT
    region,
    month,
    total,
    txn_count,
    region_rank,
    ROUND((total - prev_month) / prev_month * 100, 2) AS growth_pct
FROM ranked
WHERE region_rank <= 3
ORDER BY month, region_rank;

To query files that live in MinIO, load them into tables first. Reading Parquet or CSV files in MinIO directly from HeliosDB Full, through S3 foreign tables, is planned.

---

Encryption: SSE vs TDE

MinIO: Server-Side Encryption

MinIO offers three encryption modes, all operating at the object level:

# MinIO SSE-C: client must send key with every request
from minio import Minio
from minio.sse import SseCustomerKey
import os

key = os.urandom(32)
sse = SseCustomerKey(key)

client = Minio('minio:9000', access_key='admin', secret_key='password', secure=False)
client.put_object('secure-bucket', 'doc.pdf', data, length, sse=sse)

# Problem: lose the key, lose the data
# Problem: key transmitted over the wire on every request
# Problem: no encryption of indexes or metadata queries

HeliosDB: Transparent Data Encryption

Transparent data encryption is AES-256-GCM. In HeliosDB Nano it is compiled in but off until an operator enables it and supplies a key, and key rotation is not implemented yet (Nano FAQ · security hardening). Encryption seals values, not keys: table names, column names, row ids and timestamps remain readable on disk, as do HNSW graph snapshots and backup dumps.

-- TDE (AES-256-GCM): once enabled with a key, data at rest is
-- encrypted transparently; in Nano it is off until configured
-- No application code changes needed

INSERT INTO medical_records (patient_id, diagnosis, notes)
VALUES (42, 'confidential', 'encrypted at rest once TDE is on');

-- Queries work normally -- decryption is transparent
SELECT * FROM medical_records WHERE patient_id = 42;

---

Document Intelligence: Opaque Blobs vs Queryable Data

MinIO: Store and Retrieve

# Upload a JSON document to MinIO
client.put_object('documents', 'invoice-1234.json',
    io.BytesIO(json.dumps(invoice).encode()),
    content_type='application/json')

# To search this document, you need:
# 1. Download it
# 2. Parse it
# 3. Index it in OpenSearch/MeiliSearch
# 4. Query the search engine
# 5. Fetch the matching objects back from MinIO
# Total: 5 steps, 3 services, no ACID guarantees

HeliosDB: Store, Query, Search, Analyze -- All in SQL

-- Store the document as JSONB with full queryability
INSERT INTO invoices (id, data, embedding)
VALUES (
    1234,
    '{"vendor": "Acme Corp", "total": 4500.00,
      "items": [{"desc": "Widget", "qty": 100, "price": 45.00}],
      "tags": ["hardware", "bulk"]}',
    '[0.12, -0.45, 0.89, ...]'::vector(384)
);

-- Query deeply nested JSON fields
SELECT
    data->>'vendor' AS vendor,
    (data->>'total')::numeric AS total,
    jsonb_array_length(data->'items') AS line_items
FROM invoices
WHERE data @> '{"tags": ["hardware"]}'
  AND (data->>'total')::numeric > 1000
ORDER BY total DESC;

-- Full-text search across all invoice data
-- (@@ is true when any query term is present)
SELECT id, data->>'vendor' AS vendor
FROM invoices
WHERE to_tsvector(data::text) @@ to_tsquery('widget');

-- Vector similarity search (find semantically similar invoices)
SELECT id, data->>'vendor' AS vendor,
       embedding <-> query_embedding AS distance
FROM invoices
ORDER BY embedding <-> '[0.11, -0.44, 0.88, ...]'::vector(384)
LIMIT 5;

-- Hybrid search: combine text + vector + SQL filters
SELECT id, data->>'vendor' AS vendor
FROM invoices
WHERE data::text @@ 'hardware'
  AND embedding <-> '[0.11, -0.44, 0.88, ...]'::vector(384) < 0.3
  AND (data->>'total')::numeric > 500
ORDER BY embedding <-> '[0.11, -0.44, 0.88, ...]'::vector(384);

---

Replication and High Availability

MinIO: Erasure Coding + Site Replication

MinIO Cluster (4-32 nodes)
+-----+  +-----+  +-----+  +-----+
| N1  |  | N2  |  | N3  |  | N4  |
|EC 4 |  |EC 4 |  |EC 4 |  |EC 4 |
+--+--+  +--+--+  +--+--+  +--+--+
   +----+----+----+----+----+--+
        |  Erasure Set (16 drives)
        |  Survives N/2 drive failures

Site Replication:
  Site A <--------> Site B
  (async bucket/object/IAM sync)

• Erasure coding: survives up to N/2 drive failures per set
• Site replication: async, bucket-level, eventual consistency
• No cross-object transactions during replication
• Minimum 4 nodes for production HA

HeliosDB: WAL Streaming Replication

HeliosDB Nano replicates by WAL streaming to a warm standby, async by default with semi-sync and sync modes, and switches over on command; automatic failover runs through HeliosProxy. HeliosDB Full runs as a single server today: Raft clustering, multi-region replication and sharding are planned.

WAL Streaming (Primary -> Standby), HeliosDB Nano
+----------+    WAL Stream    +----------+
| Primary  | ---------------> | Standby  |
| (R/W)    |                  | (R/O)    |
+----------+                  +----------+
  Async by default (semi-sync or sync optional)
  Manual SWITCHOVER; automatic failover through HeliosProxy

Multi-primary replication and sharding: planned for HeliosDB Full

---

AI and RAG Pipelines

MinIO: Storage Layer for AI

# Typical MinIO + AI pipeline (5 services required)

# 1. Store documents in MinIO
client.put_object('docs', 'manual.pdf', pdf_data, len(pdf_data))

# 2. Extract text (external service: Tika, Textract, etc.)
text = tika_client.extract('s3://docs/manual.pdf')

# 3. Chunk text (external: LangChain, LlamaIndex)
chunks = text_splitter.split(text)

# 4. Generate embeddings (external: OpenAI API)
embeddings = openai.embeddings.create(input=chunks, model='text-embedding-3-small')

# 5. Store vectors (external: Milvus/Weaviate using MinIO as backend)
milvus_client.insert(collection, embeddings)

# 6. Search requires querying Milvus, then fetching from MinIO
results = milvus_client.search(query_embedding, limit=5)
docs = [client.get_object('docs', r.key) for r in results]

HeliosDB: Integrated RAG in SQL

-- Everything in one database, one transaction

-- 1. Store documents with embeddings
INSERT INTO knowledge_base (title, content, embedding, metadata)
VALUES (
    'Product Manual v3',
    'HeliosDB supports ACID transactions with MVCC...',
    '[0.12, -0.33, 0.78, ...]'::vector(384),
    '{"category": "docs", "version": 3}'
);

-- 2. RAG retrieval: hybrid search (FTS + vector + metadata filter)
SELECT title, content,
       embedding <-> $1::vector(384) AS semantic_distance
FROM knowledge_base
WHERE content @@ 'ACID transactions'
  AND metadata @> '{"category": "docs"}'
ORDER BY embedding <-> $1::vector(384)
LIMIT 5;

-- 3. Results are ACID-consistent, encrypted at rest once TDE
--    is enabled with a key, and readable as of an earlier time
--    (HeliosDB Nano; time travel for Full is planned)
SELECT title, content
FROM knowledge_base AS OF TIMESTAMP '2026-01-15 00:00:00'
WHERE content @@ 'transactions';

# Python SDK: simple RAG with HeliosDB
import psycopg2

conn = psycopg2.connect("host=localhost port=5432 dbname=helios")
cur = conn.cursor()

# Single query for semantic search + metadata filter + FTS
cur.execute("""
    SELECT title, content
    FROM knowledge_base
    WHERE content @@ %s
      AND embedding <-> %s::vector(384) < 0.5
    ORDER BY embedding <-> %s::vector(384)
    LIMIT 5
""", (user_query, query_embedding, query_embedding))

context_docs = cur.fetchall()
# Feed directly to LLM -- no intermediate services

---

Time-Travel and Data Branching

MinIO provides linear object versioning -- you can retrieve a previous version of an object by its version ID. HeliosDB provides MVCC time-travel queries and git-like data branching.

MinIO: Linear Object Versions

# List versions of a single object
versions = client.list_objects('bucket', 'key', include_version=True)
for v in versions:
    print(v.version_id, v.last_modified)

# Retrieve a specific version
obj = client.get_object('bucket', 'key', version_id='abc-123')

HeliosDB Nano: Time-Travel + Branching

Time travel reads data as far back as versions are retained (by default Nano keeps all history) and is not a substitute for backups. MERGE DATABASE BRANCH is last-writer-wins with no conflict detection, so discard the branch and re-apply the validated SQL to main instead (merging branches). Time travel and branching for HeliosDB Full are planned.

-- Time-travel: read a table as it was at an earlier time
SELECT * FROM users AS OF TIMESTAMP '2026-02-01 00:00:00';

-- Compare data between two timestamps
SELECT
    a.user_id,
    a.balance AS balance_before,
    b.balance AS balance_after,
    b.balance - a.balance AS change
FROM users AS OF TIMESTAMP '2026-01-01' a
JOIN users AS OF TIMESTAMP '2026-02-01' b ON a.user_id = b.user_id
WHERE b.balance != a.balance;

-- Git-like branching: create an isolated copy-on-write branch
CREATE DATABASE BRANCH experiment FROM main;
USE BRANCH experiment;

-- Make changes without affecting production
UPDATE pricing SET price = price * 1.1 WHERE category = 'premium';
INSERT INTO promotions VALUES ('SPRING26', 0.15, '2026-03-01', '2026-04-01');

-- Analyze impact
SELECT category, AVG(price) AS avg_price FROM pricing GROUP BY category;

-- Done: return to main and drop the branch; production is untouched.
-- Re-apply the validated SQL to main. (MERGE DATABASE BRANCH experiment
-- INTO main is last-writer-wins and reports no conflicts.)
USE BRANCH main;
DROP DATABASE BRANCH experiment;

---

Better Together: HeliosDB + MinIO

Run them side by side: MinIO stores the files, and HeliosDB stores one row per object with its bucket, key, size, metadata and embedding, so you search and join in SQL and fetch the object from MinIO by key. Reading Parquet or CSV files in MinIO directly from HeliosDB Full, through S3 foreign tables, is planned.

+-------------------------+          +---------------------+
|        HeliosDB         |          |   MinIO Cluster     |
|                         |   key    |                     |
|  objects table:         | -------> |  bucket/key -> file |
|   bucket, key, size,    |          |  (PDF, Parquet,     |
|   metadata JSONB,       |          |   images, backups)  |
|   embedding VECTOR      |          |                     |
|                         |          |                     |
|  SQL search, joins,     |          |  S3 GET by key      |
|  vector + FTS queries   |          |                     |
+-------------------------+          +---------------------+

import boto3, json, os, psycopg2

s3 = boto3.client('s3', endpoint_url='http://minio-cluster:9000')
conn = psycopg2.connect("host=localhost port=5432 dbname=helios")
cur = conn.cursor()

# 1. Store the file in MinIO
s3.upload_file('report.pdf', 'archive', '2026/report.pdf')

# 2. Record it in HeliosDB with the metadata you query
cur.execute("""
    INSERT INTO objects (bucket, key, size_bytes, metadata)
    VALUES (%s, %s, %s, %s::jsonb)
""", ('archive', '2026/report.pdf', os.path.getsize('report.pdf'),
      json.dumps({"team": "finance"})))
conn.commit()

# 3. Search in SQL, then fetch the matching files from MinIO by key
cur.execute("""
    SELECT bucket, key FROM objects
    WHERE metadata @> '{"team": "finance"}'
    ORDER BY key DESC LIMIT 10
""")
for bucket, key in cur.fetchall():
    s3.download_file(bucket, key, key.replace('/', '_'))

---

Is HeliosDB Full an S3-compatible object store?

Not today. An S3-compatible object API for HeliosDB Full is on the roadmap and is not part of the current release. The bucket-management SQL statements described in earlier release notes are planned together with it; do not rely on them for retention or legal hold (release notes · protocol hub). The rows below compare MinIO with that roadmap.

Until the object API ships, keep files in MinIO and record what you query about them in HeliosDB tables, as in the example above.

---

When to Choose MinIO

MinIO is the right choice when:

• Petabyte-scale blob storage -- You need to store and serve terabytes to petabytes of files (images, videos, backups, logs) at low cost
• S3 API compatibility -- Your existing tools, CI/CD pipelines, and applications already speak S3
• Write-once, read-many -- Your workload is primarily storing immutable objects and retrieving them by key
• Data lake foundation -- You are building a data lake where Spark, Trino, or Presto handle the query layer
• Large binary files -- Video processing, ML model artifacts, container images, and backup archives
• Erasure-coded durability -- You want drive-failure tolerance without full replication overhead

When to Choose HeliosDB

HeliosDB is the right choice when:

• You need to query your data -- SQL with JOINs, CTEs, window functions, aggregations, and subqueries
• ACID transactions matter -- Multi-statement transactions with rollback, savepoints, and isolation
• Search is a core feature -- Full-text search and/or vector similarity search built into the database
• AI/RAG applications -- Store documents, embeddings, and metadata together with hybrid search
• Embedded or edge deployment -- HeliosDB Nano is one binary of about 36-44 MB, depending on platform, that runs in-process or as a server
• Time-travel and branching -- Query historical data, create isolated branches for testing or experimentation
• Schema enforcement -- Typed columns, constraints and foreign keys, plus row-level security (in the stock Nano server it is enforced through the embedded API and the REPL; SQL CREATE POLICY and enforcement for REST and wire clients are planned; deployment modes)
• PostgreSQL client compatibility -- Connect with psql, pgAdmin, DBeaver and most PostgreSQL drivers; wire compatibility is not full semantic parity (Nano FAQ)
• Encryption requirements -- AES-256-GCM encryption at rest (in Nano, off until you enable it in the configuration with a key)

---

When HeliosDB is not the right fit

• You need an S3 API. An S3-compatible object API for HeliosDB Full is on the roadmap and is not part of the current release, and the bucket-management SQL in earlier release notes is planned with it. Release notes · Protocol hub
• You want HeliosDB to query files in MinIO in place. Reading Parquet or CSV files in MinIO through S3 foreign tables is planned for HeliosDB Full; load the data into tables today. Protocol hub
• You need proof at petabyte or multi-node scale. Published HeliosDB benchmarks are single-node on a small dataset; large-dataset and distributed results are not yet published. Benchmarks
• You need an open-source object store. HeliosDB is not an object store today. The S3-compatible object API is planned for HeliosDB Full, which is SSPL-1.0 under an annual commercial license, and file storage with buckets and signed URLs is on the roadmap for the Apache 2.0 Nano. Full FAQ: licensing · Pricing
• You need row-level security as a network-facing boundary in HeliosDB Nano. In the stock Nano server, row-level security is enforced through the embedded API and the REPL; SQL CREATE POLICY and enforcement for REST and wire clients are planned. Nano deployment modes

---

Migration Path

From MinIO to HeliosDB

If you are currently storing queryable data (JSON, CSV, metadata) in MinIO and querying it with external tools:

# Step 1: Export from MinIO
import boto3, json

s3 = boto3.client('s3', endpoint_url='http://minio:9000')
objects = s3.list_objects_v2(Bucket='documents')

# Step 2: Load into HeliosDB via PostgreSQL protocol
import psycopg2

conn = psycopg2.connect("host=localhost port=5432 dbname=helios")
cur = conn.cursor()

for obj in objects['Contents']:
    body = s3.get_object(Bucket='documents', Key=obj['Key'])
    data = json.loads(body['Body'].read())

    cur.execute("""
        INSERT INTO documents (key, data, created_at)
        VALUES (%s, %s, %s)
    """, (obj['Key'], json.dumps(data), obj['LastModified']))

conn.commit()

Keep Both: MinIO for Blobs, HeliosDB for Intelligence

Keep the files in MinIO and index them in HeliosDB, as in the side-by-side example above. Querying the files in place through S3 foreign tables is planned for HeliosDB Full.

-- Object metadata you control, joined with local tables
SELECT m.key, m.size_bytes, m.metadata->>'team' AS team
FROM objects m
JOIN teams t ON t.name = m.metadata->>'team'
WHERE m.bucket = 'archive'
ORDER BY m.size_bytes DESC
LIMIT 20;

---

Summary

MinIO excels at what it was built for: high-performance, S3-compatible object storage at massive scale. HeliosDB excels at what it was built for: making data queryable, searchable, and intelligent. Run them together: MinIO keeps the objects, HeliosDB keeps the rows that describe them and answers the queries, and an S3-compatible object API for HeliosDB Full is on the roadmap.