Build a RAG App on Nano v3.32: From fastembed to a Persistent, Compressed Vector Index in 80 Lines of Rust

v3.32 ships everything you need to run a production-shaped RAG indexer inside a single Rust process: in-process embeddings via fastembed, a persistent PQ-HNSW index, exact two-stage rerank, online deletes, and restart-survivable state. This post walks through the whole thing end-to-end. No Pinecone, no Qdrant, no separate ingest service.

1. Dependencies

[dependencies]
heliosdb-nano = { version = "3.32.1", features = ["vector-persist"] }
fastembed     = "4"
anyhow        = "1"

Two crates do the work. fastembed downloads and runs BGE-Small-EN-V1.5 in-process (dim=384, normalised); heliosdb-nano stores and searches the vectors. The vector-persist feature unlocks the new PersistentVectorIndex path — without it, you get the existing in-RAM HNSW.

2. Open the index (or create it)

use heliosdb_nano::vector::persistent::PersistentVectorIndex;
use heliosdb_nano::vector::{DistanceMetric, PqHnswConfig};

fn open_index(path: &str) -> anyhow::Result<PersistentVectorIndex> {
    if PersistentVectorIndex::exists(path)? {
        Ok(PersistentVectorIndex::open(path)?)
    } else {
        let cfg = PqHnswConfig::new(384, DistanceMetric::L2);
        Ok(PersistentVectorIndex::create_with_pq(path, cfg)?)
    }
}

Note the config. On v3.32.1+ the defaults are recall-safe — create_with_pq with a plain PqHnswConfig::new(dim, metric) picks ~4 dimensions per sub-vector (~16× compression) automatically. On v3.32.0 you had to set num_subquantizers = dim / 4 explicitly or recall collapsed; this is fixed in the patch release. Upgrade.

3. Embed and insert

use fastembed::{EmbeddingModel, InitOptions, TextEmbedding};

fn ingest(index: &mut PersistentVectorIndex, docs: Vec<(u64, String)>)
    -> anyhow::Result<()>
{
    let embedder = TextEmbedding::try_new(
        InitOptions::new(EmbeddingModel::BGESmallENV15)
    )?;
    let texts: Vec<&str> = docs.iter().map(|(_, t)| t.as_str()).collect();
    let vectors = embedder.embed(texts, None)?;
    for ((id, _), vec) in docs.iter().zip(vectors.iter()) {
        index.insert(*id, vec.as_slice())?;
    }
    index.flush()?;
    Ok(())
}

Because BGE-Small produces L2-normalised vectors, cosine similarity collapses to L2 distance — the DistanceMetric::L2 in step 2 is the right choice (PQ is L2-only). flush() commits the RocksDB batch; the index is now durable.

4. Query

fn query(index: &PersistentVectorIndex, embedder: &TextEmbedding, q: &str, k: usize)
    -> anyhow::Result<Vec<(u64, f32)>>
{
    let q_vec = embedder.embed(vec![q], None)?.into_iter().next().unwrap();
    let hits = index.search(&q_vec, k, /* ef */ 100)?;
    Ok(hits)
}

search traverses the graph using PQ codes (ADC), then reranks the candidates against full disk-resident vectors. You get exact top-k ordering. No extra plumbing.

5. Filtered queries

If your corpus is multi-tenant, post-filtering a top-k is wrong — you may return zero rows for one tenant while another has thousands. search_filtered evaluates the predicate inside the traversal:

let tenant_id: u64 = 42;
let hits = index.search_filtered(
    &q_vec, k, /* ef */ 100,
    |row_id| row_meta.get(&row_id).map(|m| m.tenant_id) == Some(tenant_id),
)?;

6. Deletes that don't rot recall

index.remove(doc_id)?;       // repairs neighbours immediately
if index.tombstone_ratio() > 0.2 {
    index.compact()?;        // reclaim space
}

Tombstone-only indexes degrade recall over time. remove() rewrites the affected neighbourhood; compact() reclaims space when the tombstone ratio gets high. Run compact() on a schedule, or after a large delete burst.

7. Restart proof

Kill the process. PersistentVectorIndex::open(path) on the next start restores the graph, the entry point, the per-element neighbour list, and the codebook from RocksDB. The first query is the same speed as the thousandth. The in-tree validation report's exact phrasing: "drop → open → search returns identical top-10".

The Claude Code / Codex skill

The Nano skill bundle has an updated heliosdb-nano-vector skill (shipped with v3.32.x) that gives Claude Code, Codex, or any MCP-aware agent a copy-pasteable recipe for exactly this flow: open-or-create, ingest, query, filtered query, delete + compact. Install with heliosdb-nano skills install (see Drop a Database into Claude Code in 30 Seconds for the broader skill-bundle story). The agent reads the skill and writes you an end-to-end indexer without reading the README.

What this replaces

An external Pinecone or Qdrant tier; a vector-rebuild job after every restart; a tombstone-only delete path; a separate filter service running over the top-k results. All four collapse into one in-process index that survives restart, compresses 16×, and reranks exactly.

HeliosDB Nano is Apache 2.0. Repository: github.com/dimensigon/HDB-HeliosDB-Nano.

← Back to Blog