Skip to content
HeliosDB HeliosDB Nano

HeliosDB Nano SQLite Compatibility Layer - Packaging Strategy

HeliosDB Nano SQLite Compatibility Layer - Packaging Strategy

Version: 1.0.0 Package Name: heliosdb-sqlite Target: PyPI Distribution Status: Production-Ready Last Updated: 2025-12-08

Executive Summary

This document defines the complete packaging, versioning, and distribution strategy for the HeliosDB Nano SQLite compatibility layer (heliosdb-sqlite), a drop-in replacement for Python’s sqlite3 module that provides enhanced features including vector search, encryption, and time-travel queries while maintaining 100% SQLite API compatibility.

Key Goals:

  • Zero-friction installation: pip install heliosdb-sqlite
  • Drop-in replacement for sqlite3
  • Cross-platform binary distribution (Linux, macOS, Windows)
  • Professional PyPI presence with comprehensive documentation
  • Automatic HeliosDB Nano binary bundling

1. Package Identity

1.1 Package Naming

Primary Package: heliosdb-sqlite

  • Rationale: Clear indication of SQLite compatibility
  • PyPI URL: https://pypi.org/project/heliosdb-sqlite/
  • Import Name: heliosdb_sqlite (Python identifier compliant)
  • Namespace: heliosdb_sqlite.*

Alternative Names Considered:

  • heliosdb-compat - Too generic
  • sqlite-helio - Less discoverable
  • heliosdb-py - Doesn’t emphasize SQLite compatibility

1.2 Version Numbering Scheme

Semantic Versioning (SemVer): MAJOR.MINOR.PATCH[-PRERELEASE][+BUILD]

Version Components:

<heliosdb-version>.<api-version>.<patch>[-<prerelease>]


Examples:
- 3.0.0      - First GA release, matches HeliosDB Nano 3.0.0
- 3.0.1      - Patch release (bug fixes, no API changes)
- 3.1.0      - Minor release (new SQLite API features)
- 3.1.0-rc1  - Release candidate
- 3.1.0-beta - Beta release
- 4.0.0      - Major release (breaking API changes)

Version Alignment:

  • Major version tracks HeliosDB Nano major version
  • Minor version increments for SQLite API additions
  • Patch version for bug fixes and optimizations

Pre-release Identifiers:

  • alpha - Early development, unstable API
  • beta - Feature complete, API stable, testing in progress
  • rc - Release candidate, production-ready pending final validation
  • dev - Development snapshots (not for production)

1.3 Package Metadata

PACKAGE_METADATA = {
    "name": "heliosdb-sqlite",
    "version": "3.0.0",
    "description": "SQLite-compatible interface for HeliosDB Nano with vector search and encryption",
    "long_description": "README.md",
    "long_description_content_type": "text/markdown",
    "author": "HeliosDB Team",
    "author_email": "team@heliosdb.io",
    "license": "Apache-2.0",
    "license_files": ["LICENSE"],
    "url": "https://github.com/dimensigon/HDB-HeliosDB-Nano",
    "project_urls": {
        "Homepage": "https://github.com/dimensigon/HDB-HeliosDB-Nano",
        "Documentation": "https://docs.heliosdb.io/sqlite-compat",
        "Repository": "https://github.com/dimensigon/HDB-HeliosDB-Nano",
        "Issues": "https://github.com/dimensigon/HDB-HeliosDB-Nano/issues",
        "Changelog": "https://github.com/dimensigon/HDB-HeliosDB-Nano/blob/main/CHANGELOG.md",
        "Discord": "https://discord.gg/heliosdb",
    },
    "keywords": [
        "sqlite", "database", "embedded", "vector-search",
        "encryption", "postgresql", "compatibility", "sql",
        "ai", "machine-learning", "rag", "semantic-search"
    ],
    "classifiers": [
        "Development Status :: 5 - Production/Stable",
        "Intended Audience :: Developers",
        "License :: OSI Approved :: Apache Software License",
        "Operating System :: OS Independent",
        "Operating System :: POSIX :: Linux",
        "Operating System :: MacOS :: MacOS X",
        "Operating System :: Microsoft :: Windows",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.8",
        "Programming Language :: Python :: 3.9",
        "Programming Language :: Python :: 3.10",
        "Programming Language :: Python :: 3.11",
        "Programming Language :: Python :: 3.12",
        "Programming Language :: Python :: 3.13",
        "Programming Language :: Rust",
        "Topic :: Database",
        "Topic :: Database :: Database Engines/Servers",
        "Topic :: Software Development :: Libraries :: Python Modules",
        "Topic :: Scientific/Engineering :: Artificial Intelligence",
        "Typing :: Typed",
    ],
}

2. Python Version Support

2.1 Supported Python Versions

Minimum Version: Python 3.8 (released October 2019)

Supported Versions:

  • ✅ Python 3.8 - Minimum supported, EOL October 2024
  • ✅ Python 3.9 - Active support
  • ✅ Python 3.10 - Active support
  • ✅ Python 3.11 - Active support
  • ✅ Python 3.12 - Active support
  • ✅ Python 3.13 - Latest stable

Python 3.7 and Earlier: Not supported (EOL)

Rationale:

  • Python 3.8+ provides typing module enhancements
  • Better asyncio support for future features
  • Modern build tools compatibility
  • 95%+ coverage of active Python users

2.2 Version Support Policy

Support Timeline:

Python Version  | First Supported | Support End    | Notes
----------------|-----------------|----------------|------------------
3.8             | heliosdb-sqlite 3.0.0 | 2025-12-31 | Extended support
3.9             | heliosdb-sqlite 3.0.0 | 2026-10-31 | Active
3.10            | heliosdb-sqlite 3.0.0 | 2027-10-31 | Active
3.11            | heliosdb-sqlite 3.0.0 | 2028-10-31 | Active
3.12            | heliosdb-sqlite 3.0.0 | 2029-10-31 | Active
3.13            | heliosdb-sqlite 3.0.0 | 2030-10-31 | Latest

Deprecation Policy:

  1. Announce deprecation 12 months in advance
  2. Issue deprecation warnings in code
  3. Update documentation and release notes
  4. Remove support in next major version

3. Dependency Management

3.1 Core Dependencies

Required Dependencies:

[project.dependencies]
# No runtime dependencies except Python stdlib
# All functionality is bundled in binary wheel

Rationale: Zero dependencies for maximum compatibility and minimal installation footprint.

3.2 Optional Dependencies

Feature-Based Groups:

[project.optional-dependencies]
# Vector operations and scientific computing
vector = [
    "numpy>=1.20.0",
    "scipy>=1.7.0",
]


# Data manipulation and analysis
pandas = [
    "pandas>=1.3.0",
    "pyarrow>=10.0.0",
]


# Async database operations
async = [
    "aiofiles>=23.0.0",
]


# Development and testing
dev = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "pytest-asyncio>=0.21.0",
    "mypy>=1.0.0",
    "ruff>=0.1.0",
    "black>=23.0.0",
    "build>=1.0.0",
    "twine>=4.0.0",
]


# Documentation generation
docs = [
    "sphinx>=6.0.0",
    "sphinx-rtd-theme>=1.3.0",
    "myst-parser>=2.0.0",
]


# Type checking stubs
types = [
    "types-setuptools>=65.0.0",
]


# All optional features
all = [
    "numpy>=1.20.0",
    "scipy>=1.7.0",
    "pandas>=1.3.0",
    "pyarrow>=10.0.0",
    "aiofiles>=23.0.0",
]

3.3 Dependency Version Constraints

Strategy: Minimum version pinning with compatible release specifiers

Rules:

  • Use >= for minimum versions
  • Avoid upper bounds unless known incompatibility
  • Pin exact versions only for security vulnerabilities
  • Test against minimum and maximum supported versions in CI

4. HeliosDB Nano Binary Bundling Strategy

4.1 Binary Distribution Approach

Primary Strategy: Platform-specific binary wheels with bundled HeliosDB Nano executable

Binary Inclusion Methods:

  1. Pre-built Binaries (Preferred)

    • Include compiled heliosdb-nano binary in wheel
    • Platform-specific wheels: manylinux, macosx, win_amd64
    • Extract binary to user-accessible location on first use

  2. Dynamic Download (Fallback)

    • Download appropriate binary on first import
    • Cache in ~/.heliosdb/ directory
    • Verify checksum before use

  3. Build from Source (Developer Option)

    • Require Rust toolchain
    • Compile during pip install with --no-binary flag

4.2 Binary Packaging Structure

Wheel Contents:

heliosdb_sqlite-3.0.0-cp39-cp39-manylinux_2_17_x86_64.whl
├── heliosdb_sqlite/
│   ├── __init__.py
│   ├── connection.py
│   ├── cursor.py
│   ├── exceptions.py
│   ├── _version.py
│   └── binaries/
│       └── heliosdb-nano-linux-x86_64    # Bundled binary
├── heliosdb_sqlite-3.0.0.dist-info/
│   ├── METADATA
│   ├── WHEEL
│   ├── RECORD
│   ├── LICENSE
│   └── entry_points.txt
└── tests/ (optional, excluded in final wheel)

4.3 Platform-Specific Binary Naming

Binary Naming Convention:

heliosdb-nano-{platform}-{arch}[-{variant}]


Examples:
- heliosdb-nano-linux-x86_64
- heliosdb-nano-linux-aarch64
- heliosdb-nano-macos-x86_64
- heliosdb-nano-macos-arm64
- heliosdb-nano-windows-x86_64.exe

4.4 Binary Extraction and Caching

First-Use Initialization:

import os
import sys
import platform
from pathlib import Path


def get_binary_path():
    """Get or extract HeliosDB Nano binary."""
    cache_dir = Path.home() / ".heliosdb" / "bin"
    cache_dir.mkdir(parents=True, exist_ok=True)


    platform_name = platform.system().lower()
    arch = platform.machine().lower()


    binary_name = f"heliosdb-nano-{platform_name}-{arch}"
    if platform_name == "windows":
        binary_name += ".exe"


    cached_binary = cache_dir / binary_name


    if not cached_binary.exists():
        # Extract from package data
        import pkg_resources
        bundled_binary = pkg_resources.resource_filename(
            "heliosdb_sqlite", f"binaries/{binary_name}"
        )
        shutil.copy2(bundled_binary, cached_binary)
        cached_binary.chmod(0o755)  # Make executable


    return cached_binary

5. Binary Wheel Support

5.1 Target Platforms

Linux (manylinux):

manylinux2014_x86_64   (CentOS 7 compatible, most common)
manylinux2014_aarch64  (ARM64 support)
manylinux_2_28_x86_64  (Newer glibc, better performance)

macOS:

macosx_10_12_x86_64    (Intel, macOS 10.12+)
macosx_11_0_arm64      (Apple Silicon, macOS 11+)
macosx_10_12_universal2 (Universal binary)

Windows:

win_amd64              (64-bit Windows)
win32                  (32-bit Windows, optional)

5.2 Wheel Building Process

Build Script (build_wheels.sh):

#!/bin/bash
set -e


PYTHON_VERSIONS=("3.8" "3.9" "3.10" "3.11" "3.12" "3.13")
PLATFORMS=("manylinux2014_x86_64" "manylinux2014_aarch64" "macosx_11_0_arm64" "win_amd64")


for py_version in "${PYTHON_VERSIONS[@]}"; do
    for platform in "${PLATFORMS[@]}"; do
        echo "Building wheel for Python $py_version on $platform..."
        python -m build --wheel --outdir dist/
    done
done

CI/CD Integration (GitHub Actions):

name: Build Wheels


on:
  release:
    types: [published]


jobs:
  build-wheels:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        python-version: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13']


    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}


      - name: Install Rust
        uses: actions-rs/toolchain@v1
        with:
          toolchain: stable
          profile: minimal


      - name: Build HeliosDB Nano Binary
        run: cargo build --release


      - name: Build Wheel
        run: |
          pip install build
          python -m build --wheel


      - name: Upload Wheel
        uses: actions/upload-artifact@v4
        with:
          name: wheels-${{ matrix.os }}-${{ matrix.python-version }}
          path: dist/*.whl

6. Installation Methods

Standard Installation:

Terminal window
# Install latest stable version
pip install heliosdb-sqlite


# Install specific version
pip install heliosdb-sqlite==3.0.0


# Install with optional dependencies
pip install heliosdb-sqlite[vector]
pip install heliosdb-sqlite[pandas]
pip install heliosdb-sqlite[all]


# Upgrade to latest
pip install --upgrade heliosdb-sqlite

6.2 conda Install (Future)

Conda-Forge Distribution:

Terminal window
# Install from conda-forge (planned for v3.1.0)
conda install -c conda-forge heliosdb-sqlite


# Create environment with heliosdb-sqlite
conda create -n myapp python=3.11 heliosdb-sqlite

Conda Package Metadata (meta.yaml):

package:
  name: heliosdb-sqlite
  version: {{ version }}


source:
  url: https://pypi.io/packages/source/h/heliosdb-sqlite/heliosdb-sqlite-{{ version }}.tar.gz
  sha256: {{ sha256 }}


build:
  number: 0
  script: {{ PYTHON }} -m pip install . -vv


requirements:
  host:
    - python >=3.8
    - pip
    - setuptools
  run:
    - python >=3.8


test:
  imports:
    - heliosdb_sqlite
  commands:
    - pip check


about:
  home: https://github.com/dimensigon/HDB-HeliosDB-Nano
  license: Apache-2.0
  license_file: LICENSE
  summary: SQLite-compatible interface for HeliosDB Nano

6.3 Build from Source

Prerequisites:

  • Python 3.8+
  • Rust toolchain (1.75+)
  • C compiler (gcc/clang/MSVC)

Build Steps:

Terminal window
# Clone repository
git clone https://github.com/dimensigon/HDB-HeliosDB-Nano.git
cd heliosdb-nano/sdks/python-sqlite


# Build HeliosDB Nano binary
cargo build --release --manifest-path ../../Cargo.toml


# Install in development mode
pip install -e .


# Or build wheel
pip install build
python -m build


# Install built wheel
pip install dist/heliosdb_sqlite-3.0.0-*.whl

6.4 Docker Installation

Docker Image:

FROM python:3.11-slim


RUN pip install --no-cache-dir heliosdb-sqlite[all]


WORKDIR /app
COPY . /app


CMD ["python", "app.py"]

Docker Compose:

version: '3.8'
services:
  app:
    image: python:3.11-slim
    volumes:
      - ./app:/app
      - heliosdb-data:/data
    command: |
      bash -c "
        pip install heliosdb-sqlite &&
        python /app/main.py
      "
volumes:
  heliosdb-data:

7. CI/CD Publishing Pipeline

7.1 PyPI Publishing Workflow

GitHub Actions (.github/workflows/publish.yml):

name: Publish to PyPI


on:
  release:
    types: [published]
  workflow_dispatch:
    inputs:
      environment:
        description: 'Publish to Test PyPI or PyPI'
        required: true
        default: 'testpypi'
        type: choice
        options:
          - testpypi
          - pypi


jobs:
  build-and-publish:
    runs-on: ubuntu-latest
    environment:
      name: ${{ github.event.inputs.environment || 'pypi' }}
    permissions:
      id-token: write
      contents: read


    steps:
      - uses: actions/checkout@v4


      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'


      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install build twine


      - name: Build package
        run: python -m build


      - name: Check package
        run: twine check dist/*


      - name: Publish to Test PyPI
        if: github.event.inputs.environment == 'testpypi'
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          repository-url: https://test.pypi.org/legacy/
          password: ${{ secrets.TEST_PYPI_API_TOKEN }}


      - name: Publish to PyPI
        if: github.event.inputs.environment == 'pypi' || github.event_name == 'release'
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_API_TOKEN }}

7.2 Release Process

Release Checklist:

  1. ✅ Update version in __version__.py
  2. ✅ Update CHANGELOG.md with release notes
  3. ✅ Run full test suite: pytest tests/
  4. ✅ Build documentation: make docs
  5. ✅ Create Git tag: git tag v3.0.0
  6. ✅ Push tag: git push origin v3.0.0
  7. ✅ Create GitHub release (triggers CI)
  8. ✅ Verify PyPI publication
  9. ✅ Announce release on Discord/Twitter

Version Bump Script (scripts/bump_version.py):

#!/usr/bin/env python3
import re
import sys
from pathlib import Path


def bump_version(version_type='patch'):
    """Bump package version."""
    version_file = Path('heliosdb_sqlite/_version.py')
    content = version_file.read_text()


    match = re.search(r'__version__ = ["\'](\d+)\.(\d+)\.(\d+)["\']', content)
    if not match:
        print("Error: Could not find version string")
        sys.exit(1)


    major, minor, patch = map(int, match.groups())


    if version_type == 'major':
        major += 1
        minor = 0
        patch = 0
    elif version_type == 'minor':
        minor += 1
        patch = 0
    else:  # patch
        patch += 1


    new_version = f"{major}.{minor}.{patch}"
    new_content = re.sub(
        r'__version__ = ["\'][\d.]+["\']',
        f'__version__ = "{new_version}"',
        content
    )


    version_file.write_text(new_content)
    print(f"Version bumped to {new_version}")
    return new_version


if __name__ == '__main__':
    version_type = sys.argv[1] if len(sys.argv) > 1 else 'patch'
    bump_version(version_type)

8. Version Compatibility Matrix

8.1 Python Version Compatibility

heliosdb-sqlitePython 3.8Python 3.9Python 3.10Python 3.11Python 3.12Python 3.13
3.0.0✅✅✅✅✅✅
3.0.1✅✅✅✅✅✅
3.1.0 (planned)❌✅✅✅✅✅

8.2 HeliosDB Nano Version Compatibility

heliosdb-sqliteHeliosDB NanoSQLite APINotes
3.0.03.0.03.43.0Initial GA release
3.0.13.0.13.43.0Bug fixes
3.1.03.1.03.44.0New SQLite features

8.3 Platform Support Matrix

PlatformArchitectureWheel AvailableStatus
Linuxx86_64✅ manylinux2014Stable
Linuxaarch64✅ manylinux2014Stable
macOSx86_64✅ 10.12+Stable
macOSarm64✅ 11.0+Stable
Windowsx86_64✅ Win10+Stable
Windowsx86⚠️ Manual buildLimited

9. License and Compliance

9.1 License Information

Primary License: Apache License 2.0

License File Inclusion:

  • LICENSE - Full Apache 2.0 license text
  • NOTICE - Attribution and third-party notices

SPDX Identifier: Apache-2.0

9.2 Third-Party Dependencies

Bundled Components:

  • HeliosDB Nano binary (Apache 2.0)
  • RocksDB (Apache 2.0 + GPLv2 dual license)
  • No proprietary components

License Compatibility:

  • ✅ Apache 2.0 allows commercial use
  • ✅ Compatible with GPLv3
  • ✅ Compatible with MIT/BSD licenses
  • ✅ PyPI distribution allowed

9.3 Compliance Requirements

Package Distribution:

  • ✅ Include full LICENSE text in wheel
  • ✅ Include NOTICE file with attributions
  • ✅ Preserve copyright notices
  • ✅ Document third-party components

Export Compliance:

  • Package includes encryption (AES-256-GCM)
  • Not subject to US export restrictions (publicly available)
  • Complies with Wassenaar Arrangement exemptions

10. Testing and Quality Assurance

10.1 Pre-Publication Testing

Test Checklist:

Terminal window
# 1. Unit tests
pytest tests/unit/ -v --cov=heliosdb_sqlite


# 2. Integration tests
pytest tests/integration/ -v


# 3. SQLite compatibility tests
pytest tests/compat/ -v


# 4. Installation tests (clean environment)
python -m venv test_env
source test_env/bin/activate
pip install dist/heliosdb_sqlite-3.0.0-*.whl
python -c "import heliosdb_sqlite; print(heliosdb_sqlite.__version__)"


# 5. Documentation tests
pytest --doctest-modules heliosdb_sqlite/


# 6. Type checking
mypy heliosdb_sqlite/


# 7. Linting
ruff check heliosdb_sqlite/
black --check heliosdb_sqlite/

10.2 Continuous Integration

CI Matrix:

  • Python: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13
  • Platform: Linux, macOS, Windows
  • Test Suite: Unit, integration, compatibility

11. Documentation and Support

11.1 PyPI Package Page

README.md Structure:

  1. Project description and features
  2. Installation instructions
  3. Quick start example
  4. SQLite compatibility statement
  5. Documentation links
  6. License information
  7. Contributing guidelines

11.2 Documentation Hosting

Documentation Sites:

11.3 Support Channels

Community Support:

  • GitHub Issues
  • GitHub Discussions
  • Discord Server
  • Stack Overflow (heliosdb-sqlite tag)

Enterprise Support:

12. Metrics and Analytics

12.1 Package Metrics

Track on PyPI:

  • Total downloads
  • Downloads by Python version
  • Downloads by platform
  • Download growth rate

Tools:

  • pypistats: pypistats python heliosdb-sqlite
  • Libraries.io: Package health score
  • Snyk: Security vulnerability scanning

12.2 Success Metrics

Targets for v3.0.0 (First 6 Months):

  • ✅ 10,000+ downloads
  • ✅ 100+ GitHub stars
  • ✅ 95%+ installation success rate
  • ✅ <5 critical bugs
  • ✅ <24hr average issue response time

13. Future Roadmap

13.1 Packaging Enhancements (v3.1.0)

  • Conda-forge distribution
  • Homebrew formula
  • Debian/Ubuntu packages (.deb)
  • RPM packages (Fedora/RHEL)
  • Windows installer (.msi)

13.2 Binary Optimization (v3.2.0)

  • Compressed binaries (UPX)
  • Lazy binary extraction
  • CDN-hosted binaries for faster downloads
  • Architecture-specific optimizations (AVX2, NEON)

Appendix A: Package Validation Checklist

Before publishing to PyPI:

  • Version number updated in all files
  • CHANGELOG.md updated
  • README.md tested with PyPI renderer
  • LICENSE file included
  • All tests passing (100%)
  • Type hints validated (mypy)
  • Documentation built without warnings
  • Wheel builds on all platforms
  • Package size < 50MB per wheel
  • Binary permissions correct (executable)
  • Dependencies pinned correctly
  • Security scan passed (Snyk, Bandit)
  • Test PyPI upload successful
  • Installation tested on clean systems

Appendix B: Troubleshooting Common Issues

Issue: Binary Not Found

Solution: Check platform detection, ensure binary extracted correctly

Issue: Permission Denied

Solution: Binary needs executable permissions (chmod +x)

Issue: Platform Not Supported

Solution: Build from source with Rust toolchain

Issue: Import Error

Solution: Reinstall with pip install --force-reinstall heliosdb-sqlite

Document Owner: HeliosDB Packaging Team Review Cycle: Quarterly Next Review: 2026-03-08