Blockchain-Secured Tables: Architecture Design

Blockchain-Secured Tables: Architecture Design

HeliosDB Append-Only Tables with Cryptographic Integrity

Version: 1.0 Date: October 11, 2025 Status: Design Phase

---

1. Executive Summary

HeliosDB will support Blockchain-Secured Tables (BST) - a specialized table type that provides:

• Immutability: INSERT-only operations (no UPDATE/DELETE)
• Cryptographic Integrity: Each block of data is cryptographically chained
• Tamper Detection: Any unauthorized modification is immediately detectable
• User-Defined Purge Policies: Automated data lifecycle management
• Audit Trail: Complete provenance of all data mutations

Use Cases:

• Financial transaction logs (compliance, audit)
• Medical records (HIPAA compliance)
• IoT sensor data (provenance tracking)
• Smart contracts and legal documents
• Security event logs (SIEM integration)

---

2. Core Concepts

2.1 Blockchain Structure

Each BST organizes data into immutable blocks linked by cryptographic hashes:

Block 0 (Genesis)
├─ Hash: SHA-256(metadata)
├─ PrevHash: 0x0000...
├─ Data: []
└─ Timestamp: CREATE TABLE timestamp

Block 1
├─ Hash: SHA-256(PrevHash + Data + Timestamp)
├─ PrevHash: Block0.Hash
├─ Data: [Row1, Row2, ..., RowN]
└─ Timestamp: 2025-10-11T10:00:00Z

Block 2
├─ Hash: SHA-256(PrevHash + Data + Timestamp)
├─ PrevHash: Block1.Hash
├─ Data: [Row101, Row102, ..., Row200]
└─ Timestamp: 2025-10-11T10:05:00Z

Properties:

• Each block contains a batch of rows (configurable: 100-10,000 rows/block)
• Blocks are immutable once written
• Chain integrity verified via hash validation
• Merkle tree structure enables efficient partial verification

2.2 Cryptographic Hash Chain

pub struct BlockHeader {
    pub block_id: u64,
    pub prev_block_hash: [u8; 32],  // SHA-256 of previous block
    pub merkle_root: [u8; 32],       // Root of data Merkle tree
    pub timestamp: i64,
    pub row_count: u64,
    pub nonce: u64,                  // For integrity verification
}

pub struct Block {
    pub header: BlockHeader,
    pub rows: Vec<Row>,
    pub signatures: Vec<Signature>,  // Optional: multi-party signing
}

impl Block {
    pub fn calculate_hash(&self) -> [u8; 32] {
        let mut hasher = Sha256::new();
        hasher.update(&self.header.block_id.to_le_bytes());
        hasher.update(&self.header.prev_block_hash);
        hasher.update(&self.header.merkle_root);
        hasher.update(&self.header.timestamp.to_le_bytes());
        hasher.update(&self.header.row_count.to_le_bytes());
        hasher.update(&self.header.nonce.to_le_bytes());
        hasher.finalize().into()
    }
}

2.3 Merkle Tree for Data Integrity

Each block contains a Merkle tree of its rows for efficient verification:

                Root Hash
               /         \
        Hash(L1+L2)    Hash(L3+L4)
         /      \        /      \
     H(Row1) H(Row2) H(Row3) H(Row4)

Benefits:

• Verify individual row integrity without reading entire block
• Efficient proof-of-inclusion
• Supports light clients (verify without full data)

---

3. DDL Syntax

3.1 Table Creation

CREATE BLOCKCHAIN TABLE financial_transactions (
    transaction_id BIGINT PRIMARY KEY,
    account_id BIGINT NOT NULL,
    amount DECIMAL(18, 2) NOT NULL,
    currency VARCHAR(3) NOT NULL,
    timestamp TIMESTAMP NOT NULL,
    metadata JSONB
)
WITH (
    -- Block configuration
    block_size = 1000,              -- Rows per block
    block_seal_interval = '5 minutes',  -- Auto-seal after 5min

    -- Integrity configuration
    hash_algorithm = 'SHA256',
    enable_merkle_tree = true,

    -- Purge policy (explained in section 5)
    purge_policy = 'TIME_BASED',
    retention_period = '7 years',

    -- Optional: Multi-party signing
    required_signatures = 2,
    authorized_signers = ['key_finance_dept', 'key_audit_dept'],

    -- Storage optimization
    compression = 'ZSTD',
    encryption = 'TDE'  -- Uses existing TDE integration
);

3.2 Querying

-- Standard SELECT (read-only)
SELECT * FROM financial_transactions
WHERE account_id = 12345
  AND timestamp >= '2025-01-01';

-- Verify chain integrity
SELECT verify_blockchain('financial_transactions');

-- Verify specific block
SELECT verify_blockchain_block('financial_transactions', block_id => 100);

-- Get blockchain metadata
SELECT block_id, block_hash, prev_hash, row_count, timestamp
FROM blockchain_metadata('financial_transactions')
ORDER BY block_id DESC
LIMIT 10;

3.3 Data Insertion

-- INSERT only (UPDATE/DELETE not allowed)
INSERT INTO financial_transactions VALUES
(1, 12345, 1000.00, 'USD', NOW(), '{"type": "deposit"}'),
(2, 12346, 500.00, 'EUR', NOW(), '{"type": "withdrawal"}');

-- Attempting UPDATE or DELETE results in error
UPDATE financial_transactions SET amount = 2000 WHERE transaction_id = 1;
-- ERROR: UPDATE not allowed on blockchain table 'financial_transactions'

DELETE FROM financial_transactions WHERE transaction_id = 1;
-- ERROR: DELETE not allowed on blockchain table 'financial_transactions'

---

4. Storage Architecture

4.1 Block Storage Layout

/var/lib/heliosdb/data/{table_name}/
├── blocks/
│   ├── block_000000.dat    # Genesis block
│   ├── block_000001.dat    # Block 1
│   ├── block_000002.dat    # Block 2
│   └── ...
├── index/
│   ├── block_index.db      # Block ID → File offset mapping
│   ├── merkle_index.db     # Merkle tree persistence
│   └── hash_chain.db       # Block hash chain index
└── metadata/
    ├── chain_metadata.json # Chain configuration
    └── purge_log.json      # Purge operation audit log

4.2 Block File Format

// On-disk block format (little-endian)
struct BlockFile {
    magic: [u8; 4],              // "HBST" (HeliosDB Blockchain Table)
    version: u16,
    header_size: u32,
    header: BlockHeader,
    row_count: u64,
    rows: Vec<SerializedRow>,    // bincode serialized
    merkle_tree: Vec<[u8; 32]>,  // All tree nodes
    signatures: Vec<Signature>,   // Optional multi-party sigs
    checksum: [u8; 32],          // SHA-256 of entire block
}

4.3 Integration with Existing Storage

BST integrates with HeliosDB’s existing storage engine:

┌─────────────────────────────────────┐
│     LSM-tree Storage Engine         │
│  (heliosdb-storage)                 │
│                                     │
│  ┌─────────────┐  ┌──────────────┐ │
│  │  Memtable   │  │  SSTables    │ │
│  └─────────────┘  └──────────────┘ │
└─────────────┬───────────────────────┘
              │
              ├─ Standard Tables (mutable)
              │
              └─ Blockchain Tables (append-only)
                 ├─ Block Writer (seals blocks)
                 ├─ Block Verifier (chain validation)
                 └─ Purge Manager (lifecycle)

Key Points:

• BST uses dedicated write path (no Memtable updates)
• Blocks are immutable SSTables with additional metadata
• Reuses existing compression, encryption (TDE), replication
• Chain verification runs asynchronously (background thread)

---

5. Purge Policies

5.1 Policy Types

TIME_BASED: Purge blocks older than retention period

purge_policy = 'TIME_BASED',
retention_period = '7 years'

COUNT_BASED: Keep only last N blocks

purge_policy = 'COUNT_BASED',
retention_blocks = 1000000

SIZE_BASED: Purge oldest blocks when total size exceeds limit

purge_policy = 'SIZE_BASED',
max_total_size = '1TB'

COMPLIANCE_BASED: Custom rules (GDPR, HIPAA, SOX)

purge_policy = 'COMPLIANCE_BASED',
compliance_rules = '{
  "gdpr_right_to_erasure": true,
  "hipaa_retention": "6 years",
  "sox_audit_trail": "7 years"
}'

CUSTOM: User-defined function

purge_policy = 'CUSTOM',
purge_function = 'my_custom_purge_logic(block_metadata)'

5.2 Purge Mechanism

Safe Purge Process:

1. Mark for Purge: Block marked as eligible (doesn’t break chain)
2. Anchor Block Creation: Create “anchor block” with hash of purged range
3. Purge Execution: Physical deletion of block files
4. Chain Repair: Update chain to include anchor block

Before Purge:
Block 1 → Block 2 → Block 3 → Block 4 → Block 5

After Purging Block 2-3:
Block 1 → [Anchor: Hash(Block2+Block3)] → Block 4 → Block 5

Anchor Block Structure:

pub struct AnchorBlock {
    pub purged_range: (u64, u64),  // (start_block, end_block)
    pub purged_hash: [u8; 32],      // Hash of purged blocks
    pub purge_reason: String,
    pub purge_timestamp: i64,
    pub authorized_by: String,      // User/system that authorized purge
}

5.3 Purge Audit Trail

Every purge operation is logged in an append-only audit log:

SELECT * FROM blockchain_purge_log('financial_transactions');

-- Result:
┌────────────┬──────────────┬─────────────┬───────────────────┬──────────────┐
│ purge_id   │ block_range  │ reason      │ timestamp         │ authorized_by│
├────────────┼──────────────┼─────────────┼───────────────────┼──────────────┤
│ 1          │ [100, 199]   │ TIME_BASED  │ 2025-10-11 10:00  │ system       │
│ 2          │ [200, 299]   │ GDPR        │ 2025-10-12 14:30  │ admin_user   │
└────────────┴──────────────┴─────────────┴───────────────────┴──────────────┘

---

6. Security Features

6.1 Multi-Party Signing

For high-security use cases, require multiple signatures before sealing a block:

pub struct Signature {
    pub signer_id: String,
    pub public_key: Vec<u8>,
    pub signature: Vec<u8>,  // Ed25519 signature
    pub timestamp: i64,
}

impl Block {
    pub fn sign(&mut self, private_key: &PrivateKey) -> Result<()> {
        let signature = private_key.sign(self.calculate_hash());
        self.signatures.push(signature);
        Ok(())
    }

    pub fn verify_signatures(&self, required: usize) -> Result<bool> {
        if self.signatures.len() < required {
            return Ok(false);
        }
        // Verify each signature against block hash
        // ...
    }
}

6.2 Integration with TDE

Blockchain tables can be encrypted at rest using existing TDE:

CREATE BLOCKCHAIN TABLE secure_logs (
    event_id BIGINT PRIMARY KEY,
    event_data TEXT
) WITH (
    encryption = 'TDE',
    tde_algorithm = 'AES256-GCM'
);

Encryption Flow:

1. Block data is first hashed (plaintext)
2. Block is then encrypted with TDE
3. Hash remains in chain metadata (unencrypted for verification)

This provides:

• Data confidentiality (TDE encryption)
• Integrity verification (blockchain hashing)
• Non-repudiation (digital signatures)

---

7. Performance Considerations

7.1 Write Performance

Optimization Strategies:

1. Batch Writes: Buffer rows in memory before sealing block
2. Async Block Sealing: Seal blocks in background thread
3. Parallel Block Creation: Multiple active blocks per table
4. Direct Block Write: Bypass Memtable for append-only ops

Expected Performance:

• Write Throughput: 50K-100K rows/sec (batched)
• Block Seal Latency: 10-50ms (depending on block size)
• Chain Verification: 1M blocks/minute (parallel verification)

7.2 Read Performance

Index Strategies:

1. Block Index: Fast lookup of block by ID
2. Merkle Index: O(log N) row verification
3. Time-Series Index: Optimize range queries by timestamp
4. B-tree Secondary Indexes: Standard indexes still work

Query Optimization:

• Block-level pruning (skip blocks outside query range)
• Parallel block scanning
• Bloom filters for existence checks

---

8. Implementation Roadmap

Phase 1: Core Infrastructure (Week 1)

• Create heliosdb-blockchain crate
• Implement Block and BlockHeader structures
• Implement Merkle tree builder
• Implement hash chain validation
• Add DDL parser support for CREATE BLOCKCHAIN TABLE

Phase 2: Storage Integration (Week 2)

• Integrate with LSM-tree storage engine
• Implement block writer (seal & persist)
• Implement block reader
• Add block metadata index
• Background chain verifier thread

Phase 3: Purge Policies (Week 3)

• Implement TIME_BASED purge
• Implement COUNT_BASED purge
• Implement SIZE_BASED purge
• Anchor block creation
• Purge audit logging

Phase 4: Advanced Features (Week 4)

• Multi-party signing (Ed25519)
• TDE integration for encrypted blocks
• COMPLIANCE_BASED purge rules
• Performance benchmarking
• Documentation and examples

---

9. API Reference

9.1 Rust API

// Create blockchain table
let table = BlockchainTable::new(BlockchainTableConfig {
    name: "financial_transactions".to_string(),
    block_size: 1000,
    block_seal_interval: Duration::from_secs(300),
    hash_algorithm: HashAlgorithm::SHA256,
    enable_merkle_tree: true,
    purge_policy: PurgePolicy::TimeBased {
        retention_period: Duration::from_secs(3600 * 24 * 365 * 7),
    },
})?;

// Insert rows (buffered)
table.insert(row1).await?;
table.insert(row2).await?;

// Force block seal
table.seal_current_block().await?;

// Verify chain integrity
let is_valid = table.verify_chain().await?;

// Query blocks
let blocks = table.query_blocks(100..200).await?;

9.2 SQL Functions

-- Verify entire chain
SELECT verify_blockchain('table_name');

-- Verify specific block
SELECT verify_blockchain_block('table_name', block_id);

-- Get block metadata
SELECT * FROM blockchain_metadata('table_name');

-- Get purge log
SELECT * FROM blockchain_purge_log('table_name');

-- Force block seal
SELECT seal_blockchain_block('table_name');

---

10. Compliance and Audit

10.1 Audit Capabilities

• Complete Provenance: Every row traceable to insertion timestamp
• Tamper Detection: Any modification detected via hash verification
• Purge Transparency: All deletions logged in audit trail
• Multi-Party Accountability: Digital signatures prove authorization

10.2 Compliance Frameworks

GDPR (Right to Erasure):

• Purge mechanism supports compliant deletion
• Anchor blocks maintain chain integrity
• Audit log proves deletion occurred

HIPAA (Healthcare):

• 6-year retention requirement supported
• Encryption at rest via TDE
• Access control via existing auth system

SOX (Financial):

• 7-year audit trail retention
• Tamper-proof transaction logs
• Non-repudiation via digital signatures

---

11. Future Enhancements

11.1 Distributed Consensus

Extend blockchain tables with distributed consensus across multiple nodes:

• Raft-based block proposal
• Multi-node block validation
• Byzantine fault tolerance
• Cross-datacenter replication

11.2 Zero-Knowledge Proofs

Enable privacy-preserving verification:

• Prove data exists without revealing content
• Range proofs for compliance checks
• zkSNARKs for scalable verification

11.3 Smart Contract Integration

Execute business logic on block insertion:

• Trigger functions on new blocks
• Automated compliance checks
• Event-driven workflows

---

12. Conclusion

Blockchain-Secured Tables provide HeliosDB with enterprise-grade tamper detection and audit trail capabilities while maintaining high performance. The design:

Integrates seamlessly with existing storage engine Provides flexible purge policies for compliance Maintains chain integrity even after data deletion Scales to billions of rows via efficient indexing Supports encryption via existing TDE infrastructure

Next Steps: Proceed to implementation (Phase 1-4)

---

Document Version: 1.0 Authors: HeliosDB Hive Mind Architecture Team Date: October 11, 2025