---
title: "ALP Compression Quick Start Guide"
---

# ALP Compression Quick Start Guide

## What is ALP?

ALP (Adaptive Lossless Floating-Point) is a lossless compression algorithm for numeric data that achieves **2-10x compression** with **zero precision loss**.

## Quick Start

### 1. It Just Works™

ALP compression is **automatically enabled** for Float4 and Float8 columns:

```rust
use heliosdb_nano::{Config, StorageEngine, Schema, Column, DataType, Tuple, Value};

let engine = StorageEngine::open("./data", &Config::default())?;

// Create table with numeric columns
let schema = Schema::new(vec![
    Column::new("id", DataType::Int4),
    Column::new("price", DataType::Float8),      // Auto-compressed
    Column::new("discount", DataType::Float4),   // Auto-compressed
]);

engine.catalog().create_table("products", schema)?;

// Insert data - compression happens automatically
engine.insert_tuple("products", Tuple {
    values: vec![
        Value::Int4(1),
        Value::Float8(99.95),
        Value::Float4(0.15),
    ],
})?;

// Query - decompression is transparent
let results = engine.scan_table("products")?;
```

### 2. Check Compression Statistics

```rust
// Get compression stats for a table
if let Some(stats) = engine.get_compression_stats("products")? {
    println!("Compression ratio: {:.2}x", stats.overall_ratio);
    println!("Original size: {} bytes", stats.total_original_size);
    println!("Compressed size: {} bytes", stats.total_compressed_size);
    println!("Space saved: {} bytes",
        stats.total_original_size - stats.total_compressed_size);

    // Per-column statistics
    for (column, meta) in &stats.column_stats {
        println!("{}: {:?} codec, {:.2}x ratio",
            column, meta.codec, meta.compression_ratio);
    }
}
```

### 3. Custom Configuration (Optional)

```rust
use heliosdb_nano::storage::compression::CompressionConfig;

let mut config = CompressionConfig::default();

// Adjust compression aggressiveness
config.min_rows_for_compression = 500;  // Compress smaller batches
config.compression_level = 8;           // Higher compression

// Disable compression for specific columns
config.column_overrides.insert("user_id".to_string(), false);

// Create engine with custom config
// Note: Currently must be set at creation time
let compression_manager = CompressionManager::new(config);
```

### 4. Batch Compression (Advanced)

For large datasets, use batch compression directly:

```rust
let manager = engine.compression_manager();

// Compress large f64 batch
let values: Vec<f64> = (0..100000).map(|i| i as f64 * 0.01).collect();
let (compressed, metadata) = manager.compress_f64_batch(&values)?;

println!("Compressed {} values:", values.len());
println!("  Original: {} bytes", metadata.original_size);
println!("  Compressed: {} bytes", metadata.compressed_size);
println!("  Ratio: {:.2}x", metadata.compression_ratio);
println!("  Codec: {:?}", metadata.codec);

// Decompress
let decompressed = manager.decompress_f64_batch(&compressed, metadata.codec)?;
assert_eq!(values, decompressed); // Lossless!
```

## Configuration Options

```rust
pub struct CompressionConfig {
    pub enabled: bool,                    // Default: true
    pub alp_enabled: bool,                // Default: true
    pub min_rows_for_compression: usize,  // Default: 1000
    pub compression_level: u8,            // Default: 6 (1-9)
    pub min_data_size: usize,             // Default: 1024 bytes
    pub min_compression_ratio: f64,       // Default: 1.2 (20% savings)
    pub column_overrides: HashMap<String, bool>,
    pub adaptive_compression: bool,       // Default: true
}
```

### Presets

**Default (Balanced)**:
```rust
CompressionConfig::default()
```

**High Compression**:
```rust
CompressionConfig {
    min_rows_for_compression: 100,
    compression_level: 9,
    min_compression_ratio: 1.1,
    ..Default::default()
}
```

**Performance**:
```rust
CompressionConfig {
    min_rows_for_compression: 5000,
    compression_level: 3,
    min_compression_ratio: 2.0,
    ..Default::default()
}
```

## Performance Characteristics

### Compression Ratios by Data Type

| Data Pattern | Ratio | Example |
|--------------|-------|---------|
| Prices | 4-5x | $99.95, $1234.56 |
| Percentages | 3-4x | 15.5%, 99.9% |
| Temperatures | 3-5x | 20.5°C, -10.2°C |
| Measurements | 3-5x | 65.3kg, 180.5cm |
| Scientific | 1.2-1.5x | π, e, √2 |

### Throughput

- **Encoding**: ~0.5 doubles per CPU cycle
- **Decoding**: ~2.6 doubles per CPU cycle
- **20-50% faster** than alternatives (Gorilla, Chimp)

### Space Savings

Example: 1M rows × 5 Float64 columns

- Uncompressed: **40 MB**
- Compressed: **10-13 MB**
- **Savings: 27-30 MB**

## Best Practices

### 1. When to Use ALP

**Good for**:
- Financial data (prices, amounts, percentages)
- Sensor data (temperatures, humidity, pressure)
- Metrics and measurements
- Time-series numeric data
- Scientific datasets

**Not ideal for**:
- Random floating-point data
- Very small tables (< 1000 rows)
- Frequently updated hot columns
- Columns requiring random single-value access

### 2. Configuration Tips

**Storage-Optimized** (data warehouse):
```rust
config.min_rows_for_compression = 100;
config.compression_level = 9;
config.min_compression_ratio = 1.1;
```

**Throughput-Optimized** (OLTP):
```rust
config.min_rows_for_compression = 10000;
config.compression_level = 3;
config.min_compression_ratio = 2.0;
```

**Balanced** (default):
```rust
config = CompressionConfig::default();
```

### 3. Column Selection

Disable compression for:
- Primary keys (frequently accessed)
- Foreign keys (join columns)
- Very small columns
- Already compressed data

```rust
config.column_overrides.insert("id".to_string(), false);
config.column_overrides.insert("user_id".to_string(), false);
```

### 4. Monitoring

Check compression effectiveness:

```rust
let all_stats = engine.get_all_compression_stats()?;

for (table, stats) in all_stats {
    if stats.overall_ratio < 1.5 {
        println!("Warning: Poor compression on table {}: {:.2}x",
            table, stats.overall_ratio);
    }
}
```

## Troubleshooting

### Low Compression Ratio

**Symptom**: Compression ratio < 1.5x

**Causes**:
- Random or encrypted data
- Already compressed values
- Very diverse value ranges

**Solutions**:
- Check data patterns with `analyze_data()`
- Adjust `min_compression_ratio` threshold
- Disable compression for problematic columns

### Performance Issues

**Symptom**: Slow inserts/queries

**Causes**:
- Compression level too high
- Batch size too small
- Compressing hot columns

**Solutions**:
- Lower `compression_level` (try 3-5)
- Increase `min_rows_for_compression`
- Disable compression on hot columns

### Statistics Not Showing

**Symptom**: `get_compression_stats()` returns None

**Causes**:
- Table has no compressed columns
- No data inserted yet
- Statistics tracking disabled

**Solutions**:
- Insert data first
- Check column data types (Float4/Float8)
- Verify compression is enabled

## Examples

### Example 1: Financial Application

```rust
let schema = Schema::new(vec![
    Column::new("transaction_id", DataType::Int8),
    Column::new("amount", DataType::Float8),        // Prices compress 4-5x
    Column::new("tax_rate", DataType::Float4),      // Percentages compress 3-4x
    Column::new("balance", DataType::Float8),       // Balances compress 4-5x
]);

engine.catalog().create_table("transactions", schema)?;

// After inserting 1M transactions
let stats = engine.get_compression_stats("transactions")?.unwrap();
// Expected: 3-5x overall compression ratio
// Space saved: ~20-30 MB per million rows
```

### Example 2: IoT Sensor Data

```rust
let schema = Schema::new(vec![
    Column::new("sensor_id", DataType::Int4),
    Column::new("timestamp", DataType::Timestamp),
    Column::new("temperature", DataType::Float4),   // Measurements compress 3-5x
    Column::new("humidity", DataType::Float4),      // Measurements compress 3-5x
    Column::new("pressure", DataType::Float4),      // Measurements compress 3-5x
]);

engine.catalog().create_table("sensor_readings", schema)?;

// After inserting 10M readings
let stats = engine.get_compression_stats("sensor_readings")?.unwrap();
// Expected: 3-4x overall compression ratio
// Space saved: ~100-150 MB per 10M readings
```

### Example 3: Scientific Data

```rust
let schema = Schema::new(vec![
    Column::new("experiment_id", DataType::Int4),
    Column::new("measurement", DataType::Float8),   // High precision, 1.2-1.5x
    Column::new("error_margin", DataType::Float8),  // High precision, 1.2-1.5x
]);

engine.catalog().create_table("experiments", schema)?;

// Scientific data has lower but still valuable compression
let stats = engine.get_compression_stats("experiments")?.unwrap();
// Expected: 1.2-1.5x overall compression ratio
// Still 20-30% space savings with zero precision loss
```

## Testing

Run tests to verify compression:

```bash
# All compression tests
cargo test compression
cargo test alp

# Integration tests only
cargo test --test alp_integration_tests

# With output
cargo test compression -- --nocapture
```

## FAQ

**Q: Is ALP lossless?**
A: Yes, 100% lossless. Every bit is preserved.

**Q: What's the performance impact?**
A: Minimal. Decoding is ~5x faster than encoding, and both are very fast.

**Q: Can I disable ALP?**
A: Yes, set `config.alp_enabled = false` or disable per-column.

**Q: Does it work with existing data?**
A: Yes, reads handle both compressed and uncompressed data transparently.

**Q: What about NaN and Infinity?**
A: Fully supported, including signed zeros and denormal numbers.

**Q: Can I change config at runtime?**
A: Not yet. Set config at engine creation time; runtime changes are planned.

## Next Steps

- Read full documentation: `docs/implementation/ALP_INTEGRATION_COMPLETE.md`
- Review benchmarks: `benches/alp_compression_benchmark.rs`
- Run integration tests: `cargo test alp_integration_tests`
- Monitor your compression ratios: `engine.get_all_compression_stats()`

## Support

For issues or questions:
- Check `docs/implementation/ALP_INTEGRATION_COMPLETE.md`
- Review test cases in `tests/alp_integration_tests.rs`
- File issue on GitHub (include compression stats output)