Time-Series Data with Vector Analysis
HeliosDB Lite analyses time-series data and its vector embeddings in one database for monitoring and anomaly investigation: timestamp and measurement columns use STORAGE COLUMNAR for aggregation scans, a VECTOR column holds an embedding of each window of readings, an HNSW index finds windows similar to a known pattern, and window functions such as LAG compute changes between consecutive readings.
Executive Summary
This unique combination delivers:
- Extreme data compression (100:1 ratio for time-series data)
- Real-time anomaly detection (patterns detected within seconds of occurrence)
- Embedded deployment (no separate time-series database required)
Problem Being Solved
The Time-Series + AI Analytics Dilemma
Option B: Relational Database + Time-Series DB
- Pro: Flexible SQL queries
- Pro: ACID compliance
- Pro: Time-series compression
- Con: Two systems to manage (complexity doubles)
- Con: Data synchronization issues (consistency problems)
- Con: Cannot do semantic queries (requires ML pipeline)
Option C: Data Lake + ML Pipeline (Modern Approach)
- Pro: Can do semantic analysis
- Pro: Flexible data storage
- Con: Complex data pipeline (5-10 components to manage)
- Con: Operational burden (requires data engineering team)
Enterprise Pain Points
Operational Complexity:
- Managing separate databases (time-series + relational)
- Synchronizing data between systems
- Debugging data consistency issues
- Scaling databases separately
- Complex disaster recovery procedures
Technical Limitations:
- No semantic queries: Cannot ask "find patterns similar to this failure"
- Batch analysis only: Pattern detection happens hours later
- Data silos: IoT data separate from business data
- Schema rigidity: Changes require pipeline reconfiguration
- No transactional guarantees: Data loss during failures
Implementation Examples
Example 2: Real-Time Monitoring Dashboard (React)
import React, { useState, useEffect } from 'react';
import { LineChart, Line, XAxis, YAxis, CartesianGrid } from 'recharts';
export function TimeSeriesMonitor() {
const [readings, setReadings] = useState([]);
const [anomalies, setAnomalies] = useState([]);
const [connected, setConnected] = useState(false);
useEffect(() => {
// WebSocket connection for real-time data
const ws = new WebSocket('ws://localhost:8080/sensor-stream');
ws.onopen = () => {
setConnected(true);
// Subscribe to sensor data
ws.send(JSON.stringify({
action: 'subscribe',
sensors: ['sensor-42', 'sensor-43', 'sensor-44'],
}));
};
ws.onmessage = async (event) => {
const message = JSON.parse(event.data);
if (message.type === 'reading') {
// Add new reading to chart
setReadings((prev) => [
...prev.slice(-99), // Keep last 100
{
timestamp: new Date(message.timestamp).toLocaleTimeString(),
sensor_id: message.sensor_id,
temperature: message.temperature,
},
]);
// Check for anomalies
if (message.anomaly_score > 0.85) {
setAnomalies((prev) => [
{
id: message.sensor_id,
pattern: message.pattern_description,
confidence: message.anomaly_score,
timestamp: new Date(),
},
...prev.slice(0, 4), // Keep last 5
]);
}
}
};
ws.onerror = () => setConnected(false);
ws.onclose = () => setConnected(false);
return () => ws.close();
}, []);
return (
<div className="time-series-monitor">
<div className="header">
<h2>Real-Time IoT Monitoring</h2>
<div className={`status ${connected ? 'connected' : 'disconnected'}`}>
{connected ? ' Connected' : ' Disconnected'}
</div>
</div>
<div className="charts-section">
<h3>Temperature Readings (Last 100)</h3>
<LineChart width={800} height={300} data={readings}>
<CartesianGrid />
<XAxis dataKey="timestamp" />
<YAxis />
<Line
type="monotone"
dataKey="temperature"
stroke="#8884d8"
dot={false}
isAnimationActive={false}
/>
</LineChart>
</div>
<div className="anomalies-section">
<h3> Recent Anomalies</h3>
{anomalies.length === 0 ? (
<p>No anomalies detected</p>
) : (
<table>
<thead>
<tr>
<th>Sensor</th>
<th>Pattern</th>
<th>Confidence</th>
<th>Time</th>
</tr>
</thead>
<tbody>
{anomalies.map((anomaly) => (
<tr key={`${anomaly.id}-${anomaly.timestamp}`}>
<td>{anomaly.id}</td>
<td>{anomaly.pattern}</td>
<td>{(anomaly.confidence * 100).toFixed(1)}%</td>
<td>{anomaly.timestamp.toLocaleTimeString()}</td>
</tr>
))}
</tbody>
</table>
)}
</div>
</div>
);
}Example 3: Time-Series Queries with Vector Analysis (SQL)
-- 1. Find all sensors with anomalous patterns (vector similarity)
SELECT
s.sensor_id,
s.sensor_name,
s.location,
fp.pattern_description,
1 - (se.embedding <-> fp.pattern_embedding) as similarity,
COUNT(*) as matching_events
FROM sensor_embeddings se
JOIN known_failure_patterns fp ON 1 = 1 -- Cross join for similarity
JOIN sensors s ON se.sensor_id = s.sensor_id
WHERE 1 - (se.embedding <-> fp.pattern_embedding) > 0.85
AND se.timestamp > datetime('now', '-1 hour')
GROUP BY s.sensor_id, fp.pattern_id
ORDER BY similarity DESC
LIMIT 50;
-- 2. Temperature gradient analysis (find rapid changes)
WITH temp_changes AS (
SELECT
sensor_id,
timestamp,
temperature,
LAG(temperature) OVER (
PARTITION BY sensor_id
ORDER BY timestamp
) as prev_temp,
ABS(temperature - LAG(temperature) OVER (
PARTITION BY sensor_id
ORDER BY timestamp
)) as temp_change_rate
FROM sensor_readings
WHERE timestamp > datetime('now', '-6 hours')
)
SELECT
sensor_id,
COUNT(*) as rapid_changes,
MAX(temp_change_rate) as max_change,
AVG(temp_change_rate) as avg_change,
STDDEV(temp_change_rate) as volatility
FROM temp_changes
WHERE temp_change_rate > 2.0 -- Degrees per minute
GROUP BY sensor_id
HAVING COUNT(*) > 5 -- Multiple changes (pattern)
ORDER BY volatility DESC;
-- 3. Correlation analysis (multi-sensor failure patterns)
SELECT
s1.sensor_id as sensor_1,
s2.sensor_id as sensor_2,
CORR(s1.temperature, s2.temperature) as temp_correlation,
CORR(s1.humidity, s2.humidity) as humidity_correlation,
COUNT(*) as reading_pairs
FROM sensor_readings s1
JOIN sensor_readings s2
ON s1.timestamp = s2.timestamp
AND s1.sensor_id < s2.sensor_id
WHERE s1.timestamp > datetime('now', '-24 hours')
GROUP BY s1.sensor_id, s2.sensor_id
HAVING CORR(s1.temperature, s2.temperature) > 0.9 -- Strong correlation
ORDER BY reading_pairs DESC;
-- 4. Predictive pattern detection (before failure occurs)
WITH recent_readings AS (
SELECT
sensor_id,
timestamp,
temperature,
humidity,
pressure,
ROW_NUMBER() OVER (
PARTITION BY sensor_id
ORDER BY timestamp DESC
) as recency
FROM sensor_readings
WHERE timestamp > datetime('now', '-4 hours')
)
SELECT
r.sensor_id,
s.sensor_name,
COUNT(*) as reading_count,
AVG(r.temperature) as avg_temp,
STDDEV(r.temperature) as temp_volatility,
MAX(r.pressure) as max_pressure,
CASE
WHEN STDDEV(r.temperature) > 5 THEN 'HIGH_THERMAL_VOLATILITY'
WHEN STDDEV(r.humidity) > 15 THEN 'HIGH_HUMIDITY_VARIANCE'
WHEN MAX(r.pressure) > 1050 THEN 'PRESSURE_SPIKE'
ELSE 'NORMAL'
END as risk_level
FROM recent_readings r
JOIN sensors s ON r.sensor_id = s.sensor_id
WHERE r.recency <= 60
GROUP BY r.sensor_id
HAVING STDDEV(r.temperature) > 3 -- Statistical anomaly
ORDER BY temp_volatility DESC;
-- 5. Semantic similarity search (find similar event sequences)
SELECT
e1.event_id as query_event,
e2.event_id as similar_event,
e2.event_type as similar_type,
1 - (e1.embedding <-> e2.embedding) as similarity,
e2.description,
e2.resolution_time_ms
FROM events e1
CROSS JOIN events e2
WHERE e1.event_id = 'event_12345' -- Find similar to this event
AND 1 - (e1.embedding <-> e2.embedding) > 0.8 -- 80% similar
AND e2.event_id != e1.event_id
ORDER BY similarity DESC
LIMIT 10;Example 5: Docker Compose - Full IoT Stack
# Dockerfile - IoT monitoring application
FROM rust:latest as builder
WORKDIR /app
COPY Cargo.* ./
COPY src ./src
RUN cargo build --release
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y curl
COPY --from=builder /app/target/release/iot-monitor /usr/local/bin/
RUN mkdir -p /data && chmod 700 /data
RUN useradd -m -u 1000 app
USER app:app
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s \
CMD curl -f http://localhost:8080/health || exit 1
ENTRYPOINT ["iot-monitor"]# docker-compose.yml - Complete IoT monitoring stack
version: '3.8'
services:
# IoT monitoring backend
iot-monitor:
build: .
environment:
RUST_LOG: info
DATABASE_PATH: /data/iot.db
MAX_MEMORY_MB: 512
COMPRESSION_LEVEL: 9
VECTOR_SEARCH_ENABLED: "true"
volumes:
- iot-data:/data
ports:
- "8080:8080"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 3s
retries: 3
# Sensor simulator (for testing)
sensor-simulator:
image: node:18-alpine
working_dir: /app
volumes:
- ./simulator:/app
- /app/node_modules
environment:
API_URL: http://iot-monitor:8080
SENSORS_COUNT: 100
READINGS_PER_SECOND: 1000
command: npm start
depends_on:
- iot-monitor
# Grafana dashboard
grafana:
image: grafana/grafana:latest
environment:
GF_SECURITY_ADMIN_PASSWORD: admin
ports:
- "3000:3000"
volumes:
- grafana-storage:/var/lib/grafana
- ./grafana/datasources:/etc/grafana/provisioning/datasources
depends_on:
- iot-monitor
volumes:
iot-data:
grafana-storage:Conclusion
By combining extreme compression, native vector search, and ACID guarantees, it enables organizations to:
- Enable predictive maintenance (prevent failures before they occur)
For any organization managing IoT or time-series data: HeliosDB Lite is transformational in cost, performance, and capability.
Performance depends on workload and hardware. Published figures are measured on representative fixtures; see the benchmarks page and reproduce them on your own workload.