HTTP Sync API Documentation

HeliosDB Nano v2.3.0 Synchronization Protocol

This document describes the REST API endpoints for client-server synchronization over HTTP.

Base URL

http://localhost:8080/api/v1/sync

Authentication

All endpoints except /health require JWT authentication. Include the JWT token in the Authorization header:

Authorization: Bearer <jwt_token>

Common Headers

Request Headers

Content-Type: application/json
Authorization: Bearer <jwt_token>

Response Headers

Content-Type: application/json
X-Request-ID: <uuid>

Endpoints

1. Register Client

Endpoint: POST /api/v1/sync/register

Authentication: Optional (can be made required)

Request Body:

{
  "type": "register_client",
  "version": 1,
  "client_id": "client-123",
  "last_known_lsn": 0,
  "vector_clock": {},
  "metadata": {
    "device": "mobile",
    "os": "android"
  }
}

Request Fields:

type (string, required): Must be “register_client”
version (integer, required): Protocol version (currently 1)
client_id (string, required): Unique client identifier
last_known_lsn (integer, required): Last known Log Sequence Number
vector_clock (object, required): Client’s vector clock state
metadata (object, optional): Additional client metadata

Response (200 OK):

{
  "success": true,
  "client_id": "client-123",
  "server_version": 1
}

Error Responses:

400 Bad Request: Invalid message format or unsupported protocol version
500 Internal Server Error: Server error during registration

Example:

curl -X POST http://localhost:8080/api/v1/sync/register \
  -H "Content-Type: application/json" \
  -d '{
    "type": "register_client",
    "version": 1,
    "client_id": "client-123",
    "last_known_lsn": 0,
    "vector_clock": {},
    "metadata": {}
  }'

2. Pull Changes

Pull changes from the server since a given LSN.

Endpoint: POST /api/v1/sync/pull

Authentication: Required

Request Body:

{
  "type": "pull_request",
  "message_id": "550e8400-e29b-41d4-a716-446655440000",
  "client_id": "client-123",
  "since_lsn": 100,
  "max_entries": 1000,
  "continuation_token": null
}

Request Fields:

type (string, required): Must be “pull_request”
message_id (uuid, required): Unique message ID for idempotency
client_id (string, required): Client identifier (must match JWT)
since_lsn (integer, required): Fetch changes after this LSN
max_entries (integer, required): Maximum entries to return (max 1000)
continuation_token (string, optional): Token for pagination

Response (200 OK):

{
  "type": "pull_response",
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "changes": [
    {
      "lsn": 101,
      "table": "users",
      "operation": "Insert",
      "key": [1, 2, 3],
      "data": [4, 5, 6],
      "vector_clock": {},
      "timestamp": "2025-11-24T10:30:00Z",
      "checksum": 1234567890,
      "compressed": false
    }
  ],
  "server_lsn": 150,
  "has_more": true,
  "continuation_token": "10",
  "vector_clock": {}
}

Response Fields:

request_id: Message ID from request (for correlation)
changes: Array of change entries
server_lsn: Current server LSN
has_more: Whether more changes are available
continuation_token: Token for next page (if has_more is true)
vector_clock: Server’s vector clock

Error Responses:

400 Bad Request: Invalid request format
401 Unauthorized: Authentication failed or client_id mismatch
500 Internal Server Error: Server error

Idempotency:

Duplicate requests with the same message_id return the cached response.

Example:

curl -X POST http://localhost:8080/api/v1/sync/pull \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "pull_request",
    "message_id": "550e8400-e29b-41d4-a716-446655440000",
    "client_id": "client-123",
    "since_lsn": 0,
    "max_entries": 100,
    "continuation_token": null
  }'

3. Push Changes

Push local changes to the server.

Endpoint: POST /api/v1/sync/push

Authentication: Required

Request Body:

{
  "type": "push_changes",
  "message_id": "660e8400-e29b-41d4-a716-446655440001",
  "client_id": "client-123",
  "changes": [
    {
      "lsn": 1,
      "table": "tasks",
      "operation": "Insert",
      "key": [1, 2, 3],
      "data": [10, 20, 30],
      "vector_clock": {},
      "timestamp": "2025-11-24T10:30:00Z",
      "checksum": 987654321,
      "compressed": false
    }
  ],
  "vector_clock": {}
}

Request Fields:

type (string, required): Must be “push_changes”
message_id (uuid, required): Unique message ID for idempotency
client_id (string, required): Client identifier (must match JWT)
changes (array, required): Array of change entries to push
vector_clock (object, required): Client’s vector clock

Response (200 OK):

{
  "type": "push_ack",
  "request_id": "660e8400-e29b-41d4-a716-446655440001",
  "accepted_lsns": [201, 202, 203],
  "conflicts": [],
  "server_lsn": 203,
  "vector_clock": {}
}

Response Fields:

request_id: Message ID from request
accepted_lsns: LSNs of successfully accepted changes
conflicts: Array of detected conflicts
server_lsn: Updated server LSN
vector_clock: Server’s vector clock

Conflict Format:

{
  "lsn": 202,
  "table": "users",
  "key": [1, 2, 3],
  "conflict_type": "ConcurrentUpdate",
  "description": "Concurrent update detected"
}

Conflict Types:

ConcurrentUpdate: Concurrent modifications detected
DeletedOnServer: Record was deleted on server
UniqueConstraintViolation: Unique constraint violated

Error Responses:

400 Bad Request: Invalid request, checksum mismatch, or message size exceeded
401 Unauthorized: Authentication failed or client_id mismatch
409 Conflict: Conflicts detected (see conflicts array)
500 Internal Server Error: Server error

Idempotency:

Duplicate requests with the same message_id return the cached response.

Example:

curl -X POST http://localhost:8080/api/v1/sync/push \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "push_changes",
    "message_id": "660e8400-e29b-41d4-a716-446655440001",
    "client_id": "client-123",
    "changes": [...],
    "vector_clock": {}
  }'

4. Heartbeat

Send a client heartbeat to indicate activity.

Endpoint: POST /api/v1/sync/heartbeat

Authentication: Required

Request Body:

{
  "type": "heartbeat",
  "client_id": "client-123",
  "timestamp": 1700000000000,
  "current_lsn": 150
}

Request Fields:

type (string, required): Must be “heartbeat”
client_id (string, required): Client identifier (must match JWT)
timestamp (integer, required): Timestamp in milliseconds since epoch
current_lsn (integer, required): Client’s current LSN

Response (200 OK):

{
  "success": true,
  "timestamp": 1700000000500
}

Error Responses:

400 Bad Request: Invalid request format
401 Unauthorized: Authentication failed or client_id mismatch
404 Not Found: Client not registered

Example:

curl -X POST http://localhost:8080/api/v1/sync/heartbeat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "heartbeat",
    "client_id": "client-123",
    "timestamp": 1700000000000,
    "current_lsn": 150
  }'

5. Health Check

Check server health and status.

Endpoint: GET /api/v1/sync/health

Authentication: Not required

Response (200 OK):

{
  "status": "healthy",
  "version": 1,
  "uptime_secs": 3600,
  "registered_clients": 42,
  "timestamp": 1700000000000
}

Response Fields:

status: Server status (“healthy” or “degraded”)
version: Protocol version
uptime_secs: Server uptime in seconds
registered_clients: Number of registered clients
timestamp: Current server timestamp (milliseconds since epoch)

Example:

curl http://localhost:8080/api/v1/sync/health

Error Response Format

All error responses follow this format:

{
  "error": "Human-readable error message",
  "code": "ERROR_CODE",
  "details": "Optional additional details"
}

Common Error Codes:

BAD_REQUEST: Invalid request format or parameters
UNAUTHORIZED: Authentication failed
AUTH_FAILED: JWT validation failed
INVALID_MESSAGE: Message validation failed
NETWORK_ERROR: Network-related error
STORAGE_ERROR: Storage backend error
SERIALIZATION_ERROR: Serialization/deserialization error
CONFLICT_ERROR: Conflict resolution failed
QUEUE_FULL: Server queue full
INTERNAL_ERROR: Internal server error

Rate Limiting

Currently, no rate limiting is implemented. This should be added for production use.

Recommended limits:

Pull: 100 requests/minute per client
Push: 100 requests/minute per client
Heartbeat: 120 requests/hour per client (once per 30 seconds)

Compression

The server supports gzip compression for responses. Include the header:

Accept-Encoding: gzip

Changes in push/pull requests can be compressed using zstd by setting compressed: true in the change entry.

CORS

CORS is enabled for all origins by default. In production, configure specific allowed origins.

Best Practices

1. Idempotency

Always use unique message_id values (UUIDv4) for pull and push requests. The server caches responses for idempotency.

2. Batch Size

Use appropriate batch sizes:

Pull: Up to 1000 entries (default)
Push: Keep total message size under 1MB

3. Pagination

When has_more is true in pull response, use the continuation_token for the next request.

4. Heartbeat

Send heartbeats every 30-45 seconds to maintain client health status. Clients without heartbeat for 60 seconds are considered inactive.

5. Error Handling

Retry on 5xx errors with exponential backoff
Don’t retry on 4xx errors (except 429 Too Many Requests)
Handle conflicts explicitly

6. Checksum Verification

Always verify checksums on received changes to detect data corruption.

Example Client Flow

// 1. Register client
const registerResponse = await fetch('/api/v1/sync/register', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    type: 'register_client',
    version: 1,
    client_id: clientId,
    last_known_lsn: 0,
    vector_clock: {},
    metadata: {}
  })
});

// 2. Pull changes
const pullResponse = await fetch('/api/v1/sync/pull', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${token}`
  },
  body: JSON.stringify({
    type: 'pull_request',
    message_id: generateUUID(),
    client_id: clientId,
    since_lsn: lastKnownLsn,
    max_entries: 100,
    continuation_token: null
  })
});

// 3. Push changes
const pushResponse = await fetch('/api/v1/sync/push', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${token}`
  },
  body: JSON.stringify({
    type: 'push_changes',
    message_id: generateUUID(),
    client_id: clientId,
    changes: localChanges,
    vector_clock: localVectorClock
  })
});

// 4. Send heartbeat
setInterval(async () => {
  await fetch('/api/v1/sync/heartbeat', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${token}`
    },
    body: JSON.stringify({
      type: 'heartbeat',
      client_id: clientId,
      timestamp: Date.now(),
      current_lsn: currentLsn
    })
  });
}, 30000); // Every 30 seconds

Performance Characteristics

Latency: <100ms for typical requests (local network)
Throughput: 1000+ requests/second on modern hardware
Batch Size: Up to 1000 entries per pull/push
Message Size: Maximum 1MB per message
Concurrent Clients: Supports thousands of concurrent clients

Security Considerations

Always use HTTPS in production
Rotate JWT secrets regularly
Implement rate limiting
Validate all inputs
Use strong JWT secrets (256+ bits)
Enable tenant isolation
Monitor for abuse
Implement request size limits
Use security headers
Enable audit logging

Troubleshooting

401 Unauthorized

Check JWT token validity
Verify client_id matches token
Ensure token hasn’t expired
Check tenant authorization

400 Bad Request

Verify JSON format
Check message type field
Validate all required fields
Check message size

409 Conflict

Review conflicts array
Implement conflict resolution
Re-pull and retry

500 Internal Server Error

Check server logs
Verify storage backend health
Check system resources

Monitoring

Recommended metrics to monitor:

Request rate per endpoint
Response latency (p50, p95, p99)
Error rate by status code
Active client count
Message size distribution
Queue depth
Storage latency

Version History

v1.0 (2025-11-24): Initial HTTP API implementation
- All core endpoints
- JWT authentication
- Idempotency support
- Pagination
- Compression
- CORS support