---
title: "HTTP Sync API Documentation"
---

# HTTP Sync API Documentation

## HeliosDB Nano Synchronization Protocol

This document describes the REST API endpoints for client-server synchronization over HTTP.

## Base URL

```
http://localhost:8080/api/v1/sync
```

## Authentication

All endpoints except `/health` require JWT authentication. Include the JWT token in the Authorization header:

```
Authorization: Bearer <jwt_token>
```

## Common Headers

### Request Headers

```
Content-Type: application/json
Authorization: Bearer <jwt_token>
```

### Response Headers

```
Content-Type: application/json
X-Request-ID: <uuid>
```

## Endpoints

### 1. Register Client

Register a client for synchronization.

**Endpoint:** `POST /api/v1/sync/register`

**Authentication:** Optional (can be made required)

**Request Body:**

```json
{
  "type": "register_client",
  "version": 1,
  "client_id": "client-123",
  "last_known_lsn": 0,
  "vector_clock": {},
  "metadata": {
    "device": "mobile",
    "os": "android"
  }
}
```

**Request Fields:**

- `type` (string, required): Must be "register_client"
- `version` (integer, required): Protocol version (currently 1)
- `client_id` (string, required): Unique client identifier
- `last_known_lsn` (integer, required): Last known Log Sequence Number
- `vector_clock` (object, required): Client's vector clock state
- `metadata` (object, optional): Additional client metadata

**Response (200 OK):**

```json
{
  "success": true,
  "client_id": "client-123",
  "server_version": 1
}
```

**Error Responses:**

- `400 Bad Request`: Invalid message format or unsupported protocol version
- `500 Internal Server Error`: Server error during registration

**Example:**

```bash
curl -X POST http://localhost:8080/api/v1/sync/register \
  -H "Content-Type: application/json" \
  -d '{
    "type": "register_client",
    "version": 1,
    "client_id": "client-123",
    "last_known_lsn": 0,
    "vector_clock": {},
    "metadata": {}
  }'
```

---

### 2. Pull Changes

Pull changes from the server since a given LSN.

**Endpoint:** `POST /api/v1/sync/pull`

**Authentication:** Required

**Request Body:**

```json
{
  "type": "pull_request",
  "message_id": "550e8400-e29b-41d4-a716-446655440000",
  "client_id": "client-123",
  "since_lsn": 100,
  "max_entries": 1000,
  "continuation_token": null
}
```

**Request Fields:**

- `type` (string, required): Must be "pull_request"
- `message_id` (uuid, required): Unique message ID for idempotency
- `client_id` (string, required): Client identifier (must match JWT)
- `since_lsn` (integer, required): Fetch changes after this LSN
- `max_entries` (integer, required): Maximum entries to return (max 1000)
- `continuation_token` (string, optional): Token for pagination

**Response (200 OK):**

```json
{
  "type": "pull_response",
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "changes": [
    {
      "lsn": 101,
      "table": "users",
      "operation": "Insert",
      "key": [1, 2, 3],
      "data": [4, 5, 6],
      "vector_clock": {},
      "timestamp": "2025-11-24T10:30:00Z",
      "checksum": 1234567890,
      "compressed": false
    }
  ],
  "server_lsn": 150,
  "has_more": true,
  "continuation_token": "10",
  "vector_clock": {}
}
```

**Response Fields:**

- `request_id`: Message ID from request (for correlation)
- `changes`: Array of change entries
- `server_lsn`: Current server LSN
- `has_more`: Whether more changes are available
- `continuation_token`: Token for next page (if has_more is true)
- `vector_clock`: Server's vector clock

**Error Responses:**

- `400 Bad Request`: Invalid request format
- `401 Unauthorized`: Authentication failed or client_id mismatch
- `500 Internal Server Error`: Server error

**Idempotency:**

Duplicate requests with the same `message_id` return the cached response.

**Example:**

```bash
curl -X POST http://localhost:8080/api/v1/sync/pull \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "pull_request",
    "message_id": "550e8400-e29b-41d4-a716-446655440000",
    "client_id": "client-123",
    "since_lsn": 0,
    "max_entries": 100,
    "continuation_token": null
  }'
```

---

### 3. Push Changes

Push local changes to the server.

**Endpoint:** `POST /api/v1/sync/push`

**Authentication:** Required

**Request Body:**

```json
{
  "type": "push_changes",
  "message_id": "660e8400-e29b-41d4-a716-446655440001",
  "client_id": "client-123",
  "changes": [
    {
      "lsn": 1,
      "table": "tasks",
      "operation": "Insert",
      "key": [1, 2, 3],
      "data": [10, 20, 30],
      "vector_clock": {},
      "timestamp": "2025-11-24T10:30:00Z",
      "checksum": 987654321,
      "compressed": false
    }
  ],
  "vector_clock": {}
}
```

**Request Fields:**

- `type` (string, required): Must be "push_changes"
- `message_id` (uuid, required): Unique message ID for idempotency
- `client_id` (string, required): Client identifier (must match JWT)
- `changes` (array, required): Array of change entries to push
- `vector_clock` (object, required): Client's vector clock

**Response (200 OK):**

```json
{
  "type": "push_ack",
  "request_id": "660e8400-e29b-41d4-a716-446655440001",
  "accepted_lsns": [201, 202, 203],
  "conflicts": [],
  "server_lsn": 203,
  "vector_clock": {}
}
```

**Response Fields:**

- `request_id`: Message ID from request
- `accepted_lsns`: LSNs of successfully accepted changes
- `conflicts`: Array of detected conflicts
- `server_lsn`: Updated server LSN
- `vector_clock`: Server's vector clock

**Conflict Format:**

```json
{
  "lsn": 202,
  "table": "users",
  "key": [1, 2, 3],
  "conflict_type": "ConcurrentUpdate",
  "description": "Concurrent update detected"
}
```

**Conflict Types:**

- `ConcurrentUpdate`: Concurrent modifications detected
- `DeletedOnServer`: Record was deleted on server
- `UniqueConstraintViolation`: Unique constraint violated

**Error Responses:**

- `400 Bad Request`: Invalid request, checksum mismatch, or message size exceeded
- `401 Unauthorized`: Authentication failed or client_id mismatch
- `409 Conflict`: Conflicts detected (see conflicts array)
- `500 Internal Server Error`: Server error

**Idempotency:**

Duplicate requests with the same `message_id` return the cached response.

**Example:**

```bash
curl -X POST http://localhost:8080/api/v1/sync/push \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "push_changes",
    "message_id": "660e8400-e29b-41d4-a716-446655440001",
    "client_id": "client-123",
    "changes": [...],
    "vector_clock": {}
  }'
```

---

### 4. Heartbeat

Send a client heartbeat to indicate activity.

**Endpoint:** `POST /api/v1/sync/heartbeat`

**Authentication:** Required

**Request Body:**

```json
{
  "type": "heartbeat",
  "client_id": "client-123",
  "timestamp": 1700000000000,
  "current_lsn": 150
}
```

**Request Fields:**

- `type` (string, required): Must be "heartbeat"
- `client_id` (string, required): Client identifier (must match JWT)
- `timestamp` (integer, required): Timestamp in milliseconds since epoch
- `current_lsn` (integer, required): Client's current LSN

**Response (200 OK):**

```json
{
  "success": true,
  "timestamp": 1700000000500
}
```

**Error Responses:**

- `400 Bad Request`: Invalid request format
- `401 Unauthorized`: Authentication failed or client_id mismatch
- `404 Not Found`: Client not registered

**Example:**

```bash
curl -X POST http://localhost:8080/api/v1/sync/heartbeat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "type": "heartbeat",
    "client_id": "client-123",
    "timestamp": 1700000000000,
    "current_lsn": 150
  }'
```

---

### 5. Health Check

Check server health and status.

**Endpoint:** `GET /api/v1/sync/health`

**Authentication:** Not required

**Response (200 OK):**

```json
{
  "status": "healthy",
  "version": 1,
  "uptime_secs": 3600,
  "registered_clients": 42,
  "timestamp": 1700000000000
}
```

**Response Fields:**

- `status`: Server state ("healthy" or "degraded")
- `version`: Protocol version
- `uptime_secs`: Server uptime in seconds
- `registered_clients`: Number of registered clients
- `timestamp`: Current server timestamp (milliseconds since epoch)

**Example:**

```bash
curl http://localhost:8080/api/v1/sync/health
```

---

## Error Response Format

All error responses follow this format:

```json
{
  "error": "Human-readable error message",
  "code": "ERROR_CODE",
  "details": "Optional additional details"
}
```

**Common Error Codes:**

- `BAD_REQUEST`: Invalid request format or parameters
- `UNAUTHORIZED`: Authentication failed
- `AUTH_FAILED`: JWT validation failed
- `INVALID_MESSAGE`: Message validation failed
- `NETWORK_ERROR`: Network-related error
- `STORAGE_ERROR`: Storage backend error
- `SERIALIZATION_ERROR`: Serialization/deserialization error
- `CONFLICT_ERROR`: Conflict resolution failed
- `QUEUE_FULL`: Server queue full
- `INTERNAL_ERROR`: Internal server error

---

## Rate Limiting

Currently, no rate limiting is implemented. This should be added for production use.

Recommended limits:
- Pull: 100 requests/minute per client
- Push: 100 requests/minute per client
- Heartbeat: 120 requests/hour per client (once per 30 seconds)

---

## Compression

The server supports gzip compression for responses. Include the header:

```
Accept-Encoding: gzip
```

Changes in push/pull requests can be compressed using zstd by setting `compressed: true` in the change entry.

---

## CORS

CORS is enabled for all origins by default. In production, configure specific allowed origins.

---

## Best Practices

### 1. Idempotency

Always use unique `message_id` values (UUIDv4) for pull and push requests. The server caches responses for idempotency.

### 2. Batch Size

Use appropriate batch sizes:
- Pull: Up to 1000 entries (default)
- Push: Keep total message size under 1MB

### 3. Pagination

When `has_more` is true in pull response, use the `continuation_token` for the next request.

### 4. Heartbeat

Send heartbeats every 30-45 seconds to maintain client health status. Clients without heartbeat for 60 seconds are considered inactive.

### 5. Error Handling

- Retry on 5xx errors with exponential backoff
- Don't retry on 4xx errors (except 429 Too Many Requests)
- Handle conflicts explicitly

### 6. Checksum Verification

Always verify checksums on received changes to detect data corruption.

---

## Example Client Flow

```javascript
// 1. Register client
const registerResponse = await fetch('/api/v1/sync/register', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    type: 'register_client',
    version: 1,
    client_id: clientId,
    last_known_lsn: 0,
    vector_clock: {},
    metadata: {}
  })
});

// 2. Pull changes
const pullResponse = await fetch('/api/v1/sync/pull', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${token}`
  },
  body: JSON.stringify({
    type: 'pull_request',
    message_id: generateUUID(),
    client_id: clientId,
    since_lsn: lastKnownLsn,
    max_entries: 100,
    continuation_token: null
  })
});

// 3. Push changes
const pushResponse = await fetch('/api/v1/sync/push', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${token}`
  },
  body: JSON.stringify({
    type: 'push_changes',
    message_id: generateUUID(),
    client_id: clientId,
    changes: localChanges,
    vector_clock: localVectorClock
  })
});

// 4. Send heartbeat
setInterval(async () => {
  await fetch('/api/v1/sync/heartbeat', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${token}`
    },
    body: JSON.stringify({
      type: 'heartbeat',
      client_id: clientId,
      timestamp: Date.now(),
      current_lsn: currentLsn
    })
  });
}, 30000); // Every 30 seconds
```

---

## Performance Characteristics

- **Latency**: <100ms for typical requests (local network)
- **Throughput**: 1000+ requests/second on modern hardware
- **Batch Size**: Up to 1000 entries per pull/push
- **Message Size**: Maximum 1MB per message
- **Concurrent Clients**: Supports thousands of concurrent clients

---

## Security Considerations

1. **Always use HTTPS in production**
2. **Rotate JWT secrets regularly**
3. **Implement rate limiting**
4. **Validate all inputs**
5. **Use strong JWT secrets (256+ bits)**
6. **Enable tenant isolation**
7. **Monitor for abuse**
8. **Implement request size limits**
9. **Use security headers**
10. **Enable audit logging**

---

## Troubleshooting

### 401 Unauthorized

- Check JWT token validity
- Verify client_id matches token
- Ensure token hasn't expired
- Check tenant authorization

### 400 Bad Request

- Verify JSON format
- Check message type field
- Validate all required fields
- Check message size

### 409 Conflict

- Review conflicts array
- Implement conflict resolution
- Re-pull and retry

### 500 Internal Server Error

- Check server logs
- Verify storage backend health
- Check system resources

---

## Monitoring

Recommended metrics to monitor:

- Request rate per endpoint
- Response latency (p50, p95, p99)
- Error rate by status code
- Active client count
- Message size distribution
- Queue depth
- Storage latency