Reading Operations

This page documents all operations for reading data from streams. The protocol supports three read modes: catch-up, long-poll, and Server-Sent Events (SSE).

Catch-Up Read

Returns immediately with available data. Used for replaying historical data or catching up after disconnection.

Request

GET {stream-url}?offset={offset}

Parameters

Parameter	Required	Description
`offset`	No	Starting position. Defaults to stream start (`-1`)

Example

GET /streams/events?offset=abc123 HTTP/1.1
Host: api.example.com

HTTP/1.1 200 OK
Content-Type: application/json
Stream-Next-Offset: def456
Stream-Up-To-Date: true

[{"event": "click", "x": 100}, {"event": "scroll", "y": 200}]

Response Codes

Code	Meaning
`200 OK`	Data returned (or empty if at end)
`400 Bad Request`	Invalid offset format
`404 Not Found`	Stream does not exist
`410 Gone`	Offset is before retention window
`429 Too Many Requests`	Rate limit exceeded

Response Headers

Header	Description
`Content-Type`	Stream’s content type
`Stream-Next-Offset`	Offset for next read
`Stream-Up-To-Date`	Present when at end of stream
`Stream-Cursor`	Optional cursor for CDN collapsing
`ETag`	Entity tag for cache validation
`Cache-Control`	Caching directives

Behavior

When offset equals or exceeds the tail offset:

Response body is empty (or [] for JSON mode)
Stream-Up-To-Date: true header is present
Stream-Next-Offset equals the requested offset

When offset is before the tail:

Response contains data from offset to tail (or server-defined chunk limit)
Stream-Up-To-Date is present only if all data was returned

Long-Poll Read

Waits for new data if none is immediately available. Enables efficient live tailing without constant polling.

Request

GET {stream-url}?offset={offset}&live=long-poll[&cursor={cursor}]

Parameters

Parameter	Required	Description
`offset`	Yes	Starting position
`live`	Yes	Must be `long-poll`
`cursor`	No	Echo of previous `Stream-Cursor`

Example - Data Available

GET /streams/events?offset=def456&live=long-poll HTTP/1.1
Host: api.example.com

HTTP/1.1 200 OK
Content-Type: application/json
Stream-Next-Offset: ghi789
Stream-Cursor: c_12345

[{"event": "new-data"}]

Example - Timeout

GET /streams/events?offset=ghi789&live=long-poll&cursor=c_12345 HTTP/1.1
Host: api.example.com

HTTP/1.1 204 No Content
Stream-Next-Offset: ghi789

Response Codes

Code	Meaning
`200 OK`	New data arrived within timeout
`204 No Content`	Timeout expired with no new data
`400 Bad Request`	Invalid parameters
`404 Not Found`	Stream does not exist
`429 Too Many Requests`	Rate limit exceeded

Timeout Behavior

Server defines the timeout duration (typically 30 seconds)
Returns 204 No Content if no data arrives before timeout
Client should immediately retry with the same offset
Servers MAY accept a timeout parameter (in seconds) as an extension

CDN Collapsing

The cursor parameter enables CDN request collapsing:

┌─────────┐
│ Client1 │──┐
└─────────┘  │  GET ?offset=X&live=long-poll&cursor=Y
             │
┌─────────┐  │     ┌─────┐        ┌────────┐
│ Client2 │──┼────▶│ CDN │───────▶│ Origin │
└─────────┘  │     └─────┘        └────────┘
             │        │
┌─────────┐  │        │  Single upstream request
│ Client3 │──┘        │  Response fanned out
└─────────┘           ▼

Clients SHOULD echo the Stream-Cursor from the previous response.

SSE (Server-Sent Events)

Maintains an open connection for real-time streaming. Best for browser clients.

Request

GET {stream-url}?offset={offset}&live=sse

Parameters

Parameter	Required	Description
`offset`	Yes	Starting position
`live`	Yes	Must be `sse`

Content Type Restriction

SSE mode REQUIRES the stream to have one of these content types:

text/* (any text type)
application/json

Binary streams (application/octet-stream) cannot use SSE mode.

Example

GET /streams/events?offset=abc123&live=sse HTTP/1.1
Host: api.example.com
Accept: text/event-stream

HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive

event: data
data: {"event": "click"}

event: control
data: {"streamNextOffset": "def456"}

event: data
data: {"event": "scroll"}

event: control
data: {"streamNextOffset": "ghi789"}

Event Types

Event	Description
`data`	Stream content
`control`	Metadata (offset, cursor)

Control Event Format

{
  "streamNextOffset": "abc123",
  "streamCursor": "c_12345"
}

Field	Required	Description
`streamNextOffset`	Yes	Offset for resumption
`streamCursor`	No	CDN collapsing cursor

Connection Lifecycle

Servers SHOULD close SSE connections approximately every 60 seconds:

┌────────┐                              ┌────────┐
│ Client │                              │ Server │
└───┬────┘                              └───┬────┘
    │                                       │
    │  GET ?offset=X&live=sse               │
    │──────────────────────────────────────▶│
    │                                       │
    │  event: data ...                      │
    │◀──────────────────────────────────────│
    │  event: control {offset: Y}           │
    │◀──────────────────────────────────────│
    │                                       │
    │     ... ~60 seconds of streaming ...  │
    │                                       │
    │  Connection closed                    │
    │◀──────────────────────────────────────│
    │                                       │
    │  GET ?offset=Y&live=sse               │
    │──────────────────────────────────────▶│
    │                                       │

Clients MUST reconnect using the last streamNextOffset received.

JSON Batching

For application/json streams, servers MAY batch multiple messages:

event: data
data: [
data: {"event": "a"},
data: {"event": "b"},
data: ]

event: control
data: {"streamNextOffset": "xyz789"}

Response Codes

Code	Meaning
`200 OK`	Streaming response
`400 Bad Request`	Content type incompatible with SSE
`404 Not Found`	Stream does not exist
`429 Too Many Requests`	Rate limit exceeded

Read Flow Diagram

Typical client read flow:

┌─────────────────────────────────────────────────────────┐
│                      Start Read                         │
└───────────────────────────┬─────────────────────────────┘
                            │
                            ▼
              ┌─────────────────────────────┐
              │  GET ?offset=X (catch-up)   │
              └──────────────┬──────────────┘
                             │
                             ▼
              ┌─────────────────────────────┐
              │   Stream-Up-To-Date?        │
              └──────────────┬──────────────┘
                             │
            ┌────────────────┴────────────────┐
            │ No                              │ Yes
            ▼                                 ▼
┌───────────────────────┐       ┌───────────────────────┐
│  Continue catch-up    │       │  Switch to live mode  │
│  with next offset     │       │  (long-poll or SSE)   │
└───────────────────────┘       └───────────────────────┘

Choosing a Read Mode

Mode	Best For	Trade-offs
Catch-up	Historical replay, initial sync	No live updates
Long-poll	CLI clients, simple implementations	Request per batch of data
SSE	Browser apps, real-time UIs	Requires text/JSON content

Next Steps

Writing Operations - Create and append
Offsets - Offset semantics
Caching - CDN and proxy configuration