Introduction
Modern applications increasingly rely on ordered, durable sequences of data that can be replayed from arbitrary points and tailed in real time. Yet the web platform lacks a simple, first-class primitive for this fundamental pattern.
The Problem
Consider these common scenarios:
- AI conversation streaming: An LLM generates tokens that need to be delivered reliably to clients, with the ability to resume if connections drop mid-response
- Database synchronization: Changes from a database need to flow to clients in order, with clients able to catch up after being offline
- Collaborative editing: Operations from multiple users need to be ordered and delivered reliably to all participants
- Event sourcing: Application state changes need to be durably recorded and replayable
Each of these requires the same core capability: an append-only stream of data that clients can read from any point, catch up on missed data, and tail for new updates.
Today, developers implement this pattern using ad-hoc combinations of:
- Databases with polling
- Message queues with complex consumer tracking
- Custom WebSocket protocols with reconnection logic
- Proprietary streaming APIs
Each implementation reinvents similar offset-based replay semantics, retry logic, and catch-up mechanisms.
The Solution
The Unbroken Protocol provides a minimal, HTTP-based interface for durable, append-only byte streams. It offers:
| Capability | Description |
|---|---|
| Durability | Data persists until explicitly deleted or expired |
| Ordering | Bytes are strictly ordered by offset |
| Resumability | Clients can resume from any offset after disconnection |
| Live Tailing | Real-time updates via long-polling or Server-Sent Events |
| HTTP Native | Works with existing infrastructure (CDNs, proxies, load balancers) |
Why HTTP?
Unlike WebSocket-based protocols, Unbroken Protocol uses standard HTTP semantics:
- CDN Compatible: Catch-up reads are cacheable; multiple clients can share cached responses
- Proxy Friendly: Works through corporate proxies and firewalls without special configuration
- Load Balancer Ready: Stateless design allows requests to hit any backend server
- Observable: Standard HTTP logging and monitoring tools work out of the box
- Simple: No connection upgrade negotiation or binary framing to implement
Comparison with Alternatives
| Feature | Unbroken Protocol | WebSocket | Kafka | Redis Streams |
|---|---|---|---|---|
| HTTP Native | Yes | No | No | No |
| CDN Cacheable | Yes | No | N/A | N/A |
| Browser Native | Yes | Yes | No | No |
| Resumable | Yes | Manual | Yes | Yes |
| Durable | Yes | No | Yes | Optional |
| Simple Setup | Yes | Yes | No | Medium |
Design Principles
The protocol follows these guiding principles:
- Minimal Surface Area: Only the essential operations for durable streaming
- Byte-Level Abstraction: Message framing and schemas are application concerns
- HTTP Semantics: Uses standard methods, headers, and status codes
- Opaque Offsets: Implementation freedom for storage backends
- Progressive Enhancement: Start with simple polling, add SSE when needed
Use Cases
Unbroken Protocol is designed for:
- AI/LLM Applications: Token streaming with reliable delivery and conversation history
- Real-time Sync: Database change feeds and state synchronization
- Event Logs: Audit trails, activity feeds, and event sourcing
- Collaboration: Multi-user editing with operation ordering
- IoT/Telemetry: Sensor data collection with resumable uploads
Example Implementations
The repository includes complete example protocols built on Unbroken Protocol:
| Example | Description |
|---|---|
| File Streaming | Resumable file uploads/downloads with real-time watching |
| Terminal Streaming | Shared terminal sessions with persistent output |
| Agent Streaming | AI agent sessions with tool use and message persistence |
These examples demonstrate how to build domain-specific protocols while inheriting Unbroken Protocol’ resumability and persistence guarantees.
Inspirations & Acknowledgments
Unbroken Protocol builds on ideas from several excellent projects:
Agent Client Protocol
The agent streaming example protocol draws inspiration from the Agent Client Protocol (ACP)—a specification for AI agent communication with support for tool use and structured message passing.
- Website: agentclientprotocol.com
- Repository: github.com/agentclientprotocol/agent-client-protocol
- License: Apache 2.0
We’re grateful to these projects for their pioneering work in streaming protocols and agent communication patterns.
Getting Started
Ready to dive in? Start with Core Concepts to understand the protocol’s building blocks, or jump to the Quick Start Guide for hands-on examples.