Skip to content
All tags

#distributed-systems

10 posts

Apache Kafka: A Replayable Event Log, Not Merely a Message Queue

Kafka is a distributed log ordered by partition and retained by policy. Consumer groups divide work through offsets, while exactly-once processing holds only inside boundaries covered by Kafka transactions.

Apache Pulsar: An Event Platform Separating Stateless Brokers from BookKeeper Storage

Pulsar separates serving from storage: stateless brokers handle connections while BookKeeper stores ledgers and subscription cursors. Elasticity and multi-tenancy come with more operational components.

Cloudflare Durable Objects: The Stateful Coordination Layer for Workers and WebSockets

Durable Objects map a name or ID to a globally unique, single-threaded actor with private SQLite storage. They fit per-room, per-user, per-tenant, and per-run coordination boundaries; the real design question is where the object key belongs.

gRPC and Connect: One Protobuf Contract across Services and Browsers

gRPC generates cross-language stubs from Protobuf services. A Connect server can support gRPC, gRPC-Web, and the Connect protocol together, avoiding a translating proxy for browsers.

NATS and JetStream: Choose Ephemeral Messaging or Durable Events First

Core NATS is storage-free, at-most-once pub/sub. JetStream adds streams, consumers, acknowledgments, retention, and replay. They share subjects but expose fundamentally different reliability contracts.

Protobuf and Buf: Put Cross-Language Schema Linting, Codegen, and Compatibility in CI

Protobuf defines binary messages with stable field numbers; Buf adds modules, linting, remote plugins, generation, and breaking-change checks to govern those schemas.

RabbitMQ: Express Routing with Exchanges, Preserve Work with Quorum Queues

RabbitMQ's strength is routing through exchanges, bindings, and queues. For replicated work, default to quorum queues and combine publisher confirms, manual acknowledgments, and idempotent consumers.

Self-Hosting Inference with Ray Serve: Python Service Graphs, GPU Scheduling, and Autoscaling

Ray Serve is a distributed serving layer on Ray. Deployments and handles compose Python service graphs, while replicas, CPU/GPU scheduling, autoscaling, and model multiplexing handle orchestration; it complements rather than replaces vLLM or SGLang.

Redpanda: Kafka API Compatibility Still Requires Broker Migration Testing

Redpanda reimplements a Kafka-compatible event log with C++/Seastar, thread-per-core execution, and a Raft group per partition. Client compatibility is broad, but operational and edge semantics still require testing.

Restate: Put Journals, Durable State, and Service Calls in One Execution Model

Restate journals operations and results, then re-executes handlers while skipping completed work. Virtual Objects and Workflows add keyed state, single-writer semantics, and long-lived coordination.