Skip to content
All tags

#event-streaming

6 posts

Apache Kafka: A Replayable Event Log, Not Merely a Message Queue

Kafka is a distributed log ordered by partition and retained by policy. Consumer groups divide work through offsets, while exactly-once processing holds only inside boundaries covered by Kafka transactions.

Apache Pulsar: An Event Platform Separating Stateless Brokers from BookKeeper Storage

Pulsar separates serving from storage: stateless brokers handle connections while BookKeeper stores ledgers and subscription cursors. Elasticity and multi-tenancy come with more operational components.

NATS and JetStream: Choose Ephemeral Messaging or Durable Events First

Core NATS is storage-free, at-most-once pub/sub. JetStream adds streams, consumers, acknowledgments, retention, and replay. They share subjects but expose fundamentally different reliability contracts.

RabbitMQ: Express Routing with Exchanges, Preserve Work with Quorum Queues

RabbitMQ's strength is routing through exchanges, bindings, and queues. For replicated work, default to quorum queues and combine publisher confirms, manual acknowledgments, and idempotent consumers.

Redis Streams: An Append-Only Log and Consumer Groups for Reliable Redis Messaging

Redis Streams stores replayable entries with `XADD`; consumer groups add a Pending Entries List and `XACK`. It is more durable than Pub/Sub, but it does not automatically become Kafka.

Redpanda: Kafka API Compatibility Still Requires Broker Migration Testing

Redpanda reimplements a Kafka-compatible event log with C++/Seastar, thread-per-core execution, and a Raft group per partition. Client compatibility is broad, but operational and edge semantics still require testing.