Skip to content
All tags

#message-queue

6 posts

Apache Kafka: A Replayable Event Log, Not Merely a Message Queue

Kafka is a distributed log ordered by partition and retained by policy. Consumer groups divide work through offsets, while exactly-once processing holds only inside boundaries covered by Kafka transactions.

Apache Pulsar: An Event Platform Separating Stateless Brokers from BookKeeper Storage

Pulsar separates serving from storage: stateless brokers handle connections while BookKeeper stores ledgers and subscription cursors. Elasticity and multi-tenancy come with more operational components.

AWS SQS and SNS: One Stores Work, the Other Fans Out Events

SQS is a pull-based durable queue and SNS is a push-based topic. Reliable fan-out commonly connects one SNS topic to multiple SQS queues rather than sharing one queue across services.

Cloudflare Queues: Move Work off the Request Path into Retryable Batches

Cloudflare Queues is the message queue beside Workers: producers enqueue slow work, consumers process it with batching, ack/retry, delays, and DLQs. It is good for single-step background jobs; durable multi-step state belongs in Workflows.

RabbitMQ: Express Routing with Exchanges, Preserve Work with Quorum Queues

RabbitMQ's strength is routing through exchanges, bindings, and queues. For replicated work, default to quorum queues and combine publisher confirms, manual acknowledgments, and idempotent consumers.

Redis Streams: An Append-Only Log and Consumer Groups for Reliable Redis Messaging

Redis Streams stores replayable entries with `XADD`; consumer groups add a Pending Entries List and `XACK`. It is more durable than Pub/Sub, but it does not automatically become Kafka.