Event-Driven Microservices: When to Use Kafka vs RabbitMQ (And When to Use Both)

Teams migrating to microservices often treat Kafka and RabbitMQ as interchangeable message brokers — pick one, configure some queues, done. This leads to using Kafka for task distribution (where RabbitMQ excels) or RabbitMQ for event streaming (where Kafka excels). Both choices create problems that appear gradually and are expensive to fix.

They are not competing tools. They solve fundamentally different problems, and in complex systems you often need both.

Kafka is a distributed, append-only log. Producers write events to topics. Consumers read from those topics at their own pace — independently, in parallel, and as many times as they want.

This means: - Events are retained for a configured period (hours, days, weeks), not just until consumed. - Multiple independent consumer groups can read the same events for different purposes. - Consumers can replay events from any offset — useful for rebuilding state, reprocessing, or debugging. - Ordering is guaranteed within a partition (useful when you partition by entity ID — e.g. user ID or transaction ID).

Kafka is the right choice when you need a durable, replayable, high-throughput event backbone. At Dashen Super App, every financial event — payment initiated, wallet topped up, lottery ticket purchased — is published to Kafka. The analytics service, the notification service, and the audit service all consume the same events independently.

RabbitMQ is a message broker with sophisticated routing. Producers publish messages to exchanges. Exchanges route messages to queues based on routing keys and bindings. Consumers pull from queues and acknowledge each message individually.

The key difference: once a message is acknowledged, it is gone. RabbitMQ is designed for task distribution, not event streaming.

This gives you: - Flexible routing: direct, fanout, topic, headers exchanges for different delivery patterns. - Dead Letter Queues (DLQ) for failed messages — critical for reliable task execution. - Per-message TTL and queue-level TTL. - Priority queues. - Consumer acknowledgement and requeue on failure.

At Dashen Super App, RabbitMQ handles tasks that need exactly-once processing with guaranteed delivery: sending SMS notifications, calling third-party banking APIs, processing scheduled reports. If the SMS service crashes mid-processing, RabbitMQ requeues the message and another consumer picks it up. With Kafka, implementing the same exactly-once guarantee requires significantly more application-level code.

Use Kafka when: - You need multiple independent consumers reading the same events (fan-out to many services) - Events need to be replayed or reprocessed (analytics, audit, event sourcing) - You need high throughput (hundreds of thousands of events per second) - Order within a partition matters - You need event retention for downstream consumers that come later

Use RabbitMQ when: - You need a task queue with exactly-once semantics - Different consumers should each handle different messages (work distribution, not fan-out) - You need sophisticated routing (route by message type, destination, content) - Failed tasks need automatic requeue and dead-letter handling - Per-message TTL or priority matters

The pattern we use at Dashen: Kafka as the event backbone, RabbitMQ for task execution.

A payment event flows through Kafka to multiple consumers. The notification service reads the event and enqueues a task to RabbitMQ's SMS queue. The SMS worker pulls from the queue, sends the SMS, and acknowledges. If the SMS API is down, the message stays in RabbitMQ until the worker can process it.

Neither broker handles both roles well. Using both — deliberately, for the jobs each is designed for — gives you a more reliable and simpler system than trying to make one tool do everything.