Part 5 of 8 · 3 min read
Two ways to share messages: load balancing and fan-out
Either each message goes to one consumer, or to all of them.
Think of two cases. In a call centre, each call goes to one free agent, so many agents share the calls. In a radio station, every listener hears the same broadcast. Both are valid ways to deliver, for different jobs.
DDIA calls them load balancing and fan-out.
Load balancing (competing consumers)
In load balancing, each message is delivered to one of several consumers, so they share the work. This is also called the competing consumer pattern.
Example: five photo workers read from one queue. Each job is handled by exactly one of them.
Fan-out
In fan-out, each message is delivered to all subscribers, so each one gets its own copy.
Example: when an order is paid, the billing service, the shipping service and the analytics service each receive the event.
The two can be combined. Two groups of consumers can each subscribe to a topic. Each group receives every message, but inside a group, only one member gets each one. A topic is a named stream of messages that many parties can subscribe to.
A worked example. A topic gets 90 messages per minute. Group A has three consumers (billing). Group B has one consumer (analytics).
| Group | Consumers | Messages seen by the group | Per consumer |
|---|---|---|---|
| A | 3 | 90 (all) | about 30 |
| B | 1 | 90 (all) | 90 |
Adding a fourth consumer to group A lowers each one's share to about 22 or 23, and group B is not affected.
Acknowledgments and redelivery. Consumers can crash at any time. The DDIA book explains the fix. A consumer must send an acknowledgment (ack) to tell the broker it has finished a message, and only then does the broker remove the message. If the connection closes or times out with no ack, the broker assumes the message was not processed and delivers it again to another consumer.
The UDS book describes a common form of this. While a consumer works on a message, the message stays in the channel but is hidden from other consumers for a period called the visibility timeout. If the consumer finishes, it deletes the message. If it crashes, the timeout runs out and the message becomes visible again. The book notes that this matches cloud queues such as Amazon SQS.
Visibility timeout
The visibility timeout is the time during which a message that one consumer has taken is hidden from the others. If it is not deleted before the time ends, it reappears for another consumer.
Example: a worker takes a message with a 30-second timeout and crashes after 5 seconds. At second 30 the message shows up again and another worker takes it.
There is a catch. A consumer may finish the work, and then crash before it sends the ack, or the ack may be lost in the network. The broker then delivers the message a second time, and the work is done twice. Page 2 of this level explains how to live with this.
Redelivery also reorders. DDIA shows a case. Consumer 1 gets m4. Consumer 2 gets m3 and crashes. Consumer 1 gets m3 again later. So consumer 1 handles m4, then m3, then m5, which is not the order the producer sent them in. Load balancing plus redelivery makes reordering unavoidable. If the messages are independent, that is fine. If they depend on each other (for example "create account" and then "close account"), one queue per consumer avoids it, at the cost of no load balancing.
Check yourself:
- In fan-out with 4 subscribers and 10 messages, how many deliveries happen in total?
- Why can redelivery change the order in which messages are processed?