Will the Guru

When there are Producer, Bounded Queue, and Consumer,

• if arrival rate > retrieval rate, the queue will be full
• if arrival rate < retrieval rate, the queue will be empty

---

Full queue problems and solutions

If queue gets full because the retrieval rate cannot follow up the arrival rate, there are couple ways to resolve it.

#1. Load Shedding

• Drops the message if the load is too high or queue size is over the limit

#2. Rate Limiting

• Producers will get a quota limiting

#3. Backpressure

• By backpressure, it can force the producer to slow down.
• If the broker says it cannot get more messages, what producer can do is
  1. Do nothing
  2. Buffer messages (in memory or on disk)
  3. Propagate the exception further up the stack
  4. Send messages over the limit to a temporary storage (e.g. another broker or system)
  5. (NEVER) Retry immediately

#4. Elastic Scaling

• Scale consumers up

---

Empty queue problems and solutions

#1. Broker sends the message by connection like WebSocket

• Broker pushes message when available. Consumer does not try to pull from the broker

#2. Long Polling

• Consumer consumes the message but broker does not reply if there is no message.
  • Broker blocks the pull request and waits for the message

---

Reference

• https://leetcode.com/explore/interview/card/system-design-for-interviews-and-beyond/726/the-importance-of-queues-in-distributed-systems/4762/