Part 10 of 12 · 1 min read
Quick notes
- A queue decouples accepting work from doing it, absorbs bursts, scales workers independently and gives you one place for retries, limits and priorities.
- Use at-least-once delivery and make handlers idempotent. Acking before the work loses jobs; acking after it duplicates side effects unless the effect is idempotent.
- Leases detect dead workers: an unacked message becomes ready again after the lease expires, so renew the lease for long agent runs.
- Retry with exponential backoff and jitter, and dead-letter poison messages after a few attempts; otherwise they starve every good job behind them.
- Schedule fairly per tenant (round-robin or caps) so one tenant's burst cannot bury another's small jobs, and scale workers on queue age, not just depth.