CSE Roadmap

Core tier · about 3 hours · 12 parts

Part 2 of 12 · 2 min read

Two different jobs: accepting work and doing it

A ticket counter for background work, and how acking can lose or duplicate a job.

Think of a ticket counter at a busy clinic. The receptionist takes your name and gives you a number; a different person, a nurse, calls numbers and sees patients. The queue is the stack of numbered tickets between them.

In plain words: a queue holds jobs that have been accepted but not yet done, so a burst of requests does not overwhelm the workers that actually run them.

The key lines. Here is the whole handoff, from code/mini-queue/jobqueue.py:

# from code/mini-queue/jobqueue.py
def receive(self, now, fair=False):
    ...
    x.state, x.attempts, x.lease_until = "leased", x.attempts + 1, now + self.lease
    return x

def ack(self, id, now): self.m[id].state, self.m[id].done_at = "done", now
def nack(self, id, now, delay=0): x = self.m[id]; x.state, x.visible_at = "ready", now + delay

Read it as: a worker calls receive to take one job, does the work, then calls ack on success or nack on failure. This module spends most of its time on one question: when should the worker call ack?

A worked example. 2,000 jobs, each with one side effect (send an email), and a 5 percent chance the worker crashes partway through. If the worker acks before doing the work, a crash after the ack means the job is marked done but the email was never sent: 99 jobs lost. If it acks after, a crash between sending the email and acking means the job is redelivered and the email goes out again: 0 jobs lost, but 65 duplicate emails. Only adding an idempotency check on the email step itself, so a repeat is recognised and skipped, gives 0 lost and 0 duplicates. The queue alone cannot give you both; the effect has to be made safe to repeat.

At-least-once delivery

A guarantee that a message is delivered one or more times, never zero. The queues used in production (SQS, RabbitMQ, Kafka and the like) are built this way, which is why idempotent handlers matter.

Example: acking after the work, as above: a crash can cause a second delivery, but never a lost one.