System Design · System Design
Queues
Work the CampusClip caller is not waiting on — bursts, producer/broker/consumer, back-pressure, and when not to queue.
Some CampusClip work can finish after the student already has a redirect. This topic is that later work: a queue that absorbs a burst, the three roles that move a message, what happens when the pile hits a depth you stated, and the work you must not put on a queue.
- System Design
- Medium level
- 5 concepts
1Synchronous versus asynchronous work
Synchronous work is work the student is waiting on. Opening a3k9 is synchronous: the browser is sitting on the response. CampusClip must look up the code and send the 302 before that wait ends. Asynchronous work is work that can finish after you have already answered. Emailing "your short link is ready" or incrementing a click counter can happen after the redirect is on the wire.
The test is not "is there a queue". The test is whether the caller needs the result in this response. If yes, the work stays on the synchronous path. If no, you may move it off that path — which is what a queue is for.
| Work | Student waiting on it? | Path |
|---|---|---|
| Redirect for a3k9 | Yes | Synchronous |
| Mint a new short code the teacher is staring at | Yes | Synchronous |
| Email "link created" | No | May be asynchronous |
| Increment a click counter | No | May be asynchronous |
A student opens a3k9. Looking up the long URL is
- Synchronous — the redirect cannot be sent without it
- Asynchronous — send an empty reply and look up later
- Neither, because queues do not apply to reads
The student is waiting on the mapping. An empty reply is not a redirect. Queues can hold async work; they do not redefine a lookup the caller needs now.
2A queue absorbs a burst
A queue is a buffer of work that has been accepted but not finished. CampusClip can finish 20 click-counter increments per second. If 40 clicks arrive in one second, a queue can hold the extra 20 so the counter worker drains them in the next second — instead of dropping them or making the student wait on a counter the browser does not need.
The queue does not raise capacity. It moves unfinished work in time. If arrivals stay at 40/s and the worker stays at 20/s, the queue grows without bound. A queue buys you a burst, not a permanently faster worker. That is the same excess arithmetic as load versus capacity, applied to work you chose to do later.
Figure. One second of click events: 40 in, 20 finished. The gap is 20 sitting in the queue — not extra worker capacity.
What the burst does
- Arrive40 click events in one second; worker finishes 20/s.
- Hold20 events sit in the queue.
- DrainThe next second, if arrivals drop, the worker eats the pile. If arrivals stay at 40/s, the pile grows by 20 again.
Queue growth in one second
Click events arrive at 40/s. The counter worker finishes 20/s. How many events wait after one second, starting from an empty queue?
- Arrivals in 1 s40
- Finished in 1 s20
- Queued = 40 - 2020
Pro tip. A second identical second makes the pile 40. The queue only shrinks when finishes exceed arrivals.
A queue in front of a 20/s worker receiving 40/s for many seconds will
- Grow without bound until something drops or the worker speeds up
- Raise the worker to 40/s by existing
- Keep the pile at 20 forever even if 40/s continues
Each second adds 20. The pile at 20 is only the first second. The queue is not extra capacity.
3Producer, broker, consumer
Three roles. The producer puts work on the queue — CampusClip, after it has already sent the redirect, enqueues "increment clicks for a3k9". The broker is the queue itself: it holds the messages in order (or as the product promises) until someone takes them. The consumer is the worker that reads a message and does the increment.
CampusClip the web host and CampusClip the counter worker can be different processes. That is the point: the student's open does not share a thread with the increment. If the worker is down, the broker still holds the messages — until it fills, which is back-pressure.
Figure. Web host produces a click event, the broker holds it, the worker consumes it. The student is already gone.
One click after the redirect
- ProduceCampusClip enqueues increment(a3k9) and finishes the 302.
- HoldThe broker keeps the message.
- ConsumeA worker pops the message and updates the counter.
After sending the 302, CampusClip enqueues a click increment. The process that updates the counter is the
- Consumer (worker)
- Producer (the web host)
- Student browser
The worker consumes the message. The web host produced it. The browser already has its redirect.
4Back-pressure when the queue fills
A queue has a depth you can state. Suppose CampusClip's click queue holds at most 100 messages (a size we choose for the lesson). If the worker is down and producers keep enqueueing, the broker eventually refuses new messages. That refusal is back-pressure: the producer is told "not now" instead of believing the work is safe.
CampusClip the web host must decide what to do with a refused increment: drop the click (the redirect already went out — the student does not see this), or retry, or slow down. What it must not do is treat an unbounded queue as infinite memory. An unbounded pile is the 40-versus-20 growth with no ceiling.
Figure. Queue depth is 100. At arrivals 40/s and a 20/s worker the pile grows at 20/s, so it fills in 100/20 = 5 s. Then the broker refuses the next increment — back-pressure — instead of pretending the work is stored.
How a full queue talks back
- FillWorker is slow or down; producers keep enqueueing.
- RefuseAt depth 100 the broker rejects the next increment.
- DecideWeb host drops, retries, or slows — it does not pretend the increment is stored.
Seconds to a full queue
Queue depth 100. Arrivals 40/s, worker 20/s, start empty. After how many seconds is the queue full?
- Growth per second = 40 - 2020 / s
- Depth / growth = 100 / 205 s
- At t = 5 sbroker starts refusing
Pro tip. Unbounded growth is 20/s forever. A stated depth turns that into a time-to-full you can compute.
A queue of depth 100 grows at 20 messages/s. It fills in
- 5 s
- 100 s
- 20 s
100 / 20 = 5 s. 100 s treats growth as 1/s. 20 s is the growth rate mistaken for a time.
5When not to queue
Do not put the student's redirect behind a queue. The browser is waiting; a broker hop adds latency and a new failure (queue down ⇒ no 302). The lookup stays on the synchronous path: balancer → CampusClip → cache or store → 302.
The same rule applies to minting a short code the teacher is staring at. If the paste response is "here is clip.campus/a3k9", that code must exist before the response. You may queue the "link created" email. You may not queue the insert and then invent a code you have not stored.
| Work | Caller waiting? | Queue? |
|---|---|---|
| Redirect lookup | Yes | No — stay synchronous |
| Insert the new mapping | Yes (teacher wants the code now) | No |
| Email "link created" | No | Yes, if you want |
| Click increment | No | Yes, if you want |
A teacher pastes a URL and the page must show the new short code in this response. The insert
- Stays synchronous — the code is not real until the store has it
- Goes on the click-counter queue so the page can return immediately
- Can be skipped if a cache already has some other code
The caller needs the new mapping in this response. A queue would return a code that might not exist yet. Another code's cache entry is the wrong row.
Notes
- Synchronous work is work this response must carry. Asynchronous work can finish after you have answered.
- A queue absorbs a burst by moving unfinished work in time. It does not raise worker capacity.
- Producer enqueues, broker holds, consumer does the work.
- A stated depth turns unbounded growth into back-pressure (refuse) at a computable time.
- Do not queue a redirect lookup or a short-code insert the caller is staring at.
Formulas
- Queue growth/s = \lambda - \mu when \lambda > \mu.
- Time to full \approx depth / growth, from empty, if growth stays constant.
Exam traps & shortcuts
- If the browser needs the result in this response, a queue is the wrong tool.
- A queue that grows at 20/s and never drains is a delayed drop, not a design.
Reference tables
Click increments may queue. Redirects and new codes may not.
| Idea | CampusClip |
|---|---|
| Sync test | Does this response need the result? |
| Burst | 40 clicks/s into a 20/s worker → 20 queued in the first second |
| Roles | Web host produces; broker holds; worker consumes |
| Back-pressure | Depth 100 at +20/s → refuse after 5 s |
Recap
Later work, a burst, and a hard no.
- Sync
- Caller needs it in this response → no queue.
- Burst
- Queue moves work in time. It does not raise μ.
- Roles
- Producer → broker → consumer. The student is already gone.
- Full
- Depth / growth = time to refuse. Unbounded is not a policy.
- Never
- Do not queue the redirect or the insert the teacher is watching.
Practise Queues
Reading is free and needs no account. Practice, mocks and progress live in the app.
- A 3-question practice set that ends the chapter
- 5 quick checks with worked explanations
- Timed mocks scored with the real marking scheme
- Readiness tracked per topic, kept on your device