System Design · System Design
Growing past one box
How CampusClip grows from one host to many — scale up versus out, a load balancer, CAP under a partition, and availability nines.
CampusClip is a campus URL shortener: a teacher pastes a long LMS or Drive link and gets back clip.campus/a3k9; a student opens that short link and is redirected. This first topic is what happens when that service no longer fits on one machine — grow the machine, or add machines, and what those choices cost. Later topics measure load against capacity, separate latency from throughput, add a cache, add a queue, and then walk the whole CampusClip design.
- System Design
- Hard level
- 5 concepts
- 2 practice questions
1CampusClip, the service we will design
CampusClip is a campus URL shortener. A teacher pastes a long LMS or Drive link; CampusClip stores that long URL and hands back a short campus link such as clip.campus/a3k9. A student taps the short link; CampusClip looks up the code and replies with a redirect to the long URL.
Every later topic uses this same app. We will never switch to a different product. When a number appears, it is a number we state for CampusClip in this course — not a claim about how large shorteners run in industry.
Figure. Teacher writes a mapping; student reads it. CampusClip sits in the middle and owns the short code.
What CampusClip does
- PasteTeacher submits a long URL.
- StoreCampusClip keeps code → long URL.
- RedirectStudent hits the short link and is sent on.
A student opens clip.campus/a3k9. CampusClip must
- Look up a3k9 and redirect to the stored long URL
- Invent a new short code for that student
- Ask the teacher to paste the long URL again
The read path is lookup-then-redirect. Minting a code is the write path. The teacher already stored the mapping.
2Vertical vs horizontal scaling
Suppose CampusClip still runs as one program on one host. Vertical scaling (scale up) means giving that one host more CPU, RAM or disk. It is simple: the code does not have to know about other machines. The cost is a hard ceiling — one box can only grow so far — and a single point of failure: if that host dies, every short link on campus dies with it.
Horizontal scaling (scale out) means running the same CampusClip program on more than one host and splitting student traffic among them. Capacity then grows by count, not by one machine's ceiling. The cost is that the program must tolerate more than one instance, and anything it used to keep only in that process's memory — the code → URL table, a login session — must live somewhere those instances can all see.
Figure. Left: one tall host (scale up). Right: three smaller hosts (scale out) — capacity grows by count.
How the choice lands
- Scale upBigger instance on one host — no distribution logic, until hardware caps out.
- Scale outMore identical hosts; any one can die without taking CampusClip down.
- Cost of outNeeds a load balancer and a store that is not in-process memory.
CampusClip still fits on one host and must double capacity tomorrow, with no code changes that make it multi-instance safe. Which path is available now?
- Vertical — buy a larger machine
- Horizontal — add identical peers behind a balancer
- Either, with identical operational risk
Horizontal needs multi-instance safety the problem forbids. Vertical grows the single host. Risk is not identical: one host remains a single failure point.
3Load balancing
Once CampusClip has more than one host, a student must not have to pick which host to hit. A load balancer sits in front of the pool and owns one address. Every open of clip.campus/a3k9 lands there first; the balancer then chooses a healthy CampusClip host and forwards the request.
Two common policies: round-robin cycles through the pool in order; least-connections sends the next request to the host with the fewest open connections. Health checks probe each host; a host that fails its check is removed from the pool so traffic stops landing on a dead process. The balancer is therefore also where high availability usually starts — but it does not, by itself, make the code multi-instance safe.
Figure. Students hit one balancer; it fans out to healthy CampusClip hosts S1–S3. A failed health check removes a host from the pool.
How a request is placed
- ArriveStudent hits the balancer address, not a specific CampusClip host.
- PickPolicy chooses a healthy backend — round-robin or least-connections.
- Fail outFailed health checks drop a host until it recovers.
Round-robin and least-connections both need a health-check layer because
- otherwise traffic keeps landing on a dead host
- they compute different CAP trade-offs
- they replace the need for horizontal scaling
Policies only choose among hosts still marked healthy. CAP is orthogonal; scaling out still needs the balancer — health checks do not replace either.
4CAP theorem under partition
The moment CampusClip has more than one machine that must agree on the same code → URL table, a network partition can split those machines so they cannot talk. The CAP theorem says that during a partition a distributed store can keep at most two of Consistency, Availability and Partition tolerance. Real networks do partition, so Partition tolerance stays; the live choice is Consistency versus Availability.
A CP CampusClip refuses or errors rather than hand out two different long URLs for the same short code. An AP CampusClip keeps answering on both sides of the split and repairs later (eventual consistency) — a student might briefly follow a stale mapping. SQL stores often lean CP; many NoSQL stores lean AP. That is a design preference, not a law that every SQL or NoSQL product must obey.
How a partition forces the choice
- Partition hitsSome replicas cannot talk; both sides may still receive student traffic.
- CP pathServe only when a quorum can agree — some opens see errors or timeouts.
- AP pathServe on both sides; reconcile after the partition heals.
| Choice | Keeps | Sacrifices | CampusClip fit |
|---|---|---|---|
| CP | Consistency + Partition tolerance | Availability (errors/timeouts) | The short-code table: one code, one URL |
| AP | Availability + Partition tolerance | Strong consistency (eventual) | Click counts, "link created" banners |
CampusClip must never redirect a3k9 to two different long URLs, even during a network partition. Which CAP pair does the mapping store prioritize?
- CP — Consistency and Partition tolerance
- AP — Availability and Partition tolerance
- CA — Consistency and Availability with no partition tolerance
One code, one URL needs strong consistency; partitions happen, so Partition tolerance stays. Availability is what gives. CA is not a live option once partitions exist.
5Availability nines
Availability is the fraction of time CampusClip answers. It is usually quoted as "nines": 99.9% is three nines. That number is a downtime budget, not a vibe. Using 365.25 \times 24 = 8766 hours in a year, three nines allows 0.001 \times 8766 = 8.766 hours down — about 8.76 hours. Four nines (99.99%) is a tenth of that, about 52.6 minutes. Five nines is about 5.3 minutes.
Each extra nine is roughly a 10\times tighter downtime budget. Plan CampusClip maintenance and failover against that number. "Highly available" without a nine-count does not tell you whether a two-hour campus outage is inside the budget.
Figure. A year is 365.25 × 24 = 8766 hours. Three nines (99.9%) allow 0.001 × 8766 = 8.766 hours down. Four nines are about 52.6 minutes; five nines about 5.3 minutes. Each extra nine is roughly a 10× tighter budget. Boxes are equal; downtime is not drawn to scale.
How to turn a percentage into hours
- Year lengthUse 365.25 \times 24 = 8766 hours in a year.
- Downtime fractionDowntime fraction = 1 - availability.
- Hours downMultiply: downtime hours = (1 - A) \times 8766.
Three nines in hours
CampusClip targets 99.9% availability. How many hours of downtime does that allow per year?
- Hours/year = 365.25 \times 248766 h
- Downtime fraction = 1 - 0.9990.001
- Downtime = 0.001 \times 87668.766 h ≈ 8.76 h
Pro tip. Memorize the three-nines peg (≈ 8.76 h/year); each extra nine divides that budget by about ten.
Four nines (99.99%) allows roughly how much downtime per year?
- ≈ 52.6 minutes
- ≈ 8.76 hours
- ≈ 87.6 hours
0.0001 \times 8766 ≈ 0.877 h ≈ 52.6 min. 8.76 h is three nines; 87.6 h is two nines.
Notes
- Vertical scaling adds power to one machine (simple, has a ceiling); horizontal scaling adds more machines (elastic, needs load balancing).
- The CAP theorem states a distributed store can guarantee only two of Consistency, Availability, and Partition tolerance during a network partition.
- Load balancers distribute traffic across servers (round-robin, least-connections) and provide health checks and high availability.
- Caching (client, CDN, Redis/Memcached) reduces latency and database load; cache strategies include write-through, write-back, and cache-aside.
- Database scaling uses replication (read replicas) for read-heavy loads and sharding (horizontal partitioning) to spread writes across nodes.
Formulas
- CAP: during a partition you must choose Consistency or Availability (CP vs AP systems).
- Little's Law: L = \lambda W (avg items in system = arrival rate x avg time in system).
- Availability with N nines: 99.9% (three nines) \approx 8.76 hours downtime per year.
- Throughput (QPS) \approx concurrency / average request latency.
- Consistent hashing minimizes key remapping when nodes are added/removed (only 1/N keys move on average).
Exam traps & shortcuts
- SQL (RDBMS) favours consistency/ACID; many NoSQL stores favour availability/partition tolerance (AP) with eventual consistency.
- Read-heavy -> add caching + read replicas; write-heavy -> shard the database.
- Use consistent hashing to distribute keys so adding a node reshuffles only a small fraction of data.
Reference tables
Every line is a rule from the concepts above — name which rule applies before you pick a path.
| Idea | Rule | Watch for |
|---|---|---|
| Scale | Up = one bigger box; out = more boxes behind a balancer | Out needs multi-instance-safe code |
| CAP under partition | Choose CP or AP (partitions happen) | CA is not a live option |
| Three nines | 99.9% ≈ 8.76 h down / year | Each extra nine ÷ ≈ 10 |
Recap
What "more than one box" costs, before we measure load.
- CampusClip
- One app for the course: paste a long URL, store a short code, redirect.
- Scale
- Up = one bigger box (ceiling, SPOF); out = more boxes behind a balancer.
- Balancer
- Round-robin or least-connections, plus health checks that pull dead hosts.
- CAP
- Under partition pick CP or AP. The short-code table → CP; click counts can be AP.
- Nines
- Three nines ≈ 8.76 h/year; each extra nine divides that budget by about ten.
Practise Growing past one box
Reading is free and needs no account. Practice, mocks and progress live in the app.
- 2 exam-style questions on this topic, with explanations
- A 3-question practice set that ends the chapter
- Timed mocks scored with the real marking scheme
- Readiness tracked per topic, kept on your device