Latency vs throughput

1Latency is time for one open

Latency is how long one CampusClip open takes, from the student's tap to the redirect reply. It is a duration, so its unit is seconds (or milliseconds). It is not how many students are opening links, and it is not how many redirects the host finishes in a second.

For this topic we give a measured mean latency: W = 0.2 s per redirect on host H when the host is keeping up. That 0.2 s is a classroom measurement we will reuse. It is not a claim about a "typical" shortener.

Figure. Latency W is the elapsed time of one CampusClip open, from the tap on clip.campus/a3k9 to the 302 on the wire. On host H that mean is the classroom measurement 0.2 s. It is a duration, not a finish count.

How to read one latency

1. StartStudent taps clip.campus/a3k9.
2. WorkCampusClip looks up a3k9 and builds the 302.
3. EndThe reply is on the wire. W is that elapsed time, averaged.

2Throughput is finishes per second

Throughput is how many CampusClip redirects finish in a second. It is a rate, so its unit is 1/s. On host H we already called the maximum finish rate capacity (20/s). Observed throughput is how many actually finished — it cannot exceed capacity, and it cannot exceed arrivals for long (you cannot finish work that did not arrive).

At 9:00, arrivals are 10/s and the host can finish 20/s, so observed throughput settles at 10/s — every arrival finishes, and the extra capacity sits as headroom. Throughput is not "the inverse of 0.2 s". That confusion is the next concept.

Figure. Steady state at 9:00: throughput matches arrivals (10/s). Capacity (20/s) is the ceiling, not the observed rate.

How to read throughput

1. Count finishesRedirects that completed in one second.
2. Cap itCannot exceed μ (20/s here) or, in steady state, λ.
3. Keep the unit1/s — not seconds, not students.

3Latency and throughput are not inverses

If CampusClip handled one open at a time, throughput would be 1/W: one 0.2 s redirect would give 5 finishes/s. Host H does not work that way. It can have several opens in flight at once — overlapping lookups — so finishes per second can be larger than 1/W.

So you cannot read 20/s capacity as "each redirect takes 1/20 = 0.05 s", and you cannot read W = 0.2 s as "the host finishes 5/s". The missing piece is how many opens are in the host at once. Little's Law names that piece.

4Little's Law and concurrency

Little's Law relates average in-flight work to arrival rate and latency: L = \lambda W. L is the average number of CampusClip opens on the host (or in the whole path you measured), \lambda is arrivals per second, and W is mean time an open spends in that same path. Units must match: if W is in seconds, \lambda is per second.

On host H at capacity, \lambda = 20/s and W = 0.2 s, so L = 20 \times 0.2 = 4 opens in flight. At 9:00, \lambda = 10/s gives L = 2. Rearranged, \lambda \approx L / W — that is throughput in terms of concurrency and latency, not a new law.

Figure. Little's Law is L = λW on one path. At host H capacity, λ = 20/s and W = 0.2 s, so L = 4 opens in flight. At 9:00, λ = 10/s gives L = 2. Units must match: seconds with per-second.

How to size in-flight work

1. MeasureTake steady-state arrival rate \lambda and mean latency W on the same path.
2. MultiplyL = \lambda W is the average number of concurrent opens.
3. BudgetThread pools, DB connections and queue depth must cover that L with headroom.

5Budget threads from L

Once L is known, it is a budget. If host H has 4 opens in flight on average at 20/s, a thread pool or connection pool sized at 2 will stall — the host is already holding more work than that. A pool sized at 4 is tight (no headroom). A pool sized at 8 covers L and a burst, using a number we chose, not an industry default.

The same identity warns you when latency grows. If W becomes 0.4 s at the same 20/s, L becomes 8. The pool that was "fine" at 4 is now the bottleneck you just created by letting latency double.

Figure. When W rises from 0.2 s to 0.4 s at the same 20/s, L goes from 4 to 8. A pool sized at 4 covered the old L and is now short by 4. Size the pool from the new L, not the old one.

How a pool size is checked

1. Compute LL = \lambda W on the path that uses the pool.
2. ComparePool size must be at least L, plus whatever headroom you choose to state.
3. Recompute when W movesIf latency doubles at the same λ, L doubles — the old pool is now short.