Retry Backoff Needs Full Jitter

Day 30 made retries safe (idempotency). Day 35 taught when to stop calling (circuit breaker). Day 36 said deadlock victims should retry. Today: how to wait between attempts so a fleet of clients does not create a second outage.

A transient failure invites a retry. Immediate retries from thousands of clients look like a DDoS on yourself. Exponential backoff spaces attempts. Without jitter, those attempts still line up like soldiers - the famous “thundering herd” after a blip. Full jitter randomizes the wait so the herd dissolves.

The problem: synchronized retries

Fifty clients hit a dependency. It fails once (deploy, GC, brief overload). Every client retries at T+0, then T+1s, then T+2s… The dependency never gets a quiet window. Rate limits (Day 9) trip. Queues (Day 14) fill. Your “resilience” code is the attack.

// Naive — everyone wakes together
async function call() {
  for (let i = 0; i < 5; i++) {
    try { return await dep(); }
    catch { await sleep(1000); } // fixed 1s — still synchronized
  }
  throw new Error("give up");
}

Worked example: multi-client storm after a blip

Walk the lab model as a story. 50 clients, each needs one success. Time is discrete slots. Every slot, all clients that are “ready” attempt; the server accepts exactly one success per slot (contention shape inspired by Brooker’s OCC-style graphs). Losers schedule a retry under one of three policies.

1. Immediate (none): every loser retries next slot. Slot 0: 50 attempt, 1 wins. Slot 1: 49 attempt, 1 wins… You finish in 50 slots, but total attempts sum to 1275 (50+49+…+1). The dependency is under constant full load until the last client lands.
2. Capped exponential, no jitter: losers sleep min(32, 1 × 2^(failStreak-1)) slots. The herd pauses together, then reappears as a spike. Median attempts stay 1275 - same work - but makespan stretches to 1489 slots because aligned waits burn calendar without cutting contention.
3. Full jitter: same exponential ceiling, but each loser sleeps U(0, exp). Clients desynchronize. Median attempts fall to 303 (~4.21× less work); makespan is only 81 slots - recovery without the square wave.

Product claim: full jitter cuts client work under multi-client contention. Exponential alone only bought delay, not fewer attempts.

Exponential backoff (necessary, not sufficient)

// wait = min(cap, base * 2^attempt)
const base = 100; // ms
const cap = 3200;
const wait = Math.min(cap, base * 2 ** attempt);

Spacing grows: 100, 200, 400, 800… That helps a single client. Under multi-client contention, clients that failed on the same attempt still share the same wait - they re-cluster on the next spike. Brooker’s graphs show work remaining high until you add randomness.

Full jitter: sleep in [0, exp]

// Full jitter (AWS blog)
const exp = Math.min(cap, base * 2 ** attempt);
const wait = Math.floor(Math.random() * (exp + 1));

Full jitter picks uniformly between 0 and the exponential ceiling. Clients desynchronize. Some retry sooner (good for recovery), some later (good for load). Equal jitter (midpoint ± random) is a cousin; full jitter is the aggressive desync option the AWS post recommends when you care about total client work.

Policy	Sleep formula	What multi-client herds do
None / fixed	0 or constant	Everyone re-hits together every slot
Exp, no jitter	min(cap, base×2^n)	Spikes space out, then re-sync
Equal jitter	exp/2 + U(0, exp/2)	Partial desync; milder than full
Full jitter	U(0, exp)	Herd dissolves; total work drops

When not to retry

Backoff only helps if another attempt can succeed and is worth the cost. Classify first; schedule second.

Signal	Retry?	Why
Timeouts, 408, network reset	Yes (transient)	May succeed next; use full jitter + cap
429 with Retry-After	Yes, honor hint	Server asks you to wait; jitter around the hint
5xx during deploy / overload	Yes, budgeted	Transient; compose with Day 35 breaker
400 validation, 401/403, 404	No	Permanent for that body / credentials; fix the request
409 same key, different body (Day 30)	No	Client bug, not a blip - do not loop
Open circuit (Day 35)	No hammer	Fail fast / fallback; probe only on half-open policy
User path with 200 ms SLO, sleep would be seconds	No long loop	No budget; queue async or return degraded
Write without Day 30 key	Not until keyed	Retries can double-charge; safety first

• Non-transient: 400 validation, 401/403 auth, 404 not found - fix the request, do not loop.
• No budget: user-facing request with 200 ms SLO cannot sleep 3 s × 5.
• Unsafe side effects: without Day 30 idempotency keys, retries can double-charge.
• Open circuit: Day 35 OPEN means fail fast / fallback - not hammer the dependency harder.

Compose the stack (Days 9 / 30 / 35 / 36)

Retries are one layer in a small stack. Order of thinking in interviews:

1. Day 30 idempotency - make the attempt safe to replay (keys on writes).
2. Classify the error - only transient classes enter the retry loop.
3. Today’s policy - capped exponential + full jitter + max attempts.
4. Day 35 circuit breaker - if the dep is dead, stop calling; do not burn retry budget into a black hole.
5. Day 9 rate limit - synchronized retries trip the limiter you built to protect capacity; jitter keeps you under the ceiling.
6. Day 36 deadlock victims - 40P01 is retriable; use the same full-jitter policy so victims do not re-lock in lockstep.

// Sketch: safe + kind + bounded
async function callWithPolicy(op, key) {
  for (let attempt = 0; attempt < 5; attempt++) {
    if (breaker.isOpen()) return failFast(); // Day 35
    try {
      return await op({ idempotencyKey: key }); // Day 30
    } catch (e) {
      if (!isTransient(e)) throw e; // 4xx permanent → stop
      await sleep(fullJitter(attempt)); // today
    }
  }
  throw new Error("retries exhausted");
}

Benchmark — multi-client contention (simulation)

50 clients each need one success. Each time slot, concurrent attempts contend; only one succeeds (OCC-style single winner - same shape as Brooker’s client-work graphs). Policies: immediate retry · capped exponential without jitter · full jitter. Base=1, cap=32. 30 runs, median attempts (work) and makespan (slots).

Policy	Median attempts (work)	Median makespan (slots)
Immediate retry (none)	1275	50
Exponential, no jitter	1275	1489
Full jitter	303	81

Headline: full jitter cuts client work by ~4.21× vs immediate and vs pure exponential (1275 → 303). Exponential alone only stretched the timeline (makespan 50 → 1489) - it did not reduce attempts, because losers stayed synchronized. That is the interview point: backoff without jitter is still a herd.

What breaks? — Anti-patterns

1. Retry storms after deploy blips. Fixed sleeps across a fleet. Fix: full (or equal) jitter + cap + max attempts.

2. Retrying 400s. Burns budget and logs noise. Fix: classify transient (408/429/5xx/network) vs permanent.

3. Retries without idempotency. Day 30: side effects multiply. Fix: key the mutation; retries return the first result.

4. Retries while the breaker is open. Fights Day 35. Fix: short-circuit; probe only on half-open policy.

5. Unbounded max attempts. A poisoned client loops forever. Fix: hard attempt budget + DLQ / user error path.

How it connects

• Day 9 (rate limit): synchronized retries trip the limiter you built to protect capacity; jitter keeps the fleet under the ceiling.
• Day 14 (queues): consumers retry poison messages carefully - backoff + DLQ, not hot loops.
• Day 30 (idempotency): makes retries correct; today makes them kind to the system.
• Day 35 (circuit breaker): stop calling when dead; today: space the calls when recovering. Half-open must not become a herd of probes.
• Day 36 (deadlocks): victim retries need the same policy - not instant re-lock storms after 40P01.

Transfer questions

What you should be able to do

• Explain retry storms as synchronized client load, not “the server is slow.”
• Write capped exponential delay: min(cap, base × 2^attempt).
• Apply full jitter: uniform in [0, exp].
• Name why exp-without-jitter may not cut total attempts under multi-client contention.
• Walk the lab: none/exp 1275 attempts vs full jitter 303 (~4.21×); makespans 50 / 1489 / 81.
• Compose retries with Day 9 rate limits, Day 30 keys, Day 35 breaker, and Day 36 deadlock victims.
• Refuse permanent errors and open-circuit hammers; only budget transient classes.

Quiz

Your teach step

Close this lesson. Write the “Explain like I’m 10” and the “60-second LinkedIn version” from memory. Focus on: storm vs jitter, why exp alone is not enough, the ~4.2× lab, and composition with idempotency + breaker. Post it, and paste the link.

Questions? Ask the agent - equal vs full jitter, token-bucket retry budgets, or adaptive retry modes in AWS SDKs.