System Design
B5

CDN and edge: shared caches near the user

A CDN keeps copies of your responses in many locations near users. A hit never reaches your servers. This sheet shows what to cache, how long, how to remove it, and what fails.

Not startedSaved in this browser only.
  1. 1A CDN is a shared cache near the user. The origin sets its rules with Cache-Control.
  2. 2Hit ratio is the number to watch. The origin sees (1 - hit ratio) of requests.
  3. 3Put a content hash in static file names, so a deploy needs no purge.
  4. 4Normalise cache keys, keep a shield, and never cache a per-user response as public.
B5
    A

    What a CDN does

    hit at the edge, miss to the origin
    Users, MumbaiEdgehit: answer hereUsers, FrankfurtEdgehit: answer hereUsers, VirginiaEdgehit: answer hereUsers, SydneyEdgehit: answer heremissOrigin shieldone regionalcachemissOriginyour service,sets Cache-Control
    • A user reaches the nearest edge through DNS or anycast routing.
    • The edge answers from its cache, or fetches from the origin and stores the copy.
    • The origin decides what is cached, and for how long, with response headers.

    Users reach the nearest edge. Hits stop there. Misses go to one shield, and only shield misses reach my origin.

    B

    What to cache, and what not

    decide per response
    responsestatus
    Static files with a hash in the name1 year
    Images, video segments, downloadsApproved
    Public HTML pagesShort s-maxage
    Public API GETs: prices, scoresSeconds, no cookies
    Pages that show the user's dataNot approved
    Responses that set a cookieNot approved
    POST, PUT, DELETENot approved
    Data that must be current: stock, balancesNot approved

    A CDN adds little when most responses are personal, traffic is low, or users sit next to the origin.

    I cache public bytes that many users share. A per-user response never goes in a shared cache.

    C

    Cache-Control and revalidation

    the origin sets the rules
    Cache-Control: public, s-maxage=60, stale-while-revalidate=30age of the copy at the edgefreshserve from the edgestaleserve, refreshtoo oldask first0 s60 s90 srevalidation: the edge asks if its copy is still currentEdgeOriginGET, If-None-Match: "v7"304 Not Modified, no bodyA match costs a round trip but no body. The edge resets the age to 0.A changed object returns 200 with the new body and a new ETag.
    directivewho obeyseffect
    max-age=Nall cachesFresh for N seconds.
    s-maxage=Nshared cachesOverrides max-age at the CDN.
    publicall cachesShared caches may store it.
    privatebrowser onlyThe CDN must not store it.
    no-cacheall cachesStore it, but revalidate before each use.
    no-storeall cachesNever store it.
    immutablebrowsersDo not revalidate while fresh.
    stale-while-revalidate=Ncaches that support itServe stale for N seconds while refreshing.
    stale-if-error=Ncaches that support itServe stale for N seconds if the origin fails.
    ETag, If-None-Matchorigin304 Not Modified when the copy still matches.
    Vary: Accept-Encodingall cachesKeep one copy per value of that header.

    HTTP caching is defined in RFC 9111. stale-while-revalidate and stale-if-error are defined in RFC 5861.

    I set s-maxage for the CDN and a shorter max-age for browsers. stale-while-revalidate hides the refresh, and ETags make it cheap.

    D

    How an edge decides

    pseudo code
    per request, at the edgepseudo code
    edge(request):
      copy = cache[key(request)]
      IF no copy:
        fetch from the origin
        store it, unless no-store OR private1
      lifetime = s-maxage, ELSE max-age2
      IF age < lifetime: RETURN copy            // a hit
      IF age < lifetime + stale-while-revalidate:
        RETURN copy; revalidate in the background3
      ask the origin: If-None-Match: copy.etag4
        304: keep the copy, age = 0             // no body sent
        200: store the new copy
        error AND age < lifetime + stale-if-error5:
          RETURN copy
    1. 1A shared cache never stores these. private keeps per-user replies out of the CDN.
    2. 2s-maxage wins at a shared cache, so the CDN can keep a copy longer than browsers do.
    3. 3The user gets the stale copy at once. One background request refreshes it.
    4. 4A conditional request. A match returns 304 with no body.
    5. 5If the origin fails, serve the stale copy for a while instead of an error.
    Tested source Go: the freshness decision and the 304 check
    Go: the freshness decision and the 304 checkgo
    
    // Decide applies the shared-cache rules: private and no-store are never stored, s-maxage wins over
    // max-age, no-cache means revalidate every time, and stale-while-revalidate lets the edge answer
    // from a stale copy for a while. originDown asks what happens when the revalidation fails.
    func Decide(d Directives, age time.Duration, originDown bool) Decision {
      if d.NoStore || d.Private {
        return DoNotStore
      }
      lifetime := d.MaxAge
      if d.SMaxAge >= 0 {
        lifetime = d.SMaxAge // the shared-cache lifetime
      }
      a := int(age.Seconds())
      switch {
      case !d.NoCache && lifetime >= 0 && a < lifetime:
        return ServeFresh
      case originDown && !d.MustRevalidate && d.StaleIfError >= 0 && a < max(lifetime, 0)+d.StaleIfError:
        return StaleIfOriginFails
      case !d.NoCache && !d.MustRevalidate && d.StaleWhileRevalidate >= 0 && a < max(lifetime, 0)+d.StaleWhileRevalidate:
        return ServeStaleRefresh
      }
      return RevalidateFirst
    }
    
    // Revalidate answers a conditional request: 304 Not Modified when the client's ETag still
    // matches, so the body is not sent again; 200 with the body otherwise.
    func Revalidate(currentETag, ifNoneMatch string) int {
      for tag := range strings.SplitSeq(ifNoneMatch, ",") {
        tag = strings.TrimSpace(tag)
        if tag == "*" || strings.TrimPrefix(tag, "W/") == strings.TrimPrefix(currentETag, "W/") {
          return 304
        }
      }
      return 200
    }
    

    Fresh copies are hits. Stale copies inside the window are served while the edge refreshes. Older copies are revalidated, and a 304 costs no body.

    E

    Cache keys

    one object, one key
    normalise the keypseudo code
    key(request):
      host   = lowercase(host)
      params = the allowed ones only, sorted1    // drop utm_*
      vary   = values of the Vary headers2       // Accept-Encoding
      RETURN host + path + params + vary
    1. 1Keep only parameters the response uses. Sorting makes ?a=1&b=2 and ?b=2&a=1 one key.
    2. 2Vary splits the copy per header value. Vary on User-Agent creates thousands of copies.
    Tested source Go: the cache key
    Go: the cache keygo
    
    // Key returns the normalised cache key of a request.
    func (p KeyPolicy) Key(rawURL string, h http.Header) (string, error) {
      u, err := url.Parse(rawURL)
      if err != nil {
        return "", fmt.Errorf("cache key: %w", err)
      }
      var b strings.Builder
      b.WriteString(strings.ToLower(u.Host)) // host names are case-insensitive
      b.WriteString(u.EscapedPath())
      q := u.Query()
      kept := slices.Sorted(func(yield func(string) bool) {
        for name := range q {
          if slices.Contains(p.Query, name) && !yield(name) { // drop utm_*, session ids, ...
            return
          }
        }
      })
      for i, name := range kept {
        vals := slices.Sorted(slices.Values(q[name]))
        sep := "&"
        if i == 0 {
          sep = "?"
        }
        b.WriteString(sep)
        b.WriteString(url.QueryEscape(name) + "=" + url.QueryEscape(strings.Join(vals, ",")))
      }
      for _, name := range slices.Sorted(slices.Values(p.Vary)) {
        b.WriteString("|" + strings.ToLower(name) + "=" + strings.ToLower(strings.TrimSpace(h.Get(name))))
      }
      return b.String(), nil
    }
    
    edge hit ratio0%50%100%94%129/s78%2111/s60%4198/s46%8270/s35%16327/s25%32374/skeys per object · origin requests a second

    Recorded from the simulation: TTL 10 min, edge holds every object once, then 1 to 32 keys per object.

    The cache key holds only what changes the response. Extra parameters and headers split one object into many copies.

    F

    Pull, push and the origin shield

    how copies arrive
    methodhowstatus
    PullThe first request at an edge fetches from the origin.Default
    PushYou upload files to the CDN before anyone asks.Large, planned releases
    Pull with a shieldEdges fill from one regional cache, which fills from the origin.Many edges
    Pull, no shield, many edgesEach edge fetches each object on its own.Small origin load
    no shield
    196.2 origin requests a second
    shield
    8.1 origin requests a second
    setting
    TTL 10 min, each edge holds 25% of objects, 4 edges, recorded

    The shield holds what 4 edges hold together. It costs one extra hop on an edge miss, and the edge copy keeps the shield copy's age.

    I use pull with a shield. In the simulation the shield cut origin load from 196.2 to 8.1 requests a second.

    G

    Purge and invalidation

    remove copies before the TTL ends
    origin requests a second, 10 s buckets0200400purge all-2-1012345678minutes from the purgeno shieldwith an origin shield
    methodstatus
    New file name per versionNo purge at all
    Purge one URLApproved
    Purge by tag: every page that shows product 42Tag header set
    Soft purge: mark stale, revalidate on useHot objects
    Purge everythingNot at peak

    I version file names so most deploys need no purge. For the rest I purge by URL or tag, never everything at peak.

    H

    Signed URLs

    access control at the edge
    sign and verifypseudo code
    sign(path, expires):                 // at the origin
      sig = HMAC_SHA256(secret, path + expires1)
      RETURN path + "?exp=" + expires + "&sig=" + sig
    
    verify(url, now):                     // at the edge2
      IF sig != HMAC_SHA256(secret, path + exp):
        RETURN 403                        // constant-time compare3
      IF now >= exp4: RETURN 403
      serve the object
    1. 1The signature covers the path and the expiry. Changing either breaks it.
    2. 2The edge holds the secret, so it checks access without the origin.
    3. 3A compare that stops at the first difference leaks timing to an attacker.
    4. 4A shared link stops working after its expiry.
    Tested source Go: sign and verify
    Go: sign and verifygo
    
    // Sign returns path with an expiry time and an HMAC-SHA256 signature over both. The origin and
    // the edge share the key; the client never sees it.
    func Sign(key []byte, path string, expires time.Time) string {
      exp := strconv.FormatInt(expires.Unix(), 10)
      return path + "?exp=" + exp + "&sig=" + mac(key, path, exp)
    }
    
    // Verify checks a signed URL at the edge, with no call to the origin.
    func Verify(key []byte, signed string, now time.Time) error {
      u, err := url.Parse(signed)
      if err != nil {
        return fmt.Errorf("verify signed url: %w", err)
      }
      exp, sig := u.Query().Get("exp"), u.Query().Get("sig")
      want := mac(key, u.Path, exp)
      if !hmac.Equal([]byte(sig), []byte(want)) { // constant time
        return ErrBadSignature
      }
      secs, err := strconv.ParseInt(exp, 10, 64)
      if err != nil {
        return fmt.Errorf("verify signed url: expiry %q: %w", exp, err)
      }
      if !now.Before(time.Unix(secs, 0)) {
        return ErrExpired
      }
      return nil
    }
    
    func mac(key []byte, path, exp string) string {
      m := hmac.New(sha256.New, key)
      m.Write([]byte(path + "\n" + exp))
      return hex.EncodeToString(m.Sum(nil))
    }
    
    • Use it for paid video, private downloads and user uploads.
    • Leave the signature out of the cache key, or every user gets a separate copy.

    The origin signs the path and an expiry time with a secret the edge shares. The edge checks it with no call to the origin.

    I

    Edge compute

    small code near the user
    task at the edgestatus
    Redirects and URL rewritesApproved
    Normalise the cache key, strip headersApproved
    Check a signed URL or a JWTApproved
    A/B bucket from a cookie, then serve a cached variantApproved
    Resize images on demand, then cache themApproved
    Insert per-user parts into a cached pageSmall parts only
    Read and write the main databaseNot approved
    Long or CPU-heavy workNot approved

    Edge runtimes limit CPU time and memory per request. A database call from the edge crosses the network back to the origin region.

    I run small, stateless logic at the edge: redirects, header rules, token checks. Anything that needs the database stays at the origin.

    J

    Capabilities used

    what each tool gives you
    toolcapabilitywhat it gives this designalso used for
    CDNEdge caches in many locationsA hit is served near the user and never reaches the origin.DDoS absorption, TLS at the edge
    CDNAnycast or DNS routingUsers reach the nearest healthy edge.Failover between regions
    CDNOrigin shield (tiered cache)Edges fill from one cache, so the origin sees one fetch per object.Large video libraries
    CDNPurge by URL, by tag, or allRemove copies before their TTL ends.Takedowns, price changes
    CDNCache key rulesIgnore parameters and headers the response does not use.Device variants
    CDNSigned URLs or tokensThe edge checks access with a shared secret.Private downloads
    CDNEdge functionsRedirects, header rules and token checks near the user.A/B tests, image resizing
    HTTPCache-Control: max-age, s-maxage, private, no-storeThe origin sets separate rules for browsers and for the CDN.Every cacheable API
    HTTPstale-while-revalidate, stale-if-errorUsers never wait for a refresh, and an origin failure serves stale copies.Status pages
    HTTPETag and If-None-MatchRevalidation returns 304 and no body when nothing changed.API polling, sync clients
    OriginContent-hashed file namesA new build gets new URLs, so no purge is needed.Mobile app assets
    OriginA tag header on each responseOne purge call removes every page that shows a changed object.News, catalogues

    The CDN gives me shared caches near users, purge and signed URLs. HTTP headers carry the rules, so the origin stays in control.

    K

    Try it: TTL and cache size

    recorded from a seeded simulation
    TTL
    edge holds
    shield
    0%25%50%75%100%edge hit ratio10 s1 min10 min1 h60.8%1%5%10%25%50%100%share of objects one edge can hold
    edge hit ratio60.8%served without the origin
    origin load196.2/sof 500 requests a second
    origin requests a secondno shield196.2shield8.1

    5,000 objects, Zipf popularity with alpha 0.8, 500 requests a second spread over 4 edges. Each edge is an LRU cache. Results cover 30 minutes after a 1 hour warm-up.

    A short TTL caps the hit ratio whatever the cache size: 44% at 10 seconds. A long TTL needs a cache big enough to keep the objects.

    L

    Failure cases

    what breaks, and the fix
    eventresultfixfixed by
    Purge everything at peakEvery edge misses at once. In the simulation, origin load went from 5 to 318 requests a second.Purge by URL or tag. Soft purge. A shield cut the first 10 s to 213 a second.Shield
    Cache key explosionTracking parameters or Vary on User-Agent split objects. The hit ratio fell from 94% to 25% with 32 keys per object.Allow-list the parameters in the key. Vary only on Accept-Encoding.Key rules
    Private data cachedA per-user page marked public, or one that sets a cookie, is served to other users.Send private or no-store on per-user responses. Never cache a response that sets a cookie.Origin headers
    Cache poisoningA header that is not in the key changes the body. An attacker's version is cached for everyone.Key on every input the response uses, or strip that header at the edge.Edge rules
    Old assets after a deployNew HTML loads an old script from the cache, and the page breaks.Content-hashed file names. A short TTL on the HTML.Build
    The origin is downMisses and revalidations fail.stale-if-error serves the last copy for a while.HTTP
    One edge location failsIts users move to the next location, which has a colder cache.Anycast or DNS failover. The shield absorbs the extra misses.CDN
    M

    Scale ladder

    start simple; climb only on a signal
    Each step adds one component1Origin, headers2+ CDN, pull3+ hashed names4+ shield5+ edge codemore load →
    Requests that reach the origin1101001kNo CDN: 500 origin requests a secondNo CDN500Edge holds 25%, 10 min: 196.2 origin requests a secondEdge holds 25%, 10 min196.2Edge holds all, 1 h: 7.8 origin requests a secondEdge holds all, 1 h7.8+ origin shield: 2 origin requests a second+ origin shield2origin requests a second, log scale
    stepaddit handlesmove up when you see
    1The origin with Cache-Control headers. Browsers cache static files.Repeat visits by one user. Every new user reaches the origin.Static files use most origin bandwidth, or users far away wait.
    2A pull CDN for static files and images.61% of requests at the edge when it holds 25% of objects with a 10 min TTL.The hit ratio stays low on popular files.
    3Hashed file names with a one-year TTL, and a bigger edge cache.98% at the edge with a 1 h TTL. Deploys need no purge.Many edges each fetch the same object from the origin.
    4An origin shield.Origin load fell from 7.8 to 2 requests a second in the simulation.Per-request logic (redirects, token checks) still goes to the origin.
    5Edge functions, and public HTML at the edge with stale-while-revalidate.Logic and pages served near the user.Top of the ladder.

    All origin figures come from the seeded simulation: 5,000 objects, 500 requests a second, 4 edges. Read them as orders of magnitude.

    I start with Cache-Control headers and a pull CDN for static files. I add a shield when edges multiply origin fetches, and edge code only for per-request logic.

    N

    Drill

    predict, then reveal

    0 of 9 known

    1. A response has Cache-Control: max-age=0, s-maxage=60. What do the browser and the edge do?

    2. Why put a content hash in static file names?

    3. You purge everything at peak traffic. What happens to the origin?

    4. Marketing adds utm parameters to every link. The hit ratio falls. Why?

    5. A page shows the user name in the header. Can the CDN cache the page?

    6. What does an origin shield change?

    7. Why does the signature of a signed URL cover the expiry time?

    8. stale-while-revalidate=30 and s-maxage=60. A request comes at age 75 s. What does the user get?

    9. When would you push content to the CDN instead of letting it pull?

    O

    Numbers to say

    simulated, derived or cited
    origin load
    The origin sees (1 - hit ratio) of requests: 95% at the edge means 20 times fewer.
    short TTL
    A 10 s TTL capped the hit ratio at 44%, even with an unlimited cache.
    long TTL
    A 1 h TTL with every object cached gave 98%.
    popularity
    The top 1% of objects get 28% of requests; the top 10% get 56%, with alpha 0.8.
    purge
    Origin load rose from 5 to 318 requests a second right after a full purge.
    distance
    Light in fiber covers about 200 km per ms: an edge 500 km away is a 5 ms round trip.
    far origin
    California to the Netherlands and back is about 150 ms (Jeff Dean's table).

    Studies of web traces fit Zipf popularity with alpha from about 0.57 to 0.97. Hit ratio grows with the log of cache size (Breslau and others, 1999).