Notes on caching

What is caching?

Caching is a technique that stores copies of data in a temporary storage (called a cache) so that future requests for that data can be served more quickly.

Example

let pi

function getPiCached() {

 // Cache misses
 if (typeof pi === 'undefined') {
  pi = computePi() // expensive call
 }

 // Cache hits
 return pi
}

Cache keys

The cache key must account for all inputs required to determine the result

Example

const cache = new Map<string, Date>();

function addDaysCached(count: number) {
  const key = `add-days:${count}`;

  if (!cache.has(key)) {
    cache.set(key, addDays(count));
  }

  return cache.get(key);
}

function addDays(count: number) {
  const msInDay = 1000 * 60 * 60 * 24;
  return new Date(Date.now() + count * msInDay);
}

Issue:
The key only includes count, but Date.now() changes on every call.

Challenge

1. It’s easy to miss an input

2. Too many inputs

3. Computing a correct cache key

Cache Revalidation

There are 3 main ways

1. Proactively Updating the Cache - On post update, update the cache

2. Time Invalidation - cache-control headers

3. Stale while Revalidate - update cache in the background

4. Forcing fresh value - manual cache updates

Caching is easy; revalidating the cache is hard.

HTTP Caching

The HTTP cache stores a response associated with a request and reuses the stored response for subsequent requests.

2 types of caches

• Private: user’s browser

• Public: Shared cache, CDNs, etc

See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Caching

What is Vary?

By default, URLs are used as cache keys

However, sometimes, the content is not the same given the same URL. For example, depending on the Accept-Language header, the server will send back different languages of the same content. In this case, we can cause the responses to be cached separately, based on the language ,by adding Accept-Language to the value of the Vary header

Vary: Accept-Language

What is validation or revalidation?

Stale responses are not immediately discarded. HTTP can transform a stale reaction back into a fresh one by asking the original server.

The way to validate is to use etag

Etag to revalidate

The server sends a unique ETag, which the client stores. On subsequent requests, the client includes this ETag in the If-None-Match header. If the server responds that nothing has changed, the cache is considered fresh and can be used.

//Server - recieving a requst

(request, response) => {
    // md5 just created a unique hash based on the html content
    const etag = md5(html)

    // check client's etag - nothing changed
     if (etag === request.headers["if-none-match"] {
        // not modify
        response.writeHeader(304)
        response.end()
     }

    respnose.writeHeader({
     200, {
         "cache-control": "max-age=0, must-revalidate"
     }
    })
}

Cache-control header

See all the cache-control header here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control

no-cache vs no-store

• no-store: don’t cache

• no-cache: store but must be validated with the server before using

no-cache is the same as max-age=0, must-revalidate But we should stick with no-cache as now all browser support it.

must-revalidate

By default, HTTP can still use a stale cached response if they are disconnected from the original server.

This means the response can be stored and used if it’s fresh. If it’s stale, must check with the original server. If the server is unreachable during revalidation, the client gets a 504 Gateway Timeout error.

📚 Resources

• To Cache or Not to Cache