Julik Tarkhanov

GETting Conditionally - The Bare Basics

A while ago, a prominent Vercel employee (two, actually) posted to the tune of:

Developers don’t get CDNs

Exhibit A etc.

It is often that random tweets somehow get me into a frenzy – somebody is wrong on the internet, yet again. But when I gave this a second thought, I figured that… this statement has more merit than I would have wanted it to have.

It has merit because we do not know the very basics of cache control that are necessary (and there are not that many)!

It does not have merit in the sense that force-prefetching all of your includes through Vercel’s magic RSC-combine will not, actually, solve all your problems. They are talking in solutions that they sell, and what they are not emphasizing is that the issue is with the “developer slaps ‘Cache-Control’” part. Moreover: as I will explain, a lot of juice can be squeezed out of you by CDN providers exactly because your cache control is not in order and they offer you tools that kind of “force” your performance back into a survivable state. With some improvement for your users, and to the detriment of your wallet. But first, let’s rewind and see what those CDNs actually do.

CDNs use something called “conditional GET requests”. Conditional GET requests mean: Cache-Control. And even I, in my hubris, haven’t been using it correctly. After reviewing how it worked on a few of my own sites, I have overhauled my uses – and built up a “minimum understanding” of it which has been, to say the least, useful.

So, there it is: the absolute bare minimum of Cache-Control knowledge you may need for a public, mostly-static (CMS-driven, let’s say) website. Strap in, this is going to be wild.

And be mindful of one thing: I do not work for Vercel, CloudFlare, AWS or Fastly. I just like fast websites and I think you deserve to have your website go fast as well.

What are web caches, anyway?

Put simply: web caches ensure that once you have downloaded something, you get to keep it as long as it doesn’t change. Or, at the very least, you will be downloading it from someplace that is faster - and closer to you - than the website you are originally trying to access.

Your browser has a cache (this is your private cache, just yours). Then there can be “intermediate” caches, which we are going to call “caching proxies”. At the very end of that chain there is the actual website you are downloading from - the origin.

If you are running the origin website - it is your responsibility to use those caches to the max - and, if possible, running one of your own. As a matter of fact, we are going to do just that.

In the times of yore, when dinosaurs roamed the earth and shared web hosting was commonplace - caching proxies were actually much more pervasive than they are now. To begin with, many hosting providers would forcibly install something called squid in front of all websites hosted by them. No “ifs”, no “buts” - if you were hosting with a certain provider, they would stick a caching proxy in the middle regardless of whether you wanted it or not, whether it was beneficial for use case - or not.

Next, company intranets were a thing and internet access (especially: fast internet access) was expensive. Doubly so - for business users, and in offices. So - in addition to the caching proxies installed by hosting providers - you would also have the “internal” caching proxies. Remember the Internet Security and Acceleration Server from Microsoft? Well, the “Acceleration” part was about doing caching proxying for your entire office.

And then, with pervasive broadband - and both the more widespread deployment of SSL and the emergence of virtual machine based hosting - those caching proxies kind of faded out of the picture, except for the use cases of the biggest, most frequented web resources on the planet. The Akamai CDN is old, and Apple has been using Akamai since… 1999? What is Akamai, you may ask? Well… it is a caching proxy!

A CDN versus a caching proxy

For ease of understanding I defined it like this: a CDN is a sophisticated caching proxy with multiple caching nodes. An ideal CDN would have multiple tiers of caches, and have those caches geographically distributed to provide multiple POEs (“points of entry”) for users in different locations, with the “hottest” (most frequently accessed) pages being available in the caches closest to the user.

An ultimate “distribution strategy” would be something similar to where Netflix and YouTube have gone. They install “caching appliances” (caching web proxy servers, essentially) in hardware form, right at the ISPs data center - so that when you go to watch the latest episode of Adolescence you get it quickly and without your ISP having to go fetch it from Netflix - as your neighbor has already watched it yesterday, and it is in the ISP’s caching appliance Netflix has given them.

So, for a web developer like myself and you: a CDN is a caching proxy, just a fairly sophisticated one. The more sophisticated - the more “tricks” you can employ with it. For example, edge functions are basically scripts/small web apps which run on the CDN edge nodes, and can route, restrict or splice requests before they get to your web application.

But for 80% of use cases - or more - you can get by without any of that. No fancy Vercel prefetch, no fancy DOs, no edge functions - just the good, old Cache-Control. But it is a fickle mistress and you have to hold it right.

For static files, cache control is usually good already

How is it that we often don’t need to think about Cache-Control at all? Well, all of the modern webservers - no exception - are quite adept at setting cache-friendly headers for static files. It is not difficult to do, and since they do it so well - for most users this is actually fully transparent - including you.

But once the output comes from your web application - a good webserver will step aside and not touch your Cache-Control at all. If it does - either the server is misconfigured by mistake, or it is deliberately misconfigured to force caching on you - by a nefarious systems administrator at your hosting provider (or a semi-hostile devops team) to save costs. This is an interesting topic which we will revisit.

Learning with your own toys

I firmly believe that you can place little trust in things you can’t run locally. At the very minimum a faithful reproduction is also good enough, provided it works well. For Ruby web apps, there happens to be an old-but-good caching proxy solution which actually embeds inside your Rails (or Rack) application. It will intercept incoming requests, examine the responses you return to the client - and cache them as headers dictate. So, let’s put together a bare-bones Rack application which serves us a cheeky little piece of content:

run ->(env) {
  require "securerandom"
  request_id = SecureRandom.uuid
  headers = {"content-type" => "text/plain"}
  [200, headers, ["Hello request #{request_id}"]]
}

We will also need another piece of kit which I have been using for ages, and it is a curl shell alias. A shell alias is just simpler to set up, you can use an HTTP client call if you prefer that. The alias goes like this:

alias headercheck='curl -s -D - $1 -o /dev/null'

and you run it like this:

$ headercheck https://blog.julik.nl
HTTP/2 200 
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Fri, 23 May 2025 17:12:38 GMT
access-control-allow-origin: *
etag: "6830ac86-e960"
expires: Thu, 12 Jun 2025 23:58:18 GMT
cache-control: max-age=600
x-proxy-cache: MISS
x-github-request-id: 9BA7:3471C9:54FA19:55EEF4:684B6741
accept-ranges: bytes
age: 0
date: Thu, 12 Jun 2025 23:48:18 GMT
via: 1.1 varnish
x-served-by: cache-ams21075-AMS
x-cache: MISS
x-cache-hits: 0
x-timer: S1749772098.153089,VS0,VE132
vary: Accept-Encoding
x-fastly-request-id: 97b8fdb7083c48fc06e988ebcc32efd7281c0be0
content-length: 59744

It does something very simple: makes a GET request (not a HEAD, and that is important!) using curl, and then prints you just the headers. It discards the response body.

From this example, we can actually already see a few interesting things:

  • The presence of etag indicates that Github calculates some kind of checksum for the index page
  • The last-modified defines the last modification date of the file
  • The via: 1.1 varnish indicates that GH Pages is using a caching proxy of its own - one called Varnish
  • The cache-control says max-age=600 - we will get to that
  • The vary: Accept-Encoding says that any caching that applies for us is content-encoding specific (so a separate cache will be used for Gzip-compressed responses and for plain responses)
  • The etag is a “strong” ETag - it is just a string in quotes, without the \W at the end. This means that the representation we have fetched is specific to the content-encoding (or, rather - the combo of our request headers mentioned in the vary: header)

We are going to use this headercheck command to experiment with our local nano-CDN (which conveniently lives inside our Rack application).

Let’s complete our Rack web app with dependency definitions and “headercheck” it:

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'puma', '~> 6.4', require: false
  gem 'rack'
end

require 'puma'
require 'rack'

# Define a simple Rack application
app = Rack::Builder.new do
  run ->(env) {
    require "securerandom"
    request_id = SecureRandom.uuid
    headers = {"content-type" => "text/plain"}
    [200, headers, ["Hello request #{request_id}"]]
  }
end

# Configure and start Puma server
server = Puma::Server.new(app)
server.add_tcp_listener '0.0.0.0', 3000
STDERR.puts "Puma server listening on port 3000"
server.run.join

When we headercheck it, we get:

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 16

Since we are interested in the headers here - and headers get cached as well - let’s add our “sentinel value” (the request ID) to the headers. If it does not change between our invocations of headercheck, we will know that the cached output is being served:

require "securerandom"

# Define a simple Rack application
app = Rack::Builder.new do
  run ->(env) {
    request_id = SecureRandom.uuid
    headers = {"x-request-id" => request_id, "content-type" => "text/plain"}
    [200, headers, ["Hello request #{request_id}"]]
  }
end

…and hit it with a couple of requests:

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: 3354a2e3-7ce1-49e7-b44c-5956c539017c
content-type: text/plain
Content-Length: 50

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: 62ae58dd-7847-484d-8cee-57e150c90dd9
content-type: text/plain
Content-Length: 50

As designed, the x-request-id is different on every request - we have a completely dynamic web application. By default, it will not cache anything.

Now, let’s add rack-cache

app = Rack::Builder.new do
  use Rack::Cache,
    metastore: "file:rack-cache",
    entitystore: "file:rack-cache",
    allow_reload: true

  run ->(env) {
    request_id = SecureRandom.uuid
    headers = {"x-request-id" => request_id, "content-type" => "text/plain"}
    [200, headers, ["Hello request #{request_id}"]]
  }
end

Initially, nothing changes:

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: dd4b4a1d-46c8-46ca-8a5c-a4d182424152
content-type: text/plain
date: Fri, 13 Jun 2025 00:07:32 GMT
x-rack-cache: miss
Content-Length: 50

We do see, however, that rack-cache has recorded a cache miss - it tried to satisfy the request using cached data, but it could not - so it has let the request “fall through” to our app.

Now let’s do some cache controlling:

headers = {"x-request-id" => request_id, "content-type" => "text/plain"}
headers["cache-control"] = "max-age=600"

and check the headers:

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: 49048426-fe8b-4a31-9828-0ef5fcfa831e
content-type: text/plain
cache-control: max-age=600
date: Fri, 13 Jun 2025 00:10:18 GMT
x-content-digest: 80c4a9709694ccde545395317456031aa50e08be
age: 0
x-rack-cache: miss, store
Content-Length: 50

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: 49048426-fe8b-4a31-9828-0ef5fcfa831e
content-type: text/plain
cache-control: max-age=600
date: Fri, 13 Jun 2025 00:10:18 GMT
x-content-digest: 80c4a9709694ccde545395317456031aa50e08be
age: 1
x-rack-cache: fresh
Content-Length: 50

Now, rack-cache starts caching our app’s output. We limit the age of the cache, and we can see that rack-cache has also computed the x-content-digest of our response, and that it stays the same. We can also see that the value of x-request-id header does not change. Our little “CDN in a box” is doing its job, and you now have what is an equivalent of the Rails page cache.

It is also broken. For example, imagine we want to to change the output of our little app:

[200, headers, ["Gruezi for request #{request_id}"]]

We make a request, and…

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: 49048426-fe8b-4a31-9828-0ef5fcfa831e
content-type: text/plain
cache-control: max-age=600
date: Fri, 13 Jun 2025 00:10:18 GMT
x-content-digest: 80c4a9709694ccde545395317456031aa50e08be
age: 223
x-rack-cache: fresh
Content-Length: 50

Oops. We are in exactly the same dreadful spot the webmasters of the early 2000s would find themselves with a provider forcibly imposing a caching proxy on them. Your page is now firmly planted in the rack-cache, and won’t be excised and re-requested from your app until age turns 600 seconds or more. Bah.

What can we do about it?

Cache validity

When you talk about caching, a fairly common term is “cache invalidation”. It’s a weirdly chosen name, because there is nothing “invalid” about it. Rather, “validation” with caches means literally this: asking whether this cache is still fresh. The validation can be done using some sort of validation handle. For example, it can be a timestamp. We can let our app tell the caching proxy that the last modification time on our resource is X, and it then works as a validator. It will work in two ways:

  • It will inform the caching proxy better as to what max-age refers to. A proxy may choose to revalidate with the origin (our app) even if max-age has not yet lapsed, or it may not. We’ll get there
  • It allows the proxy to ask our app: “Have you modified this resource since X?” If our app responds with “Nope, still the same” (which is the 304 status) - the caching proxy can choose to extend its cached version storage for another max-age, or to just serve the cached version now

Since we are inside of a script and we know when we modify it - it is a file, dammit - we can grab the mtime of our script as our Last-Modified. It does need to be formatted using Time#httpdate:

headers["cache-control"] = "max-age=600"
headers["last-modified"] = File.mtime(__FILE__).httpdate
[200, headers, ["Gruezi for request #{request_id}"]]

Now, when we do a few GETs in sequence, the behavior of the caching proxy changes:

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: a86dfec7-4c5b-4715-b83f-8bc4460b01de
content-type: text/plain
cache-control: max-age=600
last-modified: Fri, 13 Jun 2025 00:22:08 GMT
date: Fri, 13 Jun 2025 00:22:15 GMT
x-content-digest: e44ad5b6f016fa72eb8502e3d697c183b1958fed
age: 0
x-rack-cache: stale, invalid, store
Content-Length: 55

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: a86dfec7-4c5b-4715-b83f-8bc4460b01de
content-type: text/plain
cache-control: max-age=600
last-modified: Fri, 13 Jun 2025 00:22:08 GMT
date: Fri, 13 Jun 2025 00:22:15 GMT
x-content-digest: e44ad5b6f016fa72eb8502e3d697c183b1958fed
age: 3
x-rack-cache: fresh
Content-Length: 55

as does the output of our Rack app (the output is essentially the same as the x-rack-cache header):

cache: [GET /] stale, invalid, store
cache: [GET /] fresh

So now we know rack-cache has used our validator and cached the initial version, and then served it from its cache. Neat. Let’s adjust our max-age a bit:

headers["cache-control"] = "max-age=30"

restart our app, and do a headercheck again - we changed the file, so it should pick up our changes and…

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: a86dfec7-4c5b-4715-b83f-8bc4460b01de
content-type: text/plain
cache-control: max-age=600
last-modified: Fri, 13 Jun 2025 00:22:08 GMT
date: Fri, 13 Jun 2025 00:22:15 GMT
x-content-digest: e44ad5b6f016fa72eb8502e3d697c183b1958fed
age: 300
x-rack-cache: fresh
Content-Length: 55

Nope. It happily continues serving our previous version until the age value exceeds the 600 seconds we have set. Pudu. What we need to do in this case is actually tell rack-cache that it should revalidate with the origin every time:

headers["cache-control"] = "max-age=30, must-revalidate"

And - if we don’t want to wait another 5 minutes - we need to manually delete the cache:

rm -rf rack-cache
ruby app.rb

This is an important lesson (and one of the reasons “developers don’t get CDNs”).

Lesson 1: Once you have told your caching proxies to cache something, it may be a nuisance to make them “forget” that cached data.

The initial request and the subsequent ones work fine:

cache: [GET /] miss, store
cache: [GET /] fresh
cache: [GET /] fresh

and when we change the contents of our app and restart it, we see an actual invalidation followed by fresh:

cache: [GET /] stale, invalid, store
cache: [GET /] fresh
cache: [GET /] fresh

Once max-age lapses, our app again gets hit and rack-cache knows that even though last-modified is the same, the output has likely changed and checks anyway.

Actually making the GET conditional

Now let’s actually use our validator. In our previous example, rack-cache would ask us for a new rendered response regardless once the max-age has lapsed. Now, let’s check for the If-Modified-Since header - and if it hasn’t changed since our file modification time - respond with a 304:

request_id = SecureRandom.uuid
headers = {"x-request-id" => request_id, "content-type" => "text/plain"}
headers["cache-control"] = "max-age=30, must-revalidate"
mtime_http = File.mtime(__FILE__).httpdate
headers["last-modified"] = mtime_http

if env["HTTP_IF_MODIFIED_SINCE"] && Time.parse(env["HTTP_IF_MODIFIED_SINCE"]).httpdate == mtime_http
  [304, headers, []]
else
  [200, headers, ["Gruezi wieder for request #{request_id}"]]
end

Now, a few requests (and having waited 30 seconds), we see that our validator is used and is actually still valid:

cache: [GET /] stale, invalid, store
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] fresh
cache: [GET /] stale, valid, store

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: a12dbf44-41ef-4c92-a926-1daccf6749c9
content-type: text/plain
cache-control: max-age=30, must-revalidate
last-modified: Fri, 13 Jun 2025 00:38:09 GMT
date: Fri, 13 Jun 2025 00:38:17 GMT
x-content-digest: 900b1529121c74f438ae26ba4b24c0ebc858cf04
age: 27
x-rack-cache: fresh
Content-Length: 62

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: a12dbf44-41ef-4c92-a926-1daccf6749c9
content-type: text/plain
cache-control: max-age=30, must-revalidate
last-modified: Fri, 13 Jun 2025 00:38:09 GMT
x-content-digest: 900b1529121c74f438ae26ba4b24c0ebc858cf04
date: Fri, 13 Jun 2025 00:38:50 GMT
age: 0
x-rack-cache: stale, valid, store
Content-Length: 62

Note how, in the second request, while its age is reset to 0, the x-request-id header is exactly the same - and so is the content-digest! rack-cache resets the age, but it reuses the already cached response and does not call our application again.

ETags

A last-modified has a fundamental limitation. It is a low-resolution timestamp (only to a second). If our resource changes multiple times per second, the changes will not register - even though the contents of the resource may have changed. But more importantly - a last-modified does not provide us with a fundamental “webmastery” thing we would want at all times: read after write consistency

When you have a page, and you have some kind of CMS that renders that page, when you edit the page you absolutely want to see your changes immediately as you hit “Publish”. But if you do that frequently, your last-modified time stays the same. It is wall clock, not a vector clock.

This is why ETags are better. ETags are “freeform checksums” - values that only your application knows the semantics of. For example, you can have an ETag which says “version-1” and then gets incremented to “version-2”, “version-3” and so forth, effectively embedding a vector clock in the etag.

For reasons™ ETags must be in quotes. Let’s replace our modification time of the file with its checksum:

headers["cache-control"] = "max-age=30, must-revalidate"
etag = Digest::SHA1.base64digest(File.read(__FILE__)).inspect
headers["etag"] = etag

if env["HTTP_IF_NONE_MATCH"] == etag
  [304, headers, []]
else
  [200, headers, ["Gruezi wieder for request #{request_id}"]]
end

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: da28d4c7-c9dd-43e3-a855-0183808ef640
content-type: text/plain
cache-control: max-age=30, must-revalidate
etag: "WCWGtNQ5wtRieI0Z1A4bpSOag2s="
date: Fri, 13 Jun 2025 00:48:02 GMT
x-content-digest: 5f34552c880e4bf6a7938a74b5459dfcbe0f6e0e
age: 4
x-rack-cache: fresh
Content-Length: 62

And after some time:

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: da28d4c7-c9dd-43e3-a855-0183808ef640
content-type: text/plain
cache-control: max-age=30, must-revalidate
etag: "WCWGtNQ5wtRieI0Z1A4bpSOag2s="
x-content-digest: 5f34552c880e4bf6a7938a74b5459dfcbe0f6e0e
date: Fri, 13 Jun 2025 00:48:58 GMT
age: 0
x-rack-cache: stale, valid, store
Content-Length: 62

Notice something peculiar? Even though rack-cache is clearly reusing an already stored response, it prints store to the log - as if it “stores” the new response. Therefore: lesson 2.

Lesson 2: debug information from caching proxies can be very confusing.

Actually conditional GETs

Now let’s get to the meat of the matter: if you want true “read-after-write” consistency, here is what you need to do:

headers["cache-control"] = "must-revalidate"

Yes, you remove max-age entirely. It means that every request will end up with our origin app and may cause some computation, but if it is cheap enough - we would just confirm the validator and respond with a 304.

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: 831f88d1-9873-4241-8356-e80059854c73
content-type: text/plain
cache-control: must-revalidate
etag: "yxMkFgyIcrJBk8AaIxDOYdWq0VU="
x-content-digest: ba6bad0df3cb83cebf87a37d1a85ca59e73c7c85
date: Fri, 13 Jun 2025 00:53:53 GMT
age: 0
x-rack-cache: stale, valid, store
Content-Length: 62

$ headercheck http://localhost:3000
HTTP/1.1 200 OK
x-request-id: 831f88d1-9873-4241-8356-e80059854c73
content-type: text/plain
cache-control: must-revalidate
etag: "yxMkFgyIcrJBk8AaIxDOYdWq0VU="
x-content-digest: ba6bad0df3cb83cebf87a37d1a85ca59e73c7c85
date: Fri, 13 Jun 2025 00:53:54 GMT
age: 0
x-rack-cache: stale, valid, store
Content-Length: 62

If you edit the file and restart the server - the etag will change, and every subsequent revalidation will not have a matching ETag.

The subtle art of ETag divination

It is a bit of an art to come up with an ETag which accurately reflects the state of the resource displayed, but also of the application at large. For example, changes in your gems will likely lead to changes in rendered output. Changes in the objects you fetch into the view will, as well. And if your resource contains lists of things - the disappearance of an item from a list should also change the ETag.

Rails does provide something called cache_key which is available on all ActiveRecord models. The problem is that this cache_key can’t be used as the ETag, because it contains… the updated_at - and a truncated one at that. So it is actually just a Last-Modified supplemented with the model class name and ID:

User.first.cache_key #=> "users/1"

And that’s… meh. It does help with identifying queries, luckily:

User.where("id > 1").cache_key #=> "users/query-988b60a2f77c74ce4972ac63a82a82f4"

but is not really usable as ETag either. It is, after all, a digest of the query, but not of what the query produces. The way I approach it is heavy-handed but (in my opinion) very effective:

etag = Marshal.dump(model.attributes)

If you want to have multiple values contribute to the ETag, a good tool for this is Digest, used in its stateful form:

d = Digest::SHA1.new
models_involved.each do |m|
  d << Marshal.dump(m)
end
d << File.read(__FILE__)
etag = d.base64digest.inspect

Like any other cache key, ETags require finesse. For example, imagine your ETag is for a page with a list of widgets, and the URL already specifies that we will filter down to ?state=shipped widgets. You may then have an ActiveRecord relation for those widgets:

user.widgets.where(state: "shipped")`

But just the query is not enough. There are two possible cases where our resource (the page, API response etc.) will change:

  • A widget is no longer shipped or a widget has become shipped since we computed our ETag
  • A widget has been deleted entirely even though it was shipped

If we show all our widgets on the page, but may want to avoid rendering our view - and still produce a good ETag - we can do this:

shipped_widget_ids = user.widgets.where(state: "shipped").pluck(:id).map { it[:id] }
etag = Digest::SHA1.hexdigest(shipped_widget_ids.join).base64digest

This captures collection membership accurately.

Shortcutting ETag divination even more

I’ve lately enjoyed SQLite a great deal. One of the advantages with SQLite is that it does have a very good Last-Modified - the actual mtime of your entire database! Combined with the Git SHA (APP_REVESION) it allows for great ETags - while those won’t be “read-after-write” consistent, they will be quite stable.

d = Digest::SHA1.new
d << ENV["APP_REVISION"].to_s
d << File.mtime("storage/production.sqlite3").iso8601

This will give you a very reasonable ETag for the entire website.

Expiring your cache

It not only happens sometimes - it will happen to you when trying to make your site cacheable. Your caches will become stale, the site will get updates - but the visitors will not be getting the latest version, or - which is more likely - you will bodge the ETag derivation and miss something. It is a subtle art, and cache invalidation is one of the hard problems of computer science – no biggie.

With the strategy I’ve described above, you don’t need to reach for that big “Expire all the things” button in your CDN admin panel. All you need to is this:

digest_for_etag << "please expire"

or, since I bake the Git SHA into every Docker image I create:

digest_for_etag << ENV["APP_REVISION"]

This way, all of your caches will automatically invalidate on every deploy. How? Well…

  • Since your Cache-Control does not specify max-age, the max-age is considered to be 0, so…
  • …the caching proxies will always revalidate - since you said must-revalidate…
  • …so they will always do a conditional GET, which…
  • …will find a changed ETag and therefore refresh the cached resources.

As Tobi Lütke wrote many moons ago - you never want to manually expire.

Actually, I bet this is how the “instant expiry” on CloudFlare works too - they just bump a few bytes in your cache keys. But you can do that on your end as well!

Content-addressable resources and immutability

That is something you can get with ETags and systems like Git - but you can also do this if you store some kind of checksum with every model in your system (which is a bit tricky - but doable). Imagine every model you display has some kind of checksum_bytes column in the database, which stores a SHA1 digest of all the attributes on save. Kind of like this:

module AttributeDerivedChecksum
  extend ActiveSupport::Concern

  included do
    after_validation :update_attribute_checksum
  end

  def update_attribute_checksum
    return unless self.class.column_names.include?("_attribute_checksum_bytes")
    new_value = Digest::SHA1.digest(Marshal.dump(attributes.except("_attribute_checksum_bytes")))
    write_attribute("_attribute_checksum_bytes", new_value
  end

  def cache_key(...)
    if persisted?
      super + "/" + _attribute_checksum_bytes
    else
      super
    end
  end

  def serializable_hash(options = nil)
    options = if options
      options.merge(except: Array.wrap(options[:except]).concat([:_attribute_checksum_bytes]).uniq)
    else
      {except: [:_attribute_checksum_bytes]}
    end
    super
  end
end

Then you can output the _attribute_checksum_bytes (converted to Base64 or other palatable string representation) as an ETag, or mix it into your ETag for invalidation:

headers["etag"] = user.cache_key.inspect

Another measure you can use is the optimistic locking which conveniently provides a lock_version:

module LockVersionCacheKey
  extend ActiveSupport::Concern

  def cache_key(...)
    if self.class.column_names.include?(self.class.locking_column)
      [super, self[self.class.locking_column]].join("/")
    else
      super
    end
  end
end

which would also enable easier conversion of an ActiveRecord into a stable cache key, and then - to an ETag.

Avoid immutability

The temptation would then also be to say (which is possible with some CDNs)

headers["cache-control"] = "immutable"

But check this out: imagine that Mallory became one of the patrons of your website. Mallory posts a piece of CSAM on your site, which your system duly checksums, and marks “immutable”. The piece of CSAM ends up in your CDN caches, and since it’s marked “forever” the CDN is not going to “ring back” and ask whether it is still in place. It will happily continue serving it to visitors, oblivious to the fact that you have actually received a court order commanding you to erase the resource within 12 hours - which you totally did.

Therefore: unless it concerns some very benign, bespoke resources that you can be absolutely positive will never ever need to be deleted - and most likely you have none of those - just don’t rely on that feature. Ever.

A 404 is ephemeral

There is a reason that 404 Not Found and 410 Gone are different status codes. Here is a hypothetical scenario:

  • There is an address on your site called /exciting-announcement. It now returns a 404
  • It was never used, but now the marketing department wants to use it to put out an announcement
  • The announcement is there, and is happily present for a week or so, until…
  • The legal department finds that the /exciting-announcement contains conflicting (and illegal) claims about a competitor, and thus has to be pulled immediately. It now returns a 404
  • …but later on a way is found to make the announcement without disparaging the competitor. The URL is reinstated, but with different content.

All of those scenarios will be ruined if your 404 gets cached by a proxy. And it can get cached provided that you enable caches for non-2xx responses - this is usually not the default for caching proxies, but you may want to do it sometimes because one of the ways your web app can be brought to its knees is by bombarding it with requests to non-existent pages.

  • Instating the page will not work as proxies that have, by accident, cached the 404 page
  • Once the page is in place, the proxies will happily cache it even further…
  • …so the request from legal to delete the page will not work…
  • and so on

This is what the 404 code is for. If you are dealing with user-uploaded content, do yourself a favor and design a workflow with “permadeletion” where a resource would return a 410 status. And that response can have cache headers and be cached, because it is one of the very few truly immutable responses.

My advice? Do cache 404s but they should have a max-age, and maybe not even that. And do not play with immutable or content-addressable resources until you have a firm grasp of the consequences.

An interesting part of CDN offerings

CloudFront, for one, is an incredible product. I was very, very impressed at the depth and effectiveness of the package they offer. But take a look at this page:

One of the headliner features in CloudFlare used to be Cache Settings Override - which would allow you to ignore origin cache headers and set your own caching rules at the edge, effectively overriding whatever cache control headers your origin server sends.

Surprising, isn’t it? So a feature is explicitly in not honoring the Cache-Control headers. If you put this and Guillermo’s post together (even though Vercel and CloudFlare bump heads constantly) – you can figure out what is going on, but I will outline it just in case. There is a strong business use case which those CDN providers are addressing.

  • You are a large organization called AcmeCorp. You have a number of web properties, which all have become slow. Very, very slow.
  • At properties 3, 7 and 12 the freshly formed DevOps™ Agile Transformation Team formed just 3 quarters ago has investigated performance and has experimented with Cache-Control to make things better.
  • Sadly, the team was disbanded, the tech lead on it fired and all the ops work of that team got outsourced to the secondary HQ in Saarbrücken. They are in the middle of their third reorg – and no end in sight, so there is nobody who owns the cache configuration.
  • Meanwhile, you get a mandate to do something because even the COO notices that web property number 3 has become insufferably slow. Users complain, support queues are growing.
  • …but due to having removed all technical leads from your part of the organization there is nobody to call Saarbrücken, get access to the repository, take ownership of the code setting the headers and fix them. Even where you are at, close to the COO.

Therefore, what do you do?

  • One option is retooling your web property 3 into something very fancy with maximum Vercel lock in, and hoping that their magic tricks with streaming RSCs will magically make things go brr. You also need an entire team completely locked-in on all the newest React/Next.js practices.
  • Put web property 3 behind CloudFlare, but then use the admin console to finally override those pesky bodged headers that those blokes in Saarbrücken just don’t want to fix (they are still having their meeting about whether the title of Doktor Fachinformatiker should be a requirement on their job descriptions or not), and be done with your day.

Of course you choose the latter.

The CDNs are having a ball solving the problem of organizations either being unable to get their Cache-Control right, or with organizations being dysfunctional enough that they can get them right - but utterly unable to drive the change through. With this perspective in mind, Guillermo’s post reads differently, doesn’t it?

This is another reason why knowing your way around Cache-Control is imperative. Don’t be the team in Saarbrücken and don’t let yourself be sucked into the suffocating Vercel embrace.

Balancing validations and read-after-write

It may be that you do want to have read-after-write only for people who actually can change the content on the page. That is actually harder to do than one would like, but very simple. All you need to do is bump the URL.

Yes, that easy. Your cache key in a caching proxy is always a combination of the full URL, the values of headers mentioned in Vary and the validators. To force a refresh for a URL, you may get by with just this:

<a href="/pages/{id}?b={rand}">View the updated page</a>

For a site with public-facing pages I would actually not do that, though, and just speed up my revalidations enough that they become usable for every request. This is the next important lesson.

You want less config, not more

The header we have specified in the end (Cache-Control: must-revalidate) is actually implicitly expanded into this:

Cache-Control: public, max-age=0, no-cache, must-revalidate

And this is confusing as well, because no-cache actually means “do cache, but always revalidate first”.

Lesson 3: Cache control is intricate, and you should aim to do as little of it as you can.

This actually applies to more things in modern web development, and it is a pity that both the modern platform teams and modern frontend teams work in the opposite direction (more configs and more systems, as far as the eyes can see).

Conclusion

First, let’s reiterate the lessons one more time:

  • Lesson 1: Once you have told your caching proxies to cache something, it may be a nuisance to make them “forget” that cached data.
  • Lesson 2: debug information from caching proxies can be very confusing.
  • Lesson 3: Cache control is intricate, and you should aim to do as little of it as you can.

And here is how you can benefit from Cache-Control, today:

  • Use an embedded caching proxy inside your application to model the side-effects
  • Once your embedded caching proxy is rock-solid and configured to your liking, set up a CDN
  • Read your CDN’s documentation regarding Cache-Control. For example, here’s CloudFlare’s
  • For a publicly-accessible “content” website, you likely want either public, must-revalidate with ETags or public, max-age=30, must-revalidate with ETags.

While Cache-Control is fairly well-defined, there are still peculiarities to how CDNs treat it. For example, stale-while-revalidate is very handy, but it only functions in combination with max-age being higher than 0 on CloudFlare. s-max-age can be useful too. But before you reach for these - get your basic, locally-testable Cache-Control rock-solid.

Because I want your website to be fast, and I am not selling you a CDN. Serious.