Configuring Your CDN for Efficient Static Content Delivery
If you are configuring a Content Delivery Network (CDN) to speed up your static content, it can be tricky to choose the correct settings for caching and expiry. The goal of using a CDN is to improve performance and serve the latest version of your files without adding unnecessary costs. Although the specific settings will depend on your application’s needs, and the CDN you are using, the principles remain the same. In this article I’ll walk you through these principles with examples.
The Basics of a Static Content CDN
A static content CDN speeds up the delivery of your files (images, CSS, JS, documents) to your users by serving the files from a location near to them. The files are originally stored on your server or cloud storage - also called the “origin”. The CDN fetches the files from your origin and caches them at various locations around the world. These locations are called edge locations or Points-of-Presence (POPs).
Why Not Serve From Your Own Server Or a Cloud Bucket?
Serving static content directly from your web server or from a cloud storage bucket has disadvantages if you have a global audience.
The costs for outgoing bandwidth can be prohibitive if there is too much data transfer.
Your end users may face high latency as your server is in one location. Even if you have a distributed cloud storage like S3 or GCS, they are in a limited set of regions, and there is no guarantee your cloud provider will serve from a bucket closest to the user.
There is no protection against malicious attacks like DDoS.
I had to set up a CDN recently called Bunny.net with the origin server as a Google Cloud Storage bucket. I’m going to use that for the examples but the principles are generally applicable.
Configuring Cache Settings for Optimal Performance
Browsers talk to the CDN POPs using HTTP. The CDN usually talks to the origin using HTTP as well. The key to fine-tuning your CDN settings for static content is to understand HTTP cache related headers.
Understanding HTTP Cache-Related Headers
Headers sent by the browser in the request
If-Modified-Since - Conditional request. “I have the image with this timestamp. Do you have a newer version?” The server response is either 304 (no newer version) or 200 (have a newer version).
Cache-Control - this controls caching in both browsers and CDNs, including intermediate proxies sitting between them. Important values in this context are:
no-cache - “Check with the origin server if there is a newer version of the image I have in my cache”. This does not translate to the English meaning of “do not cache” - it just means that a cache (a browser’s or some intermediate cache) can store a local version of the image but needs to revalidate it with the server. This is what gets triggered when you “hard-refresh” a page.
no-store - “Do not cache, i.e. store, this response in any caches”
Headers sent by the server in the response
Cache-Control
no-cache, no-store - Same as above, in the request headers section.
max-age=N (seconds) - Browsers and other caches can cache this for N seconds and serve the cached response.
So there are two places that cache your content - the CDN POPs and the user’s browser.
Caching Configuration for Origin to CDN
A CDN pulls files from the origin server and caches them at multiple locations around the world. When you change a file, you want it to cache the latest version so that your users get it.
What you should care about here
How long should the CDN cache the file at its POPs? If it caches it for a very short period, it will keep refetching from the origin, potentially increasing cost and load on the origin. If it caches for a long period, it might not fetch the latest version immediately. However, there is a way to force the CDN to fetch the latest version by calling “purge”. A purge invalidates a file in the CDN POPs so that it is forced to fetch it again from the origin. Purge is just another term for cache invalidation on the CDN.
CDN Cache Invalidation or Purging
In Bunny CDN, this is under “Cache expiration time”. You can choose to honor the origin server’s Cache-Control or override it. If the origin returns 3600 seconds as the Cache-Control: max-age header value, the CDN will cache it for 3600 seconds.
Caching Configuration for CDN to End User
An end user fetches the file from the CDN POP nearest to them.
What you should care about here
How long should the browser cache the file it fetched from the POP? If it caches for a short while, it will keep fetching the file from the POP, defeating the purpose of a CDN. If it caches for a long time, the user might not see the new version of a file for a long time. The correct setting here depends on your application.
This setting controls the Cache-control header from the CDN to your end users.
Fine-Tuning the HTTP Headers
Let’s look at some specific use cases to understand this better.
Case 1
File : The company logo on your website.
How often does it change? Not very often. Perhaps once in 5 years?
Clients can cache it for a long time.
CDNs can also cache it for a long time until asked to fetch a newer version using purge.
Case 2
File : User uploaded profile picture on a social media site. This shows up on their profile, visible to them and all their connections, and possibly everyone if it’s a public profile.
How often does it change? Not often but possibly more often than a company logo. 3-6 months?
Clients cannot cache it for a long time.
CDNs can cache it for a long time until asked to fetch a newer version using purge.
Case 3
File : A data.json which is updated every 30 minutes. The json is used to render charts.
How often does it change? Every 30 minutes.
Clients cannot be asked to cache it for too long. If the file changed at 10:00, and the browser accessed it at 10:15, the new version will be available 15 minutes later. Depending on the data, you can choose something like 5 minutes, or have a feature in the app to auto-refresh it.
CDNs can cache it for 30 minutes, starting from the time when a new version was pushed. The CDN will start caching it from the time it fetches from the origin, so you might have to sync a purge call once with a new version at the origin.
So if you know when the file will change, setting the correct values of the headers is slightly easier. In most other cases you have to guess. When you don’t know, you will probably err on the side of caution and ensure your users get the latest version at a possible cost of higher CDN-origin traffic (I would).
You can always force the CDN to fetch a new version of a file, but you cannot force your users to do that since it’s controlled by the browser. But there’s a way around it - read below.
Another Strategy
Another strategy used in web apps is to change the URL of the static files each time there is a change.
E.g. instead of index.html referring to
https://cdn.myapp.com/css/global.css
https://cdn.myapp.com/js/app.js
https://cdn.myapp.com/images/logo.svg
We create a dynamic URL at build time so that index.html refers to
https://cdn.myapp.com/202510011200/css/global.css
https://cdn.myapp.com/202510011200/js/app.js
https://cdn.myapp.com/202510011200/images/logo.svg
The app will get deployed each time there is a static content change, with each static content file uploaded to the dynamically created URL. Since the URL is different from what is cached in the end user’s browser, it will be fetched again. “202510011200” can be any unique id - here it’s a date-time combination.
In practice, this is what the build process will do:
Generate the unique ID.
Change any static content references in HTML files to include the unique ID in the URL.
Upload all static content to a CDN folder which has has the unique ID as the same.
There is no need to call purge.
With this approach, you can effectively set a high Cache-control header on the browsers. There are potential downsides:
Increased build complexity in your application.
A need for regular cleanup of older versions of the files in the CDN, as well as in the origin.
It might not be straightforward to change static content references if the HTML is generated by a app framework during the build.
War Story - Bunny CDN with Google Cloud Storage
A while ago, I was configuring Google Cloud Storage as the origin with Bunny CDN. A user uploads a file which would show up on their personalized status page. The original image file would get stored on GCS, and the CDN would pull it from there. I chose the following settings:
CDN cache expiration time -1 week. My app calls the CDN’s purge API after the user uploads the file, and I wanted to minimize calls to Google (and thus, cost). Since I called purge anyway in my app after the user uploaded the file, I did not want to set this to a low value.
Browser cache expiration time - 20 minutes.
Uploading a newer version of the image and invoking the CDN’s purge API did not result in the new image being served from the CDN. So something was not right.
It turned out that there was another caching layer in the flow - at Google. Google has its own cache in front of GCS. The Cache-control header that is set when I uploaded a file to GCS was being set to 3600 seconds - which seems to be the default. GCS would cache every file for 3600, so even if the CDN tried to fetch the file as a result of the purge call, it would get the old one until 3600 seconds had elapsed.
The solution was to set the header to “public, no-cache” while uploading the file to GCS. This resulted in every fetch from the CDN getting the latest version of the file as it existed on the origin server.
References
Mozilla Developers - HTTP headers reference https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers
Google Cloud Storage Docs - Consistency https://cloud.google.com/storage/docs/consistency#cache-control
Google Cloud Storage Docs - Setting Metadata https://cloud.google.com/storage/docs/metadata#cache-control
Google Cloud Storage Built-in Caching - https://cloud.google.com/storage/docs/caching#built-in