Running a website as big as GOV.UK would be impossible without caching. Storing a copy of a page, instead of building it from data each time it’s viewed, allows the page to be served very quickly. It enables us to keep up with the millions of users accessing GOV.UK without having too much expensive hardware.
However, caching comes at a cost. Once a page has been cached, it may not be updated until the cache has expired - even if someone has published some new content. We’ve set the expiration time on content to 30 minutes, meaning it can take up to that long for updated content to appear on the website.
To get around this issue, it’s possible to manually clear a page from the cache when we need to - something our technical support team often do while some high priority publishing is going on.
Luckily, this is not such a problem with new content as the page is unlikely to have already been cached. Although if someone were to try and access new content before it’s published, we may cache the 404 page for that particular URL.
To get around these issues, we’ve decided to automate the process so that pages will be removed from the cache automatically when content gets updated. We already have a system in place that allows tools to be notified when new content is published or old content is updated - this is the same thing we use to send out email notifications.
We can build on top of this existing service to listen for when a piece of content changes and then remove that page from the cache. We deployed our new cache clearing service to production at the beginning of September and we’ve already seen some promising results.
What we've found so far
On 13 September, the government published the second round of technical notices related to preparations for if the UK leaves the EU without a deal.
The first round had been published a few weeks prior, and our technical support team at the time had to manually intervene on a few steps to ensure it all went out smoothly. Specifically, it became necessary to clear the cache to remove draft versions and pull through last-minute changes.
For the second round of publishing, we’d deployed the cache clearing service a few days before and we thought this would be an excellent opportunity to test it all worked. We documented the things we had to do manually for the first round so that we were prepared to do it again should we have to.
When it was time to publish, we were pleased to see the caches fully cleared themselves automatically. We did not have to manually intervene at any point to get the content visible on the website. What’s more, we measured the change and users were able to see updated content within 3 minutes - 10 times sooner than before.
It’s a huge improvement for users, meaning we can serve the most up-to-date content sooner. It’s an improvement in line with the GOV.UK vision: the best place to find government services and information. Simpler, clearer, faster.
Thomas and Elliot are developers on GOV.UK.
