This post outlines a recent production issue on GOV.UK and how it was resolved. We’ve blogged in the past about what happens when things go wrong on GOV.UK, and also how we classify and prioritise incidents.
On the afternoon of 5 January 2017, we upgraded to a newer version of jQuery, which briefly caused some GOV.UK pages to fail to display content.
When a library is updated, the maintainers of that library will assess the potential damage of their changes. If the change is very minor, it is considered ‘non-breaking’ and therefore unlikely to break anything on sites which include it. Although this update in jQuery was documented as non-breaking, it caused a breaking change in the jQuery Tabs plugin that we use on calendars and transaction start pages.
What users saw
The most affected pages were:
The /bank-holidays page, for instance, failed to show the upcoming bank holidays for the different parts of the UK.
Users should have been presented with the dates of upcoming bank holidays:
How we responded
We rolled back the jQuery upgrade in the Static application to the previous jQuery version. Static is where global templates (headers, footers and components that are common to a lot of pages) are defined for GOV.UK pages.
We make use of caching on GOV.UK - once a user has visited a page, content will be stored in memory to prevent the browser from having to re-request it from the server. The benefit of this is that the page should load faster.
After deploying the rollback of Static, we purged the most affected pages from our caches so that users would see the fixed version straight away.
What we’re doing to prevent this from happening again
We are investigating how to integrate visual regression testing into the deployment process for the Static application. Visual regression testing will generate screenshots of webpages on different environments (in this case staging and production), with the differences between them highlighted. This provides a mostly automated way of telling if something doesn’t look quite as expected.
Rosa is a developer on GOV.UK. You can follow her on Twitter.