Between 8 and 9 August 2017, there was a problem with all GOV.UK smart answers that ask users to select a country from a drop-down menu. This post explains what caused the issue and how we solved it.
We’ve blogged in the past about what happens when things go wrong on GOV.UK, and also how we classify and prioritise incidents.
What happened
On Tuesday 8 August, we updated Whitehall publisher - one of the publishing apps that content designers across government use to publish content to GOV.UK. The update changed the value of the ‘world locations’ format.
On 9 August at 10:14am during code review, we discovered that the country drop-down menu for selecting a country on the UK benefits abroad smart answer was empty. We then immediately noticed the same issue on the first question of the smart answer for getting married abroad.
The reason these smart answers were affected is that the value of the ‘world location’ format was changed from ‘world location’ to ‘world location news’ in Whitehall publisher.
This format provides information to applications we use to publish content about worldwide services, including the smart answers publisher. The change affected all smart answers that ask users to select a country from a drop-down menu.
What users saw
We estimate a total of 7 smart answers had empty drop-down menus:
- Check if you need a UK visa
- Help if you're arrested abroad
- Getting married abroad
- Register a birth abroad
- Register a death
- Report a lost or stolen passport
- UK benefits if you’re going or living abroad
The exact number of users affected isn't clear at this point. We are sure that users didn’t see any error messages, so the total number of hits that encountered pages with the an empty country drop down isn't clear.
How we responded
We reverted the change we’d made to the ‘world location’ format in Whitehall. As a result, smart answers drop-downs started to provide the list of countries again.
What we're doing to prevent this from happening again
After assessing what happened, we realised that our high-level ‘smoke tests’ (the tests we run after deployment) don't cover verification of the content in drop-down menus.
The test we carried out made sure that the ‘world locations’ update didn’t produce errors that would stop pages from fully rendering their contents; but it didn’t check the contents of other components that rely on services from the Whitehall app, like drop-down lists. This incident was similar to a previous incident involving smart answers and Whitehall publisher.
We have now included tests that make sure that components with external data are checked. We have also expanded our smoke tests to exercise all the parts of the worldwide API that smart answers rely on.
Our long-term plan is to remove the ‘worldwide services’ format from Whitehall, so we are looking at possibly migrating worldwide API endpoints to the content store (the main repository that holds all content items on GOV.UK). We hope this will allow us to handle any future service hiccups gracefully.
Ikenna was a developer on GOV.UK. Kelvin is a developer on GOV.UK. You can follow Kelvin on Twitter.
