This post is about a recent incident on the GOV.UK website. We routinely publish incident reports because we believe we should be open about our mistakes and share our learning. For more information on incidents on GOV.UK, you can check the status of the GOV.UK site or read more about what happens when things go wrong on GOV.UK.
Background
On GOV.UK, there are several ways that users can sign up to receive email alerts when a document is published or updated. Our most popular method is via the GOV.UK Announcements page where users can create email alerts that suit their needs, including filtering by policy area. Another way is to sign up for email alerts relating to specific policies through the Policies page. This incident report is specifically about email subscriptions created through the Policies page and does not affect the more commonly used Announcements and Publications pages, travel advice or medical alerts.
What users saw
Between May and November 2015, emails weren’t sent when documents relating to policies were updated, if the user had signed up to receive email alerts through the Policies page. However, if users were signed up to receive emails through the Announcements or Publications pages on GOV.UK then there was no impact and they received any updated documents regardless.
These documents were available on GOV.UK as normal, and could be seen by visiting the policy page, looking at the page’s RSS feed or on the announcements page on GOV.UK.
All other email alerts, including those for travel advice, medical alerts, announcements and publications, were sent out as usual. This issue was limited specifically to users who signed up for email alerts from a policy page.
Cause of the problem
In the run up to the 2015 general election, GOV.UK developed a new policy system which was in use from 8 May 2015. This system was designed to trigger email alerts when policies changed using a GOV.UK-built service called 'email-alert-service'.
However, when we were doing some other work on the email-alert-service, we discovered that the service ignored messages for content which wasn’t tagged to a topic. This meant that unless a policy document was tagged to a topic, subscribers to the policy alert didn't receive emails when policies were published or updated.
How we are preventing this from happening again
We fixed this by making a change to email-alert-service so that it checked for content tagged to policies. This was done on 17 November 2015, and all users subscribed to policies started receiving email alerts when those policies were published or updated.
This bug was hard to spot. There was nothing in routine contacts from our users to indicate that there was a problem, and the pre-election period meant that the rate of publishing on GOV.UK, and therefore emails being generated, was lower than normal making it very hard to proactively spot problems. We are prioritising work to review and improve how emails are sent from GOV.UK and we are also planning to look at our monitoring systems to what improvements we can make.
