During the recent GOV.UK firebreak, we made some improvements to info pages. These are the ‘shadow’ pages that exist for almost every page on GOV.UK. They show user needs for that page and metrics about how well the page meets those needs.
Info pages for the multi-part guides
The 650 multi-part guides account for more than 20% of unique pageviews on GOV.UK. By adding info pages, there’s now a lot more data available on these pages. You can now see the number of pageviews, searches and problem reports aggregated across all pages in a guide. For example, we now know that the Housing Benefit guide averages 2 problem reports a week across all its pages.
Fixed metrics for smart answers
There are around 40 smart answers on GOV.UK, and previously we only showed problem reports for the first page. Now we look at all the problem reports across all steps. So we now know that the page to check if you need a UK visa averages 75 problem reports a week.
Putting metrics in context
The metrics for the PIP checker smart answer showed 240 problem reports in six weeks. But we don’t know if that’s high or low, given that the page had 100,000 pageviews? We started comparing these numbers across the whole of GOV.UK, to add ‘very high’ or ‘high’ flags where needed. This will make it easier to spot pages that need work.
Identifying the most problematic content
Now that we know what pages users complain about, we should be able to come up with lists of the pages with the most problems. Interestingly, we found some smart answers, like UK visa and PIP checker answers, do have a high numbers of problem reports. We’re now looking into why that is, and how we can fix it.
We didn’t finish the last 2 things during firebreak, but we think they are important. So we’re planning on finishing them in the next few sprints. We’re also using these metrics as part of our internal performance reporting. Watch this space for news.
We’ll post soon on the GDS data blog about how we’re identifying problematic content.
8 comments
Comment by Lana Gibson posted on
This is awesome guys. Having an aggregate view of the performance of different pages gives so much potential for improving the site. Nice work.
Comment by John Ploughman posted on
Are there any plans to provide info pages for organisation homepages? It seems to be the one missing piece of the jigsaw.
Comment by David Heaton posted on
Prioritising page updates where there are most complaints is perhaps not always sensible. The information on NIC employment allowance that was recently changed to reflect a forthcoming change to the rules at 6 April 2015 is wrong, and employers who follow it are very likely to incur financial penalties levied by HMRC. Surely these page updates, where content errors can cost the citizen real money, should be prioritised?
Comment by Andy Keen posted on
Thanks for drawing our attention to this. We've now corrected the line in this content on the new rule for care and support workers (https://www.gov.uk/claim-employment-allowance).
Ongoing content improvement and iteration is a separate process to making factual changes, which we respond to and prioritise as soon as we become aware of them. There's more detail in point 19 of https://insidegovuk.blog.gov.uk/gov-uk-support-requests-our-processes-and-response-times/.
Comment by Martin Baker posted on
Hi where can one find background detail.
For example https://www.gov.uk/info/search-bankruptcy-insolvency-register shows a number of complaints received each week, but where do I find out what those complaints are. I receive an Anonymous feedback report, but this doesn't seem to cover these complaints.
Comment by Jake Benilov posted on
Hi Martin,
Apologies that you haven't been receiving feedback for that URL - today we found a bug with the script that generated the reports that meant that pages with the 'answer' format weren't being included in the report. That URL will be included in subsequent reports that you will receive.
In the meantime, you can use the Feedback Explorer (https://www.gov.uk/guidance/content-design/data-and-analytics#feedback-explorer) to see the feedback for individual pages.
Comment by Martin Baker posted on
Jake, thanks for that. Could you email me (martin.baker@insolvency.gsi.gov.uk) or explain here what, the fast majority of, results mean when they say "Action: bing. Problem: Bing" and no other detail.
Comment by Martin Baker posted on
That should obviously read "vast not fast"