https://insidegovuk.blog.gov.uk/2015/12/09/using-analytics-within-product-teams-at-gov-uk/

Using analytics within product teams at GOV.UK

In the Core team (one of 4 product teams at GOV.UK) we look after the development and maintenance of many applications behind GOV.UK, including Whitehall Publisher.

We collect lots of data about the usage of GOV.UK and how it performs technically. It helps us do things like assessing development requests, amongst other things.

We’ve identified a need to be more structured in our use of this data, so we've created a simple analytics report. It shows activity in the apps, focusing on how they are used by government publishers.

This is the first time we've collected and interpreted data in such a consistent way for GOV.UK’s publishing tools.

What are publishers doing?

Our first report was really simple and showed who was using the different apps. We produce this weekly so we can start to understand things like:

  • how many people use each app every day
  • what the busiest day of the week is
  • the busiest time of day for publishing activity

Over the last six months, the busiest day and time in Whitehall Publisher by number of users, was between 11am and midday on Thursdays.

Analytics dashboard
A section of the Core team's first analytics dashboard.

This first report helps us understand how the apps are used, and plots activity against longer term trends. Some changes will be seasonal, but this information can also be used to help us do things like deploy code at low impact times, or anticipate an increase in support ticket requests if we spot an increase in new users.

Creating a team dashboard

The next iteration of the report, which we're working on now, will continue to report on the usage of Whitehall Publisher by publishers, but it will also provide information on activity by the public and performance of the apps. That’ll be things like the technical speed of delivery (how fast a page loads), or 404 error pages (how many, and of what type), for example.

The aim of collecting this additional information is so that we can:

  • identify problems before they are reported by users
  • validate or challenge requests for development or change
  • inform future work

Brad Wright has already written about our focus on improving the architecture of GOV.UK and this report will be useful for showing if we are making performance improvements through the architectural migration too. For example, are we seeing an improvement in page load speeds, meaning publishers and the public are waiting less time for information?

Analytics dashboard version 2
Notes on the analytics we want to track in the second iteration of the Core team dashboard

We have already started to make observations from data that we want to collect in the next dashboard. For example, on average, there are 74 page views of Whitehall content by the public for every page view in Whitehall by a publisher.

Setting a benchmark

Once we’ve collected data for a few months, the next stage will be agreeing team metrics that we can measure our performance against.

Once we understand what the regular activity of ours products is, we can measure the success of the work we do, and allow us to track changes in user behaviour too. This is important for team motivation because we can see how usage changes dependent on what we do. It also keeps us focussed on the team priorities.

Focus

We're keen to produce a report that only contains information that can affect the work we do within the Core team - we don’t want to get bogged down in data that doesn’t help us.

The report should provide just the right amount of information so that if there are issues or points of interest we can delve into the data. The team dashboard will be something that is quick and easy for us to look at every week.

We'll keep sharing what we learn from this work in future blog posts.

Jennifer Allum is the GOV.UK Core Formats Product Manager. Keep in touch: follow Jennifer on Twitter and subscribe to email alerts from this blog.

2 comments

  1. E. Brown

    Hi Jennifer,

    Thanks for this post. It's a part of the service I don't know much about.

    In this context: what is an 'app'? b'c I think of apps as icons that appear on my phone. I'm guessing you mean something different.

    You mention products too:

    "Once we understand what the regular activity of ours products is, we can measure the success of the work we do, and allow us to track changes in user behaviour too."

    Is an app the same as a product?

    RSVP, cheers,

    E. Brown

    Link to this comment
    • Jennifer Allum

      Hi Elizabeth,

      Thanks for pointing out the need to be clearer on the definition of applications and products.

      In this context, by app I mean a piece of software. That application may also, in technical terms, be the product. In the case of GOV.UK though, our product consists of multiple applications, which we use in combination (like components) to manage parts of GOV.UK.

      Thanks,
      Jennifer

      Link to this comment