Everything we do on GOV.UK is centred around providing the best possible experience for our users. We want the site to be fast and straightforward for everyone. To achieve this we need to understand how people use the site - this is where analytics are useful.
Data gathered by analytics helps us answer questions like:
- what are the most common tasks that people do?
- are people finding what they want?
- which parts of the site are slowest?
Understanding how people use GOV.UK helps us to learn what is working well and what isn't. We can then use this information to make the site better for everyone.
The only direct analytics tool we use is Google Analytics (GA). Google intends to replace the current version of GA (Universal Analytics or UA) with a new version in the near future. This means we need to upgrade to the new version or we’ll lose a valuable way of understanding GOV.UK users.
We’ve already talked about our preparations for upgrading to Google Analytics 4 (GA4). Now that we have successfully implemented GA4 onto GOV.UK, we wanted to explain:
- the technical side of the implementation
- what we’ve learned
- what’s coming next
What we’re doing and why
We wanted to implement GA4 using Google Tag Manager (GTM). Whilst we did consider alternatives, GTM came out as our best option. It allows for easier sharing of data collection methodologies, provides a preview and testing mode, and is already widely used across government.
As Daisy explained in her earlier blog post, we’ve decided to run both the old and new versions of GA together on GOV.UK for 12 months. We want to test everything is working, compare the data and understand any discrepancies, and to give users and departments time to migrate. If we just flipped from UA to GA4 we'd run the real risk of not discovering any issues until it was too late.
This is a huge change for GOV.UK, and we didn’t know the full effect of running both analytics products together. We knew this would place a small additional burden on the site - both in terms of the amount of JavaScript needed and the performance impact of this extra code - but we wanted to understand the specific impact and be prepared in case it was deemed unacceptable.
What we’ve learned
Early on we created a test site to measure the size of the JavaScript that has to be included in order to use GTM.
We found that changes made in the GTM interface can increase or decrease the size of the JavaScript. Disabling enhanced measurement - a feature we’re not using - slightly decreased the size. By contrast, we were able to increase the size significantly by adding custom HTML tags. This was achieved by adding deliberately bloated code that would never be added in reality, but suggests what could happen over time without care.
The best size that we could achieve was around 99 KB, which was nearly 5 times the size of the JavaScript needed for UA. This was concerning, but didn’t provide information about the real impact for users, since JavaScript can be loaded separately while the rest of the page is loading.
GOV.UK is already fast to load, so users with modern devices on good connections probably won’t notice any difference. We’re more concerned about the impact on users with older devices on slower connections.
Plans for going live
GOV.UK uses Real User Monitoring (RUM) to help us understand how the site performs for users. It only works on the live site, so we couldn’t properly test the performance impact until we put the new code live.
We discussed the potential performance impact of dual-running with a range of people across GOV.UK, including developers, analysts and senior management. We had to communicate clearly what this meant, and have plans in place if the performance wasn’t acceptable.
We decided to build a minimum viable product using GTM, consisting of a few key parts of our analytics, and put it live on GOV.UK for 2 weeks. We would measure the performance of the site but be ready to go with an alternative to GTM if it wasn’t acceptable. Our alternative was the Google tag, which has a slightly smaller JavaScript size and works in a similar way to GTM, but lacks many of its features.
The performance impact
We monitored the performance of 5 of the most visited pages on GOV.UK for 2 weeks using RUM. We focused particularly on older devices that would be more likely to notice a performance change.
Some metrics increased as expected, such as the overall JavaScript size and the total time for the page to load. The good news was that, although these metrics increased, it had little to no impact on how fast the pages appeared to load (perceived performance) for users or how soon they were able to interact with them.
Interaction time and perceived performance are important as they let the user know that the page is now in a usable state. Even a 50 millisecond difference in the performance of a page can make a page feel slow to a user, especially on low specification devices, which will likely feel the impact a lot more than faster devices.
Using RUM data we were able to show that the impact of GTM was negligible. This confirmed our initial thinking and meant we can now proceed to use GTM to implement GA4 without an impact on our users.
Managing GTM
We’re also doing all we can to ensure that nothing unnecessarily inflates the GTM JavaScript size. Although GTM can give analysts a lot of control over tracking, we’re going to use it in a very minimal way.
Once the initial setup is complete, any further changes made in the GTM interface must be approved by a developer before being published to the live site. This will include checks to ensure that nothing increases the size of the GTM JavaScript unnecessarily.
We’ve also implemented a blocklist in our code to prevent any custom HTML or scripts from being added to GOV.UK. The blocklist only stops these things from executing - they are still downloaded as part of the GTM JavaScript. This means the size could still get out of hand if not monitored properly.
What’s next
Since the launch at the end of September 2022, we’ve been continuing to add the remaining analytics to our GA4 setup, as well as testing and adjusting the existing work. This is a large task and we have limited time to complete it before UA is shut down in 2024.
We’ll continue to monitor the performance of GOV.UK as the work continues. We’ll also continue to ensure that all of the data collected fully respects people's consent, is anonymous, and is free of personally identifiable information.
Subscribe to Inside GOV.UK to keep up to date with our work.
If you work in government, join the Performance Analyst Basecamp where we will be posting regularly (information on how to join is on the performance analysis community page).
8 comments
Comment by Janet Dyson posted on
I was interested to receive this today, having just upgraded some sites to GA4 using GTM this week. I'm hoping you will write another blog when there's been time to build some data comparing the current GA version to GA4, as I would be interested to know how other people are getting on with the differences.
Comment by Will Huthnance posted on
Out of interest, are you folks also using Google Signals as part of your analytics?
Comment by Andy Sellick posted on
Hi Will, we’re not, no. Signals looks like it’s more geared towards sites with advertising, which GOV.UK doesn’t have.
Comment by Simon Clayton posted on
Thanks for the useful update. In the linked blogpost from Daisy Wain, it suggests that you intend to share the tracking schema and DPIA with gov departments. Is this still the intention following your period of testing?
Comment by Andy Sellick posted on
Hi Simon, yes it is. We’re still working out how best to do that and the work is ongoing. In the meantime if you have any ideas or suggestions then please do get in touch.
Comment by Jeremy posted on
Thank you for another update. Looking forward to more sharing.
Comment by Simon Williams posted on
Looking forward to following this story. We've already implemented Google Tag manager and are tracking a number of our websites, but the explorations are a whole new world. Interested to know if you have noticed discrepancies between GA4 and Universal Analytics data so far.
Comment by Andy Sellick posted on
Hi Simon,
Thanks for the comment. We’re still looking through the data and have found some discrepancies, but it’s a bit too early to give our findings. If you subscribe to this blog at https://insidegovuk.blog.gov.uk/subscribe/ you’ll get a notification next time we publish an update.
Thanks, Andy