If users consent to cookies on GOV.UK we can use Google Analytics to help improve our services and content for the millions of people that visit every day. We look at data on things like page views, link clicks and some element interactions, such as when a user clicks something to show or hide text.
For some high traffic pages, we also use scroll tracking. This blog post is about how we identified some problems with the existing scroll tracker and went about building a new one.
What is scroll tracking?
Scroll tracking is commonly used to show how much of a page has been read. Our scroll tracking can either tell us how many users have scrolled a certain percentage of the page or which headings they’ve reached.
This data can help us identify problems with content - for example, is the whole page being read, or just the first part? Are users missing out on important information that could be positioned more prominently? Do users engage differently on desktop and mobile?
Problems with the existing scroll tracker
We were looking at improving our foreign travel advice pages and adding scroll tracking to get more usage information was a sensible first step. Unfortunately this highlighted some problems with the way our current scroll tracker worked.
In order to add scroll tracking to a page on GOV.UK it was necessary to add the URL of that page into the scroll tracking script itself. There are 226 countries and territories listed on the foreign travel advice page and each one consists of up to 10 different pages, covering topics including security, health and money. That would mean adding almost 2,260 URLs to the JavaScript. This might sound reasonable except that this script is included on every page on GOV.UK, even the pages without scroll tracking.
Another problem was that to track headings on the page, the text of those headings also had to be included. Since the heading text had to be hard-coded in the JavaScript, if the content changed, that heading would no longer be tracked.
Some of the travel advice pages have 8 or more headings, which could increase our initial figure of 2,260 additional lines to nearly 10,000. Adding this to the JavaScript would significantly slow down every page on GOV.UK for users.
This was an untenable way to do scroll tracking both now and in the future, so we started thinking about how to make a new scroll tracker that would use a different approach.
How does scroll tracking work?
When you scroll a page in your browser, a scroll event occurs. JavaScript can be written to listen for that event and carry out instructions when it happens. This means that when the page is scrolled, the code can check which headings are currently visible within the window, and collect that information.
Determining whether an element like a heading is currently visible in the window or not is computationally expensive, and scrolling even a little way down a page can cause the scroll event to happen hundreds of times. Combined without consideration, these things could make scrolling down a page on GOV.UK unbearably slow for some users, which we had to avoid.
Modern browsers can use Intersection Observers to get around this problem, but we couldn’t use them because we need to support old browsers like Internet Explorer. A polyfill was considered, but that would add too much extra JavaScript.
Instead, when the scroll tracker starts it begins by storing the positions of headings (and percentage positions) and using those numbers inside a throttled scroll event listener to reduce the load. Our new scroll tracker worked, but there were still some problems that we had to consider.
Problems to solve
If we pre-determine the position of things on a page (headings or percentage positions), we need to take account of the fact that the height of a page (and therefore the position of those things) can change after the page has finished loading. This can happen when the browser window is resized, or if a device is rotated, or if the font size or zoom level in the browser is changed. It can also happen when the user interacts with the page, for example by expanding a collapsed element or dismissing the cookie banner.
We only wanted to track headings in the page content, not headings in the footer or other common elements. We also needed to consider that headings might appear or disappear after a page has loaded, for example if they are inside collapsible content like an accordion.
We wanted to make sure that tracking occurs for any headings that are visible when the page loads, before scrolling occurs. This might be a different set of headings depending on what size device or browser window is being used. This would also be a different set of headings if the user was following a jump link to a point on the page that wasn’t the top.
Finally, for both heading and percentage tracking, we didn’t want to track things more than once, so we needed a mechanism for remembering what had already been tracked.
Our new scroll tracker
We’ve built a new scroll tracker that considers all of the problems above. It doesn’t need URLs to be included in the JavaScript, but runs based on a simple meta tag added to specific pages. It detects headings and their text on page load, and determines whether or not they should be tracked based on whether they’re inside the page content.
It also detects changes in page height and resizing of the browser, and recalculates everything appropriately. Finally, it uses fewer lines of code than the list of URLs to track in the old scroll tracker.
We successfully tested our new scroll tracker on the foreign travel advice pages and discovered that some users were missing important information, which led to some content changes. We then carried out a complete review of scroll tracking across GOV.UK. This resulted in the removal of scroll tracking that either wasn’t needed or wasn’t working properly, as well as some further improvements to the new scroll tracker.
We’ve since fully migrated to the new scroll tracker and retired the old one, which has led to a small reduction in our overall JavaScript size and means that scroll tracking can now be more easily updated when required.
5 comments
Comment by Dale posted on
This sounds really cool. Is this something that's happening elsewhere or is it unique to GOV.UK?
Comment by Andy Sellick posted on
Thanks Dale. This piece of work only applies to some pages on http://www.gov.uk, but scroll tracking is quite common on the wider internet - the latest version of Google Analytics has basic scroll tracking available as a built-in feature, for example (https://support.google.com/tagmanager/answer/7679218?hl=en).
Thanks, Andy.
Comment by Allan Keys posted on
NICE!
Comment by Will posted on
This is really interesting! Do you guys have any inclination to move away from Google Analytics to something less invasive (In regards to privacy) - possibly something hosted by GOV.UK?
Comment by Tina Mermiri, Head of User Data and Insights at GOV.UK posted on
Thanks Will! We understand the concern and user privacy is really important to us. We have conducted a lot of work on our GA implementation to reduce the potential privacy risks and consequently we intend to continue using Google Analytics.
For example, our cookie banner gives clear accept/reject options, and unless this is accepted none of the Google Analytics code is executed. If users initially accept and then later change their mind we provide a clear way for them to opt out at a later date.
We also take measures to ensure the data we collect with GA cookies does not directly identify any users. This includes redacting any personal data from URLs / page titles, anonymising IP addresses and clearly stipulating in our Privacy Notice that we do not combine GA data with other data in a way that could identify users.
There’s more information in our privacy notice here: https://www.gov.uk/help/privacy-notice. Thanks, Tina