In the first blog post we covered "How and why we removed jQuery from GOV.UK", let’s now turn our attention to what performance difference it has made for users. Please read that blog post before this one.
Image data versus compressed data
We will focus on these users for the rest of the blog post.
Helping users with low specification devices
Lower specification devices are often considered to be older laptops and tablets, or ‘budget’ mobile devices (often Android devices on a limited data plan).
Since these users have the slowest devices, they’ll need the most help to make sure their visit to GOV.UK is as fast, efficient and as cheap as possible. Last October, Ofcom estimated 2 million households were "experiencing affordability issues with either their fixed broadband and/or smartphone", so our work here has the potential to reduce their internet costs.
At this point, it’s worth mentioning that the anonymous Real User Monitoring (RUM) data we collect from user devices shows that GOV.UK is already a very quick website. For the majority of users, the site loads in less than 1 second. But the distribution graph (seen above) also shows that there are some users that see a page load time of up to 2.35 seconds. Examining these users’ RUM performance data more closely, we see:
- 75% of users are based in the UK
- 35% of users are on an Android device
- it’s taking almost 2 seconds for the first page pixels to display on their screens (this is called the start render metric)
From the statistics above, we can assume it’s highly likely that many of these Android users are on low specification devices where CPU speed and memory capacity is limited.
Testing the impact
This is where our “lab-based” performance testing is very useful. We run tests every day on GOV.UK pages using specific simulated devices and connection speeds. Because we can repeat these tests every day, it allows us to monitor what the changes we are making to the site are doing for real users visiting GOV.UK.
For example, for a simulated user visiting the Universal Credit start page on a low specification device and 2G mobile connection, we can clearly see from the graph where the jQuery change was made.
The reason why the Universal Credit page was chosen is because it’s the most visited page on GOV.UK according to our analytics data, so it is important that it loads quickly for all users, no matter what device or connection they are using. As the graph above shows, the time it took for the page to completely draw the pixels to the screen (visually complete) dropped from 11.3 seconds to 9.4 seconds (a 17% improvement).
As well as visually complete Improvements, page load time improvements were also clearly seen. The graph below shows the time until the page has fully loaded dropped from 20.42 seconds to 18.75 seconds (an 8% improvement) and total page load time dropped from 19.44 seconds to 17.75 seconds (a 9% improvement).
Finally, we also see a significant improvement in the page's interactive performance metrics, meaning a user can interact with the page a lot sooner. Interactivity metrics are important to users as they show when the user can first interact with the page. Being able to interact with a page gives users confidence that the page is still working.
The graph below shows time to interactivity dropped from 11.34 seconds to 9.43 seconds (an improvement of 17%), and the devices First CPU idle dropped from 11.32 seconds to 9.42 seconds (an improvement of 17%).
Improving GOV.UK for everyone
Improving web performance is often made up of lots of smaller incremental changes over time. So it’s been great to see support for this change from the wider web performance community.
If you’re interested in this work, why not join the team? Apply to work at GDS
Comment by Barnok Balavan posted on
Honestly, I think the whole case for removing jQuery is overstated.
It's a tiny library that allows developers to write very terse code that overcomes browser differences without issue. The tests quoted are unrealistic worst case scenarios: "The graph below shows the time until the page has fully loaded dropped from 20.42 seconds". Nobody waits that long. Those numbers sound cooked, frankly.
Comment by Jon posted on
"For example, for a simulated user visiting the Universal Credit start page on a low specification device and 2G mobile connection, we can clearly see from the graph where the jQuery change was made."
Comment by Dave Howes posted on
There speaks a man with a fast device and connection who's utterly failed to understand any of the page he's just read :~)
Comment by Mr Oz posted on
Your privilege is showing the "cooked" data is out there: https://httparchive.org/reports/loading-speed#ol
Read the comments:
"3g would be good to have in some parts of the country!!!"
"I still have to walk around my Essex garden trying to obtain a phone line let alone 5G."
Comment by CK posted on
@Barnok: Oh dear, you missed the point.
@Gov.uk: Good work guys, I roll my own and it makes a noticeable difference on older devices. Plenty of people still using them.
Please work on streamlining the self assessment tax returns next, I hate them 🙂
Comment by CK posted on
hmmm, seems comments are removed even if they support gov.uk...weird
Comment by Ka posted on
Does this page play a “ding” at the end? it really scare me, I am try to found out if my glass fallen, LOL.
Comment by Mads posted on
It would be interesting to know if asset caching or deferred loading was used, prior to ditching jQuery, and/or if the daily tests ran in a no-cache configuration.
Disabling the cache will give you the worst-case scenario, you're probably looking for, but it also does skew the results a bit.
It's not really obvious, from the little info we get about your RUM data, if the 17% with the "bad experience" are all just first time visitors or if they get the same experience on repeat visits.
Otherwise interesting post(s) on real-world improvements.
Comment by Matt Hobbs posted on
Hi thanks for your questions,
For the daily synthetic testing by default nothing is cached between tests, this is the default for WebPageTest on which the synthetic tests are built, unless you set it up for repeat views with cache persistence, the idea with these tests is to identify worse case scenarios given how aggressive browser caching can be on resource limited devices I'm not sure how long assets would be kept in their cache, but I do agree it will likely skew the results.
I know Facebook and Yahoo studied cache efficiency on devices / browsers back in 2015 (https://engineering.fb.com/2015/04/13/web/web-performance-cache-efficiency-exercise/) and found the browsers at the time to be very aggressive with cache life, I'm assuming this will likely still be true for low spec devices (I'd love to see an updated study for 2022).
The 2G question is an excellent one and something we are working on at the moment. As GOV.UK has a number of FE rendering apps, a user's journey across the website involves multiple rendering apps which don't always use the same assets for each page. This means that it's currently likely a user is always going to be in a position where there is something not sitting in the browser cache, and will need to be loaded on the next page (we know not ideal at all, but architectural constraints from years ago are still a limitation with this).
For the RUM data question, the RUM tool we use gathers anonymous data across a user's journey and then splits them into the load time histogram you see above. This means we cannot determine if the visit is a new or repeat visitor - details are in GOV.UK’s Privacy Notice (https://www.gov.uk/help/privacy-notice). What we do know is that 17% of users do get a bad experience, so let's concentrate on trying to improve that statistic and fix it for both new and returning visitors.
I hope that helps answer some of your questions.
Comment by dfl posted on
This is a wonderful analysis of a case where jQuery was just sitting idly in the page.
However, for pages that actually *use* jQuery, the question is - what was it used for and what alternative has been deployed.
Not to sound dismissive - this post is very useful for web authors who include jQuery, or any other asset, for religious or other non functional reasons. This is obviously the common case. However, for the tiny minority that are including libraries and actually *using* them, the value of the post is quite limited.
Comment by Andy Sellick posted on
Hi thanks for your questions,
Comment by Nige posted on
This article is misleading as it uses 2G speeds for the final comparison. The actual benefit to the slowest-loading of your visitors is more like 0.2 seconds rather than 2 seconds, while increasing pressure on your web developers who can no longer use the convenience functions jQuery provides.
Comment by Proteus posted on
The article writes: "But when it’s on every page, from a web performance perspective, it equates to a lot of data".
Each script is downloaded once. It's not a lot of data, it's a single download, and then it's cached in memory.
Further, when you remove jquery, you have to replace it with something else and refactor the entire site - because (guess what!), jquery was used by other scripts. So the comparison is rather tricky.
To put it mildly, this whole article is not very scientific.
Comment by Mr Oz posted on
Why would a team want to continue using a 3rd party with a known security issues? It's technical debt at the end of the day, Unless they just leave everything in there forever?? That sounds like a terrible idea!
Well done to the team working on this! The privilege of some people on some of the comments in this post is really terrible. Empathy for others at all??
As I mentioned in another reply feel free to read comments from this article: https://www.bbc.co.uk/news/technology-48458280#comments
Some folks in the country can't even get a stable 3G connection let alone high speed internet, gov.uk has to cater for everyone, not just people on high speed connections and the latest mobile device. I'm sure many people are very happy they can get to gov.uk content on low specification devices!
Comment by ManoDestra posted on
jQuery was fine back in the day when browser compatibility was a serious concern, but this is no longer the case. That, along with the security aspect and the additional skill requirement, are all great reasons to ditch it along with the obvious performance benefit especially on low end devices.
We made the same change recently at my company and haven't looked back (despite some initial resistance). So much better off without it. Good riddance, I say, and roll your own!
Comment by Hahah posted on
Comment by Dave Methvin posted on
Something is strange with the scripts on, for example, the http://www.gov.uk site. There is a lot of duplicated code in the two files loaded at the bottom of the page, and it seems to include a shim that mentions IE8! Whatever bundler being used is pulling in the same code several times. The site would be faster if all that duplication was eliminated.
Comment by Matt Hobbs and Andy Sellick posted on
Thanks, Matt and Andy