Back in October, Ben and Henry posted about planned changes to the Business and self-employed browse page. Testing revealed that on some things we achieved our goals - but in others we didn't, and we'll need to iterate further. So here's quite a #longread about how digital analysis and user research explored user behaviour in some depth.
What the changes to the Business browse page were
The main changes were:
- Replacing the page description with shortcuts to popular pages, to get the majority of users to their destination faster.
- Changing the categories list to a single column and removing the descriptions. The goal was to make them easier to scan and review.
- Introducing highlighted content areas. The goal was to support users exploring areas where they may need a range of content.
As the changes were deployed, we used digital analytics and user research to understand the impact of the changes and whether our goals were met.
The answer is a conditional 'yes'. The redesign has been effective in getting users directly to the popular pages and to the highlighted section, but user research sessions showed people focusing on the popular pages section and not exploring further down the page.
How the page performed originally
We began by looking at how the Business browse page performed in September, before any changes were made.
From this original page, 15.2% of traffic exited GOV.UK and 84.8% visited another page.
The chart shows the 15 most popular links. Setting up a business browse page and the homepage were most popular, with nearly a third of all click-throughs. Note that the original browse page only had links to other browse pages, to the homepage and search.
The top site search terms made from the business browse page were: self employed, sole trader, jobs, redundancy and self assessment.
Measuring the effect of changes
The team tested and measured the page over several iterations, but as each iteration was implemented quickly we only had short time periods to compare data. The following data is for the 20 days following the implementation of the final version, which provided a longer comparison.
From the ‘new’ Business browse page, 14.0% of traffic exited GOV.UK and 85.0% visited another page; so the exit rate fell slightly.
The chart here again shows the 15 most popular links. The most popular link was direct to Starting up a business, with the homepage still second, but less important.
Mapping these most popular links to the page layout shows that the redesign has been effective in getting users directly to the popular pages and to the highlighted links sections:
We were also interested in what users do after selecting one of the links on the /business page. Initial analysis shows that the the links at the top of the 2nd level browse pages perform better than those lower down. Links in the Get started in business section performed quite strongly too.
We also checked for any differences in click-through popularity for tablet and mobile users. As the table below shows, there were few:
Using qualitative research to understand the analytics
The data gathered during the trial showed a small uplift in the number of click-throughs to the most popular link - starting up a business, and a reduction in the exit rate.
To give us greater confidence in interpreting the data, we also carried out user research so we could understand why users were clicking on these links.
We carried out research with 19 participants in total, asking them to complete tasks based on analytics data. So we asked users to find business content that we already knew was popular on GOV.UK were looking for. We saw consistent patterns of behaviour across participants.
Are the 'popular pages' too popular?
The most striking observation was that participants were not interacting with the whole Business browse. Instead, we saw participants repeatedly selecting one section of the page and then using that as the starting point for the majority of the tasks.
The section that most participants ‘chose’ at first, was the Popular Pages, and we saw a number of participants who failed to scroll down to the other sections until they were well into the session and it was clear to them that the Popular Pages links wouldn’t suffice as starting points for all the tasks that they had been set.
We were testing the page at the most popular screen resolution (1366x768), and at this setting, the Popular Pages links take up the entire screen above the fold. If we think about this in relation to what we saw in the analytics data, we could make some hypotheses:
- that the uplift we saw in the number of click-throughs was due more to the fact that the Popular Pages are at the top of the page and take up the entire screen above the fold, rather than the success of these links in meeting user needs
- based on the lack of scrolling that we observed, that users did not realise that the Popular Pages were only the top 4 most popular links, but interpreted these as the main route into Business content
- that the number of click-throughs to ‘Starting up a business’ and ‘Growing your business’, which appeared were the 3rd and 8th most popular links in the analytics data, were influenced by the visual design - appearing more visible to the user in their vertical line of sight as they scanned down the page
The task completion rate was highest for those participants who were using the A-Z list as their starting point, but we also observed users repeatedly only scrolling halfway down the A-Z list, suggestion that the scannability and length of the list were barriers to engagement. Participants also commented on the lack of congruence between the links in the list (e.g. Maritime vessels and work at sea vs. Selling your business and closing down)
Next steps: iterate and retest
So we will revisit the design concepts to try to better distinguish the three sections and encourage exploration of the whole page. Of course, we’ll do further research that allows us to assess, evaluate and iterate the design.
We plan to test test new designs with content from other areas as well as Business, so that we can distinguish design issues from those to do with the information architecture and content about business.
From an analytics perspective, we want to measure the ‘quality’ of interactions of visitors arriving through the Business page. How will different designs of the Business page impact on how users get to and engage with content after they have followed a link?