Here’s an example of how a multi-disciplinary team of user researchers, performance analysts, content designers and developers work together to improve the user experience on GOV.UK.
In our last round of user research benchmarking we saw a worrying pattern of behaviour. When people reached a page with a ‘Start now’ button on it, instead of clicking this button they navigated to other pages about the subject.
These pages led them back to the ‘Start now’ page, where they would continue the pattern of ‘circular navigation’, becoming increasingly frustrated that they couldn’t complete the task.
Because we observed this behaviour in so many users in the lab, we analysed the data for these pages to see how common it was in real life.
We found that 1 in 3 people (177,000) looked at the ‘Start now’ page more than once, and 31,000 people saw it more than 4 times. This confirmed that ‘circular navigation’ exists and is a real problem. We also found feedback comments like, "your site sent me round in circles".
Usability testing is something we do all the time. It lets us ask users why they behave the way they do. In this case, they told us the words ‘Start now’ scared them - they thought it would start some process, rather than giving them a customised answer to their question.
So we decided to test whether changing the wording on the buttons would alter how many people clicked it.
We tested 3 different pages with ‘Start now’ buttons on them. We tested one more neutral and one more specific way of wording the button, as well as the original ‘Start now’ on each of the 3 pages.
We ran the test for 2 weeks to allow for variations in traffic on different days. The pages get high volumes of traffic so we were confident the results would give us some statistical significance.
The test pages
We tested 3 pages with the green 'Start now' button:
This table shows the change in how many people clicked on the button.
|Page||Button text||Click rate change|
|Overseas passport application||'Start now'||0|
|'Get application information'||+1.9%|
|Child maintenance calculator||'Start now'||0|
|'Estimate your child maintenance'||+3.5%|
|Getting married abroad||'Start now'||0|
|'Find out how'||-13.6%|
|'Get more information'||-18.8%|
The results instantly raise more questions. Why did the ‘Getting married abroad’ test produce such different results?
One possibility is that both variations make it much clearer what people will find when they click the button - just more information. So if someone was expecting more than that, clicking the button wouldn’t have helped them, and so the reduction in clicks is a good thing.
It’s also important to remember that the content of the page is most important. If the content is confusing or does not match the user’s needs that is likely to make more of a difference than the wording of the button. We think the content may be the relevant factor with 'Getting married abroad'.
What we learnt
The button text does make a difference
The differences between the click rates were statistically significant. They were not just random variation.
Don’t stop with just one test
Testing 2 variations each time demonstrated the risk of being satisfied with small improvements.
If we’d just tested ‘Get application information’ on ‘Overseas passport applications’ we might have thought a 2% increase was a strong signal. We wouldn’t have known we could get more than twice that increase with ‘Next’. It’s a great example of the ‘local maxima’ trap.
There is no magic button
Don’t jump to the conclusion that something which has one effect in one context will have the same effect in another context.
‘Get application information’ resulted in a 2% increase in clicks on one page, but ‘Get more information’ generated a 19% drop in clicks on another. The words are very similar but the results are very different. There is no magic button.
You never can tell
No matter how much experience and expertise you have, you can never really know which version will produce which result every time.
What happened after people clicked
As part of the analysis we also checked on the abandon rate after people clicked on the button. It would be bad if changing the button text lured more people into clicking it only for them to decide the information was not what they needed. We would’ve been wasting their time.
It turned out that changing the wording on the buttons didn’t alter the continuation rate.
The analysis also highlighted the difference in the rates between different subjects. We can see this easier if we look at ‘abandon’ rates rather than ‘continuation’ rates. The abandon rate for ‘calculate child maintenance’ was much lower than for ‘overseas passport applications’, even though the questions which follow are more complicated.
|Page||Button text||Abandon rate|
|Overseas passport application||'Start now'||11%|
|'Get application information'||11%|
|Child maintenance calculator||'Start now'||4%|
|'Estimate your child maintenance'||4%|
|Getting married abroad||'Start now'||6%|
|'Find out how'||7%|
|'Get more information'||7%|
This is a reminder to consider the context and the user needs. Someone trying to estimate child maintenance may be more motivated than someone looking for information about passport applications.
Of course we don’t know the context for any of the users - so it’s important to remember that the wording of the button may only be a minor factor influencing behaviour.
But what happened before people clicked
I’ve left the bad news until last. We intended to see if changing the button text would reduce ‘circular navigation’. Unfortunately, because of a problem with the tracking, the data on this was flawed. This question is going to have to wait for the next round of tests.
We’ll carry on testing.
In particular we want to re-test this same group of pages to check the effect of the neutral ‘Next’ button in other contexts.
As human beings we cling to the idea of finding something we could apply everywhere to improve user experience. When we do that we’ll also be able to test another set of variations to explore ‘Getting married abroad.’
We’ll also be able to investigate the tracking issues and have another go at reporting on ‘circular navigation’.
And of course we also want to carry on testing other things and other pages as well.
Tim Leighton-Boyce is a Senior Performance Analyst at GOV.UK. Follow him on Twitter.