We’ve made quite a few improvements and fixes to the GOV.UK search engine this year, even though our main focus has been on improving navigation and how content is organised.
Many of these changes to search are behind the scenes and not very visible, but solve some longstanding annoyances. Looking back at what we’ve achieved this year, I’d like to highlight some of these less obvious improvements.
Searching for an exact phrase
You can now use double quotes to search for pages that contain an exact phrase. A few thousand people each month were already trying to do this, as it’s a familiar technique used in other search engines like Google.
To show what a difference it can make, take the current most popular exact phrase search. Without double quotes, searching for ‘national service frameworks’ gives you 19,521 results containing some or all of those words. Narrow it down to the exact phrase "national service frameworks" and you get just 11 results.
We still use 'stemming' in these phrase searches, so the results may contain different forms of a root word: singular or plural, nouns or verbs, past or present tense, etc.
At the moment, you can only use one exact phrase as the search query, with the double quotes at the beginning and the end. You can’t yet combine multiple phrases, as that would involve more complicated technical work.
Better spelling suggestions
The original ‘Did you mean’ suggestions used a standard English dictionary, so they’d sometimes offer irrelevant corrections for valid searches, especially for names or acronyms that aren’t in the dictionary.
For example, someone searching for ‘Falkirk’ was asked ‘Did you mean flakier?’. Searching in Welsh for ‘pleidleisio’ (which means ‘voting’) resulted in ‘Did you mean lidless?’. A search for ‘apostile’ used to suggest ‘apostle’ rather than ‘apostille’ (a legalised document).
We’re now using the actual content of the GOV.UK site instead of a generic dictionary, so you’re less likely to see unhelpful suggestions when you search for government-specific terms or other words that are used on the site. And when people do misspell or mistype something, the suggestions should be more appropriate - though they’re still not always right.
Curly apostrophes
Straight and curly quotes or apostrophes are now interchangeable in search. You might take that for granted, but for technical reasons our search engine used to treat them differently.
That meant some content didn’t appear in search results when it should have. For example, if you typed in a search term with a straight apostrophe, but the page you were looking for had been copied and pasted from a Word document with smart (curly) quotes, the search engine couldn’t recognise it as a matching result.
We’ve fixed that by treating all the variations as a plain apostrophe in search. So now, all the relevant content is returned, regardless of the exact type of characters used in the search query or the content.
Fewer results per page
Earlier this year we reduced the number of search results from 50 to 20 per page. From looking at analytics, we could see that 95% of clicks were on the top 20 results, so the rest of the page was just overload that slowed down most users.
Strange as it now sounds, originally there was no ‘next page’ of results. When GOV.UK had much less content, you just got a single page of up to 50 results. We got round to adding pagination in 2014 as the site grew larger and some types of users did need to look through more than 50 results.
There is a trick for advanced users who’d prefer more results at once: you can add ‘&count=100’ to the search URL to see up to 100 results per page. For example: https://www.gov.uk/search?q=tax&count=100
Highlighting search terms
The words you searched for are now highlighted in bold if they appear in the page descriptions. This should help users to see the context more easily, and to identify whether the results are likely to be relevant.
We tried a few different designs but some were too intrusive, so we settled on bold. This is similar to Google’s search results, but we’re still using handwritten summaries rather than automatic extracts, so you won’t see the words in their actual context within the page.
Decluttered search results
We’ve stopped displaying a 'breadcrumb' on search results for mainstream services and information. It used to show the browse category that the content belonged to, but that wasn’t particularly helpful for users, and sometimes it was downright confusing.
To save space, some of the category names had been shortened, so for example when you searched for ‘Housing Benefit’ the breadcrumb said ‘Benefits > Heating’ rather than the full name ‘Benefits > Heating and housing benefits’. Rather than expanding these, we decided to try removing them altogether.
We also took away the grey box around external links to related websites (such as NHS Choices and the National Careers Service). Those results stood out too much, and users either thought they were special promoted items, or ignored them like adverts. Now the external links look similar to other results.
What’s next?
There’s plenty more that we can do to make searching GOV.UK better. The wider work on developing the topic hierarchy will help group and filter search results better, and in the meantime we’ve started work on some ranking improvements to the search algorithms. Stay tuned for more news from the Finding Things team in 2016.
5 comments
Comment by Ady Garrett posted on
GOV.UK still needs to be more intuitive. For example, when tested (November 2015), entering “taxcode” as a search term brings up no results, yet by adding a space and entering “tax code” as a search term fetches 1,112 results. A better search facility would have been presented this when the term “taxcode” was used.
Comment by Alan Cooke posted on
Sadly when I used the phrase "Home Office counting rules' in the GOV.UK search I got everything but the counting rules. The same phrase in Google and the page I was looking for popped straight up at the top of the search list.
Comment by Tara Stockford posted on
Hi Alan
I've just seen your comment. I'm not sure what happened there, but the 'Counting rules for recorded crime' page is the current top result for https://www.gov.uk/search?q=%22Home+Office+counting+rules%22 with double quotes for the exact phrase.
That page was last updated on 5 January, the same day as your comment, so it may have been a temporary glitch.
Comment by Miran Norderland posted on
Is there any chance to list all FILTERED results on 1 page within Announcements, Consultations, Publications and Statistics sections?
When I select filters for certain segments, I still get results spread across several pages and for the purpose of research it would be much helpful if I could somehow list them on one page. Is that possible, and if so, how can I apply that function?
Miran
Comment by Tara Stockford posted on
Hi Miran
At the moment you can't increase the number of results on those filterable pages, but we'll bear it in mind for possible future development.
If you're comfortable with a more technical approach, you could get more results at once from our search API - a way of accessing structured data in the JSON format, using various filters. I'm planning to write a blog post about the search API soon, which will be on our 'Data at GDS' blog (https://gdsdata.blog.gov.uk/).
Tara