There are many places on GOV.UK where users enter their postcode to find services local to them. These include services provided by their local authority, such as paying council tax or disposing of hazardous waste, or finding the nearest number plate supplier.
These postcode lookups are used hundreds of thousands of times each month, and it’s obviously important that users get matched to the correct results.
The problem
Behind the scenes, these lookups used a combination of datasets from different sources to match a postcode to a geographical area or local authority. This was a very hands-on process, requiring regular updates by our developers as new data became available. This happened when new postcodes were created or local authorities merged, or their boundaries changed. The new data needed to be manually imported, which took a long time and generally required additional tweaking of the data to make it work properly. Between updates, users would potentially be getting incorrect information.
It was also slow and difficult to scale up the lookups when there was more demand - we found during some peak times that things struggled and users couldn’t get results.
What we did
After a thorough review process, we concluded that we should move to using an API-based solution, which means the data is fetched regularly and automatically.
When exploring this type of solution, we considered a few factors:
Reliability and scalability
How many requests can the solution handle? At the peak of coronavirus tier checking, our postcode checker was receiving up to 6,000 requests per second.
Accuracy
How frequently will the data be updated, so that users receive accurate results? Our developers manually updated the data every 3 months so we were looking for something more frequent but with less manual work needed.
Cost
We needed to make sure we were spending taxpayers’ money efficiently and delivering great value.
Technical fit
We carried out some investigations to ensure that there were no technical limitations preventing us from integrating the API into our systems.
The solution
Ultimately, we settled on using the Ordnance Survey’s (OS) Places API. This ended up being the best solution for a couple reasons. Firstly, GOV.UK can access the API for free under the Public Sector Geospatial Agreement. Secondly, their data is updated daily (compared to every 3 months when we updated the data manually) and their API has a 99.9% uptime.
To make the Places API work best for us, we built our own application, called Locations API, which fetches data from the OS and stores it ready for our various tools to use. This means we have full control over how it works with those tools, and can easily and quickly scale it up if necessary.
We had to make some changes to our tools that use the data, as they work with slightly different data structures.
We took care to design our application so that it doesn’t overload the Ordnance Survey’s system, using what’s known as a caching approach. In practice this means our application stores all of the postcode data and continually checks them against the OS Places API. We do this at a rate well within the API’s threshold of 600 requests per minute, cushioning the impact of any spikes in demand.
What this means
As a result of these changes, users entering their postcode to find information local to them will get more accurate results. We also plan to make use of the new data source to improve how we deal with situations where a user’s postcode spans several local authorities.
Also, far less developer time and effort is now needed to maintain things. The system runs automatically and requires minimal maintenance.
Subscribe to Inside GOV.UK to keep up to date with our work.
Check out the blog OS published on the work we did using their API.
8 comments
Comment by John Ploughman (DVSA) posted on
Thanks for the update, Jonathan.
With the new PlacesAPI, are places returned 'as the crow flies' (as in the past) or by road?
Comment by Jonathan Nichols, Lead Product Manager posted on
Hi John, thanks for the question. How we come up with lists of places, like on https://www.gov.uk/number-plate-supplier, hasn’t changed. As you say, basically it’s worked out by the straight line distance from the central point of a given postcode. We’re working on some improvements to that.
Comment by Tim Leighton-Boyce posted on
I'm really pleased to hear about this and I'm particularly interested that you hope to do something about the situation where a postcode can cover more than one local authority. Could that also include allowing users to enter only the first part of their postcode which they might expect to be sufficient in many cases.
Comment by Jonathan Nichols, Lead Product Manager posted on
Thanks Tim. That’s a good point and definitely something that can be considered by the team going forward.
Comment by Ian Evans (Royal Mail Address Management Unit) posted on
Hi Jonathan - really interested to know if there's anything we can do to help, ref. the issue of a postcode spanning several Local Authorities.
Comment by Jonathan Nichols, Lead Product Manager posted on
Hi Ian, thanks for reaching out. We actually just launched a solution where users who live in postcodes that span several Local Authorities can pick their actual street address to narrow it down. If you want to talk more please reach out to us on interaction-and-personalisation-govuk@digital.cabinet-office.gov.uk
Comment by Stuart Andrews posted on
Would you expect this API to be used in the SDLT service?
https://www.gov.uk/sdlt-online
Comment by Jonathan Nichols, Lead Product Manager posted on
Hi Stuart, at the moment this is just for use on the "www.gov.uk" domain which is what our team here at GDS manages, rather than in services that are owned and run by departments.