GOV.UK has been working hard to migrate formats to use the new publishing platform over the last year. By the end of April, we expect to have migrated 16 Whitehall formats, with many sub-types of content, totalling almost a quarter of a million documents.
We’ve also rebuilt Specialist Publisher to use the new publishing platform.
This work is important because it reduces technical debt and enables us to speed up and start the work we’ve planned for the 2017 to 2018 roadmap. We blogged about the benefits of using the new publishing platform last year.
We’ve also been migrating mainstream formats to use the new publishing platform. I wanted to explain how we’ve approached it and give an update on the progress we’ve made so far.
What are mainstream formats?
By mainstream, we mean the formats that meet the most common user needs for citizens coming to GOV.UK. There are 13 mainstream formats, which made up 55% of pageviews on GOV.UK in the last 12 months. This was a little bit daunting for the team - we were concerned about causing disruption to a large number of users if something went wrong.
Mainstream Publisher - the publishing application used for publishing mainstream formats - is one of our oldest applications and is heavily dependent on our old publishing architecture. This makes migrating these formats complex and a bit risky.
Discovery
We learnt from the work on Whitehall migration that migrating formats can be really complex and we wanted to identify any complexity as early as possible.
Our aims for the discovery were to answer 3 broad questions:
1. How should mainstream formats work using the new publishing platform?
Here we looked at:
- the workflow and functionality of each format
- the data needed to display each format
- the reliance of each format on our old publishing architecture
2. How should we migrate the formats to use the new publishing platform?
Here we looked at:
- how much preparatory work should we do to simplify the old publishing architecture before
- migrating individual formats to minimise the risk and complexity
- the path for migrating each format at a high level
- the order we should migrate each format
- which formats needed to be migrated together
- which formats are more risky to migrate
- which applications each format should use (both publishing applications and frontends)
3. What dependencies are there on the publishing platform team?
Here we looked at what would block us technically from being able to migrate each format.
Decisions from discovery
Completing the discovery meant we were also able to make important decisions about the scope of the work, for example:
- by doing significant preparatory work to simplify the old publishing architecture, migrating individual formats would be simpler, quicker and less risky
- using the publishing platform to publish and render the content would be enough to unblock important work for next year’s roadmap, such as the template consolidation.
- continuing to use the same frontend application rather than building a new one (as we’ve done for some other formats)
We also decided to retire 4 formats:
- Videos - these weren’t meeting a user need and had very little traffic.
- Programmes - these were so similar to the guide format that we didn’t need both, and we could easily convert all programmes to guides without any impact for users.
- Campaigns - we’ve built a new campaigns platform so we should move campaigns there.
- Business support finder - we decided to make this format consistent with other finders which have already been rebuilt on the new publishing platform as they’re so similar. This also means we'll be retiring the business support finder API.
Preparatory work
We simplified the old publishing architecture to make migrating individual formats simpler, quicker and less risky. Sharing this work across multiple product teams was a good example of working collaboratively to achieve a common goal.
We’ve also been able to retire an old application (Panopticon) which was responsible for tagging content to browse pages, topics, organisations and other related content. It was also responsible for unpublishing content and managing URLs. We’ve significantly reduced the complexity of migrating each format by breaking their dependency on Panopticon.
Migrating formats
We started by migrating help pages, because there aren’t many of them and their traffic is fairly low. Since the team hadn’t worked on migration previously the developers used mob programming for the first format. This was a great way for the team to develop their knowledge together.
Since then, we’ve been working in pairs to migrate the remaining formats - we’ve done 7 of the 9 so far. We’re really happy with the progress we’re making and think spending time on discovery and the preparatory work has made the process of migrating a format much quicker and simpler.
Retiring formats
We’re also making good progress to start retiring the video, programme, campaign and business support formats from Mainstream Publisher.
Getting over the line
When we’ve migrated all mainstream formats to the new publishing platform, we’ll have significantly reduced technical debt in our platform and simplified our architecture. We’re confident we’re on track to finish by the end of March so that we’re in a good place to start on the 2017 to 2018 roadmap in April.
Mark Mcleod is a product manager on GOV.UK. You can follow him on Twitter.
2 comments
Comment by Cait posted on
What does 'reduces technical debt' mean?
Comment by Mark Mcleod posted on
Hi Cait,
You can find a good summary of what we mean by 'reducing technical debt' in this context by reading this blog post - Rebuilding GOV.UK’s publishing platform.
Generally, we mean reducing complexity in our architecture and code so that developers and designers working on GOV.UK can:
- begin work more quickly and with greater confidence
- find and fix problems more quickly
- iterate components and design elements and share what we've learnt across all our applications
- add new back-end functions or front-end templates faster because they’re built on shared components
- reduce the risk of code changes causing unintended consequences
- speed up or start the work we've planned for the 2017-2018 roadmap
Hope this is helpful.
Thanks,
Mark