https://blog-digitalparliament.scot/merging-the-past-with-the-present-web-archiving-and-the-scottish-parliament-website

Merging the past with the present: web archiving and the Scottish Parliament website

This blog post has been provided by National Records of Scotland (NRS) and reflects the partnership working between NRS and the Scottish Parliament on web archiving and the release of the Scottish Parliament’s new website.

The Scottish Parliament (SP) launched its new corporate website two weeks ago. One of the challenges they faced was – what do we do with the old site and web content going back to 1999, when the Parliament came into being.

The historical value of this older content was not in question, but the SP team sought options to safely remove this older content from their new site, so long as it remained available elsewhere. To do so, SP collaborated with NRS’s Web Continuity Service to create a solution which incorporated live and archived web content. Read on to learn how we were able to put theory into practice.

Web archiving – a tool to support website redevelopment

The World Wide Web is over 30 years old – relatively ‘old’ in digital terms - and many websites have already gone through several generations.

Figure 1: Early prototype homepage for Google, courtesy of Internet Archive, 11 Nov 1998, available at Google! Search Engine (archive.org)
Figure 1: Early prototype homepage for Google, courtesy of Internet Archive, 11 Nov 1998, available at Google! Search Engine (archive.org)

When organisations change their website, they often need to migrate much of the current information (or ‘content’) to the new site, so that it remains online for current business purposes. However it can be challenging to migrate all content, especially where it is outdated, large in scale, or relies on older technology. What happens to this older content? Does it have historical value? Should it still be available online?

One tool to help address these questions is web archiving – the active process of  selecting, capturing and making available archived web content in a web archive. The most long-established and popular web archive is the Internet Archive, but many archives and libraries undertake web archiving to preserve crucial aspects of society’s memory.

National Records of Scotland, keeper of Scotland’s national archive, is no different: in 2017 NRS established the Web Continuity Service (WCS) to capture, preserve and make available snapshots of our clients’ websites for the archival record e.g. Scottish Government, NHS Special Boards, Scottish Courts etc. NRS recognises websites of Scottish public authorities as part of the public record, and we aim to capture regular snapshots of these websites in order to record their evolving narrative. NRS also invests resource in quality assuring archived content before it is made available, which mitigates technical challenges and supports service quality and access. Our commercial supplier, Internet Archive, supports us in this important work.

The product of this is the NRS Web Archive, and we encourage our clients – e.g. Scottish Government, NHS Scotland, Historic Environment Scotland etc. - to consider how this high quality collection of open, accessible and permanently preserved content could support their live website design and redevelopment.

How the Scottish Parliament put theory into practice

The Scottish Parliament launched its new website on 29 March 2021, replacing the current site which had been running for the last 10 years and which hosted historic information relating to previous sessions. NRS has archived the SP website since February 2017, and captures snapshots of https://www.parliament.scot/ on a quarterly basis.

Figure 2: earliest snapshot of SP website in NRS Web Archive, February 2017. Copyright Scottish Parliament
Figure 2: earliest snapshot of SP website in NRS Web Archive, February 2017. Copyright Scottish Parliament

SP and NRS assessed how archived content could be incorporated with live content on the main SP site. For example, we looked at whether it may be possible to offer a search functionality which queried live and archived content – this is not scalable at present, but is something to explore in the future.

What SP did instead was establish a layered-access approach to the large pool of content relating to Parliamentary Bills. Content on current or recently passed Bills, along with summary information on historic bills is available on the live site. However, if the user needs more, they are presented with direct links to relevant content in the NRS Web Archive, where full, original information on the historical passage of the Bill can be found.

This approach intellectually and technically drew together the history of devolved law-making in Scotland back to 1999, and also enabled the SP to remove large swaths of historical content from the live site.

Mark Sproul, the Product Owner for the new service at Parliament believes this approach is best for the users requiring to access this information – because they can easily and clearly find what they are looking for on the new website. But also they can consume the original content as it was published originally, accessible seamlessly from the NRS Web Continuity Service. It did take a little bit of time to convince Parliament colleagues that this was the best thing to do – web archiving is, still, a relatively new and abstract concept - but once we fully explained how it would work they quickly got on board with the approach. ‘From my perspective, it provided a strong and consistent user experience, removed the ongoing cost of maintaining this information, and was the most cost effective route as it required minimal content migration’.

Diagram highlighting where all Bill information can be found.

Garth Stewart, Head of NRS Digital Records Unit: ‘We are really excited to see the integration of archived content from NRS’s Web Continuity Service with the Scottish Parliament’s live website.  We put a lot of effort into the quality of our web archiving, and it’s great to see how this, in turn, benefits website owners such as the Scottish Parliament’. It’s been great to showcase the potential of web archiving to website redevelopment. Web archiving is a permanent activity however, and NRS will begin capturing snapshots of the new SP website once it goes live, so that future generations will be able to review and understand this unprecedented period in our nation’s history.