The Archives Unleashed Project: Technology, Process, and Community to Improve Scholarly Access to Web Archives

Loading...
Thumbnail Image

Date

2020-08

Authors

Ruest, Nick
Lin, Jimmy
Milligan, Ian
Fritz, Samantha

Journal Title

Journal ISSN

Volume Title

Publisher

ACM/IEEE

Abstract

The Archives Unleashed project aims to improve scholarly access to web archives through a multi-pronged strategy involving tool creation, process modeling, and community building -- all proceeding concurrently in mutually --reinforcing efforts. As we near the end of our initially-conceived three-year project, we report on our progress and share lessons learned along the way. The main contribution articulated in this paper is a process model that decomposes scholarly inquiries into four main activities: filter, extract, aggregate, and visualize. Based on the insight that these activities can be disaggregated across time, space, and tools, it is possible to generate "derivative products", using our Archives Unleashed Toolkit, that serve as useful starting points for scholarly inquiry. Scholars can download these products from the Archives Unleashed Cloud and manipulate them just like any other dataset, thus providing access to web archives without requiring any specialized knowledge. Over the past few years, our platform has processed over a thousand different collections from over two hundred users, totaling around 300 terabytes of web archives.

Description

Keywords

Apache Spark, technology adoption, cloud platform, Information systems~Digital libraries and archives, web archives

Citation