Yale University Library Research Guides: Web Archiving @ Yale: Documentation (Recommendations &amp; Shared Practices)

Documentation

Documentation produced or sponsored by the Web Archiving Working Group

Recommendations and Reports:

Quality Assurance Report
WAWG identified and assessed the available tools and methods for conducting quality assurance (QA) on archived web pages. This report provides guidance and resources on web archiving WA and an overview of how QA is completed at Yale University Libraries and other academic institutions.
Preservation Challenges and Recommendations for Captured Websites
This report explores the challenges, needs, and considerations for long-term preservation of captured web archives. In addition to an overview of captured web content, technical issues, and current systems, this report outlines recommended administrative considerations for the WAWG in their goals to support broad participation in web archiving and the development of a community of practice.

Metadata Guidelines for Archived Websites
Developed by the Web Archive Metadata Working Group, this guideline provides information on the creation and maintenance of catalog records for web archives

Web Archiving Strategy for FY19
This document builds off of the previous strategy, setting new priorities for the WAWG based on the successes and challenges faced during the previous two years. It outlines the plans to develop a robust Web Archiving Community of Practice (CoP) and continued opportunities for advocacy about web archiving with library leadership
Strategic Plan for Web Archiving at Yale, 2016-2018
This document outlines the goals of the WAWG towards developing a firm foundation for the newly emerging web archiving activity at Yale, articulating a two-year vision. It provides a framework to articulate stakeholder value, foster community and professional collaboration, develop funding models, streamline and expand harvesting, and provide essential guidance on rights issues.

Templates and Checklists:

Template Preservation Plan for Web Archives
This template outlines the considerations that should be addressed when acquiring web archives with the intention of long term preservation.
Web Archiving Appraisal Checklist
This checklist is a tool to help determine whether a website has archival or permanent value and if it should be included in a collection.

Additional Resources

Ivy Plus Libraries Web Resources Collection Program
The Ivy Plus Libraries Web Resources Collection Program is a shared initiative across the Ivy Plus Libraries to collect publicly accessible but at-risk web content and make it available for researchers. So far the program has developed thirty collections on a wide variety of topics.
Literature Review of User Needs (OCLC Research Supplemental, Descriptive Metadata for Web Archiving)
This report was developed by the OCLC Research Library Partnership Web Archiving Metadata Working Group to provide an overview of existing literature related to web archiving metadata. The review focuses on the needs of end users and practitioners.
Review of Harvesting Tools (OCLC Research Supplemental, Descriptive Metadata for Web Archiving)
In this publication, the OCLC Research Library Partnership Web Archiving Metadata Working Group analyzes eleven different web harvesting tools to determine their functionality and ability to generate descriptive metadata.

Tools

Webrecorder
Webrecorder provides a suite of open source tools and packages to capture interactive websites and replay them at a later time as accurately as possible.
Heritrix
Heritrix is an open source web crawler developed by the Internet Archive. It provides a web interface and command line utility to initiate web crawls.
Perma.cc
Perma.cc is developed and maintained by the Harvard Law School Library as a resource for creating permanent links to cited online sources. Users with a Perma.cc account can input a URL in exchange for a new permalink (a “Perma.cc link”) to the Perma.cc archived version of the page. This is a tiered subscription service for individuals, but organizations that are registrars of Perma.cc can provide free accounts to their users. The Yale Law Library is a Perma.cc partner and serves as a registrar for the Law School only.
YUL Digital Preservation Services
Digital Preservation Services is part of the work of the Preservation Department, which is responsible for supporting the Library's mission by ensuring continued access to collections through proactive stewardship, specialized expertise, and collaboration with partners at Yale and other institutions. The unit manages the Library’s digital preservation system (Preservica), which provides tools to crawl websites and ingest the captured web content into the system for long-term preservation.
Archive-IT QA Tool
The Archive-IT QA tool allows users to export a manifest of all missing content as a CSV file which can then be filtered in Excel.

Web Archiving @ Yale: Documentation (Recommendations & Shared Practices)

Documentation

Additional Resources

Tools

Site Navigation

Yale's Libraries