THE LEVELS OF DIGITAL PRESERVATION A NATIONAL DIGITAL

THE LEVELS OF DIGITAL PRESERVATION A NATIONAL DIGITAL STEWARDSHIP ALLIANCE PROJECT JEFFERSON BAILEY, @jefferson_bail NDSA INNOVATION WORKING GROUP CO-CHAIR ALA Digital Preservation Interest Group, 2013 -06 -30 Upstairs, by Jo Szczepanska, from The Noun Project

THE LEVELS: THIS PRESENTATION • • • About the NDSA The Levels: Goals & Intent The Levels: In brief The Levels: In detail Early feedback Your feedback! Upstairs, by Jo Szczepanska, from The Noun Project 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE NDSA: ABOUT • “The mission of the National Digital Stewardship Alliance is to establish, maintain, and advance the capacity to preserve our nation’s digital resources for the benefit of present and future generations. ” • Started in July 2010 by Library of Congress, NDIIPP • Has 160+ members; free to join & all-volunteer • Features a diverse range of institutions and sizes • Organized in 5 Working Groups: Innovation, Infrastructure, Content, Standards, Outreach • http: //www. digitalpreservation. gov/ndsa/ 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE NDSA: ACTIVITIES • • • Storage Survey & Staffing Survey Web Archiving report Annual Innovation Awards Content Case Studies (geospatial, citizen journalism, etc) Wiki. Project Kickstarter featured projects Webinars on OSS Innovation Interview Series These Levels! http: //www. digitalpreservation. gov/ndsa/activities 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: GOALS • Simple guidance to help institutions of any type and size mitigate the risks to digital content • Basic assistance for planning and undertaking digital preservation activities • Content and resource agnostic • Evolving document of community-accepted minimums for digital stewardship • To be used 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: ORIGINS • NDSA Supergroup: Abbie Grotke (Library of Congress), Amy Kirchhoff (ITHAKA), Kris Klein (California State Library), Jane Mandelbaum (Library of Congress), Shawn Rounds (Minnesota Historical Society), Robin Ruggaber (University of Virginia) and Linda Tadic (Audiovisual Archive Network), Trevor Owens (Library of Congress), Andrea Goethels (Harvard University), Meg Phillips (NARA), me • Community-development • Blog posts on NDIIPP blog, The Signal 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: WHAT THEY COVER • Concepts & process areas • Accepted best practices & validation • Baseline considerations, “boxes off the floor” • Progressive, accessible scalability • Functional independence 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: WHAT THEY DON’T COVER Technology: Policy: • Infrastructure • Hardware, software • Tech ownership • Scope of Collecting • Standards & Workflows • Staffing Activity: Elasticity: • Budgeting • Project Management • Vendor lock, systems, OSS • Data I/O • Institutional change/knowledge 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: USES • Identify consensus of best practices • Bootstrapping • Self Assessment • Strategic Planning • Institutional advocacy • A roadmap & a starting point 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: STRUCTURE • FOUR LEVELS • Level 1 – Protect your data • Level 2 – Know your data • Level 3 – Monitor your data • Level 4 – Repair your data • FIVE ACTIVITY AREAS • Storage & Geographic Location • File Fixity & Data Integrity • Information Security • Metadata • File Formats • From Bit-level Preservation to Long-term Stewardship 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Level 1 (Protect your data) Level 2 (Know your data) - Two complete copies that are not collocated Storage and - For data on heterogeneous media Geographic (optical discs, hard drives, etc. ) get the content off the medium and into your Location storage system File Fixity and Data - At least three complete copies - At least one copy in a different geographic location - Document your storage system(s) and storage media and what you need to use them - At least one copy in a geographic location with a different disaster threat - Obsolescence monitoring process for your storage system(s) and media - At least three copies in geographic locations with different disaster threats - Have a comprehensive plan in place that will keep files and metadata on currently accessible media or systems - Check file fixity on ingest if it has been provided with the content - Create fixity info if it wasn’t provided with the content - Check fixity on all ingests - Use write-blockers when working with original media - Virus-check high risk content - Check fixity of content at fixed intervals - Maintain logs of fixity info; supply audit on demand - Ability to detect corrupt data - Virus-check all content - Check fixity of all content in response to specific events or activities - Ability to replace/repair corrupted data - Ensure no one person has write access to all copies - Identify who has read, write, move and delete authorization to individual files - Restrict who has those authorizations to individual files - Document access restrictions for content - Maintain logs of who performed what actions on files, including deletions and preservation actions - Perform audit of logs - Inventory of content and its storage location - Ensure backup and noncollocation of inventory - Store administrative metadata - Store transformative metadata and log events - Store standard technical and descriptive metadata - Store standard preservation metadata Integrity Information Security Metadata Level 3 (Monitor your data) Level 4 (Repair your data) - When you can give input into the creation of digital files encourage use of a File Formats limited set of known open formats and codecs - Inventory of file formats in - Monitor file format use obsolescence issues - Performat migrations, emulation and similar activities as needed 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and Geographic Storage Geographic. Location Level 1 Level 2 Protect your data Know your data Level 3 Monitor your data Level 4 Repair your data Two complete At least three copies that are not complete copies collocated At least one copy For data on in a different heterogeneous geographic media (optical location discs, hard drives, etc. ) get the Document your content off the storage medium and into systems(s) and your storage media system and what you need to use them At least one copy in a geographic location with a different disaster threat At least three copies in geographic locations with different disaster threats Obsolescence monitoring for your storage system(s) and media Have a comprehensive plan in place that will keep files and metadata on currently accessible media systems 00100000 or 01110000 01100100 01101001 01100111 01101001 01110100 01100001 01101100 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and Geographic Location File Fixity Data Integrity Level 1 Level 2 Protect your data Know your data Level 3 Monitor your data Level 4 Repair your data Check file fixity on Check fixity on all ingest if it has ingests been provided with the content Use write-blockers when working with Create fixity info if original media it wasn’t provided with the content Virus-check high risk content Check fixity of content at fixed intervals Check fixity of all content in response to specific events or activities Maintain logs of fixity info; supply audit on demand Ability to detect corrupt data Virus-check all content Ability to replace/repair corrupted data Ensure no one person has write access to all copies 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Information Security Level 1 Level 2 Protect your data Know your data Level 3 Monitor your data Level 4 Repair your data Identify who has read, write, move and delete authorization to individual files Maintain logs of who performed what actions on files, including deletions and preservation actions Perform audit of logs Restrict who has those authorizations to individual files Document access restrictions for content 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and Metadata Geographic Location Level 1 Level 2 Protect your data Know your data Level 3 Monitor your data Level 4 Repair your data Inventory of content and its storage location Store administrative metadata Store standards technical and descriptive metadata Store standard preservation metadata Ensure backup and noncollocation of inventory Store transformative metadata and log events 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and File. Geographic Formats Location Level 1 Level 2 Protect your data Know your data Level 3 Monitor your data Level 4 Repair your data When you can Inventory of file give input into the formats in use creation of digital files, encourage use of a limited set of known open formats and codecs Monitor file format obsolescence issues Performat migrations, emulation and similar activities as needed 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and Geographic Location Level 1 Storage & Geographic Location File Fixity & Data Integrity Information Security Metadata File Formats - Two complete copies that are not collocated - For data on heterogeneous media (optical discs, hard drives, etc. ) get the content off the medium and into your storage system - Check file fixity on ingest if it has been provided with the content - Create fixity info if it wasn’t provided with the content - Identify who has read, write, move and delete authorization to individual files - Inventory of content and its storage location - When you can give input into the creation of digital files encourage use of a limited set of known open formats and codecs 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and Geographic Location Level 2 Storage & Geographic Location File Fixity & Data Integrity Information Security Metadata - At least three complete copies - At least one copy in a different geographic location - Document your storage system(s) and storage media and what you need to use them - Check fixity on all ingests - Use writeblockers when working with original media - Virus-check high risk content - Document access restrictions for content - Inventory of - Store administrative file formats in use metadata - Store transformative metadata and log events File Formats 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and Geographic Location Level 3 Storage & Geographic Location File Fixity & Data Integrity Information Security Metadata File Formats - At least one copy in a geographic location with a different disaster threat -Obsolescence monitoring process for your storage system(s) and media - Check fixity of content at fixed intervals - Maintain logs of fixity info; supply audit on demand - Ability to detect corrupt data - Virus-check all content - Maintain logs of who performed what actions on files, including deletions and preservation actions - Store standard technical and descriptive metadata - Monitor file format obsolescence issues 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: THE LEVELS Storage and Geographic Location Level 4 Storage & Geographic Location File Fixity & Data Integrity Information Security Metadata - At least three copies in geographic locations with different disaster threats - Have a comprehensive plan in place that will keep files and metadata on currently accessible media or systems - Check fixity of - Perform audit of - Store standard all content in logs preservation response to metadata specific events or activities - Ability to replace/repair corrupted data - Ensure no one person has write access to all copies File Formats - Performat migrations, emulation and similar activities as needed 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: EARLY FEEDBACK Dramatic Chipmunk, via You. Tube user cregets 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: PRIMARY FEEDBACK AREAS Level 1 (Protect your data) Level 2 (Know your data) Level 3 (Monitor your data) Level 4 (Repair your data) - Two complete copies that are not collocated - For data on heterogeneous media (optical discs, hard drives, etc. ) get the content off the medium and into your storage system - At least three complete copies - At least one copy in a different geographic location - Document your storage system(s) and storage media and what you need to use them - At least one copy in a geographic location with a different disaster threat - Obsolescence monitoring process for your storage system(s) and media - At least three copies in geographic locations with different disaster threats - Have a comprehensive plan in place that will keep files and metadata on currently accessible media or systems - Check file fixity on ingest if it has been provided with the content - Create fixity info if it wasn’t provided with the content - Check fixity on all ingests - Use write-blockers when working with original media - Virus-check high risk content - Check fixity of content at fixed intervals - Maintain logs of fixity info; supply audit on demand - Ability to detect corrupt data - Virus-check all content - Check fixity of all content in response to specific events or activities - Ability to replace/repair corrupted data - Ensure no one person has write access to all copies - Identify who has read, write, move and delete authorization to individual files - Restrict who has those authorizations to individual files - Document access restrictions for content - Maintain logs of who performed what actions on files, including deletions and preservation actions - Perform audit of logs Metadata - Inventory of content and its storage location - Ensure backup and noncollocation of inventory - Store administrative metadata - Store transformative metadata and log events - Store standard technical and descriptive metadata - Store standard preservation metadata File Formats - When you can give input into the creation of digital files encourage use of a limited set of known open formats and codecs - Inventory of file formats in use - Monitor file format obsolescence issues - Performat migrations, emulation and similar activities as needed Storage and Geographic Location File Fixity and Data Integrity Information Security 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS: NEXT STEPS & FEEDBACK Next Steps: • Community Feedback • Release Candidate One is live • Next Iterations – Late Summer Your Feedback: • Clarity? • Usability? • Levels and Activity Areas? • Additions? 01100100 01101001 01100111 01101001 01110100 01100001 01101100 00100000 01110010 01100101 01110011 01100101 01110010 011101100001 01110100 01101111 01101110

THE LEVELS OF DIGITAL PRESERVATION A NATIONAL DIGITAL STEWARDSHIP ALLIANCE PROJECT • Levels of Digital Preservation Resources • http: //bit. ly/ndsa_levels • NDSA Homepage • http: //www. digitalpreservation. gov/ndsa THANKS! jeffersonbailey@gmail. com