The Oxford Common File Layout OCFL https ocfl

  • Slides: 13
Download presentation
The Oxford Common File Layout (OCFL) https: //ocfl. io Neil Jefferies Bodleian Libraries, University

The Oxford Common File Layout (OCFL) https: //ocfl. io Neil Jefferies Bodleian Libraries, University of Oxford/Data Futures Gmb. H neil. jefferies@bodleian. ox. ac. uk @Neil. SJefferies

OCFL is. . . an open community effort to define an application-independent way of

OCFL is. . . an open community effort to define an application-independent way of storing versioned data for long-term access

Completeness so that a application can be rebuilt from the files it stores ●

Completeness so that a application can be rebuilt from the files it stores ● The data is stored together with its metadata. ● Falls in line with standards about preserving access to data. ● Allows ease of mapping from one system to another, with the ability to keep a picture of the past.

Parsability both by humans and machines, to ensure data can be understood in the

Parsability both by humans and machines, to ensure data can be understood in the absence of original software ● In disaster recovery situations, humans should be able to understand the data. ● Machine readability allows for simple applications to be placed on top of an existing OCFL storage root.

Robustness against errors, corruption, and migration between storage technologies ● Strong fixity is built

Robustness against errors, corruption, and migration between storage technologies ● Strong fixity is built into OCFL. ● Data can easily be validated using the inventory. json. ● Data can be completely selfcontained.

Versioning so applications can make changes to data allowing its history to persist ●

Versioning so applications can make changes to data allowing its history to persist ● Changes to data are tracked over time. ● Forward delta is employed to reduce the amount of data stored. ● Previous versions of can be reconstructed using the inventory. json file

Storage Diversity to ensure data can be stored on diverse storage infrastructures including cloud

Storage Diversity to ensure data can be stored on diverse storage infrastructures including cloud object stores ● Designed to work with various storage infrastructures including object stores prevalent in cloud offerings (e. g. Amazon S 3). ● Supports conventional filesystem metaphor. ● Can be implemented to ensure deduplication of data, lowering overall storage costs.

OCFL Status Release milestones ● ● Alpha 0. 1 - Oct 2018 Alpha 0.

OCFL Status Release milestones ● ● Alpha 0. 1 - Oct 2018 Alpha 0. 2 - Feb 2019 Beta 0. 3 - Jun 2019 Release 1. 0 - July 2020 Release criteria ● ● 1. 0 Validator ✔ Test suite ✔ Fixture objects ✔ Two institutional commitments ✔

Who is investing in OCFL? Tools ● Johns Hopkins University - Go client ●

Who is investing in OCFL? Tools ● Johns Hopkins University - Go client ● Cornell University - Python client, validator, and test suite ● Penn State - Go client ● University of Wisconsin, Madison - Java client ● Stanford - Ruby client, validator, and test suite ● University of Technology, Sydney - Javascript client ● Brown University - Clojure HTTP server

Who is investing in OCFL? Systems ● University of Technology, Sydney - Research Data

Who is investing in OCFL? Systems ● University of Technology, Sydney - Research Data Portal ● Pacific & Regional Archive for Digital Sources in Endangered Cultures - Catalog ● University of Cologne - Analysis and Archiving of Audio-Visual Data ● LYRASIS - Fedora Repository ● Data Futures - hasdai Network

Watching https: //groups. google. com/forum/#!forum/ocfl-community https: //ocfl. io/ -- latest release and live current

Watching https: //groups. google. com/forum/#!forum/ocfl-community https: //ocfl. io/ -- latest release and live current drafts Jumping in ★ Monthly calls announced on ocfl-community Google Group ★ Join us on Slack (channel #ocfl) ★ Review and contribute use cases. ★ Review spec & implementation notes, discuss and submit issues ★ Implement and share your experiences

Thank you Neil Jefferies Bodleian Libraries, University of Oxford/Data Futures Gmb. H neil. jefferies@bodleian.

Thank you Neil Jefferies Bodleian Libraries, University of Oxford/Data Futures Gmb. H neil. jefferies@bodleian. ox. ac. uk @Neil. SJefferies