Research Data Services RDS AAF ANDS Ne CTAR

Research Data Services RDS / AAF / ANDS / Ne. CTAR / AARNET Data Lifecycle framework Ian Duncan – Director, Research Data Services (RDS) Icons made by Freepik from www. flaticon. com

Research Data Services What is the research data lifecycle? / Creation Discovery Preserve / Archive / Discard Analysis / Manipulation Description / Provenance Integration / Storage Icons made by Freepik from www. flaticon. com

Research Data Services Another way of looking at it http: //www. lib. ua. edu/wiki/sura/index. php/Data_Life_Cycle_Models Icons made by Freepik from www. flaticon. com

Research Data Services And another http: //www. lib. ua. edu/wiki/sura/index. php/Data_Life_Cycle_Models Icons made by Freepik from www. flaticon. com

Research Data Services What’s the problem? reliability http: //retractionwatch. com/ Icons made by Freepik from www. flaticon. com

Research Data Services What’s the problem? accessibility http: //retractionwatch. com/ Icons made by Freepik from www. flaticon. com

Research Data Services Findable Accessible Interoperable Reusable Icons made by Freepik from www. flaticon. com

Research Data Services (RDS) * Australian National Data Service (ANDS) * National e. Research Collaboration Tools and Resources (Ne. CTAR ) * The Australian Access Federation (AAF) Australia’s Academic Research Network (AARNET) * funded by the National Collaborative Research Infrastructure Strategy (NCRIS) Icons made by Freepik from www. flaticon. com

Existing Components/Services Research Data Services ingest analyse/process share analyse/process inc NCI, Pawsey, local HPC, etc dropbox-like researchers Identity projects / Autho risation / Acces s Various Storage Resources including portal 1 portal 2 portal 3 Existing components / Services repo 1 repo 2 repo 3 Research Data Australia Icons made by Freepik from www. flaticon. com

Proposed New/Enhanced Components Research Data Services ingest DMP and Provisioning share analyse/process Data Management Plan inc NCI, Pawsey, local HPC, etc dropbox-like researchers Project ID projects Local DMP systems DLCF Metadata Store (DLCF-MS) portal 1 repo 1 A P I National Storage Resources Enhanced Identity / Authorisation / Access Services (ORCi. D, edu. GAIN) portal 2 repo 2 portal 3 repo 3 New “connector” components: • • • Global Project ID Group ID and Group management service Minimal & extensible DMP metadata definition Project-based resource allocation Provisioning API (storage and allocation metadata) Research Data Australia Icons made by Freepik from www. flaticon. com

Phases / Workflows Research Data Services DMP & provisioning ingest 1 3 share 4 analyse/process 5 5 grants db dropbox-like inc NCI, Pawsey, local HPC, etc researchers Project ID projects Local DMP systems metadata A P I National Storage Resources 2 Data Lifecycle framework Outline – phases/workflows Provisioning (phase 1) 6 Use (phase 2) portal 1 repo 1 portal 2 repo 2 Research Data Australia portal 3 archive/publish/share/reuse/discard (phase 3) 1 “What can I access? ” 2 “Where is it? ” 3 “How can I feed my data into it? ” 4 “How can I share and use it with my group and my collaborators? ” 5 “Can I process it on the Cloud? ” “And here at Uni of X? ” “I need a bigger machine. . ” 6 ”I’ve finished my project and I think this data could be useful to someone in the future, please pack it away and make it available somehow” “I don’t want to share it just yet, please hold on to it and let me know if someone wants access” repo 3 Icons made by Freepik from www. flaticon. com

Possible Project Components Research Data Services DMP & provisioning ingest 1 3 share 4 analyse/process 5 grants db dropbox-like 5 inc NCI, Pawsey, local HPC, etc researchers Project ID projects Local DMP systems 2 metadata A P I National Storage Resources Data Lifecycle framework – All components – aspirational target 6 portal 1 repo 1 portal 2 repo 2 Research Data Australia portal 3 repo 3 1 Researchers access grants database which indicates which grants they ‘own’ or have access to. This “surrounding” metadata is registered with the metadata db. 2 Space marked with this metadata is provisioned on dropbox-like storage which is visible to the Ne. CTAR cloud – this space should belong to a project, not a person. 3 Automated and manual ingest processes feed data to this store, harvesting additional metadata where possible and relevant 4 Provisioned space should be as dropbox like as possible. 5 Storage is immediately visible to Ne. CTAR cloud and processes developed to ship data to local HPC or peak facilities using existing high-speed networks and tools 6 Once project is complete, the data is packaged and shipped to and indexed by the relevant domain repository as well as registered with the RDA index. Icons made by Freepik from www. flaticon. com

Possible Project Areas of Responsibility Research Data Services ingest DMP & provisioning share analyse/process grants db inc NCI, Pawsey, local HPC, etc dropbox-like researchers Project ID Identity / Authorisation / Access projects DMP System metadata National Storage Resources A P I Identity / Authorisation / Access portal 1 portal 2 portal 3 Data Lifecycle framework Potential Areas of Responsibility RDS Ne. CTAR repo 1 repo 2 repo 3 ANDS AAF AARNet Others (eg unis, NCRIS projects, etc) Research Data Australia Icons made by Freepik from www. flaticon. com

Examples: Virtual Lab Research Data Services DMP & provisioning Local grants db National grants db ingest 1 3 share 4 analyse/process 5 5 Ethics db inc NCI, Pawsey, local HPC, etc dropbox-like researchers Local DMP systems projects metadata A P I Local Storage 2 Example: portal 1 portal 2 Uni Research Data portal University Workflow Provisioning (phase 1) Use (phase 2) repo 1 Local repo 2 archive/publish/share/reuse/discard (phase 3) 6 Research Data Australia Icons made by Freepik from www. flaticon. com

Examples: Amazon Research Data Services DMP & provisioning National grants db Local grants db ingest 1 3 share 4 EC 2 5 processing 5 Ethics db inc NCI, Pawsey, local HPC, etc dropbox-like PAP portal metadata A P I Amazon S 3 2 Example: portal 1 portal 2 portal 3 Amazon Provisioning (phase 1) Use (phase 2) Glacier repo 1 repo 2 archive/publish/share/reuse/discard (phase 3) 6 Research Data Australia Icons made by Freepik from www. flaticon. com

Examples: Cloudstor Research Data Services DMP & provisioning ingest 1 3 share 4 analyse/process 5 grants db 5 inc NCI, Pawsey, local HPC, etc dropbox-like researchers projects Local DMP systems metadata A P I 2 Example: portal 1 portal 2 portal 3 Cloudstor a National Solution Provisioning (phase 1) repo 1 repo 2 repo 3 Use (phase 2) archive/publish/share/reuse/discard (phase 3) 6 Research Data Australia Icons made by Freepik from www. flaticon. com

Examples: Own. Cloud Federation Research Data Services DMP & provisioning ingest 1 3 share 4 analyse/process 5 grants db 5 inc NCI, Pawsey, local HPC, etc dropbox-like researchers projects Example: Local DMP systems Own. Cloud Federation metadata A P I portal 1 2 portal 3 repo 1 repo 2 repo 3 6 Uni A 2 2 2 Uni B Uni C (uses the national provisioning portal) Uni A provisioning portal Uni C provisioning portal Research Data Australia Icons made by Freepik from www. flaticon. com

Research Data Services www. dlc. edu. au Icons made by Freepik from www. flaticon. com

Research Data Services Engagement Initial steps for DLCF was engagement - a “market scan” of who was doing what, with whom, using what. i. e. Identifying the blocks Icons made by Freepik from www. flaticon. com

Research Data Services Thought Bubbles Output #1: DLCF Summary Document https: //goo. gl/9 Iufe. T This allowed us identify and communicate the scale of the challenge as well as to zero in on one area to focus on Provisioning Phase Icons made by Freepik from www. flaticon. com

Research Data Services MVP – DLCF Connectors Framework: • Metadata Store + REST API • (inter? )National Project ID • Group Id and Group Management Service Resulting in (for step 1) • Project based data resource allocation • Using AARNet Cloudstor+ Icons made by Freepik from www. flaticon. com

Research Data Services Minimal & Extensible Metadata Section Schema Comments Phase Research Project ID <ID> Auto-generated (by national RDS system? ) 1 Collaborators ORCi. D’s Identified by ORCi. D's, 1 Data Links <Defined by Data Providers> Data providers to define a JSON fragment. 2 Service Links <Defined by the services> Service providers to define a JSON fragment 3 Project Title <text> May contain sensitive information 3 Public Funding URI's Links to ARC/NHMRC/other funders 3 Institutional proj ID institution specific As per institutional requirements 4 Ethics Approval HREC? As per institutional requirements 4 Finance institution specific As per institutional requirements 4 Institutional Storage <UID> Local, Dropbox, One. Drive, as per local requirements 4 … … … 4 Required for DMP Connector Optional for DMP Connector Not Aggregated (institution specific) Icons made by Freepik from www. flaticon. com

Research Data Services Existing DMP tools Icons made by Freepik from www. flaticon. com

Research Data Services Metadata store & REST API Project_ID Request <Project_ID > Single API Pass-through service Two-way traffic Group_ID Request <Group_ID > Where and how. . ? National Services Metadata Store API/Service REST API Group Membership <Group_ID > <ORCi. D 01> <ORCi. D 02> <…> Icons made by Freepik from www. flaticon. com

Research Data Services National Project ID User story: As a researcher, after completing a DMP for a project I want to connect with the DLC tool and have a DLC project ID automatically allocated. This will provide a common key to all my resources for the duration of the project and post-publication. National Services – Project ID DLCF developed Project_ID <CERIF, ANDS, ORCi. D? > This identifier is a critical part of the DLC process; it provides a unique key for identifying not only the project but also all associated project entities and collaborations. Icons made by Freepik from www. flaticon. com

Research Data Services Group ID & Group Management User story: As a data provider I want to determine who has specific access permissions for a research data set and associated project assets. User story: As a project collaborator I need to have access to the datasets and tools associated with one or more projects. I will use my ORCID as my primary identifier and will then be able to access assets for which I have permissions, across all projects for which I am a collaborator. People ID’s vs Role ID’s. (Data Custodians) User story: As a research organisation I want to ensure that research data and associated project assets have a reliable custodian assigned. I want the custodian to be aligned with an organisational role and not a specific person, although it is understood that a person will be assigned that role for a duration of time. National Services - Groups AARNet developed VOOT Group_ID <? ? ? > Group_ID <ORCi. D 1> <ORCi. D 2> <ORCi. D 3> <…> <Institution_ID 01> <…> <Google. ID 01? > <. . . > Icons made by Freepik from www. flaticon. com

Research Data Services DLCF Connectors DMP <Project_ID 01> National Services – Project ID DLCF developed National Services Metadata Store API/Service Institution or national example DMP service New project ID request Project_ID Request <Project_ID > t Req ojec w Pr Ne uest National Services - Groups AARNet developed Project Name <Group_ID 01> <Group_ID 02> New Group request Group ID Grou p Qu REST API National Services – Resource Provisioning ery Group_ID Request <Group_ID > Group Membership <Group_ID > <ORCi. D 01> <ORCi. D 02> <…> Project_ID <CERIF or ANDS> New Group Request Group Query VOOT Group_ID <? ? ? > Group_ID <ORCi. D 1> <ORCi. D 2> <ORCi. D 3> <…> <Institution_ID 01> <…> <Google. ID 01? > <. . . > Icons made by Freepik from www. flaticon. com

Research Data Services Roadmap & Minimum Viable Product Icons made by Freepik from www. flaticon. com