Metadata Management in DILIGENT CERN Geneva Dec 16

Metadata Management in DILIGENT CERN, Geneva, Dec 16 2004 Bhaskar Mehta, Peter Fankhauser, Fraunhofer IPSI Darmstadt, Germany

Outline Introduction to WP 1. 3 Goals of Content and Metadata Management Component interaction Analysis of Metadata in DILIGENT Questions and issues wrt to EGEE and data management 02/10/2020 DILIGENT N° 004260 2

WP 1. 3 Overview Start Month 8 End Month 18 (continued) Involved Partners: ETH, Fh. G-IPSI, UMIT, USG Deliverables D 1. 3. 1 Content & Metadata Management services specification interim report (Month 11) D 1. 3. 2 Content & Metadata Management services specification report (Month 14) D 1. 3. 3 Content & Metadata Management services detailed design report (Month 17) 02/10/2020 DILIGENT N° 004260 3

Metadata Management Goal: support the management of metadata for digital objects handled by the content management service used by Metadata Broker, Annotation Service, Content Description & Selection, Search, Index, and other service of the DILGENT infrastucture Key functionalities: Optimal partitioning and placement of metadata among the GRID storage nodes (with replication) Efficient modification operations on distributed and replicated metadata Handling of concurrent access and updates on metadata Related Tasks Task 1. 3. 4 a and b Metadata Management Service design and implementation 02/10/2020 DILIGENT N° 004260 4

Content Component Interaction Metadata Existing Content and Metadata Collections 02/10/2020 DILIGENT N° 004260 5

Content Component Interaction Metadata Existing Content and Metadata Collections 02/10/2020 DILIGENT N° 004260 6

Content Component Interaction Metadata Existing Content and Metadata Collections 02/10/2020 DILIGENT N° 004260 7

Content Component Interaction Metadata Existing Content and Metadata Collections 02/10/2020 DILIGENT N° 004260 8

Content Component Interaction Metadata Existing Content and Metadata Collections 02/10/2020 DILIGENT N° 004260 9

Meta data : What, Why and How Metadata Data about data Types include Technical, source, structural metadata etc DILIGENT scenario Why do we need meta data ? What Meta data is needed ? How can this metadata be found/ generated ? 02/10/2020 DILIGENT N° 004260 10

A VDL connects … Users Collections Objects 02/10/2020 Services DILIGENT N° 004260 11

IMPECT- Use cases User Management Collaborative report generation, Explore users, Invite users User Object management Search, Import, Register, Annotate, Remove Object Collections Navigate, browse Create Collection Service Management Retrieve, Explore , Import, Search, Register new, browse, Annotate Retrieve Services Objects Services Annotate objects, retrieve objects, meta data generation 02/10/2020 DILIGENT N° 004260 Annotate, generate metadata 12

Meta Data in the DILIGENT scenario Collections can be treated as a kind of Objects 3 kinds of mediation possible Meta data at each end of the triangle to support mediation User profiles used for team building and user managements, and also for mediating with objects and services User profile User context Collections Object Meta data Indexing data, Annotations schema, 02/10/2020 Skills and preferences DILIGENT N° 004260 Services Service Meta Data Enhanced Service Descriptions, Qo. S Parameters 13

Object Meta Level Indexing Data Annotations , Annotation Schema Source Schema (Syntax) Dublin Core (Who, where , …) Ontological Annotations ( Semantics) Versions Size Cost 02/10/2020 DILIGENT N° 004260 14

Collections Meta data Indexing Data Annotations , Annotation Schema Dublin Core (Who, where , …) Operational Specifications Ontological Annotations ( Semantics) Versions Size Cost 02/10/2020 DILIGENT N° 004260 15

User Meta data Identification and Authorization Skills ( themes, areas of expertise ) Preferences User context Tasks Roles Relationships with other people 02/10/2020 DILIGENT N° 004260 16

Service Meta Data Semantic data Ontological references for input output Quality of service Performance, efficiency Computational resources available Operation time Information from service provider User ratings Availability 02/10/2020 DILIGENT N° 004260 17

Data management issues in using EGEE (Meta) Data Management Distributed or centralized ? Level of granularity Duplication of metadata Association of Meta data with individual objects Meta data maintainance Analysis of meta data: Support from EGEE Experiance of EGEE with Metadata in other e. Science applications 02/10/2020 DILIGENT N° 004260 18

Questions ? 02/10/2020 DILIGENT N° 004260 19

Content Component Interaction Metadata Existing Content and Metadata Collections 02/10/2020 DILIGENT N° 004260 20

Services for Content and Metadata Management part of the DILIGENT Digital Library Layer Five interacting service components Content Management Metadata Broker Annotation Service Content Security Support 02/10/2020 DILIGENT N° 004260 21

Content and Metadata Management Content Management Goal: support of transparent access to the DILIGENT content storage nodes as well as to external content providers. Metadata Management Goal: support the management of metadata for digital objects handled by the content management service Key functionalities: Incorporating exists source of data Optimal partitioning and placement of data among the GRID storage nodes (with replication) High and reliable access to data Efficient modification operations on distributed and replicated data Handling of concurrent access and updates on data used by Metadata Broker, Annotation Service, Content Description & Selection, Search, Index, and other service of the DILGENT infrastucture Related Tasks Task 1. 3. 4 a and b Metadata Management Service design and implementation Task 1. 3. 1 a and b: Content Management Service design and implementation Task 1. 3. 2 a and b Wrapper and Monitor Service design and implementation 02/10/2020 DILIGENT N° 004260 22

Metadata Broker Goal: framework for achieving metadata interoperability among disparate and heterogeneous metadata sources. uses: Metadata Management Service Key functionality: transformation of metadata dealing with metadata interoperability and integration detecting hidden equivalences based on the actual data, and knowledge based techniques for taking into account semantic context Light-weight broker for on the fly transformation Related Tasks: Task 1. 3. 6 a and b: Metadata Broker Service design and implementation 02/10/2020 DILIGENT N° 004260 23

Content Management Goal: support of transparent access to the DILIGENT content storage nodes as well as to external content providers. Key functionalities: Integration of content collections into the Grid environment enabling High reliability and availability of data Good scalability with the amount of data and the number of concurrent accesses Content wrapper for the integration external data sources Monitoring of DILIGENT storage nodes Related Tasks Task 1. 3. 1 a and b: Content Management Service design and implementation Task 1. 3. 2 a and b Wrapper and Monitor Service design and implementation 02/10/2020 DILIGENT N° 004260 24

Annotation Service Goal: support of manual and automatic content annotation and annotation-based information services; uses Metadata Broker, Metadata Management, Key functionalities Support of creation and use of annotation in VDLs Development of an ontology for annotation types Combination of generated semantic annotation and manually created annotations Use of annotation for resource selection Related Tasks Task 1. 3. 5 a and b Annotation Service design and implementation 02/10/2020 DILIGENT N° 004260 25

Content Security Goal: Content protection in a Grid-based Digital library system Key functionalities: Partial Encryption Digital Watermarking for copy right protection, avoiding unwanted change of content, tracing of copies, protection against unauthorized access, etc. Related Tasks: Task 1. 3. 3 a and b Content Security Service design and implementation 02/10/2020 DILIGENT N° 004260 26

02/10/2020 DILIGENT N° 004260 }Month 17 }Month 15 }Month 8 Tasks in WP 1. 3 - Summary 27

Next Steps Coordination with the efforts in WP 1. 4 (Indexing and Search) Query Processing in WP 1. 3 vs. WP 1. 4 Combination of IR (WP 1. 4) with XML querying (WP 1. 3) Metadata requirements from WP 1. 4 Refinement of architecture of Metadata and Content Management component Detailed plan for all tasks Alignment of planned functionalities with requirements collected from user scenarios 02/10/2020 DILIGENT N° 004260 28