CESSDA Question Databank Tender results and future Maarten

  • Slides: 19
Download presentation
CESSDA Question Databank Tender, results and future Maarten Hoogerwerf, CESSDA expert seminar 2009

CESSDA Question Databank Tender, results and future Maarten Hoogerwerf, CESSDA expert seminar 2009

Introduction • Data Archiving and Networked Services – Institute of both KNAW and NWO

Introduction • Data Archiving and Networked Services – Institute of both KNAW and NWO – Mission – Departments: • Archive and dissemination • Infrastructure • Software development Maarten Hoogerwerf, CESSDA expert seminar 2009

Outline • • • Background Question Bank Tender Discussion of technical specifications Conclusion Approach

Outline • • • Background Question Bank Tender Discussion of technical specifications Conclusion Approach Maarten Hoogerwerf, CESSDA expert seminar 2009

Background • Cross-national survey programmes introduce comparability and harmonization issues. • Supporting infrastructure: –

Background • Cross-national survey programmes introduce comparability and harmonization issues. • Supporting infrastructure: – Constructs, Classifications, Conversions Database (CCCDB or CHARMCATS) – Question Database (QDB) • Pre- and post harmonization Maarten Hoogerwerf, CESSDA expert seminar 2009

Tender • Specification of tender – Requirements, use cases – Need for CESSDA-wide architecture

Tender • Specification of tender – Requirements, use cases – Need for CESSDA-wide architecture • Execution – Metadata Technology – Marratech Sessions – Involvement of architecture WP • Report and review Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • General – QDB should not function stand alone • References to variables,

Report • General – QDB should not function stand alone • References to variables, questionnaire, etc. • DDI 3 metadata model • Webservice architecture – DDI v 1 and v 2 in use by CESSDA archives • Discussion – Will tools be able to migrate to DDI v 3? Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • Purpose and Functionality – Link questions via concepts, variables – Link additional

Report • Purpose and Functionality – Link questions via concepts, variables – Link additional survey metadata / physical data – Query questions based on references – QDB needs to include references • Discussion – Either use DDI 3 – Use generic model Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • Architecture – Repositories povide content – Registry indexes content – 3 CDB

Report • Architecture – Repositories povide content – Registry indexes content – 3 CDB and QDB provide functionality – Increasing identification and communication • Discussion – Question bank vs. QDB? – Identification designed for DDI 3 context Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • Repository – Contains content from one or more archives – Contains one

Report • Repository – Contains content from one or more archives – Contains one or more banks • Studies, variables, concepts, universes, questions, . . . – Dedicated or on top of existing systems – Additional administration, logs, etc. • Discussion – Existing systems fall short (identification, version, . . . ) – Quality essential for stability Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • Registry – Banks register content – Minimal metadata required for searching –

Report • Registry – Banks register content – Minimal metadata required for searching – Responsible for searching / locating, not for retrieval – Use SDMX approach • Discussion – How much metadata is needed for proper functioning? Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • QDB – Function as repository for local questions and proxy for non-local

Report • QDB – Function as repository for local questions and proxy for non-local questions – Stores comparison information • Discussion – Should QDB archive questions / comparison information – Who is responsible for QDB (LTP) Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • Requirements and use cases – A ‘Gold Standard’ promotes the use of

Report • Requirements and use cases – A ‘Gold Standard’ promotes the use of certain proven objects and increases comparability – Use registry for searching • Discussion – Assign to existing questions or define them centrally? – Use registry or QDB for searching questions? Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • Metadata and technology overview – Many open source components – Database might

Report • Metadata and technology overview – Many open source components – Database might require proprietary software • Discussion – Start with open source database. Good design allows replacement when needed. Maarten Hoogerwerf, CESSDA expert seminar 2009

Report • Implementation – Start prototype implementations to demonstrate functionality – Start improving legacy

Report • Implementation – Start prototype implementations to demonstrate functionality – Start improving legacy metadata – Use / extend SDMX registry • Discussion – Deadlock-situation: get tools to improve metadata, improve metadata to demonstrate functionality – How DDI 3 is improved metadata from Nesstar without workflow, versioning, identification? DDI 3 -ready? Maarten Hoogerwerf, CESSDA expert seminar 2009

Alternative Solution • MT approach is similar / better than intuitive solution – DDI

Alternative Solution • MT approach is similar / better than intuitive solution – DDI 3 metadata approach is essential – Web service is more flexible than harvesting – MT approach is more distributed Maarten Hoogerwerf, CESSDA expert seminar 2009

Conclusion • DDI 3 is an obvious choice, adopt it and improve it •

Conclusion • DDI 3 is an obvious choice, adopt it and improve it • It will change workflow, infrastructure and responsibility • How can archives justify, pay, risk and achieve this? • What is the role of CESSDA? Maarten Hoogerwerf, CESSDA expert seminar 2009

Approach Maarten Hoogerwerf, CESSDA expert seminar 2009

Approach Maarten Hoogerwerf, CESSDA expert seminar 2009

Approach • Phase 1: search, browse and access questions – Question text + response

Approach • Phase 1: search, browse and access questions – Question text + response domain – Results in having some base material • Phase 2: add references – To/from concepts and questionnaires – Implement registry to facilitate search – Explore organiation, publishing issues Maarten Hoogerwerf, CESSDA expert seminar 2009

Approach • Phase 3: Add QDB/3 CDB – What functions do these provide –

Approach • Phase 3: Add QDB/3 CDB – What functions do these provide – What metadata functions do these require • Etc. Maarten Hoogerwerf, CESSDA expert seminar 2009