Marching to the Same Drummer Overview of a

Marching to the Same Drummer Overview of a Proposed Open Source Question Bank Olivier Dupriez François Fonteneau Geoffrey Greenwell Mark Mc. Conaghy International Household Survey Network World Bank PARIS 21/OECD DFID - UK

About IHSN • A network of international agencies … • A coordinating mechanism to: – Improve quality and use of household survey data in developing countries /MDG monitoring – Harmonise international recommendations for survey design, data analysis, etc – Produce and disseminate international good practices

The problems…. • In developing countries, survey data are produced mostly by official agencies for monitoring / assessing / targeting purposes • High dependency on external financial and technical assistance • Poor coordination creates problems: – Accessibility and use (no re-purposing of data) – Reliability and relevance – Comparability across sources and over-time

Water & Sanitation example • Access to safe water and improved sanitation – 2. 5 billion lack improved sanitation facilities globally – In sub Saharan Africa 58% (2006) access improved drinking water, MDG target 75%, world target 89%. – Globally 1. 5 million children die every year due to poor hygiene practices – Evidence suggests we are off track to meet the MDG goal to ‘halve the proportion of people without access to basic sanitation’ worldwide (54% - 77%). Figure for 2006 is 62% projected 67% 2015.

Nigeria ‘evidence’ case study • Monitored by WHO &UNICEF – Model with 3 estimates from national surveys for 1990 -2006 • Access to improved rural sanitation has changed little 21% (1990) to 24% (2006) – poor hygiene practices means Diarrhoea 2 nd largest cause of child death • Sluggish progress for improved rural water too 34% (1990) – 31% (2006)

Limited data points, based on 3 DHS surveys – already ‘similar’ limited syntax written to code to the common JMP standards for ‘improved’

New sources (April 2009) • Nigeria data archive: 14 datasets documented (DDI) and catalogued; many have data on water/sanitation • Problem: inconsistencies in questionnaire design and calculation of indicators in different sources • But good documentation and access to all datasets permitted reconciliation (national workshop) and production of standard guidelines for future surveys • Decision can be taken on using these new microdata, recomputed to international standards with existing sources

Inclusion of microdata – after reconciliation workshop - shifts estimate Promotes use of data which is made accessible through National Data Archive

… a common obstacle • Same situation in most other sectors • But in many cases, reconciliation impossible • Proposed solution: harmonised questions, guidelines, classifications, methods easily usable by survey managers – A collection of recommended questions, not a repository of questions from existing surveys – Objective is to help collect better data, not to help fix existing data

Q-Bank in a suite of solutions • 2005/07: DDI Microdata Management Toolkit improved usability of data • 2008/09: Survey cataloguing tool improved visibility • 2009: Dissemination / anonymization tools & guidelines improved access • 2009: Preservation guidelines (with ICPSR) improved preservation • 2009 – 2010: IHSN Question Bank improved quality and comparability for future surveys

IHSN Q-Bank • A repository of international recommendations • A bank of indicators, concepts, classifications, questions and instructions, and related references • Based on an XML schema (borrowed much from DDI and DCMI) • Application: includes a metadata editor (developed in dot. net) and a user interface developed in Adobe Flex (web) and AIR (stand-alone); all will be open-source • Content provided by many experts and specialized agencies (ILO, UNICEF, WB, consultants, etc) • Work in progress; expect to cover many topics (priorities are health, education, labour …)

Recap - questions and concepts Disease prevention & better livelihood Concepts Latent constructs Probable access to clean/safe water Indicators Improved/ unimproved drinking water sources Questions What is the main source of drinking water for your household?

Description of indicators (definition, methodology, limitation, references, etc) Each indicator provides links to the related concepts, questionnaire module(s), and classification(s) and other indicators

Description of survey modules Questions - Modular structure - Multiple ‘options’ per topic but ‘best practice’ e. g. Mode, burden, regularity - Overall module instructions, purpose and link to ‘concepts’ - Discussion forum, linking user community

Detailed description and instructions for each question Question ‘Item’ level Automatic link to classifications - Answer categories - Validation checks - Coding instructions - Type - Routing and universe - Visual rendering

Inclusion of technical resources Literal question –What kind of toilet facility do members of your household usually use? Answer categories Flush/pour flush Flush to piped sewer system 11 Flush to sceptic tank 12 Ventilated Improved Pit latrine 21 Pit latrine with slab 22 Pit latrine without slab/open pit 23 …. Post question text If ‘flush’ or ‘pour flush’ probe – Where does it flush to? IF NECESSARY ASK PERMISSION TO OBSERVE THE FACILITY Notes for Interviewer Pit latrines are the best conventional sanitation option when there is no source of water on or close to the plot so that water use is restricted.

A repository of classifications with detailed description • Maintain complex coding and classifications separately • Promote access and consistency across users • Improve international comparisons

All content can be added to a “shopping cart” for easy export and re-purposing

Conclusions • A tool for survey statisticians, researchers and analysts, aiming to: – Foster adoption of standards and good practices (improve reliability and comparability of data) – Reduce the cost/time of designing surveys • Launch expected in September 2009 • Open-source • Expect some countries to develop national question banks in time

Thank you www. surveynetwork. org International Household Survey Network