Framework for Quality Assurance in Population Censuses under

Framework for Quality Assurance in Population Censuses under Fragility and Adverse Conditions Dr. Luay Shabaneh Chief Technical Advisor, Population Census UNFPA-Iraq

Introduction • NSOs are committed to provide accurate, timely, and cost-effective population count through censuses. • Experience reveals areas of potential risk that could hamper efforts in conducting censuses, particularly regarding 1. quality assurance (QA) 2. quality control (QC)

Concept of Quality QC and QA No single definition for quality in the census context Here are some of the many definitions of “Quality. ” • Conformance to requirements. • Conformance to mutually agreed-to expectations. • Fitness for use. Quality is “getting the job done, on time, within budget, so that it meets the specified requirements”

Why Quality in Census • N Em ew D Census taking became more complex • • Ne erg eve Po w d ing lo • Outsourcing human resource • • Off litici ime issu pme Fr ice za ns e nt • R am c tio io • S egi e b ens n ns • I am ste ase us CT pl r b d 1. Justify public expend. ou e b ase tso ase d 2. Gain Trust of public ur d cin g 3. Deserve Authority 4. Protect from interven. QA program can decrease the risks associated with these factors. An effective QA function can reduce the chances of expensive missteps that require rework or last minute work-around solutions

Example on addressing QA/QC in the Region Egypt 2006 External Audit Palestine 2007 Structural system PES Sudan 2008 Mo. C Iraq 2010 QA Sector (NECIO 2010)

Conceptual Framework Quality= Fitness of use. It has to satisfy: Utility: usefulness for user Objectivity: accurate, reliable, and unbiased, clear Integrity: protection from unauthorized access

Conceptual Framework For operational purpose, some countries operationalized the definition of utility, objectivity, and integrity in terms of six dimensions: 1. Relevance 2. Accuracy 3. Timeliness 4. Accessibility 5. Interpretability 6. Transparency.

Case Tradeoff Issue Tradeoff Decision Iraq Census Quality of Postponing the census 2004 census data, and ability to keep professional implementation Iraq Census Quality of 2009 census data Postponing the population counting stage and continuing the work in the listing and numbering, in addition to updating cartography tools. Afghanistan Ability to cover Postponed the census. Census 2009 nationwiall census areas

Palestine Census 1997 Palestine Census 2007 Ability to cover all census areas particularly in Jerusalem Decided to go ahead regardless of Occupation position in allowing census teams to work in Jerusalem. The census ended up by not covering part of Jerusalem city. Would a QA Ability to Census management decided to framework lead to cover all make prior data collection in different outcome? census areas Jerusalem to avoid prevention by particularly in Israel, and accepted time lag Jerusalem between the data of Jerusalem and the rest of the country.

Message 1 Quality is very important dimension. It has. Not an Abstract Definition, it has to be derived from the domestic situation, in line with 1. Technical capacity 2. Census infrastructure and capacity of staff 3. National and Political Context of the census 4. Potential use or misuse of the data

Quality Control and Quality Assurance QC refers to quality related activities associated with the creation of census deliverables. It is used to verify that deliverables are of acceptable quality. 1. Peer reviews 2. Testing process. QA refers to the process used to create census deliverables, and can be performed by census management. 1. Checklists 2. Preventive measures system 3. In-process census audits.

Adverse Conditions and Risk Analysis Adverse conditions are processes of transition towards consolidated statehood. Types of Adverse Conditions: 1. A crisis state is a state under acute stress, and unable to manage conflict and shocks. 2. A fragile state is a state that is failing, or at risk of failing, with respect to authority, legitimacy and comprehensive service entitlements of citizenry. 3. 4. A post-conflict state is a state that has emerged out of conflict, crisis or fragility and is progressively establishing institutions

Country Type Challenges to launch a census Rwanda Conflict 1. Impact of census on 1994 genocide 2. Effect of genocide on migration and people’s displacement. (legitimization) Nigeria Conflict 1. 2. 3. 4. Public trust Conflict over resources Violence Potential isuse and misinterpretation 5. Acceptance of results; 1990 census data were accepted in 1998 because of political disagreements

Cambodia Post conflict 1. Lack of expertise after 3 decades 2. Use to reconstruct a country that had been ravaged by civil strife and genocide. Iraq Post conflict 1. 2. 3. 4. Sudan Post conflict 1. Distribution of wealth 2. Distribution of powers 3. Use data for referendum South Africa Post conflict 1. Post apartheid situation (legitimization) 2. Mistrust Allocation of resources Politicization over national reconciliation Security problems Allocation of power

Palestine Conflict Afghanistan Fragile State 1. 2. 3. 4. 5. Political instability Security problems Counting in Jerusalem Division between WE and Gaza Use of census for political purposes 1. Legitimacy of the state 2. Security and counting in the mountains

Message Two 1. Census is not only a statistical activity, it has political significance 2. Census launching tradeoff is important 3. Quality tradeoff is important 4. Out-of-box ideas are always needed

Risk Assessment Framework 1 Select census stages & form risk assessment team 8 Validation & feedback and modify accordingly 2 Decompose each census stage into main activities 7 Put measures and actions into a systematic pastern 3 Analyze risks and identify its sources 4 Identify the likelihood of each risk and vulnerabilities 6 Identify preventive measures and interventions 5 Identify the causes of threats

Challenge Problem Risk

Framework of QA modalities - Regular problems Characteristics Measure /Modality Regular difficulties that take place in almost all censuses. These difficulties occur because the census is a huge statistical undertaking. This includes 1. Management of large numbers 2. Mobility of people and goods 3. Trransfer of knowhow 4. Technical difficulties 5. Security of questionnaires and form 6. ICT utilization 7. Coverage and content errors Good management, well defined plans and procedures, better training, QA plans, systematic pattern of actions, mobilization of national and/or external support, etc

Framework of QA modalities- Challenges Difficulties that are country-specific and related to domestic natural and/or human made factors with limited impact on census undertaking, and over passing such difficulties depends mostly on the census team. For example: 1. Natural catastrophes 2. Reaching all remote areas 3. Lack of resources 4. etc Special traditional measures, separate teams and modalities, emergency plans, etc.

Framework of QA modalities- Risks and Threats Difficulties that are country-specific and related to domestic human factors (mostly political and/or security) and over passing these risks depend mainly on other players. This includes 1. Politicizing of the census 2. Political division across the nation 3. Potential fraud by certain group, 4. Potential violent resistance to census 1. Out-box measures 2. External monitoring 3. Mechanisms, special out-of-box measures (finger print, eye mark, observatory, etc), 4. Political dialogue 5. International intervention 6. Security measures 7. etc

Activities within QA framework- design stage • Central/regional committees for monitoring and controlling field work. • Adequate selection and training of staff. • Well established database on other sources information to compare with. • Cross fertilization with other running surveys (sources: registers) in terms of concepts, classifications and operational definition. • Developing accurate maps and checking its accuracy on regular basis through field visits and office checks.

Activities within QA framework- Census Fieldwork • Involve population, local authorities, religious/political/ethnic groups/leaders, associations/NGOs, field staff (directly or indirectly) and other relevant bodies. • Analyze the modalities of which a risk could occur including threats on field staff, call for boycott census among certain ethnicity/group/area, etc. • Prevent addition/omission of housing units, household, persons in households, persons in institutions, destruction of forms collected and intrusion in the information technology systems. • Communicate on the goals of the census, addressing the concerns of political/religious/ethnic groups

Activities within QA framework- Census Fieldwork • Communicate on the strong quality control mechanism, and identify potential risks, areas and groups. • Maintain the general features of field control which should be a part of the quality assurance and improvement programme of the census • Ensure the commitment of all partners and standard application country wide. • Involve many persons – mix ethnicities to neutralize fraud and include as much as possible comparison with other sources • Make listing, previous censuses, administrative sources, etc) and applicable and acceptable by the public

Activities within QA framework- Census Fieldwork • Involve census stakeholders on the results at each stage of launching the census and consistency/inconsistency of results. • Establish good enumeration and implementation environment that does not arouse suspicion except for minor boundary issues. • Ensure transparent, open and competitive procedure, avoid party or special interest lists, recruit enumerators from community using catchment zone strategy • Recruit supervisors from other communities to avoid misuse of power.

Activities within QA framework- Data Processing Stage • Plan dissemination at an early stage of census design • Identify what to release and when • Take into consideration other national events to avoid misinterpretation of dissemination timing, particularly in the highly politicized atmosphere • Try not to surprise the political level and plan very well for releasing the census results • Release in an official event where all relevant bodies are invited or informed on the day of releasing the results

Activities within QA framework- Data Processing Stage • Do not make any compromise on the professional ethics, but also absorb the situation and not provoke any influential bodies • Report on the development of the progress of the census in a transparent way to all relevant bodies • Release census data once and not to give privilege to any person or institution or party • Deal with census results as “public Goods” • Do not release the data before checking it very well, including PES

Conclusions • Adverse conditions are not exceptional anymore • A well articulated practical and operational guidelines on census organizations and management is needed to complement the standard United Nations P&R • UNSD as well as regional commissions and UNFPA country offices might play a vital role to develop such guidelines • In adverse conditions, census might be employed as a peacemaking and national reconciliation tool • Adverse conditions might hamper census if the census management was unable to mobilize an enabling environment to incubate it

Conclusions • Openness on international experiences is a wise step to provide sound technical support and motivate enabling environment • Understanding of national and political context of the census is a key factor • Compilation of tradeoffs regarding the census implementation and/or its quality based on religious risk analysis including sources of risks, its likelihood to happen and its impact on the census is a wise step • Setting up a comprehensive quality assurance system based on the results of the risk analysis is important

Conclusions • QA system should cover all census stages including design, planning, implementation, supervision, dissemination and evaluation. • QA system should be comprehensive, integrated, sound, demand driven and country-specific. • Census management has to absorb national local conditions, public acceptance and political sensitivities. • Out of box ideas, tools and approaches might be employed within the international standards boundaries.