Preserving Census Microdata and Making Them Useful IPUMS

Preserving Census Microdata and Making Them Useful: IPUMS & Sudan Prof. Awad Hag Ali Central Bureau of Statistics, Sudan Prof. Robert Mc. Caa University of Minnesota, USA First Arab Statistical Conference Amman, Jordan November 12 -13, 2007 Preserving Census Microdata: IPUMS & Sudan 1

The Problem: 1973 census tapes at risk! Preserving Census Microdata: IPUMS & Sudan 2

A Solution: Data recovery Preserving Census Microdata: IPUMS & Sudan 3

IPUMS-International Goals: 1. Recover and preserve census microdata (“raw data”) 2. 3. and documentation Integrate anonymized, high precision samples Disseminate extracts of samples to researchers (trusted users) UN Statistical Division Principles and Recommendations (2007) » » Preserve census microdata and documentation Disseminate anonymized microdata Note: No recommendation regarding prior censuses IPUMS pays to recover, document, and archive Preserving Census Microdata: IPUMS & Sudan 4

First Population Census of Sudan: 1955/56 • Conducted by the British • Time span: one year and a half • Used traditional administration (chiefs of the tribes). • Total population: 10. 1 million, adjusted to 10. 3 for under-count • A landmark in global census history: – First African census to use key-punch cards. – Thoroughly modern in design, execution, and evaluation Preserving Census Microdata: IPUMS & Sudan 5

First Population Census of Sudan: 1955/56 Preserving Census Microdata: IPUMS & Sudan 6

Preserving Census Microdata: IPUMS & Sudan 7

Preserving Census Microdata: IPUMS & Sudan 8

Preserving Census Microdata: IPUMS & Sudan 9

Population censuses of Sudan 1. 1956 - 10. 3 millions (adjusted from 10. 1) 2. *1973 - 14. 8 millions (adjusted from 12. 3 to account for Gezira camps and Nomads) 3. *1983 - 20, 594, 197 4. *1993 - 24, 941, 000 5. *2008 - ? ? * Since 1973: uniform methodologies; long & short forms; urban (long), rural (sample long + short) Preserving Census Microdata: IPUMS & Sudan 10

The Recovery process 1. 2. 3. 4. 5. Test 3 tapes by data recovery firm (Muller Media) When tests were successful, ship remaining 32 tapes Due to US economic blockade, tapes were first shipped to a Spanish University, re-packaged then trans-shipped to Muller Media 8 tapes were readily recovered; 3 tapes were recovered with “hand spooling”; 4 tapes were unrecoverable and 17 tapes were deemed earlier versions of recovered data; total cost US$3, 000 Construction of database is now underway at the Minnesota Population Center Preserving Census Microdata: IPUMS & Sudan 11

Feast in Khartoum Preserving Census Microdata: IPUMS & Sudan 12

1973 Tapes recovered by IPUMS Preserving Census Microdata: IPUMS & Sudan 13

Preserving Census Microdata: IPUMS & Sudan 14

Preserving Census Microdata: IPUMS & Sudan 15

Beyond data recovery: anonymized, integrated samples for research » » IPUMS: the “Hubble Telescope” project of statistics From 1999, funded by National Science Foundation of the USA, to: » » Construct high precision samples Anonymize the microdata Integrate microdata and metadata Disseminate to researchers world-wide at no cost 70+ official statistical agencies have endorsed project Memorandum of Understanding Samples for 80 censuses (26 countries), 200 million person records are now available Preserving Census Microdata: IPUMS & Sudan 16

IPUMS-Sudan • 1956: complete documentation scanned • 1973: recovery of 100% data by IPUMS • 1983: recovery of 100% data by CBS • 1993: archived on CDs • Documentation for all censuses is translated • In 2008: construct samples, anonymize and integrate • In 2009: launch integrated samples for International Statistical Institute meeting (Durban, South Africa) Preserving Census Microdata: IPUMS & Sudan 17

Recovered Preserving Census Microdata: IPUMS & Sudan 18

Beyond data recovery: anonymized, integrated samples for research » » IPUMS: the “Hubble Telescope” project of statistics From 1999, funded by National Science Foundation of the USA, to: » » Construct high precision samples Anonymize the microdata Integrate microdata and metadata Disseminate to researchers world-wide at no cost 70+ official statistical agencies have endorsed project Memorandum of Understanding Samples for 80 censuses (26 countries), 200 million person records are now available Preserving Census Microdata: IPUMS & Sudan 19

IPUMS: “Best practice” – Dennis Trewin (Australian Statistician and chairman of UNECE committee on Managing Microdata Access) » » » Anonymization to protect statistical confidentiality Integration of microdata and metadata Dissemination to researchers: fair and free --topics for a second presentation Preserving Census Microdata: IPUMS & Sudan 20

Invitation to participation in IPUMS: What needs to be done? » Official formalities: 1. Endorse memorandum of understanding 2. MPC pays US$5, 000 per census for microdata, nonexclusive license, and documentation » Unofficial—each statistical authority agrees to details: » What census microdata are to be recovered: 197? , 198? , 199? » Calendar: begin now? or in 2008? » Historical censuses: do microdata tapes or computer cards still exist? » Can earlier microdata be recovered? » IPUMS does the work and provides the funding Preserving Census Microdata: IPUMS & Sudan 21

Preserving Census Microdata: IPUMS & Sudan 22

The Fifth Sudan Census According to article 215(1) of the Constitution: A population census throughout the Sudan should be conducted and completed by the end of the second year of the interim period. Therefore this census is a constitutional one. Preserving Census Microdata: IPUMS & Sudan 23

The Central Bureau of Statistics (CBS) and the Southern Sudan Commission for Statistics and Evaluation (SSCSE) According to article 215 (2) and based on the comprehensive peace agreement (CPA )the Census shall be conducted by the Central Bureau of Statistic and the Southern Sudan Commission for Statistics and Evaluation Preserving Census Microdata: IPUMS & Sudan 24

Monitoring & Quality assurance A committee for monitoring and quality assurance was established by the PCC in March 2006. It consists of : • • • International Experts. National Experts. Civil organizations. Political parties. The committee is chaired by the states council. Preserving Census Microdata: IPUMS & Sudan 25

Achievements : (1) A detailed and comprehensive project document has been prepared through project appraisal meetings attend by (CBS) and (SSCCSE) world bank, UNFPA, and other supporting bodies. The document consist of the followings : – – A detailed work plan. A detailed Budget with activities and time frame implementation for each activity. Preserving Census Microdata: IPUMS & Sudan for 26

(4) Cartographic Work was: Started in the early 2006 and continuing up to now. GPS is used for the accurate location of settlement and delineation of (EAS). It is also used for allocating service centers. Almost all cartographic work have been finished in the North except Darfour which in a half way. The south is also in a half way. We except to finish mapping by the end of September. Preserving Census Microdata: IPUMS & Sudan 28

Decentralization : – Decentralized data capturing will be followed for the first time on Sudan. – Final data processing and tabulation will be performed centrally. Preserving Census Microdata: IPUMS & Sudan 30

Data dissemination-: • National and regional tabulation will be analyzed and published using different data dissemination methods such as: • Printed reports. • Electronic media (websites, Emails). • Data archiving. • Seminars and workshops. Preserving Census Microdata: IPUMS & Sudan 31

Challenges : • Lack of Proper infrastructure and experienced manpower in South Sudan due to the war. • Mines and insecurity in some parts of Sudan. • Continues population movement to the South (returnees) from inside and out side the country. • Insecurity in some areas in Darfur States and Equatoria. • Problems concerning nomadic population and their continues movements in huge areas and across the boards. • Lack of continues flows of Funding from both government and Donors. Preserving Census Microdata: IPUMS & Sudan 32