Results Generated from the questionnaire disseminated prior to

Results Generated from the questionnaire disseminated prior to the workshop �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

The objective of the questionnaire • To better understand data processing activities at the country level • To invite country experiences with the goal of providing a forum for further collaboration on the effective use of techniques and methods in data processing • To support the development and management of the workshop and future activities • To understand what information and technical training is needed on the use of specific daata processing methods �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Capture: Methods for census/survey data capture • Common methods used for census survey data capture were: • manual data entry • OMR • ORC/ICR • Several countries are interested advancing efficiency through the use of PDA’s and Internet �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Capture: Scanners and features used by countries: • Kodak i 610 80 - 320 images/sheets per min. (Philippines) • Kodak 3510 200 dpi (60 - 75 ppm) 300 dpi: (40 -50 ppm (Singapore -2000 round) • Fujitsu M 4099 D ~90 ppm [simplex] to 180 images per minute [duplex], up to 400 dpi (Malaysia) • All dependent on resolution, orientation, feeding, etc. �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Capture: outsourcing of processes • With concern to manual data entry, the data capture process is not always outsourced. • • Methods included the use of a database management system such as Oracle along with CSPro where data entered, edited and coded in-house. With concern to OMR & OCR/ICR the data capture process is often partially or entirely outsourced. �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Capture: Planned data capture method for next census round • Some countries are undecided as of which method to choose • OMR/OCR/ICR is planned for use by many countries will all or part of the process outsourced (e. g. Bangladesh, Indonesia, Sri Lanka) • Mobile Devices/Internet are also proposed for use (e. g. Singapore, Iran- [surveys]) �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Capture: Archiving methods and policies used for storing forms • Many countries use electronic means for the storage of forms. Some countries store forms both electronically and in hardcopy format. • Several countries have laws requiring the storage of forms for a given time. • Issues raised in the storage of hardcopy forms are that they take up space and may be damaged after a given time period. �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Editing: Coding for Major Classifications of Occupations • All offices use coding for major classifications of occupations, industry and education. • • Occupation- most use ISCO with several countries using nationally specific systems also Industry- most use ISIC with several countries using nationally specific systems also Education- Most countries use ISCED with several countries using nationally specific systems also Ethnicity was also mentioned as a major classification in which coding is used. �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Editing: Manual or Automated Coding • Coding is done manually in most cases with some countries using both manual and automated methods. • • When automated, the software is developed in house (e. g. Egypt) or through a commercial produced such as Oracle (Lebanon) or developed by a private contractor and configured further by NSO staff (e. g. Morocco) Most countries have an editing system as a part of the census/survey processing phase • The dominant error detection systems expressed within the questionnaire were • validity check & consistency check • Also mentioned- Across and Within record, macro tabulation �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Data Editing: • In many cases manual methods for imputation are used with the following software CSPro, IMPS, SPSS, Oracle. • Countries create automated routines using statistical software tools such as SPSS and STATA and batch editing programs attached with the data entry program (CSPro batch editing tool). • Several countries expressed in the questionnaire that alongside software such as CSPro, editing system routines will be developed in-house (e. g Bangladesh, Rep. of Korea, Phillipines) �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Staff and Training Iran Data capture Data coding Error detection Imputation Part-time 414 729 450 10 Philippines Data capture Data coding Error detection Imputation Full-time 400 + 150 + 50 10 Rep. of Korea Data capture Data coding Error detection Imputation Sri Lanka Data capture Data coding Error detection Imputation Part-time Full-time Ad-hoc 700 13, 000 5 70 20 350 10 150 50 Full-time 75 40 20 20 Nepal Full-time Data capture 10 Data coding 20 Error detection 5 Imputation 2 Part-time 100 5 2 Indonesia Full-time Data capture 132 Data coding 165 Error detection 165 Imputation 99 Ad-hoc 702 1, 053 1, 332 594 �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Cont… • Training for each step ranges widely across countries • Data Capture ~5 days to 1 month • Data Coding ~5 days to 2 weeks • Error Detection ~5 days to 3 weeks • Imputation ~ 1 Day to 3 weeks Example: Manual Data capture - 1 week Data coding - 3 days Error detection - 3 days Imputation - 1 day Example: OMR Example: PDA Data capture - 12 days Data capture - 14 days (for Buildings & Housing Units Data coding - 5 days Census) & 14 days (for Error detection - 5 days Population Census) Data coding - 7 days Error detection - 7 days �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

Quality control procedure Country Examples in relation to the various steps of data processing: • Cambodia: • Data processing coordinator, QA Team, Verifiers (Supervisor), Verification form, Production form, QA news bulletin, Small QA meeting in a group, Targeted Training for specific individuals • Indonesia: • Supervisor is usually assigned to supervise three to five data capture operators to assist the operators through the completion of the data capture process. • Malaysia: • 1. Data Capture - Sampled check on data interpretation and verification • 2. Data Coding - Checks on sampled forms • 3. Imputation - Consistency check and production of a summary table • 4. Tabulation - Production of dummy tables • Philippines: • ICR-Based Data Capture -Hand-written characters and other shading and marks are interpreted by the machine are subject to key verification through a key-from-image entry. • Data Entry/Manual Coding -Entered records are subjected to sample key verification, usually using a 10%/20% sampling rate based on specified thresholds • Marginal Frequencies -Items are subjected to frequency tabulations before and after the edit/imputation step to determine potential abnormalities/anomalies in the edit/imputation rules or its implementation in the software. �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving

END Thank You �UNSD-UNESCAP Regional Workshop on Census Data Processing: Contemporary technologies for data capture, methodology and practice of data editing, documentation and archiving