Results Generated from the questionnaire disseminated prior to

Results Generated from the questionnaire disseminated prior to the workshop Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

The objective of the questionnaire • To better understand data processing activities at the country level • To invite country experiences with the goal of providing a forum for further collaboration on the effective use of techniques and methods in data processing • To support the development and management of the workshop and future activities • To understand what information and technical training is needed on the use of specific daata processing methods Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Capture: Methods for census/survey data capture • Common methods used for census survey data capture were: • manual data entry • OMR • ORC/ICR • Several countries are interested advancing efficiency through the use of PDA’s and have expressed that they plan to use the for the next census round Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Capture: Scanners and features used by countries: • Fujitsu w/Twain software driver (Egypt) • OPScan 7 & OPScan 10: 500 sheets/hr w/validation OR 5000 sheet per/hr no validation used with scantools software (Kuwait) • Kodak ds Digital Science Scanner 3520 40 to 85 ppm (depending on resolution, orientation, feeding, etc. ) (Morocco) Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Capture: outsourcing of processes • With concern to manual data entry, the data capture process is not outsourced. • • Methods included the use of a database management system such as Oracle along with CSPro where data entered, edited and coded in-house. With concern to OMR & OCR/ICR the data capture process is often outsourced. Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Capture: Planned data capture method for next census round • Some countries are undecided as of which method to choose • OMR/OCR/ICR is planned for use and outsourced (e. g. Iraq) • PDA’s are also proposed for use (e. g. Kuwait, Palestine, Qatar) Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Capture: Archiving methods and policies used for storing forms • Many countries use electronic means for the storage of forms. Some countries store forms both electronically and in hardcopy format. • Several countries have laws requiring the storage of forms for a given time ranging from ~10 to ~60 years. • Issues raised in the storage of hardcopy forms are that they take up space and may be damaged after a given time period Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Editing: Coding for Major Classifications of Occupations • All offices use coding for major classifications of occupations, industry and education. • • Occupation- most use ISCO with several countries using nationally specific systems also Industry- most use ISIC with several countries using nationally specific systems also Education- Most countries use ISCED with several countries using nationally specific systems also Nationality was also mentioned as a major classification in which coding is used. Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Editing: Manual or Automated Coding • Coding is done manually in most cases with some countries using both manual and automated methods. • • When automated, the software is developed in house (e. g. Egypt) or through a commercial produced such as Oracle (Lebanon) or developed by a private contractor and configured further by NSO staff (e. g. Morocco) All countries have an editing system as a part of the census/survey processing • The dominant error detection systems expressed within the questionnaire were • validity check • consistency check Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Data Editing: • • Editing consists of two main stages (Iraq) • Stage 1: during data entry sessions validation rules are used within the data entry program; • Stage 2: after data entry process completed using additional validation rules by different processes like merging files in order to find out & resolve outliers In many cases manual methods for imputation are used with the following software CSPro, IMPS, SPSS, Oracle. Countries create automated routines using statistical software tools such as SPSS and STATA and batch editing programs attached with the data entry program (CSPro batch editing tool). Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Staff and Training Syria Process: Full-time: Part-time: Data capture 350 25, 000 Data coding 200 500 Error detection 20 20 Imputation 200 1, 000 Qatar Process: Part-time: Data capture 100 Data coding 120 Error detection 120 Imputation 120 Palestine Process: Full-time: Data capture 100 Data coding 120 Error detection 200 Imputation 10 Lebanon Process: Full-time: Data capture 700 -800 Data coding 50 Error detection 50 Imputation - Jordan Process: Full-time: Part-time: Ad- hoc: Data capture 350 16, 000 700 Data coding 25 155 20 Error detection 10 80 5 Kuwait Process: Full-time: Data capture Data coding Error detection 10 Part-time: 100 30 30 Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Cont… • Training for each step ranges widely across countries • Data Capture ~5 days to 2 months • Data Coding ~5 days to 2 months • Error Detection ~5 days to 2 months • Imputation ~ 1 Day Example: Manual Data capture - 1 week Data coding - 3 days Error detection - 3 days Imputation - 1 day Example: OMR Example: PDA Data capture - 12 days Data capture - 14 days (for Buildings & Housing Units Data coding - 5 days Census) & 14 days (for Error detection - 5 days Population Census) Data coding - 7 days Error detection - 7 days Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

Quality control procedure country examples in relation to the various steps of data processing: • • • Egypt: • Training is given to data capture/data entry staff to help better understand questionnaire and fieldwork approaches Jordan: • Design special data entry programs for error detection; tables of frequencies for reviewing results, their consistency and accuracy Lebanon: • Specially trained team for verification and performance tests with the objective of establishing quality standards of a desired level for each operation (i. e. manual editing, coding, data entry and computer editing) Palestine: • Performing checks which are added to editing applications according to the feedback of specific subject matter departments Qatar: • Double keying; verification of data keyed from paper are some of the quality control procedures implemented Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008

END Thank You Workshop on International Standards, Contemporary Technologies and Regional Cooperation, Noumea, New Caledonia, 04– 08 February 2008