European Statistical System and European Social Survey Input

  • Slides: 22
Download presentation
European Statistical System and European Social Survey Input and output harmonisation Ineke Stoop (SCP)

European Statistical System and European Social Survey Input and output harmonisation Ineke Stoop (SCP)

Harmonisation strategies (Körner and Meyer, 2005) Harmonised concept Measurement procedure National survey/ Specific concept

Harmonisation strategies (Körner and Meyer, 2005) Harmonised concept Measurement procedure National survey/ Specific concept Input harmonisation Ex-ante output harmonisation Ex-post output harmonisation www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Type of harmonisation Ex post output harmonisation • Use existing sources Ex ante output

Type of harmonisation Ex post output harmonisation • Use existing sources Ex ante output harmonisation • Set up new survey (or develop new instrument) Output harmonisation • Best national quality • Or, national survey tradition • But, what about optimal comparability Input harmonisation • Design new survey • Optimal comparability • Or, are identical methods and instruments really equivalent in different countries? • But, what about optimal quality www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

European Statistical System: output harmonisation Registers or surveys? • Comparison within countries Target population

European Statistical System: output harmonisation Registers or surveys? • Comparison within countries Target population Survey mode Question text (examples, scales) Fieldwork • Proxy, substitution Nonresponse • Efforts, rate, bias www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

European Social Survey Attitudes, values and beliefs Bi-annual 2002, 2004, 2006, 2008 Small sample

European Social Survey Attitudes, values and beliefs Bi-annual 2002, 2004, 2006, 2008 Small sample size: effective size 1500 30+ countries • EU, Switzerland, Norway, Russia, Ukraine, Turkey, Israel www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Aims ESS To monitor and interpret public attitudes and values within Europe and to

Aims ESS To monitor and interpret public attitudes and values within Europe and to investigate how they interact with Europe’s changing institutions • Provide data on beliefs, attitudes and values for scientific and policy making purposes • Measure attitude change in a changing Europe www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Substance: core modules Socio-demo-graphic items • • household composition education housing Occupation Attitudes and

Substance: core modules Socio-demo-graphic items • • household composition education housing Occupation Attitudes and behaviour patterns • • religious affiliation and identity ethnic and national identity political trust party affiliation and voting behaviour media consumption value orientations social exclusion www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Rotating modules: scientific community R 1 • • R 2 • • • R

Rotating modules: scientific community R 1 • • R 2 • • • R 3 • • R 4 • • R 5 • • Immigration Citizenship, involvement and democracy Family, work and well-being Opinions on health and care-seeking Economic Morality in Europe: Market Society & Citizenship Personal & Social Well-being: Creating indicators for a flourishing Europe The Timing of Life: The organisation of the life course in Europe Experiences and Expressions of Ageism Welfare attitudes in a changing Europe Family, work and well-being: implications of economic recession Trust in criminal justice and the police www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Can this be measured by output harmonisation Second aim ESS To advance and consolidate

Can this be measured by output harmonisation Second aim ESS To advance and consolidate improved methods of cross-national survey measurement in Europe and beyond • • Collect data according to highest standards Generate methodological research Develop and disseminate new best practices Develop and improve social survey research infrastructure in Europe • Input harmonisation www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Harmonisation model Central coordination Sampling protocol • Strictly random, no proxy, no substitution Source

Harmonisation model Central coordination Sampling protocol • Strictly random, no proxy, no substitution Source questionnaire Translation procedures Fieldwork specifications Interview mode (f 2 f) Contracting checklist Fieldwork monitoring Guidelines and protocols www. europeansocialsurvey. org National implementation Survey organisation • Stats, uni, market Interviewer training Sampling design Call schedule Response enhancement strategies Q 2010, Helsinki, Harmonisation models

Input harmonisation Some things cannot be harmonised • Language • Sampling frames • Response

Input harmonisation Some things cannot be harmonised • Language • Sampling frames • Response rates Level you aim for • • Lowest Highest Average National best www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Coverage Random sample, but some groups are excluded Elderly in institutions Military, student halls,

Coverage Random sample, but some groups are excluded Elderly in institutions Military, student halls, hostels Non-native speakers Not in population register Opt-out Reality differs across countries www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Nonresponse Response rate Response efforts • Required/made Response composition • Noncontact/refusal Nonresponse bias •

Nonresponse Response rate Response efforts • Required/made Response composition • Noncontact/refusal Nonresponse bias • Interest in politics • Trust in government • Voluntary work www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Response rates ESS 80 70 60 50 2002 40 2004 30 2006 20 2008

Response rates ESS 80 70 60 50 2002 40 2004 30 2006 20 2008 10 0 CH FR ES PT DE UK BE NO SE PL NL DK HU SI FI www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Questionnaire design Ask the same question Ensure that various concepts are actually represented as

Questionnaire design Ask the same question Ensure that various concepts are actually represented as precisely as possible Reliability and validity tests (SQP) Cognitive interviews Translation from source language into 1 target language for two-country pilot Two-nation pilot Comments national coordinators Final source questionnaire and annotations Translation and documentation Verification Small pilots in every participating country www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

No back-translation in ESS Back translation • • It’s raining cats and dogs Have

No back-translation in ESS Back translation • • It’s raining cats and dogs Have you had your tea yet? Geniessen in vollen Zügen Dear undertakers Perfect back translation does not guarantee a good question in the target language • Good help in finding obvious errors (wealthy/healthy) www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

TRAPD Translation • More than 1 translator • What is a good questionnaire translator

TRAPD Translation • More than 1 translator • What is a good questionnaire translator • Training of translators Review • Discuss and compare Adjudicate • Decide on 1 version Pretest Documentation • Also of review process Verify www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Survey mode Face-to-face • • • Crucial role of interviewer Let someone in house

Survey mode Face-to-face • • • Crucial role of interviewer Let someone in house Interviewer can have an effect on answers Usually higher response rates Long fieldwork period Expensive Mail • • • Less socially desirable Lower response rates Who answers the questionnaire? Telephone • • • Crucial role of interviewer Interviewer can have an effect on answers Lower response rate No landline phone, ex-directory Short interviews Internet • • • Less socially desirable Lower response rates Who answers the questionnaire? www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Survey modes Face-to-face only Applicable in every country? Do modes mean the same in

Survey modes Face-to-face only Applicable in every country? Do modes mean the same in every country? www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Fieldwork Different organisations House effects Interviewers • Training, remunerations Tradition National survey culture www.

Fieldwork Different organisations House effects Interviewers • Training, remunerations Tradition National survey culture www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

IN SOME CASES INPUT HARMONISATION IS THE ONLY OPTION BUT YOU HAVE TO BE

IN SOME CASES INPUT HARMONISATION IS THE ONLY OPTION BUT YOU HAVE TO BE AWARE OF CROSS-NATIONAL DIFFERENCES www. europeansocialsurvey. org

Thank you for your attention www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models

Thank you for your attention www. europeansocialsurvey. org Q 2010, Helsinki, Harmonisation models