1 Training and accreditation of interviewers to carry

1 Training and accreditation of interviewers to carry out physical measurements on the Millennium Cohort Study Age 11 survey Nickie Rose (Ipsos MORI) and Lisa Calderwood (Centre for Longitudinal Studies, Institute of Education) © Ipsos MORI Version 1 Internal Use Only Version 1 | Internal Use Only 04/07/2013

Background 2 • Number of surveys in different countries include physical measurements • One element of a broader range of biomeasures collected alongside survey data • Three main models for biomeasure data collection in social surveys • Medically trained personnel in specially equipped clinics • Home-visit setting with portable equipment • Mobile nurse fieldforce • Specifically trained interviewers • Well-established that field interviewers are able to carry out more common physical measurements • Has advantages if collected alongside other study elements • Higher response rates • Cost effective • But requires additional training and support and appropriate quality control during fieldwork © Ipsos MORI Version 1 Internal Use Only

Millennium Cohort Study (MCS) 3 • Cohort study of around 19, 000 children born 2000/2001 – home visitbased • Families surveyed when children 9 months, 3 years, 5 years, 7 years and most recently 11 years old • Data collected from main carer (typically mother) and partner (typically father or step-father): interview and self-completion • Direct assessments of children’s cognitive function • From age 7, data collected from cohort children themselves via a selfcompletion • Data collected from older siblings in some sweeps via self-completion • Some bio-measures collected (e. g. teeth, oral fluid via mouth swabs (not for DNA extraction)) • Physical measurements (e. g. height, weight and body fat percentage, waist measurement, physical activity monitoring) © Ipsos MORI Version 1 Internal Use Only

This paper 4 • Millennium Cohort Survey (MCS) uses interviewers to collect physical measurements from children • Response rates to the physical measurements are high - 98. 8% at age 7 • Challenge to maintaining this at Age 11 survey • 11 year olds more self-conscious about being measured and handled by an interviewer • For many Ipsos MORI interviewers it was the first time they would have collected physical measurements • The CLS team were keen to review the approach to training and monitoring interviewers, following best practice on other similar studies • Led to the development of a formal accreditation process and additional early accompaniment for all interviewers • Paper describes how we achieved this in the context of an already very full briefing programme © Ipsos MORI Version 1 Internal Use Only

Physical measurements on MCS 5 • At age 11, interviewers required to measure height, weight and body fat • Height and weight used to calculate BMI • Measure of obesity • Body fat provides an estimate of fat-free mass • Children who are overweight tend to become overweight adults • Risks of illness and disease • Psychological consequences • Levels of obesity are increasing • In 1965 National Child Development Study found 11% of 7 year olds were overweight • In 2008 MCS found 21% of 7 years olds were overweight • Major concern for UK Government who are committed to delivering a sustained downward trend in levels of excess weight © Ipsos MORI Version 1 Internal Use Only

Experience from other surveys 6 • Accreditation schemes are standard practice in many US-based studies • • Health and Retirement Study National Social Life Health and Aging Project Fragile Families National Longitudinal Survey of Youth • More recently adopted in the UK • Health Survey for England • Understanding Society • English Longitudinal Study of Aging • Ensures interviewers are able to accurately follow study protocols © Ipsos MORI Version 1 Internal Use Only

What is accreditation? 7 • Accreditation takes place after comprehensive training and practice • Performance is assessed against a set of standardised criteria • Certification checklist • Interviewers pass or fail • Feedback is provided • Interviewers are required to pass before starting work • Benefits • • © Ipsos MORI Interviewers are accurate and consistent Interviewers are confident in abilities Promotes data quality Maximises response rates Version 1 Internal Use Only

Building on existing MCS training 8 • Existing training for MCS is well-established and robust • 3 days in classroom • “Active learning” – taught elements, demonstrations and practice sessions • Practice session with two children of same age as cohort between day 2 -3 • Home learning before and during briefing • Informal practice administering CAI © Ipsos MORI Version 1 Internal Use Only

Briefing the physical measurements 9 Existing briefing programme • Day 1: • • Detailed classroom session on measurement protocols lasting 2 hours 15 minutes Videos Demonstration in small groups Practice measuring height and familiarised themselves with scales operation • Between Day 2 -3 • Rehearsal with two specifically recruited 11 year olds • Day 3 • Feedback on in-home practice sessions • Quiz on measurement protocols © Ipsos MORI Version 1 Internal Use Only

Height protocol 10 • Height measured using a Leicester stadiometer • Protocol covers • • Correctly assembling the height measure Preparing the child Correctly positioning child Taking an accurate measurement • Bring down measuring arm • Move head to achieve Frankfurt Plane • Child to duck and step off • Take reading to nearest completed millimetre • Record in CAI immediately • Dismantle and pack away © Ipsos MORI Version 1 Internal Use Only

Height measurement 11 © Ipsos MORI Version 1 Internal Use Only

Measuring weight and body fat 12 • Calibrated Tanita scales • Weight • Body fat • Weak electrical current sent around body from one foot to another measures resistance of muscle and fat • Measured for all children except those who cannot stand unaided or were wearing an electronic device such as a pacemaker • If body fat refused able to measure weight only • Check scales measuring in the right mode (kilograms) • Enter child’s age and height (taken from CAI) • Record in CAI immediately © Ipsos MORI Version 1 Internal Use Only

Developing the accreditation process 13 • CLS team trained in measuring height by expert at Child Growth Foundation • Worked with a member of Ipsos MORI team to develop protocol and accreditation process • Developed an accreditation checklist based directly on the measurement protocols • Ensured trainers were consistent in their assessment • Tested only on taking an accurate measurement, not tested on other elements e. g. explanations or rapport • Full briefing procedures tested in a session that trained and accredited the briefing team • Height measurement accredited in full • Weight/body fat accreditation involved showing competency with scales © Ipsos MORI Version 1 Internal Use Only

Guidelines for interviewer accreditation 14 • General and specific guidelines developed to supplement checklist • General guidelines covered environment and testing conditions • • Accreditation should take place in pairs, matched by gender/height Separate room or quiet area No feedback, including non-verbal clues may be given during accreditation Feedback will be given by accreditor to each interviewer separately once both have been observed • Interviewers may use the Physical Measurements Summary Sheet and Frankfurt Plane card • Mistakes could be self-corrected • Subject should remove shoes and be co-operative • Specific guidelines referred to accreditation checklist • Re-sits organised for those not passing © Ipsos MORI Version 1 Internal Use Only

Incorporating accreditation into the briefing programme 15 • MCS briefing programme was already very full • Not possible to extend further • Challenge was to include accreditations within existing plan • Interviewers wanted more time to practice • At each briefing up to 20 interviewers to accredit • Considered accrediting during in-home practice sessions but felt inappropriate • Decided accreditation should take place on day 3, following practice • Time and resource constraints led to decision to accredit on height only • Measurement is more complex • Technical issues if not removing socks/tights for weighing • Concerns about weighing peers © Ipsos MORI Version 1 Internal Use Only

Incorporating accreditation into the briefing programme 16 • Allocated 40 minutes in schedule for accreditation • In pairs, 15 minutes per interviewer plus feedback • Additional 10 minutes for re-sits • Resources to achieve this were problematic • • Overall needed to accredit 309 interviewers over 19 briefing sessions Up to 8 -10 pairs of interviewers per briefing But only 2 Ipsos MORI briefers available at each briefing Could take over 3 hours elapsed time • Solution was to accredit supervisors in advance • All attended a quality control session on day 2 • Trainers had to be flexible balancing number of interviewers and accreditors and accomodating resits during breaks © Ipsos MORI Version 1 Internal Use Only

Fieldwork quality control 17 • Interviewers required support beyond initial practice and training • Interviewer accompaniments • All interviewers were accompanied by a supervisor within first 4 weeks on study • Priority given to those identified at briefing as needing extra support and/or new to this type of research • Modified standard accompaniment form • Supervisors checked adherence to protocols and offered feedback • Exception reporting • Unusual patterns in data by interviewer • Response rates for physical measurements/body fat refusal • Identified interviewers for additional training with equipment /feedback on performance © Ipsos MORI Version 1 Internal Use Only

Findings – learning and confidence 18 • This was a successful exercise • Improved training and support for interviewers • Consolidated learning • Confirmed interviewers could follow the protocols • Emphasised importance of following the protocols • All interviewers passed before leaving the briefings • Around four interviewers per briefing were required to resit • Interviewer confidence • Developed confidence in abilities • Reinforced by accompaniments • Interviewers found these invaluable • 17 were accompanied twice, 2 were accompanied three times • 3 interviewers were removed from the MCS panel as a result © Ipsos MORI Version 1 Internal Use Only

Findings – response rates 19 • Response rates to the physical measurements were high • 98. 4% productive at Age 11 survey • 98. 8% at Age 7 survey • Just 0. 8% refused body fat measurement • A success given expectation that response rates would be more challenging among 11 year olds © Ipsos MORI Version 1 Internal Use Only

Findings – practical lessons 20 • Incorporating the accreditations in the limited time available required careful planning • • Number of interviewers and accreditors Provision of adequate space Briefers needed to be flexible while ensuring approach was standardised The guidance provided was helpful • Managing interviewers expectations • Ensure interviewers are aware of need and rationale for accreditations in advance • Accreditation of briefers improved quality of training © Ipsos MORI Version 1 Internal Use Only

Recommendations 21 • MCS 5 team successfully improved training and support for interviewers in collecting physical measurements and ensuring high response rates • Achieved this by • • © Ipsos MORI Formalising the skills required Developing an accreditation process Training and accrediting the trainers Providing ongoing support in the field Version 1 Internal Use Only

Recommendations 22 • Review does suggest some lessons for the future • Allow more time for accreditation and feedback, particularly resits • Could be achieved by running parallel sessions • Slight amendment to accreditation form to indicate pass/fail and ensure that trainers use a separate form for each sitting • Include accreditation of height and weight/body fat • Formalise rules about expiry of accreditation • If working on other studies (which may have different protocols) • Or on same study but for subsequent waves or after an elapsed period © Ipsos MORI Version 1 Internal Use Only

23 Thank you nickie. rose@ipsos. com | l. calderwood@ioe. ac. uk © Ipsos MORI Version 1 | Internal Use Only This Version work 1 will Internal be carried Use Only out in accordance with the requirements of the international quality standard for market research, ISO 20252: 2006 and with the Ipsos MORI Terms and Conditions which can be found here