CONFERENCE EVALUATION DATA COLLECTION WHEN TO COLLECT DATA

CONFERENCE EVALUATION DATA COLLECTION

WHEN TO COLLECT DATA? Before Survey, Interviews, etc. During Polling, Observation, Focus groups, Interviews, etc. After Interviews, Surveys, etc

WHAT TO COLLECT? Ø You have to move from the objectives you want to evaluate to concrete questions – “operationalisation” Concepts Variables “Assess what were key learnings of the conference” Change to knowledge Questions Which of the following issues did the conference address for you? To what extent did the conference increase your knowledge on the following issues? Ø You then need to put questions into the appropriate surveys, interview guides, etc.

HOW MUCH DATA TO COLLECT? Ø The number or survey responses needed or persons interviewed largely depends on the size and type of the conference ØYou are not trying to achieve scientifically valid sampling but collecting enough feedback to make your evaluation credible and representative ØSome general recommendations: Ø Surveys: aim to have 30 – 60% of all conference participants respond to your survey Ø Interviews: 8 -12 interviews per demographic group is usually sufficient to gather enough feedback

GUIDANCE FOR DATA COLLECTION Ø Evaluators have to be impartial, cooperative and respectful Ø Good data collection guarantees anonymity to participants Ø Collect only data you need – evaluations tend to collect too much data that is never used Ø Develop a feeling for when you have collected enough data – this is called “saturation” – e. g. in interviewing when you keep hearing the same things with no variations

THE INTERVIEW - DATA COLLECTION Ø Semi-structured interviews often work well in conference evaluations ØWhen interviewing: Ø Ask for amplifications, examples and have participants define terms Ø Use phatic language to encourage responses “I see. . ” “ah huh. . ” Ø Listen to what the respondents don’t tell you Ø Write up your notes as soon as you can after the interview

THE SURVEY - DATA COLLECTION General advice: Ø Key when to give out the survey – during, at the end or after the conference (paper vs. online versions…. ) Ø Keep the survey short! Always link the survey questions back to your evaluation objectives to avoid “survey creep” Ø Start with more general questions (e. g. attitudes) and move to the more specific (e. g. demographic) Ø No need to reinvent the wheel – look at other conference surveys and use commonly used question scales and terms Ø Repeat questions from year to allow comparison Ø Test your survey before deployment

THE SURVEY - DATA COLLECTION Structuring the survey: Introduction - What for? - How long? General questions - Process - Outcomes - Impacts Specific questions - Attend this session? - Did you participate? Demographic questions - Who? - From where? - Past participation? Conclusion - Thank you - Next steps

THE SURVEY - DATA COLLECTION Type of questions ØDifferent question types are suited to different types of information sought: Ø Open questions Ø Closed questions: Ø Likert scale Ø Rank order scale Ø Multiple choice (singular response) Ø Multiple choice (plural response) Ø Binary (e. g. yes/no) Ø Numerical response

THE SURVEY - DATA COLLECTION Writing question ØAll surveys should have at least one open-ended question often useful for the “why” and the “how” ØOnly ask one piece of information per question ØBe consistent in the use of terms and words ØEnsure that questions are answerable ØAvoid biased questions ØAvoid having similar questions with same response set ØFor closed questions with pre-listed responses, make sure list is exhaustive (include “other” if in doubt)