Sampling and Questionnaires Bill Reimer reimeralcor concordia ca

Sampling and Questionnaires Bill Reimer reimer@alcor. concordia. ca http: //reimer. concordia. ca/teaching 2/14/2022 Bill Reimer 1

Outline Sampling n Questionnaire construction n 2/14/2022 Bill Reimer 2

Sampling Objective n Get a lot of information with little cost Sample Population 2/14/2022 Bill Reimer 3

Sample sizes for a population of 10, 000 2/14/2022 Bill Reimer 4

3 Rules of Scientific Sampling each unit in the population has an equal chance of being chosen n we must know the chance of each member being chosen n each selection must be independent from the others n 2/14/2022 Bill Reimer 5

To what do you want to generalize? People n Households n Trees n Leaves These are the Units of Analysis n 2/14/2022 Bill Reimer 6

To which units do you want to generalize? People in Canada? n Women in Québec? n Citizens of Montréal? n Trees in boreal forests? This is the Population n 2/14/2022 Bill Reimer 7

The Population Is the collection of units to which you want to generalize n It may be abstract n It may be concrete But it must be clear n 2/14/2022 Bill Reimer 8

From what will you choose your sample? A n list of units? region or place? drawer or computer file? particular time? This is your sample frame 2/14/2022 Bill Reimer 9

The Sample Frame may not match the Population SF e th P n i the its n U ot n t bu Un it s no in th tt e he P SF but Sample Frame 2/14/2022 …but try to get them close Bill Reimer 10

Sampling Requirements: Identify the population n Is the sample frame reasonable? n Does it meet the 3 conditions of scientific sampling? n We can seldom meet all 3 conditions! 2/14/2022 Bill Reimer 11

Types of Samples n Probability Generalizability n Efficient n n Non-probability Exploratory n Strategic n 2/14/2022 Bill Reimer 12

Simple Random Sampling Equal chance of being chosen n We know the chance n Each selection independent n From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 13

Systematic Sampling Random selection Determine sample rate (8/40) n Random selection of 1 st case n Every nth case (n=5) n Watch for regularities in the sample frame n From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 14

Stratified Random Sampling Separate sample frame into strata n Take random (or systematic) sample from each strata n May be proportionate or disproportionate n From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 15

Cluster Sampling Divide population into clusters n Randomly select clusters n Collect data on all cases in cluster n Watch cluster composition n From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 16

Cluster Sampling - Montréal 2/14/2022 Bill Reimer 17

Non-Probability Samples Accidental n Purposive n 2/14/2022 Bill Reimer 18

Convenience Sample Choose most convenient people n Subject to multiple biases n From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 19

Volunteer Sampling n Ask for volunteers Directly n Ads n Selected groups n n Unclear biases From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 20

Strategic Sampling Handpick respondents for strategic purposes n Useful for marginalized or small groups n Control biases by specifying criteria n From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 21

Snowball Sampling Identify strategic respondent n Ask for referrals n Continue the process n Use multiple starts to avoid network biases n From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8. 2/14/2022 Bill Reimer 22

Probability Samples – Multiple n Mixture of various techniques 2/14/2022 Bill Reimer 23

Sample Size Depends On Size of the population n Accuracy desired n Confidence desired n Variation in the phenomenon investigated n 2/14/2022 Bill Reimer 24

Assignment n n n Specify a research question you would like to answer. Describe a sampling procedure along with the data you would collect to answer the question. Identify the following: n n the the 2/14/2022 population sample unit sample frame type of sample Bill Reimer 25

Samples n Non-probability n n n Accidental Purposive Probability n n n Simple random Systematic Stratified: • Proportionate • Disproportionate n n 2/14/2022 Cluster Multiple Bill Reimer 26

Constructing Questionnaires 2/14/2022 Bill Reimer 27

Types of Research Interviews n Questionnaires n n Semi-structured interviews n n Closed and some open-ended questions Open and some closed-ended questions Unstructured interviews n Open-ended questions Research Interviews are NOT what you see on TV 2/14/2022 Bill Reimer 28

Advantages and Disadvantages n n n n Questionnaires Economic Speedy Minimal interviewer bias Anonymity Low response rates Limited exploration Comparison easy 2/14/2022 n n n n Bill Reimer Semi-structured High response rates Can ask complicated questions Can follow-up Qs Can observe Expensive Time consuming Comparison difficult 29

Preparing Questions Resist the temptation to start with specific questions n Clarify the RESEARCH question n Identify the types of information required to answer the research question n Within each of the types of information brainstorm for specific questions to ask n Rearrange questions n 2/14/2022 Bill Reimer 30

Question Order Leave sensitive issues for later - once rapport has been established n Follow from general to specific n Use tables and grids where appropriate n Use cards for sensitive issues if appropriate n Prepare your probes if interview n Follow a natural flow n 2/14/2022 Bill Reimer 31

Anticipate the Analysis Produce dummy tables n Work backwards from your analysis n 2/14/2022 Bill Reimer 32

Format examples Contingency question n Full filter question n Quasi-filter question n Closed question n Partially open question n Open question n Probes n 2/14/2022 Bill Reimer 33

Simple, Mutually Exclusive n What is your gender? (Circle the appropriate number) Female. . . . 1 n Male. . . . 2 n 2/14/2022 Bill Reimer 5 34

Multiple Choice, not Mutally Exclusive n What modes of transportation did you used to get to school last week? (Circle all that apply) • Metro. . . . • Bus. . . . . • Car. . . . . • Bicycle. . . . • Walk. . . . • Other (please specify) ___________ 2/14/2022 Bill Reimer 1 1 1 6 7 8 9 10 |__|__| 11 -12 35

Rank-order n Rank order the 3 most important aspects of your job (Place 1 beside the most important, 2 beside the next most important, and 3 beside the next most important) 2/14/2022 • Money. . . . |___| • Security. . . . |___| • New experiences. . |___| • Power. . . . |___| • Excitement. . . |___| • Other (please specify) ____________ |___| Bill Reimer 13 14 15 16 17 |__|__| 18 -19 36

Pretest, pretest Use your friends and family n Discuss n Where it was clear or not clear n What they thought about when responding to each question n What they thought you were getting at n Move to colleagues n Check it out on people who are similar to those you will eventually survey n 2/14/2022 Bill Reimer 37

Problem Questions n Jargon, slang, abbreviations n n Ambiguity, confusion, vagueness n n What is your income? Emotional language and prestige bias n n n What is your Internet browser? Should we put murderous terrorists in jail? Do you support Prime Minister Chretien’s policy on world trade? Double-barrelled questions n Should marijuana be legalized for medical or other purposes? 2/14/2022 Bill Reimer 38

Problem Questions – con’t n Leading questions n n Beyond respondent’s competence n n Do you feel that governments should have less power to interfere in business? Does your mother support extramarital sex? False premises n How can we halt the rising crime rate? 2/14/2022 Bill Reimer 39

Problem Questions – con’t n Future intentions n n How will you vote in the next federal election? Overlapping or unbalanced response categories n What is your opinion regarding abortion? 2/14/2022 • Very favourable • Favourable • Neutral • Opposed Bill Reimer 40

Questionnaire Construction The amount of time spent in the preparation of the questionnaire is directly related to the value of the results. n Surveys usually cost a lot and you can't redo them. n Know what you are wanting to discover n 2/14/2022 Bill Reimer 42

Research Strategies Questionnaires for generalizability n Exploration: well selected, in-depth interviews n Use interviews to design survey n Use comparisons n 2/14/2022 Bill Reimer 43

Sampling and Questionnaires Bill Reimer reimer@alcor. concordia. ca http: //reimer. concordia. ca/teaching 2/14/2022 Bill Reimer 44