Chapter 10 NonExperimental Research Part 2 Survey Research

Chapter 10 Non-Experimental Research, Part 2: Survey Research

How to Design a Questionnaire 1. Determine the Types of Questions • Open-ended Question – People can answer in their own words. • Example Sentence Completion: “I feel most betrayed when__________

How to Design a Questionnaire 1. Determine the Types of Questions • Close-ended Question – Answers limited to the choices available. [next slide example]

Psychological Entitlement Scale Please respond to the following items using the number that best reflects your own beliefs. 1 = strong disagreement 2 = moderate disagreement 3 = slight disagreement 4 = neither agreement nor disagreement 5 = slight agreement 6 = moderate agreement 7 = strong agreement ____I honestly feel I'm just more deserving than others. ____Great things should come to me. ____If I were on the Titanic, I think I would deserve to be on one of the first lifeboat ____Quite honestly, I feel I can demand the best because I'm worth it. ____I do not necessarily deserve special treatment. (reversed) ____I believe I deserve more things in my life. ____People like me deserve an extra break now and then. ____Things should go my way. ____I generally tend to feel entitled to more of everything. [discussion, next slide]

Discussion Points • What are the advantages and disadvantages of open ended versus closed ended questions? • Why do researchers change the direction of acknowledgement of items i. e. , reverse-score items i. e. , ____I do not necessarily deserve special treatment. (reversed) • Note how Likert Scale options are worded consistently using the same word stem (agreement) i. e. , “agreement” to “disagreement” • Video on nuclear power plant survey and bias

Writing Questionnaire Items • One issue per item (or question) • Example of bad item: • “I tend to react fearfully to dogs and become nauseous”. • Make closed-ended questions clear • Example of a bad item: • “How likely is it that you would volunteer? ”

Writing Questionnaire Items • Avoid Bias In Wording Response Options: • Example of Bias: • “To reduce insurance rates for all, should motorcyclists be required to wear helmets? ” VERSUS • “Should motorcyclists have the right to choose whether they wear a helmet or not? ” • Note: most survey questions are somewhat biased

Writing Questionnaire Items • Make sure your response options are mutually exclusive • no case can belong to more than one alternative at a time Example of a bad item: “Licenses should be required of: a) b) c) d) state forest recreational users those who hike or picnic in state forest land; campers on state forest land; those who fish or hunt on state forest land”

Writing Questionnaire Items • Make sure your response options exhaustive • all cases fall into one or another of the alternatives Examples: • cover all age groups of interest in survey • Cover all behavioral options (e. g. , sports activities)

Writing Questionnaire Items • Determine the format of the item • Closed-end formats • • Likert scale, check boxes branching items visual analogue scale • Space to write for open-ended questions • Sequence the items • Put things in the right order

Beware of… • Social Desirability • Stimulus items or options prompt the respondent to answer primarily in a socially desirable; these are uninformative items Examples: True/False • “I like to try to keep physically fit” • “I’ve never exaggerated the truth in telling any story” • Counter with a Verification Key • Also called a lie scale • Collection of items that detect dishonest answers • Examples: “I read every page of the newspaper each day” ; “I have never felt like swearing”;

Beware of… • Acquiescence • Tendency to agree with any statement • Example: Tendency to over-acknowledge nearly all items on an inventory of psychological problems • “Are you considerate of others? ” • “Are there times of the day in which you are extremely hungry? ” • Video on Sampling, Survey Research

Determine how to analyze the results • Determine Scoring • How will each question be represented in an analysis? • Use a Coding Guide to write it down • Examples: 1=always, 2=often, 3=rarely 4=never • 1. “I struggle finding the energy to get things done” (scored + • 2. “My energy level is adequate” (reverse scored) • Scoring convention for reverse scored items 1=4, 2=3, 3=2 4=1 • Compute Total = question 1 + reverse-scored question 2. • Example: • If less than 3 errors are present in the memory task, score a “ 0” • If 3 or more errors are present in the memory task, score a “ 1” • Example: • Code the following responses: 1=Anglo; 2= Latino/Hispanic; 3=Asian; 4=African-American

Determine how to analyze the results • Determine Scoring • Use a Coding Guide • Data Reduction and Transcription • Take care moving from individual sheets to analysis software • Computerized responses to computer database reduces scoring errors • Automated response system (recordings, Scantron forms)

Administration Logistics and Procedures • Methods of Survey Administration: Strengths, Weaknesses of Each Method? • Depends on “circumstances” e. g. , feasibility, available technology (wireless transmission of responses via phone), expense, risks to confidentiality • Face-to-Face: Strengths • Best rapport • Can influence motivation of respondent (for better or worse) • Assess understanding of questions asked

Administration Logistics and Procedures • Methods of Survey Administration: Strengths, Weaknesses of Each Method? • Depends on “circumstances” e. g. , feasibility, available technology (wireless transmission of responses via phone), expense, risks to confidentiality • Face-to-Face: Weaknesses • High expense of travel and time for live interviewers • Risk of influencing the responses because due to verbal, nonverbal cues from interviewer • Potential for interviewer effects is highest in F-to-F • Extensive training of interviewers is required

Administration Logistics and Procedures • Methods of Survey Administration: Strengths, Weaknesses of Each Method? • Written and/or Computer-based Formats • Advantages: • Much lower expense than face to face • Highly standardized, consistent presentation of questions • Research suggests more “honest” answers • Can offer some anonymity to respondents • Easier to code and score; create a data set of all responses • Group administrations, written format are very economical, with computerized (internet-based) systems the least expensive of all

Administration Logistics and Procedures • Methods of Survey Administration: Strengths, Weaknesses of Each Method? • Written and/or Computer-based Formats • Disadvantages: • Paper and pencil methods require handling, erasure marks, having to “hand in” a response form (i. e. , possible damage, missed answers • Reading level of respondent? • Lower response rates (by 25 -80%) with written and/or computer-based formats • Erroneous answers---respondent may uniformly answer all questions the same way to quickly finish the task

Administration Logistics and Procedures • Methods of Survey Administration: Strengths, Weaknesses of Each Method? • Telephone • Advantages: Similar to live interviews, but lower expense • Use of “call centers”, lower social pressures compared to face-to-face on interviewee • More training required compared to written or computer-based response systems

Response Rate Issues • Response Rate: The percentage of individuals who return the completed survey • Selection of stratified, random sample is undermined by the issue of “who, among the selected participants, actually elect to complete the survey” • Elective “magazine polls” or elective internet-based surveys have worst response rate, most biased sampling (response rates of 1 -2%) • Mail-in and Paper and Pencil surveys 10 -50) • Elective, in-class research surveys in college have 4060% return rate, on average

People & Surveys • Population • All of the people or animals in the group involved in the research • Sample • Sub-group of the population. • May be selected in several different ways: • • Haphazard sample Purposive sample Convenience sample Probability sample

Random Sample • Type of probability sample • Each member of the population has the same chance to be chosen, independently of any other member • A special type of random sample in which at least 2 subgroups are represented in the same proportions as the population is called a Stratified Random Sample.

Non-Random Probability Samples • Systematic sample • Used for practical reasons • Likelihood of any element being selected is still known • Cluster sample • Groupings from a larger population • Multistage sampling • Clusters further broken down by taking samples from each cluster