CH 6 The Survey pp 144 174 1

CH 6: The Survey (pp. 144 -174) 1

WHAT IS A SOCIAL SURVEY? o o Social surveys are the most widely used datagathering technique in the social sciences and other fields The survey technique gives us data based on selfreports In a social survey, a large number of people (called respondents because they respond to a query) hear/read and provide answers to the exact same questions Opinion polls are one type of survey, which ask about opinions on current issues n Besides opinions, surveys may ask about knowledge, social background and characteristics, general beliefs or behaviors 2

Survey data are considered correlational o o Survey data are sometimes called correlational because they are strongest at meeting the 2 nd condition of causality – association/correlational research: any nonexperimental study in which correlations in data are examined and cause-effect relations are shown indirectly 3

Cause-effect explanations based on survey data o o Cause-effect explanations based on survey data differ somewhat from other research techniques, such as the experiment Recall from Ch. 2 that if you want to say that one variable causes another, you must meet 3 conditions 1) The IV must come before the DV in time 2) The 2 variables must be associated, or correlated with one another; and 3) No alternative cause for the relation exists, or there is no spuriousness: when two variables appear to be causally connected but in reality, they are not because an unseen third factor is the true cause 4

Condition 1: time order? o o Meeting the 1 st condition – time order – is complicated in survey data collected at a single point in time Without data at multiple time points, you must logically show that information from one survey question (e. g. , experience of severe trauma) occurred earlier in time than information from another question (e. g. , suicidal ideation) 5

Condition 3: no alternative cause? o o To meet the 3 rd condition, no alternative cause, you must think of variables that are possible alternative causes and measure them in a survey control variables control variable: a variable that represents a possible alternative explanation to the main hypothesis being tested; often control variables are measured in survey research with questions in addition to measures of the independent and dependent variables 6

HOW TO CONDUCT A SOCIAL SURVEY o 3 stages /6 steps n n n 1. Start-up stage – plan and prepare survey questionnaire (steps 1, 2, 3) 2. Execution stage – collect and record data (step 4) 3. Data analysis stage – analyze and interpret data and report final results (steps 5, 6) 7

Start-up o In this stage, you address the following 3 questions: 1) 2) 3) Who will be respondents? What information do you want to learn from them? How can you effectively get that information? -What will be your data collection instrument? 8

Start-up: Step 1 o o Develop hypotheses Decide on type of survey: n n o o o questionnaire: a fixed collection of questions used in a social survey that respondents answer interview schedule: a questionnaire specifically designed for an interviewer asking respondents the questions o Face-to-face or by phone/online? Write survey questions Decide on response categories Design layout 9

Start-up: Steps 2 and 3 o Step 2 n n o Plan how to record data Pilot test the survey instrument Step 3 n n Decide on target population Get sampling frame Decide on sample size Select sample 10

Execution: Step 4 o o o Locate respondents Conduct interviews Carefully record data 11

Data analysis: Steps 5 and 6 o Step 5 n n n o Enter data into computers Recheck all data Perform statistical analysis on data Step 6 n n Describe methods and findings in research report Present findings to others for critique and evaluation 12

WRITING GOOD SURVEY QUESTIONS Two core principles: 1) Avoid confusion, e. g. , double-barreled questions o n double-barreled question: a confusing survey question that includes two or more ideas 2) Keep the respondent’s perspective in mind n Dilemma: You want each respondent to hear the exact same question (b/c you want to measure the same thing) but if respondents have diverse backgrounds, the same wording may not have the same meaning for everyone 13

What Are Leading Questions? o A good survey question is one in which respondents do not know which answer you expect, and they feel free to state what they really think or feel n leading question: a survey question worded such that respondents are pushed to a specific answer or position. 14

Some leading questions o o “Stable gay and lesbian couples have the same rights as any other law-abiding citizens. Do you agree that your fellow citizens who are gay or lesbian have the same right to the benefits and responsibilities of a legal marriage as anyone else? ” n leading to overstate support “Should we alter the law so that sexually deviant people get the privileges and benefits of a marriage, or should we uphold the natural basis of a marriage as between one man and one woman as it has been for centuries in custom, law and religious teachings? ” n leading to understate support 15

Should you use open or closed format? o o open-ended question format: allows respondents to give any answer closed-ended question format: respondents must choose among fixed answer choices 16

Writing Good Closed-Format Response Choices o Writing good answer choices is just as important as writing a good question. Answer choices should have 3 features: n n n mutually exclusive: response categories don’t overlap exhaustive: each respondent has a choice balanced: favorable and unfavorable choices are equally available in set of responses o e. g. , ‘outstanding, excellent, very good, and satisfactory’ is an unbalanced set of responses to rate performance 17

Should You offer a ‘Don’t Know’ or ‘No Opinion’ Response Choice? o o Opinions differ, but you want to avoid two errors: n a) getting ‘no opinion’ or ‘don’t know’ response when a respondent actually holds a non-neutral opinion n b) forcing a respondent to choose a position when s/he has no opinion on an issue or knows nothing about it standard-format question: a closed-ended survey question that does not offer a DK or no opinion option quasi-filter question: a closed-ended survey question that includes a choice for respondents who have no opinion or do not know about an issue full-filter question: a contingency survey question that first asks whether a respondent knows about an issue, then only asks those who do 18

Factors that interfere with respondent recall of past events o o o A sensitive or threatening topic Events that occurred simultaneously Events that occurred after that being recalled An issue or event that was not significant A person’s need to be consistent and not appear to contradict him/herself 19

Asking Respondents about Sensitive Issues o o People often underreport having an illness or disability or engaging in illegal or deviant behavior—and overreport positive, desirable behavior social desirability bias: a tendency for survey respondents to answer in a way that conforms to social expectations or makes them look good rather than to answer honestly 20

Contingency questions o 1) contingency question: a two-part question in which a first question screens who gets the second question, e. g. , Did you vote in the midterm elections last week? [] Yes (Go to Q 2) [] No (Go to Q 3) 2) 3) Which gubernatorial candidate did you vote for? Had you voted in the 2008 elections? 21

Wording Effects - Learning from History: The Power of Words (p. 166) o o ‘forbid’ vs. ‘not allow’? spending ‘to help the poor’ vs. ‘for welfare’ ‘inner city’? ‘impartial’? 22

EFFECTIVE QUESTIONNAIRE DESIGN TIPS o o o Length of a survey/questionnaire depends on survey format Organize questions to minimize discomfort and confusion Reduce order effects n o order effects: when the ordering of survey questions influences how respondents answer them Reduce context effects by being aware how preceding questions and the interview setting may influence responses. Ask more general questions before specific ones. 23

Mail & Self-Administered Questionnaires o Advantages n n n o Easy and inexpensive If mailed, cover wide geographical area, allow respondents to check personal records at home Offer anonymity and avoid interviewer bias Disadvantages n n Low response rates No control over survey conditions Unsuitable for Qs that require visual aids, openended Qs, many contingency Qs, complex Qs Ill-suited for the illiterate or semi-literate 24

Telephone Interviews o Advantages n n o Cover wide geographical area Flexible, with most strengths of F 2 F interviews Can use contingency Qs effectively, esp w/CATI Supervisors can monitor quality by listening in Disadvantages n n Higher cost Limited interview length Use of interviewer reduces anonymity and introduces potential interviewer bias Cooperation with telephone interviews has steadily declined 25

Face-to-Face Interviews o Advantages n n o Highest response rates Allow longest questionnaires Interviewers can observe nonverbal communication Interviewers can ask all types of Qs, complex Qs, and can use extensive probes Disadvantages n n n Training, travel, supervision, and personnel costs are high Interviewer bias Less supervision than for telephone interviews 26

Web Surveys o Advantages n n o Lowest cost format Fastest Span geographic space Can include visual materials Disadvantages n n n Unrepresentative samples Cannot control conditions of survey No one present to clarify or probe 27

SURVEY INTERVIEWING o o o The Interviewer’s Role Interview Stages Training Interviewers Using Probes Interviewer Bias Computer-Assisted Telephone Interviewing 28

The Interviewer’s Role o o o Survey research interviewing is specialized interview in which the primary goal is to obtain accurate information from another person It is a social relationship with expectations, social roles, and norms Interviewers must obtain cooperation and build rapport yet remain neutral and objective A survey interviewer is nonjudgmental and never reveals his or her opinions, verbally or nonverbally The interviewer helps define the situation for respondents, determining whether they have the information sought, understand what is expected , are motivated, and provide relevant and serious answers 29

Interview Stages 1. 2. Introduction and entry The main part of the interview probe: a neutral request made by an interviewer to clarify an ambiguous answer, complete an incomplete answer or obtain a relevant response 3. The exit 30

Training Interviewers o o A professional-quality interview requires carefully selected and trained interviewers Researchers consider the interviewer’s physical appearance, age, race, sex, languages spoken, even the voice 31

Using Probes o o o Probes are not substitutes for writing clear questions or creating a framework of understanding for the respondent Unless carefully stated, probes might shape respondent’s answers Yet, flexible, conversational interviewing using many probes can improve accuracy on questions about complex issues 32

Interviewer Bias o o o An interviewer’s uncontrolled visible characteristics, including race and gender, often affect interviews and respondent answers Thus, you should note the race and gender of both interviewers and respondents In general, interviewers of the same gender or ethnic-racial group as the respondent get the most accurate answers 33

Computer-Assisted Telephone Interviewing o computer-assisted telephone interviewing (CATI): telephone survey technology that integrates interviewing over the phone with a computer for the questionnaire and data entry. 34