Survey Research Social Research Methods 2217 6501 Fall

Survey Research (調查研究) Social Research Methods 2217 & 6501 Fall, 2006 1

Part II. Constructing the Questionnaire (Continued) wording issues u questionnaire design issues u 2

Wording Issues (用字遣詞的考量) n Two wording issues: q q n Use simple vocabulary and grammar (使用簡單的 字彙與文法) Effects of specific words or phrases (特殊詞彙的影 響) Wording effects q q Avoid certain words Be aware of minor wording changes 3

Questionnaire Design Issues: Length of Survey or Questionnaire n Length of Survey or Questionnaire q q q No absolute proper length The length depends on the survey format and on the respondent’s characteristics For students’ projects: ~10 pp. 4

Questionnaire Design Issues: Question Order (問題順序) n Organization of Questionnaire q q q Remember: minimize the discomfort and confusion of respondents The opening section: easy and interesting questions first Background questions usually in the ending section Put questions on the same topic together Question topics flow smoothly and logically Always end with a “thank you” 5

Questionnaire Design Issues: Question Order Effects (順序效應): q q Order effects strongest for people who lack strong views, less educated, older respondents or those with memory loss (對立場不堅定、教育程度愈低、 年紀愈大、記憶力差的人影響越大) Respondents will be affected by the set of issues and order of question presentation in questionnaires 6

Questionnaire Design Issues: Question Order n Context Effects (脈絡效應) q q q Context includes question order, the effects of the interviewer, interviewing setting, etc. Evident in mail surveys Can do two things: n n q Use a funnel sequence (from general to specific questions) Test the question order Respondents answer questions based on a context of preceding questions and the interview setting. (前面的問題可作為回答後面問題的脈絡基 礎) n The more ambiguous a question’s meaning, the stronger the context effects. 7

Question Design Issues: Format and Layout (格式與設計) n Questionnaire layout q q q Questionnaire should be clear, neat, and easy to follow Have a cover sheet: for administrative use Give respondents instructions n q Instructions in different color or font Layout is crucial for self-administered or mail surveys n Details matters 8

Question Design Issues: Format and Layout n Question format q q q Make responses unambiguous: boxes, brackets, or numbers Arrows or instructions for contingency questions Use matrix questions (矩陣式問項) 9

Question Design Issues: Format and Layout 10

Questionnaire Design Issues: Nonresponse (未作答或拒答) n Nonresponse is a major problem for survey research. q q n n The issue of representativeness Nonresponse in most academic surveys: 25 -33% Five types of nonresponse: nonlocation, noncontact, ineligible, refusal to participate, incomplete participation Some techniques to reduce nonresponse q q Reduce each type of nonresponse Initial contact critical Use incentives tailoring 11

Part III. Types of Surveys: Advantages and disadvantages 12

Mail and Self-Administered Questionnaires (郵寄與自填式問卷) Advantages n The cheapest n Cover a wide geographical area n Convenient for respondents n Offer anonymity and avoid interviewer bias n effective Disadvantages n A low response rate n Do not know the conditions under which a mail questionnaire is completed n Cannot observe respondents’ reactions, characteristics, or settings n Some types of questions are not ideal 13

Web Survey: over the Internet or by email Advantages n Very fast and inexpensive n Allow flexible design and use visual image n Principles of paper questionnaire generally apply n efficiency Disadvantages n Coverage: sampling issues n Privacy and verification q n Technological fixes Design issues q q Various software and hardware combination Screen-by-screen questions A progress indicator Provide instructions 14

Telephone Interviews (電話訪談) Advantages Disadvantages n A popular survey n Relatively high cost and method: quickly reach limited interview legnth people across long n Cannot reach people distances without phones n Flexible, cost lower than n Possible interviewer face-to-face interviews bias n Computer-assisted n Some questions cannot technologies: CATI & be used IVR n Only notice serious disruptions 15

Face-to-Face Interviews (面對面訪談) Advantages n Have the highest response rates n Permit the longest questionnaire n Can observe the surroundings and use nonverbal communication and visual aids n Interviewers can ask complex questions and probes Disadvantages n High cost n Interviewer bias greatest n Interviewer supervision is less 16

Interviewing (訪談) n The role of the interviewer: difficult q q q n Survey interviewing is special Nonjudgmental and do not reveal their opinions Provide human warmth, sense of trust, and rapport Stages of an interview q q q Asking questions by following the exact wording Recording answers verbatim: never summarizes or paraphrases Use probes (追問: 中性的要求) effectively 17

Interviewing—interviewer bias (訪員偏 差) n n n Training interviewers Interviewer bias: concerning the issue of standardization Factors affecting interviewer bias q q Interviewer expectations Interviewers’ visible characteristics (race and gender) 18

Cultural Meanings and Survey Interviews n n The concern: how people create social meanings and cultural understanding? Reconsider naïve assumption model of survey interviews (pp. 310 -311) Remember a survey interview ≠ordinary conversation The standardization of survey research: the construction of a shared meaning among different people 19

Pilot Testing n n Pilot-testing survey interviews and questionnaires before implementation The purpose: to improve questionnaire design (see pp. 312) q n Cognitive interviewing Pre-test your questionnaire! 20

The Ethical Survey (合乎研究倫理的調 查) n The invasion of privacy q n n n Provide mutual trust and confidentiality Voluntary participation The exploitation of surveys and pseudosurveys Misuse survey results or use poorly designed surveys q Poor media reports 21

Summary: Survey research is complicated and requires great knowledge, planning, and skills work hard to minimize errors n be careful in analyzing data and generalizing results n 22