Social Research Methods Questionnaire Design 1 References Oppenheim

Social Research Methods Questionnaire Design 1

References Oppenheim, A. N. 1978 Questionnaire Design and Attitude Measurement. London: Heinemann l de Vaus, D. A. 1996 Surveys in Social Research, 4 th Ed. London: UCL Press (Chapters 6 & 15) l Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires, Cambridge: Cambridge University Press. Plus l Hoinville, G. , Jowell, R and associates (1985) Survey Research Practice. London: Gower l Moser, C. A. and Kalton, G (1971) Survey Methods in Social Investigation. Aldershot: Gower l 2

When is a questionnaire appropriate? Have good idea of what to expect l Literate respondents or l Time to administer if not literate l Interested in counts, frequencies etc. l Want to make generalisations to larger population l 3

Procedure l l l l Working out the aims of the questionnaire Selecting the question styles Designing the questions Structuring the questionnaire Piloting the questionnaire Revising the questionnaire Administering the questionnaire. 4

Information which can be obtained from questionnaires l l l l Facts and Knowledge (biography) Actions Opinions/beliefs Attitudes Motives Past Behaviour Likely Future Behaviour 5

Choosing the questions l l What type of information do you need from this question? How will the answers serve the aims of the questionnaire? How will you present the findings from answers to this question? Do you need numerical scores from the answers? 6

Open questions 1. What do you think the Government should be doing about global warming? 7

Closed questions 1. What do you think is the most important factor to consider when buying food? Tick just ONE box a) Value for money 1 b) Hygienic processing 2 c) No chemical additives 3 d) Good flavour when cooked 4 e) Easy to prepare 5 8

Open or closed questions? Open questions Closed Questions More representative Very easy to score of respondent’s true and analyse opinions Less open to Give predictable researcher’s bias outcomes Useful to indicate Useful for statistical points of interest or illustrations or weaknesses summaries 9

Questionnaire structure Title Introductory paragraph Content questions in sections use filter questions put more difficult questions towards the end do not give away answers to later questions End and thanks 10

11

Questionnaire layout l l Allow space Instructions (general, section, question, filter question) Numbers for computer coding One side of page only (space for comments) 12

Telephone Questionnaires Retention problem = too many categories in answer to remember 1. 2. 3. l l use numbered scale get direction of feeling then intensity include in question then repeat list Provide guides for interviewer on questionnaire – distinguish from questions Make clear what is to be read to respondent and what is not (e. g. instructions for prompt) 13

Checklists and ratings 14

Semantic Differentials How would you rate your mother on these scales? Warmhearted I I I Coldhearted Patient I I Impatient Self-assured I I I Always worried Unselfish I I I Selfish 15

Why pilot a questionnaire? 1. 2. 3. 4. To expose the questionnaire to new forms of thinking and understanding. To evaluate the quality of the answers provided by the questionnaire. To assess and revise the phrasing and content of the questions. To explore issues of questionnaire structure and question sequence. 16

Use Piloting to check for: l l l l l Variation in type of answer Meaning Redundancy Scalability Non-response (too much) Acquiescent response (too much agreement with agree/disagree Q) Flow Filtering Timing 17

Scaling Needs to be l Unidimensional l Linear (equal intervals) l Reliable l Valid (esp. for standard scales) 18

Major types of scales l Thurstone l l Lickert l l Judges score large list of items Guttman/Scalogram l l Judges rank agree/disagree items Items eliminated to form a ranked scale Paired comparisons l Pairwise comparing of items 19

Lickert Scales Not applicable Strongly disagree Disagree Neutral � There’s not much we can do to reduce carbon dioxide emissions. � I worry a lot about rising sea levels. New technology will stop global warming. Agree Strongly agree The experts are divided on global warming. � � 20

21

22

Some problems of question-writing: Leading questions Are you against giving too much power to trades unions? Why don’t you go more often to the supermarket? 23

Ambiguous questions Is your work made more difficult because you are expecting a baby? Also words like ‘Books’, ‘Dinner’ Multiple-content questions Have you suffered from headaches or sickness lately? 24

Over-complex vocabulary Simple alternatives Avoid big words and technical terms e. g. “Personality”, “Social mobility”, “Anomie”, “Patriarchy”. acquaint inform or tell assist help consider think initiate begin; start major important; chief; main; purchase buy require want; need reside live state say sufficient enough terminate end 25

Over-elaborate phrasing and too complex question Has it happened to you that over a long period of time, when you neither practised abstinence, nor used birth control, you did not conceive? Yes No l l l vague (what is a ‘long period’ too factual (‘has it happened to you that …’) complex - Long sentence and double negative 26

Implicit questions (based on assumptions) l l How many children do you have? (Asked of all people) How old is your car? 27

How often? Use either: l l l In the last week how often. …? On average …? When did you last. …? 28

Embarrassing questions e. g. sexual mores, alcohol consumption, law breaking, taboo subjects (e. g. death, menstruation) Create permissive atmosphere by range of possible answers. e. g. not “Does your child steal? ” but, “Which of the following best describes your child? 1. He or she never steals 2. He or she sometimes steals (including from home) 3. He or she has stolen on several occasions. 29

Vague terms e. g. a long time recently near you kind of fairly often on the whole 30

Prestige bias l l l “Doctors say that …” “Research has shown that …” “Is the government right to. …” 31

Patronising tone Avoid this. “Answer these questions to the best of your ability” “‘Dwelling’ means the place where you regularly live”. Use the Sun/Mirror style 32