Chapter 6 Quantitative Research Designs Manipulation of independent

Chapter 6 Quantitative Research Designs Manipulation of independent variables Random assignment of participants to conditions 1 Experimental forms Quasiexperimental forms Nonexperimental forms Present : researcher controlled Present : natural variation Absent Present Absent Introduction to Communication Research, BU-CA

The Experimental Framework l l Used to determine causation Conducted in lab or other controlled setting – – l 2 To control for extraneous influences Independent variable is manipulated by researcher Participants randomly assigned to conditions or treatments Introduction to Communication Research, BU-CA

The Classical Experiment l Researcher controls – – l 3 Selection of treatment & control groups – based on theory Random assignment of participants to conditions – ensures participants in each condition are equivalent before any treatment Manipulation checks Introduction to Communication Research, BU-CA

Posttest Experimental Design 4 Introduction to Communication Research, BU-CA

Pretest-Posttest Experimental Design 5 Introduction to Communication Research, BU-CA

Factorial Experimental Design 6 l Treatment groups based on two or more independent variables l Dependent variable measured once after treatment given l Can test for main and interaction effects Introduction to Communication Research, BU-CA

Longitudinal Experimental Designs l l l 7 Multiple measurements of the dependent variable Time between measurements depends on the nature of the communication phenomenon studied Used for studying training effects or degree of retention Introduction to Communication Research, BU-CA

Evaluating Experimental Design l l l Strengths Researcher controls manipulation of IV Assumes that any effect on DV is the cause of the IV Precision l l l 8 Limitations Not all communication can be studied using experiments Other external influence may be the cause of DV change May not reflect reality Introduction to Communication Research, BU-CA

Quasi-Experiments l l l 9 Researcher relies on natural variation of independent variable Posttest, pretest-posttest, factorial, and longitudinal designs can be used Includes mock experiments, simulations, and field experiments Introduction to Communication Research, BU-CA

Non-experimental Designs l l 10 Also called correlational or descriptive studies Researcher does not control manipulation of IV Participants not randomly assigned to conditions Predictor and criterion are better labels for IV and DV as they do not imply causality Introduction to Communication Research, BU-CA

Researcher Effects and Procedural Bias l Introduced in interaction with participants – – l 11 Similarity of researcher to participants How researcher communicates to participants Researcher unknowingly encourages desired responses Demand characteristics created when topic has socially desirable elements Use research protocol to detail procedural steps Introduction to Communication Research, BU-CA

Comparing Research Designs 12 Introduction to Communication Research, BU-CA

Surveys, Questionnaires, and Polls l Most commonly used quantitative method – – 13 Used for obtaining information about what people do, and respondents’ attitudes or characteristics In experimental, quasi-experimental, and nonexperimental research designs Introduction to Communication Research, BU-CA

What is a Survey? l System for collecting comparable information across many people – – – 14 Self-administered or self-reports Face-to-face Telephone Mail Computer-assisted Introduction to Communication Research, BU-CA

Designing a Survey l l Develop the survey objective Evaluate existing questionnaires/surveys – – – 15 Recommended over creating your own Has undergone extensive testing and revision Minor changes are okay; substantial changes will require that you pretest or pilot test the questionnaire Introduction to Communication Research, BU-CA

Writing Your Own Questionnaire l l Start with literature review Designing survey items – – – 16 Straightforward One complete thought written in sentence or question format Respondent should know how to answer Avoid abbreviations and slang expressions Shorter items are better than long ones Introduction to Communication Research, BU-CA

Open Questions l Respondents use their own words to respond – l Consider what constitutes an adequate answer – l l l 17 Makes data less comparable and more difficult to interpret Build that request into the question Use a recall cue to draw participants’ attention to issue, topic, or timeframe Record everything participant says Code responses after all data is collected Introduction to Communication Research, BU-CA

Closed Questions l Respondents given a question or statement and given a set of responses to select from – – l 18 All responses must be known in advance Creates easily comparable responses Use a recall cue or stimulus statement to draw participants’ attention to issue, topic, or timeframe Introduction to Communication Research, BU-CA

Response Sets for Closed Questions l Nominal or categorical responses – – – l Likert-type scales – – l 5 - or 7 -point scale Includes middle or neutral response Semantic differential scales – 19 Exhaustive Mutually exclusive Equivalent Bipolar adjectives anchor 7 -point scale Introduction to Communication Research, BU-CA

Examples of Response Sets for Likert-Type Scales 20 Very often Fairly often Occasionally Rarely Never Very positive Generally positive Mixed Generally negative Very negative Completely agree Generally agree Unsure Generally disagree Completely disagree Strongly agree Agree Unsure Disagree Strongly disagree Introduction to Communication Research, BU-CA

Pretesting the Survey or Questionnaire l l l Also called pilot testing Should be done if you developed a questionnaire or modified an existing one Four approaches – – 21 Cognitive approach Conventional Behavior coding Expert panel Introduction to Communication Research, BU-CA

Surveys and Questionnaires as Part of a Sound Research Design l l l 22 Who will respond to the survey? What is the survey’s objective(s)? When will the survey be taken? Where? Why is this particular survey needed? Introduction to Communication Research, BU-CA

Criteria for Questionnaire l l l 23 Are the words simple, direct and familiar to all. Is the question as clear and specific as possible. Is it double question. Does the question have a double negative. Is the question too demanding? Are the questions leading or biased. Is the question applicable to all respondents. l l l Can the item be shortened with no loss of meaning. Will the answers be influenced by response styles. Have you assumed too much knowledge. Is and appropriate time referent provided. Does the question have several possible meanings. Are the response alternatives clear and mutually exclusive (and exhaustive. ( Introduction to Communication Research, BU-CA

Sampling Issues for Surveys l Sample size and response rate are not the same – l Acceptable response rates vary by survey technique – – 24 Response rate or return rate = number of people who provide usable responses after being asked to participate May be differences in those who choose to respond and those that don’t respond Unusable responses Introduction to Communication Research, BU-CA

Survey Reliability and Validity l Internal reliability must be computed for multiple item questionnaires – Cronbach’s alpha l l l Three types of validity should be considered – – – 25 Varies from 0 to 1. 00 Generally. 70 considered acceptable Content validity Face validity Construct validity Introduction to Communication Research, BU-CA

Analyzing Survey Data l l Data of all participants combined to create a picture of the whole Data is often limited to descriptive purposes Data can test existing models Data from closed questions – l Data from open questions – 26 Descriptive statistics Categorized or content analyzed Introduction to Communication Research, BU-CA