Quantitative Research Overview Nonexperimental Qualitative Quantitative Experimental Case

Quantitative Research

Overview Non-experimental Qualitative Quantitative Experimental Case study Phenomenology Ethnography Historical Literature Review Observational Survey Archival Meta-analysis Experiment Quasi-experiment

Observational • Definition - Directly observing naturally occurring behavior unobtrusively, typically in the field, but can also take place in laboratory settings • Pros - Goal is unobtrusive observation so that your presence does not alter the participant’s affect, behavior, cognition - Allows for continuous measurement - Avoid participant self report error • Other - Construct Validity – depends on usage External Validity – Excellent b/c naturally occurring Bias – coding bias Error – coding errors

Observational • Choose “Observational” if… - interested in naturally occurring behavior - interested in studying a phenomenon as it is naturally occurring - interested in gathering information about naturally occurring boundary conditions or moderators - interested in depth/breadth of continuous measurement

Survey • Definition - Using self-report measures in any type of collection method ---► in-person, online, telephone, and mail • Pros - Relatively easy to collect data - Multiple collection methods for larger and representative sample sizes • Other - Construct – depends on usage - External – excellent because large sample sizes and representative samples - Bias – social desirability bias (except online) - Error – participant self-report error

Survey • Choose “Survey” if… - your topic can be analyzed using self-report - interested in collecting a lot of data – many variables and questions - interesting in collecting a lot of data – many subjects, power - interested in representative sample and generalizability - have little money, time, resources (except for mail/telephone if using Survey Company)

Archival • Definition - Using previously collected materials to analyze new research questions by using quantitative (numbers) analysis • Pros - No longer restricted only to present-day people and events so access to larger sets of data - Unobtrusive so reduced chances of experimenter error • Other - Construct – Insensitive measures since not collected for purpose of your study, No control over how information collected so possibly flawed - External – depends on inclusion criteria - Bias – coding bias - Error – coding error

Archival • Choose “Archival” if… - Same as with “Historical”, such as • • interested in origins and growth interested in particular historical events no current data on point so look to past data want to “generalize” from past events to current or future events • research question can only be answered by previously collected data - Plus… • have resources like coders, time • want to minimize experimenter bias • want to synthesize and compare data quantitatively

Meta-analysis • Definition - A meta-analysis statistically combines the results of several studies that address a shared research hypotheses • Pros - Central tendency - whether X affects Y, is the effect significant, and how strong is that effect? - Variability - If there is heterogeneity, then look for moderating variables that explain the variability. Does the effect of X on Y differ with moderator? • Other - Construct – Depends on CV of included works - External – Depends on EV of included works - Bias – Inclusion/Exclusion bias, Interpretation bias - Errors – Inclusion/Exclusion error

Meta-analysis • Choose “Meta-analysis” if… - Same as with “Literature Review”, such as • have an argument that can be supported by published research • interested in “summarizing” the literature for a variety of reasons • interested in in “interpreting” the literature, for a variety of reasons • interested in communicating the “quality” of the literature, for a variety of reasons - Plus… • have resources like coders, time • want to minimize experimenter bias • want to synthesize and compare data quantitatively

Meta-analysis • What are those “reasons”? - Same as “Literature Review”, such as • No one has previously summarized and/or interpreted the primary articles • The literature has grown to the point that it necessitates guidance or direction • There is a new topic that cross-cuts many previous literatures so new Literature Review needed to synthesize disparate literature relevant to new topic • There are controversies or disagreements that need resolution or support from summarizing/interpreting the literature - Plus… • Interested in “overall effect” of literature • There are new/old “moderators” that you want to test and/or can be tested across studies

Experiment • Definition - Testing cause-and-effect relationships by: (1) random assignment of Ss (2) to two or more conditions (3) which differ in terms of (only) IVs • Pros - Can prove causation - Tight controls • Other - Construct – depends on usage - External – experiments are artificial; alternative is conduct field study but then problem is loss of control and influence of extraneous variables - Internal – see the information from the previous Power. Point slides about internal validity - Bias – experimenter bias - Error – experimenter error

Experiment • Choose “Experiment” if… - want to prove causation

Quasi-Experiment • Definition - Contains aspects of both experiments and nonexperiments because deficient in at least one of the three aspects of experimental designs • Pros - Depends on which aspects of experiment and which aspects of non-experiment are involved in the quasiexperiment • Other - Construct – depends on… External – depends on… Bias – depends on… Error – depends on…

Quasi-Experiment • Choose “Quasi-Experiment” if… - want scientific rigor of experiments but can’t satisfy all three requirements for variety of reasons (see next Power. Point presentation about all the types of quasi-experiments such as hybrid, matched-pairs, within-subjects, mixed-designs, and single-n studies

Advanced Sources • Observational Participant Observation: A Methodology for Human Studies, by Jorgensen, Sage Publications • Survey Chapter 9 (Survey Research) of The Handbook of Research Methods in Social and Personality Psychology, Edited by Reis and Judd • Archival Strategies and Techniques, by Hill • Meta-analysis Practical Meta-analysis, by Lipsey and Wilson • Experiment and Quasi-experiment Experimental And Quasi-experimental Designs For Research, by Campbell Experimental and Quasi-Experimental Designs for Generalized Causal Inference, by Shadish et al