Research Designs in Education Qualitative vs Quantitative Experiments

Research Designs in Education Qualitative vs Quantitative Experiments vs Quasi-Experiments

Goals For Today 1. Help you read more efficiently for classes like this one 2. Help you understand the value of both quantitative and qualitative research 3. Help you understand more about how to evaluate causal claims

How to Read an Academic Research Article for Classes Like This

Know why you are being asked to read it Read it strategically, not front to back For classes like mine, look for: 1. The main (1 or 2) questions or arguments 2. The basic (1 or 2) conclusions 3. The very basics of the research design For other purposes, this is not enough

Academic research articles are almost all structured the same way 1. 2. 3. 4. 5. 6. 7. Abstract Introduction Review of theory and prior evidence Description of Research Design Presentation of Results Discussion and Conclusions References

For classes like mine, read in this order: 1. 2. 3. 4. 5. 6. 7. Abstract 1 Introduction 2 Review of theory and prior evidence Description of Research Design 4 (maybe skim) Presentation of Results Discussion and Conclusions 3 References

Qualitative vs Quantitative

QUANTITATIVE RESEARCH Studies social phenomena through quantifiable evidence, and often relies on statistical analysis of many observations to yield valid, reliable, and generalizable claims

QUALITATIVE RESEARCH Emphasizes understanding of social phenomena through direct observation, communication with participants, or analysis of texts, and may stress contextual subjective accuracy over generalizability

QUANTITATIVE Studies social phenomena through quantifiable evidence, and often relies on statistical analysis of many observations to yield valid, reliable, and generalizable claims

QUANTITATIVE Emphasizes understanding of social phenomena through direct observation, communication with participants, or analysis of texts, and may stress contextual subjective accuracy over generalizability

QUANTITATIVE Curriculum Tracking • Formally quantified outcome (math test scores) • Large probability sample to maximize generalizability • Description of broad and general patterns • Less attention to meaning or on-the-ground process

QUANTITATIVE Curriculum Tracking • Small, strategically, smaller selected sample • Description of process and meaning, in context • Less attention to generalizable patterns than to the mechanisms that generate them

QUANTITATIVE Racialized Peer Pressure • Formally quantified outcome (test scores) • Large probability sample to maximize generalizability • Description of broad and general patterns • Less attention to meaning or on-the-ground process

QUANTITATIVE Racialized Peer Pressure • Small, strategically, smaller selected sample • Description of process and meaning, in context • Less attention to generalizable patterns than to the mechanisms that generate them

OBVIOUS POINTS Neither is “better, ” more objective, more scientific, more useful, or more valid The two approaches are complementary, and help understand different aspects of social phenomena

LESS OBVIOUS POINTS Both kinds of research are expensive to conduct Qualitative vs Quantitative Few people are well trained to do both Arguments about which is “better” are based more on ignorance and myopia that on fact

Experiments vs Quasi-Experiments

CAUSALITY How can we establish that one thing “causes” an outcome in education research? Qualitative vs Quantitative X Y

CONFOUNDING Factors associated with X also matter for Y Qualitative vs Quantitative X Z Y

CONFOUNDING Factors associated with X also matter for Y Qualitative vs Quantitative X=Parent Involvement Z=Family Income Y = HS Completion

CONFOUNDING Factors associated with X also matter for Y Qualitative vs Quantitative X=School Funding Z=County Poverty Rate Y=Achievement

EXPERIMENTS Value of the treatment variable is assigned only through chance Qualitative vs Quantitative X Z Y

EXPERIMENTS Confounding is not possible Qualitative vs Quantitative X Z Y

QUASI-EXPERIMENTS Value of the treatment variable is not assigned through chance … and thus risks confounding Qualitative vs Quantitative X Z Y

QUASI-EXPERIMENTS Why not always do experiments? Because you can’t always randomize the treatment Qualitative vs Quantitative X=Parent Involvement Z=Family Income Y=H. S. Graduation

QUASI-EXPERIMENTS Why not always do experiments? Because you can’t always randomize the treatment Qualitative vs Quantitative X=School Funding Z=County Poverty Rate Y=Achievement

QUASI-EXPERIMENTS Statistical approaches are used to eliminate bias from confounding … but there is always a risk Qualitative vs Quantitative X Z Y

EXPERIMENT Levels of cultural capital assigned entirely by chance

QUASI-EXPERIMENT Levels of municipality-level homicide not assigned by chance Statistical procedures designed to minimize risk of bias from confoudning

NO CLASS ON THURSDAY