MetaAnalysis Basics Concepts and History What is Metaanalysis

Meta-Analysis Basics Concepts and History

What is Meta-analysis? Summary of quantitative results of empirical studies – data analytic part of a systematic review – data analysis where studies are data Can be used like any other statistical technique (e. g. , to test a theory or moderator rather than to summarize) Effect size – quantitative association of IV with DV Model – an equation used to represent data M-A Considers a distribution of effect sizes Sort of an empirical sampling distribution Mean of the distribution Variance of the distribution Accounting for variance in the distribution

Common Objectives Show (and interpret) the distribution Find the mean Find the variance Explain the variance Look for problems Bias Outliers Sensitivity to decisions about data

Goals for You By the end of the class, you should be able to complete your own publishable meta-analysis. Some students have published papers that came from this class: Allen, T. D. , French K. A. , Dumani, S. , & Shockley, K. M. (2015). Meta-analysis of work-family conflict mean differences: Does national context matter? Journal of Vocational Behavior, 90 -100.

Literature Review Narrative – traditional, but less common now Meta-analytic Some do not distinguish meta-analysis from quantitative review. But meta-analysis is really just a kind of statistical analysis like regression. Metaanalysis uses ES, but regression uses individual data as observations. Most statisticians consider metaanalysis just data analysis where the studies have different magnitudes of errors.

Appropriateness of M-A Empirical vs. theoretical Quantitative vs. qualitative Study outcome (level 1) vs individual data (level 0) Single IV and DV for ES (congeneric measures – corr b/t measures of same IV or same DV should approach 1. 0) Same or similar design (e. g. , do not combine experiments and correlations)

Common Effect Sizes Correlation coefficient: r Odds ratio (converted to logit): ln(OR) Died Lived Exper A B Ctrl C D Standardized mean difference: d

Main Steps Find studies (Effect Sizes) Code ES (convert if necessary) Code study characteristics (e. g. , date of publication) Calculate distribution of ES (garden variety meta-analysis) Examine ‘moderators’ Sensitivity analyses (data and model) Draw conclusions Share (present and publish)

M-A Pros & Cons Structured, open to replication Superior to global impression & vote counting Quantitative relations between outcome & study characteristics Handles large k Time & effort Expertise in analyses Missing data & bias Exchangeability (apples & oranges) Methodological rigor (all studies vs. ‘good’ studies)

Example in R This course is not about R, it is about metaanalysis However, we will use R to do meta-analysis R is free R is fairly easy to use R can save a cluster of related commands – repetition The ‘metafor’ package in R is really good Many different meta-analysis algorithms Many different publication-quality graphs Many diagnostics just for meta-analysis

Dataset Mc. Daniel Metafor package contains several datasets. Mc. Daniel 1994 is a collection of studies in which employee interview scores were correlated with job performance measures. The effect size is the correlation, r. N is the sample size. There also moderators for kind of interview type = j, s, p – job related, situational, psychological Structure = u, s – unstructured, structured Download from Canvas & run in R

Easy, isn’t it? The example I just ran is your basic metaanalysis. Simple to do, but you must understand what it means and whether the analysis is a good representation of the data. That is why you should complete this course.