http theveryclosedcircle blogspot com201204360 metaanalysis html Special Topics

http: //theveryclosedcircle. blogspot. com/2012/04/360 -meta-analysis. html

Special Topics in Biological Data Analysis: Meta-analysis Jarrett Byrnes UMass Boston Spring 2014

IS ONE STUDY EVER DEFINITIVE?

Tilman et al. 1997 Science Huston et al. 2000 Nature

HOW CAN WE DETERMINE THE ANSWER?

Narrative Reviews • Author searches and summarizes evidence from their perspective • Can be biased by author’s own opinion or searching abilities (if they did a search)

“In a strictly defined sample of competition studies using controlled field experiments, covering 215 species and 527 experiments competition was found in most of the studies, in somewhat more than half of the species, and in about two-fifths of the experiments. ” Connell 1983 “Rare until recently, field-experimental studies of interspecific competition now number well over 150. Competition was found in 90% of the studies and 76% of their species, indicating its pervasive importance in ecological systems. ” Schoener 1983

Systematic Review • Defined repeatable criteria for searching • Clear inclusion/exclusion strategy • Systematic coding/grading of results from studies • Not often done outside of medicine • But see http: //www. environmentalevidencejournal. org/ http: //www. campbellcollaboration. org/ http: //www. cochrane. org/

Vote-Counting “Rare until recently, field-experimental studies of interspecific competition now number well over 150. Competition was found in 90% of the studies and 76% of their species, indicating its pervasive importance in ecological systems. ” Schoener 1983 • Says nothing about strength of effect • Weak evidence = no evidence, biased by experiments with large and small sample sizes

One Giant P-Value • One study contains p(data|Hypothesis) for one data set • Why not combine them? For example: p=1 -pnorm(Szi, m=0, s=1) • Unequal sample sizes, methodologies, assumptions, etc……

Meta-analysis! An analysis of analyses! Systematic combination of information from published studies to derive the weight of evidence to a particular question

The First Meta-Analysis Pearson, K. (1904). Report on certain enteric fever inoculation statistics. British Medical Journal, 3, 1243 -1246.

1970's: Dawn of Modern Meta-analysis • First, medical: Anti-coagulants as a treatment for heart attacks (Chalmers 1977) • Psychotherapy (Smith and Glass 1977) • Education: Class size and outcomes (Glass and Smith 1979) • Opposition: "an exercise in Mega-Silliness"

Why would you object to metaanalysis as a tool?

Characteristics of Meta-Analysis • Systematic search for studies • Strict criteria for inclusion • Common effect size Example: Hedge's D • Analysis that accounts for difference in power of different studies

Why did it take so long to get to Ecology? eigenfactor. org

Example 1: Strength of Competition Gurevitch et al. 1992

Example 2: Does Diversity Influence Ecosystem Function

Cardinale et al. , Nature 2006 93 studies 175 experiments LRAve = ln(yp / ym) LRMax = ln(yp / ymax) Standing stock or Resource depletion Monoculture, m Most diverse polyculture, p Species richness

LRAve LRMax

Standing stock (YS / Ym) Resource depletion (YS / Ym)

Example 3: Are Seagrass Beds Changing? Waycott et al. 2009 PNAS

For LOTS more in EEB • Ecology Volume 80 Issue 4 in 1999 • Ecology Volume 91 Issue 9 in 2010: Is meta-analysis a mega-mistake when it comes do diversity and productivity?

SO WHAT ARE WE DOING HERE?

Course Goals 1. Learn how to do a meta-analysis 2. DO a meta-analysis – and publish it! 3. Learn how to teach yourself new statistical skills 4. Learn how to participate in a working group type of environment

Course Outline • Now-March: Gathering and Assessing Data • March-April: Analytic Techniques • April-May: (not so) Unique Challenges

Class Structure • 1 -2 or 2: 15 You lecture on the topic of the week • 2: 15 -2: 45 Code or other demo (if needed) • 2: 45 -4: Working group

I’m going to lecture? ! • You will encounter many new analytic techniques in the future • This class provides an introduction to learning a new topic • Meet with me at 1 pm on Tuesday to go over your lecture for the week • I’ll be lecturing on some of the hairy stuff & all of the code

a Requirement • • We will use R for analysis metafor package for meta-analysis nlme and lmer for mixed models MCMCglmm and JAGS for Bayesian methods http: //www. metafor-project. org/

http: //swirlstats. com/ By March 1 st!

http: //www. nceas. ucsb. edu/handbook_of_meta-analysis

Other Books! • Introduction to Meta-Analysis. 2009. Michael Borenstein, Larry Hedges, Julian Higgins, and Hannah R. Rothstein. Wiley. • The Handbook of Research and Synthesis and Meta-Analysis, 2 nd Edition. Edited by Harris Cooper, Larry Hedges, and Jeffry Valentine. Russel Sage Foundation.

What will you do in this course?

Choose a Question

Search the Literature, with a Protocol http: //agris. fao. org/

(Re)Choose a Question

What will you do in this course?

Harvest Data http: //arohatgi. info/Web. Plot. Digitizer/app/

Design a Database

Generate Strict Meta-data

Go back to the beginning… • As you begin to search, you will learn about what information is out there • Starting over (sometimes more than once) is normal!

Be Circumspect: Assess Bias, Power, Sensitivity, etc.

Analysis! http: //www. metafor-project. org/

Publish?

Is species diversity decreasing at local scales in the sea?

Problems with Vellend et al. • Sampling bias • Low power of test misses trend • Didn't test fundamental assumptions of conclusions • Extrapolation from data they had to conclusions about things not measured • Terrestrial only

Is species diversity decreasing at local scales in the sea?