Substrate Type slate granite concrete Predation Caged Uncaged

Substrate Type = slate, granite, concrete Predation = Caged, Uncaged, Cage Control Experimental Design: Classic Looks and Causal Meaning Barnacl es

Experimental Design from a Causal Model Perspective • What is important today is not the ‘bestiary of designs’ • What is important is seeing the connection between a causal diagram and the experimental design • If you can diagram an experiment, the basics of the design and what choices you need to make will become (mostly) apparent • After that, it’s about how your decisions can increase precision

Where We Ended Tuesday: What Controls Barnacle Densities in the Intertidal Other Site Factors Predation Recruitment Substrate Type Barnacles wikimedia. org

Focusing on a Few Drivers x Predation Recruitment Substrate Type Other Site Factors x x Barnacles x • Experiment at single site to eliminate site factors and between site recruitment differences • Interested in both substrate type and, potentially, predation

Our Experimental Manipulations Focus on Predation and Substrate Predation = Caged, Uncaged, Cage Control ? Substrate Type = slate, granite, concrete Barnacles • We included caged controls to evaluate flow reduction effects of cages • We talked about a possible interaction, but, maybe? • We recognized possibility of lower external validity

Let’s Derive Some Inference! 1. Simple One-Way Designs and Associated Issues 2. Two-Way Designs to Solve Psuedoreplication 3. Factorial Designs 4. Reality Check for a Complex World

To Start: Substrate Only – One-Way Layout Substrate Type = slate, granite, concrete Barnacles • How many replicates of each treatment? • Placement of replicates • Scale over which to run experiment • Dispersion of treatments to ensure independence

A Word on Continuous v. Categorical Designs Substrate Rugosity = 0100 • Regression-based designs can work like ANOVA-based designs • You can assigns treatment levels evenly • You can assign discrete levels and add random noise Barnacles • The causal model is *THIS SAME*, it’s the details of implementation and statistical modeling that are different

Our One-Way Design Substrate Type = slate, granite, concrete Slate Granite Concrete Barnacles

How Many Replicates Do I Need? • Generally, p^(3/2) / ntot should approach 0 • Portnoy 1998 • So, 3 means • p^(3/2) ~ 5 • So, 5 / (3*n) should be close-ish to 0 • Practically, 5 -10

5 -10 Replicates? That’s it? • Not so fast! • The noisier the system and smaller the effect, the more replicates you need for good precision • Using NHST, noise = higher chance of Type II error – 1 -b = power

OK, How do I Determine Power (or Likely Precision) • SIMULATION! • Make a simulated data set with your design, fit a model, get SE of parameters or p-values • Rinse and repeat to see how often you fall in an acceptable range

Replicate Placement – In An Area of Minimal Variation in Other Conditions • Randomize coordinates • Or petri dish placement, labbies! • Once you accommodate, gradients, etc. , it’s a different design • Note – this is done at one *time* as well

Bad Replicate Placement: Non. Independence of Plots must be spatially or temporally separate This goes double for plots with the same treatment!

Bad Replicate Placement: Non. Independence of Treatments Pseudoreplication sensu Hurlbert

Subsampling As a Form of Pseudoreplication

Is it Pseudoreplication – How Many Replicates Are There? If treatments AND plots are nonindependent, this is a problem with n = 2, not 4

Subsampling (Nested Design) Can be Great! • If you take subsamples from true replicates, can minimize within replicate variation • Average subsamples • Or use mixed models . .

Repeated Measures as Subsamples • Let’s say these were samples through time • Analyze the same way • UNLESS – there is change through time • Then, need to consider plot AND a time effect . .

Let’s Derive Some Inference! 1. Simple One-Way Designs and Associated Issues 2. Two-Way Designs to Solve Psuedoreplication 3. Factorial Designs 4. Reality Check for a Complex World

What if There is a Gradient? Tide Height

Two-Way Blocked Design: Additivity with n = 1 per block/treatment Substrate Type Block What model would you use? Barnacles

Randomized Controlled Blocked Design (RCBD) • Randomize treatment placement within blocks • Accommodates potential other gradients • nblock = ntrt replicates

What Can Blocks Be? • Areas along a gradient • Plots close together in patches • Replicates run at the same time • And more!

Many Gradients? Latin Squares! Rows Columns Substrate Type Column Row Barnacles

Let’s Derive Some Inference! 1. Simple One-Way Designs and Associated Issues 2. Two-Way Designs to Solve Psuedoreplication 3. Factorial Designs 4. Reality Check for a Complex World

Two Way Designs are Not Just for Blocks Predation = Caged, Uncaged, Cage Control Barnacles Substrate Type = slate, granite, concrete But assumes additivity

Beyond Additivity: Factorial Designs Treatment A, Level 1 Treatment A, Level 2 Treatment B, Level 1 N=5 Treatment B, Level 2 N=5

Factorial Blocked Design: Does your Treatment Vary by Block? Substrate Type Block 1 Block 2 Block 3 What model would you use? Barnacles Useful for site variation, temporal variation, and more

Factorial Designs are Not Just for Blocks Predation = Caged, Uncaged, Cage Control Predation* Substrate Type = slate, granite, concrete Barnacles

You can Mix Things – e. g. , Blocks and Factorial Design (re-randomize each block) Block 1 Block 2 Block 3

Notes On Experimental Design Basics • Elements of any and all of these designs can be combined • Thinking about site, time, location, arrangement in a room, etc. , are all key in designing effective experiments • Always diagram out not just your system of interest, but the experimental system you are constructing

Let’s Derive Some Inference! 1. Simple One-Way Designs and Associated Issues 2. Two-Way Designs to Solve Psuedoreplication 3. Factorial Designs 4. Reality Check for a Complex World

Reality imposes constraints that make designs not always so straightforward

Taking Advantage of Natural Experiments

Taking Advantage of Natural Experiments • Diagramming is key to understand drivers and what you need to control for in a matching site • You have to have non-impacted sites that match • Or, non-impacted sites you can follow to make sure ”natural experiment” impacts are not just temporal variability

The Before-After Control-Impact Design (BACI) Site Impact Temporal Variability Response of Interest Schroeter et al. 1993

The Before-After Control-Impact Design (BACI) Difference Between Control and Impact Time Response of Interest Schroeter et al. 1993

BACI Data to See Difference

The Split Plot Problem • What if one treatment has to be applied to a non-independent set of a plots? • E. g. You have been gifted with barnacle cages that cover 3 plates at once, and you don’t have the $$$ to afford more • Or, you are studying the effect of a drug on different cell types in subjects, but all of the cell types from each subject have experienced the same treatment

Split-Plot Design: If You Have to Have Non. Independence, What are Your Sources of Error?

Split-Plot Design: Start Getting into Thinking About Error in your Causal Model Predation = Caged, Uncaged, Cage Control Predation* Substrate Type = slate, granite, concrete error Barnacles Random Effects – a Whole New World! “Plot” error

Final Thoughts • If you can create a simple enough “world” to run an experiment, you will save yourself a lot of headache! • But, don’t sacrifice external validity for headache if you can • Experimental design is an “art”, but always let your biological knowledge expressed in causal diagrams guide you