Experimental Design The Research Process Defining a Research

Experimental Design The Research Process Defining a Research Question

Research & the scientific process What is the scientific process? n n Rationalism Empiricism

Scientific theories Inductive theory n Specific to general Deductive theory n General to specific Functional theory n Elements of both Models n “mini-theories”

Inductive approach Theory Hypothesis Pattern Observations

Deductive approach Theory Hypotheses Observation Confirmation/Non-confirmation

Phases of a research study 1. 2. 3. 4. 5. 6. Idea-generating Problem definition Design of procedures Data collection Data analysis Interpretation

Research Designs* Constraint level Naturalistic observation Case study Correlational Differential Experimental • Taken from Graziano • Not all research studies fit neatly into one of these categories

Strengths of low constraint research Can be used to generate hypotheses Can be used to negate a proposition Can be used to identify contingent relationships

Limitations of low constraint research Cannot be used to test hypotheses Poor representativeness Poor replicability Observer bias Ex post facto fallacy

Strengths of correlational/differential research Good for situations where manipulation of an independent variable is not practical or ethical! Higher constraint than observations or case studies

Limitations of correlational/differential research Influence of confounding variables Correlation does not imply causation n A causes B, B causes A, some other factor causes A and B The researcher measures but does not manipulate the variables

Strengths of experimental designs Causation can be determined (if properly designed) The researcher has considerable control over the variables of interest Can be designed to evaluate multiple independent variables

Limitations of experimental designs Not ethical in many situations Often more difficult and costly

Developing the research question/hypothesis Initial idea Initial observations Literature search Problem statement (Graziano, 2000) Research hypothesis Operational definitions

Good characteristics of a problem statement States the expected relationship between variables The problem should be in the form of a question Implies the possibility of an empirical test of the problem

Problem statements Observations & Case studies n Given A what is the probability of B? Correlational research n Is variable A correlated to a specific change in variable B Differential research n Will group A differ from group B by variable X? Experimental design n Does variable A cause a specific change in variable B?

Operational definitions Definition of the variables of interest n n n How are they defined? How will they be measured? A good operational definition of variables defines the procedure so precisely that another researcher could replicate it

Research hypothesis Develop the problem statement into a specific testable prediction n n States the direction Represents a declarative statement e. g. , Brown bullheads exposed to PAH-contaminated sediments will develop skin tumors at a higher rate than controls

What is an experiment? An inquiry in which an investigator chooses the levels (values) of input or independent variables and observes the values of the output or dependent variable(s).

What is a statistical experimental design? Determine the levels of independent variables (factors) and the number of experimental units at each combination of these levels according to the experimental goal. What is the output variable? Which (input) factors should we study? What are the levels of these factors? What combinations of these levels should be studied? How should we assign the studied combinations to experimental units? Experimental unit: the unit we apply the factors on to get the response.

Example: soft drink beverage • What is the output variable? Taste of the drink; score 1 to 10 (from poor to good) • What factors and at which levels should we study? A, B • Type of sweetener • Ratio of syrup to water Low, High • Carbonation level • Temperature

Example: soft drink beverage • What combinations of factors should be studied? All 2 x 2 x 2 x 2 combinations. • How should we assign the studied combinations to experimental units? Assign equal number of units to each combination. (unit: the “null” beverage or say the plain water)

The Six Steps of Experimental Design Plan the experiment. Design the experiment. Perform the experiment. Analyze the data from the experiment. Confirm the results of the experiment. Evaluate the conclusions of the experiment.

Plan the Experiment Identify the dependent or output variable(s). Translate output variables to measurable quantities. Determine the factors (input or independent variables) that potentially affect the output variables that are to be studied. Identify potential combined actions between factors.

Example: Which brand of battery should we buy? • What is the output variable? Battery life. (in hours) • What are the input variables (factors)? Three available brands; Prices etc.

Design topics vs. variable types Response (output) Predictor (input) Continuous Categorical w. Response Continuous Categorical w. Standard surfaces (RS) ANOVA designs (eg. factorial w. Uniform designs (UD) designs) w. Optimal designs (OD) UD, OD

Prior experimental information The model is known but the parameters are not: The shape of the model is somewhat clear: The model is completely unknown: Optimal designs Response surfaces Uniform designs