Chapter 5 Producing Data 5 1 Designing Samples

Chapter 5 Producing Data 5. 1 Designing Samples

Key Concept • Statistical inference produces answers to specific questions, along with how confident we can be that the answer is correct.

Key Concepts

Key Concept

Key Concept

Examples: Call-in Polls, Online Polls

Example: Classroom survey, Mall Surveys

When taking a SRS You may use a Table of Random Digits

Steps in creating a Simple Random Sample Example: Page 336, Example 5. 5 Example: Select 5 Students at Random

General Framework for a method that uses chance to choose a sample

Example: Males/Females, Senior/Juniors….

Example: Randomly choose homerooms, then survey all students in the homeroom

Other Sample Design • Multistage samples select successively smaller groups within the population in stages, resulting in a sample consisting of clusters of individuals. Each stage may employ an SRS, a stratified sample and combine them to form the full sample.

Cautions about Sample Surveys Example Undercoverage: Phone Poll about the role of religion performed on Sunday Morning.

Cautions about Sample Surveys Response Bias Example Response Bias: IRS Poll of Are you Honest in filing your tax return? Do you still wet your bed at night?

Homework • Read 5. 2 • Exercises #1 -13, 15 -25, 28

Chapter 5 Producing Data 5. 2 Designing Experiments

Observation and Experiment An observational study observes individuals and measures variables of interest but does not attempt to influence the responses An experiment, on the other hand, deliberately imposes some treatment on individuals in order to observe their responses.

Experimental Units, Subjects, Treatment The individuals on which the experiment is done are the experimental units. When the units are human beings, they are called subjects. A specific experimental condition applied to the units is called a treatment.

Factors, Levels The explanatory variable in an experiment are often called factors. Many experiments study several factors. In such an experiment, each treatment is formed by combining a specific value (often called a level) of each of the factors. Example 5. 14 on page 355

Example 5. 14, page 355 Factor B: Repetitions 1 Time 3 Times Factor A: 30 sec Length 90 sec 1 4 2 5 5 Times 3 6 This experiment has 2 factors: length of commercial with 2 levels and repetitions, with 3 levels. The 6 Combinations of one level of each factor form 6 treatments. Look for the optimal combination.

Comparative Experiments Lab experiments in the sciences and engineering often have a simple design with only one treatment applied to all the units. The design of such an experiment looks like: Observation 1 Treatment Observation 2

Example: Comparative Experiments Gastric Freezing Example 5. 16 on page 358 Poorly designed experiment. Did not take into account the placebo effect. Placebo Effect - A beneficial effect in a patient following a particular treatment that arises from the patient's expectations concerning the treatment rather than from the treatment itself.

The basic Principles of Experimental Design of Experiments are 1. Control the effects of lurking variables on the response, most simply by comparing two or more treatments. 2. Replicate each treatment on many units to reduce chance variation in the results. 3. Randomization – use impersonal chance to assign experimental units to treatments.

Control Group The group of patients who receive a sham treatment is called a control group, because it enables us to control the effects of lurking variables on the outcome. Control of the effect of lurking variables is the first principle of statistical design of experiments. Uncontrolled studies of new treatments give a much higher success rate than proper comparative studies.

Replication • Use enough subjects to reduce chance variation

Completely Randomizing Experiments The use of chance to divide experimental units into groups is called randomization (the second principle of statistical design of experiments). In a completely randomized experimental design all experimental units are allocated at random among all the treatments.

Randomization Example: 40 Subjects, 1 Factor, 2 Levels

Other Experimental Designs • A block is a group of experimental units or subjects that are similar in ways that are expected to effect the response to the treatments. In a block design, the random assignment of units to treatments is carried out separately within each block.

Block Design: Example 5. 20 page 367

Other Experimental Designs • Matched pairs designs compare just two treatments. • Redesign Shoe – Randomly assign subject to two groups. Group 1 wears new design on left foot, old design of right foot. Group 2 wears new design on right foot, old design on left foot. After one month, wear the opposite design. After an additional month compare wear and tear.

Cautions about Experimentation • To avoid hidden bias, a double-blind experiment is employed where neither the subjects nor the people who have contact with them know which treatment a subject received.

Cautions about Experimentation: Example 5. 25, page 370 • Lack of realism – Real marijuana used in Treatment Group 1, Placebo Marijuana use in Treament Group 2. • One Group Complained…Guess Who? ? • Also, if subject know they are involved in an experiment, their behavior is often modified.

The Logic of Experimental Design • Randomization produces groups of experimental units that should be similar in all respects before the treatments are applied • Comparative design ensures that influences other than experimental treatments operate equally on all groups • Therefore, differences in the response variable must be due to the effects of the treatments.

Statistical Significance An observed effect too large to attribute plausibly to chance is called statistically significant. Note: Experiments with many subjects are better able to detect differences among the effects of the treatments than similar experiments with fewer subjects.

Basic Principles of Statistical Design of Experiments 1. Control of the effects of lurking variables on the response, most simply by comparing several treatments. 2. Randomization, the use of impersonal chance to assign subjects to treatments. 3. Replication of the experiment on many subjects to reduce chance variation in the results.

Lack of Realism • The most serious potential weakness of experiments is lack of realism. The subjects or treatments or setting of an experiment may not realistically duplicate the conditions we really want to study. • Example 5. 14 page 278

Homework • Exercises #33 -38, 40 -44, 45 -49, 51 -56 • Complete Take Home Quiz