Primary Data Collection Experimentation Chapter Seven Copyright 2006

Primary Data Collection: Experimentation Chapter Seven Copyright © 2006 John Wiley & Sons, Inc.

Learning Objectives 1. To understand the nature of experiments. 2. To gain insight into requirements for proving causation. 3. To learn about the experimental setting. 4. To examine experimental validity. 5. To learn the limitations of experimentation in marketing research. 6. To compare types of experimental designs. 7. To gain insight into test marketing. John Wiley & Son, Inc 2

What is an Experiment? To understand the nature of experiments. An Experiment: The researcher changes an explanatory, independent, or experimental variable to observe changes in the dependent variable. Experimental variable John Wiley & Son, Inc Dependent variable price total sales advertising market share 3

Demonstrating Causation To gain insight into requirements for proving causation • Causal Research: – The only types of research that has the potential to demonstrate that a change in one variable causes some predictable change in another variable • To Demonstrate Causation: – Concomitant Variation (Correlation – Appropriate Time Order Of Occurrence – Elimination Of Other Possible Causal Factors John Wiley & Son, Inc 4

Demonstrating Causation • To gain insight into requirements for proving causation Scientific Definition—Causation and Causality 1. X is only one of a number of determining conditions that caused the observed change in Y. 2. X can be a cause of Y if the presence of X makes the occurrence of Y more probable or likely 3. One can never definitively prove that X is a cause of Y but only infer that a relationship exists 4. Causal relationships are always inferred and never demonstrated conclusively beyond a shadow of a doubt. John Wiley & Son, Inc 5

Demonstrating Causation • To gain insight into requirements for proving causation Concomitant Variation: – A predictable statistical relationship between two variables—A caused a particular change in B • Appropriate Time Order of Occurrence: – A change in an independent variable must occur before a change in the dependent variable • Elimination of Other Possible Causal Factors – The change in B was not caused by some factor other than A John Wiley & Son, Inc 6

The Experimental Setting To learn about experimental setting • Laboratory experiments: – Conducted in a controlled setting. – Advantages of laboratory experiments: • Ability to control all variables • Greater internal validity – Disadvantages of laboratory experiments: • External validity • Not transferable to the actual marketplace John Wiley & Son, Inc 7

The Experimental Setting To learn about experimental setting • Field experiments: – Tests conducted outside the laboratory – Advantages • Creates realism of the environment – Disadvantages • Internal validity • No control over spurious factors – Action of competitors – Weather – economy John Wiley & Son, Inc - societal trends - political climate 8

Experimental Validity To examine experimental validity • Validity – Degree to which an experiment actually measures what it is trying to measure • Internal and External Validity – Internal Validity: • The extent to which competing explanations can be ruled out. – External Validity: • The extent to which causal relationships can be generalized to outside persons, settings, and times John Wiley & Son, Inc 9

Experimental Notation To understand the nature of experiments • Experimental Treatment – Factors whose effects are to be measured and compared • Experimental Notation: – X indicates exposure – O for observation – O 1 X O 2 for time periods – X 1 O 1 for simultaneous exposure and measurement X 2 O 2 John Wiley & Son, Inc 10

Extraneous Variables To examine experimental validity • Extraneous Variables: Threats to Experimental Validity – – History Maturation Instrument Variation Selection Bias • Randomization or matching – Mortality – Testing Effect – Regression to the Mean John Wiley & Son, Inc 11

Extraneous Variables • • To examine experimental validity Confounding Variables—extraneous causal factors because they confound the treatment condition Four Basic Approaches to Control Extraneous Factors – – Randomization Physical Control Design Control Statistical Control John Wiley & Son, Inc 12

Experimental Design, Treatment, and Effects To learn about experimental setting • Experimental Design – Ex Post Facto Research—non-experimental designs • Factors of the Experimental Design: – Researcher has control of variables and manipulates them. • • Treatment Subjects Dependent variable Plan for dealing with extraneous causal factors John Wiley & Son, Inc 13

Experimental Design, Treatment, and Effects • To learn about experimental setting Treatment Variable – Independent variable that is manipulated • Experimental Groups – Control Group – Test Group • --Experimental Effects – The effect of the treatment variable on the dependent variable John Wiley & Son, Inc 14

Limitations of Experimental Research To learn the limitations of experimentation in marketing research • The High Cost of Experiments • Time • Money • Security Issues • exposing to the actual marketplace • competitors • Implementation Problems • cooperation • contamination • lack of control group John Wiley & Son, Inc 15

Selected Experimental Designs To compare experimental designs • Three Pre-experimental Designs – Offer little or no control over extraneous factors • One-Shot Case Study – X O 1 – Basic weakness • No pretest observations • No control group of test units • Lacks internal validity John Wiley & Son, Inc 16

Selected Experimental Designs To compare experimental designs • One-Group Pretest-Posttest Design – O 1 X O 2 – Pre- and postmeasurements but no control group – Threats • History • Maturation • Only one pretest observation John Wiley & Son, Inc 17

Selected Experimental Designs To compare experimental designs • Static-Group Comparison Design Experimental Group: X O 1 Control Group O 2 – Pre-experimental with a control group – Threats • Absence of pretests • Not assigned to the groups randomly John Wiley & Son, Inc 18

Selected Experimental Designs To compare experimental designs • True Experimental Designs – Before and After with Control Group Design • Experimental Group (R) O 1 • Control Group (R) O 3 X O 2 O 4 – Threats • Mortality • History John Wiley & Son, Inc 19

Selected Experimental Designs To compare experimental designs – Solomon Four-Group Design • • • Second experimental group no pretest Second control group only posttest measurement Experimental Group 1: (R) O 1 X O 2 Control Group 1: (R) O 3 O 4 Experimental Group 2: (R) X O 5 Control Group 2: (R) O 6 John Wiley & Son, Inc 20

Selected Experimental Designs To compare experimental designs – After-Only with Control Group • Involves random assignment of test units to experimental and control groups, no premeasurement of the dependent variable. • Experimental Group: (R) X O 1 • Control Group: (R) O 2 John Wiley & Son, Inc 21

Selected Experimental Designs To compare experimental designs • Quasi-Experiments – Developed to deal with external validity – Researcher lacks complete over the scheduling of treatment or must assign treatment in a nonrandom manner. • Interrupted Time-Series Designs – The treatment interrupts repeated measurements. – Cannot control history – O 1 O 2 O 3 O 4 X O 5 O 6 O 7 O 8 John Wiley & Son, Inc 22

Selected Experimental Designs To compare experimental designs • Multiple Time-Series Designs – An interrupted time-series design with a control group – Experimental Group O 1 O 2 O 3 X O 4 O 5 O 6 – Control Group: O 1 O 2 O 3 O 4 O 5 O 6 John Wiley & Son, Inc 23

Test Markets To gain insight into test marketing • Test Markets Defined – Any research that involves: • Testing a new product or change in an existing marketing strategy. • The use of experimental or quasi - experimental procedures • Test Market Usage and Objectives – – Estimate of market share Effect on sales of similar products Characteristics of consumers Behavior of competitors • Simulated Test Markets John Wiley & Son, Inc 24

Test Markets • Direct Costs of Test Marketing • Costs include: – – – – Commercials Advertising Media Research POP Coupons Sampling trade allowances John Wiley & Son, Inc To gain insight into test marketing • Indirect Costs of Test Marketing • Costs include: – Management time – Diversion of sales activity – Negative impact of test market failure – Negative trade reactions to products – Cost of letting competitors know what the firm is doing 25

Test Markets To gain insight into test marketing • Decision To Conduct Test Marketing • Benefits – Good estimate of product’s sales potential – Identify weaknesses of the product and the proposed marketing strategy • Factors to Consider – Weight the cost and risk of failure – Likelihood and sped product can be copied – Damage an unsuccessful new product would inflict John Wiley & Son, Inc 26

Test Markets • To gain insight into test marketing Steps in a Test Market Study 1. Define the Objective 2. Select a Basic Approach 1. Simulated test market 2. Standard, or traditional, test market 3. Controlled test market 3. Develop Detailed Procedures for the Test John Wiley & Son, Inc 27

Test Markets 4. Select Markets for the Test 1. Minimum of 2 test sites 2. Geographically dispersed 3. Demographically representative of US 4. Run at least 6 months 5. Media 50% of current levels 6. Variety of media outlets 7. TV spill-over less 15% 8. TV spill-in less 15% 9. Dominant newspaper 10. Market should be at least 0. 2% of US no more 2% John Wiley & Son, Inc To gain insight into test marketing 5. 6. Execute the Plan Analyze the Test Results 1. 2. 3. 4. Purchase data Awareness data Competitive response Source of sales 28

Test Markets • Simulated Test Market (STMs) – • To gain insight into test marketing Not actual test markets—rely instead on laboratory Steps in the STM 1. 2. 3. 4. 5. Intercept consumer at shopping malls Screen Expose to the new product concept or prototype Give subject opportunity to buy new product After time interval interview those who made a purchase determine their assessment of it 6. Use the trial and repeat purchase for input into a mathematical model John Wiley & Son, Inc 29

Test Markets • To gain insight into test marketing Four Reasons for the Popularity of STMS 1. 2. 3. 4. Competitors less likely to know about test STMs can be completed quickly STMs are much cheaper than standard test STMs can be very accurate – Other Types of Test Marketing – Rolling rollout – Lead country strategy John Wiley & Son, Inc 30

SUMMARY – What is an Experiment – Demonstrating Causation – The Experimental Setting – Experimental Validity – Experimental Notation John Wiley & Son, Inc – Extraneous Variables – Experimental Design, Treatment and Effects – Limitations of Experimental Research – Selected Experimental Design – Test Markets 31

The End Copyright © 2006 John Wiley & Son, Inc 32