Experiments in Marketing Research Dr Michael R Hyman

Experiments in Marketing Research Dr. Michael R. Hyman, NMSU

Criteria for Establishing Causal Relationships • Concomitant variation • Temporal ordering of variables • Control over other possible causal factors 2

Preliminary Definitions: Independent Variable • Experimenter controlled variable • Value can be manipulated by experimenters to whatever they wish it to be 3

Dependent Variable • Value expected to be dependent on experimenter’s manipulation • Criterion or standard by which results are judged 4

Test Units Subjects or entities whose response to the experimental treatment are measured or observed 5

What is an Experiment? • Research investigation under controlled conditions • At least one independent variable is manipulated • Its effect on at least one dependent variable is measured • Purpose: To test hypothesis 6

Unit Pricing Experiment 7

Basic Experimental Design Issues • Manipulation of independent variable • Selection of dependent variable • Assignment of subjects (or other test units) to treatments/conditions • Control over extraneous variables 8

How Can Experimenters Control for Extraneous Variation? • • • Eliminate Extraneous Variables Hold Conditions Constant Randomization Matching Subjects Presentation Order Blinding 9

Control Groups Isolate extraneous variation 10

Erroneous Inferences Caused by Lack of Control 11

Field vs. Laboratory Experiment 12

Types of Research Environments 13

Artificiality of Laboratory vs. Field Experiments 14

Laboratory Experiment Field Experiment High Internal Validity High External Validity Demand Artifacts High Few Extraneous Variables Demand Artifacts Low Many Extraneous Variables High Control Low Cost High Cost Short Duration Long Duration Subjects Aware of Participation Subjects Unaware of Participation Few Participants Many Participants Easier to Do Harder to Do 15

Demand Characteristics Experimental procedures that intentionally hint to subjects something about experimenter’s hypothesis • Guinea pig effect • Hawthorne effect 16

Internal Validity • Ability of experiment to determine if the experimental treatment was the sole cause of changes in a dependent variable • Did the manipulation do what it was supposed to do? 17

Threats to Internal Validity • • • History Maturation Testing Instrumentation Selection bias Mortality 18

Type of Extraneous Variable Example History - Specific events in the environment between the Before and After measurement that are beyond the experimenter’s control A major employer closes its plant in test market area Maturation - Subjects change during the course of the experiment Subjects become tired Testing - The Before measure alerts Questionnaire about the traditional or sensitizes subject to nature of role of women experiment or second measure. triggers enhanced awareness of women in an experiment. 19

Type of Extraneous Variable Example Instrument - Changes in instrument result in response bias New questions about women are interpreted differently from earlier questions. Selection - Sample selection error because of differential selection comparison groups Control group and experimental group is self-selected group based on preference for soft drinks Mortality - Sample attrition; some subjects withdraw from experiment Subjects in one group of a hair dying study marry rich widows and move to Florida 20

21

Types of Experimental Designs 22

Sample Marketing Experiments Using Different Experimental Designs 23

24

25

26

27

28

29

30

31

32

33

Test Marketing 34

Test Market Field test of a new product or marketing mix element(s) of a current product using experimental or quasi-experimental designs 35

Uses and Objectives • Evaluate proposed natural programs • Estimate market shares and volumes • Estimate cannibalization rate on existing product line • Collect classification data on potential customers • Analyze competitor reactions 36

Benefits and Disadvantages Benefits • Estimate sales potential under realistic conditions • Find and correct weaknesses in marketing mix Disadvantages • Costs • Tipping hand to competitors 37

What to Consider • Tradeoff of costs and risk of product failure versus potential profits and probability of success • Speed of competitor response • Cost to produce product for test market • Impact of failure on company reputation 38

Cost of Two-Market Test Direct costs: $500, 000 • POP materials, coupons, samples, syndicated research, media time/space, ad agency payments, initially high trade allowances Indirect costs • Management time, diversion from current products, negative internal/external impact of test failure, etc. 39

Steps in Test Market Study 1. Define objectives 1. Estimate market share, purchase frequency, cannibalization rates 2. Discover who buys product and where 2. Select testing approach 1. Standard 2. Simulated (mall testing facility) 3. Controlled test market (e. g. , Behavior. Scan) 40

Steps in Test Market Study 3. Develop marketing plan 4. Select test markets 1. Should reflect regional differences if important, but have similar demographics and little media spillover from/to other markets 2. Avoid over-tested and idiosyncratic markets 41

42

Steps in Test Market Study 5. Run test markets 1. Sales estimates take 6 -12 months because initial estimates of repeat purchases high 6. Analyze results 1. Check purchase data (i. e. , trial versus repeat purchases), awareness data, competitive response, and sales of all similar products 43