Advanced MMBR Conjoint analysis Conjoint analysis Multilevel models

Advanced MMBR Conjoint analysis

Conjoint analysis -> Multi-level models You have to understand: - What it is - Which different kinds of Conjoint Analysis there are - How it can be of use in typical TIW research - How it can lead to different kinds of statistical analyses (of the repeated measures or multi-level kind) For this, you can use these slides AND the literature online In the laptop exam, running a repeated measures analysis is part of the requirements. Advanced Methods and Models in Behavioral Research

The logic of the course • binary Y logistic regression • conjoint analysis: way of data collection that might come in handy "repeated measures" / "multilevel" data We practice on self-collected data some practice/training in survey design and execution Advanced Methods and Models in Behavioral Research

Conjoint Analysis Underlying assumption: for each user, the "utility" of a product can be written as - 10 Euro p/m 2 year minimum free phone. . . How do you rate this proposal? (-5. . . +5) U(x 1, x 2, . . . , xn) = c 0 + c 1 x 1 +. . . + cn xn Advanced Methods and Models in Behavioral Research

Two kinds of research questions • Which phone do people tend to prefer? • How do different attributes of a proposition affect the utility of a proposition? (and how does that differ across different kinds of people) Advanced Methods and Models in Behavioral Research

Why is this important? It is an important tool in social science when you want to investigate how someone’s behavior or evaluations depend on circumstances and a useful tool in typical TIW Master’s Theses Advanced Methods and Models in Behavioral Research

One of the main advantages = more data: Example - Adoption of technology 1. If you ask for behavior only (“did you adopt” etc), then you get one piece of info person, and the rest you have to infer by comparing different persons. -> OK, and gives “real” behavioral data, but data are sparse. 2. If you offer different scenarios and then ask whether someone would adopt, you get how adoption depends on the context for this person. -> Richer data person, but not behavioral Given that sample sizes of >200 are often necessary, often option 2 is more feasible than option 1. Advanced Methods and Models in Behavioral Research

Or, use it on. . . New light bulb - price of bulb - color of light - expected lifetime Submit a motion in parliament -. . . Would you be interested in buying this light bulb? (-5. . . +5) - strongly favor issue personally core of party's strategy many other parties against it gets a lot of media attention. . . Do I submit a motion? (-5. . . +5) ? ? -. . .

Example: properties of mobile phones Suppose you get a phone that has … - large b/w screen long battery life not flashy costs 6 Euro/month free monthly contract How do you rate this option? (-5. . . +5) Hmm. . . How would that work anyway? 1. Give each person 200 of these cases (“vignettes”) 2. Per person, run multiple regression on their 200 answers -> you get (personal) values for each of the dimensions. You could then: 1. Create groups of people with the same kind of values, or … 2. … get an estimate of the average trade-off between dimensions, or … 3. … compare different groups of respondents Advanced Methods and Models in Behavioral Research

The data would look like this: ISSUES Y D 1 D 2 D 3 D 4 D 5 id +4 -1 -1 0 1 • The D-values are chosen by the researcher -> experiment -3 1 1 1 0 -1 1 0 0 0 1 0 -1 1 0 1 +1 … … … 1 • 200 is way too much +1 … … … 1 -1 … … … 1 • 35 = 243 > 200 (or even: 410 = …) +4 … … … 1 … … … … • If you look at the complete data set, this is repeated measures: the data are clustered -> standard multiple regression will not work Other considerations - How many dimensions? How many levels per dimension? How many cases person? Which cases from all possible cases person? How many persons (sample size)? Judgment or choice? How do I make sure that all given cases are ok? Advanced Methods and Models in Behavioral Research …

- Sidestep: design of experiments literature • Suppose you have a large set of potential vignettes (way too large to use them all) • One option: choose a random subsample of all possible vignettes • . . . but that might not be optimal, here’s why: -1 (X’X) n g i s e d l a m i t p o D Taguchi de sign A, C, E, T, G, I, V – optimal design Advanced Methods and Models in Behavioral Research

Alternatives (1) • Ask people directly what their favorite phone is If you could choose, which phone (and monthly plan) would you prefer? + much easier to collect - you will end up with as many suggestions as you have respondents - does not capture trade-offs between attributes Advanced Methods and Models in Behavioral Research

Alternatives (2) • Ask people directly for "their" utility model For instance: Which of the following factors do you find important when it comes to <buying a phone> / <adopting an innovation> / <choosing a buyer on e. Bay> /. . . ? Please divide 100 points over the following factors: - price - battery life -. . . + much easier to collect - asks for introspection about choice process, not for a “real” choice - nonlinear effects can never occur like this Advanced Methods and Models in Behavioral Research

Advantage of conjoint analysis (subtle) Usually, the comparison of interest is within persons person person 1 2 3 4 5 But if you consider only the <sold phones get the comparison between persons >, you (to get a feeling for this difference, think about what would happen to phones that are typically evaluated in 2 nd place)

Disadvantages of conjoint analysis • All fictituous decisions (do you like / would you buy / how would you evaluate. . . ) • (Often) assumes weighted average. This does not allow "all or nothing" weights • Can be quite complicated and/or boring for the respondents • Can be quite complicated for the researcher, both to implement and to analyze • . . .

Kinds of Conjoint Analysis 1. CVA: Conjoint Value Analysis Respondent evaluates single vignette at a time 2. ACA: Adaptive Conjoint (Value) Analysis Adapting the cases you offer based on previous answers of the respondent. 3. CBC: Choice Based Conjoint Asking for preferences between 2 (or 3 or 4) cases. 4. PP-CBC: Partial-profile Choice Based Conjoint Comparing only part of the attributes of the cases. - 10 Euro p/m 2 year minimum free phone. . . How do you rate this proposal? (-5. . . +5) Advanced Methods and Models in Behavioral Research

1. CVA: Conjoint Value Analysis Respondent gets to evaluate one vignette at a time Note • • Researcher chooses the dimensions The variance of the multiple regression estimator equals (X’X)-1 (with X the matrix of D’s) • • This implies that choosing the subset of cases that you are going to use, affects how broad your confidence intervals will be -> Experimental design literature: full-factorial design, Doptimal designs, etc Advanced Methods and Models in Behavioral Research

2. ACA: Adaptive Conjoint Analysis Adapting the cases you offer based on previous answers of the respondent. • You give, say, 20 cases to each respondent. • Which cases the respondent gets, depends on his answers to the first couple of cases. Much more efficient than just randomly choosing cases But impossible to do off-line or by phone, and even online quite difficult to implement Advanced Methods and Models in Behavioral Research

Software … is problematic - Rare (part of general survey software mostly)… Expensive … Visually not very nice… No output of raw data … Advanced Methods and Models in Behavioral Research

Sawtooth software / SKIM Research www. sawtoothsoftware. com www. skimgroup. com/software Advanced Methods and Models in Behavioral Research

3. CBC: Choice Based Conjoint Asking for preferences between 2 (or 3 or 4) cases. - 10 Euro p/m 2 year minimum free phone internet = per Mb - 15 Euro p/m 1 year minimum phone costs 70 internet = per Mb - 30 Euro p/m 2 year minimum free phone internet = free Which of these three offers do you prefer? Or … Rank these three offers Or … Distribute 10 points over these 3 offers Advanced Methods and Models in Behavioral Research

3. CBC: Choice Based Conjoint (2) Resp chooses preferred vignette Issues: • Which of these kinds of questions works best? We don’t know. • If you have only choice data (or ordinal data), how can you arrive at values for the different dimensions? Y D 1 D 2 D 3 D 4 D 5 id set 1 -1 -1 0 1 1 1 0 -1 1 1 0 0 0 1 0 -1 1 0 1 2 1 … … … 1 2 0 … … … 1 2 1 … … … 1 3 0 … … … 1 3 … … … … 3 Y = 0/1 => Logistic regression, but … … that does not use all the available information and if we use the data for all persons we have dependencies in the data (more cases person) and … what if we have ordinal data? Advanced Methods and Models in Behavioral Research

Which method to choose when? We have rules of thumb only … www. sawtoothsoftware. com/products/advisor/ Advanced Methods and Models in Behavioral Research

Sample sizes in conjoint analysis • This is difficult. . . for many reasons – Quite a lot of degrees of freedom (#people, vignettes pp, dimensions per vignette, values per dimension, . . . ) – Not clear up front how different answers from the same person are going to be • Rules of thumb (see paper online) • Generally 150 – 2000 participants • Comparing groups? 200+ participants per group • Testing purposes not comparing groups: 300+ Advanced Methods and Models in Behavioral Research

Sample sizes in conjoint analysis (2) For choice-based conjoint: nta > 500 c With n t a = number of respondents = number of tasks person = number of alternatives per task c = number of “analysis cells” (main effects model: max number of levels in an attrib. ) (with interactions: max number of interaction levels) Advanced Methods and Models in Behavioral Research

Statistical issues We need to know how we can deal with “nested data” (for instance, more than one answer person) Can't the Sawtooth people take care of this? Advanced Methods and Models in Behavioral Research

Marketing vs other social science Marketing ("conjoint analysis") Try to come up with the weights for each dimension for each respondent (e. g. to then segment the population) Other social science ("vignette study") Try to come up with the average weights for groups of people (so other aim, and you need less choices person)

To Do • Read and understand the literature on the course website on Conjoint Analysis. Advanced Methods and Models in Behavioral Research