Convince them its real or tell them its

Convince them its real or tell them its hypothetical? The Effect of Alternative Explanations for Implausible Alternatives in Choice Experiments Mark Morrison and Olvar Bergland

The problem of hypothetical bias • In CVM and CM there is an emphasis on ensuring that the scenarios presented to respondents are as real and consequential as possible • Otherwise can lead to hypothetical bias and unreliable results

Mitchell and Carson (1989) • Suggested several factors that affect the realism of a hypothetical scenario, including respondents: – being familiar with the key scenario elements (eg the good being valued, and the payment vehicle) – being able to understand the scenario, and – finding the scenario plausible.

Arrow et al (1993) • “If CV surveys are to elicit useful information about willingness to pay, respondents must understand exactly what it is they are being asked to value…and must accept the scenario in formulating their responses”.

Experimental evidence • Cummings and Taylor (1998) demonstrated that it is only when respondents “place somewhat high probabilities on the realness of the survey”, that hypothetical and real estimates converge.

Implausible/Unlikely Alternatives in Choice Experiments • Arise from the use of orthogonal designs • Hard to always understand what respondents will consider implausible • Can be reduced by the use of correlated/ nested designs

Options for explaining implausible/unlikely alternatives • Many studies have sought to explain why implausible/unlikely alternatives are possible despite focus group results indicate that many doubt the veracity of these comments (eg Blamey et al 1997; Morrison, Bennett and Blamey, 1997, Morrison and Bennett 2004). • Others have admitted that they are hypothetical and explained why they are being asked (eg Mallawaarachchi et al 1999, Morrison 2000).

Examples • The outcomes in each of the options have been specifically defined so that you have a broad range of choices. Within this range, some options may seem strange according to your experience, but bear in mind that there are many different ways of managing water. For example, wetland vegetation requires regular floods of various sizes to remain healthy. In contrast, waterbird breeding requires flooding of sufficient size at a suitable time of year (Morrison, Bennett and Blamey 1999).

Examples Some of the options may also seem a little odd – for example an option that involves less area and more native species. However, such a possibility could occur if the smaller area is all high quality habitat, and some of the larger area is of lower quality (Lockwood and Carberry 1998).

Examples • To keep matters simple, we do not describe how each management option would work. For example, we do not specify how trees would be protected from clearing, and which areas of the Desert Uplands would be protected. Rather, we simply describe what effect it would have on jobs, the natural environment etc. Indeed, there a lot of ways that the issue of tree clearing can be managed. When answering the questions below, please keep in mind that some options which seem a little odd may in fact be quite possible (Blamey, Rolfe, Bennett and Morrison 2000).

Examples • We would now like to find out your views about some options for the use of land in the Herbert River District. To do this we have prepared eight sets of hypothetical options that produce a broad range of possible land use outcomes. In each question, Option A describes current practice and Options B and C describe the outcomes associated with two possible incentive schemes. By indicating the option you prefer in each of these sets, we will be able to identify the outcomes you think are most important, and the land use options that best suit people like yourself (Mallawaarachchi et al 1999).

Examples • We are interested to find out your views on some possible management options for the Macquarie Marshes. To do this, we have prepared eight sets of hypothetical options. From each set of options we would like you to choose the option you prefer…These options have been designed to cover a broad range of possible wetland management options. By indicating the option you prefer in each of these sets, we will be able to identify the outcomes that you think are most important, and the options that best suit people like yourself (Morrison 2000).

What are the likely effects of these alternative approaches? • If the admissions made are congruent with respondents’ beliefs about the information provided, respondents will be more likely to take the survey seriously, thereby reducing bias and variance • May increase strategic behaviour

Kelman (1967) Deception may cause the subject to dismiss the stated purpose of the study and search for other interpretations. May cause the subject to receive contradictory messages – if a contradiction between the researcher’s statements about the purposes of the study and the information the subject receives from the experimental conditions

Bailey (1978) • May cause different subjects to have their own definition of the meaning of the questionnaire and thus adhere to different response strategies. • ‘This is somewhat analogous to survey respondents having various understandings of an ambiguous question, so that in a real sense different respondents are answering different questions while all are ostensibly answering the same question” (pp. 390 -391).

Case Study • Land use change in Sunshine Coast area of Queensland • Sunshine Coast has a very high rate of urban development – Threatening the viability of the Sugar Cane Industry – Affecting areas of natural vegetation – Three competing priorities • Except for natural vegetation, no strong a priori view about whether willingness to pay would be positive or negative

Survey logistics • Questionnaire was sent to 825 respondents on the Sunshine Coast • Mail survey was used – initial questionnaire plus two reminders • 40. 7% response rate was achieved (312 valid responses)

Sample characteristics

Questionnaire description • 20 page booklet • Accompanying the questionnaire was a brochure that provided detailed information about the three main land uses in the Sunshine Coast region. • After several introductory questions respondents were asked four contingent ratings questions. • The first alternative that each person rated was the status quo, then they rated three alternatives that were different to the status quo.

Questionnaire description • Ratings scale in the form of a Likert scale. • Labels and small number of points on the ratings scale likely to reduce differences in the way respondents use ratings scales • Use of support/oppose labels makes use of the ratings scale much more like choice

Questionnaire description • First respondents answered four ratings questions

Questionnaire description • Next respondents answered six trinary choice questions

Two Treatments (1) We would now like to find out your views about some options for the use of land on the Sunshine Coast. To do this, we have prepared a number of hypothetical options for you to evaluate. By evaluating these options, we will be able to identify the outcomes you think are most important, and the options that best suit people like yourself. You will need to read the enclosed brochure before evaluating these options. The brochure describes the features of the options we would like you to consider in this section. The options have been specifically designed so that you have a broad range of options to consider.

Two Treatments (2) We would now like to find out your views about some options for the use of land on the Sunshine Coast. To do this, we have prepared a number of options for you to consider. You will need to read the enclosed brochure before evaluating these options. The brochure describes the features of the options we would like you to consider in this section. The options have been specifically designed so that you have a broad range of options to consider. Some options may seem a little odd, but bear in mind that there a number of ways of determining land uses.

Results

Comparison of ratings models (full ratings model, ordered logit) Treatment 1 (non-hypo) Treatment 2 (hypo) Urban -0. 05 (-2. 63)*** -0. 08 (-3. 90)*** Sugar 0. 53 e-2 (0. 43) 0. 31 e-1 (2. 32)** Vegetation 0. 46 e-1 (2. 17)** 0. 49 e-1 (2. 28)** Rates -0. 45 e-3 (-0. 45) -0. 47 e-2 (-4. 62)** rho 2 0. 011 0. 046 Note: t-stats in brackets, *** sig at 1%, ** sig at 5%, LR Test: X 2(8)=18. 17, p=0. 02

Comparison of ratings models (ratings recoded to choices) Treatment 1 (non-hypo) Treatment 2 (hypo) Urban -0. 80 e-2 (-0. 38) -0. 69 e-1 (-3. 12)*** Sugar 0. 39 e-2 (0. 31) 0. 12 e-1 (0. 93) Vegetation 0. 32 e-1 (1. 61) 0. 77 e-1 (4. 09)*** Rates -0. 19 e-2 (-1. 89)* -0. 69 e-2 (-6. 25)*** rho 2 0. 004 0. 049 Note: t-stats in brackets, *** sig at 1%, ** sig at 5% %, LR Test: X 2(5)=15. 62, p=0. 008

Comparison of choice models Treatment 1 (non-hypo) Treatment 2 (hypo) -0. 79 e-1 (-0. 69) -0. 97 e-1 (-0. 92) Urban -0. 61 e-1 (-5. 90)*** -0. 60 e-1 (-5. 70)*** Sugar 0. 47 e-2 (0. 81) 0. 14 e-2 (0. 37) 0. 77 e-1 (6. 79)*** 0. 74 e-1 (6. 87)*** Rates -0. 60 e-2 (-11. 44)*** -0. 47 e-2 (-8. 79)*** rho 2 0. 066 0. 048 ASC Vegetation Note: t-stats in brackets, *** sig at 1%, ** sig at 5% %, LR Test: X 2(5)=3. 56, p=0. 61

LR Tests for Ratings vs Choice Models X 2 Value P value Ratings Models vs Choice Models Non-hypo 24. 89 0. 0001 Hypo 3. 51 0. 4760 Ratings (recoded as choices) vs Choice Models Non-hypo 17. 30 0. 0017 Hypo 2. 29 0. 6828

Implicit Prices 35 Veg 30 25 20 Veg 15 10 5 0 Ratings_NH Ratings_H RR_NH RR_H Choice_NH Choice_H

Implicit Prices 0 Ratings_NH Ratings_H Urban RR_NH RR_H Choice_NH Choice_H -5 -10 -15 Urban -20 -25 -30 -35

Final Observations • Admitting alternatives were hypothetical and explaining why they were asked was more meaningful and less objectionable to respondents • Lead to (1) robust ratings results, and (2) convergent validity between ratings and choice results • Little evidence of strategic behaviour • Choice results were equivalent regardless of treatment – Are choice models less affected by the explanation given for implausible/unusual alternatives? – Effect of choice tasks being asked after the ratings questions? • Equivalence of implicit prices for ratings and choice tasks in Treatment 2 promising for ratings