First Regional Training Assessing Costs and Benefits of

First Regional Training Assessing Costs and Benefits of Adaptation: Methods and Data SAMPLING AND NON-SAMPLING ERRORS ISSUES TO CONSIDER BISHWA NATH TIWARI UNDP-APRC Email: bishwa. tiwari@gmail. com BANGKOK 1 14 MARCH 2013

CONTENT OF PRESENTATION Household Survey design and errors • Steps for conducting survey • Errors Sampling and non-sampling errors 2 Suggestion to minimize those errors – Issues to consider

NEED FOR DATA • Data is critical for effective planning, design & implementation of adaptation projects • BUT there is lack of disaggregated data by: • Region, districts, settlements (informal), villages • Sex and age groups • Caste & ethnic groups • Economic groups • Vulnerable/disadvantaged groups • Lack of updated data, and regular availability of data 3 • Lack of quality data (with less errors)

SOURCES OF DATA Sources of Data Secondary National international Primary Household survey Institutional survey Some international sources of data: Internal Energy Agency, Global Footprint Network, Centre for International Earth Science Information Network, Centre for Research on the Epidemiology of Disasters, World Bank World Development Indicators 4 Generally, secondary data lack disaggregation CC is global externality but its impacts are local, therefore need for hh level data

STEPS FOR CONDUCTING A HOUSEHOLD SURVEY 1. Define universe –households where the adaptation project is to be implemented 2. Determine sample size 3. Decide sampling technique 4. Prepare sampling frame and sample households 5. Develop questionnaire taking into account the variables affecting the adaptation capacity and pretest the questionnaire 6. Organize field survey – selection and training of enumerators, organization in groups with division of responsibility including supervisory responsibility 5 7. Check data in the field & minimize non-response

EXAMPLE OF HH SURVEY: DUMANGAS FARMERS ARE WILLING TO PAY FOR ADAPTATION PROGRAM q Gay (2005) conducted a survey in coastal town of Dumangas in the Philippines to know if farmers are willing to pay to reduce their vulnerability, and if so, how much do they value a planned adaptation program to climate change. q Using CVM, a WTP survey was conducted among 450 hhs; 391 (87%) were willing to pay for planned adaptation program, 59 were not. q The mean and median WTP for planned adaptation program were PHP 34. 37 and PHP 23. 96 per month, respectively. q Factors influencing farmers’ WTP for planned adaptation program were education, farm experience, farm size, knowledge about climate change, land tenure, access to credit, and household income. q Relocation for people affected by sea level rise was the least preferred option Source: Defiesta, Gay (2010). Social Vulnerability and Willingness to Pay for Adaptation to Climate Change of Farmers in 6 q Provision of alternative livelihoods and training that are less affected by climate change were the most important planned adaptation projects for farmers

EXAMPLE OF HH SURVEY: SOUTH AMERICAN FARMERS ADAPT TO CLIMATE CHANGE BY CHANGING CROPS Estimating a logit model across 949 farmers in 7 countries, Seo and Mendelsohn (2008) found that both temperature and precipitation affect the crops that South American farmers choose. Farmers choose fruits and vegetables in warmer locations and wheat and potatoes in cooler locations. …. Global warming will cause South American farmers to switch away from maize, wheat, and potatoes towards squash, fruits and vegetables. Predictions of the impact of climate change on net revenue reflect not only changes in yields per crop but also crop switching. The paper use data collected in 7 South American Countries. The Household surveys asked detailed questions on farming activities during the one year period of July 2003 to June 2004. 7 Seo, S. Niggol and Robert Mendelsohn (2006). An analysis of crop choice: Adapting to climate change in South American farms. E C O L O G I C A L E C O N O M I C S 6 7 ( 2 0 0 8 ) 1 0 9 – 1 1 6

ERROR OR BIAS Error due to sampling design and sampling technique Nonsampling error All other error over and above the sampling error Total error 8 Sampling error

HOW TO MINIMIZE SAMPLING ERRORS – POINTS TO CONSIDER q Suggestions to minimize sampling errors • Use of appropriate probability sampling technique • Minimize stages of sampling – design effect • Determine sample size taking into account the variance of the main indicators/attributes of the universe • Prepare updated sampling frame so as to minimize the non-response 10 q Sampling error depends on: • Sample size • Sampling technique • Heterogeneity of universe

SOURCES OF NON-SAMPLING ERROR Nonsampling error Respondent enumerator processor • Strategic bias • Under/over- reporting • Non-response • Lack of training • Poor coding and editing of questionnaire • Mistake in data entry • Programming errors 11 Instrument • Definition/specification • Defective questionnaire • Defective Measurement tools

SOURCES OF NON- SAMPLING ERRORS – EXAMPLES Strategic Bias: While piloting questionnaire of NMIS cycle 3, it was found that some respondents were reporting false info about sources of drinking water with a strategy that they will get tube well. Error from enumerator due to lack of training: (i) Use of lead question; (ii) Lack of probing 12 Specification error: Unless all agricultural crops are included in a survey it is difficult to enumerate income of farmers from agriculture

HOW NON-SAMPLING ERRORS CAN OCCUREXAMPLES Because irrigated farms are less sensitive to climate, where water is available, irrigation is a practical adaptation to climate change in Africa. A specification error of dry land or irrigated crops can change the result of the study! Strategic bias (eg, reporting less revenue) from the farmers with irrigated farms can change the conclusion of the study! Source: Kurukulasuriya et. al (2006). Will African Agriculture Survive Climate Change? The World Bank Economic Review. 13 Using data from a survey of more than 9, 000 farmers across 11 African countries, Kurukulasuriya et. al (2006) estimates how farm net revenues are affected by climate change compared with current mean temperature. Revenues fall with warming for dry land crops and livestock, whereas revenues rise for irrigated crops. At first, warming has little net aggregate effect as the gains for irrigated crops offset the losses for dry land crops and livestock. …. The final effects depend on changes in precipitation, because revenues from all farm types increase with precipitation.

HOW TO MINIMIZE NON-SAMPLING ERRORS – POINTS TO CONSIDER Use of structured questionnaire and FGDs – skip pattern, codes Use of survey manual with standard definition and specification Pre-test of questionnaire Use of mix of instruments – structured and unstructured questionnaire/FGD q A thorough training of enumerators with background knowledge on the subject q Good rapport building with respondents q No lead question; no guess q Use of probing questions – who, what, where, why and how q Request for time if the questionnaire is not completed in one sitting q Interview place – calm and quiet environment q Improved coding of questionnaire q Programming for data entry and double data entry to minimize possible entry errors q Data cleaning and editing 14 q q

15 THANK YOU