Measuring socioeconomic impact of postdisaster shelter Developing a

Measuring socio-economic impact of post-disaster shelter Developing a standardized methodology Shelter Meeting 09 a, Geneva May 2009 Simone van Dijk Alexander van Leersum

Intro Measuring the impact of post-disaster shelter: short- or long-term? Status Quo: ◦ Long term (i. e. impact) vs short-term evaluation Experience ◦ Vietnam ◦ Indonesia of case studies Ø Development of standardized methodology

Objectives of the presentation Long term effects of housing 2) Aims of a standardized methodology 3) A proposed methodology 1) ◦ ◦ Measuring socio-economic dimensions 4) ◦ ◦ ◦ 5) Before-after With-without Direct measurement Indirect measurement (e. g. Wealth-index; PCA) Self-assessment Cross-programme comparison

1) Long term effects of housing What kind of a relation can be distinguished? Intertwined elements, problems with causalities and counterfactual analysis

2) Aims of a standardized methodology (1/2) What is the added value? Examples values of expected gained insights/added ◦ Proportion of income spent on housing adaptations ◦ How long does a HH uses a house, how do they use it? ◦ Is the house cultural appropriate?

2) Aims of a standardized methodology (2/2) What do we want to achieve? Developing a methodology for a long-term impact study that allows for comprehensive analysis of a household’s socio-economic situation. Developing a general approach and methodology for longterm socio-economic impact studies irrespective of location. Developing a general methodology that is suitable for the evaluation of different types of post-disaster housing programs.

3) A proposed methodology (1/4) How to tackle causality questions? With-without (intervention ◦ Selection of groups Before-after (impact ◦ Timing; project cycle vs control) of intervention)

3) A proposed methodology (2/4) Before and after: The importance of timing With-without & Before-after effect research Living standards Beneficiaries Impact of intervention Control group (i. e. nonbeneficiaries) Comparability of groups 2000 2007 Time

3) A proposed methodology (3/4) With-without: selection of intervention- & control group(s) Control group, intervention group(s) Socio-economic position should be comparable between groups with respect to: Ø Ø Ø Average of the household Labor force Average size of the household Predominant sources of income Educational level

3) A proposed methodology (4/4) Different (housing) interventions, e. g. in Ache With-without & Before-after effect research T. S. + P. H. Living standards Permanent Housing (P. H. ) ! f n? o o ct nti pa rve Im te No-support in Only T. S. Transitional Shelter (TS) Dec. 2004 2008 Time

4) Measuring socio-economic dimensions What kind of data is required and feasible? Type of measurement Discipline Data processing Type of data Direct measurement • Income & consumption • Access to social services - - Indirect measurement • Housing characteristics • Durable assets • In-house services Wealth index (Factor analysis) Objective Self-assessment Housing Economic Living Standards Social standards 5 -point scale Subjective

4) Measuring socio-economic dimensions. Do you have days of food deficits per week? No [ ]1 Sometimes [ ]2 What kind of data is required and feasible? Yes [ ]3 Discipline Data Type of processing datauses? What type of latrine does your household Direct measurement Flush • Income & (setting) - [ ]1 toilet consumption Toilet (standing) [ ]2 • Access to social services Other, unhygienic latrine [ ]3 Type of measurement Indirect measurement Self-assessment • Housing Wealth index Objective No latrine [ ]4 characteristics (Factor • Durable analysis) The qualityassets of your present house is. . . • In-house services Very bad [ ]1 Housing 5 -point scale Subjective Bad [ ]2 Economic Living Standards Ok [ ]3 Social standards Very good [ ]4 Good [ ]5

5) Cross-programme comparison (1/3) (How) can shelter programmes be compared among each other? Examples of questions which can make comparison possible Ø Only relative comparison is possible; comparison of different socio-economic status of groups E. g. Proportion of income spent on housing adaption/extension/repair Standardization of methodology Partly standardization of questions

5) Cross-programme comparison (2/3) Why a standardized methodology? Strength of method lies in the combination of ◦ Focus: socio-economic dimensions related to housing ◦ Approach: intervention + control group(s) ◦ Timing: different moments in time, to measure impact ◦ Data collection: objective and subjective ◦ Data collection: quantitative and qualitative

5) Cross-programme comparison (3/3) What are the underlying aims of a standardized methodology? What can we expect from insights of impact studies? Support from donors 2) Improvement of the program design 3) Improvement of the (shelter)design 1)

Thank you

6) Extra Measurement of impact further explained Measuring mean-difference between groups H 0 =µintervention ˃ µcontrol Statistically significant different? (e. g. T-test, ANOVA, Cross-tabs) Control Group Intervention Group 5% overlap (1 st order error, Alpha) 95% Confidence Interval Answer scale’s

6) Extra PCA further explained Data type Housing Actual variables to construct 'Wealth Index', based on Principle Component analysis floor type Analyzed variables rooms/person, type of wall, floor, roof Consumer Durables radio, refrigerator, bicycle, TV, radio, bicycle, TV, electric fan, motorbike, car, ploughing rice cooker machine/tractor, other agricultural equipment, mobile phone, fixed telephone, sewing machine, electric fan, rice cooker Services water, sanitation, electricity, fuel electricity, sanitation for cooking Wealth index values ranging between 0 to 5 (most poor towards least poor)