Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES

Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities. Use qualitative information to integrate measures of targeting accuracy. Screen 1 of 27

Targeting Monitoring and Evaluation of Targeting Screen 2 of 27 INTRODUCTION Which types of information and analysis are useful in monitoring and evaluating how well targeting works in practice? Good feedback information can be very important in fine-tuning or correcting the targeting during an ongoing operation. Food security information systems can make an important contribution.

Targeting Monitoring and Evaluation of Targeting Screen 3 of 27 INTRODUCTION KEY QUESTIONS: • Has the targeting system identified the right people? • How many of the target group have been assisted? • Has the targeting system succeeded in concentrating resources on the intended beneficiaries? • Have any of the resources intended for the target group been diverted elsewhere?

Targeting Monitoring and Evaluation of Targeting Screen 4 of 27 CAUSES OF TARGETING “ERROR” In the real world, no targeting system is perfect. The imperfections are called targeting “errors”. Some potential causes of targeting error: Technical problems Governance problems • Inaccurate needs or vulnerability assessment • Malpractice, especially diversion or misuse of resources • Weak administrative capacity • Poor accountability • Poor analysis of power structures and interest groups • Clash of values • Cultural issues • Gaps in data or information • Poor design • Flawed implementation

Targeting Monitoring and Evaluation of Targeting THREE APPROACHES TO MEASURING ACCURACY Key questions: 1. How many people were correctly or incorrectly targeted? 2. How much aid reached the right people? 3. Who are the beneficiaries (and the non-beneficiaries)? Screen 5 of 27

Targeting Monitoring and Evaluation of Targeting Screen 6 of 27 COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? 1. How many people? The accuracy is often measured in terms of how many people are correctly or incorrectly receiving benefits. TOTAL POPULATION Target population EXCLUSION ERROR Beneficiaries (population receiving assistance) CORRECT TARGETING INCLUSION ERROR The diagram shows how the target group and the beneficiaries may not exactly overlap

Targeting Monitoring and Evaluation of Targeting Screen 7 of 27 COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Correct targeting is usually measured in terms of coverage. Coverage is the percentage of the target group who receive benefits. Coverage and exclusion error are complementary. Example – Coverage and exclusion error TARGET GROUP Exclusion Error Coverage

Targeting Monitoring and Evaluation of Targeting Screen 8 of 27 COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Perfect targeting Community of 100 people. 20 meet the targeting criteria. Perfect targeting Beneficiaries Non-beneficiaries

Targeting Monitoring and Evaluation of Targeting Screen 9 of 27 COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Perfect targeting total Beneficiaries total 100 Target group 20 Nontarget group 80 Non-beneficiaries 20 20 Coverage = 20/20 = 100% 0 Inclusion error = 0/20 =0% 80 0 Exclusion error = 0/20 =0 80

Targeting Monitoring and Evaluation of Targeting Screen 10 of 27 COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Imperfect targeting (with adequate resources) Beneficiaries Non-beneficiaries Total Beneficiaries Total 100 Target group 20 Non-target group 80 20 15 Coverage = 15/20 = 75% 5 Inclusion error = 5/20 = 25% Non-beneficiaries 80 5 Exclusion error = 5/20 = 25% 75

Targeting Monitoring and Evaluation of Targeting Screen 11 of 27 COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Imperfect targeting (with under-resources) Imperfect targeting B (with under-resourcing) Beneficiaries Non-beneficiaries Total Beneficiaries Total 100 Target group 30 Non-target group 70 Non-beneficiaries 20 80 19 11 Coverage =__% Exclusion error (or undercoverage) = __% 1 Inclusion error =__% 69

Targeting Monitoring and Evaluation of Targeting Screen 12 of 27 COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Imperfect targeting (with under-resources) Imperfect targeting B (with under-resourcing) Beneficiaries Non-beneficiaries Total Beneficiaries Total 100 Target group 30 Non-target group 70 Non-beneficiaries 20 19 19 /30 Coverage = 63 % 1 1 /20 Inclusion error = 5 % 80 11 11 / 30 Exclusion error (or undercoverage) = 37 % 69

Targeting Monitoring and Evaluation of Targeting Screen 13 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? 2. How much aid? Another approach to measuring the success of targeting is by assessing the quantity or proportion of aid actually received by the target group.

Targeting Monitoring and Evaluation of Targeting Screen 14 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Example – Imperfect targeting (with adequate resources) Beneficiaries (20) Target group (15) Beneficiaries Non-target group (5) (25% inclusion error) Target group Non-target group

Targeting Monitoring and Evaluation of Targeting Screen 15 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Targeting Effectiveness = the percentage of transfer resources actually delivered to the intended target group. Target group Non-target group Effectiveness Leakage targeting effectiveness = 75% Effectiveness and leakage are complementary.

Targeting Monitoring and Evaluation of Targeting Screen 16 of 27 EFFECTIVENESS, IDENTIFYING TARGET LEAKAGE GROUPS AND DILUTION – METHODS – HOW OF TARGETING MUCH AID? Example – Imperfect targeting (with adequate resources) - 2 The 20 sacks have been distributed as follows: • 18 to the correctly targeted households; and • only 2 to the non-target group. Target group Non-target group Effectiveness = ? Leakage = ?

Targeting Monitoring and Evaluation of Targeting Screen 17 of 27 EFFECTIVENESS, IDENTIFYING TARGET LEAKAGE GROUPS AND DILUTION – METHODS – HOW OF TARGETING MUCH AID? Example – Imperfect targeting (with adequate resources) - 2 The 20 sacks have been distributed as follows: • 18 to the correctly targeted households; and • only 2 to the non-target group. Target group Non-target group Effectiveness = 18/20 Leakage = 2/20 (or 100 -90) = 90% = 10% …of course, the opposite can also happen. . .

Targeting Monitoring and Evaluation of Targeting Screen 18 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Example – Imperfect targeting (with adequate resources) - 3 Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for. Target group Effectiveness = ? Non-target group ? ? Missing resources Leakage = ?

Targeting Monitoring and Evaluation of Targeting Screen 19 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Example – Imperfect targeting (with adequate resources) - 3 Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for. Target group Effectiveness = 12/20 = 60% Non-target group ? ? Missing resources Leakage = 100 -60 = 40%

Targeting Monitoring and Evaluation of Targeting Screen 20 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Targeting effectiveness can be compared with the results of an untargeted distribution. TOTAL POPULATION TARGET GROUP The accuracy of the targeting can be measured in terms of how much more the target group receive from the targeted programme. Targeting Performance = % of resources received by the target group / % of target group in population.

Targeting Monitoring and Evaluation of Targeting Screen 21 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? PERFORMANCE > 1 Targeting has improved the outcome for the target group. PERFORMANCE = 1 The targeting has had no effect. PERFORMANCE < 1 It is worse than no targeting.

Targeting Monitoring and Evaluation of Targeting Screen 22 of 27 EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Dilution = the amount of aid received by each targeted beneficiary, as a percentage of the planned amount. Dilution is the opposite of concentration (the aim of targeting). It is a common targeting problem in which aid is shared among a larger number of people than the programme was designed for.

Targeting Monitoring and Evaluation of Targeting Screen 23 of 27 BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT? 3. Who are the beneficiaries? The third group of targeting indicators falls under the question of who is included and who is excluded by the targeting system. It is very useful to compare the characteristics of the beneficiary and non-beneficiary populations. If there is a clear difference between them it means that the targeting has been effective.

Targeting Monitoring and Evaluation of Targeting Screen 24 of 27 BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT? It is also useful to look at the profiles of people in the exclusion and inclusion errors. Exclusion Do the excluded target group members have any factor in common that might suggest systematic marginalization? Inclusion Who are the people wrongly included among the beneficiaries? Why and how the error is happening?

Targeting Monitoring and Evaluation of Targeting Screen 25 of 27 THE BOTTOM LINE Standard of accuracy and targeting priorities are different in different contexts. Targeting is more accurate in countries with higher incomes, good administrative capacity, and high levels of inequality. In emergency operations, the priority is often to maximize coverage , while in other situations, the priority may be to eliminate inclusion error and leakage.

Targeting Monitoring and Evaluation of Targeting Screen 26 of 27 THE BOTTOM LINE Contextual information and qualitative assessment of how the targeting system is working help to interpret the error measurements in a meaningful way. • Why are problems or errors happening? What can be changed in order to reduce them? • What do beneficiaries and their community think about the targeting? • How are targeting decisions actually made, and by whom? • How transparent and accountable is the process? • Does the targeting match the programme’s objectives? • Could the targeting design have been better?

Targeting Monitoring and Evaluation of Targeting Screen 27 of 27 SUMMARY No targeting system is perfect. Usually some degree of inaccuracy must be accepted to make the programme workable and affordable. A judgement has to be made about which kinds of error, and what degree of error, are acceptable given the programme’s objectives and context. The most useful indicators for measuring the accuracy of targeting fall under 3 key questions: 1. How many people were correctly or incorrectly identified as beneficiaries? • Coverage = % of target group receiving benefits • Exclusion = % of target group not receiving benefits • Inclusion = % of beneficiaries who are not target group members 2. How much of the resources for distribution reached the right people? • Effectiveness = % of transfer resources received by the target group • Leakage = % of transfer resources not received by the target group • Performance = % of transfer resources received by the target group, divided by the % they would have received from an untargeted distribution 3. Who is included, who is excluded?