Lesson 7 Performance Macerata 28 th October Andrea

Lesson 7: Performance Macerata, 28 th October Andrea Gramillano, t 33 srl

Verify the consistency of the analysis of needs • Obstacles (needs) • Potential for solutions S T T O Ostacoli – colli di bottiglia

Agenda v Definition and meanings v Indicators v Multicriteria

Performance: problematic defintion. OECD definition: The degree to which a development intervention or a development partner operates according to specific criteria/ standards/ guidelines or achieves results in accordance with stated goals or plans. Commission Defintion: The meaning of the word performance is not yet stable; it is therefore preferable to define it whenever it is used. Performance might mean that intended results were obtained at a reasonable cost, and/or that the beneficiaries are satisfied with them. Efficiency and performance are two similar notions, but the latter extends, more broadly, to include qualitative dimensions.

Performance: When it does take place On going Evalaution Criteria the project activities are delivered on time, the outputs respect the targets, the resources are duly absorbed, the procedures are done according to the rules. Needs Resources (inputs) Output (implementation) Result (Specific Objective)

Performance questions: The project: - is able to spend all the financial resources; - meets the procedural deadlines; - achieved the target in terms of physical realization. And in the next period: - What is needed to increase the project performance? (human resources? political support? administrative enforcement? - Will the project be successfully completed? - Which might be future challenges Performance evaluation is based on monitoring indicators (procedural, financial, physical, output indicators)

Operational life of a Project Setting Test, End and final payment Implementation Tender or Purchaise procedures Start of Work / service

Procedural monitoring Most public activities have to follow a more or less rigid schedule in which the different steps are mandated and the deadlines fixed (i. e. ). Procedural monitoring usually provides information about how project pipelines are progressing (where and when calls for tenders have been published, contracts have been awarded, …). Final payment …… Contract awarded Call published Specifications ready …… …… Actual …… …… …… Expected Actual …… 19 -08 -10 Expected Actual 08 -08 -10 Expected 14 -08 -10 Actual Status Open Expected 08 -08 -10 Procedure P 1

Physical monitoring Example of physical monitoring: Indicator (number of enterprises) Unit of Measurement Target Achievement Micro N 142 91 Small N 133 111 Medium N 39 21 Owner (women) N 50 40 Owner (<30 y) N 26 3 Start-up N 54 3

Financial monitoring Example of financial monitoring: Priority P 1 Expected expenditures Resources committed Expenditures Amount % (a) (b/a) (c/a) 133. 4 100. 4 75. 2 71. 8 53. 9

Multicriteria Tool used to compare several interventions in relation to several criteria. Multicriteria analysis is used also in the ex ante evaluation for comparing proposals. It can also be used in the ex post evaluation of an intervention, to compare the relative success of the different components of the intervention. Finally, it can be used to compare separate but similar interventions, for classification purposes. Multicriteria analysis may involve weighting, reflecting the relative importance attributed to each of the criteria. It may result in the formulation of a single judgement or synthetic classification, or in different classifications reflecting the stakeholders' diverse points of view. In the latter case, it is called multicriteria-multijudgement analysis. (from EVALSED)

Process Step 1 define criteria Step 2 scoring or ranking Step 4 aggregating Step 3 weighting

STEP 1: setting criterion Criterion 1: Financial Performance Project 1 Project 2 Project 3 Project 4 Project n… Criterion 2: Procedural Criterion 3: Physical realisation

STEP 2: Score or rank for judgment - It is needed to find a way to appraise the project according to different aspects since we used different measuring units for different aspects of different process - We can opt for: - A) Scoring: by assigning a numeric value to different “interval” of performance. For example 3 for “above average” – 1 for “on line with average” – 3 for “below average” - B) Ranking: we simply order the different projects according to their performance from the first to the last

STEP 2: Scoring Criterion 1: Criterion 2: Criterion 3: Project 1 1 Project 2 0 2 1 Project 3 3 1 1 Project 4 3 3 0 Project n…

STEP 3: Establishment of weight If some Criteria is more important than others it shall be given more importance. To do it we simple apply a multiplication factor > 1 (e. g. 1, 5). Some criteria may have such importance that they have to be singled out. This is the case for criteria determined by a veto threshold (For example “Physical” if some project has 0 performance, it is excluded by the analysis).

STEP 3: Apply the weight Criterion 1: ( * 1, 5) Criterion 2: Criterion 3: Project 1 1, 5 1 1 Project 2 0 2 1 Project 3 4, 5 1 1 Project 4 4, 5 3 out Project n…

STEP 4: Aggregate the score Criterion 1: Criterion 2: Criterion ( * 1, 5) 3: Total (with weight) Total (without weight) Project 1 1(1, 5) 1 1 3. 5 3 Project 2 0 2 1 3 3 Project 3 3(4, 5) 1 1 6, 5 5 Project 4 3(4, 5) 3 out 6 Project n…

WORK OUT: SME INCUBATOR Criterion 1: Economic (average increase of turnover) Project 1 Project 2 Project 3 Project 4 Criterion 2: New Jobs Criterion 3: Satisfaction for quality service Total