Session 12 Indicators III Quantitativequalitative indicators critical analysis

Session 12 Indicators (III) Quantitative/qualitative indicators & critical analysis of data 1

Quantitative and qualitative indicators

Quantitative & qualitative indicators § number of qualified and experienced teachers § assessment by a jury during a competition (e. g. score given) § cost per pupil § § number of qualified and experienced teachers per 1, 000 children of primaryschool age. ranking by people based on their opinions, perceptions etc. § parents' opinion on the possibility of their children attending a primary school class with a qualified and experienced teacher. QUANTITATIVE INDICATORS Fact-based information using a counting process (elementary indicator) or can be computed using elementary indicators (ratios, rates, i. e. derived indicators) QUALITATIVE INDICATORS Information produced through a qualitative method (opinion based). It is not based on a measure using a standardized 3 instrument.

Qualitative indicators are useful § Qualitative indicators are as useful as quantitative ones § Reflecting the opinion of stakeholders provides key info on whether the change process is taking place (or not) § Diverse and disaggregated qualitative information helps highlight the diversity of perceptions about an action § Allow cross-checking with the opinions/perceptions of different stakeholders § Can facilitate a participatory approach to measurement

Exercise| Balancing quantitative and qualitative indicators Using examples and in discussion with 1 or 2 colleagues, think about how we could: § Make a quantitative indicator more qualitative? § Make a qualitative indicator more quantitative? Activity– small group discussion followed by a plenary discussion 5

Comparison methods & critical analysis of data

With indicators we can compare… § § § The baseline situation vs. actual situation Actual vs. planned Trends over time and comparisons between periods Geographical variability Variability between groups Standards and benchmarks For more info on comparability see chapter 2. 4 of the guide on « Performance measurement… » 7

Activity | Indicators and comparison Three main methods of comparison to assess the added value of an action § Before / after (6 months, 1 year of) the action based on observation of the actual situation and the initial situation § With / without the action based on a scenario/model § With / without the action based on observation of a control group and the target group Each method has its advantages and disadvantages. Take a real example to identify the respective pros & cons. Summarize in a table (+/-). Activity – 3 groups 8

Interpreting data Indicators and resulting data/information help tell us what is happening, but not necessarily why A project may not be achieving objectives and targets for a variety of reasons, including: § Weak understanding of the pathway of change § Unrealistic initial objectives/targets § Constraints outside of the project’s control; or § Poor project management he r t ok fo Lo … s r use » ! Data on why s « rea Reasons need to be understood if appropriate corrective action is to be taken 9

Data analysis Figu re dime s have a n illusi sion wh mathem ic o a facts n that th h create tical … K s e eep y are sc the think i ing ! entific § Data needs to be analysed to turn it into useful management information § Some actions generate data that lends itself to some quantitative analysis – others do not § Data used for monitoring/assessing performance should always be treated with caution and analysed before drawing conclusions § Operational Managers can Ø provide ideas/advice to implementing partners to improve their analysis Ø contribute their own opinions on what data may be indicating about project performance 10