Monitoring and Evaluation Experience of Centralised framework at

Monitoring and Evaluation Experience of Centralised framework at sector level Mr B Khama Ministry of Education, Namibia CABRI Conference 2006 28 -30 November 2006 ADDIS ABABA

Structure of presentation Introduction n Description of Namibia’s PEMP and its application in education n Analysis of role of PEMP in education n ¨ Design, process and implementation ¨ Development partners, sector strategic planning and PEMP n Way forward and lessons from Namibia

Introduction n Namibia case illustrates key issues regarding Monitoring and Evaluation frameworks between central oversight and control agencies and spending agencies n Where should objectives, actions, desired outputs and targets be set? ¨ ¨ n Important system objectives ¨ ¨ ¨ n Balance between central oversight and sector knowledge Desirability of policy coherence in government points to centralising efforts Need to keep frameworks simple and comprehensible to limit overload at centre Need to make framework comprehensive and useful for sector management Process Data availability and integrity Efficiency Multiple frameworks not desirable ¨ Donor requirements complicates

PEMP in Namibia n Centrally developed government-wide monitoring system Aim is to link government’s operations to to National Development Plans ¨ Sets out objectives and desired outcomes for ministries ¨ n Part of budget management system ¨ ¨ MTEF system: Macro-fiscal framework and Ministerial MTPs Critical analysis of past performance and MTPs informs thinking on resource allocation n n ¨ ¨ Queries on PEMP data integral part of Budget Hearings and assessment Forward analysis look at stretch/achievability of future targets; potentially redundant and overlapping spending; realism. In-year monitoring Future reforms include accountability reports, quarterly inyear monitoring, top level review and expenditure audits

Education sector and M&E n Sector overall in period of transition and significant reform Drive to reform sector policies and management vs legacy thinking – “line items”, “needs”, “funding gap” ‘tyranny of targets” Producer capture and unwillingness to be accountable for results ¨ Hierarchy and decision making within Education ¨ Effects of decentralization ¨ Initiative fatigue ¨ n Issues affecting M&E ¨ ¨ ¨ EMIS late Disjunct financial year/academic year PEMP: Age + turnover = not ours

PEMP in Education: substance n 8 Objectives ¨ ¨ ¨ ¨ ¨ n All children receive quality education All Namibians are functionally literate Enhance learners’ potential for beneficial participation in the economy and in society Increased employment A workforce with the competencies to enable the labour market to operate efficiently Well-educated citizens Increased level of value-added research Advancement of scientific knowledge Plus indicators for administration Total of 96 indicators…just for one ministry

Examples of indicators All Children receive quality education Problematic: Enrolment numbers, number of schools, quality of teaching Appropriate: enrolment rates, progression rates, survival rates, textbook ratio’s, teaching hours per learner, pass rates, drop out rates, PTRs All Namibians are functionally literate Problematic: Number of people passing literacy and numeracy tests Appropriate: Adult literacy rate, numeracy level, Enhance learners’ potential for beneficial participation in the economy and society Problematic: Employment rate for school leavers after X months; attendance levels; %lessons rated satisfactory by inspectors; Employment levels after graduation by subjects offered

Examples of indicators Increased employment Problematic: Relative employment levels of qualified and unqualified 6 months after completion; Unit cost of processing applications Appropriate: Youth unemployment (Mo. E target 2006/7, 80 000); Formal and informal employment levels (Mo. E target for 2006/7, 43 000) A work force with the competencies to enable the labour market to operate efficiently Problematic: Levels of skills shortage; 25% enrolling in VET; Levels of qualifications by field, length and year completed Well educated citizens Problematic: Unit cost of all qualifications; number of Namibians educated to first degree etc; internet users within the last month Appropriate: % of unemployed youth trained in SME promotion Appropriate: Attendance levels at tertiary institutions

Examples of indicators Increased level of value added research Leveraged funding ratio (GRN: Foreign); Total R&D spend in private sector; BUT ALSO: Number of research projects by sector Advancement of scientific knowledge Subject breakdown of citations (Mo. E target 2006/7 90 000) Unit cost of citations Administration Number of rapes by hostel BUT ALSO: Utilisation rate of school building; average of outstanding payments; unit cost of catering and laundry per boarder

Does system work for Ministry? n Some positives Existence of M&E framework used by both Mo. F and Mo. E positive ¨ Some objectives, indicators are SMART ¨ n Specific, measurable, attributable, relevant and time-bound Go. N framework linked to NDPs But other issues problematic for Ministry to use PEMP as core of performance framework Issues of design, process and implementation ¨ Multiplicity of stakeholders, processes and monitoring frameworks ¨ Ministry own processes under-developed ¨

Issues of design DESIGN OF OBJECTIVES Overlap between objectives; at same time not all Ministry functions captured; at same time relationship between objective and Mo. E not straightforward or singular ¨ Objectives and indicators not aligned clearly with key sector concerns, eg access, quality, equity, efficiency. ¨ DESIGN INDICATORS Too many indicators: overlap between indicators Relationship between objectives and indicators not always clear Not clear what direction indicator should move Where indicator come with set target, not clear on what basis target was chosen ¨ Data problematic and expensive to source for many indicators ¨ Indicators often use absolute numbers instead of relative numbers – indicator on own won’t indicate progress/regress ¨ Many indicators can be manipulated ¨ ¨

Process and implementation issues DEVELOPMENT PROCESS n n Framework developed centrally without sufficient consultation with MOE Efforts to adjust framework to be more in line with Mo. E strategic plans met with resistance ¨ Although annual request for adjustments IMPLEMENTATION PROCESS n While on paper framework given prominence in budget process, in practice role not as prominent Limited analytical capacity in Mo. F to do justice to information and perception of window dressing ¨ System still bedding-in, yet to see full benefits. Targets and IFMS only from March 06 ¨ n n Can Ministry’s budget be allocated against indicators outside of its control? Framework not suitable for use in ministry to manage performance of subsectors: not all sub-sectors included Data not available for many of indicators Multiplicity of stakeholders

Multiplicity of stakeholders n n DP attempts at imposition/override Creation of separate processes and frameworks ¨ n n n JAR, ESPAG, ISCBF SC, ETSIP Pareto 80: 20 rule and DPs ETSIP monitoring framework some sector ownership Education and Training Sector Improvement Plan ¨ ¨ ¨ Developed with intense consultations within the sector and with stakeholders over 2 year period Based on thorough research on key challenges for sector Comprehensive, focused, specific, resource-bound Integrates operational and development spending against available internal and external resources Approved by Cabinet and tabled in Parliament Plan has Monitoring and Evaluation framework

ETSIP M&E framework n Main objective clear and attributable ¨ n Indicators against main objective: ¨ n To increase the immediate supply of middle-high level skilled labour to meet labour market demands and support overall national development goals Objective and indicators SMART: Increased net enrolment ratios in Senior Secondary ; Increased proportion of students entering Grades 5, 8 and 11; Significantly reduced drop-out rates; National average SACMEQ scores improved; Employment rates of VET students increased; Survey of employers verifies improved skill levels of graduates; Improved throughput to tertiary education with increasing emphasis on priority subjects Clear, non-overlapping sub themes with detailed SMART indicators 8 programmatic sub-themes with sub-objectives aligned to each Limited number of indicators with how indicators will be used specified (process, output and outcome indicators) ¨ Key performance issues of access, quality, equity, efficiency and HIV/AIDS addressed at objective level ¨ ¨

Role of ETSIP framework n n ETSIP framework driving management in sector ETSIP framework used between Ministry and Development Partners and other funders ¨ Negotiated framework with high Mo. E ownership ¨ Relevant, specific, attributable objectives and indicators ¨ Indicators aligned with existing data, but limited additional data collection being set up ¨ M&E framework will be used to manage donor funds

Issues and way forward n Key issues for Mo. E: ¨ MOE indicators to be agreed upon with MOE; ¨ ETSIP framework linked to key additional resources, captures all Ministry activities at high level of output and outcome, linked to approved strategic plan ¨ Urgent to reconcile two frameworks to make M&E activities streamlined, comprehensive, strategic, effective and efficient n New NDP development ¨ Harmonize ETSIP and NDP 3 ¨ Agree on PEMP indicators; ¨ Resolving of these issues urgent

Conclusion n Lessons from a Namibian line ministry Mutually agreed performance frameworks very important for principal agent contract between Mo. F and Mo. E ¨ There is a need to design a M + E framework ¨ n ¨ Criteria for design performance is whether it can be used by ministry itself n ¨ Specific, measurable, attributable, relevant and time-bound If that possible ties policy, budgeting and implementation together Line-ministries understand sector policy issues and performance measurement better n Superior knowledge should be used in design process But participation (effectively oversight) by centre important to ensure integrity of framework (stretch and achievement) ¨ Frameworks and their functionality are symptomatic of institutional setup of policy making and financing ¨ Single framework needs to be agreed between centre, line ministry and external funders ¨ n First requirement for integration of activities in sector towards single set of strategic goals