SABER Systems Approach for Better Education Results Tertiary

SABER – Systems Approach for Better Education Results Tertiary Education Governance World Congress of Colleges and Polytechnics Halifax, May 26, 2012 Quentin Wodon, World Bank 1

Structure of presentation �SABER goals, scope, and methodology �Context for tertiary education governance �System-wide and institution-level indicators �Example of data collection for institutions: MENA �Analysis of MENA data on autonomy �Financial autonomy �HR autonomy �Academic autonomy �Conclusion 2

What SABER is �New initiative to advance Learning for All �First detailed, disaggregated database of education policies/institutions in core areas �Open data tool for empowering stakeholders �Two key areas: �Maps out policies/institutions �Links to implementation data 3

Goal: Make visible what’s underwater Inputs and (some) outcomes Everything else: • Policies • Institutions • Implementation • Effects of interventions 4

Domain development: Key steps �What Matters paper �Indicators and scoring rubric �Data-collection instrument �Data collection �Analysis �Data validation �Publication of data & analyses 5

What SABER provides (not just ratings) �Analytical framework for thinking about domain �Descriptive data on policies/institutions �Evaluative judgments (ratings, not rankings) �“Latent” (poor performance) �“Emerging” (insufficient performance) �“Established” (adequate performance) �“Advanced” (outstanding performance) 6

SABER informs policy choices & diagnoses gaps in implementation in each domain SABER Toolkit POLICIES + IMPLEMENTATION Catalog & assess quality of policy framework Collect & analyze data on policy execution Tools to benchmark policies Tools to assess implementation based on available data • Survey data (e. g. PETS, • Framework • Collection instrument • Rubric • Manual OUTCOMES QSDS, Absence, Household) • Proxy indicators (e. g. economy-wide metrics, survey data from other countries) Country, regional, and policy domain reports with interpretation, including expert judgment Online knowledge base 7

Tertiary education governance �Large variation in performance of higher education institutions – governance as a key determinant �Differences in objectives: research, teaching, contribution to local economy, etc. Not one “best” model �Benchmarking as step towards monitoring performance �Three levels of analysis �Institutions level: Identification of Strengths and weaknesses, base line for comparison overtime �Country level: Identification of models, differences between institutions, correlation between models and performance �International level: Comparisons between models, correlations between models and performance 8

System Level Context, Mission and Goals Management Autonomy Accountability Participation Government- Governmentdriven defined missions and policies Governmentappointed president Centrally managed budget Central control for new programs and curriculum Central HR management Central audits Central QA National driven curriculum Low accountability-no links between performance and rewards Mainly on consultation basis Autonomous- Mission-oriented Government- Institutions steered Strategic plans prepared by Institutions Governing boards led Competitive funds allocation Autonomy to introduce new programs and set curriculum HR autonomy External audits Independent external QA Performance-based salaries High participation of stakeholders throughout the decisionmaking process 9

Context, Mission and Goals Corporate Mission-oriented Decentralized Academic Mission-oriented. Defined in consultation with academic staff Management Orientation Results-based Trustee Results-based Accountability Participation High autonomy High in all three accountability in areas, academic, financial and HR High academic autonomy Representational Mission-oriented. Defined in consultation with trustee Autonomy High internal academic accountability High participation of academic staff High external accountability High participation of stakeholders High internal accountability 10

System-wide Policy Goals �Goal #1: Vision The country or government has a vision and plan for the tertiary education sector, a willingness to translate its vision into a concrete action plan, and an ability to implement and monitor reforms �Goal #2: Regulatory Framework The tertiary education system is governed by an appropriate regulatory framework including for private providers �Goal #3: Leadership The TEA has an appropriate policy on the role and functions of the boards of tertiary education institutions, as well as for the selection of the leadership of tertiary education institutions, and the respective responsibilities of the Board and leadership �Goal #4: Financial Autonomy and Equity The regulatory framework provides enough financial autonomy to tertiary education institutions while still promoting equity 11

System-wide Policy Goals �Goal #5: Staffing Autonomy The regulatory framework provides enough staffing autonomy to tertiary education institutions �Goal #6: Academic Autonomy The regulatory framework provides enough academic autonomy to tertiary education institutions �Goal #7: Performance-based Funding The TEA negotiates performance targets and uses financing as incentives for institutions to achieve the targets. � Goals #8: Quality assurance and transparency The TEA has an independent quality assurance and accreditation agency for both public and private institutions. Institutions are held to specific standards of transparency around financial health, fraud, student engagement and employment of graduates. 12

A Note on QA and quasi-corruption Question Share of household with positive value for annual official school cost Share of household with positive value for annual unofficial school cost Annual unofficial school cost Did you or anyone in your family make unofficial payments to get admission? Yes No It is common for parents to make some “unofficial payments” to gain admission? Never Seldom Sometimes Frequently Always When unofficial payments are required, how is it done? A school official indicates or asks for a payment The parent or family member offer a payment on his/her own accord It is known before hand how to pay and how much to pay, so it is not discussed Respons e 95. 7% 44. 1% 1. 2 M 0. 4 M 27. 2% 72. 8% 1. 1% 2. 2% 43. 0% 47. 3% 6. 5% 30. 3% 37. 1% 32. 6% 13

Institution-level: MENA University Governance Screening Card �Tool to assess to what extent Universities are following good Governance practices aligned with their Institutional Goals, but also to allow Universities monitor their progress and compare themselves with other institutions �Inspiration: �Guidelines and Good Practice Codes that have been revised by OECD �Autonomy Score Card- European University Association �CUC in the UK �Benchmarking guidelines- Australian Universities �West Coast Guidelines, USA 14

Institution-level screening card v DIMENSION 1: CONTEXT, MISSION and GOALS Are the missions of the University formally stated? v DIMENSION 2: MANAGEMENT Are the management mechanisms results-based or traditional? v DIMENSION 3: AUTONOMY What is the degree of academic, HR Management, and financial autonomy? v DIMENSION 4: ACCOUNTABILITY AND PARTICIPATION How much is the university held responsible vis à vis its stakeholders? Do the stakeholders have a voice in decision making? 15

Example of Autonomy - Financial v. Financial autonomy - ability of universities to: v set tuition fees v accumulate reserves v keep surplus on state funding v borrow money v invest money in financial or physical assets v own and sell the land buildings they occupy v deliver contractual services; v attract funds on a competitive basis. 16

Example of Autonomy - Academic v. Academic autonomy v Responsibility for curriculum design v Extent to which universities are autonomous to introduce or cancel degree programs and to determine academic structure v Overall number of students v Admissions criteria v Admissions per discipline; v Evaluation of programs; v Evaluation of learning outcomes v Teaching methodologies. 17

Example of Autonomy – HR v. Human Resources autonomy v Recruitment procedures for appointment of academic and other staff – hiring and firing v Status of employees (whether they are considered civil servants or not) v Procedure for determining salary levels, salary incentives, and workloads v Human resources policies v Career development policies v Performance management. 18

MENA Case study: Sample size �Egypt: 12 universities �Morocco: 9 universities �Palestine: 9 universities �Tunisia: 10 universities 6 Public 8 Public 2 Public 7 Public 6 Private 1 Private 7 Private 3 Private �Statistical Analysis �MCA for construction of indices of autonomy �Comparisons between countries/types of universities �Assessment of correlation structure & causality 19

MENA Case study: MCA results % Coord. Has autonomy to define revenue structure of the University (No) 0. 08 1. 994 Has autonomy to set the level of fees (No) 0. 074 1. 724 Has autonomy to set the level of fees (Yes) 0. 074 -1. 724 Has the autonomy to run a deficit (No) 0. 07 1. 678 Has the autonomy to run a deficit (Yes) 0. 07 -1. 678 Has autonomy to set bonuses to be paid to private owners (Yes) 0. 063 -1. 969 Allowed to own Financial Assets (No) Allowed to own Land (No) 0. 056 0. 055 1. 928 2. 497 Has autonomy to define revenue structure of the University (Yes) 0. 053 -1. 329 Categories Financial Autonomy 20

Normalized Indices of autonomy Country Egypt Morocco Palestine Tunisia Academic Autonomy 0. 62 0. 57 0. 74 0. 29 Human Resources Autonomy 0. 76 0. 27 0. 75 0. 36 Financial Autonomy 0. 69 0. 57 0. 81 0. 55 All Dimensions of Autonomy 0. 80 0. 49 0. 86 0. 42 Indices of autonomy Status All Public Private Academic Autonomy 0. 48 0. 65 0. 55 Human Resources Autonomy 0. 30 0. 88 0. 55 Financial Autonomy 0. 52 0. 84 0. 65 All Dimensions of Autonomy 0. 46 0. 90 0. 65 21

Correlations between indices 0 1 2 -0. 5 -1 -2 -1 0 -1 -0. 5 1 2 -1. 5 -2 -2. 5 Staffing Autonomy -2. 5 Academic Autonomy 1. 5 1 1 1 0 -1 -2 1. 5 0 0 -0. 5 -1 -1. 5 -2 -2. 5 Staffing Autonomy 1 2 0. 5 0 0 1 2 3 4 -0. 5 -1 6 0 0 1 2 3 4 5 6 -0. 5 -1 -1. 5 -2 5 Financial Autonomy 0. 5 Staffing Autonomy Financial Autonomy 2 -1 -2 Academic Autonomy -1 0 -2 -0. 5 -1. 5 -2 1 0. 5 Financial Autonomy 0 -3 0 -1 0. 5 Financial Autonomy Staffing Autonomy 0. 5 -2 1 1 1 -3 1. 5 -2 Autonomy Self Assessment -2. 5 Autonomy Self Assessment 22

Importance of various forms of autonomy VARIABLES Autonomy Academic autonomy index Staffing autonomy index Financial Autonomy index Size of the university Log of the number of students Countries (ref= Egypt) Morocco Palestine Tunisia Type of program (ref=Ph. D) Undergraduate Under & Graduate Status (ref=Public) Private Constant Observations Autonomy index Subj. Autonomy 1. 72*** 2. 69*** 1. 86*** 1. 17*** 1. 27*** 0. 86*** -0. 05 0. 04 -0. 19 -0. 24 -0. 19 0. 34* 0. 19 0. 29* -0. 21 0. 01 -0. 22 0. 09 -0. 02 -2. 13*** 40 0. 13 1. 40*** 40 23

Conclusion �SABER: New effort and framework to document and assess policy frameworks �System-wide data and institutio-level data �Institution-level data helpful for implementation, but also calibration (weights) for system-wide indicators �Institution-level tool available for deployment in case of interest among participants at World Congress �Thank you! 24