The CHE ranking The multidimensional way of Ranking

The CHE ranking The multi-dimensional way of Ranking Isabel Roessler CHE – Centre for Higher Education Development International Conference “Academic Cooperation and Competitiveness – University Ranking Methodologies” Cluj-Napoca, 17 – 20 September 2009

Presentation I. The CHE – Centre for Higher Education Development II. Rankings – Aims and methodology III. The CHE ranking approach IV. Towards a European ranking Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 2

I. CHE – Center of Higher Education Development private, not-profit organisation founded in 1994 by Bertelsmann Foundation and German Rectors Conference purpose: promotion of reforms in German higher education Ranking of German universities among founding tasks of CHE; first ranking in 1998 activities: HE policy issues consulting ranking, since 1998 staff: ~ 30 people more information: www. che. de Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 3

II. Rankings: Aims and Methodology Rankings differ by target groups, particular goals information for prospective students (US News, CHE) information about global positioning (Shanghai Jiatong, THES) Information for HE community (Germany: National Science Foundation Ranking of Research Grants, CHE Research Ranking) even: basis for accreditation (e. g. Nigeria) Rankings vary in aims and target groups as well as „in terms of what they measure, how they measure it and how they implicitly define quality“ (Usher & Savino) Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 4

II. The „classical“ model: ranking orthodoxy The worldwide known Rankings (Shanghai, THES) follow the classical league table approach of ranking of whole institutions aggregation indicators into a single composite overall indicator by using fixed weights league table with individual numerical positions (like soccer table) There is a growing diversity of alternative approaches: deviations from the classical model One example for an alternative approach is CHE ranking Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 5

III. CHE Ranking: Background § ~ 35 subjects § All three years updated § Development of results are shown (Ups and downs) § Board of experts twice a year Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 6

III. CHE-Ranking: Basic Principles no ranking of whole institutions field specific ranking Berlin Principles: Rankings should be „Be clear about their purpose and their target groups. Rankings have to be designed with due regard to their purpose. “ Target groups of CHE ranking (prospective students, universities /academic staff) are interested in information about „their“ field Universities are heterogeneous units; fields differ in their performance ranking of whole institutions gives misleading averages Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 7

III. CHE-Ranking: Basic Principles no composite overall indicator multi-dimensional ranking Composite indicators blur profiles and strengths & weaknesses There are neither theoretical nor empirical arguments for specific weights for single indicators Heterogeneity of preferences about indicators among target groups /users (“quality is in the eye of the beholder”) given weights patronise users / target group of rankings Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 8

III. CHE-Ranking: Basic Principles no league table group approach (top, middle, bottom) League tables tend to exaggerate differences between universities („ 7 th is better than 12 th“) Small differences in the numerical value of an indicator lead to big differences in league table positions (ignoring issues of statistical errors and insecurity) CHE ranking: three groups top group is significantly better than bottom group no differentiation within groups Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 9

III. The CHE-Ranking: Indicators city, university students study outcome internationalisation teaching ressources 20 – 25 indicators. . . Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 10

III. The CHE-Ranking: Indicators. . . from different data sources… < publications /citations (bibliometric analysis) < research grants (faculties/departments) < research reputation (professors survey) < relation to research (students survey) Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 11

III. The CHE-Ranking: Indicators . . . facts as well as judgements < student-staff-ratio (fact) < student assessment of contact between students and professors < student assessment of course organisation Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 12

III. The CHE ranking: Data sources Survey among universities / departments ~1150 faculties facts on teaching & learning, research, resources etc. Student survey ~ 80 000 students up to 500 students per programme Professor survey ~ 10 000 professors on reputation Bibliometric analysis Patent analysis (engineering, sciences) Graduate survey Official higher education statistics Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 13

III. CHE ranking: presentation of results Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 14

III. CHE ranking: presentation of results Compact Ranking § 4 -6 indicators § First ordered by name § All analysed HEI included Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 15

III. CHE Ranking: faculty level Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 16

III. CHE ranking: presentation of results Individualised Ranking § Own priority § up to five indicators § grouped Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 17

III. CHE ranking: presentation of results Step 2: Giving priorities to indicators Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 18

III. CHE ranking: presentation of results Step 3: Individualised results Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 19

III. CHE ranking: presentation of results. . . looking completely different for different indicators Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 20

III. CHE ranking: Impact - Individual 2/3 of students use ranking as one source of information differences by fields /types of students: Ýlaw, medicine, engineering Þhumanities studies show: ranking covers needs of information of prospective students (indicators) Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 21

III. CHE ranking: Impact - Institutional Ranking not used for funding decisions /allocation of money !!! Institutions use data (published data & additional analysis) as a starting point for analysis of strengths and weaknesses for internal comparison / benchmarking between faculties, incl. contracts between president - faculties for external comparison / benchmarking with other institutions ranking helps to identify deficits & asking questions, but does not give all answers Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 22

IV. Towards a European ranking Emergence of European Higher Education area: growing mobility of students and staff growing demand for transparency about European HEIs Internationalisation of CHE ranking: since 2004: Austrian universities 2005: Swiss universities 2006/07: EU-funded pilot project with Dutch/ Belgian (Flemish) universities 2008: Dutch universities, University Bozen/Bolzano (I) 2009: new universities in specific fields: Babes Bolyai, Semmelweis… Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 23

IV. Towards a European ranking Berlin Principles: Rankings should: “Recognize the diversity of institutions and take the different missions and goals of institutions into account. ” No ranking of all 4. 000 European HEIs who have different structures, missions and profiles Rankings within types/clusters of institutions Demand for a European Classification Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 24

IV. Towards a European ranking CHE - European Excellence Ranking ranking of top research departments in mathematics, physics, chemistry, biology focus on information for Master, Ph. D students and research pre-selection of universities based on bibliometric analysis, Marie Curie programme, (active!) Nobel Price Winners disproportional to countries basic CHE approach: field specific, multi-dimensional, no league tables Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 25

IV. Towards a European ranking MGUR: Multi-Dimensional Global University Ranking, EU-Tender ranking of departments in business studies (research focus) and engineering (regional focus) pre-selection of universities disproportional to countries basic CHE approach: field specific, multi-dimensional, no league tables CHERPA Academic Cooperation and Competitiveness/ Cluj-Napoca 17 -20 Sept. 2009 26

Thank you very much ! International Conference “Academic Cooperation and Competitiveness – University Ranking Methodologies” Cluj-Napoca, 17 – 20 September 2009