Evaluation mechanisms of HK tertiary institutions FOK Ping

Evaluation mechanisms of HK tertiary institutions FOK, Ping Kwan (霍秉坤) Department of Curriculum of Instruction, Faculty of Education, The Chinese University of Hong Kong E-mail: pkfok@cuhk. edu. hk Tertiary Institution evaluation 1

Outline Cases p University Grant Council (UGC) p Comments p Tertiary Institution evaluation 2

Cases Hong Kong Institute of Education p The Hong Kong Polytechnic University p The Chinese University of Hong Kong p Tertiary Institution evaluation 3

Hong Kong Institute of Education p p Scoring Worksheet for Academic Staff Performance Appraisal (Annex VII) Summary of Submission by Appraisee for Academic Staff Performance Appraisal (Annex V) Performance Criteria and Scoring Models (Annex III) Minimum Weighting for Each Performance Domain for Academic Staff (Annex IV) Tertiary Institution evaluation 4

HKIEd items Teaching p Research and scholarly activities p Tertiary Institution evaluation 5

Teaching portfolio (1) p p 1) Teaching Performance – Student Evaluation 2) Teaching Performance – Innovations in Teaching n (e. g ability to demonstrate a reflective self-critical attitude to teaching, willingness to innovate and improve his/her teaching, preparation [including originality of approach and planning] of lectures and tutorials, etc, n innovative use of teaching resources and choice of teaching methods and strategies, willingness to employ new teaching materials / resources, appropriate introduction and use of multimedia and other educational technologies etc. ) Tertiary Institution evaluation 6

Teaching portfolio (2) p p 3) Teaching Performance – Observation of Teaching n Possible sources of evidence may include: p Comments / Feedback from Ho. D, peer nominated by Ho. D and/or peer nominated by the staff member p Any relevant data available to the reviewer / DRC 4) Supervision of Practicum n (e. g. quality of supervision on practicum, projects and/or fieldwork , initiative in practical work, field trips and placement of students in work experience, etc. Tertiary Institution evaluation 7

Teaching portfolio (3) p 5) Scholarship of Teaching and Learning n n p a) Contributions to teaching / learning related projects e. g. TDG projects, and the quality of the involvement / projects completed b) Teaching materials for use by the Institute or schools, such as CDRom, video, teachers’ manuals, teaching packages, children’s texts, etc. 6) Professional and Pastoral Care to Students n (e. g. relations with students, genuine care and support for students, sensitivity to needs of students, a demonstrated willingness to make oneself available for consultation to students and to assist individual students in times of need, ability to interact with and inspire students, encourage students to enquire and search for ideas / information in other venues e. g. library, etc. ) Tertiary Institution evaluation 8

Research and Scholarly Activities (1) p 1) Contributions to Research n p Projects (External funded projects can include QEF, EMB, RGC funded projects from which research output arises. ) 2) Research Output n n (e. g. articles in refereed journals, book chapters / books published, commissioned reports such as Education and Manpower Bureau Commissioned Reports, original music score, artwork, design etc. ) NOTES: For each multiple-authored work, the applicant should indicate his/her percentage contribution to each work. Tertiary Institution evaluation 9

Research and Scholarly Activities (2) p 3) n Providing Support to Staff in Research Service to HKIEd p 1) Contributions to Committee Work at Institute / Departmental Level or Faculty Responsibilities / Representing the Institute on external committees p 2) Contributions to Programme/Curriculum Development, Programme Management, and/or Review Exercises p 3) Participation in New Initiatives / Special Projects / Self-funding activities Tertiary Institution evaluation 10

Service to HKIEd – Scoring Model p Contributions to Committee Work at Institute / Departmental Level or Faculty Responsibilities p Contributions to Programme Development, Programme Management, Review Exercises p Participation in New Initiatives / Special Projects Tertiary Institution evaluation 11

Professional and Community Contribution p 1) Recognition by and Contributions to the Community and / or the Profession n Possible examples of evidence may include: Contributions to public committees p Consultancies to government and non-government bodies p Appointment as External Examiner / Assessor / Advisor / Teaching Consultant p Positions of responsibility in professional bodies p Engagement in public debate (e. g. in media) p p 2) Participation in the Establishment of School Networks / Services to Schools, Involvement in School Development / Consultancy Activities Tertiary Institution evaluation 12

HKIED Professorial salary Academic Chair Professor Min Max 107, 800 133, 600 (average) 72, 100 108, 200 Associate Professor 58, 200 87, 300 Assistant Professor 69, 300 46, 200 Tertiary Institution evaluation 13

HKIEd teaching staff salary Teaching Senior Teaching Fellow / Senior Instructor Teaching Fellow I / Instructor I Teaching Fellow II / Instructor II Tertiary Institution evaluation Min Max 39, 300 60, 900 28, 700 44, 500 21, 100 30, 600 14

Evaluation of CUHK staffs p Annual Departmental Records 2007 -08 (See Appendix A) n Activities Undertaken by Individual Teaching Staff (Instructor and above) Tertiary Institution evaluation 15

Items in CUHK record (1) p 1. Teaching (from August 1 to July 31) n Classroom Teaching (corresponding course evaluation results) n Postgraduate Supervision and Examination n Teaching-Other Contributions Tertiary Institution evaluation 16

Items in CUHK record (2) p Research June 30) and Scholarship (from July 1 to In filling out items 2(a) to 2(c), the teacher should cut and paste below the information provided by the RAO for Appendices C and D of the Annual Departmental Records. n Publications n Research Grants n Research and Scholarship-Other Contributions n Tertiary Institution evaluation 17

CUHK evaluation report p Annual Departmental Records 2007 -08 (See Appendix A) p Summary of Research (See Appendix B) Tertiary Institution evaluation 18

Items in CUHK record (3) p Service (from August 1 to July 31) n Service to Department/Faculty n Service to College/University n External Service Tertiary Institution evaluation 19

CUHK salary ($) Teaching Posts Point Research Assistant Professor RAP 1 -4, ($34, 950 -47, 175) B 1 -2 ($47, 990 -54, 320) Research Associate Professor B 3 -6 ($55, 145 -67, 390) Assistant Professor B 1 -5 ($47, 990 -64, 125) Associate Professor B 6 -9 ($67, 390 -77, 145) Professor 2 B 10 -12 ($81, 205 -82, 030) Professor 1 A 1 -7 ($85, 280 -104, 670) Professor of a subject or specialty Commencing A 8 ($107, 795) Tertiary Institution evaluation 20

Hong Kong Polytechnic University p Manpower Quality Review n n p Frequency n p For academic staff For non-academic staff To be carried out annually Operation n Conducted by a committee p p n n Head of department Two senior staff members in the department Staff members are not required to fill in any forms, submit any document or attend any interviews Based on readily available information Tertiary Institution evaluation 21

Information based on (academic staff) Results of SFQ p Peer review p Submissions of i. RAE and RAE p Results of applications for external funding p Any critical incidents which have occurred in the year --- significant incidents that may have affected or influenced the actual performance of the staff member and significant contributions, achievements or innovative practices of the staff made to the Department / University p Tertiary Institution evaluation 22

Outcomes of Review p p Staff normally will not be informed of the results (unless need follow up actions) Head of department needs to record the results of Review Only confirm if the staff is performing satisfactorily. No specific grading will be given Enable head of departments to n n n Review the roles and / or responsibilities of staff members in general Consider the workload of staff members in general Recognise development needs of staff in general Plan manpower Identify which staff should be appraised by the Staff Appraisal System in the coming year Identify candidates for various HR functions (e. g. salary review, performance reward, contract renewal, promotion and sanction) Tertiary Institution evaluation 23

Manpower Quality Review Report Tertiary Institution evaluation 24

The dominance of UGC p UGC n Excellent Results from the Research Assessment Exercise (RAE) 2006 p n p RAE guideline (see appendix C) Teaching quality assessment RGC report n CERG / GRF p (RGC report) Tertiary Institution evaluation 25

RAE p 2 March 2007 announced the results of the Research Assessment Exercise (RAE) 2006 p Dr Alice Lam (林李翹如), Chairman of UGC, "The UGC is pleased to learn that all UGC-funded institutions have made great strides in their research performance since the last RAE in 1999. Many of our colleagues are internationally known top notch researchers and this reflects the effort of our institutions in excelling in research over the years. “ n RAE 2006 results will be factored into the 'Research' portion of the institutions' recurrent grants. n Tertiary Institution evaluation 26

History of RAEs p The previous three RAEs were carried out in 1993, 1996 and 1999. For RAE 2006, the UGC has raised the standard of assessment. p Roland n Chin : Many of the external assessment panel members are impressed with the performance of our institutions. They think that we compared well with top higher education institutions in Europe and North America. Some members think that Hong Kong has demonstrated an emergence of intellectual prominence in many research disciplines Tertiary Institution evaluation 27

The RAE result p RAE 2006: Research Indices (See Appendix D) n p http: //www. ugc. edu. hk/eng/ugc/publication/prog/rae. ht m RAE example (See Appendix E) Tertiary Institution evaluation 28

RAE result Panel City. U HKBU LU CUHK HKIEd Poly. U HKUST HKU Total 1. Biology 95. 00% 92. 50% -- 78. 79% 25. 00% 72. 73% 95. 16% 85. 61% 85. 29% 2. Health Sciences -- 100. 00 % -- 90. 23% -- 58. 42% -- 94. 44% 86. 84% 3. Physical Sciences 98. 96% 93. 75% -- 95. 16% 29. 17% 87. 25% 92. 61% 94. 62% 91. 57% 4. Electrical & Electronic Engineering 75. 16% -- -- 88. 46% -- 85. 19% 98. 44% 92. 97% 86. 34% 5. Computer Science /IT 74. 19% 69. 64% -- 88. 75% 29. 17% 68. 75% 82. 64% 68. 75% 74. 06% 6. Engineering 83. 93% -- -- 97. 50% -- 67. 59% 89. 58% 82. 05% 80. 69% 7. Built Environment 71. 74% -- -- 50. 00% -- 53. 37% 75. 00% 65. 99% 60. 91% 8. Law 74. 81% 50. 00% -- 100. 00% -- -- -- 86. 48% 81. 03% 9. Business Studies & Economics 73. 08% 80. 96% 84. 14% 88. 41% 3. 67% 65. 09% 80. 90% 81. 04% 76. 72% 10. Social Sciences 75. 83% 63. 52% 80. 36% 88. 18% 25. 00% 44. 83% 89. 29% 81. 47% 75. 16% 11. Humanities 65. 34% 79. 00% 68. 37% 82. 85% 44. 92% 40. 25% 80. 21% 80. 95% 67. 63% 12. Creative Arts, Performing Arts & Design 48. 08% 66. 67% -- 82. 81% 27. 94% 52. 34% -- 86. 11% 57. 18% 13. Education 12. 50% Tertiary Institution evaluation 54. 31% -- 71. 44% 23. 83% -- 29 -- 75. 83% 49. 34%

CERG / GRF Grant Project Funding (see Appendix E) p All year CERG / GRF results (see Appendix C by UGC) p 2008 -09 CERG results (2008 -09 funding result PDF) p Tertiary Institution evaluation 30

What is GRF p The majority of the Earmarked Research Grant (ERG) of the Research Grants Council (RGC) is allocated in response to competitive bids for grants for academic research projects. GRF stands for "General Research Fund" and is the major annual funding exercise of the RGC to support academic research in the UGCfunded institutions. All GRF proposals are subject to a rigorous peer review process via the RGC's four subject panels supported by an international network of expert referees. The closing date for applications is usually the end of October. Tertiary Institution evaluation 31

Criteria for Consideration of Competitive Bids p The following criteria are used in considering competitive bids : academic quality n institutional commitment n contribution to academic/professional development n potential for social, cultural or economic application n availability of, and potential for, non-RGC funding n Tertiary Institution evaluation 32

Academic quality p As regards item (a), members have agreed that "academic quality" should cover : - scientific and scholarly merit of the proposal; - qualifications and track record of the investigator(s) - originality; - feasibility within the time-scale of the proposal. Tertiary Institution evaluation 33

My comments (1) p "The international expert panels had conducted the research assessments in a fair and rigorous manner, " Dr Lam added. n n p (Fok: Is it really fair and rigorous? Not able to conclude) Prof Chin cautioned readers of the RAE results that, "We should not read the data out of context. It is neither fair nor appropriate to compare directly the institutions' performance in research. We need to give due regard to the different roles, missions, discipline focus, and history of the institutions. “ n (FOK: Really no comparison? ) Tertiary Institution evaluation 34

My comments (2) p The UGC will allocate a total of about $11 billion to institutions for research during the 2005 -08 triennium. This figure includes research grants administered by the Research Grants Council and part of the recurrent grants provided by the UGC to the institutions supporting research activities. n (Fok: Important implications) Tertiary Institution evaluation 35

Importance of publication Journal articles p Books p International handbook chapters p Conference paper not counted p Local journals not counted p Chinese Journals is not the first priority p Tertiary Institution evaluation 36

Importance of RAE Image of the university p Funding p Tertiary Institution evaluation 37

Importance of CERG / GRF p Every year: a very keen competition (RGC website) n n Financial implication Media The first priority (Chair Professor) p The first priority (allocation of rooms) p Financial support for staffs (CUHK & HKIEd) p Tertiary Institution evaluation 38

Teaching is important Only the baseline p Not below this p Distinguished Teaching Award p n Not renewing contract in HKIEd Tertiary Institution evaluation 39

Important notes UGC domination p Research is very important --- CERG and GRF p RAE is the most important p Extremely focus by UST, CUHK, HKU p Tertiary Institution evaluation 40

Differences between institutions p Not the formalities that counted n n p HKIEd has the most detailed procedures and wellstructured CUHK and Poly U are not having very clear instruction Not working according to the said items n n n HKIEd fired someone who had over 70 marks HKIEd fired someone who had good comments from students Indeed, most institutions assessed according to RAE and (external) research grants Tertiary Institution evaluation 41

Thank you!! Tertiary Institution evaluation 42