Project TEMPUS IV 159328 1 2009 FR TEMPUS

Project TEMPUS IV 159328 – 1 – 2009 – FR – TEMPUS - SMHES Life Long Learning Framework for Medical Teaching Staff Present Status of Evaluation of Medical Schools in Italian University Prof. Stefano Elia, Department of Surgery, Italy Tempus Coordinator Prof. Claudio Cortese, Department of Internal Medicine Prof. Antonio Volpi, Department of Internal Medicine ROME TOR VERGATA UNIVERSITY ITALY

The main goals of Medical Faculty are: • To improve student’s learning process • To ensure it • To evaluate it

• It is important to realize that true learning process can only be managed by the student himself • It occurs in his mind where teachers cannot enter • Teachers can only create a stimulating environment and support mental activities necessary to facilitate the learning process • Therefore we have to switch from the historical « teacher-centered » education to the « studentcentered » education

Any reform of medical education has to take into consideration the simultaneous reform of: • Curriculum • Teaching strategies for an effective learning • Evaluation

Curriculum The Association of American Medical Colleges (AAMC) has identified 5 main roles for the future medical doctor: a) Life-long learner b) Clinician c) Teacher/communicator d) Researcher e) Manager Any type of reform of curriculum should take these roles into due consideration

Teaching strategies for an effective learning Teacher’s role is to stimulate in the student’s mind: • Creativity • Problem solving ability • Mind flexibility • Criticism skillness • Opening of ideas • Analysys and synthesys of complex material

Teaching strategies for an effective learning Present worldwide medical teaching tends to emphasize scientific knowledge more than encouraging scientific comprehension, clinical reasoning, practical skills, and the development of character, empathy and integrity

Teaching strategies for an effective learning Medical teachers have to adapt themselves to behavioural changes in the present society focusing their action on: • More effective practical training by systematically repeated clinical tasks • Development of new evaluation methods preferably focused on competences • Improvement of research standards • Change the negative concept ofevaluation into an information step that helps students to improve their daily work

Evaluation • Medical teachers traditionally think that the evaluation procedure comes at the end of the learning process • This creates a gap between student’s learning and evaluation result • Evaluation is part of the teaching/learning process • Teachers have to identify and communicate end-of-course goals to the students in order for them to actively participate in their achievement • New evaluation models have to be used during learning process e. g. through progress test

Progress Test • Should be available along the whole educational track and used to continuously monitor: Ø progression of knowledge Ø implementation of knowledge Ø possible loss of previous knowledge

Progress Test Advantages • It avoids the student’s approach to study because « I need to take the exam » • It gives the opportunity to identify student’s strengths and weaknesses both globally and in single topics • It evaluates long term knowledge and gives confirmation of the achievement of goals • It prevents wasting of time in repeating exams

Progress Test Goals • To make students identify the important concepts • To motivate students in studying • To identify weak areas that need further application • To identify weakness both in the course and teacher • To determine the final score ( give or deny pass)

Italian System of Education High School Bachelor Degree 3 years Master of Science 2 years One-cycle courses (5 or 6 years) Architecture/Law/Medicine Ph. D programs

Italian University System First Reform: 1999 Law n. 509 By-laws and rules AUTONOMY Financial Teaching Organizational

1. 1 st cycle: Degree Courses (goal: to ensure appropriate empowerment of general scientific methods and contents and acquisition of specific professional knowledge) TEACHING AUTONOMY 2. 2 nd cycle: Postgraduate Courses and 1° level Master (goal: provide an advanced level of education to perform high quality activity in a specific environment) 3. 3 rd cycle: a) Postgraduate Courses and 2°nd level Master (goal: provide knowledge and capability for functions required by peculiar professional activities) b) Ph. D. Courses (goal: acquisition of a correct research methodology)

Medical Faculty is an exception since it is articulated in a unique cycle of 6 years

Study Classes (Law 270) Courses of the same level that share goals and qualifying educational activities University Education Credits (Crediti Formativi Universitari - CFU) One credit = 25 working hours (comprehensive of lesson, individual study, lab, verification, etc. ).

Teaching and research autonomy <<art and sciences are free and free is their teaching » …. , meaning that the teacher is independent in exercising his profession as far as programmes and methods are concerned>> Art. 33 of Italian Constitution

Nucleo di Valutazione di Ateneo (University Evaluation Committee) According to present law it has to provide by 30 th April each year the National Committee for Evaluation of University System (CNVSU) the following data : • Educational offer • Students’ data • Personnel’s data • Financial data • Structures • Scientific research data

From 2006 (law no. 262) Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca (ANVUR) National Agency for Evaluation of University System and Research

Quality control system Efficiency of training process is evaluated through: a) Mean teaching hours per year and average number of CFU ( Crediti Formativi Universitari) per student b) Enrolled/attending students c) External yearly evaluation of quality system by CNVSU (Comitato Nazionale per la Valutazione del Sistema Universitario) d) Tracking of educational process by means of : withdrawal rate in the first two years, average yearly CFUs per student and yearly percentage of graduates within planned course duration. Efficacy of training process is evaluated through: a) Level of students’ satisfaction (anonymous questionnaire) b) Employment rate at one, two and three years from graduation.

BEME guidelines for medical education §Structure by problems; §Central role of clinical education; §Investigation of emerging problems, with reference and verification of scientific literature (aiming at searching for evidence proofs); §Evaluation of evidence proofs (accessibility, reliability, relevance, pertinence) ; §Analysis of transferability of evidence into an actual clinical case; §Self-evaluation in terms of self-learning of the work being done.

Main reasons for resistance to change §Sluggishness that pushes to defend your own action §Recognition of clinical committments as prioritary compared to teaching tasks §Non acceptance of education as a science with its principles §Ignoring fundamental elements of the educational process §Poor interest and minor care for educational activities §Complaining for lack of services and support for educational activities

Life-Long Learning Framework for Medical University Teaching Staff

Life-Long Learning Framework for Medical University Teaching Staff Goals: 1. Professional update >> all medical doctors 2. Educational pedagogic update >> only for those physicians who play an educational role Transversal tools: a. Guidelines; b. Of scientific literature (and meta-analysis); c. Network of experts. undergraduate (to include into core curriculum) postgraduate Ability to read and correctly and constantly interpret the reviews. It is a type of self-learning that has to be taught within the basic education of the medical doctor in order to build a mental dress oriented to constant education (and self-education).

Life-Long Learning Framework for Medical University Teaching Staff Pedagogics and medicine Epistemiological specificity: üThe object of the study is a subject, the human being, that can never be reduced to an object; üIt is not applied in a neutral context but in a world of values. Clinical VS Experimental Method

Life-Long Learning Framework for Medical University Teaching Staff Pedagogics and Education in Medical School 1 Pedagogics Last goals of education. It elaborates on the values included and pursued by educational action. Educational goals Docimology: it assesses if , how and how much the established objectives have been achieved. Education Means and strategies to be used to achieve educational objectives

Life-Long Learning Framework for Medical University Teaching Staff Pedagogics and Education in Medical School 2 Therefore the educational process is made of three dimensions on which to work: Core curriculum Planning >> pedagogics Implementing >> education Evaluating >> docimology Educational management Teaching function

Life-Long Learning Framework for Medical University Teaching Staff Undergraduate Implementation of clinical activity Core curriculum Implementation of humanities: phylosophy, sociology, history of medicine, psychology Postgraduate for medical doctors/teachers Pedagogics (education – educational technology) Docimology (experimental pedagogics)

Life-Long Learning Framework for Medical University Teaching Staff Ethical aspects üWhich are the reference values of the learning subject ? üWhich criteria he has to use in decisional processes that involve ethical aspects ? üWhich responsibilities? ü How to harmonize ethics and deontology?

Life-Long Learning Framework for Medical University Teaching Staff Epistemological aspects üWhat is the scientific culture of contemporary medicine ? üHow research models and methodological presumptions are applied? üWhich competences for continuous updating ? Clinical Biomedical

Educational Management Academic institutions devoted to education Administrative, legal, logistic ties Plays the role of putting into relationship each other The management acts besides institutions and teachers in macroplanning and cooperates in the evaluation of the entire course. Some transversal work settings may be: 1) Study curriculum; 2) Courses and lessons such as: “history of medicine”; 3) Contextualize humanities that live within biomedical and clinical knowledge; 4) Recognize and implement research methodologies; 5) Reconstruct the formation of fudamental concepts for professional practice such as health/disease”

Qualification requirements for medical university teaching staff (Teaching skills)

Pedagogics Normative NOT Describing Science The following hypothesis is a proposal of competence NOT a picture of the existing

Qualification requirements for medical university teaching staff (Teaching skills) Lesson Planning Management Implementation Relationship Evaluation Communication

Qualification requirements for medical university teaching staff (Teaching skills) Management competences §Frontal lesson (mimics, voice management, space occupation…) §Cooperative didactics (organization of working groups for more complex goals than simple memorization such as decision making e problem solving). §Management of groups (for the analysis fof conflicts and ability to negotiate) §Simulation and role playing (active and participating didactic to elicit transformation of knowledge into competence and to create situations which are similar to professionale ones) §Technique of case-study (decision making and individual problem solving).

Relational competences § § Physical Setting (spaces, times, disposition of participants) Psychological Setting (space of relation: care is not given to “removed” psychological aspects but to conditions that may be necessary to achieve different specific goals. E. g. : leadership type, definition of roles Communication competences § § § Verbal/paraverbal (oral exposition: times, ways, organization, clearness) Written (needed to transfer the experience) Tecnifor didactic communication (e. g. ppt, audiovisuals, electronic blackboard, videorecorded simulations, up to elearning and teleconference or online working/study groups) Communication: content and exposition Site of power exerting on a single individual concerning the management of knowledge

Qualification requirements for medical university teaching staff (Teaching skills) Planning competences Evaluation competences § § § Times/ways of evaluation (ex ante - in itinere - ex post) Quantitative Methodologies (test creation) Qualitative Methodologies (creation and interpretation) Metareflexive Competences § § § Observation Analysis Euristics

Lab and practising Metareflexive § § § Definition of methodoloigcal premises Research methodology Logic of discovery Management § § § Tutoring Scaffolding Planning of significant working settings Evaluation Observation: § Of practical and tehnical procedures § Of strong communications signals § Of weak communication signals § Of communication abilities § Of relational ailities § Of team working abilities

Didactic evaluation: 1) Teachers’ performance 2) Students’ performance 3) Course performance The evaluation may be quantitative (tests, questionnaires, multiple choice and filling answering, standard interview) and qualitative (non standard interview, participated observation, etnographic method) Medical didactic goals have to be related to the need for care therefore the overall evaluation of a Medical School/Faculty has to include the analysis of the relationship between personnel education and long term changes produced as an answer to individual needs.

Qualification requirements for medical university teaching staff (Teaching skills) How to put this model into practice? 1. Need for an appropriate normative 2. Need for “education for educators” undergraduate postgraduate 3. Evaluation system