AUNQA Assessment at Program Level Dr Pepen Arifin

AUN-QA Assessment at Program Level Dr. Pepen Arifin Satuan Penjaminan Mutu Institut Teknologi Bandung Sosialisasi Sertifikasi Internasional Program Studi di lingkungan UPI, 24 Mei 2018

ABET-ASIIN-JABEE-AUN-QA-BAN PT ABET ASIIN JABEE AUN-QA BAN-PT Accreditation Assessment Accreditation International Regional National Outcomes IPOOutcomes SSR - visit SAR - visit SER - visit SAR - visit Professional Semi-Gov Government Program & Institutional Program & Institutional Eng, Comp, Tech Engineering Informatics, Sciences Engineering All Programs

Strategies for International Accreditation p Establishment of Quality Assurance System n In-line of IQA and EQA Systems Shifting in Curriculum Paradigm p Outcomes Based Education p

In-line of IQA and EQA systems Internal Quality Assurance (IQA) (Assessment, Audit) External Quality Assurance (EQA) (Accreditation, Assessment)

AUN-QA Model for Program Level Assessment (3 rd version)

OBE Business Process Vision & Mission Int. Accr. Agency Prog. Ed. Objectives IQF Program Learning Outcomes Stakeholders Survey Prog. Structure & Curriculum Faculty Module 1 Module 2 Facilities Module 3 Module n Alumni Survey Teaching & Learning Std. Advice & Support Students Institutional Support Prog. Management Continuous Improvements Outcomes Assessment Continuous Improvements : ABET/ASIIN Criteria

Definition (1) • Program Educational Objectives/Tujuan Prodi: Statements that describe the expected accomplishments of graduates during the first few years after graduation • Learning outcomes/Capaian pembelajaran: Statements of what a learner knows, understands and is able to do on completion of a learning process, defined in terms of knowledge, skills and competences. • Knowledge: Outcome of the assimilation of information through learning (theoretical and / or factual knowledge).

Definition (2) • Skills: Ability to apply knowledge to complete tasks and solve problems (cognitive skills such as logical, intuitive and creative thinking and practical skills such as manual dexterity and the use of methods, materials, tools and instruments). • Competence: Proven ability to use knowledge, skills and personal, social and / or methodological abilities, in work or study situations and in professional and / or personal development

Learning Outcomes ? Source: ASIIN Consult

Source: ASIIN Consult 10

Source: ASIIN Consult 11

Steps in Program Design Based on Learning Outcome • • Prog. Edu. Objectives General competences/Academic and professional profiles Learning outcome at program level (PLO) Program structure and module composition Module/course outcome Module contents Module assessment Monitoring, Assessment and Evaluation of learning outcome

Program Educational Objectives • Professional Accomplishment • Academic Accomplishment • General/Social Accomplishment

Program Educational Objectives (example) To produce graduate to become professionals that are capable to be researcher, lecturer, teacher, industrial expert as well as entrepreneurs. This includes the following objectives: • Mastering knowledge and methodology of physics, and capable to apply it for problem solving in their work. • Capability to continuously develop knowledge for further study, either formally or informally. • Mastering scientific methods for observing, analysing, and understanding of physical phenomenon, in addition of mastering induction as well as deduction methods and its application to support their carrier. • Keeping with development on their physics field interest along with its interaction to the dynamically development of science and technology, industry and life in general. • Having capability to communicate their ideas orally and written, either scientifically or popularly, and capable to take appropriate initiative, and lead a working group in relevant fields.

KKNI

Program Learning Outcomes • Professional/specialist outcomes • Social/generic outcomes – – – Communication Team work Management and leadership Life-long learning Self development

Profil Lulusan Fisika (Contoh) p p Menguasai fisika klasik dan fisika modern dan menggunakannya untuk menyelesaikan masalah yang dihadapi dalam tugas-tugas keprofesinya secara multidisiplin dan kreatif. Merancang, men-setup, dan melaksanakan ekperimen tentang sistem fisis, menganalisis data serta menga, bil kesimpulan. Mampu menggunakan perlatan terkini untuk melakukan eksperimen tentang sistem fisis. Dapat mencari informasi dari berbagai sumber dan menggunakannya untuk meningkatkan solusi suatu permasalahan secara efektivitas dan efisiensi

p p Mengikuti perkembangan terkini bidang-bidang fisika terkait (misal: fisika teoretik, fisika material dll), dan menerapkannya dalam problem solving masalah-masalah fisika terkini, serta dapat meningkatkan pengetahuannya. Dapat berperan sebagai manager/supervisor untuk sebuah tim yang mengerjakan tugas-tugas lapangan, laboratorium atau penelitian yang mampu mengembangkan aktivitas berorientasi tujuan.

Dari Profil ke Learning Outcomes Knowledge Skill • Memahami konsep fisika klasik dan fisika modern • Menguasai pemodelan sistem fisis • Memahami pengetahuan dalam bidang lain (multidisiplin) • Mengikuti perkembangan fisika terkini • Dapat memecahkan masalah suatu sistem fisis standar secara komprehensif • Dapat menganalisis sistem fisis • Merancang, melaksanakan eksperimen dan menganalisis data • Menggunakan peralatan terkini • Life long learning To know To be able Competence • Mempunyai kemampuan dasar dalam komunikasi lisan & tullisan • Mencari informasi • Menjadi manager/ supervisor To decide

Learning outcomes p p p Mampu menunjukkan pengetahuan tentang konsep-konsep fisika klasik dan fisika modern melalui identifikasi sifat-sifat fisis dari suatu sistem fisis. Dapat memformulasi sistem fisis standar ke dalam model fisis dengan menggunakan matematika. Dapat memecahkan masalah suatu sistem fisis standar secara komprehensif menggunakan matematika dan perangkat komputasi. Dapat menganalisis sistem fisis dengan mengaplikasikan matematika dan perangkat komputasi/ICT. Dapat mendesain dan melaksanakan eksperimen dalam bidang fisika atau bidang lain terkait fisika, serta dapat mengolah, menganalisis, dan menginterpretasi data yang diperoleh.

p p p Mempunyai kemampuan dasar dalam komunikasi lisan dan mampu menulis laporan ilmiah dalam format penulisan yang sesuai. Dapat bekerja secara efektif, baik secara individu maupun dalam grup. Dapat mengaplikasikan pengetahuannya dalam bidang fisika ke bidang yang lebih luas / permasalahan interdisiplin. Memiliki karakter dasar seorang ilmuwan yang baik. Mempunyai kemampuan untuk meningkatkan pengetahuannya dan dapat melanjutkan studi ke tingkat yang lebih tinggi.

Dari Learning Outcomes ke Matakuliah Memahami konsep fisika klasik dan fisika modern • Mekanika • Listrik Magnet • Termodinamika • Gelombang • Fisika Kuantum • Fisika Matematika • Fisika Zat Padat • Fisika Inti Merancang, melaksanakan eksperimen dan menganalisis data • Elektronika • Teknik Pengukuran • Fisika Komputasi Mempunyai kemampuan dasar dalam komunikasi lisan & tullisan • Komposisi • Scientific communication

Outcomes Matakuliah p p Masing-masing matakuliah memiliki course outcomes (CO). Setiap CO harus inline dengan PLO Setiap CO harus mendukung PLO yang terkait Setiap Matakuliah bisa memiliki LO knowledge/skill/competence saja, atau gabungan diantaranya.

What do we have to do first? • Formulate Program Educational Objectives (PEO) alumni profile • Ensure that PEO's are consistent with the mission of the institution. • Ensure that PEO's are well-stated and measurable. • Ensure that PEO's can be linked with the curriculum. • Involve constituencies in formulating the Program Learning Outcomes (PLO’s).

• • • Ensure that PLO’s satisfy AUN QA criteria Ensure that PLO’s are measurable Map PLO’s into POE's Map PLO’s into AUN QA criteria Ensure that curriculum of each program: – achieves PLO’s – satisfies professional component – meets AUN QA criteria

Exercise 1: Constructive alignment of PEO-PLO-Standard/Criteria-Curriculum • Dokumen Kurikulum: – – – – Tujuan Prodi (PEO) Capaian pembelajaran Prodi (PLO) AUN QA criteria Matriks Tujuan Prodi vs Capaian pembelajaran Matriks Capaian pembelajaran vs AUN QA criteria Struktur program Kurikulum Matriks Mata kuliah vs Capaian pembelajaran

What is PLO Assessment? PLO Assessment is a curriculum development process that is intended as a means to improve program success and student learning based on real evidence. • PDCA process • Programs must demonstrate their graduates have outcomes as specified • Programs must have an assessment process with documented results • Evidence that the outcomes are being measured • Evidence that the results of the assessment process are applied to the further development and improvement of the program

Assessment model • Define program learning outcomes to ensure that the PEO's will be met. • Develop and deliver a curriculum that prepares all students to satisfy the learning outcomes. • Monitor/assess the program’s performance in meeting your customers’ needs. • Assess and review process that involves a regular evaluation of the achievement of objectives and assessment of the student learning outcomes. • Continuously improve your program to meet your customers’ needs better. • Model your assessment/review/continuous improvement process after the best business/industry quality initiatives. • Strive to continuously improve your program and your product

What is Assessment? Assessment: Processes that identify, collect, and prepare data to evaluate the achievement of PLO dan PEO. Assessment Tools: Quantitative: • Examination • Exit Survey • Licensing Engineering in Training Exam results Qualitative: • Student portfolios • Alumni Survey • Employers Survey

Goals Program Learning Outcomes: • PLO’s are consistent with PEO's and AUN-QA criteria • Achievement of PLO’s are quantitatively measured • PLO’s are assessed, monitored, evaluated and reported • PLO’s are reviewed and adjusted • Program improvements are made in response to assessment

Tips and Suggestion • Faculty involvement/engagement in assessment activities is absolutely necessary • Keep the process as simple as possible from a procedural standpoint • Avoid complex matrix mappings and redundant assessments • Focus on high-quality direct assessment of student work at key points in the curriculum. • Planning and course design with outcomes and assessment in mind is a key to success. • The real focus of accreditation is program and processes continuous improvement, with a vision of improving to satisfy stakeholders

Faculty Tasks and Responsibilities Prepare and maintain course portfolio: • • • Course outlines Course Learning Objectives (CLO’s) Contribution of Course to Meeting the Professional Component Relationship of Course to Program Learning Outcomes Teaching methods Assessment method Assessment frequency Performance criteria Feedback methods Relationship to the PEO's Relationship to the AUN-QA criteria

• Examples of student performance on homework, quizzes, examinations, projects, etc… • Evidence of operation of the feedback mechanism. Such evidence can include, but is not limited to, graded work, written comments made to students by the instructor, and written comments from students. • Results of a standard end-of-semester questionnaire that addresses overall achievement of LO’s.

AUN-QA Criteria 1. Expected Learning Outcomes 2. Programme Specification 3. Programme Structure and Content 4. Teaching and Learning Approach 5. Student Assessment 6. Academic Staff Quality 7. Support Staff Quality 8. Student Quality and Support 9. Facilities and Infrastructure 10. Quality Enhancement 11. Output

Criteria 1: Expected learning outcome p p p The expected learning outcomes have been clearly formulated and aligned with the vision and mission of the university The expected learning outcomes cover both subject specific and generic (i. e. transferable) learning outcomes The expected learning outcomes clearly reflect the requirements of the stakeholders

Criteria 2: Program Specification p p p The information in the programme specification is comprehensive and up-to-date The information in the course specification is comprehensive and up-to-date The programme and course specifications are communicated and made available to the stakeholders

Description Learning outcomes Program structure modules courses (curriculum document) p Publication of program specification p

Criteria 3: Program Structure and Content p p p The curriculum is designed based on constructive alignment with the expected learning outcomes The contribution made by each course to achieve the expected learning outcomes is clear The curriculum is logically structured, sequenced, integrated and up-to-date

Description University vision-mission Program Educational Objectives Program Learning Outcomes Modules Courses

Program Structure Module 1 Fisrt year Module 2 Module 3 Second year Module 4 Module 5 Module 6 Module 7 Module 8 Third year Fourth year

PLO 1 (Knowledge) Course 1 Module 1 = Basic course Course 2 Course 3 = intermediate course Course 4 Module 2 Course 5 One course may relate to two or even three PLO

PLO 2 (Specialized skill) Course 6 Module 3 Course 7 Course 8 Course 9 Module 4 Course 10 = specialized course

PLO 3 (Generic skill) Course 11 Module 5 Course 12 Course 13 Module 6 Course 14 Course 15

Course matrix PLO 1 Course 1 x Course 2 x Course 3 x Course 4 x Course 5 x PLO 2 PLO 4 x x Course 6 x Course 7 x dst PLO 3 PLO 4

Course outcome vs PLO Course 1 CO 1 PLO 2 PLO 3 PLO 4 x CO 2 x CO 3 CO 4 Course 2 x PLO 1 CO 2 CO 3 x PLO 2 PLO 3 PLO 4 x x x PLO 4

Thank you