ASSESSMENT ADIBAH BINTI ABDUL LATIF SCHOOL OF EDUCATION

ASSESSMENT ADIBAH BINTI ABDUL LATIF SCHOOL OF EDUCATION FACULTY OF SOCIAL SCIENCES AND HUMANITIES

ACTIVITY 1 • Please write down one thing that you want to know about the process of assessment related to your job scope. (5 Minutes)

4 A BASIC CONCEPT IN ASSESSMENT

“Assessment is at the heart of student experience” Brown and Knight (1994) “If you want to change student learning then change the method of assessment” Brown, Bull & Pendlebury (1997)

5

Introduction • Educational measurement and evaluation are essential to sound educational decision making • Should be based on accurate, relevant information and that the responsibility of gathering and imparting that information belongs to educators • Will affect many people for example, parent’s or only a single person

TERMINOLOGY Assessment Evaluation Measurement Testing

TESTING • A tool to determine student’s ability to complete specific tasks or demonstrate mastery of a skill or knowledge of content • The most critical basis in ensuring the validity of students interpretation score • Examples: Q&A session in class, assignment, performance task, test, quiz and final exam

MEASUREMENT • A systematic process of assigning numerals (quantitative) to the test administered. • It can be in raw scores, percentile, standard score, etc. • Examples: Assignment marks, total sore in a final exam, mean of PLO, KPI score, mean of elppt, ranking score.

MEASUREMENT Ability 03 March 2021 Score SPP 2032: : Educational Measurement and Evaluation

EVALUATION • The process of describing, obtaining, and providing useful information for judging decision alternatives. This process allows one to make a judgment about the desirability or value of something. • Examples: ABCD, Pass and Fail, HL, TM, MM, Description from the value (Baik, Cemerlang, Sederhana), Description of P 1 to P 5 in e-LPPT.

ASSESSMENT • The process of gathering information to monitor and reflect the progress in learning and teaching and to make educational decisions if necessary. • Examples: Dr A found out that her students are excellent in formative assessment, but they cannot perform well in their final exam. What to do? • Dr B realises that there are two groups of ability/level of skills in his class. How can he do to make sure the learning environment can help students’ learning?

ACTIVITY 2 • Provide three situations related to testing, measurement, evaluation and assessment. Discuss in groups and other groups will guess the answer. (20 minutes)

Differences between ASSESSMENT and EVALUATION • Let us discuss together!!

Dimensions of Difference Between Assessment and Evaluation • • Timing Focus of Measurement Relationship Between Administrator & Recipient Findings, Uses thereof Ongoing Modifiability of Criteria, Measures thereof Standards of Measurement Relation Between Different Objects of A/E

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing to Improve Learning Timing

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing Summative: Final to to Improve Learning Gauge Quality Focus of Measurement

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing Summative: Final to to Improve Learning Gauge Quality Process-Oriented: Focus of Measurement How Learning Is Going

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing to Improve Learning Process-Oriented: How Learning Is Going Administrator/Recipient Relationship Summative: Final to Gauge Quality Product-Oriented: What’s Been Learned

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing to Improve Learning Process-Oriented: How Learning Is Going Reflective: Internally Defined Criteria/Goals Summative: Final to Gauge Quality Product-Oriented: What’s Been Learned Administrator/Recipient Relationship

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing to Improve Learning Process-Oriented: How Learning Is Going Reflective: Internally Defined Criteria/Goals Findings, Uses Thereof Summative: Final to Gauge Quality Product-Oriented: What’s Been Learned Prescriptive: Externally Imposed Standards

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing to Improve Learning Process-Oriented: How Learning Is Going Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Summative: Final to Gauge Quality Product-Oriented: What’s Been Learned Prescriptive: Externally Imposed Standards Findings, Uses Thereof

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing to Improve Learning Process-Oriented: How Learning Is Going Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Ongoing Modifiability of Criteria, Measures Summative: Final to Gauge Quality Product-Oriented: What’s Been Learned Prescriptive: Externally Imposed Standards Judgmental: Arrive at an Overall Grade/Score

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Formative: Ongoing to Improve Learning Process-Oriented: How Learning Is Going Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Flexible: Adjust As Problems Are Clarified Summative: Final to Gauge Quality Product-Oriented: What’s Been Learned Prescriptive: Externally Imposed Standards Judgmental: Arrive at an Overall Grade/Score Ongoing Modifiability of Criteria, Measures

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Process-Oriented: How Learning Is Going Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Flexible: Adjust As Problems Are Clarified Standards of Measurement Product-Oriented: What’s Been Learned Prescriptive: Externally Imposed Standards Judgmental: Arrive at an Overall Grade/Score Fixed: To Reward Success, Punish Failure

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Process-Oriented: How Learning Is Going Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Flexible: Adjust As Problems Are Clarified Absolute: Strive for Ideal Outcomes Product-Oriented: What’s Been Learned Prescriptive: Externally Imposed Standards Judgmental: Arrive at an Overall Grade/Score Fixed: To Reward Success, Punish Failure Standards of Measurement

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Flexible: Adjust As Problems Are Clarified Absolute: Strive for Ideal Outcomes Relation Between Objects of A/E Prescriptive: Externally Imposed Standards Judgmental: Arrive at an Overall Grade/Score Fixed: To Reward Success, Punish Failure Comparative: Divide Better from Worse

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Flexible: Adjust As Problems Are Clarified Absolute: Strive for Ideal Outcomes Coöperative: Learn from Each Other Prescriptive: Externally Imposed Standards Judgmental: Arrive at an Overall Grade/Score Fixed: To Reward Success, Punish Failure Comparative: Divide Better from Worse Relation Between Objects of A/E

Assessment n Evaluation (various sources, but especially Dan Apple 1998) Reflective: Internally Defined Criteria/Goals Diagnostic: Identify Areas for Improvement Flexible: Adjust As Problems Are Clarified Absolute: Strive for Ideal Outcomes Coöperative: Learn from Each Other Prescriptive: Externally Imposed Standards Judgmental: Arrive at an Overall Grade/Score Fixed: To Reward Success, Punish Failure Comparative: Divide Better from Worse Competitive: Beat Each Other Out

Summary of Differences Dimension of Difference Assessment Evaluation Timing Formative Summative Focus of Measurement Process-Oriented Product-Oriented Relationship Between Administrator and Recipient Reflective Prescriptive Findings, Uses Thereof Diagnostic Judgmental Ongoing Modifiability of Criteria, Measures Thereof Flexible Fixed Standards of Measurement Absolute Comparative Relation Between Objects of A/E Coöperative Competitive

• Discuss some test/task that called as evaluation and called as assessment. • Can evaluation and assessment happen concurrently in one test/task? • Are there any differences between summative assessment and evaluation?

PRINCIPLE OF ASSESSMENT • Well-aligned with educational learning outcomes. • Assessment should be valid and reliable • Formative assessments needs to scaffold students in the summative assessment • Student should receive feedback on their work in timely manner. • Assessment should be inclusive and equitable for all students • Assessment is not used to threaten and intimidate students. • Assessment should help student mastery learning.

ASSESSMENT IN OBE Identify outcomes Determine assessment Learning activities Curriculum

TYPES of ASSESSMENT ■Assessment Of Learning (Ao. L) ■Assessment For Learning (Af. L) ■Assessment As Learning (Aa. L)

Types of Tests Essay or Subjective -Short-answer -Discussion or extended response -Oral -Mathematical Problem Objective -Short-answer single word, formula multiple words/phrase -True or False / Yes or No -Multiple Choice -Matching

Types of Tests Test is classified of required answers by the student and not by the questions Answer Capacity Low Moderate High Short Answer Essay Multiple-choice Structured Response Cloze Test

Types of Tests Test might be in any form such as Informal test – classroom tests that are not arrange or plan Teacher’s made test – a procedure whereby the teacher can reliably and validly evaluate how effective their students have been taught

Types of Tests Achievement test – To measure student’s progress or performance after a period of lesson Aptitude test – To determine and estimate the student’s achievement and what students’ can do when performing at their best Diagnostic test – To identify learning difficulties

Purpose of the Test Measure of maximum performance To determine the students’ ability The students are motivated to obtain as high a score as possible Examples: IQ test Subject/course test Aptitude/achievement test

Purpose of the Test Measure of typical performance To measure students’ interest, personality and attitude Responses are classified thru the preset criteria, not the highest marks Examples: Affective Test Personality Test Career Test

Types of Classroom Evaluation Placement • Pretests designed that is before teaching process Formative • To identify the presence or absence of prerequisite skills Diagnostic • Concern with the students’ readiness to begin the instruction Summative • Concern with the appropriateness of our planned instruction Example: A pretest in algebra might be confined to computational skill in arithmetic

Types of Classroom Evaluation Placement Formative Diagnostic Summative • Tests are given periodically and it is ongoing during the process • To reinforces successful learning and reveals learning weaknesses • Covers some predefined segment of instruction • Help the teachers in making decision about students’ progress Example: Monthly/weekly tests, oral questions during teaching process, assignments, worksheets SPP 2032 Educational Measurement and Evaluation

Types of Classroom Evaluation Placement Formative Diagnostic Summative • As needed during instruction • To detect students’ learning difficulties • To pinpoint the specific types of error each student is making • Takes up where the formative test leaves off

Types of Classroom Evaluation Placement Formative Diagnostic Summative • At the end of a course • Results are used primarily for assigning grades or certifying student • Used for evaluating the effectiveness of the instruction Example: Final examination.

Assessment Procedures Test Preparation Test Administration Inspection and Measurement Analysis Interpretation and Action

Norm Referenced Test and Criterion Referenced Test Norm Referenced Test Criterion Referenced Test 47

Let’s discuss Issues on testing, measurement, evaluation and evaluation that you face as a lecturer 48

OBJECTIVITY INTERPRETABILITY ADMINISTRABILITY VALIDITY RELIABILITY USABILITY

HOW YOU MEASURE YOUR VALIDITY AND RELIABILITY OF YOUR TEST? HOW YOU CHOOSE YOUR ITEM IN YOUR ITEM BANK? 50

CHARACTERISTIC OF A GOOD TEST • VALIDITY (KESAHAN) • Measuring what should be measured • The appropriateness of the interpretations made from test scores and other evaluation results with regard to a particular use.

VALIDITY • CONTENT • CONSTRUCT • CRITERION RELATED

CONTENT VALIDITY • Most related with achievement test • The test represent the topic and cognitive process towards the syllabus. • Does it measuring learning objectives? –cognitive / affective / psychomotor • Table of specification • Subject matter expert

KUALITI INSTRUMEN 1. KESAHAN KANDUNGAN Boleh dilakukan melalui semakan pakar dalam kursus berkenaan dan juga melalui Jadual Spesifikasi Item (JSI). Kesahan kandungan adalah sesuatu yang bersifat kualitatif.

JADUAL SPESIFIKASI ITEM PEPERIKSAAN AKHIR TAHUN SPPP 2032

Table of Spesification

KUALITI INSTRUMEN 2. KESAHAN KONSTRUK Merujuk kepada kecukupan dan ketepatan item dalam menguji sesuatu pembolehubah / konstruk yang dikaji. Contoh: Cukupkah item yang digunakan dalam menguji pengetahuan pelajar dalam kursus anda? Dilakukan secara kuantitatif. Menggunakan Raw Variance Explained by Measure.

Kesahan konstruk Rubrik ini dapat menguji 51. 6% kemahiran komunikasi pelajar

Kesahan konstruk Rubrik ini dapat menguji 24. 6% kemahiran komunikasi pelajar

KUALITI INSTRUMEN 3. KEBOLEHPERCAYAAN Merujuk kepada kebolehulangan ujian untuk ditadbirkan kepada pelajar lain yang homogen dengan pelajar yang sedang diuji. Menggunakan analisis kebolehpercayaan Item.

RELIABILITY Test-retest reliability. • reliability coefficient is obtained by administering the same test twice and correlating the scores. • an excellent measure of score consistency as one is directly measuring consistency from administration to administration.

RELIABILITY Split Half Test • Coefficient is obtained by dividing a test into halves, correlating the scores on each half, and then correcting for length (longer tests tend to be more reliable). • The split can be based on: odd versus even numbered items, randomly selecting items, or manually balancing content and difficulty. • Advantage: only requires a single test administration. • Weakness: - the resultant coefficient will vary as a function of how the test was split. - not appropriate on tests where speed is a factor

RELIABILITY Internal Consistency • Internal consistency focuses on the degree to which the individual items are correlated with each other and is thus often called homogeneity. • The Coefficient is determined by - Cronbach’s alpha, - Kuder- Richardson Formula 20 (KR-20) - Kuder-Richardson Formula 21 (KR-21) • The advantages: • only require one test administration and they do not depend on a particular split of items The disadvantage: They are most applicable when the test measures a single skill area

RELIABILITY Alternate Form Reliability ØMost standardized tests provide equivalent forms that can be used interchangeably. ØThese alternative forms are typically matched in terms of content and difficulty. ØScores on pairs of alternative forms for the same examinees are correlated to provide a measure of consistency or reliability.

RELIABILITY CORRELATION = RELIABILITY

Kebolehpercayaan Item 98% pengulangan keputusan boleh Berlaku jika ditadbir kepada kumpulan Pelajar lain

Activity WHAT CAN BE CONCLUDED FROM THE GIVEN DIAGRAM? 67

Validity and Reliability

Validity and Reliability

Validity and Reliability 70

TABLE OF SPECIFICATION (TOS) JADUAL SPESIFIKASI ITEM (JSI)

PENGENALAN • JSI ialah satu jadual yang menunjukkan perwakilan secara grafik berkaitan silibus pelajaran dengan hasil pembelajaran topikal (TLO) dan selari dengan hasil pembelajaran kursus [CLO]

PENGENALAN • JSI juga menunjukkan jumlah item dan pembahagian markah untuk setiap item. Jenis item juga boleh ditentukan.

PENGENALAN • JSI dirancang oleh pensyarah sebagai asas dalam pembinaan ujian samada peperiksaan akhir tahun / peperiksaan pertengahan tahun / ujian / kuiz.

PENGENALAN �Kubiszyn & Borich, (2003) emphasized the following significance and components of TOS: 1. A Table of Specifications consists of a two-way chart or grid relating instructional objectives to the instructional content. The column of the chart lists the objectives or "levels of skills" (Gredlcr, 1999) to be addressed; The rows list the key concepts or content the test is to measure.

AKTIVITI 1 Bilakah Jadual Spesifikasi Item dibina?

TUJUAN JSI • Menjamin content validity • Memastikan sample item yang representative secara adil. • Ujian memfokuskan kepada kandungan yang penting • Menentukan pemberat / masa yang akan ditetapkan dalam kuliah.

TUJUAN JSI • JSI juga dapat membantu pensyarah sebagai panduan dalam perancangan menetapkan topik yang lebih penting, masa yang diperlukan untuk topik tertentu dan apakah tugasan / projek yang boleh dilakukan untuk membantu pelajar belajar topik tersebut lebih bermakna.

TUJUAN JSI According to Bloom, et al. (1971), "We have found it useful to represent the relation of content and behaviors in the form of a two dimensional table with the objectives on one axis, the content on the other”.

TUJUAN JSI 2. A Table of Specifications identifies not only the content areas covered in class, it identifies the performance objectives at each level of the cognitive domain of Bloom's Taxonomy. Teachers can be assured that they are measuring students' learning across a wide range of content and readings as well as cognitive processes requiring higher order thinking.

TUJUAN JSI 3. A Table of Specifications is developed before the test is written. In fact it should be constructed before the actual teaching begins.

TUJUAN JSI The cornerstone of classroom assessment practices is the validity of the judgments about students’ learning and knowledge. A TOS is one tool that teachers can use to support their professional judgment when creating or selecting test for use with their students.

TUJUAN JSI In order to understand how to best modify a TOS to meet your needs, it is important to understand the goal of this strategy: improving validity of a teacher’s evaluations based on a given assessment. Validity is the degree to which the evaluations or judgments we make as teachers about our students can be trusted based on the quality of evidence we gathered (Wolming & Wilkstrom, 2010).

TUJUAN JSI A Table of Specifications helps to ensure that there is a match between what is taught and what is tested. Classroom assessment should be driven by classroom teaching which itself is driven by course goals and objectives. Tables of Specifications provide the link between teaching and testing. (University of Kansas, 2013)

Ujian yang sah dan boleh dipercayai Pemberat yang sesuai bagi setiap topik Ujian yang adil dan seimbang KELEBIHAN JSI Sample ujian yang representative Keyakinan pelajar terhadap ujian

LANGKAH PEMBINAAN JSI • Enam langkah utama ialah: 1. Menganalisis objektif pengajaran 2. Mengkaji sukatan / topik pelajaran 3. Penekanan keutamaan topik dalam ujian 4. Menentukan masa yang digunakan untuk sesuatu topik 5. Menentukan jenis soalan 6. Menentukan bilangan soalan

FORMULA Formula A Relative weight for the importance of content = ( The number of the TLO / class period for one topic ÷ total TLO/ class period ) × 100% (3/10)*100 = 30

Content TLO / Hours spent Relative weight of the subject 3 30% Topic 2 1 10% Topic 3 1 10% Topic 4 2 20% Topic 5 1 10% Topic 6 2 20% 10 100% Topic 1 Total TLO / class periods for teaching the unit

FORMULA Formula B Relative weight for the item = (% of weight in each Bloom level x total item of the test) (0. 3*20)= 6

Objectives Totals 100% Knowledge and Comprehension 30 % Application and Analysis 50% Evaluation and Synthesize 20% Totals 100% 6 10 4 20 Topics Topic 1 (30 %) Topic 2 (10 %) Topic 3 (10 %) Topic 4 (20 %) Topic 5 (10 %) Topic 6 (20 %) Weight for item

FORMULA Formula C Identify the number of questions in each topic for each level of objectives = (The total number of test x relative weight of the topics x relative weight of Bloom level) (20*0. 3)= 1. 8

Objectives (Totals 100%) Knowledge and Comprehension 30 % Application and Analysis 50% Evaluation and Synthesize 20% Totals 100% Topic 1 (30 %) 1. 8 (2) 3 (3) 1. 2 (1) 6 Topic 2 (10 %) 0. 6 (1) 1 (1) 0. 4 (0) 2 Topic 3 (10 %) 0. 6 (1) 1 (1) 0. 4 (0) 2 Topic 4 (20 %) 1. 2 (1) 2(2) 0. 8 (1) 4 Topic 5 (10 %) 0. 6 (0) 1 (1) 0. 4 (1) 2 Topic 6 (20 %) 1. 2 (1) 2(2) 0. 8 (1) 4 6 10 Topics Number of questions 4 20

Kajian Kes Jumlah jam kredit yang ditetapkan adalah tidak tepat. Apabila dijumlahkan hanya mendapat 38 jam sahaja

CADANGAN • Menggunakan formula dalam pengiraan excel

Kajian Kes Tidak sejajar di antara CLO dan aras taksonomi

CADANGAN • Kuiz, test, dan peperiksaan akhir dirangkum dalam satu CLO sahaja. • CLO lain memfokuskan pentaksiran secara praktikal dan kemahiran generik

Kajian kes Bagaimanakah sebenarnya bilangan item ditetapkan? Berasarkan jam kuliah? Pemberat CLO? Ini kerana jumlah Item yang sama sahaja bagi jumlah jam kuliah yang berbeza

CADANGAN • Aplikasikan formula yang dicadangkan • Masukkan hanya topik yang paling signifikan terutama bagi soalan berbentuk subjektif

Kajian Kes Tidak salah sebenarnya bagi peringkat sijil / diploma untuk mempunyai CLO / aras item yang melebihi aras Aplikasi

CADANGAN • Gunakan satu sahaja CLO bagi mengukur PLO 1 (Pengetahuan) • Aplikasikan konsep penjajaran konstruktif.

Kajian Kes Soalan MCQ bukanlah dibina untuk aras mudah sahaja. Malah bagi ujian berbentuk sumatif seharusnya memenuhi konsep normal distribution

CADANGAN • Kemahiran dalam pembinaan item peperiksaan • Memenuhi prinsip pembinaan item

Kajian kes Antara contoh CLO yang tidak sesuai diukur dengan ujian bertulis

CADANGAN • Tidak semua CLO perlu diukur dengan peperiksaan • Bina satu CLO khas untuk pengetahuan (PLO 1)

Kajian Kes Item ini bukanlah item berbentuk struktur

CADANGAN • Pensyarah perlu dapat membezakan apa yang dimaksudkan dengan item jawapan pendek, respon terhad, respon terbuka dan soalan berstruktur.

Kajian Kes Bagaimana penetapan dan pemilihan topik bagi ujian berbentuk esei? Adakah perlu melibatkan semua topik? Bagaimana dari segi aras kesukaran?

Kajian Kes Instead of letakkan bilangan item, letakkan nombor soalan, Contohnya soalan 1 (a), 1(b), 2(a),

Kajian Kes Tiada jajaran dengan CLO Tiada pecahan aras kesukaran item

Practical skills should not be assessed by examination

MASA UJIAN Carey (1988) pointed out that the time available for testing depended not only on the length of the class period but also on students' attention spans.

MASA UJIAN Linn & Gronlund (2000): 1. A true-false test item takes 15 seconds to answer unless the student is asked to provide the correct answer for false questions. Then the time increases to 30 -45 seconds. 2. A seven item matching exercise takes 60 -90 seconds.

MASA UJIAN 3. A four response multiple choice test item that asks for an answer regarding a term, fact, definition, rule or principle (knowledge level item) takes 30 seconds. The same type of test item that is at the application level may take 60 seconds.

MASA UJIAN 4. Any test item format that requires solving a problem, analyzing, synthesizing information or evaluating examples adds 30 -60 seconds to a question.

MASA UJIAN 5. Short-answer test items take 30 -45 seconds. 6. An essay test takes 60 seconds for each point to be compared and contrasted.

TEST THEOREM If an individual can perform the most difficult aspects of the objective, the instructor can "assume" the lower levels can be done. However, if testing the lower levels, the instructor cannot "assume" the individual can perform the higher levels.

ARAS KESUKARAN UJIAN Aras kesukaran • Memastikan item yang dibina adalah bersesuaian dengan aras keupayaan pelajar. • Membuktikan aras kesukaran item yang ditetapkan dalam JSI.

• Analisis aras kesukaran ini dilakukan untuk penetapan aras item untuk disimpan di dalam bank item. • Bagi mengkaji semula aras kesukaran item yang diletakkan semasa penulisan Jadual Spesfikasi Item. • Analisis secara CTT boleh dilakukan sebagai asas analisis item

Item Difficulty Level: Definition The percentage of students who answered the item correctly. 0 High (Difficult) Medium (Moderate) Low (Easy) ≤= 30% > 30% AND < 80% ≥=80% 10 20 30 40 50 60 70 80 90 100

c T T • Menentukan indeks kesukaran bagi item objektif: pengiraan itu boleh dilakukan dalam bentuk jadual seperti dibawah.

c T T • Menentukan indeks kesukaran bagi item subjektif: -Pengiraan dibuat seperti jadual di bawah:

Item Difficulty Level: Discussion • Is a test that nobody failed too easy? • Is a test on which nobody got 100% too difficult? • Should items that are “too easy” or “too difficult” be thrown out?

KUALITI INSTRUMEN Indeks Diskriminasi Item Bagi memastikan item yang dibina berfungsi dengan baik. Boleh dianalisis menggunakan CTT dan IRT. Item yang baik seharusnya dapat membezakan keupayaan pelajar yang berpencapaian tinggi dan berpencapaian rendah. Indeks diskriminasi membantu penetapan item dibuang dan disimpan dalam bank item Bagaimana anda menganalisis indeks kesukaran item?

What is a “good” value? If the item has Ratio of Students answered the item correctly Positive Discrimination High achievers >Low achievers Negative Discrimination High achievers < low achievers No discrimination High achievers = low achievers

What is a “good” value? Discrimination Index 0. 40 and above Item Evaluation Very good 0. 30 -0. 39 Good and can be improved 0. 20 -0. 29 Marginal and need improvement Below 0. 19 Bad, cant be accepted and need proper checking

c T T • Contoh: Jika terdapat 40 orang murid dalam satu kelas, bahagikan mereka kepada dua kumpulan iaitu 20 murid pencapaian tinggi dan 20 murid pencapaian rendah. Misalnya bagi item 8, 16 murid dari kumpulan berpencapaian tinggi dapat menjawab dengan betul manakala hanya 4 orang murid dari kumpulan berpencapaian rendah yang menjawab betul bagi item tersebut. Maka: K t = 16 = 0. 8 atau ( 80 % ) 20 K r = 4 = 0. 2 atau ( 20 % ) 20 D = K t – K r = 0. 8 - 0. 2 = 0. 6 ( Kesimpulannya item 8 adalah item yang baik)

GRADED ASSIGNMENT • Develop one table of specification for your final exam, using the formula given in this workshop. • Send in softcopy (using excel form) • Individual/ Group Assignment according to your course.

GRADED ASSIGNMENT • Analyze your final examination item using Classical test theory • Send in softcopy (using excel form) • Individual/ Group Assignment according to your course

THANK YOU

4 B PLANNING AND MANAGING ASSESSMENT

“Students can escape bad teaching but they cannot escape bad assessment. ” Boud, 1995 131

ACTIVITY • Share with others the BEST PRACTISE of your assessments including how you plan and manage them ü Lab assessment ü PBL ü Generic skills ü Case Study ü Project ü Etc.

Guiding Questions • What is the purpose? What do you hope to learn from this assessment? • What will be assessed? (learning objective) • Who will be assessed? • How will it be assessed? • In what setting will the assessment be conducted? • How will the results be analyzed? • How will the results be used? How will the results be helpful to you and your students? • To whom will the results be communicated? 133

The student voice - what they would like… For the assessment load to be spread out More advice & support Clear criteria I never realised that assessment was for learning Feedback to help me learn 134 Ref: MESA Project Choice of assessment topics Relevant & real-life topics Variety of assessment methods

Start Small q Step 1: Plan - Clarify the evidence requirements. - Choose most appropriate assessment methods. q Step 2: Implement & Act - Design and develop the assessment tools. - Make sure students know what you are doing and that they clearly understand the procedure. Collect the responses and analyze them as soon as possible. q Step 3: Respond & Reflect - Refine tools, to help maximise confidence that the tool/s can be used flexibly and assist to make valid, reliable and fair judgements. 135

Assessment Plan • Should be manageable and meaningful • Takes into account specified programme outcomes • Identify course learning outcomes which are mapped to PLO • Takes into consideration that assessment is a continuous improvement process. 136

Assessment Cycle 137

138

1. Getting Started • Know program goals – What programme outcomes to be achieved – Knowledge, technical/ professional skills, soft skills and values to be acquired • Determine course outcomes 139 – Specify activities, products or performances to be measured/ assessed/ evaluated – State what students will know,

Linking Assessment and Instruction q What we assess q How we communicate the results 140 q What is worth learning q How it should be learned q How well we expect them to perform

2. Designing Strategies • What to assess? – Cognitive, psychomotor, affective • How to assess – methods, quantity • When to assess? • How to manage? 141

Common Teaching Situations & Associated TLA Situations Lecture Tutorial Project PBL 142 Teaching Activities • Talk, explain, clarify Learning Activities o. Listen, take notes, accept, query, discuss with peers, one-minute paper • Set/answer o Pre-read, prepare questions, provide questions, learn from feedback peers, critique, analyse • Set brief, provide o. Apply, create, self-monitor, ongoing feedback communicate, teamwork • Set problems, provide o. Set learning goals, design, feedback apply, access desired content and skills, integrate, solve problems

Assessment Tasks and Learning Opportunity Communicate information & ideas in other than the written word Become more adept at writing in different forms, formats & ‘genres’ To understand the benefits & challenges of working collaboratively 143 Seminar, poster, video conferencing or other multimedia Design or proposals; book reviews; case report/ studies; web Group pages; problem journal articles solving; joint book reviews; team presentations; role plays; group

Assessment Methods - How you gather the evidence Methods Examples Direct observation Real work/real time activities at the workplace Work activities in a simulated workplace Structured assessment activities Simulation exercises/role-plays, Projects Presentations Activity sheets Written questions Interviews Self-assessment Verbal questioning Questionnaires Oral or written examinations Questioning 144

Cont. Methods Examples Evidence compiled Portfolios by the candidate Collections of work samples Products with supporting documentation Historical evidence Journal/log books Information about life experience Review of products Products as a result of a project Work samples/products Third party Testimonials/reports from feedback employers/supervisors Evidence of training Authenticated prior achievements 145 Interviews with employers, supervisors or

3 Deciding Methods of Assessment • Determine methods/tasks – choose measure & instruments to collect appropriate data • Use multiple approaches – measure a wide range of students’ capabilities • Consider administration times • Provide opportunities for students to demonstrate their learning • Identify methods that can assess multiple outcomes o Eg: projects – can assess written and oral communication, group work, independent learning, problem solving skills 146

Assessment Tasks (ATs) provide students the opportunity to demonstrate whether or not they have achieved the LOs and what level their performance is in those LOs. should be appropriately designed or selected to address the LOs that we want to assess. different assessment methods/tasks address different LOs. There should therefore be several kinds of task. provide the evidence allowing lecturers to make a judgment about the level of a student’s performance against the LOs and to award a final grade. 147

AT - Should we offer a choice? • Standardized tests • Exam (Open or close book) • Project • Resumes, letters, memos • Learning logs • Reflection journals • Indiv/Group presentation • Poster • Portfolio • Literature search • Audio / video tape • Spreadsheet • Questionnaire • Book or article review • Essay/essay plan • Strategic or business • • • plan Laboratory report In-tray exercise Case study Group work Exhibition Journal article POD/VODCAST MCQs Radio presentation Project plan/work Web pages 148

Timing of Assessments Student workload + timing Do you know: The total workload & timing in the semester for your students? Hours required to complete a piece of assessed work? If there is bunching of work towards the end of the semester? Where do you teach time management skills?

COURSE INFORMATION UTM 2018

Student Learning Time • 1 kredit bersamaan dengan 40 jam kredit • Diagihkan mengikut kesesuaian perlaksanaan proses pengajaran dan pembelajaran • Penetapan jam kredit adalah selari dengan weight pentaksiran • Contoh: - CLO 2 menggunakan pentaksiran individual presentation dan diberi markah sebanyak 10 markah • Maka (10/100)*80 (jam kredit) = 8 jam • Maka agihkan 8 jam untuk F 2 F (L/T/P/O), NF 2 F secara guidance atau non guidance

COURSE INFORMATION UTM 2018

Managing Assessment Preparation Coordination Implementation Sharing the results 153

Preparation - stage before assessment is carried out üDistribution of work load – preparing and setting questions or projects, marking schemes and distribution of marks, marking of student works üAgreed on the date of tests, projects and submission dateline for reports. 154

ü Organization of tests/exams – – Booking of rooms for test facilities for assessment of student works printing of test questions checking of numbers of printed questions and answer sheets – arrangement of tables in accordance to the test regulation – invigilation 155

Coordination üElect coordinator, identify key players. üDetermine equity in grading standards üAgree on distribution of total marks, type, nature and number of assessment üAgree on scope of assessment plan, what to include and not to include üAlways refer to learning outcomes 156

Implementation üFrequent meetings to review and to receive feedback on the progress üDepends on when, where and how many students üEstablish some procedures to monitor students’ behaviour 157

Example: Procedures in Examination ü Students to enter rooms 10 -15 min. before test/exam starts. ü Students given clear instruction on test/exams requirements. ü Lecturers to ensure no negative activities during test/exams. ü Students to be informed of time remaining - 15 min. before end of test ü Inform students to stop writing. 158

Test/exam procedures üCollection of answer script üChecking number of answer scripts üStudents allowed to leave test rooms after answer scripts have been counted correctly üAnswer scripts tied and passed and given to faculty’s collection centre 159

Examples of Unforeseen Events ü Retest or repeat quiz or special test ü Irregularity in marks for some sections or classes ü Disciplinary problems ü Copying during test or exam ü Independent study ü Unforeseen administration decisions, Cocurricular activities, service learning ü Natural disaster 160

Sharing the Results: Gather Assessment Data and Report • How well are the students learning? o Documenting student learning o Data evaluated relative to the learning objectives and outcomes 161

Sharing the results of assessment (Cont. ) üRequired for feedback to students and lecturers to improve teaching and learning üKeep an efficient, effective and informative record of assessment activities üKeep good records of students’ performance 162

Make Changes : Use Assessment Data to Improve Program What will improve student learning? Respond to evidence at any point in the assessment cycle Assessment evidence should show what students have learned and what processes contributed to that learning. 163

What should be changed? Are current learning strategies (curriculum and instruction) effective? Are the assessment methods measuring appropriately? Are the objectives and outcomes appropriate? Are documentation and records clear for long term? 164

Conclusion Plan & manage the learning experience Assessment plan should be manageable, meaningful and sustainable. Administration should be effective, efficient and easy to handle. 165

Reflection • What was the muddiest point so far in this session? -what was least clear to you 166

Thank you 167