Formative and summative e Assessment Paul Lam Assistant

Formative and summative e. Assessment Paul Lam Assistant Professor Christina Keing Computer Officer 29 March 2007

Assessment n Formative ¨ Assessment used as a learning tool: students realize what they know or don’t know through answering questions n Summative ¨ Assessment used as a student learning evaluation tool and is typically used to assign students a course grade

Formative assessment n “Formative evaluation takes place while a class is ongoing. It is the indispensable part of assessment that provides a way for us to continuously monitor our students’ learning and our own teaching practices. ” (Nuhfer, 1996; p. 385)

Summative assessment n Assessment must be matched to the desirable learning outcomes

Student learning needs Suggested approach to course planning 2. Learning activities 1. Aims/ desired learning outcomes 4. Feedback for evaluation 3. Assessment Actual learning outcomes 2. Content/ fundamental concepts

Formative e. Assessement Traditional e. Assessment Engagement No time Out of class practice Feedback Not immediate Not individualized Immediate Can be individualized Workload High Auto-marking

Summative e. Assessment Traditional e. Assessment Fairness Fair Also fair Question types Varied question Closed-ended types Multi-media Workload High in marking Auto-marking High in setting questions

Case 1 To engage students in extended learning through quizzes Weekly (Molecular Biology) n Competitive and game-like n n For details, please contact Prof Lam Honming, Biology, CUHK

Case 2 Train students’ decision-making with cases (Chinese Medicine) n Self-learning resources n Rich in information, pictures and feedback n n For details, please contact Prof Lin, Chinese Medicine, CUHK

Case 3 n n Realistic assessment on the ability of students to defend main concepts of the course in interviewlike situations (Investment banking) Multimedia and resemble real-life situation Timed For details, please contact Mr Joseph Mohan, School of Commerce, City University of Hong Kong

Sample student answers …What is the difference between Investment Banking and Financial Services? n “I would like to use a special example to answer this question. Have you ever know about the gardener and the flowers? What will people focus on? The answer is without doubt, the flowers. However, the gardener plays an important role; he provides nourishment to the flowers, and takes care of them. Without the gardener, flowers cannot grow well, and may even die. I think the relationship between the investment banking and HK’s financial services industry is like that. Investment banking helps to maintain the financial services’ performance. For example, it helps in the merger and acquisition, making the loan or even help the companies/ countries to manage their risks/ debts. . . ”

Case 4 n n n Summative assessment as a tool for a large cohort of students (Common English Foundation) 900 students in 45 classes taught by 21 ELTU teachers 40 MC, and 40 shortanswer questions 1 hour of duration Large question bank 20 % final mark

Why e. Assessment? n n Large group of students, heavy workload in marking examination papers. 45 classes, scheduled to take the exam at different time slots in one week. Security is a concern. Teachers wanted to analyze the performance of students on each question and make necessary adjustment. The assessment will be progressively reused.

What are the benefits? n The CEF examinations were conducted in four computer labs. The 45 classes were assigned to different examination time slots within one week. During the peak period, there were about 150 students working on the examinations at the same time. n For MC and short-answer questions, the platform supported auto-marking which would significantly save teachers’ time. n It was possible to randomly select questions from a question bank and compile a different examination paper for each student even during the same examination session. n It would be very convenient for teachers to do item analysis and performance analysis.

What are the limitations? n e. Assessments is relatively weak in auto-marking open-ended questions. n It could help to collect students’ answers to the open-ended questions electronically but teacher may not like marking it electronically. n There are quite some efforts in importing questions to the database and checking the format of the input questions.

Demonstration n n Various type of questions Individual timing Immediate marking and feedbacks (optional) Submission summary ¨ Number of trials ¨ Records of each ¨ Score ¨ Time spent n n trial Detail item analysis Summary statistics

Demonstration n Adding a question ¨ Marks n and comments Creating a quiz ¨ Assigning questions ¨ random select from different n Setting up a quiz ¨ Number of attempts ¨ Availability ¨ Security set?

Students’ comments n n Students' acceptance of the new method was also not unanimous. The responses to the survey question about “I prefer the online test to a pen-and-paper test” are represented below.

Students’ comments n The open-ended questions showed some positive remarks concerning the new method: ¨ ¨ ¨ “It is more convenient than a pen-and-paper test. ” “Not so tiring as a pen-and-paper test. ” “Not as nervous as doing a paper test. ” “Easy to see which question I have answered. ” “I can amend my answer without using a correction pen. ” “The timers on computers are accurate. ”

Students’ comments n There also negative remarks concerning e. Assessment: ¨ ¨ ¨ “It is better to use the traditional pen-and-paper test. ” “Too tiring for eyes. ” “I can’t collect my thoughts in front of a computer. ” “The quality of the mouse and the computer has an effect on performance. ” “It would be unfair for those who type words slowly and wrongly. ” “It is not fair that the questions are not the same for all students. To make the test fair, all students should answer the same set of questions. ”

More tips 1. 2. Make the purpose of each quiz clear. Is it for learning processes? Does it focus on revision of terms? Is it for application of concepts to new contexts? There are many sites which provide technical rules for writing good items. Some useful URLs are http: //www. park. edu/cetl/quicktips/writingtest. html http: //www. au. af. mil/au/afiadl/curriculum/icwguide/unit_2. htm 3. 4. Remember to think about how readable the items will be onscreen. Competition can be fun.

Some more tips 1. 2. 3. 4. 5. Release quizzes progressively during the course. The feedback you provide for each item is essential to making quizzes into learning experiences. Good items evolve over time. Ask a colleague to review them for you, or give them to a few previous students or higher degree students. If you are using multiple-choice items, do an item analysis on the results. Which alternatives were popular? Which were not at all appealing, and hence need to be replaced? How difficult was each item? Run a few quiz items in class to show students how useful the resource can be.

Reference n Nuhfer, E. B. (1996). The place of formative evaluations in assessment and ways to reap their benefits. Journal of Geoscience Education, 44, 385 -394.

Thank you