Assessment and Evaluation BSCSE Program Neelam Soundarajan Chair

Assessment and Evaluation BS-CSE Program Neelam Soundarajan Chair CSE Undergrad Studies Comm.

Objectives, Outcomes n PEOs (fairly standard): n n . . . related to employment in profession. . . related to advanced studies. . . related to being informed/involved members of community and profession. . . Student Outcomes (fairly standard): n n (a) - (k) (slightly modified) (l) - (n) (as required by CAC Criteria)

Assessment Methods Alumni Surveys (next round? ) n Advisory Board (annual) n Exit Survey n Annual Student Forum n POCAT (Exit test) n Rubrics in Capstone Courses All of the above are standard except for POCAT (Program Out. Comes Achievement Test) n

Assessment Methods n Slides on Alum Surveys etc. to be created. . .

POCAT n n n n All students are required to take it; Typically in the graduation semester; Assesses the program, not the students; Each test paper has a unique code on it; no record of (student code) mapping! Only the student knows his/her code; So: when results are posted, only the student can know he/she did; Goal: Assess the program, not the students;

POCAT (contd) n n n n 20+ multiple-choice questions; Topics: Reqd. courses, popular electives; Conceptual questions designed to tease out common misunderstandings; Maybe "cross-cutting" (across courses); Questions are freely reused! No "security" concerns; Students never cheat. . . why would they? !

POCAT: Example Question Your sibling is very ill and needs medication that you cannot afford, so you steal it. Which of the following is the ethical question that arises in this scenario: a. b. c. d. e. f. g. What kind of illness does your sibling have and how do we prevent its spread among populations? Why is the medication not affordable and how do we make it more affordable? Is it ever right to steal, even if you have a great need? What consequences might you face if you are caught? All of the above are ethical questions. All of the above are important questions but none is an ethical question. I cannot decide based on the given information.

POCAT Results Note: Correct answers are in boldface

POCAT Results (contd) • Shown: No. of students who picked each answer (bold=correct) • Nos. in () ("good") and [] ("bad") are important; "good": measures of how many top students picked correct ans/ did not pick wrong answer; "bad": measures of how many top students picked wrong ans/ did not pick correct answer; • For ethics question, (e) is a good distractor.

POCAT Results (contd) n n n Another example: In a question related to databases, students were often picking one particular wrong answer in preference to another wrong ans. although both looked equally plausible. . . We couldn't fix the problem (by modifying the course content) without knowing why! One of the student reps on the Undergrad Comm. came up with a plausible explanation. . .

POCAT Results (contd) n n So: we have revised the question to test his hypothesis and plan to use it in the next POCAT; If it turns out right, we will know what changes are needed in the DB course to fix the problem;

Summary That is the essence of assessment, evaluation, and program improvement: n Evaluate the results of the assessments to help identify specific problems in the program and make changes in the program to address the problems. n Conclusion: The approach works well and is sustainable!