ASSESSMENT ALTERNATIVE ASSESSMENT RUBRIC DEVELOPMENT 11 SEPT 2020

ASSESSMENT : ALTERNATIVE ASSESSMENT & RUBRIC DEVELOPMENT 11 SEPT 2020@ RAZAK SCHOOL UTMKL Rohaya Talib School of Education FSSH Ph. D (Measurement & Evaluation) M. Ed (Measurement & Evaluation) BBA (Accounting)

OBE : CONSTRUCTIVE ALIGNMENT COURSE INFO (CLO, PLO, TAXONOMIES) CURRICULUM QUIZ / TEST PEDAGOGY ANDRAGOGY = CYBERGOGY ASSESSMENT EXAM [MASTERY]

CONSTRUCTIVE ALIGNMENT CURRICULUM STUDENT-CENTERED LEARNING (MEANINGFUL LEARNING) PEDAGOGY ANDRAGOGY PEERAGOGY COURSE INFO (CLO, PLO, TAXONOMIES) ALTERNATIVE ASSESSMENT ASSIGNMENT ASSESSMENT TRADITIONAL ASSESSMENT FINAL EXAM

WHY CONSTRUCTIVE ALIGNMENT IS IMPORTANT? ? VALID AND RELIABLE TH ES E J U D G E M E N T • ABILITY • ACHIEVEMENT • COMPETENCE • MASTERY CO R

ALTERNATIVE ASSESSMENT is a form of student performance grading that allows for a more holistic approach to student assessment. The traditional form of student assessment involves the average grading of a cumulative set of work for a given time period. With alternative assessments, students are enabled to provide their own responses rather than simply selecting from a given list of options.

ALTERNATIVE ASSESSMENTS referred to as performance or authentic assessments are used to determine what students can and cannot do, in contrast to what they do or do not know. In other words, an alternative assessment measures applied proficiency more than it measures knowledge. learning requires that the learner engage in problem-solving to actively build mental models. Knowledge is attained not just by receiving information, but also by interpreting the information and relating it to the learner's knowledge base.

COMMON CHARACTERISTICS OF ALTERNATIVE ASSESSMENT § Asks students to perform, create or produce something § Evaluation criteria and standards are known to the student § Involve interaction between assessor (instructor, peers, self) and person assessed § Provides self-assessment opportunities for students § Provides opportunities for both individual and group work § Encourages students to continue the learning activity beyond the scope of the assignment

COMMON CHARACTERISTICS OF ALTERNATIVE ASSESSMENT n Assesses a learning process and encourages student selfreflection n Based on authentic task with real-world application n Topics and means of expression are of interest to the students

PORTFOLIO FORMATIVE ASSIGNMENT ALTERNATIVE ASSESSMENT PROJECT TOTAL SCORE REPORT FINAL EXAM TRADITIONAL ASSESSMENT

ON 1 VERSI

ASSESS PROCESS OF LEARNING § Flexible § Show development § Increases communication § Promotes reflection § Provides feedback

ASSESSMENT TASKS PORTFOLIOS REFLECTIVE JOURNAL ALTERNATIVES ASSESSMENT SELF/PEER ASSESSMENT PRESENTATIONS

AA - The utilization of non-traditional approaches in judging student performance. BUSSINESS PLAN ARTICLE REVIEW DIARY/LOG LETTERS/ MEMOS GROUP WORK / INDIVIDUAL WEB PAGE AUDIO/VIDEO /VLOG POSTER LITERATURE REVIEW CASE STUDY PROJECT

SONG NEWS REPORT BILL BOARD MODEL/ DIORAMA BROCHURE / PAMPHLET CARTOON

SOFT SKILLS/GENERIC SKILLS/GRADUATE ATTRIBUTS CS AD TH GC SC ES TW

ASSESSMENT TOOL PERFORMANCE RUBRICS CRITERIA STANDARD

RUBRIC : Assessment Tool BRIDGING Teaching Learning…TO Assessment

Criterion Referenced Assessment (CRA) ASSESSMENT TOOLs • Checklist • Rating Scale • Rubric Criterion-referenced assessments are designed to measure student PERFORMANCE against a fixed set of predetermined criteria or learning standard; concise, written descriptions of what students are expected to know and be able to do at a specific stage of their education. Criterion referencing is OFTEN defined in terms of proficiency levels. www. edglossary. org

Eg Proficiency Level www. edglossary. org

RUBRIC TEST

Performance ? Performance-based learning and assessment represent a set of strategies for the acquisition and application of knowledge, skills, and work habits through the performance of tasks that are meaningful and engaging to students. http: //www. ascd. org

MULTIPLIER EFFECTS OF PERFORMANCE ASSESSMENT INDUSTRY COMMUNITY www. emergingedtech. com

Traditional Assessment Versus Performance Assessment ?

Why Performance Assessment is significant? PA requires students to actively demonstrate what they know…. . PA more valid indicator of students' knowledge and abilities. www 2. ed. gov

What Does The Research Say? Research suggests that learning how and where information can be applied should be a central part of all curricular areas Students exhibit greater interest and levels of learning when they are required to organize facts around major concepts and actively construct their own understanding of the concepts in a rich variety of contexts PA requires students to structure and apply information, and thereby helps to engage students in Active Learning www 2. ed. gov

Rubrics give structure to observations Matching your observations of a student's work to the descriptions in the rubric averts (prevents) the rush to judgment that can occur in classroom evaluation situations. The rubric describes the performance. The resulting judgment of quality based on a rubric also contains within it a description of performance that can be used for feedback and teaching. www 2. ed. gov

What Does It Cost? PA requires a greater expense of time, planning and thought from students and teachers. Richard P. Mills Commissioner Vermont Department of Education Montpelier, VT 05602 www 2. ed. gov

Types of Performance Type of Performance Examples Processes • Physical skills • Use of equipment • Oral communication • Work habits • Playing a musical instrument • Preparing a slide for the microscope • Making a speech to the class • Reading aloud • Conversing in a foreign language • Generate scientific analysis report • Laboratory report • Term paper • Written analysis of the effects of the XXX • Model or diagram of a structure (atom, solar, planetary system, etc. ) • Concept map Products • Constructed objects • Written essays, themes, reports, term papers • Other academic products that demonstrate understanding of concepts

R NORMALLY DISTRIBUTED SCORES? ? U B R IC

• MEASURE ABILITY, ASSESSMENT PERFORMANCE, COMPETENCY, MASTERY CRITERIONREFERENCED ASSESSMENT • INTERPRETATION OF SCORE PERFORMANCE ASSESSMENT • KNOW AND DO RUBRIC • SCORING TOOL

1 CHECKLIST 2 RATING SCALE 3 RUBRIC ?

RUBRIC • Latin word means RED • An authoritative RULE • A guide listing specific criteria for grading or scoring academic papers, projects and any performance assessment Merriam-webster Dictionary

A rubric is a scoring tool that explicitly describes the instructor’s performance expectations for an assignment or piece of work. https: //www. cmu. edu/teaching/assessment/assesslearning/rubrics. html

A rubric is a coherent set of criteria for students' work that includes descriptions of levels of performance quality on the criteria. Effective rubrics have appropriate criteria and well-written descriptions of performance. http: //www. ascd. org

ü ü ü Rubrics help lecturers to: Reduce the time spent grading by allowing lecturers to refer to a substantive description without writing long comments Help lecturers more clearly identify strengths and weaknesses across an entire class and adjust their instruction appropriately Help to ensure consistency across time and across students Reduce the uncertainty which can accompany grading Discourage complaints about grades https: //www. cmu. edu/teaching/assessment/assesslearning/rubrics. html

Rubrics help students to: ü ü Understand lecturers’ expectations and standards Use lecturers’ feedback to improve their performance Monitor and assess their progress as they work towards clearly indicated goals Recognize their strengths and weaknesses and direct their efforts accordingly https: //www. cmu. edu/teaching/assessment/assesslearning/rubrics. html

The main purpose of rubric is to assess PERFORMANCE

VERSUS HOLISTIC RUBRICS: Gives students an overall score for their paper Views the final product as a set of interrelated tasks contributing to the whole A score is not assigned for each criterion Does not provide detailed feedback to the student It is efficient and quick. ANALYTIC RUBRICS: Analytic rubrics allows for educators to provide feedback on each criterion Each part is scored independently to avoid bias. Provides meaningful and specific feedback along multiple dimensions Are more difficult to construct and is time consuming http: //www. queensu. ca

VERSUS HOLISTIC RUBRICS: Holistic rubrics single criteria rubrics (onedimensional) used to assess participants' overall achievement on an activity or item based on predefined achievement levels; performance descriptions are written in paragraphs and usually in full sentences. http: //www. queensu. ca ANALYTIC RUBRICS: Two-dimensional rubrics with levels of achievement as columns and assessment criteria as rows. Allows you to assess participants' achievements based on multiple criteria using a single rubric. You can assign different weights (value) to different criteria and include an overall achievement by totaling the criteria; written in a table form.

A HR does not list separate levels of performance for each criterion. Instead, a HR assigns a level of performance by assessing performance across multiple criteria as a whole. HR AR http: //www. queensu. ca/ An AR articulates levels of performance for each criterion so the lecturer can assess student performance on each criterion.

1 (50 -59%) 2 (60 -69%) 3 (70 -79%) 4 (80 -100%) SCORE CRITERIA Research paper demonstrates complete understanding and execution of the assigned objectives. Thesis statement/argument is clearly stated, complex and original, and the writing does not spend excessive time on any one point of development at the expense of developing other points in the body of the paper. Writing is also error-free, without ambiguity, and reads smoothly, creatively, and with a purpose. Research paper demonstrates considerable understanding and execution of the assigned objectives. Thesis statement/argument is stated, verges on the complex and original, and the writing shows accuracy and balance in developing body points, but may exhibit occasional weaknesses and lapses in correctness. Writing also has some errors and ambiguities, yet does read clearly and coherently. Research paper demonstrates some understanding and execution of the assigned objectives. Thesis statement/argument is faintly stated and/or expected and not confident, and the writing is inconsistent in terms of balance in developing body points, and exhibits weaknesses and lapses in correctness. Writing also has many errors and ambiguities, and may read confusingly and incoherently. Research paper demonstrates limited understanding and execution of the assigned objectives. Thesis statement/argument is simplistic, unoriginal, and/or not present at all, and the writing is unbalanced in developing body points, weak, and incomplete. Writing also has numerous errors and ambiguities, and reads confusingly and incoherently. Adapted from John Bean, Engaging Ideas, Exhibit 15. 4: Holistic Scale for Grading Article Summaries

STANDARDS CRITERIA ADEQUATE (50 -59%) Knowledge of forms, Demonstrates limited knowledge conventions, terminology, of forms, conventions, and strategies relative to terminology, and strategies the importance of sources relative to importance of sources to subject COMPETENT (60 -69%) GOOD (70 -79%) Demonstrates some knowledge of forms, conventions, terminology, and strategies relative to importance of sources to subject Demonstrates considerable knowledge of forms, conventions, terminology, and strategies relative to importance of sources to subject Demonstrates thorough and insightful knowledge of forms, conventions, terminology, and strategies relative to importance of sources to subject Uses critical and creative thinking skills with considerable effectiveness Uses critical and creative thinking skills with a high degree of effectiveness Critical and creative thinking skills Uses critical and creative thinking skills with limited effectiveness skills with moderate effectiveness Communication of information and idea Communicates information and idea with limited clarity Communicates information and ideas Communicates information and with some clarity and ideas with considerable ideas with a high degree of clarity and with confidence Quality of argument and Argument is simple and Argument takes on a fair and writing unoriginal, and the writing is weak expected position, and the writing is and inconsistent moderately clear and coherent Spelling and grammar Several errors in spelling and grammar EXCELLENT (80 -100%) Argument bridges on the complex and original, and the writing is clear and coherent Argument is complex and original, and the writing is strong, fluid, and creatively coherent A few errors in spelling and grammar Some errors in spelling and No errors in spelling and grammar Adapted from Centre for Teaching Excellence, Appendix B: Sample Analytic Rubric (“Rubrics: Useful Assessment Tools.

An Analytic Scoring Rubric • An analytic rubric is an assessment instrument for scoring a constructed response. The response can be a written response, a performance, or a product. • The response can be from an individual or a group • An analytic rubric includes a score scale of 1 to 3 points or up to 1 to 5 points with criteria and descriptors for assigning the points Carriveau R. S. (2016), Connecting the dots: developing student learning outcomes and outcome based assessments (2 nd ed, Stylus Publications, Sterling, VA)

Anatomy of An Analytic Rubric (CDPL) Criteria • the aspects of performance (e. g. , argument, evidence, clarity) that will be assessed Descriptors • the characteristics associated with each dimension (e. g. , argument is demonstrable and original, evidence is diverse and compelling) Performance Levels • a rating scale that identifies students’ level of mastery within each criterion https: //www. cmu. edu/teaching/assessment/assesslearning/rubrics. html

Construction of Rubric Structure DESCRIPTORS The two basic principles in rubric construction are parallelism in structure and homogeneity in content q Parallelism means the basic grammatical structure of the descriptions are similar from row cell (column 1) to row cell (column 2…) q Homogeneity means the content is similar for row cell (column 1) to row cell (column 2…) so that you are measuring the same content. teachingcommons. unt. edu

Allen, M. J. (2004). Assessing academic programs in higher education. Bolton, MA: Anker.

Rubric for Position Paper [an essay that presents an arguable opinion about an issue] Criteria/ Outcomes 4 PS is clearly stated and is Position consistently Statement [PS]: CLARITY maintained 30% throughout the paper. Structure of the argument (So. A) : CLEAR TRANSITION 50% Structure of the argument is well developed with clear in all transitions. 3 2 1 An attempt at a PS is clearly stated PS is not clearly position statement but shifts slightly stated and shifts is evident, but it throughout the cannot be clearly paper. determined. Structure of the argument is moderately well developed, and most transitions are clear Structure of the argument is moderately well developed, and many transitions are weak and unclear. Structure of the argument is weak, and most transitions are weak and unclear.

Rubric for Position Paper: is an essay that presents an arguable opinion about an issue Criteria/ Outcomes 3 4 PS is clearly stated Position and is consistently Statement maintained [PS]: throughout the CLARITY paper. 30% 2 1 1 Criteria: the aspects of performance 2 3 3 2 PS is clearly stated but shifts slightly throughout the paper. PS is not clearly stated and shifts throughout the paper. 2 2 Descriptors: the characteristics associated with each dimension 3 1 An attempt at a position statement is evident, but it cannot be clearly determined. 2 Performance levels: a rating scale that identifies students’ level of mastery within each criterion

Rubric for Position Paper Criteria/ Outcomes 4 3 PS is clearly stated and is Position stated but shifts consistently Statement slightly [PS]: CLARITY maintained throughout the 30% throughout the paper. Parallelism : Basic grammatical structure of the descriptions are similar 2 1 An attempt at a PS is not clearly position stated and shifts statement is throughout the evident, but it paper. cannot be clearly determined. Homogeneity : content is similar for row cell to row cell, measuring the same content.

The Value of Consistency in Terms of Structure And Content • The primary focus of assessment is to determine what the student knows and can do so that instructional and evaluation decisions can be made. • To make valid decisions about a student requires well-constructed assessment instruments (rubrics) that consistently produce valid scores. Carriveau R. S. (2016), Connecting the dots: developing student learning outcomes and outcome based assessments (2 nd ed, Stylus Publications, Sterling, VA) • Validity means that the scores can be used and interpreted as they were intended to be used and interpreted.

The Value of Consistency in Terms of Structure And Content • Parallel structure and homogeneous content help ensure rubric score validity because they make it easier for the rater or raters to more objectively score the student’s work. • The rater needs to be able to see clearly the progression from a high to a low quality student response. Carriveau R. S. (2016), Connecting the dots: developing student learning outcomes and outcome based assessments (2 nd ed, Stylus Publications, Sterling, VA) • Any inconsistency or confusing language would reduce the ability of the rater to make valid decisions.

Rubric for Position Paper Criteria/ Outcomes 4 3 2 1 An attempt at Position PS is clearly PS is not clearly a position Statement stated and is stated but statement is stated and [PS]: consistently shifts evident, but it CLARITY maintained slightly throughout the cannot be 30% throughout paper. clearly the paper. determined. Weightage for each criteria? Criteria 1 30% Criteria 2 %50% Score Total Score ? ? ? Composite Score? ? ? Weightage for the PA? ? ?

How Many Levels of Performance Should I Include in my Rubric? It is not easy to write good descriptors for each level and each criterion. So, when you first construct and use a rubric you might not include descriptors. That is okay. You might just include the criteria and some type of labels for the levels of performance. Once you have used the rubric and identified student work that fits into each level it will become easier to articulate what you mean by "good" or "excellent. " Thus, you might add or expand upon descriptors the next time you use the rubric. Copyright 2018, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL.

How Many Levels of Performance Should I Include in my Rubric? A rubric can have as few as two levels of performance (e. g. , a checklist) or as many as. . . well, as many as you decide is appropriate. There is no "right" answer as to how many levels of performance there should be for a criterion in an analytic rubric; that will depend on the nature of the task assigned, the criteria being evaluated, the students involved and your purposes and preferences. Copyright 2018, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL.

How Many Levels of Performance Should I Include in my Rubric? Level of Performance Students better understand what good (or bad) performance on a task looks like if levels of performance are identified, particularly if descriptors for each level are included Copyright 2018, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL.

How Many Levels of Performance Should I Include in my Rubric? Also, it is NOT TRUE that there must be an EVEN number or an ODD number of levels. Again, that will depend on the situation. Copyright 2018, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL.

4 Level of Performance in the Rubric POOR BEGINNING SELDOM UNACCEPTABLE GOOD BASIC SOMETIMES MARGINAL EXCELLENT PROFICIENT USUALLY PROFICIENT SUPERIOR ADVANCED OFTEN DISTINGUISHED BEGINNING DEVELOPING COMPETENT EXEMPLARY NEEDS IMPROVEMENT SATISFACTORY GOOD ACCOMPLISHED UNACCEPTABLE MARGINAL PROFICIENT DISTINGUISHED NEEDS IMPROVEMENT SEVELOPING SUFFICIENT ABOVE AVERAGE Center for Teaching and Learning 25 E. Jackson Blvd. Chicago, IL 60604 (312) 362 -8000 resources. depaul. edu

3 Level of Performance in the Rubric NEEDS IMPROVEMENT MEETS EXPECTATION EXCEED EXPECTATION BASIC PROFIENT ADVANCED 5 Level of Performance in the Rubric POOR MINIMAL SUFFICIENT ABOVE AVERAGE EXCELLENT NOVICE INTERMEDIATE PROFICIENT DISTINGUISHED MASTER UNACCEPTA BLE EMERGING MINIMALLY ACCEPTABLE ACCOMPLISHED Center for Teaching and Learning 25 E. Jackson Blvd. Chicago, IL 60604 (312) 362 -8000 resources. depaul. edu

SUMMARY Rubrics are NOT used to compare the performance of students, but TO COMPARE A SINGLE STUDENT AGAINST THE SET OF CRITERIA. Copyright 2018, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL.

COURSE INFO PROJECT-BASED ASSESSMENT PROJECT OUTLINE PRESENTATION (FORMATIVE) 5% CS COMMUNICATION SKILL RUBRIC Copyright 2018, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL.