Defining Scales of Competency Levels Standardize MEES Evaluation

Defining Scales of Competency Levels Standardize MEES Evaluation Results ISABELLE BERTOLOTTI PROCEDE CURRICULUM CONSULTANT

Where are we now? IMPLEMENTING MEES CRITERION REFERENCED RUBRICS

Criterion Referenced Rubrics

Using Criterion Referenced Rubrics q. Road map to end of course outcomes and end of course evaluations q. Is provided by BIM for CCBE (published exams only) q. Is provided by the MEES for DBE secondary 4 and 5 (except for local courses) q. Is/Will be included in DEDs for DBE courses secondary 4 and 5 (except local courses) q. Adult learner can view criterion referenced rubrics q. Is used to evaluate each and every MEES sanctioned exam q. Cannot be altered, changed or modified q. Must be handed in with final exams

Additional Lists q. Some rubrics have observational lists that provide more detail v. French second language (Observational Lists) v. Computer science (Observational Lists) v. Math (Information gathering tool) v. Science (Checklist) q. Cannot be shared or published q. Contains the “answers” to exams

Competency q. Review competency being evaluated q. Only evaluate that competency

Evaluation criteria and task q. Can be found in the DEDs q. Proposes an Explanation of the Evaluation Content q. The numbers correspond to the Information Clarifying the Evaluation Criteria

Rating Scale q. Use only the scale provided

The Zero Mark q. In some cases where specified, you can assign a mark of zero

Deductions q. There is no “deduction system”

Converting to Instructional Grids q. Created with learner friendly language, but uses same MEES content q. Can be used formative assessment and ongoing feedback q. Cannot be used to evaluate MEES sanctioned exams q. Can be found on www. dbeimplementation. weebly. com website MEES Criterion Referenced Rubric Instructional Grid

Where are we going? TOWARDS STANDARDIZING FEEDBACK/MARKS

Example of Current Situation q. Based on collective corrections ENG 4112 -2 question 3: learner sample 2 teachers 3 teachers 2 teachers

Learner Sample ENG 4112 -2 Reads Excellent Q 1 Very Good Weak 4 3 1 1 2 7 Q 3 2 3 Q 4 3 4 1 Q 5 4 3 1 Q 2 Produces Text 1 Excellent Very Good Weak 5 2 1 3. 1 2 3. 1 1 2 2 3. 2 1 5 1 1 3. 4 1 3 3 2 3. 1 3. 3 Very Weak teacher evaluation results for the sample corrected

Learner Sample FRE 5103 -3 Produire des textes variés FRE-5103 Excellent Très bien Bien Faible 3. 1 – Communication langagière 3 3 1 1 3. 1 – Grammaire du texte 1 3. 2 – Orth. usage+ Grammaire 1 5 2 2 3 4 2 3. 2 – Syntaxe + Ponctuation 3. 2 -Vocabulaire 5 1 1 Très faible 1 teacher evaluation results for the sample corrected

Desired Situation: Closing the Gap q. Based on collective corrections ENG 4112 -2 question 3: learner sample OR 3 teachers 4 teachers 7 teachers

How will we get there? § BACKWARD DESIGN § DISCIPLINARY “EXCELLENCE” § EXEMPLARS OF PERFORMANCE § LITERACY BENCHMARKS § COLLECTIVE TEACHER EFFICACY

1) Using Backward Design q. Backward Design instructional design method by Jay Mc. Tighe & Grant Wiggins q. Refer to DED ENG 4112 -2 Information Clarifying the Evaluation Criteria 2. 1 Coherent construction of meaning from texts: * explanation of the development of the conflicts and the possible resolution * examination of the setting(s) of the text q. Refer to course guide ENG 4112 -2 “They identify literary elements such as characterization: reactions/roles/traits of main and minor characters; climax; conflicts; internal/external and their resolution; dialogue; main and minor themes; narrative point of view; plot; protagonist and antagonist; setting and symbolism 1. Identify desired results by referring to end of course evaluations and DEDs 2. Determine acceptable evidence by referring to courses and programs 3. Plan learning experiences and instruction by using Learning and Evaluation situations

2) Disciplinary “Excellence” q. Refer to disciplinary expertise Ex: texte narratif, literary terms etc. q. Disciplinary resources q. SOFAD qwww. dbeimplementation. weebly. com

3) Exemplars of Performance q…many studies have shown that it is difficult to make assessment criteria and standards explicit through written documentation or through verbal descriptions in class (Rust et al. , 2003). q As Yorke (2003, p. 480) notes: “Statements of expected standards, curriculum objectives or learning outcomes are generally insufficient to convey the richness of meaning that is wrapped up in them. ” q. An approach that has proved particularly powerful in clarifying goals and standards has been to provide students with ‘exemplars’ of performance (Orsmond et al. , 2002). Sample argumentative essay provided to learner

4) Literacy benchmarks q. Vision of excellence in reading, writing, or oral communication skills for each specified level as proposed by Literacy Leadership Brief: Exploring the 2017 NAEP Reading Results: Systemic Reforms Beat Simplistic Solutions ( International Literacy Association 2018) and Regie Routman Example ENG 4112 -2 Q 1: What is the shared vision of excellence in communicating thoroughly and clearly in secondary 4? Example FRE 5103 -3: Qu’elle est la vision partagée d’un texte qui “respecte parfaitement les éléments » ?

Sample Tool 2. Disciplinary Excellence 1. Backward Design 3. Performance Exemplars 10/10 or 5/5 4. Literacy Benchmarks 3. Performance Exemplars 6/10 or 3/5 4. Literacy Benchmarks

5)Collective Teacher Efficacy Albert Bandura defines collective efficacy as “a group’s shared belief in the conjoint capabilities to organize and execute the courses of action required to produce given levels of attainment. ” Visible Learning Summer school d=-0. 23 Changing schools d=-0. 34 Vocabulary programs d=0. 67 Collective teacher efficacy d=1. 568