DEVELOPING ASSESSMENT RUBRICS FOR CIVIL TECHNOLOGY Werner Engelbrecht

DEVELOPING ASSESSMENT RUBRICS FOR CIVIL TECHNOLOGY Werner Engelbrecht Department of Science and Technology Education, Faculty of Education, University of Johannesburg 15 March 2014 Co. P for Civil Technology

Common weaknesses in Assessment • Tasks do not match the stated outcomes • Criteria do not match the tasks or outcomes • Criteria not known to and/or not understood by the students • Overuse of one method of assessment • Overload of students and/or staff • Absence of well defined criteria so consistency is difficult to achieve • Unduly specific criteria which create a straightjacket for students • Inadequate or superficial feedback provided to students • Wide variation in marking between assessors

Assessment Methods • • • Must improve student performance Must relate to subject objectives Must be authentic Must have clearly defined criteria Must be fair, valid and reliable

Some Assessment Methods • Formal Examinations • Knowledge and Understanding • Analytical Ability • Problem Solving • Communication Skills • Progress tests • Knowledge and Understanding • Simple Problem Solving • Formal Practical Experiments • Following Instructions and Recording Results • Practical Ability • Written Reports or Software Documentation • Projects • Requirement Analysis, Research, Problem Solving / Synthesis • Practical Development • Oral and Written Communication • Possibly Group Working • Other Coursework Assignments

Assessment Criteria – a definition • Not to be confused with assessment methods or tasks • Assessment Criteria provide a clear indication of how achievement may be demonstrated • Often specified with respect to each learning objective, they describe what a learner is expected to do in order to demonstrate that the learning objective has been achieved. • Assessment Criteria may be used in three ways: • To confirm achievement of standards • To define what is expected in order to achieve each of the grades being awarded • To specify a template of characteristics or qualities against which the students’ performance of the assessment task will be judged

Writing Assessment Criteria 1. 2. 3. 4. 5. 6. 7. Consider the learning objective being tested Consider the assessment task set Develop requirements for, or attributes of, successful performance of the assessment task If necessary specify the range to clarify contextual factors and the level Focus on what is essential and categorise the requirements or attributes into clearly worded criteria Check that the criteria are measurable or assessable in valid and reliable ways and that the criteria are clear and unambiguous Publish the assessment criteria with the assessment task and identify the intended learning objective or objectives that the task is assessing (Dasoo 2011)

What are rubrics? • A rubric is an assessment instrument containing pre-determined performance criteria used in assessing learner performance or evaluating products resulting from a performance task.

Rubrics Answer… • By what criteria will the work be judged? • What is the difference between good work and weaker work? • How can both students and lecturers focus their preparation on excellence? 9

Why are they used? • Focus instruction---intentionally. • Guide feedback---descriptively. • Characterize desired results---objectively. • Operationalize performance standards---purposefully. • Develop self-assessment competence---constantly. • Involve students---thoughtfully. 10

Holistic vs Analytic rubrics • Holistic rubrics require the assessor to assess the learner’s performance or evaluate a product as a whole, without judging different aspects or components separately. • With an analytic rubric the assessor judges a variety of aspects or components separately and ads the marks awarded to each aspect to obtain a total assessment mark. 11

Rubric Template Beginning Stated criteria Developing 1 2 Description of identifiable performance characteristics reflecting a beginning level of performance. Description of identifiable performance characteristics reflecting development and movement toward mastery of performance. Accomplished 3 Description of identifiable performance characteristics reflecting mastery of performance. Exemplary 4 Description of identifiable performance characteristics reflecting the highest level of performance.

Aspects to include in a analytical rubric for practical work • Clare Benson identifies the following aspect as the framework for product analysis: • • • Physical features Construction Function Design Value - What does it look and feel like? How is it made? What was it made for? Who was it made for? Is it well designed? What is it worth?

Process to develop a analytical rubric 1. Re-examine the learning objectives to be addressed by the task – match with the objectives and actual instruction. 2. Identify specific observable attributes that you are looking for in the learner’s performance or product. Consider the aspects, skills or behaviours that you will be looking for, as well as common mistakes you do not want to see. 3. Develop characteristics that describe each attribute. Identify ways to describe average, above average, and below average performance for each observable attribute identified. 4. Write thorough narrative descriptions for excellent work and poor work for each individual attribute. Describe the highest and lowest levels of performance using descriptors for each attribute separately.

Process to develop a analytical rubric 5. Complete the rubric by describing other levels on the continuum that ranges from excellent to poor work for each attribute. Write descriptions for all intermediate levels of performance for each attribute separately. 6. Collect samples of students’ work that exemplify each level. These will help you to fine tune your description and serve as benchmarks. 7. Revise the rubric as necessary. Be prepared to reflect on the effectiveness of the rubric and revise it prior to its next implementation. (Mertler)

Example of a analytical rubric for practical work

Example of a analytical rubric for the technological process

PAT for civil Technology • The Practical Assessment Task consists of three components: the design portfolio and working drawings which each counts 25% of the PAT and the model which counts 50% of the PAT. • The teacher must take into account the requirements of the assessment of all the components of the PAT and therefore plan the assessment programme for the PAT accordingly.

Group activity • Develop a rubric for a PAT in Civil technology.

Discussion and conclusion