Assessor training 1 Welcome from AUA and NBS

Assessor training 1

Welcome from AUA and NBS Graham Pitcher • Why GBA and what is it? • The basic matrix format • Examples of how the matrices are used at NTU • Key assessment regulations Ann Hartley • The assessment titles for new cohorts • The assessment matrix for KA 1 • Provision of good feedback • Issues arising from moderation • Referencing style • Activity on the assessment matrix • Q & A 2

Grade Based Assessment Part 1 - overview Grading Scheme for Postgraduate Taught Courses 2013 3

What is Grade Based Assessment • Grade based assessment describes the ‘quality’ of learning (Biggs, 2003) • The term ‘distinction’ is used to capture the idea that a student with a distinction ‘thinks differently’ from a student with a pass. • The descriptors of assessment criteria describe a level of achievement. 4

The NTU Grading Scheme for PG taught courses • The generic level and grading descriptors are provided in the booklet, NTU ‘Grading Scheme for Postgraduate Taught Courses’. • Generic level descriptors describe the level of achievement required by all NTU PG students. • The general grading descriptors provide the standard of performance across a 14 point grading scale. 5

Assessment A – Grade awarded Mid Pass (55) Assessment B – Grade awarded Mid Com (65) Weighting 40% - 60% Numerical Mark Class Grade Range equivalent Aggregate numerical equivalent = Exceptional Distinction 93 - 100 (40% x 55) + (60% x 65) = 96 High Distinction 89 DISTINCTION 22 + 39 =85 - 92 61 (Excellent) Mid Distinction 78 - 84 81 Grade for module = Commendation The Grading Scheme COMMENDATION (Very good) PASS (Good) FAIL (Insufficient) ZERO Distinction 70 - 77 74 High Commendation 67 - 69 68 Mid Commendation 64 - 66 65 Commendation 60 - 63 62 High Pass 57 - 59 58 Mid Pass 54 - 56 55 Pass 50 - 53 52 Marginal Fail 45 - 49 47 Mid Fail 40 - 44 42 Low Fail 1 - 39 24 Zero 0 0 6

So how does it work? Principle of constructive alignment (Biggs, J. (1999) Teaching for Quality Learning at University, Buckingham, SRHE/Open University Press) 7

Constructive alignment What do we want students to be able to do at the end of the course, module, session etc. ? What is the most appropriate way of assessing whether Learning Outcome students can do what we want students to be able to do? What are the students expected to do in order to demonstrate that the learning outcomes has been Assessment task achieved? GBA Incentive What. Define is the content and learning strategies that for willhigher enable threshold through students to acquire the necessaryachievement knowledge and skills. graded assessment criteria Learning and teaching strategy to enable learners learning The grade defines the levelto of reach achievement. outcomes 8

Learning outcomes and GBA 100% Higher levels of achievement Student achievement Pass: award credit 50% learning outcomes achieved Learning outcomes partially or not achieved 0% 9

Grade Based Assessment Part 2 – Learning Outcomes and Assessment Criteria 10

Learning outcomes A learning outcome contains: • A verb - what the learner is expected to be able to do; • Words that indicate on what or with what the learner is acting; • Words that indicate the nature of the performance required 11

Verbs for learning outcomes 12

Learning outcome At the end of the module students should be able to: • Demonstrate a systematic understanding of customer profitability analysis and how it can be utilised to enhance the earnings per share of a commercial organisation. What the learner has to do: DEMONSTRATE What is the nature of the performance required: a systematic understanding of customer profitability analysis; how it can be utilised to enhance earnings per share In what context: enhancing earnings per share in a commercial organisation 13

Assessment task / criteria Learning outcome: Demonstrate a systematic understanding of customer profitability analysis and how it can be utilised to enhance the earnings per share of a commercial organisation. – To demonstrate a systematic understanding requires theoretical knowledge and practical knowledge. – To demonstrate how requires the use of some numerical analysis. – Appropriate task would be to apply CPA to a given situation and write a report to the Board of Directors. 2 nd Learning outcome Communicate key issues and arguments in a written format to a professional standard 14

Assessment criteria Learning outcome: ØDemonstrate a systematic understanding of customer profitability analysis and how it can be utilised to enhance the – Write a report to the Board of Directors critically earnings per share of a commercial organisation. evaluating the use of Customer Profitability Analysis ØCommunicate key issues and arguments in a written format as a means of enhancing the Earnings Per Share of to a professional standard • Task - scenario provided with numerical element XYZ plc. • Criteria to be met by students Written Report writing skills to professional standardcommunication – Theoretical and practical – Use of appropriate theory/knowledge elements, systematic – Evaluation of CPA (benefits and drawbacks) understanding and context of – Relate to particular facts in scenario (Applied) commercial organisation. – Appropriate numerical analysis Applied in context and enhance EPS 15

Grade Based Assessment Part 3 – Completing the assessment matrix descriptors 16

Assessment matrix Header information Learning outcomes being assessed Assessment criteria Comments 17

Assessment matrix Criteria Low Fail Mid Marg. Fail Pass Com’n Dist’n Except’l Dist’n Report writing skills to professional standard Evaluation of CPA (including theoretical knowledge) Relate to particularly facts in scenario Appropriate numerical analysis 18

Completing the grade descriptors • The NTU general grading descriptors use the terminology of: Exceptional • Exceptional Distinction – Excellent • Distinction – • Commendation – Very good • Pass – Good • Marginal Fail – Marginally insufficient • Mid fail – Insufficient • Low fail – Highly insufficient 19

SOLO Taxonomy • The NTU descriptors also utilise the SOLO taxonomy which was developed by Biggs and Collis (1982). 20

21

22

Completing the matrix - Pass standard • Decide what constitutes a pass standard, i. e. what is adequate or satisfactory to achieve a pass standard. • Once this is decided work up towards a Exceptional Distinction, then work down towards a Low Fail. • It is an iterative process. You may find you need to split criteria into more specific ones, or even merge some together in order to distinguish the levels of achievement. • Get a colleague to review it. • Ask the students. 23

Assessment matrix Criteria Low Fail Mid Marg. Fail Pass Com’n Dist’n Except’l Dist’n Report writing skills to professional standard Evaluation of CPA (including theoretical knowledge) Relate to particularly facts in scenario Appropriate numerical analysis 24

Appropriate numerical analysis Exceptional Dist’n Exceptional numerical analysis provided throughout the report to illustrate benefits and drawbacks and imaginative use of analysis (including sensitivity analysis) to illustrate the impact of Board decisions about strategic choices associated with customers on the EPS of XYZ plc. Distinction Excellent numerical analysis with sensitivity analysis provided to illustrate the impact of Board decisions about customers on the EPS of XYZ plc. Commendation Very good numerical analysis provided and used to highlight the impact of Board decisions relating to CPA on the EPS of the XYZ plc. Pass Numerical analysis is correct and used within report to illustrate points, but limited to basic calculations. Marginal Fail Numerical analysis contains some errors that are significant to the results, but narrative reduces the impact of the errors. Mid Fail Numerical analysis contains significant errors and could result in misleading the Board of Directors in their decision making. Low Fail Little or no numerical analysis offered. 25

Assessment matrix Criteria Low Fail Mid Marg. Fail Pass Com’n Dist’n Except’l Dist’n Report writing skills to professional standard Evaluation of CPA (including theoretical knowledge) Relate to particularly facts in scenario Appropriate numerical analysis Evaluation of CPA (including theoretical knowledge and application to scenario) 26

Evaluation of CPA Exceptional Dist’n An exceptional evaluation demonstrating significant insight and grasp of the scenario situation, highlighting the most significant benefits and drawbacks providing the Board with an objective appraisal of the situation and evaluation of the use of CPA to achieve their objective of enhancing the EPS. Distinction An excellent evaluation of theoretical concept covering the benefits and drawbacks. Shows insight gained from the scenario information that provides the Board of XYZ plc with an excellent evaluation on the use of CPA to achieve the objective of enhancing the EPS. Commendation Very good explanation of theoretical concept and evidence of evaluation. A thorough coverage of benefits and drawbacks that utilised the information in the scenario effectively to provide the Board of XYX plc with a very good evaluation of the use of CPA to enhance the EPS. Correct use of CPA model and good evidence of evaluation. Pass Covers both benefits and drawbacks with explanation but limited to basic knowledge and obvious links to XYZ plc’s situation. 27

Evaluation of CPA Pass Marginal Fail Mid Fail Low Fail Correct use of CPA model and good evidence of evaluation. Covers both benefits and drawbacks with explanation but limited to basic knowledge and obvious links to XYZ plc’s situation. Some misunderstanding of CPA. Insufficient coverage of benefits and drawbacks or insufficient explanation. Very limited or no use of scenario information to link to XYZ plc’s situation. Misunderstanding of CPA. Very few benefits or drawbacks included, or some incorrectly stated. Very poor link to scenario information or no mention of XYZ plc’s situation. Complete misunderstanding or no explanation of CPA. Benefits and drawbacks are incorrectly stated. No attempt to relate to the scenario information (basically ignored altogether). 28

Report writing skills to professional standard Exceptional Dist’n Exceptional presentation of information and arguments that reflects a highly professional style of writing. Clear and objective presentation of arguments. Imaginative presentation of numerical analysis integrated into the report in a way that supports the arguments. Very effective use of supporting appendices. Distinction Excellent structure with information and discussion presented in logical, clearly defined sections that aided the understanding of issues relevant to XYZ plc. Written in a clear and objective, professional style. Excellent integration of numerical analysis with use of summaries supported by detail shown in appendices. Commendation Clear structure with very good use of headings and signposting of sections. Very good appropriate writing style. Clear presentation of summary numerical analysis in report with good use of appendices for detail. Pass Clear structure Good, appropriate writing style. Clear presentation of numerical analysis. 29

Comments section • At the end of the assessment matrix is a comments section. – Strengths – Areas for improvement (refer to guidance sheets) – Overall performance • Stay on the lookout for updates as there are enhancements being made to the matrix that will be filtered through at a later date. 30

That completes the GBA presentation. Any questions? 31