Lecture 7 Evaluation Evaluation n n Purpose Assessment

  • Slides: 19
Download presentation
Lecture 7 Evaluation

Lecture 7 Evaluation

Evaluation n n Purpose Assessment of the result n n n n Against requirements

Evaluation n n Purpose Assessment of the result n n n n Against requirements Qualitative Quantitative User trials Etc Assessment of and Reflection on process Actions

Purpose of Evaluation n To demonstrate that requirements have been met An assessment of

Purpose of Evaluation n To demonstrate that requirements have been met An assessment of the fitness for purpose To identify shortcomings To identify opportunities To reflect upon product/process etc n Demonstrate you did it right – and to learn n n

Purpose n The aim is to honestly assess the results and processes to n

Purpose n The aim is to honestly assess the results and processes to n n n Demonstrate successful completion of the project Improve the product or identify potential improvements Learn from mistakes

Methodology for Evaluation n n There are many different ways to evaluate systems/products etc

Methodology for Evaluation n n There are many different ways to evaluate systems/products etc There may be software or methodological tools to support some forms of evaluation n n E. g. see web page assessment tools Note: Evaluation is NOT testing

Methodology for Evaluations There are many dimensions to the evaluation: n Subjective v Objective

Methodology for Evaluations There are many dimensions to the evaluation: n Subjective v Objective n Qualitative v Quantitative n Analytical v Experimental n System v Component n Perspectives: Customer v User v Developer …. n …….

Evaluation against Requirements n n Requirements should define a set of criteria that the

Evaluation against Requirements n n Requirements should define a set of criteria that the product must meet. So, check whether they are met n n Are they sufficiently precise? Are they measurable? How can they be assessed? Develop a methodology for evaluation – before development

Evaluation against Requirements n Aside: Acceptance tests n A precise set of tests that

Evaluation against Requirements n Aside: Acceptance tests n A precise set of tests that must be passed for acceptance n n Necessary but not sufficient Defined in contract Some overlap with evaluation Aside: -testing, -testing …

Subjective v Objective assessment n n For some criteria there will be hard, measurable

Subjective v Objective assessment n n For some criteria there will be hard, measurable features For others, we have to rely upon an assessors judgement

Qualitative v Quantitative n n Many features can be assessed quantitatively Others may have

Qualitative v Quantitative n n Many features can be assessed quantitatively Others may have to be qualitative n n Excellent, good, acceptable, poor Ranking

Quantitative assessment n What can we measure? n n File size Speed n n

Quantitative assessment n What can we measure? n n File size Speed n n n To perform a task Latency Memory requirements Reliability etc

Qualitative measures n Attractiveness Ease of use Fitness for purpose Extensibility etc n Often

Qualitative measures n Attractiveness Ease of use Fitness for purpose Extensibility etc n Often these are the more important criteria n n

Perspectives n Who are the stakeholders? n n n Customer User Developer Maintainer Etc

Perspectives n Who are the stakeholders? n n n Customer User Developer Maintainer Etc Look at each perspective

What is being evaluated? n Some assessments may focus upon different parts of the

What is being evaluated? n Some assessments may focus upon different parts of the artifact n n The whole system The user interface Documentation The implementation n n Maintainability Reliability Extensibility etc

Trials n An evaluation from the users’ perspective may require trialling with typical users

Trials n An evaluation from the users’ perspective may require trialling with typical users n Assess user’s reaction to the work n n How? May have different classes of user n n Expert, novice Frequent, occasional

How to evaluate n Develop a plan for the evaluation n Identify the criteria

How to evaluate n Develop a plan for the evaluation n Identify the criteria Specify how they will be assessed Plan will necessarily focus on some aspects – at the expense of others

Reporting n n n Define the evaluation methodology Report the results of the evaluation

Reporting n n n Define the evaluation methodology Report the results of the evaluation Discuss recommendations n n n Improvements Changes Enhancements

Reflection n Evaluate the process n n n What was good What was bad

Reflection n Evaluate the process n n n What was good What was bad Recommendations n How to improve it next time

Actions n Review the set of recommendations n n n Product Process Identify which

Actions n Review the set of recommendations n n n Product Process Identify which of these n n n Must be addressed within the project Can be addressed within the project Should be carried forward to future work

PURPOSE Purpose and Food Value Chain PURPOSE PurposePURPOSE Purpose and Food Value Chain PURPOSE Purpose
Analyse The Use Evaluation Evaluation Assessment Evaluation AssessmentAnalyse The Use Evaluation Evaluation Assessment Evaluation Assessment
Assessment Purpose of Assessment Assessment is the processAssessment Purpose of Assessment Assessment is the process
Financing Purpose Types and Rules Purpose Purpose FinancingFinancing Purpose Types and Rules Purpose Purpose Financing
Determining Purpose Determining Purpose Purpose 1 Do youDetermining Purpose Determining Purpose Purpose 1 Do you
Whats the purpose AUTHORS PURPOSE NOTES Authors PurposeWhats the purpose AUTHORS PURPOSE NOTES Authors Purpose
Whats the purpose AUTHORS PURPOSE NOTES Authors PurposeWhats the purpose AUTHORS PURPOSE NOTES Authors Purpose
Text Purpose What does purpose mean Purpose meansText Purpose What does purpose mean Purpose means
E Assessment EAssessment E Assessment E Assessment AssessmentE Assessment EAssessment E Assessment E Assessment Assessment
Summative assessment Assessment for learning Formative assessment AssessmentSummative assessment Assessment for learning Formative assessment Assessment
Assessment and Evaluation PERFORMANCEBASED AND PORTFOLIO ASSESSMENT AssessmentAssessment and Evaluation PERFORMANCEBASED AND PORTFOLIO ASSESSMENT Assessment
Evaluation and Assessment Presentation Evaluation and Assessment presentationEvaluation and Assessment Presentation Evaluation and Assessment presentation
Assessment and Evaluation Mazyad Alotaibi Assessment and EvaluationAssessment and Evaluation Mazyad Alotaibi Assessment and Evaluation
Evaluation Assessment Athletic Injury Terminology Evaluation Assessment AthleticEvaluation Assessment Athletic Injury Terminology Evaluation Assessment Athletic
INTRODUCTION TO ASSESSMENT DESIGN INTRODUCTION PURPOSE INTRODUCTION PURPOSEINTRODUCTION TO ASSESSMENT DESIGN INTRODUCTION PURPOSE INTRODUCTION PURPOSE
Employability Skill Assessment by Purpose The purpose ofEmployability Skill Assessment by Purpose The purpose of
Evaluation Economic Evaluation Economic evaluation considers assessment ofEvaluation Economic Evaluation Economic evaluation considers assessment of
PAVEMENT EVALUATION Pavement evaluation Functional evaluation Structural evaluationPAVEMENT EVALUATION Pavement evaluation Functional evaluation Structural evaluation