Designing Conducting Formative Evaluation Based upon Dick Carey

Designing & Conducting Formative Evaluation Based upon Dick & Carey

Why Formative Evaluation? 4 To determine Weakness(es) in the Instruction

Weaknesses 4 Teachers/Trainers are not good predictors of the effectiveness of instructional content 4 Instructional Theory is not a perfect science 4 Few instructional products are ever evaluated with learners 4 Designer can’t see the forest for the trees

Six Stages 4 Design Review 4 Expert Review 4 One-To-One 4 Small Group 4 Field Trials 4 Ongoing Evaluation

Design Reviews 4 Instructional goal match problem? 4 Learner analysis match audience? 4 Task analysis include prerequisites? 4 Test items reliable and valid? 4 Test items match the objectives?

Expert Review 4 Content accurate & up-to-date? 4 Present a consistent perspective? 4 Are elements realistic & accurate? 4 Pedagogy consistent with current instructional theory? 4 Instruction appropriate to audience?

One-On-One Review 4 Is the message clear? 4 What is the impact on: < learner attitudes < achievement of objectives & goals 4 Feasibility of training

Small Group Review 4 Look for the effects caused by the changes made in the one-to-one review 4 Identify any remaining learning problems

Field Trials (Review) 4 Look for effects in changes made in small group 4 Can the instruction be used in the context in which it was intended

Ongoing Evaluation 4 Viewed through the Lens of: 4 Project Size 4 Life span of content 4 Audiences change 4 One-To-One 4 Small Group Tryouts 4 Field Trials

Learner Evaluation “Testing” 4 Do learners understand the instruction? 4 Do they know what to do during the practice & the tests? 4 Can they interpret graphics in the text? 4 Can they read all the material? 4 How much time does it take?

Summary 4 Formative Evaluation is critical 4 Complete as many stages as practical 4 Don’t depend on opinion 4 Document your evaluation