Evaluating Online Learning Issues and Strategies Mark Hawkes
- Slides: 29
Evaluating Online Learning: Issues and Strategies Mark Hawkes Dakota State University Innovations, Educating New Generations March 1, 2002
Presentation Objective Discuss online learning evaluation approaches in graduate programs at two universities Identify criteria/indicators suitable for the evaluation of online learning environments
Distance Learning Literature Evaluation. . . . No (79%) Yes (21%) Focus. . Training Education (87%) (13%) Impact on Learning. . . . No (58%) Yes (42%)
Instructor Student Online Learning Architect ure Internet Student Login Interface Content Resources Management and Organizational Information Communication Modes Assessment Institutional Support Services System Resources
Familiar Online Learning Evaluation Targets. . . interface design instructional design student satisfaction technology access faculty satisfaction economic viability departmental capacity interdepartmental collaboration
Common Dimensions of Effective Online Learning: Relevant and challenging assignments Providing adequate and timely feedback through teacher-student interaction Flexibility in teaching and learning Constructing coordinated learning environments Constructing rich environments for student to student interaction
DSU’s Educational Technology Program Students: 36 Credit hour MS program 80% Education (K 12, Technical, Higher Ed. ) 20% Business/industry 90% Online; 10% On campus Female 68%; Male 32% Project-based curriculum
DSU’s ET Environment Pervasive technological culture Consistency between program goals and the state/region-wide initiatives Campus-wide faculty support Institutional experience in Web-based instruction delivery Multi-delivery methods Client: teachers, teacher developers, instructors, trainers, technology coordinators, etc.
An Evaluation Model. . . Illuminative Operation of Components And Subcomponents Integrative Holistic perspective on The learning experience Components Infra- Course & Work structure Program Interaction Impact Flow /System Design Process Observing and Detecting Functional Problems Product Focused on Performance Outcomes
Infrastructure/System Input/output devices Network speed and connectivity Network design/Topology Technical support systems and maintenance
Course and Program Design Nature of the Design n n Situation Based Role of State and National Standards Sequencing/Instructional Strategies Assessment Motivation: Learning vs. Performance Visualization Tools and Media User Interface Course Management
Work Flow Use of discussion tools Software usage Message redundancy (audio, video, web pages, emails). Progression n n Do learners progress through their work tasks in a linear fashion? (novice-like) Nonlinear opportunistic fashion (expertlike)
Interaction Social and instructional Must account for all of the following relationships: Instructor Learners Technology Content
Interaction Example Learner Assessment Content Learning Resources (Technology) Performance Catalog info Instructor Query Learner Records Preferences (Metcalf, Snitzer, Austin,
Online Course Interaction Announcements Email Discussion Board (Web. Board Conferencing) Synchronous text chat Streaming media using Real Player Desktop Video File Loading Online assessment Audio/video clips Audio-narrated Power. Point's Room-based Video
Impact Course performance Collaborative learning Retention/attrition (course and program) Professional relevance and utility Learner productivity
Evaluation Attributes Multi-sourced data (students, server log files, etc) Internal and external Performance based Comparison and criterion based
The breadth of this course was: 1 2 3 Not nearly enough 4 5 6 The right amount 7 Way too much Compared to a traditional course 1 2 3 A much narrower range much wider range of of material was covered 4 5 6 7 About the same range A of material was covered Online: 4. 61 Compared to traditional 4. 65 n=32
The depth of this course was: 1 2 3 Not nearly enough 4 5 6 The right amount 7 Way too much Compared to a traditional course 1 2 3 4 5 6 7 Material was covered in much less depth about the same depth much more depth Online: 4. 48 Compared to traditional 4. 42 n=32
The extent of critical thinking required: 1 2 3 4 Not nearly enough Way too much 5 6 7 The right amount Compared to a traditional course 1 2 Much less 3 4 5 6 About the same 7 Much more Online: 4. 61 Compared to traditional 4. 94 n=32
The amount of effort put into the course: 1 2 3 4 Much less more 5 6 7 About the same Much Compared to a traditional course 1 2 Much less 3 4 5 6 About the same Online: 5. 65 Compared to traditional: 5. 26 n=32 7 Much more
Evaluation Processes Mostly formative Mixed data collection and analysis methods (document analysis, student artifact analysis, survey, interview, text analysis, etc. ) Course and program evaluation
Typical Problems with Online Courses Facilitating and encouraging collaboration Time management Student proficiency with course tools Ambiguous directions Timeliness of feedback
Factors Beyond ID Control Student sophistication with technology tools System capacity Learner availability/accessibility Enthusiastic, responsive instructor Good learner support Motivated learners
How to Design and Effective Online Course? Follow basic ID principals Build a climate of disclosure and full participation Institute informal student evaluation and check-in mechanisms Active and intensive instructor participation Build in as much interactivity as possible Create visually interesting screens/pages Ensure instructions are very clear Multi-mode interaction is critical Redundancy
- Sectoriapp
- Transformer hawkes process
- Hawkes bay refrigeration
- Rumelt's criteria
- Most quantitative criteria are geared to
- Evaluating trading strategies
- Activity 2 evaluating online resources
- Collaborating via social networks and groupware
- Chapter 7 strategic management
- Mis issues in strategy implementation
- Implementing strategies: management and operations issues
- Implementing strategies management and operations issues
- Implementing strategies management and operations issues
- Checklist for evaluating learning materials
- Cuadro comparativo e-learning m-learning b-learning
- Dse english electives
- The miracle of language
- Background issues in language learning
- Inductive bias in decision tree learning
- Active learning strategies to promote critical thinking
- Classroom strategies for interactive learning
- Reactive discipline
- Language learning strategies
- Active learning strategies to promote critical thinking
- Siop strategies
- Calla model
- Assessment strategy
- Cognitive style
- Marzano strategies
- The effectiveness of online and blended learning