Telecommunications in Education Examining Distributed Teaching and Learning
Telecommunications in Education: Examining Distributed Teaching and Learning Environments (EDER 677 L. 91) teaching and Learning Environments: Guides for Critical Reviews and Evaluation for the Asynchronous Class and for the Web Site on November 7 from Calgary © E. Kowch 2002 1
Housekeeping • Some pointers • If you would like to discuss your online project with me, I will be sure to be available at a time when you need… If you would like to discuss your DL portfolio, I would be delighted to discuss that with you too… just let me know… • © E. Kowch 2002 2
Presentation Overview In this lesson, we will: • Explore the issues around designing, developing and utilizing web based assessment techniques for online learning environments or their parts - from the teacher or designer perspective. • Discuss the importance of formative and summative program/course evaluation in the online learning context - from the teacher or designer perspective. © E. Kowch 2002 3
Assessment and Evaluation - Definitions • Assessment - determine changes in learner performance • Evaluation - make judgments and decisions about learners • Program or course evaluation - appraise the success of your program or course © E. Kowch 2002 4
Assessment and Evaluation - Self Activity (something to ponder as you create evaluations for your own online learning project) • List examples of web based assessment activities that you prefer (see the resource section of this lesson) • What types of student learning do these activities support (e. g. Bloom’s Taxonomy – knowledge, comprehension, application, analysis, synthesis, evaluation)? • Identify issues surrounding the use of web based assessment activities that you feel strongly about, and address those issues in your project. © E. Kowch 2002 5
Triangle of Assessment • The multidimensional approach to assessment Knowledge Transfer and Performance Knowledge Constructs and Reflection Bloom’s Taxonomy Concrete Knowledge Acquisition http: //www. coun. uvic. ca/learn/program/hndouts/bloom. html © E. Kowch 2002 6
Concrete Knowledge Acquisition Concept • "Factual Foundation Knowledge" provides the ground floor in a learning hierarchy. You can evaluate things that must be known - facts. • The working knowledge or language of a subject discipline can be considered as a foundational knowledge to succeeding in a learning event, online or anywhere. © E. Kowch 2002 7
Concrete Knowledge Acquisition Working Knowledge Examples • Calculus • The student can recognize and count numbers • The student can add, subtract, multiply and divide • Literary criticism and creative writing • The student can apply the alphabet and the basic rules of grammar © E. Kowch 2002 8
Concrete Knowledge Acquisition Assessment Techniques • Measuring student mastery of the factual “foundation knowledge” Some in instructional methodologies where you use specific learning evaluation techniques (Great source book: Gronlund: The Assessment of Student Learning” 7 th Ed. ) We take this up in EDER 673 in detail. Drill and practice is one assessment technique: with components designed to check the level of mastery of basic knowledge. • Drill and Practice • multiple choice • true and false • matching • ordering • short answer © E. Kowch 2002 9
Concrete Knowledge Acquisition More Assessment Ideas • • Give a Preview quiz – on preparatory textbook, article or online readings Give a Review quiz – on lecture content Give a Collaborative quiz – online discussion about the questions and answers on a quiz Students can develop their own quiz questions – learning how to design tests Online, other students can respond and evaluate the documents. For stronger students in the class – the writing of mid-term and final exam questions can also result in credit for this online “assignment”. Online quizzes - Integrity test questions – first or last question – Did you do this quiz on your own? (on site supervised testing is a simple and effective option). © E. Kowch 200210
Concrete Knowledge Acquisition Web Based Tools for Evaluation: See the Resource Section in the Lesson Plan for November 7 th for more of these: • • Hot Potatoes http: //web. uvic. ca/hrd/halfbaked/ Quiz Lab http: //www. funbrain. com/quiz/index. html Te. Ach-nology. com's Crossword Puzzle Maker http: //www. teach-nology. com/web_tools/crossword/ Discovery School's Puzzle. Maker (Word Searches) http: //www. puzzlemaker. com/ © E. Kowch 200211
Knowledge Constructs and Reflection The Concept of Knowledge Constructs • This is the process of students constructing and/or discovering their own knowledge. ( Self evaluation with cognitive (model) direction). • Internalization and integration of new information with previously held knowledge (in the good old bad old days, we called this “synthesis” ; -) © E. Kowch 200212
Knowledge Constructs and Reflection The Concept of Reflection • Students reflecting on the construction of their knowledge as part of the evaluation process: • Zone of proximal development can be created online too: the instructor watches and guides students to evaluate each other after they evaluate themselves. • Students are comparing thoughts and ideas with others. © E. Kowch 200213
Knowledge Constructs and Reflection Assessment Techniques • Structured journals (e. g. responding to specific questions) • Open-ended reflection journals (e. g. responding to broad or open questions) • Collaborative discussions and summaries © E. Kowch 200214
Knowledge Constructs and Reflection Assessment Techniques for Online Discussions • See our home page links © E. Kowch 200215
Knowledge Constructs and Reflection Assessment Examples Example One - Class Discussion Scoring Guide http: //www. tageducation. org/webct/level 1/6 discussrubric. htm • a rubric for a 100 -point project (60% for responses to weekly readings, 40% for responses to classmates postings) Example Two - Grading Online Discussions http: //www. oznet. ksu. edu/dp_iet-edtech/Techfor. Teaching/grading. htm • various models based on support/clarity/helpfulness, an A-F scale, and groups grading each other © E. Kowch 200216
Knowledge Constructs and Reflection Assessment Examples Example Three - Evaluation and Grading Criteria for Web. Board Discussion Assignments http: //www. ivcc. edu/eng 1001/handout_evaluating_discussions. htm • an outline of four evaluation criteria for discussion assignments Example Four - Discussion Guidelines and Grading Standards http: //www. stedwards. edu/ursery/discuss. htm • discussion grade guidelines on an A to F scale, as well as a 10 -point guideline for participation © E. Kowch 200217
Knowledge Constructs and Reflection Recall the Modes of Web Based Computer Mediated Communication Tools • Asynchronous • • E-mail File transfers Threaded discussion forums Synchronous • • • Virtual chats Shared whiteboards, URLs, and applications MUDs(multi-user dimensions) and MOOs (multi-user dimension - object oriented) © E. Kowch 200218
Knowledge Constructs and Reflection Web Based Resources that help you evaluate your online learning environs, considering the mode of delivery / interaction chosen by the designer: • Using Asynchronous Conferencing to Promote Critical Thinking http: //crlt. indiana. edu/publications/crlt 99 -8. pdf • Instructional Design Principles for Teaching in Computer Conferencing Environments http: //www. cvm. tamu. edu/wklemm/instruct. html • Shaping Cyberspace Into Human Space http: //css. sfu. ca/update/vol 6/6. 3 -harasim. main. html • Handbook for Instructors on the use of Electronic Discussions http: //www. osu. edu/education/ftad/Publications/elecdisc/pages/index. htm • Check for Plagiarism http: //www. turnitin. com © E. Kowch 200219
Knowledge Transfer & Performance Concept • Performance-based assessment students demonstrate the (learned) behavior to be measured (by the evaluator). • Authentic assessment - students demonstrate the acquisition and mastery of a 'real world' skill or concept. • Learners applying knowledge and skills rather than simply remembering facts © E. Kowch 200220
Knowledge Transfer & Performance Web Based Resources: Hotlinks to Assessment tools: • • Performance Assessment Educational Issues Series Strategic Questions - What to Consider When Planning for Electronic Portfolios Electronic Teaching Portfolios Electronic Portfolio Checklist Portfolio Assessment Student Portfolios: Classroom Uses Student Portfolios: Administrative Uses • Alternative Assessment and Technology • Assessment Terminology: A Glossary of Useful Terms • Collected Assessment Resources Center for Research on Evaluation, Standards, and Student Testing http: //www. ed. gov/databases/ERIC_Digests/ed 365312. html http: //www. newhorizons. org/assmtterms. html http: //www. cse. ucla. edu/ © E. Kowch 200221
Online Program/Course Evaluation - Remember this Model from earlier in the course? Now we look at the “Evaluate” loop: (More detail will come in EDER 673) © E. Kowch 200222
Online Program/Course Evaluation - Definitions Formative Evaluation • The ongoing analysis of your program/course design during development and implementation. Much more on this in EDER 673 and EDER 675 Summative Evaluation • Is conducted to determine whether your course or program is worthy of adoption or continuation • leads to a “go/no go” decision on the program. © E. Kowch 200223
Online Program/Course Evaluation - Formative Some of the on-going process that designers use at each stage of the course development process: • They have colleagues and students constantly reviewing the development and implementation of your web course site – internal review • They develop an evaluation checklist or form • They ensure that each module is piloted • They make the necessary revisions based on informal and formal feedback mechanisms © E. Kowch 200224
Online Program/Course Evaluation (Summative) Judging the worth or merit of a program/course: • This evaluation is conducted after a program/course/design has been in place for a complete cycle – usually an external review to the institutional course distributor Key Questions used in this type of program evaluation: • Effectiveness – Do students learn the basic learning objectives or exceed them? • Appeal – Does the program/course/design engage learners and students – do they like it? (Affective) • Efficiency – Is the program/course cost effective? © E. Kowch 200225
Program/Course Evaluation - Resources Article Shrock, S. A. and Geis, G. L. (1999). Evaluation. In H. D. Stolovitch and E. J. Keeps (Eds. ) Handbook of human performance technology (2 nd ed. ) ( Chapter 10 – Evaluation, pp. 185209). San Francisco, Jossey-Bass Pfeiffer. Web sites See the resources section in the lesson plan for hundreds of resources. © E. Kowch 200226
For next session • • Contribute to this week’s discussion forum. Check out the Web Site Home Page for further instructions (core readings). Your online project production should have started by now - your professor will review anything that you like ahead of time… Your Personal DL portfolio web site should be under construction now. Adieu from Calgary. . © E. Kowch 200227
- Slides: 27