OUM SEMINAR SERIES LATIFAH ABDOL LATIF 29 MARCH

OUM SEMINAR SERIES LATIFAH ABDOL LATIF 29 MARCH 2016

Course Evaluation of Online Courses • Instrument: • Survey Administration: • Data Analysis: a) Data were retrieved from the Survey Monkey portal on 28 Sept 2015. The original data file downloaded from the Survey Monkey portal comprised 734 cases. Only 397 were used for final data analysis; b) Data cleaning and data analysis were carried out using SPSS Statistics 22; c) Text responses were not included in the analysis (requires SPSS Text Analytics software)

Demographic Variables Table 2: Gender Freque Percent ncy Table 4: Fully Online Course Valid Percent Cumulati ve Percent Freque Percent ncy Valid Percent Cumulati ve Percent Female 287 72. 3 Yes 214 53. 9 Male 110 27. 7 100. 0 No 183 46. 1 100. 0 Total 397 100. 0 Table 3: Age Categories Age Range Freque Percent ncy Table 5: CGPA Profile Valid Percent Cumulati ve Percent CGPA Freque ncy Percent Valid Percent Cumulati ve Percent 0. 00 -1. 00 11 2. 8 1. 01 -2. 00 45 11. 3 14. 1 98. 0 2. 01 -3. 00 182 45. 8 59. 9 100. 0 3. 01 -4. 00 159 40. 1 100. 0 Total 397 100. 0 18 to 24 76 19. 1 25 to 34 177 44. 6 63. 7 35 to 44 106 26. 7 90. 4 45 to 54 30 7. 6 55 to 64 Total 8 2. 0 397 100. 0

Time spent logging in online learning 24. When I log on to the fully online course, my fully online sessions averaged … Frequency Percent Valid Percent Cumulative Percent Less than 30 minutes 104 26. 2 28. 6 30 to 60 minutes 150 37. 8 41. 2 69. 8 1 to 2 hours 69 17. 4 19. 0 88. 7 2 to 4 hours 25 6. 3 6. 9 95. 6 More than 4 hours 16 4 4. 4 100 Total 364 91. 7 100 Missing 33 8. 3 Total 397 100

Reliability of Instrument • A reliability analysis was undertaken to determine the internal consistency of the 2 rating scale constructs in the study, i. e. (I) 7 items under “My e-Tutor” construct, and (II) 17 items under “Course Learning Outcomes” construct. Table 1 shows the reliability coefficients (Cronbach alpha). • The high values of Cronbach Alpha indicate well-constructed scale items and the respondents did not face problems in interpreting the questions. This also shows the relatively good questionnaire construction techniques used and the good understanding of respondents. Constructs No. of Items Reliability Coefficients 1. Construct 1: My e-Tutor 7 . 948 2. Construct 2: Course Learning Outcomes 17 . 950

RESULTS Mean Std. Deviation N 8. The e-tutor was very knowledgeable about the subject matter of the course 3. 71 0. 9030 397 9. The e-tutor was actively involved in facilitating the course 3. 64 0. 9755 397 10. The e-tutor encouraged my active involvement in learning. 3. 61 0. 9525 397 11. The e-tutor expect me to be responsible for my own learning. 3. 66 0. 8289 397 12. The e-tutor cared about my individual learning in this course. 3. 48 0. 9092 397 13. The e-tutor was responsive to my concerns in learning. 3. 59 0. 9078 397 14. I would recommend this e-tutor to other students. 3. 55 1. 017 397 Construct 1: My e-Tutor

Mean Std. Deviation N 15. The learning outcomes in this module are clearly identified. 3. 75 0. 8159 397 16. Assessment tasks for this module evaluate my achievements of the learning outcomes. 3. 72 0. 7909 397 17. The course material / learning resources help me to achieve the learning outcomes. 3. 71 0. 8051 397 18. The workload for this course is appropriate for the achievement of the learning outcomes. 3. 67 0. 7720 397 19. The quality of teaching of this module helps me achieve the learning outcomes. 3. 65 0. 8269 397 20. I make the best use of the learning experience (self test, forum, digital library, readings and video lectures) in this module. 3. 72 0. 8066 397 21. I think about how i can learn more effectively in this module. 3. 83 0. 7169 397 22. I am motivated to achieve the learning outcomes in this module. 3. 79 0. 7731 397 23. Overall, I am satisfied with this fully online course 3. 39 0. 9712 397 25. The overall usability of the course website was good. 3. 51 0. 8332 397 26. I am goal-directed; if I set my sights on the results, I usually can achieve it. 3. 73 0. 7129 397 27. I put forth the same effort in the fully online course as I would in a blended course. 3. 63 0. 8153 397 28. I prefer to express my ideas and thoughts in writing rather than through oral expression. 3. 61 0. 8308 397 29. I understand directions better when I see a map than when I receive oral directions. 3. 62 0. 7861 397 30. The quality of the learning experience in fully online courses is better than in blended courses. 3. 06 0. 9490 397 31. I would choose to enroll in a fully online course rather than a blended course if I was given a choice. 2. 99 1. 0236 397 32. I would take a fully online course at OUM again in the future. 3. 14 1. 0220 397 Construct 2: Course Learning Outcome

The 9 Items on Online Course evaluation • Looking at questions 15 – 23 which evaluates the fully online courses; • 15 out of the 35 courses were analysed (to ensure no. of respondents >4); • Average of the exam mean scores for Sept’ 14, Jan’ 15 and May’ 15 were used for each course;

MEANS FOR THE 15 ONLINE COURSES MEANS (%) SD 15. The learning outcomes in this module are clearly identified. 3. 73 (74. 6) 0. 8159 21. I think about how i can learn more effectively in this module. 3. 84 (76. 8) 0. 7169 20. I make the best use of the learning experience in this module. 3. 73 (74. 6) 0. 8066 22. I am motivated to achieve the learning outcomes in this module. 3. 71 (74. 2) 0. 7731 18. The workload for this course is appropriate for the achievement of the learning outcomes. 3. 69 (73. 8) 0. 772 16. Assessment tasks for this module evaluate my achievements of the learning outcomes. 3. 66 (73. 2) 0. 7909 17. The course material / learning resources help me to achieve the learning outcomes. 3. 64 (72. 8) 0. 8051 19. The quality of teaching of this module helps me achieve the learning outcomes. 3. 60 (72. 0) 0. 8269 23. Overall, I am satisfied with this fully online course 0. 9712 ITEMS 3. 21 (64. 2)

Course Evaluation Items: Q 15 -23 Online Course Code 16. Assessment 15. The tasks for this learning No. of module outcomes in evaluate my respon this module dents are clearly achievement s of the identified. learning outcomes. 17. The 18. The 19. The course workload for quality of 20. I make 21. I think 22. I am 23. Overall, material / this course is teaching of the best use about how i motivated I am learning appropriate this module of the can learn to achieve satisfied resources for the helps me learning more the learning with this help me to achievement achieve the experience in effectively in outcomes in fully online achieve the of the learning this module. course learning outcomes. BBPS 4103 STRATEGIC MANAGEMENT 11 3. 82 3. 64 3. 73 3. 82 3. 55 OUMM 2103 ENTREPRENEURSHIP 15 4. 05 3. 87 3. 78 3. 73 3. 80 3. 67 3. 87 3. 53 6 4. 17 4. 00 4. 17 4. 33 4. 11 4. 33 3. 50 7 3. 71 3. 86 4. 14 4. 00 3. 43 4. 14 4. 00 3. 86 3. 90 3. 86 4. 07 4. 00 3. 39 20 3. 75 3. 63 3. 50 3. 69 3. 85 3. 36 21 3. 67 3. 62 3. 52 3. 62 3. 48 3. 57 3. 81 3. 67 3. 33 8 3. 50 3. 63 3. 50 4. 00 4. 13 3. 88 3. 30 MPW 2113 BAH ASA KEBANGSAAN A BBPM 2203 MARKETING ANAGEMENT II CBMS 4303 MANAGEMENT INFORMATION SYSTEM 14 BBPM 2103 MARKETING MANAGEMENT I BBPB 2103 HUMAN RESOURCE MANAGEMENT MPW 1133 MALAYSIAN STUDIES

Course Evaluation Items: Q 15 -23 Online Course Code CBKI 4103 KNOWLEDGE MANAGEMENT 16. 18. The 17. The 19. The Assessment workload for 20. I make 21. I think 15. The course quality of 22. I am 23. Overall, tasks for this course is the best about how i learning material / teaching of motivated to I am No. of module appropriate use of the can learn outcomes in learning this module achieve the satisfied evaluate my for the learning more respon this module resources helps me learning with this dents are clearly achievement me to achievement achieve the experience effectively outcomes in fully online s of the in this identified. the learning this module. course learning module. outcomes. 7 3. 71 4. 14 4. 29 4. 00 3. 29 4 4. 25 4. 00 4. 25 3. 50 4. 00 3. 50 3. 25 5 4. 00 3. 80 3. 60 3. 80 3. 20 5 3. 40 3. 20 3. 40 3. 60 3. 80 3. 40 3. 57 3. 80 3. 20 5 3. 95 3. 74 3. 94 3. 73 3. 40 3. 80 4. 00 3. 40 3. 00 7 3. 29 3. 14 3. 43 3. 29 2. 86 4 2. 50 2. 75 2. 00 2. 75 3. 00 3. 25 2. 50 2. 00 139 3. 73 3. 66 3. 64 3. 69 3. 60 3. 73 3. 84 3. 71 3. 21 CBAD 2103 SYSTEM ANALYSIS AND DESIGN BBCM 4103 COMPENSATION MANAGEMENT MPW 2133 MALAYSIAN STUDIES MPW 2153 MORAL EDUCATION MPU 3223 ENTREPRENEURSHIP MPW 1153 MORAL EDUCATION Overall Means

INFORMATION SYSTEM BBPM 2103 MARKETING MANAGEMENT I BBPB 2103 HUMAN RESOURCE MANAGEMENT MPW 1133 MALAYSIAN STUDIES Average Mean /100 Sept 14, Jan 15, Mei 15 CBMS 4303 MANAGEMENT 23. Overall, I am satisfied with this fully online course MANAGEMENT II 22. I am motivated to achieve the learning outcomes in this module. BBPM 2203 MARKETING 21. I think about how i can learn more effectively in this module. KEBANGSAAN A 20. I make the best use of the learning experience in this module. MPW 2113 BAH 19. The quality of teaching of this module helps me achieve the learning outcomes. ENTREPRENEURSHIP 18. The workload for this course is appropriate for the achievement of the learning outcomes. OUMM 2103 17. The course material/learning resources help me to achieve the learning outcomes. MANAGEMENT 16. Assessment tasks for this module evaluate my achievements of the learning outcomes. BBPS 4103 STRATEGIC 15. The learning outcomes in this module are clearly identified. Online Course Code No. of Respondents Online Course Evaluation 11 3. 818 3. 636 3. 727 3. 818 3. 545 64. 3 15 4. 050 3. 848 3. 867 3. 778 3. 733 3. 800 3. 667 3. 867 3. 533 62. 4 6 4. 167 4. 000 4. 167 4. 333 4. 108 4. 333 3. 500 64. 0 7 3. 714 3. 857 4. 143 4. 000 3. 429 56. 7 14 4. 143 4. 000 3. 857 3. 903 3. 857 4. 071 4. 000 3. 385 53. 9 20 3. 750 3. 633 3. 500 3. 686 3. 850 3. 359 49. 3 21 3. 667 3. 619 3. 524 3. 619 3. 476 3. 571 3. 810 3. 667 3. 333 59. 1 8 3. 500 3. 625 3. 500 4. 000 4. 125 3. 875 3. 299 50. 1

MPU 3223 ENTREPRENEURSHIP MPW 1153 MORAL EDUCATION Average Mean /100 Sept 14, Jan 15, Mei 15 MPW 2153 MORAL EDUCATION 23. Overall, I am satisfied with this fully online course STUDIES 22. I am motivated to achieve the learning outcomes in this module. MPW 2133 MALAYSIAN 21. I think about how i can learn more effectively in this module. MANAGEMENT 20. I make the best use of the learning experience in this module. BBCM 4103 COMPENSATION 19. The quality of teaching of this module helps me achieve the learning outcomes. ANALYSIS AND DESIGN 18. The workload for this course is appropriate for the achievement of the learning outcomes. CBAD 2103 SYSTEM 17. The course material/learning resources help me to achieve the learning outcomes. MANAGEMENT 16. Assessment tasks for this module evaluate my achievements of the learning outcomes. CBKI 4103 KNOWLEDGE 15. The learning outcomes in this module are clearly identified. Online Course Code No. of Respondents Online Course Evaluation 7 3. 714 4. 143 4. 286 4. 000 3. 286 49. 7 4 4. 250 4. 000 4. 250 3. 500 4. 000 3. 500 3. 250 47. 8 5 4. 000 3. 800 3. 600 3. 800 3. 200 58. 8 5 3. 400 3. 200 3. 400 3. 600 3. 800 3. 400 3. 566 3. 800 3. 200 54. 8 5 3. 950 3. 744 3. 943 3. 733 3. 400 3. 800 4. 000 3. 400 3. 000 57. 2 7 3. 286 3. 143 3. 429 3. 286 2. 857 65. 8 4 2. 500 2. 750 2. 000 2. 750 3. 000 3. 250 2. 500 2. 000 56. 9

Results Courses with lowest exam mean score & lowest in Quality of Teaching and Overall Satisfaction: 1. BBPM 2103: MARKETING MANAGEMENT I 2. CBKI 4103: KNOWLEDGE MANAGEMENT 3. CBAD 2103: SYSTEM ANALYSIS AND DESIGN Courses lowest in Overall Satisfaction: due to Quality of teaching, Assessment, Clarity of LO, etc. : 1. MPW 1153: MORAL EDUCATION 2. MPU 3223: ENTREPRENEURSHIP 3. MPW 2153: MORAL EDUCATION

Background 1. In OUM, evaluation of COURSES has not been carried out consistently throughout the faculties. ; 2. However, for all PROGRAMMES offered, an external examiner is invited to carry out an evaluation of the programme, after going through almost a full cycle; 3. This is a onetime evaluation which does not allow formative feedback wherein improvements can be instituted along the students’ study cycle; 4. ITLA monitors and evaluates tutors’ performance through a tutor evaluation exercise. However, the results are not integrated with students’ feedback on the course that they are taking; 5. This makes it difficult for OUM to introduce specific intervention in improving the course as the evaluation of the tutors and the courses are not done in an integrated manner.

Why implement this system-wide evaluation? Implementing this university wide survey will help OUM in the following ways: 1) for curricular and pedagogic review & reform and decision making. Data can be also be used to help institutions identify, promote, and demonstrate effective T&L; 2) for quality improvement and also managing academic performance, professional development of tutors, academic performance rewards, promotion processes and as key performance indicators for academics (in the aspect of T&L); 3) the qualitative data can be used by academics as feedback to support and improve their academic practice (T&L);

Research Questions 1. What are the student’s perceptions of their learning experiences in fully online courses? 2. What tutor attributes do students perceive that help them learn? 3. What tutor attributes do students perceive that hinder their learning?

Course Evaluation Quantitative items : 1 -11 Using the 1 -4 Likert-Type scale: (1: Strongly Disagree; 2: Disagree; 3: Agree; 4: Strongly Agree) 1. The learning outcomes in this course are clearly identified. The learning outcomes are what you are expected to know, understand or be able to do in order to be successful in this course. 2. The learning experiences in this course help me to achieve the learning outcomes. The learning experiences could include: face-to-face tutorials, laboratories, clinical practicums, learning tasks/activities, online forum, online interactions, etc 3. The learning resources in this course help me to achieve the learning outcomes. Learning resources could include print modules, multimedia and online study materials, and equipment available in laboratories, clinics or studios, digital library materials, etc 4. The assessment tasks in this course evaluate my achievement of the learning outcomes. Assessment tasks are those which are rewarded by marks or grades or feedback. Assessment tasks directly assess your achievement of the learning outcomes. 5. Feedback on my work in this course helps me to achieve the learning outcomes. Feedback includes written or verbal comments on your work.

Course Evaluation 6. The workload in this course is appropriate to the achievement of the learning outcomes. Workload includes reading, researching, group activities and assessment tasks. 7. The quality of teaching in this course helps me to achieve the learning outcomes. Quality teaching occurs when knowledgeable and enthusiastic teaching staff interacts positively with students in well-organised teaching and learning experiences. Items 8 -11 ask students to report on their motivation, enthusiasm and commitment in the learning experiences and their overall satisfaction with the course. 8. I am motivated to achieve the learning outcomes in this course. Being motivated means having the desire or drive to learn, to complete tasks and to willingly strive for goals. 9. I make best use of the learning experiences in this course. I prepare for and follow up on the learning experiences offered in this course. 10. I often think about how I can learn more effectively in this course. I take time to think about how I can learn more effectively. 11. Overall, I am satisfied with this course. Overall, this course provides a quality learning experience.

Course Evaluation Qualitative items 12. 13. What are the most helpful aspects of this course? How do you think this course might be improved?

Tutor Evaluation Quantitative Items: 1 -7: using the 1 -4 Likert-type Scale 1: Strongly Disagree; 2: Disagree; 3: Agree; 4: Strongly Agree for each item. [The Tutor's name]: ………………… 1) Is knowledgeable in the subject area 2) Is enthusiastic in teaching the course The tutor has a good understanding of the subject. The tutor makes the subject interesting and conveys his or her enthusiasm for the subject. 3) Is well organised 4) Communicates clearly 5) Is approachable The tutor has material prepared on time, is punctual and structures activities in ways that help learning. The tutor is easy to understand in face-to-face, online, written and other formats and explains concepts clearly. The tutor encourages students to ask questions and seek help.

Tutor Evaluation 6) Provides useful feedback The tutor provides timely and helpful feedback so you can learn. 7) Is an effective tutor Overall, this tutor helps you to learn. Qualitative items 8) Please comment on the tutor's teaching strengths. 9) Please comment on how you think the tutor might improve the teaching and learning in this course

Thank you