Learning Management System Survey LMS Taskforce October 2018

I. Respondent Characteristics

Respondent College - Frequency 45 40 40 35 The overall sample consisted of 144 individuals 32 30 27 25 20 15 15 15 10 The majority of the sample (69%) came from three colleges: 5 7 2 3 2 0 AG BUS EBS ECS CLAC NHP SM US Respondent College - Percentage of Sample • • • US 2% Liberal Arts and Communications Nursing and Health Professions Education and Behavioral Science Those selecting “Other”: Library (3), AOS (1), Emergency Management (1), Global Outreach & Engagement (1), and ISS (1) AG 1% Other 5% SM 11% BUS 11% EBS 19% NHP 22% ECS 1% CLAC 28% Other

Respondent Teaching Load (Spring 2018) The majority of the sample (75%) taught three or more courses during the Spring 2018 semester RESPONDENT TEACHING LOAD (SPRING 2018) 5+ Courses 17% 1 Course 10% 2 Courses 15% • Three courses was the most endorsed option • Ten respondents were not employed by AState in Spring 2018 4 Courses 19% 3 Courses 38%

Respondent BBL Use (Spring 2018) The majority of the sample (70%) reported using Blackboard Learn for 3 or more courses during Spring 2018 Only 3 respondents indicated they did not use Blackboard Learn during Spring 2018 Respondents’ teaching load and Blackboard Learn use are highly related – most used Blackboard Learn in most or all of their courses during Spring 2018 Respondent BBL Use (Spring 2018) 5+ Courses 15% Not at all 2% 1 Course 11% 2 Courses 17% 4 Courses 20% 3 Courses 35%

II. Blackboard Learn (BBL)

How Do Respondents Use BBL? Use of Blackboard Learn Frequency of BBL Use Disseminate information Promote interactions out of class 75, 6% Grade student assignments/exams 74, 8% Communicate with students Less than monthy 1% Weekly 16% 62, 6% Conduct course assignments/exams 0, 0% Monthly 1% 98, 5% 92, 4% 20, 0% 40, 0% 60, 0% 80, 0% 100, 0% Daily 82% All of the provided forms of BBL use were highly endorsed The majority of respondents use BBL on a daily basis during a typical semester

Perceptions of BBL Overall respondents had positive perceptions of the role of BBL in education: Perceptions of BBL useful tool to enhance student learning 7, 0% 5, 4% 4, 7% 22, 5% 60, 5% • Useful tool to enhance learning (M = 4. 24) 3, 8% BBL useful tool to enhance my teaching 7, 7% 4, 6% 16, 2% 67, 7% 3, 8% BBL critical to my teaching 8, 5% 0, 0% 7, 7% 10, 0% Strongly Disagree 13, 1% 20, 0% Disagree 30, 0% 66, 9% 40, 0% Neutral 50, 0% Agree 60, 0% Strongly Agree 70, 0% 80, 0% 90, 0% 100, 0% • Useful tool to enhance teaching (M = 4. 32) • Critical to teaching (M = 4. 26)

Satisfaction with BBL Overall staisfaction 8, 6% 43, 1% Ease of use 37, 9% 36, 6% Initial use training 12, 9% 13, 7% Posting content 28, 2% Managing assignments 8, 9% Entering student progress 46, 6% 33, 1% 61, 1% 27, 4% 58, 9% 6, 3% 27, 6% Receiving student assignments 9, 8% Engaging in meaningful interactions with students 17, 5% 34, 7% 57, 5% 25, 4% 54, 9% 31, 6% 42, 1% 0, 0% 10, 0% 20, 0% 30, 0% 40, 0% 50, 0% 60, 0% 70, 0% 80, 0% 90, 0% 100, 0% Very Dissatisfied Neutral Overall, respondents reported high levels of satisfaction with Blackboard Learn Satisfied Very Satisfied Satisfaction with initial training and instructions did receive more dissatisfaction ratings compared to other aspects of BBL

III. Desired LMS Features

Features Desired in an LMS Respondents were asked to indicate the importance of six core sets of features for any LMS being considered by A-State: • • • Communication Content Grading Gradebook Assessment Miscellaneous Features

Features Desired in an LMS Importance ratings were analyzed in two ways: 1. Endorsement frequency for each response category 2. The mean ratings were tested to determine if they were significantly higher or lower than a rating of 3 (i. e. “of average importance”) • The effect size (Cohen’s d) was also calculated to gauge whether features were overall deemed very important, of average importance, or unimportant Absolute values of Cohen’s d are conventionally interpreted as indicating no effect (<. 20), a small effect (. 20 -. 50), a moderate effect (. 50 -. 80), or a large effect (≥. 80)

Desired Communication Features Deemed Very Important COMMUNICATION FEATURES Course announcements Course calendar 4, 9% 13, 4% 8, 0% 15, 2% 14, 5% 29, 0% 34, 1% 23, 6% Chat (multimedia) 33, 9% 23, 4% Email integration 6, 5% 10, 9% 4, 3%7, 2% Video conferencing Wikis/blogs 0, 0% Not All Important 10, 9% 10, 8% 18, 5% Cohen’s d Course announcements 4. 69 2. 33 Course calendar 3. 64 0. 50 11, 4% Discussion boards 3. 96 0. 75 12, 1% Email integration 4. 29 1. 10 33, 3% 13, 0% Features Deemed of Average Importance 67, 6% 20, 9% 14, 0% 23, 4% Of Little Importance 13, 7% 47, 8% 10, 1% 20, 0% 17, 9% 16, 9% 23, 9% 28, 1% 10, 0% Mean 79, 6% Chat (text only) Discussion boards Feature 30, 0% 40, 0% 20, 2% 26, 4% 22, 7% 50, 0% Of Average Importance 60, 0% 14, 8% 70, 0% Somewhat Important 80, 0% 10, 9% 90, 0% Very Important 100, 0% Feature Mean Cohen’s d Video conferencing 3. 15 0. 10 Features Deemed Unimportant Feature Mean Cohen’s d Chat (text only) 2. 49 -0. 36 Chat (multimedia) 2. 47 -0. 38 Wikis/blogs 2. 57 -0. 32

Desired Content Features Deemed Very Important CONTENT FEATURES Ability to make organization Access files across classes 18, 9% 7, 3% Link to external sites Media integration OER integration 19, 8% 7, 3% 4, 3%8, 0% 3, 6% 4, 3% 10, 2% 0, 0% 10, 2% 20, 0% Of Little Importance Ability to make organization 3. 34 0. 21 Access files across classes 4. 09 0. 85 69, 6% Link external sites 4. 36 1. 17 70, 5% Media integration 4. 46 1. 41 OER integration 3. 81 0. 60 16, 1% 57, 7% 27, 3% 44, 5% 10, 0% Cohen’s d 38, 7% 17, 3% 10, 2% Mean 14, 4% 5, 1% 13, 0% Social media integration Not All Important 11, 7% 8, 1% Feature 42, 2% 19, 5% 30, 0% 40, 0% 50, 0% Of Average Importance 60, 0% 18, 8% 70, 0% Somewhat Important 8, 6% 80, 0% 90, 0% Very Important 8, 6% 100, 0% Feature Deemed Unimportant Feature Mean Cohen’s d Social media integration 2. 17 -0. 63

Desired Grading Features Deemed Very Important GRADING FEATURES Quizzes/Exams 2, 9% 7, 9% Question pools 7, 4% Question randomization Add comments to assignments Grade with rubric 9, 7% 10, 2% 13, 4% 4, 3% 5, 7% 10, 9% 0, 0% 14, 7% 53, 7% 14, 6% 60, 6% 12, 7% 23, 1% 41, 0% 20, 1% 61, 9% 15, 0% 5, 8% 9, 4% 72, 1% 18, 0% 21, 9% 20, 0% Of Little Importance Feature Mean Cohen’s d Quizzes/Exams 4. 47 1. 37 Question pools 3. 99 0. 76 Question randomization 4. 13 0. 87 Surveys 3. 72 0. 53 Plagiarism detection 4. 30 1. 19 Add comments to assignments 4. 48 1. 44 Grade with rubric 4. 26 1. 09 Peer review 3. 25 0. 19 75, 7% 16, 2% 3, 6% 6, 5% 7, 9% Peer review Not All Important 8, 1% 8, 0% 6, 6% Surveys Plagiarism detection 4, 3%9, 3% 61, 9% 23, 4% 30, 0% 40, 0% 50, 0% Of Average Importance 18, 8% 60, 0% 70, 0% Somewhat Important 25, 0% 80, 0% 90, 0% Very Important 100, 0%

Desired Gradebook Features Deemed Very Important GRADEBOOK FEATURES Delete columns 11, 3% 85, 1% Make customized reports 5, 1% 10, 9% Weight grades 8, 7% 5, 8% 11, 6% Order columns as desired Import/export to spreadsheet 73, 0% 10, 9% 63, 0% 7, 8% 5, 7% 7, 9% Add specialized column 0, 0% Not All Important 8, 0% 6, 3% 10, 0% 87, 2% 4, 3% 13, 6% 68, 6% 12, 7% 20, 0% Of Little Importance 76, 1% 30, 0% 40, 0% 50, 0% Of Average Importance 60, 0% 70, 0% Somewhat Important 80, 0% 90, 0% Very Important 100, 0% Feature Mean Cohen’s d Delete columns 4. 79 2. 88 Make customized reports 4. 41 1. 26 Weight grades 4. 14 0. 86 Order columns as desired 4. 79 2. 69 Import/export to spreadsheet 4. 31 1. 08 Add specialized column 4. 57 1. 71

Desired Assessment Features Deemed Very Important ASSESSMENT FEATURES Collect data for single course 4, 7% 8, 7% Collect data across courses 6, 5% 7, 3% Flexibility of setup for data collection 6, 5% 7, 3% Overcome setup errors Item analysis 6, 8% 10, 0% 62, 9% 15, 4% 65, 0% 21, 5% 10, 3% 7, 6% 10, 3% 20, 0% 30, 0% 54, 6% 24, 1% 17, 8% 6, 5% Of Little Importance 54, 0% 16, 9% 22, 8% 0, 0% 16, 9% 3, 8%9, 2% 14, 4% Use of EAC data Not All Important 6, 5% 16, 4% Rubric-row analysis 63, 8% 15, 3% 9, 4% 10, 8% Psychometric data 18, 1% 38, 8% 16, 9% 19, 6% 40, 0% Of Average Importance 43, 2% 14, 1% 50, 0% 60, 0% 37, 0% 70, 0% Somewhat Important 80, 0% 90, 0% Very Important 100, 0% Feature Mean Cohen’s d Collect data for single course 4. 31 1. 18 Collect data across courses 4. 05 0. 84 Flexibility setup for collection 4. 23 0. 99 Overcome setup errors 4. 28 1. 07 Item analysis 4. 05 0. 79 Psychometric data 3. 59 0. 39 Rubric-row analysis 3. 67 0. 46 Use of EAC data 3. 36 0. 23

Desired Management Features Deemed Very Important MANAGEMENT FEATURES Copy course between semesters 5, 0% 89, 9% Export/archive courses 8, 7% 83, 3% Merge courses 10, 2% 83, 9% Import existing content to new courses 4, 3% Adaptive release Track student access 0, 0% Not All Important 91, 4% 11, 0% 7, 1% 81, 6% 4, 3% 10, 0% Of Little Importance 20, 0% 70, 7% 30, 0% 40, 0% Of Average Importance 50, 0% 60, 0% 70, 0% Somewhat Important 80, 0% 90, 0% Very Important 100, 0% Feature Mean Cohen’s d Copy course between semesters 4. 79 2. 43 Export/archive courses 4. 68 1. 97 Merge courses 4. 75 2. 65 Import to new course 4. 83 2. 78 Adaptive release 4. 69 2. 18 Track student access 4. 41 1. 29

Desired Miscellaneous Features Deemed Very Important MISCELLANEOUS FEATURES 12, 6% Customize tabs 22, 2% Customize landing page 7, 2% 11, 6% Compatibility with publishers 8, 8% Faculty access as a student 6, 3% Faculty access via mobile 8, 7% Student access via mobile Student ID verification (exams) 13, 9% 48, 9% 78, 6% 13, 4% 0, 0% 39, 5% 55, 1% 26, 1% 59, 4% 16, 4% 5, 3%6, 9% 12, 9% 10, 0% 16, 8% 15, 7% 5, 3% 6, 8% 8, 3% 6, 9% 14, 3% 16, 5% 7, 1% Online communities Not All Important 60, 1% 19, 7% 16, 0% Adaptive release Student ID verification (assignments) 19, 6% 4, 3% 13, 6% Tegrity 70, 7% 17, 3% 20, 0% Of Little Importance 62, 4% 13, 7% 18, 5% 67, 2% 14, 5% 30, 0% Feature Mean Cohen’s d Customize tabs 4. 30 1. 27 Customize landing page 4. 30 1. 26 Compatibility with publishers 3. 85 0. 63 Faculty access as student 4. 64 1. 96 Tegrity 3. 50 0. 33 Adaptive release 4. 07 0. 86 Faculty access via mobile 4. 37 1. 45 Student access via mobile 4. 50 1. 61 Student ID verification (assignments) 4. 25 1. 06 Student ID verification (exams) 4. 29 1. 05 Online communities 3. 41 0. 29 58, 5% 40, 0% Of Average Importance 22, 6% 50, 0% 60, 0% 31, 5% 70, 0% Somewhat Important 80, 0% 90, 0% Very Important 100, 0%