Analytics Planning Considerations Jacqueline Bichsel John Fritz Margie

Analytics: Planning Considerations Jacqueline Bichsel, John Fritz, Margie Jantti, and Celeste Schwartz with Sondra Smith EDUCAUSE 2012 | Preconference Seminar 10 P | November 6, 2012

Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations

Seminar Speakers Jacqueline Bichsel Senior Research Analyst, EDUCAUSE John Fritz Assistant Vice President, Instructional Technology & New Media University of Maryland, Baltimore County Margie Jantti University Librarian, University of Wollongong Celeste M. Schwartz Vice President for Information Technology and College Services Montgomery County Community College Sondra Smith Director, Analytics Outreach and Education, EDUCAUSE

(Selections from) a Year of Analytics http: //www. educause. edu/library/analytics “SLIDER BAR” TOOL ANALYTICS SPRINT ECAR ANALYTICS STUDY

Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations

The 2012 Study of Analytics in Higher Education Jacqueline Bichsel, Ph. D. Senior Research Analyst, ECAR

Defining Analytics

EDUCAUSE Definition of Analytics § Analytics is the use of data, statistical analysis, and explanatory and predictive models to gain insights and act on complex issues.

Analytics Is a Priority

But Analytics Use Is Lagging …particularly in areas of administrative and faculty activities

Higher Education’s Progress

ECAR Analytics Maturity Index

Benchmark Your Institution § Measure your progress on the ECAR Analytics Maturity Index § http: //www. educause. edu/library/resources/ 2012 -ecar-study-analytics-higher-education

Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing Q & A and evaluations

About Blackboard @ UMBC • Began using Bb in Spring 2000 • Current version: 9. 1, SP 6 • Adoption – – 95% of all students 75% of all instructors 65% of all courses 350 Communities • Analytics Initiatives – – – 2012 Bb A 4 L LFT & Implementation 2011 Learning Analytics in Gateway Courses sub-grant 2010 Adopted i. Strategy for analysis of all Bb courses 2009 Code release at Bb. World 2009 2008 Check My Activity for Students available 2007 www. umbc. edu/blackboard/reports • Support Staff: – 2 FTE (Admin & Support) – 1 Server Admin

www. umbc. edu/blackboard/reports

Bb Activity by Grade Distribution

New Method: Bb A 4 L (formerly i. Strategy) Bb. A 4 L Data Warehouse (Queries) Student Admin. (Grades, Demographics) Blackboard (Activity) Check My Activity (Students) Most Active Bb Courses (Faculty) www. umbc. edu/blackboard/reports

Accesses by Grade (SP 2012)

New Reports

Evolving CMS Use by Faculty 1. User & Document Management (Pull) – – Password-protected class & group space Attach or Copy/Paste Documents (expiration) 2. Communications (Push) – – – Announcements Email, Messages Discussion & Chat 3. Assessments (Push & Pull) – – – Electronic assignment delivery & collection Quizzing, Surveys, Course Usage Adaptive release of content based on prior student action.

Tim Hardy & ECON 122 Tim attends hybrid course design workshop

Grade Center Impact on Activity

Personal Analytics in Action

Can IT Architect User Choices?

Demo & Practice Bb Analytics for Learn (BA 4 L) Instance

Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing Q & A and evaluations

Impact and effect A study on library use and student performance Margie Jantti, University Librarian University of Wollongong Library, Australia, 2012

There are many approaches to evaluating the value of libraries. As an academic library, the focus is on the transformative power of information

Typical measures • Satisfaction measures • Rankings • Contingency valuation • Information literacy assessment • Usage rates Limitations • Scale, e. g. samples versus population • Subjective • Often one-dimensional Limited focus on the transformative power of information

Problem statement I: Does a student’s academic performance improve through the use of library information resources?

Problem statement II: How do we know which students make little or no use of library information resources?

Libraries and other units produce lots of discrete data

The challenge was to build a relational database (or cube) • books • ereadings • databases • ebooks • student grades

Data sources for the Cube Student data – PIU Loans – snapshot exported weekly Electronic resource usage – ezyproxy logs

Eresources: >Authentication logs (ezyproxy) >Monitor use in 10 minute blocks (144 ten minute periods p/day) >When used – database name captured Other benefits – logs are updated weekly

Figure 1. Correlation between electronic resource usage and student grades

Undergraduate student resource usage (Faculty of Commerce) Table 1. Weighted Average Marks (WAM) for Undergraduate Students (Faculty of Commerce) Electronic resources: ejournals, ebooks, ereadings etc

Is it perfect? >Some limitations in correlations >Arbitary measures; business rules >Many external factors affect grades, e. g. academic influence

But……. A first at UOW Library: >for true integration of data silos >getting answers to our problem statements >evaluating our communication and intervention strategies >for a new way of demonstrating the value of the Library

QUESTIONS?

Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations

Data-Informed Decision-Making throughout the Institution Celeste M. Schwartz, Ph. D. Vice President for Information Technology and College Services

What is the Question? What Data can help answer the Question? What Actions will be recommended based on the data?

Astra – Room Scheduling

Student Success

Enrollment Analysis

Help Desk Weekly Lab Usage • Central Library – 70% utilization at 12: 00 pm(noon) Wednesday 10/3/2012 • West Library – 70% utilization at 11: 00 am Monday 10/2/2012 • CH Tutoring – 30% utilization at 10: 00 am Monday 10/1/2012 • West Library Laptops – 20% utilization at 7: 00 pm Monday 10/1/2012 • Central Library Laptops – 20% utilization at 12: 00 pm(noon) Thursday 10/4/2012 Monday Tuesday

Total number of tickets created: 456 Summary of Weekly calls by Status & Assignment.

Total Calls Per Hour 10: 00 pm 0 9: 00 pm 12 8: 00 pm 1 6 7: 00 pm 11 5 0 5 6: 00 pm 15 5: 00 pm 4 4 4: 00 pm 6 3: 00 pm 6 4 1 4 4 8 2: 00 pm 6 14 1: 00 pm 9 19 6 7: 00 am 5 0 7 8 2 11 5 6 7 14 8: 00 am 5 9 12 9: 00 am Password ID 6 14 10: 00 am Helpdesk/other 8 9 11: 00 am Classroom emergency 4 12 12: 00 pm 3 5 5 2 10 15 20 25 30 35

Help Desk Feedback Report Date Representative 10/1/2012 robert gehring 5: 02 PM 10/1/2012 Robert Gehring 6: 43: 32 PM 10/2/2012 Jasmyne Smith 3: 07: 37 PM 10/3/2012 Ryan Foster 4: 37: 26 PM Email Not provided Problem Resolved? Service Rating Yes Very Satisfied Not provided Yes Satisfied 10/4/2012 Jasmin 10: 08: 00 AM 10/4/2012 Jennifer 11: 55: 31 AM 10/4/2012 Jen 1: 44: 50 PM Not provided Yes Very Satisfied Great. Thanks dalder@mc 3. edu Yes Very Satisfied Not provided Yes Very Satisfied This is the third time I've contacted the help desk. On each occasion, you were EXTREMELY patient and helped me resolve the problem. Additionally, you do it WITHOUT making me feel like an idiot! Thank you! Kathy is always willing to listen and ask questions until everyone is clear on what the difficulty/need is! Many, many thanks to Kathy and Mary Lou! Robert is remarkable and we are very fortunate to have him. Because it was so late at night when my computer crashed, he was able to talk me through re-booting so that I could finish the class and not waste time and inconvenience all my students. 10/5/2012 Kathy Miller 8: 53: 06 AM 10/6/2012 Robert 11: 39: 35 AM Comments Representative was very helpful, and answered all my questions. He was fast, polite and extremely helpful. Thank you. Ryan did well to resolve my loss of available drives. It did take almost two full weeks to resolve this issue after first being reported on 9/20/12. Sincere thanks, Lee

What is the Question? What Data can help answer the Question? What Actions will be recommended based on the data?

Now it’s Your Turn What Actions will you recommend based on the Data? • Read the daily enrollment email from the VP for Student Affairs based on the daily updated warehouse data. • Take ten minutes with your team to discuss recommended actions • Each team will report out

Data-Informed Decision-Making throughout the Institution QUESTIONS Cschwartz@mc 3. edu

Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations

Thank You. § § § jbichsel@educause. edu fritz@umbc. edu margie@uow. edu. au cschwartz@mc 3. edu srsmith@educause. edu