Analytics Planning Considerations Jacqueline Bichsel John Fritz Margie
Analytics: Planning Considerations Jacqueline Bichsel, John Fritz, Margie Jantti, and Celeste Schwartz with Sondra Smith EDUCAUSE 2012 | Preconference Seminar 10 P | November 6, 2012
Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations
Seminar Speakers Jacqueline Bichsel Senior Research Analyst, EDUCAUSE John Fritz Assistant Vice President, Instructional Technology & New Media University of Maryland, Baltimore County Margie Jantti University Librarian, University of Wollongong Celeste M. Schwartz Vice President for Information Technology and College Services Montgomery County Community College Sondra Smith Director, Analytics Outreach and Education, EDUCAUSE
(Selections from) a Year of Analytics http: //www. educause. edu/library/analytics “SLIDER BAR” TOOL ANALYTICS SPRINT ECAR ANALYTICS STUDY
Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations
The 2012 Study of Analytics in Higher Education Jacqueline Bichsel, Ph. D. Senior Research Analyst, ECAR
Defining Analytics
EDUCAUSE Definition of Analytics § Analytics is the use of data, statistical analysis, and explanatory and predictive models to gain insights and act on complex issues.
Analytics Is a Priority
But Analytics Use Is Lagging …particularly in areas of administrative and faculty activities
Higher Education’s Progress
ECAR Analytics Maturity Index
Benchmark Your Institution § Measure your progress on the ECAR Analytics Maturity Index § http: //www. educause. edu/library/resources/ 2012 -ecar-study-analytics-higher-education
Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing Q & A and evaluations
About Blackboard @ UMBC • Began using Bb in Spring 2000 • Current version: 9. 1, SP 6 • Adoption – – 95% of all students 75% of all instructors 65% of all courses 350 Communities • Analytics Initiatives – – – 2012 Bb A 4 L LFT & Implementation 2011 Learning Analytics in Gateway Courses sub-grant 2010 Adopted i. Strategy for analysis of all Bb courses 2009 Code release at Bb. World 2009 2008 Check My Activity for Students available 2007 www. umbc. edu/blackboard/reports • Support Staff: – 2 FTE (Admin & Support) – 1 Server Admin
www. umbc. edu/blackboard/reports
Bb Activity by Grade Distribution
New Method: Bb A 4 L (formerly i. Strategy) Bb. A 4 L Data Warehouse (Queries) Student Admin. (Grades, Demographics) Blackboard (Activity) Check My Activity (Students) Most Active Bb Courses (Faculty) www. umbc. edu/blackboard/reports
Accesses by Grade (SP 2012)
New Reports
Evolving CMS Use by Faculty 1. User & Document Management (Pull) – – Password-protected class & group space Attach or Copy/Paste Documents (expiration) 2. Communications (Push) – – – Announcements Email, Messages Discussion & Chat 3. Assessments (Push & Pull) – – – Electronic assignment delivery & collection Quizzing, Surveys, Course Usage Adaptive release of content based on prior student action.
Tim Hardy & ECON 122 Tim attends hybrid course design workshop
Grade Center Impact on Activity
Personal Analytics in Action
Can IT Architect User Choices?
Demo & Practice Bb Analytics for Learn (BA 4 L) Instance
Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing Q & A and evaluations
Impact and effect A study on library use and student performance Margie Jantti, University Librarian University of Wollongong Library, Australia, 2012
There are many approaches to evaluating the value of libraries. As an academic library, the focus is on the transformative power of information
Typical measures • Satisfaction measures • Rankings • Contingency valuation • Information literacy assessment • Usage rates Limitations • Scale, e. g. samples versus population • Subjective • Often one-dimensional Limited focus on the transformative power of information
Problem statement I: Does a student’s academic performance improve through the use of library information resources?
Problem statement II: How do we know which students make little or no use of library information resources?
Libraries and other units produce lots of discrete data
The challenge was to build a relational database (or cube) • books • ereadings • databases • ebooks • student grades
Data sources for the Cube Student data – PIU Loans – snapshot exported weekly Electronic resource usage – ezyproxy logs
Eresources: >Authentication logs (ezyproxy) >Monitor use in 10 minute blocks (144 ten minute periods p/day) >When used – database name captured Other benefits – logs are updated weekly
Figure 1. Correlation between electronic resource usage and student grades
Undergraduate student resource usage (Faculty of Commerce) Table 1. Weighted Average Marks (WAM) for Undergraduate Students (Faculty of Commerce) Electronic resources: ejournals, ebooks, ereadings etc
Is it perfect? >Some limitations in correlations >Arbitary measures; business rules >Many external factors affect grades, e. g. academic influence
But……. A first at UOW Library: >for true integration of data silos >getting answers to our problem statements >evaluating our communication and intervention strategies >for a new way of demonstrating the value of the Library
QUESTIONS?
Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations
Data-Informed Decision-Making throughout the Institution Celeste M. Schwartz, Ph. D. Vice President for Information Technology and College Services
What is the Question? What Data can help answer the Question? What Actions will be recommended based on the data?
Astra – Room Scheduling
Student Success
Enrollment Analysis
Help Desk Weekly Lab Usage • Central Library – 70% utilization at 12: 00 pm(noon) Wednesday 10/3/2012 • West Library – 70% utilization at 11: 00 am Monday 10/2/2012 • CH Tutoring – 30% utilization at 10: 00 am Monday 10/1/2012 • West Library Laptops – 20% utilization at 7: 00 pm Monday 10/1/2012 • Central Library Laptops – 20% utilization at 12: 00 pm(noon) Thursday 10/4/2012 Monday Tuesday
Total number of tickets created: 456 Summary of Weekly calls by Status & Assignment.
Total Calls Per Hour 10: 00 pm 0 9: 00 pm 12 8: 00 pm 1 6 7: 00 pm 11 5 0 5 6: 00 pm 15 5: 00 pm 4 4 4: 00 pm 6 3: 00 pm 6 4 1 4 4 8 2: 00 pm 6 14 1: 00 pm 9 19 6 7: 00 am 5 0 7 8 2 11 5 6 7 14 8: 00 am 5 9 12 9: 00 am Password ID 6 14 10: 00 am Helpdesk/other 8 9 11: 00 am Classroom emergency 4 12 12: 00 pm 3 5 5 2 10 15 20 25 30 35
Help Desk Feedback Report Date Representative 10/1/2012 robert gehring 5: 02 PM 10/1/2012 Robert Gehring 6: 43: 32 PM 10/2/2012 Jasmyne Smith 3: 07: 37 PM 10/3/2012 Ryan Foster 4: 37: 26 PM Email Not provided Problem Resolved? Service Rating Yes Very Satisfied Not provided Yes Satisfied 10/4/2012 Jasmin 10: 08: 00 AM 10/4/2012 Jennifer 11: 55: 31 AM 10/4/2012 Jen 1: 44: 50 PM Not provided Yes Very Satisfied Great. Thanks dalder@mc 3. edu Yes Very Satisfied Not provided Yes Very Satisfied This is the third time I've contacted the help desk. On each occasion, you were EXTREMELY patient and helped me resolve the problem. Additionally, you do it WITHOUT making me feel like an idiot! Thank you! Kathy is always willing to listen and ask questions until everyone is clear on what the difficulty/need is! Many, many thanks to Kathy and Mary Lou! Robert is remarkable and we are very fortunate to have him. Because it was so late at night when my computer crashed, he was able to talk me through re-booting so that I could finish the class and not waste time and inconvenience all my students. 10/5/2012 Kathy Miller 8: 53: 06 AM 10/6/2012 Robert 11: 39: 35 AM Comments Representative was very helpful, and answered all my questions. He was fast, polite and extremely helpful. Thank you. Ryan did well to resolve my loss of available drives. It did take almost two full weeks to resolve this issue after first being reported on 9/20/12. Sincere thanks, Lee
What is the Question? What Data can help answer the Question? What Actions will be recommended based on the data?
Now it’s Your Turn What Actions will you recommend based on the Data? • Read the daily enrollment email from the VP for Student Affairs based on the daily updated warehouse data. • Take ten minutes with your team to discuss recommended actions • Each team will report out
Data-Informed Decision-Making throughout the Institution QUESTIONS Cschwartz@mc 3. edu
Seminar Agenda • Introductions • Analytics and the maturity index • John Fritz and academic analytics at UMBC • Break • Margie Jantti and the Wollongong library cube • Break • Celeste Schwartz and data informed decisions at MCCC • Closing questions, comments, and evaluations
Thank You. § § § jbichsel@educause. edu fritz@umbc. edu margie@uow. edu. au cschwartz@mc 3. edu srsmith@educause. edu
- Slides: 71