LESSONS LEARNED FROM SUMMON USABILITY TESTING Lydia Pyburn
LESSONS LEARNED FROM SUMMON USABILITY TESTING Lydia Pyburn Off-Campus Services Librarian Mitch Stepanovich Arts and Humanities Liaison University of Texas at Arlington
SPECIAL THANKS Xin Wang, University of North Texas Molly Shade, University of North Texas Derek Reece, University of Texas at Arlington Gretchen Trkay, University of Texas at Arlington Fen Lu, University of Texas at Arlington Peter Zhang, University of Texas at Arlington
BACKGROUND In Fall 2013, UTA Libraries launched the discovery tool Summon. A need to conduct a usability study to determine effectiveness of Summon to gather information to improve the discovery tool. The study was a collaboration with University of North Texas.
KNOWN ITEM SEARCHES Classic Catalog Summon
DEVELOPING TASKS User complaints Helpdesk tickets Chat transcripts Talk to service desk staff Curiosity
USABILITY TESTING TASKS Keywords Task Great Gatsby Locate physical copy of book Obesity Prevention Find peer-reviewed journal articles Microsoft Excel Find relevant materials in different formats Martin Luther King, Jr. Find a newspaper article within a date range Course Reserves Identify which editions are reserved for a course Roommate’s Book Troubleshoot when a known item doesn’t appear
INSTITUTIONAL REVIEW BOARD (IRB) A board to review human subject research to ensure the rights and welfare of participants. It ensures the research is ethical and protects the privacy of the participants.
IRB APPROVAL Looked at previous library IRB submissions Human Subjects Protection training Everything must be decided on submittal Overestimate than underestimate Allow time for submission
MARKETING AND RECRUITMENT
MARKETING THE STUDY • • • Worked with Marketing and Communications department within the library Wanted to get faculty and students Mavwire and Trailblazer Library digital signage Library website Flyers around campus
RECRUITMENT FLYER
RECRUITMENT SURVEY
RECRUITMENT SURVEY
SCHEDULING
SCHEDULING TESTING Look at school calendar to avoid scheduling during or around midterms or finals Have a quiet space without distractions Test the software and hardware Count on about 20% of who you scheduled to test Send reminders
SCHEDULING SURVEY
ASSESSMENT
A COMBINED APPROACH Quantitative Qualitative Morae-behavior tracking software Surveys Think aloud interviewing during tasks Exit interviews
MOVING FORWARD Moved chat box top of page A/B/N testing of library homepage Tutorial
QUESTIONS? llpyburn@uta. edu
- Slides: 20