LESSONS LEARNED FROM SUMMON USABILITY TESTING Lydia Pyburn

LESSONS LEARNED FROM SUMMON USABILITY TESTING Lydia Pyburn Off-Campus Services Librarian Mitch Stepanovich Arts and Humanities Liaison University of Texas at Arlington

SPECIAL THANKS Xin Wang, University of North Texas Molly Shade, University of North Texas Derek Reece, University of Texas at Arlington Gretchen Trkay, University of Texas at Arlington Fen Lu, University of Texas at Arlington Peter Zhang, University of Texas at Arlington

BACKGROUND In Fall 2013, UTA Libraries launched the discovery tool Summon. A need to conduct a usability study to determine effectiveness of Summon to gather information to improve the discovery tool. The study was a collaboration with University of North Texas.

KNOWN ITEM SEARCHES Classic Catalog Summon

DEVELOPING TASKS User complaints Helpdesk tickets Chat transcripts Talk to service desk staff Curiosity

USABILITY TESTING TASKS Keywords Task Great Gatsby Locate physical copy of book Obesity Prevention Find peer-reviewed journal articles Microsoft Excel Find relevant materials in different formats Martin Luther King, Jr. Find a newspaper article within a date range Course Reserves Identify which editions are reserved for a course Roommate’s Book Troubleshoot when a known item doesn’t appear

INSTITUTIONAL REVIEW BOARD (IRB) A board to review human subject research to ensure the rights and welfare of participants. It ensures the research is ethical and protects the privacy of the participants.

IRB APPROVAL Looked at previous library IRB submissions Human Subjects Protection training Everything must be decided on submittal Overestimate than underestimate Allow time for submission

MARKETING AND RECRUITMENT

MARKETING THE STUDY • • • Worked with Marketing and Communications department within the library Wanted to get faculty and students Mavwire and Trailblazer Library digital signage Library website Flyers around campus

RECRUITMENT FLYER

RECRUITMENT SURVEY

RECRUITMENT SURVEY

SCHEDULING

SCHEDULING TESTING Look at school calendar to avoid scheduling during or around midterms or finals Have a quiet space without distractions Test the software and hardware Count on about 20% of who you scheduled to test Send reminders

SCHEDULING SURVEY

ASSESSMENT

A COMBINED APPROACH Quantitative Qualitative Morae-behavior tracking software Surveys Think aloud interviewing during tasks Exit interviews

MOVING FORWARD Moved chat box top of page A/B/N testing of library homepage Tutorial

QUESTIONS? llpyburn@uta. edu