Making Library Assessment Work Practical Approaches for Developing

Making Library Assessment Work: Practical Approaches for Developing and Sustaining Effective Assessment Phase I Update ARL VISITING PROGRAM OFFICERS 2004 -06 Steve Hiller Library Assessment Coordinator, University of Washington Jim Self Director, Management Information Services, University of Virginia & Martha Kyrillidou ARL, Director of Statistics and Measurement Program

ARL New Measures 1999 Key Areas • User Satisfaction/Service Quality – Lib. QUAL+™ – Measuring Library Service Quality Lyceum • Market Penetration • Ease and Breadth of Access • Library Impact on Teaching and Learning – Project SAILS, Learning Outcomes • • Library Impact on Research Cost Effectiveness Library Facilities/Space E-Metrics – MINES for libraries – ARL supplementary statistics

Barriers to Management Information and Using Data Effectively in Libraries • • Organizational culture/leadership support Time/Staff/Resources Data issues – Too much, compatibility, validity Establishing priorities Knowing what to measure and methods to use Inexperience, perceived lack of skills and expertise Understanding, presenting and knowing what to do with the results Hiller, S. and Self, J. (2004). From Measurement to Management: Using Data Wisely for Planning and Decision-Making. Library Trends.

Making Library Assessment Work: Practical Approaches for Developing and Sustaining Effective Assessment • 2 year ARL project to assist libraries with moving assessment efforts forward • Led by Steve Hiller (UW) and Jim Self (UVa) • Recognition that libraries understand value of assessment but have trouble sustaining efforts and using results • Site visits conducted to evaluate assessment and develop practical approaches to effective local practices • Identify common barriers and facilitators to assessment • 7 libraries participating in Phase I Spring 2005; 12 -14 during 2005 -06 academic year; final report in 2006

Phase I Participants • • University of Arizona State University of Connecticut University of Illinois at UC New York University of Notre Dame University of Oregon February 2005 March 2005 May 2005 March 2005 June 2005 May 2005

Phase I Process • Each library chooses a contact person • Survey sent to contact person 2 months before visit – Assessment activities, motivators, organizational structure – What’s worked/hasn’t, specific areas to address, expectations • Schedule of meetings established • Site visit • Short report completed within 30 days – Specific recommendations made on moving forward • Library evaluates process after final report received

Sample Site Visit Schedule • Meet with University Librarian/Contact person • Presentation on effective assessment – 90 minutes to 2 hours with Q&A and discussion • Group Meetings – – with management/administrative group assessment-related group (if formed different departments functional areas (e. g. info literacy) • Wrap-up

ARL Project Preliminary Observations after Phase I Visits • • • No “cookie-cutter” libraries; diverse organizational cultures Sustainability seen as resource issue Management information system seen as crucial underpinning Integrating data use effectively into library management Many internal statistics (non-ARL related) kept Few libraries allocate sufficient resources (staff and funding) for assessment or management information Facilities renovation as catalyst to good assessment practices Good assessment work being done but not published/ disseminated or coordinated Need for basic/practical training on assessment methods/analysis Little understanding of what’s happening outside the library in data warehousing/management

Phase I Feedback from Participants • • • One day is too short More resource materials would be helpful More examples presented Follow-up activity would maintain momentum Establishing an assessment-related “community” of practitioners would assist ongoing communication and development

Phase II Changes • Site visit time increased from 1 day to 1 1/2 days • Appropriate resource materials provided • Consulting on a follow-up activity – Assessment plan implementation – Specific assessment effort • Individual meeting at a professional conference • Group meeting at a professional conference • Stronger project Web presence

Phase II Participants • Confirmed – – – – – Cornell Emory Kansas Louisville Pennsylvania State Southern California Texas Tech Wayne State Western Ontario • Willing to commit (not confirmed) – – Boston Public Library Colorado State Nebraska Purdue • Interest pending (maybe) – Alabama – Alberta – Marquette

Future Reporting • Upcoming Presentations – Northumbria International Conference on Performance Measurement, Durham, England, August 22 -25 – 3 rd International Evidence-Based Librarianship Conference, Brisbane, Australia, October 16 -19 – ARL Membership Meeting, Washington, D. C. , October 25 -28 or in May 2006 • Making Library Assessment Work Web Site – http: //www. arl. org/stats/newmeas/VPO_Hiller_Self. html • Final Report late summer 2006 • ARL Newsletter • Conference in Fall 2006?