Determining the Most Appropriate Types of Assessment in

Determining the Most Appropriate Type(s) of Assessment in Library Instruction Bonnie L. Fong, Physical Sciences Librarian, Rutgers-Newark bonnie. fong@rutgers. edu VALE / NJ ACRL / NJLA CUS 14 th Annual Users' Conference Friday, January 4, 2013

Assessment

Assessment Difficulties • Many possibilities to consider • Many factors to consider Who, What, When, Where, Why, How

Why Assess • For someone else? – College/university – Academic department – Professor – Library • For ourselves? – What are we trying to accomplish?

What to Assess • Evaluation of library instruction session: – Students (e. g. , useful, confusing, enjoyable) – Professor (e. g. , fitting, effect on assignments) – Librarian (e. g. , timing, activities that worked) • Students’ information literacy skills: – Objective knowledge – Higher-order thinking – Application of knowledge (e. g. , paper) • Assessment tool

How to Assess Sessions • Student self-evaluation – Typical one-minute paper: • Questions: – What did you learn today? – Is there anything that is still confusing? • Distribution? – Paper / index card – Online • Quick • Self-reflective

How to Assess Sessions • Student self-evaluation – My modified version (for one-shot sessions): • Did you learn something new? • Do you think the session has helped you with doing the kind of research your ENG 102 assignment requires? • What did you find most helpful about this session? • What would you have liked more of? • What would you have liked less of? • Additional Comments

How to Assess Sessions • Student self-evaluation – Survey in a science writing-intensive course: • Did you find our discussion on XYZ helpful? a) Yes, I found it very helpful and used it for my assignment(s). b) Yes, it was helpful, but I did not wind up using it for my assignment(s). c) No, it was not helpful - I found it unclear. d) N/A - I already knew everything that was discussed. e) N/A - I was not in class that day. • Which of the Reserve items did you use? • Did you find the “Reserve items” handout helpful?

How to Assess Sessions • Student self-evaluation – Survey for graduate students in mini-course: • After our mini-course, how confident are you about using the following (17) resources? • How often do you now use library resources, after our mini-course? • Did you find our mini-course helpful? • What did you find most helpful? • What would you have liked more of? • What suggestions do you have for improvement?

How to Assess Sessions • Student self-evaluation – Survey in a mixed-enrollment seminar course: • Beginning & end: – How confident are you about doing/using the following? (24) – What are some of the challenges or difficulties you are still encountering when looking for or using information? • End: – How helpful do you think the library sessions were/will be for XYZ? – Which resources did you use to do complete your oral presentation project? – What was the most difficult part of the course?

How to Assess Sessions • Professor evaluation – How did the library instruction session go? • Did I cover everything you were hoping I would? • Did anything seem unclear? • Might you have any suggestions for how I could improve the session? – How did the assignment(s) turn out? • Did student assignments indicate retained knowledge? • Did student assignments show improvement?

How to Assess Sessions • Professor evaluation – When: • Right after library instruction session • Within a week after library instruction session • After the end of the semester – How: • In-person • E-mail • Survey

How to Assess Sessions • Librarian (self-)evaluation – Observations during class: • Walking around during student work • Students’ faces – confusion / boredom • Student questions – Activities: • Effectiveness • Student interest – Timing

Assessing Information Literacy • What are we trying to assess? – What are the learning objectives? – What is being taught? • When can we assess? • How can we measure skills/knowledge? – Identify appropriate type(s) of assessment – Develop / identify suitable assessment tool(s): • Content (e. g. , test questions, assignments) • Device (e. g. , software, hardware)

How to Assess Info. Literacy • Objective knowledge: – True/False – Multiple choice – Choose all that apply – Matching – Matrix – Short answer completion

How to Assess Info. Literacy • Higher-order thinking – Typically: • • Essay Long answer Oral response Student demonstration – Maybe: • True/False • Multiple choice • Choose all that apply

Ex: Multiple Choice

Ex: Choose All That Apply Choose all that apply: C 3. Where should you look to find spectra (e. g. , IR, NMR, UV) information? a) CRC Handbook of Chemistry and Physics b) Pub. Med c) Reaxys d) Sci. Finder e) Web of Science

Ex: Matrix / Likert Scale

Ex: Short / Long Answer 1. Find all the items in Sci. Finder that’s been published in English by your assigned seminar speaker. a) What is your search strategy? Please be specific. i. Which section of Sci. Finder are you searching in? ii. What did you type in the search box? iii. If applicable, which of the “Author Name Candidates” did you choose? b) Identify the one that was cited the most. i. ii. How many times was it cited? Provide the citation in Nature formatting. c) How many of the references are journal articles? d) What has your assigned seminar speaker’s research focused on since 2007? How did you determine this? Please be specific. 2. Identify ONE person who has been doing research on the same topic as your assigned seminar speaker and might be considered a potential collaborator or competitor. This person cannot be someone s/he has co-written a paper with since 2007. How did you determine this (i. e. , where did you look)?

How to Assess Info. Literacy • Application of knowledge – In-class exercises – Evaluate assignment(s): • • Annotated bibliographies Papers Posters Oral presentations – Obstacles: • Permissions • Time

Ex: Rubric / Checklist • Oral presentation:

How to Assess Info. Literacy • Application of knowledge – Process: • Demonstration / record thought process • Search history • Journal – Reflection – Choices made – Obstacles: • Time • Permissions

When to Assess Info. Literacy • Pre- & post session • Pre-session: – Before session – At start of session • During session • Post-session – At end of session – Shortly afterwards – Some time later

Limitations to Assessment • No magic bullet – Different assessment tools for measuring different things – May need more than 1 method • General limitations: – Time – Resources – Some things are difficult to measure

Notes • Suggested further reading: Radcliff, C. J. , Jensen, M. L. , Salem Jr. , J. A. , Burhanna, K. J. , & Gedeon, J. A. (2007). A practical guide to information literacy assessment for academic librarians. Westport, Conn: Libraries Unlimited. • Unless otherwise noted, all images come from Microsoft Office’s clip art

Questions? Thank you! bonnie. fong@rutgers. edu