Standardized Information Literacy Assessment SAILS and Beyond Juliet

Standardized Information Literacy Assessment: SAILS and Beyond Juliet Rumble, Auburn University Cheryl Cecil & Beth Ashmore, Samford University Library April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

Project SAILS Standardized Assessment of Information Literacy Skills § An initiative of Kent State University Libraries § Purpose: develop a test instrument for the assessment of information literacy skills that: --Is standardized --Assesses at institutional level --Provides for both external and internal benchmarking April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

Project Structure § 3 phase pilot testing of instrument (2003 -2005) --Samford and AU participated in pilot testing --82 institutions participated (including 7 Canadian) --42, 304 students tested § Diverse group of participating institutions --Carnegie Doctoral/Research level universities --2 -4 year colleges April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

SAILS Test Instrument § Multiple choice test § 45 questions randomly drawn from a data bank of 252 items § Each test question addresses an ACRL Objective for Information Literacy Instruction April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

SAILS Administration at Samford l Sample: --mostly freshmen; students enrolled in UCCA 102 --approximately 40 juniors and senior from research methods and senior seminar courses l Campus stakeholders: --University Core Communication Arts Program --Office of Institutional Research and Assessment --Center for Teaching, Learning & Scholarship l Incentives: some instructors gave credit for taking the assessment; others required it; others just encouraged students to take it April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

SAILS administration at Auburn l Sample: --freshmen through seniors participated --students from 12 disciplines/subject majors l Campus stakeholders: --English Freshman Composition program --Office of Institutional Research and Assessment -- Core Curriculum Oversight Committee l Incentives: participants entered in drawing for 3 Apple i. Pods April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

SAILS Data Reports Results reported at two levels of specificity: --4 ACRL Info Lit Competency Standards --12 skill sets (derived from ACRL’s Objectives for Information Literacy Instruction) l Measures the performance of groups, not individuals l Test questions plotted according to difficulty level April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

What we learned from the SAILS data…. § On all standards and skill sets, the average student at Samford and Auburn scored at about the same level as the average student from all participating institutions. § Both Samford & Auburn focused on person-item maps for the 12 skill sets to identify areas of strength and weakness in students’ performance. April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

A Difficult Question for Our Students If you have a research paper due, and the course instructor has not advised you to use a particular citation style, which of the following is the best thing to do? CHOOSE ONLY ONE ANSWER. �� Select a citation style and use it consistently. �� Use various citations styles based on the type of resource. �� Use your own citation style and use it consistently. �� You should always use APA if no other style is requested. �� You should always use MLA if no other style is requested. April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

What the numbers don’t tell us…. § The national “benchmarks” associated with standards and skill sets do not indicate “mastery” of information literacy § Scores are not based on success in actually performing tasks associated with learning outcomes (although they are intended to be predictors of success). § Samford & Auburn’s test results do not track development of cohort groups (no longitudinal studies were conducted). § There’s still a lot we don’t know about cohort groups that we’re comparing. -- E. g. : Did test takers receive library instruction? -- E. g. : How much (and what kind of) instruction in research skills did students receive in other classes? April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

What we learned about doing assessment § Sharing resources and expertise with other campus groups charged with programmatic assessment is a key to success. § Programmatic assessment involves a serious commitment of time and money. Support must come, not only from individual faculty members and departments, but also from university administration. § Assessment can be done without a standardized tool. In all likelihood, we need a variety of different assessment tools. --E. g. Pre- and post- tests that address specific learning objectives April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006