IL Best Practices Innovative Approaches to Fit Your

IL Best Practices: Innovative Approaches to Fit Your Institutional Needs Sheril Hook, Instruction Coordinator, University of Toronto Mississauga Stephanie Sterling Brasley, Manager, Information Literacy Initiatives, California State University, Office of the Chancellor

Participants will be able to n n n Articulate the major ideas from categories 5 and 10 of the Best Practices Guidelines Identify assessment tools for information literacy development Identify basic strategies for how categories 5 and 10 of the Best Practices Guidelines can be applied to their own instructional and institutional environments

Agenda n Category 5: articulation with the curriculum q q n Playing with the IL Standards IL by design: embedding IL into course outcomes Category 10: assessment and evaluation q q Instruction statistics Horizontal and vertical integration

ALA/ACRL Characteristics of Programs of Information Literacy that Illustrate Best Practices Category 5: Articulation with the Curriculum Articulation with the curriculum for an information literacy program: n is formalized and widely disseminated; n emphasizes student-centered learning; n uses local governance structures to ensure institution-wide integration into academic or vocational programs; n identifies the scope (i. e. , depth and complexity) of competencies to be acquired on a disciplinary level as well as at the course level; n sequences and integrates competencies throughout a student’s academic career, progressing in sophistication; and n specifies programs and courses charged with implementation. http: //www. ala. org/ala/acrlstandards/characteristics. htm

IL Standards Standard One The information literate student determines the nature and extent of the information Performance Indicator 2. The information literate student identifies a variety of types and formats of potential sources for information. Outcomes include n Knows how information is formally and informally produced, organized, and disseminated n Recognizes that knowledge can be organized into disciplines that influence the way information is accessed n Identifies the value and differences of potential resources in a variety of formats (e. g. , multimedia, database, website, data set, audio/visual, book) n Differentiates between primary and secondary sources, recognizing how their use and importance vary with each discipline n Realizes that information may need to be constructed with raw data from primary sources "Information Literacy Competency Standards for Higher Education. " American Library Association. 2006. http: //www. ala. org/acrl/ilcomstan. html (Accessed 15 May, 2007)

Let’s play with the standards

IL Standards Standard One The information literate student determines the nature and extent of the information Performance Indicator 2. The information literate student _____ a variety of types and formats of potential sources for information. Outcomes include n ____ information is formally and informally produced, organized, and disseminated n _____ knowledge can be organized into disciplines that influence the way information is accessed n _____the value and differences of potential resources in a variety of formats (e. g. , multimedia, database, website, data set, audio/visual, book) n _____between primary and secondary sources, recognizing how their use and importance vary with each discipline n _____ that information may need to be constructed with raw data from primary sources "Information Literacy Competency Standards for Higher Education. " American Library Association. 2006. http: //www. ala. org/acrl/ilcomstan. html (Accessed 15 May, 2007)

Student Engagement: IL depth & complexity n n n research-based learning problem-based learning inquiry-based learning case-based learning discovery learning knowledge building Scardamalia, M. , & Bereiter, C. (2003).

Research Question (inquiry-based) § How have myths changed over time? Humanities 2.

Assignment: Myth over Time n Outcomes q q q Explore the dynamism of myth by comparing and contrasting a selection of ancient and modern primary sources of a myth (at least one literary, one material) Identify the most significant changes from ancient to modern source and discuss those changes in light of the context in which each source was created Interpret those changes in terms of how they affect the meaning of the myth and how they came about in the first place Humanities 1.

Outcomes • compare and contrast a selection of primary sources (art) • Students begin by finding primary sources--art works, music, scripts, opera and background information on artists Google has images, but no provenance information Camio has images, plus provenance and usage rights information Humanities 3.

Outcomes • identify the most significant changes. . . in light of the context in which each source was created. Students build on the learning acquired by finding background information on a time period/place Humanities 4.

Outcomes • identify the most significant changes. . . in light of the context in which each source was created. Students place a myth in the cultural context in which it’s being used or retold Humanities 5.

Outcomes • compare and contrast a selection of primary sources (music) Students listen to a symphony to identify the dynamism of the myth and interpret its significance Humanities 6.

ALA/ACRL Characteristics of Programs of Information Literacy that Illustrate Best Practices Category 10: Assessment/Evaluation Assessment/evaluation of information literacy includes program performance and student outcomes and: for program evaluation: n establishes the process of ongoing planning/improvement of the program; n measures directly progress toward meeting the goals and objectives of the program; n integrates with course and curriculum assessment as well as institutional evaluations and regional/professional accreditation initiatives; and n assumes multiple methods and purposes for assessment/evaluation -- formative and summative -- short term and longitudinal; http: //www. ala. org/ala/acrlstandards/characteristics. htm

ALA/ACRL Characteristics of Programs of Information Literacy that Illustrate Best Practices Category 10: Assessment/Evaluation (cont’d) Assessment/evaluation of information literacy includes program performance and student outcomes and: for student outcomes: n acknowledges differences in learning and teaching styles by using a variety of appropriate outcome measures, such as portfolio assessment, oral defense, quizzes, essays, direct observation, anecdotal, peer and self review, and experience; n focuses on student performance, knowledge acquisition, and attitude appraisal; n assesses both process and product; n includes student-, peer-, and self-evaluation; http: //www. ala. org/ala/acrlstandards/characteristics. htm

Assessment is the process of gathering and discussing information for multiple and diverse purposes in order to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences; the process culminates when assessment results are used to improve subsequent learning. Mary E. Huba and Jann E. Freed. Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Allyn & Bacon, 2000.

Evaluation is “any effort to use assessment evidence to improve institutional, departmental, divisional, or institutional effectiveness”

Program Evaluation Components n An Assessment/Evaluation Program Plan q q q q What are we assessing? What are our IL instruction goals? What are the measurable outcomes? How will we measure at the course/curricular level? Who are our campus partners? How can we tie into institutional efforts? What data do we need to collect?

Program Evaluation: Snapshot of Course Penetration n 100/200 level course penetration q Where n q What n q Core courses, instructor status Standards 1 -5 How n Concept-based or tool-based

Instruction Database

Instruction Statistics

Stats tell a story n n n Redundancy Gaps Planning

Stats identify approaches Fig. 5 Number of unique instruction sessions given by type

Class snapshot

Class Snapshot, contd

Interpreting Statistics

Assessment of Student Outcomes

Direct Assessment n Definition q q q Direct evidence of student performance collected from students Actual samples of student work Assess student performance by look at their work products n Examples q q q q Assignments Research papers Portfolios Dissertations/theses Oral presentations Websites Posters/videos Instructor-designed exams or quizzes

Indirect Assessment n Definition q q Provides perspectives and perceptions about what has been learned Faculty must infer students’ skills, knowledge, and abilities rather than observing from direct evidence n Examples: q q q q Exit interviews Surveys: student satisfaction, Course instruction Focus groups Self-reported reflections Research journals or diaries Student ratings of skills Graduation Rates Job Placement Rates

Formative Assessment n n Ongoing measure of student learning Provides feedback to student and instructor on learning process Takes place during the learning process Examples: q Classroom assessment techniques, targeted questions, in-class exercises, research journals/diaries, concept maps

Summative Assessment n n Information gathered at the end of instruction Used to evaluate the efficacy of the learning activity Answers the question of whether learners learned what you had hoped they would Typically quantitative: q Examples: test scores, letter grades, graduation rates

Formative vs. Summative “When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative. ” - Robert Stakes http: //library. cpmc. columbia. edu/cere/web/ACGME/doc /formative_summative. pdf

Classroom Assessment n n n n Provides continuous flow of accurate information on student learning Learner-centered Teacher Directed Mutually Beneficial Formative Context-specific Ongoing Rooted in Good teaching practice

Classroom Techniques-Examples n n 1 -minute paper Muddiest Point 1 -sentence Summary 3 -2 -1

CATS – Web Form Example http: //www 2. library. ucla. edu/libraries/college/11306. cfm One Minute Paper form n Quarter n Class n Instructor n Librarian n Your name n Your phone number n n n Your email address 1. What is the most significant or meaningful thing you have learned during the session? (Feel free to add more items) 2. What question(s) remain uppermost in your mind?

Example: Indiana University Bloomington Libraries http: //www. indiana. edu/~libinstr/Information_Literacy/assessment. html Basic Goal 1. Appreciates the richness and complexity of the information environment. Objective 1. 1. You will be able to describe the wide array of information sources available and discuss their appropriateness for a given information problem. Possible Measurement Techniques: Essay examination Oral report Practicum in the library** Written evaluation assignment Basic Goal 4. Design and use a search strategy tailored to a specific information need. Objective 4. 1. You will be able to describe and execute an appropriate search strategy in a given information source or in multiple information sources. Possible Measurement Techniques: Annotated bibliography with search strategy discussion included Collaborative learning exercise in class Practical exercise Practicum examination** Research journal Research paper proposal Research portfolio Research worksheet** n

Diagnostic Assessment n n Assesses knowledge and skills of students before instruction is designed Determines where student is in terms of learning – identifying gaps in student learning Can provide a baseline for student achievement in a subject area Examples: standardized tests, review of students’ prior work, pre-tests/instructordeveloped tests.

Assessment in Practice: i. Skills, SAILS, i. Lit, ILT, ICDL

Assessment Tools: Evaluation Considerations n n n n Audience Test Development – Reliability and Validity Costs: Development and Administration Test Type: performance-based? Multiplechoice? Delivery mechanism – web? Print? Simulations? Content/Standards Scoring and Reporting Features

SAILS – Standardized Assessment of Information Literacy Skills n n n n Librarians at Kent State University, 2000 Test of information literacy skills Web-based test Multiple choice questions 142 whole items in American English Students answer 40 items + 5 pilot Based on 4 or 5 ACRL Standards (not 4)

i. Skills Assessment (Formerly ICT Literacy Assessment) n n n ETS staff with CSU and 7 core institutions Assesses Information and Communication Technology (ICT) Literacy Web-based, performance-based, scenario-based test Core and Advanced Versions; 75 minutes 15 Tasks – 14 short (1 -3 min) and 1 long (15 min) Test tasks aligned with and informed by ACRL Standards, ISTE NETS standards

Information Literacy Test (ILT) n n n n James Madison University’s Center for Assessment and Research Studies and the Library Designed to assess the ACRL standards (not 4) Web-based Multiple Choice test 60 operational items and 5 pilot Password-protected-Secure administration Reliability: 0. 88 Cost - ? ?

i. LIT n n n Developed by CA. Community College librarians, CSU Subject Matter experts, Test Development experts, Psychometricians Web-based Multiple choice questions Aligned to ACRL standards “Affordable” High-Stakes, proctored

Your Institution’s Needs The Characteristics in Practice