Documenting Student Learning From Classroom Assessment to Program

Documenting Student Learning: From Classroom Assessment to Program Improvement Pat Hutchings National Institute for Learning Outcomes Assessment A conversation

How We Got Here October 2018 February 2018 Designing Effective Assignments: Intellectual Work Worth Sharing Are My Students Learning What I Think I’m Teaching? Quick and Easy Classroom Assessment Strategies

Plan for Today • Brief review of assessment principles, history • Why documenting student learning matters • Models and strategies for moving from individual classroom assessment to program and institutional accountability • A conversation

NILOA’s mission is to discover and disseminate effective use of assessment data to strengthen undergraduate education and support institutions in their assessment efforts. ● SURVEYS ● WEB SCANS ● CASE STUDIES ● FOCUS GROUPS ● OCCASIONAL PAPERS ● WEBSITE ● RESOURCES ● NEWSLETTER ● PRESENTATIONS ● TRANSPARENCY FRAMEWORK ● FEATURED WEBSITES ● ACCREDITATION RESOURCES ● ASSESSMENT EVENT CALENDAR ● ASSESSMENT NEWS ● MEASURING QUALITY INVENTORY ● POLICY ANALYSIS ● ENVIRONMENTAL SCAN ● DEGREE QUALIFICATIONS PROFILE AND TUNING ●ASSIGNMENT LIBRARY INITIATIVE www. learningoutcomesassessment. org

First Things First Assessment is… • Finding out whether my students are learning what I think I’m teaching (faculty member at a long-ago workshop) • A systematic process for understanding and improving student learning (Angelo, 1995) • An integral component of learning (Alverno College) • Required by accreditors—regional and professional • A professional responsibility

The History of Assessment: An Enduring Tension Between Improvement and Accountability* • • Involvement in Learning (1984) Time for results (1986) New accreditation requirements Investment in external tests and instruments Shift toward home-grown approaches Recognizing the need for faculty engagement… Without which, improvement is unlikely. * See Ewell, 2011

Where Assessment Is Headed FROM: assessment as “exoskeletal, ” an add-on TO: Assessment that’s integral to T&L, homegrown, with collaboratively designed assignments at the center

Why Should Programs and Institutions Document Student Learning? • Ongoing, incremental improvement • Professional responsibility • Accreditation – Federal rules – Western Association – Professional accreditation

The Challenge How do we go from the rich evidence provided by course assignments and classroom assessment techniques to evidence that addresses broader issues of quality, accountability, and improvement. --Kristel Nazzal, at the end of the previous webinar

Models for Student Learning Outcomes Assessment that Build on Classroom Work

Building Blocks 1. Explicit, agreed upon, program and institutional outcomes. 2. Curriculum mapping—alignment 3. Rubrics or criteria for evaluating student work

1. Explicit, Agreed Upon Outcomes • What students should know and be able to do to succeed in our program, our institution • Active verbs—vs “appreciate” • Can be evaluated, judged, measured • Local and national outcomes frameworks

The Degree Qualifications Profile • Intellectual Skills • Specialized Knowledge • Specialized knowledge Broad Integrative …. Knowledge • Civic and Global Learning • Applied and Collaborative Learning

Specialized Knowledge at MA level: Student… • Elucidates the major theories, research methods and approaches to inquiry and schools of practice in the field of study, articulates their sources and illustrates both their applications and their relationships to allied fields of study. • Assesses the contributions of major figures and organizations in the field of study, describes its major methodologies and practices and illustrates them through projects, papers, exhibits or performances. • Articulates significant challenges involved in practicing the field of study, elucidates its leading edges and explores the current limits of theory, knowledge and practice through a project that lies outside conventional boundaries.

2. Curriculum Mapping/Alignment Thanks to Josh Potter, KU

BONUS slide

3. Rubrics and Criteria for Evaluation: Valid Assessment of Learning in Undergraduate Education: VALUE Rubrics INTELLECTUAL AND PRACTICAL SKILLS Inquiry and analysis Critical thinking Creative thinking Written communication Oral communication Reading Quantitative literacy Information literacy Teamwork Problem solving PERSONAL AND SOCIAL RESPONSIBILITY Civic engagement Intercultural knowledge and competence Ethical reasoning Foundations and skills for lifelong learning Global learning INTEGRATIVE AND APPLIED LEARNING Integrative learning https: //www. aacu. org/value-rubrics

MODELS FOR USING COURSE WORK TO DOCUMENT PROGRAM AND INSTITUTIONAL STUDENT LEARNING OUTCOMES

1. Capstone Assignments and Common Rubrics --Mt. St. Joseph, Washington State U, Point Loma Nazarene… --programs design own culminating, capstone experiences --but all use a common rubric --eg specialized knowledge, or integrative learning --Assessment can sample or look at all student work

2. Assignment Templates Organized Around Shared Outcomes —Cabrini College --”signature assignments”—designed to fit broad specifications for a particular outcomes/curricular area --E. G. : Individual and Society: 3 -4 pp written product; requires collecting, analyzing, and interpreting data. --Assessed with a common rubric

3. e. Portfolio at Program or Inst Level --collections of student work, across contexts, time --students select (but with guidelines) --structured, curated --with student reflections --build student agency (owned by student) but also --provide grist for assessment at program level --using common rubric or criteria --rely on work students are already doing in classes

4. Online Assessment Communities --seeing assessment as professional practice (vs technical accounting, reporting) --builds on “COP” idea (Lave and Wenger), of building a community of expertise, learning through practice --a group of faculty who work together online, designing assignments/tasks, looking at student work, developing shared metrics, refining rubrics --building an institutional culture --see Banta et al in resources

Discussion • How do these options match what’s possible and needed at PAU? • What is your experience with them? • What else would be useful or have you found useful?

Note to Self—or to Each Other Something you’d like to do as follow up to this session?

Resources • Banta, T. W. , Griffin, M. , Flateby, T. L. , & Kahn, S. (2009, December). Three promising alternatives for assessing college students' knowledge and skills. (NILOA Occasional Paper No. 2). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment. • Eynon & Gambino (2017). High-Impact e. Portfolio Practice. Sterling, VA: Stylus • Green & Hutchings (2018). Faculty engagement with integrative assignment design: Connecting teaching and assessment. New Directions for Teaching and Learning, 155, 39 -46. • Hutchings, Jankowski, & Ewell, (2014). Catalyzing assignment design activity on your campus: Lessons from NILOA’s assignment library initiative. Urbana, IL: NILOA.

• Metzler, E. T. , & Kurz, L. (2018, November). Assessment 2. 0: An organic supplement to standard assessment procedure. (Occasional Paper 36). Urbana, IL: NILOA. • National Institute for Learning Outcomes Assessment. (2016, May). Higher education quality: Why documenting learning matters. Urbana, IL: NILOA • Point Loma Nazarene University: Case Study. http: //degreeprofile. org/wpcontent/uploads/2015/08/DQP-Case-Study-Point. Loma. pdf

Thank you