Improving Undergraduate Education through the Assessment of Student

Improving Undergraduate Education through the Assessment of Student Learning SACS-COC Institute on Quality Enhancement and Accreditation July 30, 2006 Peter T. Ewell National Center for Higher Education Management Systems

Looking Back: Origins of the Assessment Movement Two Decades Ago • Undergraduate Reform Reports of 1985 -86 • Internal Stimulus: Call for More Coherent Teaching/Learning Approaches and Information for Improvement • External Stimulus: Stakeholder Demands for Information on “Return on Investment” • Tensions in Motive and Message Ever Since

Why Didn’t Assessment Go Away? • Pressure to Produce Evidence of Student Learning Outcomes Never Let Up • By Early 1990 s, Accreditors Replace States as Primary External Stimulus to Get Started • Intermittent Federal Interest in Assessment as an Element of National Accountability • But Resulting Faculty Ambivalence About a Process Seen as “External” and “Administrative”

Looking Back: What’s Been Accomplished? • Assessment Is for the Most Part Perceived as Inevitable and Legitimate • Vast Majority of Institutions Have Statements of Learning Outcomes (General and Programmatic) • A “Semi-Profession” of Folks Involved in Assessment • Steadily Growing Sophistication with Respect to Methods of Gathering Evidence

Looking Back: What Hasn’t Happened? • Authentic Integration of Assessment into Faculty Cultures and Behaviors • Assessment Activities Still Largely “Added On” to the Curriculum Instead of Being Embedded In It • Systematic and Widespread Use of Assessment Results for Institutional and Curricular Improvement • Proactive and Sincere Institutional Engagement with Accrediting Organizations Around Topics of Assessment

Assessment as a “Perpetual Movement? ” • Most Social Movements… – Fade as Fads, or – Go Away Because Core Ideas are Mainstreamed • The “Assessment Movement” Has Done Neither. . . – External Requirements Keep it Alive – But External Pressures also Constrain Faculty Buy-In and Meaningful Institutional Use • Moving Beyond a “Perpetual Movement” Will Demand. . . – Making Assessment Real to Faculty by Connecting It to the Actual Practice of Teaching and Learning – Re-Focusing Accountability on the Authentic Student Abilities that Society Says it Needs

Some Prominent Changes in Higher Education’s Operating Environment • Doing More with Less - Need for Curricula that are Effective and Efficient - Need Information About Curricular Functioning to Enable Effective Action • Changes in Instructional Delivery - Competencies and “Deep Learning” - Student Engagement and Role of Technology • New Views of Accountability - “Quality”— From Inputs/Processes to Results - Stakeholder Voices — Students and Employers

Challenges to the Academy: The Internal Dimension • Changing Paradigm of Teaching and Learning • Resulting Changes in Academic Roles, Behaviors and Structures – Nature and Role of “Faculty” in Instruction – Patterns of Student Enrollment • [Technology as a “Wild Card”]

A Changing Paradigm of Teaching and Learning • From “Faculty Teaching” to “Students Learning” • Students “Make Their Own Paths” through Multiple Learning Opportunities • Explicit Designs for Learning Based on Research

Changing Paradigm: Possible Lines of Response • Demonstrated Achievement Becomes Paramount, not “Seat Time” • Students “Teach” One Another • Individual (and Asynchronous) Paths • Technology Seen as Opportunity to Re. Think [not as a “Solution”]

Some Implications for Student Assessment Processes • Assessments Reinforce Common Standards for Learning Across Curricula and Classes • “Seamless” Assessments Become an Integral Part of Curriculum and Pedagogy • Assessments Emphasize Connections and Longitudinal Development, not Just Attainment

Assessment Approaches: A Resulting Shift in Emphasis • Accountability-Based: Assessments Added onto Instruction to “Check Up” on the System in the Aggregate • Scholarship and Improvement: Assessments Built into the System to Simultaneously Assure Standards and Provide Feedback on Collective Performance

Kinds of Information Needed • Alignment of Key Learning Outcomes Across Units, Sequences, and Courses • Match Between Curricular Design, Delivery, and Student Experience • Match Between Instruction and Needs of Diverse “Student Bodies” • Effectiveness of Particular Innovations and Interventions

Changing Nature of the Faculty Role • Dis-aggregation of Instructional Roles • More Things for Faculty to Do • New Potential Career Patterns and Paths

New Faculty Roles: Possible Lines of Response • Emphasize Peer Support and Collaboration in Faculty Development • Recognize and Regularize Alternative Career Paths • Recognize “Mentorship” as the One Thing You Can’t Responsibly Outsource

Kinds of Information Needed • Capturing and Re-Aggregating Data About Discrete Instructional Functions • Accounting Technology-Based Costs • Tracking Faculty “Assets”

Changing Patterns of Student Attendance • Increased Levels of Multi-Institutional Attendance • Increased Complexity in Course-Taking Behavior within Institutions • Greater (and Unpredictable) Time Lapses Between Instructional Encounters

Changing Patterns of Attendance: Possible Lines of Response • Coherence Based on Common Practices [and Outcomes], not Common Content • Stress Ways for Students to Exploit and Reflect on Their Own Experiences • Establish Clear Transition Points at which to Assess Student Mastery of Key Concepts

Kinds of Information Needed • Relationships Between Particular Institutional Experiences and Particular Outcomes • Tracking Student Learning Styles and Individual Paths of Development • “Episode-based” [as opposed to time-based] Data Structures

Challenges to the Academy: The External Dimension • Increased Accountability • Changing Expectations Regarding What Students Know and Can Do • The “New Competition”

What Are States Doing? • Forces Influencing State Approaches – Decreased Agency Capacity Due to Funding Cuts – Momentum of K-12 Reform (NCLB) – Political Uncertainty and Instability • Types of State Policy Responses – P-16 Alignment – Performance Measures and “Report Cards” – Achievement Testing (Driven by K-12)

What’s Shaping Accreditation? • Forces Influencing Accreditors – Pressure for Specific Performance from Federal Government – Demands from Institutions to “Add Value” – New Models from Other Sectors and Abroad • “New Looks” in Accreditation – – Focus on Outcomes and Effectiveness Presenting Evidence [e. g. “Portfolios”] Review Approaches [e. g. “Academic Audits”] Connection to Institutional Planning [e. g. QEP]

Learning Outcomes: What Employers Expect • Higher-Order “Literacies” as Well as Specific Skills • Framed in Terms of “Practice” [not “Knowledge”] • As Much About Attitudes as Academics [“Soft Skills”]

External Forces: Some Resulting Influences on the Academy • Increased Emphasis on Credentialing • Modularity and Acceleration to Increase Accessibility • Pressure to Respond to “Students as Customers” • Accountability “Superstructures” that Divert Attention and Information Resources

Kinds of Information Needed • Outcomes and Performance Measures • Data on What Experiences/Services Students Can Expect • Peer Comparisons [Increasingly Outside the Academy] and Comparative Performance • Needs and Satisfaction of External Stakeholders

Attributes of a Meaningful “Culture of Evidence” • Shared Recognition That Many (But Not All) Things Are Knowable • An Accessible Store of Information About Organizational Condition and Performance • An Attitude Toward Problem-Solving that Minimizes “Finger-Pointing” • Clear Follow-Through On Decisions Made and Why They Were Taken

Cultures of Evidence: Success Factors • Visible Metaphor of Scholarship • Beginning with Real Problems and Processes, not with “Method” • Consistent Messages from Leadership • Periodically “Re-Socializing” the Community • “Closing the Loop” with Action

Cultures of Evidence: Inhibiting Factors • Either Excessive or Non-Existent Consequences • Alien Language and “Management Culture” • Excessive Complexity • Burnout and “Committee Fatigue”

Meaningful Assessment is More About Mindset than Method • Questions About Learning are not Just Matters of Opinion – What Information Might We Collect? – What Might We Expect to Find? – What Difference Would Finding Out Make? • Assessment is About Improving Practice – What are We Trying to Fix? – How Good is Good Enough? – What Changes are Implied?

The Bottom Line • For Internal Management, “Seat of the Pants” Decision-making is no Longer Sufficient – Information Used Openly, Consistently, and Continuously to Inform Academic Decisions • For External Constituencies, “Trust Me” is no Longer Sufficient – Need Clear, Understandable Evidence of Student Academic Attainment • Be Vigilant about the Information You Choose and the Signals it Sends – Make Sure that What You Measure is What You Value – Harness the Accreditation Process to Make it Happen!