Telling Our Story About Our Students and Engaging

Telling Our Story About Our Students and Engaging Others in Doing the Same

Why we need to tell our own story about student learning? • Measures of quality are imposed –Standardized exams (comparison institutions) –Return on Investment measures (e. g. , job placement and median starting salaries –Program progression measures (e. g. , retention and completion rates) • Additional scrutiny is required –making student work available to outside parties –including outside parties in the review of student work

Voluntary System of Accountability (VSA) • Public universities to supply straightforward, comparable information on the undergraduate student experience through a common web report • Three objectives • Demonstrate accountability, transparency • Support innovation in the measurement, reporting of student learning outcomes • Use as a search tool for informational and comparison purposes

Student Learning Outcome (SLO) Component of the VSA • Use results from one of the required instruments to assess campus-level outcomes – ETS Proficiency Profile critical thinking and written communications sub-scores – CLA performance and analytic writing tasks – ACT CAAP critical thinking and writing essay modules • Results can be no more than three years old

Stories Reported on the VSA • CLA longitudinal study –Results were posted but they were not useful in contributing to campus discussions about student learning outcomes. –The faculty have not found the results of the study useful in directing improvements. • ETS Proficiency Profile in 2014 –Results and comparisons to participating institutions were posted to our College Portrait. –Test results were shared with the deans and department chairs with the recommendation that departments identify learning activities in the curriculum that are linked to the critical thinking competencies and provide greater emphasis or infuse additional learning activities to strengthen students' critical thinking skills.

The Story Reported on the University’s Website • Student Achievement – Retention and Graduation Rates – Teacher Licensing Exam Results – National Survey of Student Engagement Results – Post-Graduation Survey Results – Enrollment http: //academics. uncc. edu/node/11

How can we tell our story better? • Program Learning Outcome examples: –Africana Studies –College of Education –Respiratory Therapy –Computing and Informatics –University Writing Programs

VSA’s New Student Learning Outcome Component • SLO reporting is no longer restricted to specific instruments • The National Institute of Learning Outcomes Assessment (NILOA) Transparency Framework was adopted as the preferred reporting method • Greater flexibility in assessing student learning • Greater clarity in messaging to stakeholders • Required to report on the components of the Transparency Framework on the University’s website

NILOA Transparency Framework National Institute for Learning Outcomes Assessment. (2011). Transparency Framework. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Retrieved from: http: //www. learningoutcomesassessment. org/Transparency. Framework. htm

Benefits of the Transparency Framework • Standardized exam is no longer required to assess campus-level outcomes • Supports regional and discipline compliance requirements for SLOs • The following are already in place: • Student Learning Outcome Statements, Assessment Plans and Activities, Evidence of Student Learning , and Use of Student Learning Evidence for Programs • Assessment Resources

NILOA Framework Readiness Dimension Readiness to meet the Criterion Student Learning Outcomes Statements Student learning outcomes statements are clearly stated Student learning outcomes statements are integrated Student learning outcomes statements are prominently posted and available to students Campus-level Assessment Plan Comprehensive institutional-level assessment plan Alignment between institution-level assessment plan and program-level assessment plans Stakeholder involvement in the development and review of assessment plans Campus-level Assessment Resources Centralized assessment resources availability and use Dimension Readiness to meet the Criterion Campus-level Assessment Activities Campus-level assessment information shared with internal and external stakeholders Presentation format and accessibility of campus assessment information Participation in and sharing of information regarding campus-level assessment activities Campus-level Evidence of Student Learning Engagement in monitoring and compiling campus-level assessment results and analysis Distribution and sharing of campus-level results of student assessment Integration of campus-level results with measures used at other levels to create complete picture of student learning Faculty and staff development activities to promote campus -level assessment best practices availability and use Use of Campus-level Evidence of Student Learning Integration of campus-level results with measures used at other levels to guide campus decision-making Evidence of use of assessment results from all levels of campus provided to stakeholders Campus policies and procedures recognizing and providing support for faculty and staff assessment activities Communication of changes made as a result of assessment evidence from all levels of campus Communication of outcomes from changes made as a result of assessment evidence from all levels of campus Green = available or no outside input necessary Light Gold = moderately difficult, as a process or collaborative efforts are needed Dark Gold = difficult, as a process or collaborative efforts are needed

Outcome Linkages

Why the need for campus-level outcomes? • Faculty & Staff are able to – Communicate skills to the public in terms they understand agree upon – Help students connect the dots between courses – Make linkages between the various initiatives and their impact • Students are able to – Make connections between general education and academic major outcomes – Make connections between curricular and cocurricular outcomes

Prioritizing and Ranking Campus level Outcomes 1. Critical Thinking 5. Inquiry 2. Ethical Reasoning 6. Oral Communication 3. Global Learning and Intercultural Competence 7. Quantitative Literacy 4. Information Literacy 8. Written Communication

Campus-Level Assessment Plan The plan consists of: • campus-level student learning outcomes with measures that are integrated with program- and course-level student learning outcomes and measures • an assessment plan for collecting, analyzing, disseminating, and reviewing evidence of campus-level student learning and evidence-based decision making • involvement of external and internal stakeholders in: – development of the assessment plan – review of assessment data – decisions for improving student learning • publishing to website

NILOA Framework Readiness Dimension Readiness to meet the Criterion Student Learning Outcomes Statements Student learning outcomes statements are clearly stated Student learning outcomes statements are integrated Student learning outcomes statements are prominently posted and available to students Campus-level Assessment Plan Comprehensive institutional-level assessment plan Alignment between institution-level assessment plan and program-level assessment plans Stakeholder involvement in the development and review of assessment plans Campus-level Assessment Resources Centralized assessment resources availability and use Dimension Readiness to meet the Criterion Campus-level Assessment Activities Campus-level assessment information shared with internal and external stakeholders Presentation format and accessibility of campus assessment information Participation in and sharing of information regarding campus-level assessment activities Campus-level Evidence of Student Learning Engagement in monitoring and compiling campus-level assessment results and analysis Distribution and sharing of campus-level results of student assessment Integration of campus-level results with measures used at other levels to create complete picture of student learning Faculty and staff development activities to promote campus -level assessment best practices availability and use Use of Campus-level Evidence of Student Learning Integration of campus-level results with measures used at other levels to guide campus decision-making Evidence of use of assessment results from all levels of campus provided to stakeholders Campus policies and procedures recognizing and providing support for faculty and staff assessment activities Communication of changes made as a result of assessment evidence from all levels of campus Communication of outcomes from changes made as a result of assessment evidence from all levels of campus Green = available or no outside input necessary Light Gold = moderately difficult, as a process or collaborative efforts are needed Dark Gold = difficult, as a process or collaborative efforts are needed

NILOA Transparency Framework Gaps Dimension Student learning outcomes statements are integrated NILOA Criterion Statement Student learning outcomes statements at the campus-level are integrated with student learning outcomes statements from the college-, program-, and course-level. Comprehensive institutional-level assessment The institution has a comprehensive assessment plan at the institution-level that includes plan common assessment activities for all students designed to provide evidence of student learning across campus. Participation in and sharing of information regarding campus-level assessment activities Faculty, staff, students, and external stakeholders such as employers or graduate school admissions professionals actively participate in the decision processes related to campuslevel assessment. Engagement in monitoring and compiling campus-level assessment results and analysis Faculty, staff, students, and external stakeholders such as employer or graduate school admissions professionals participate in the monitoring and/or compiling of campus-level assessment results and analysis. Integration of campus-level results with Campus-level assessment results have been considered in combination with assessment measures used at other levels to guide campus results at other levels of the institution to guide decision-making related to changes in decision-making policies and practices that may lead to improved student learning. Campus policies and procedures recognizing and providing support for faculty and staff assessment activities Breakout Group Team #1 Team #2 Campus policies and procedures regarding faculty and staff review processes (e. g. , tenure and promotion, performance reviews, etc. ) are structured to provide support and/or recognition for faculty and staff working to improve or advance their assessment practices. Alignment between institution-level Institutional-level assessment is integrated with and scaffolds from program-, course-, and assessment plan and program-level assessment student-level assessment data. The review processes from all levels of assessment provides plans supporting evidence for all learning outcomes at the institutional-level. Stakeholder involvement in the development Stakeholders from programs and departments, including student affairs and students, are and review of assessment plans involved in the development and review of assessment plans, results, and recommendations. Team #3

Breakout Groups/Lunch/Report Out Team 1 - (Salon E) Integration of campus-level outcomes and developing a campus-level assessment plan Team 2 - (Salon C) Involving external partners in campus-level assessment processes and changes to internal policies and practices Team 3 - (Salon D) Identifying campus-level measures and a review process and involving internal stakeholders

Publishing Student Learning Outcome Assessment Plans Dimension Campus-level assessment information shared with internal and external stakeholders NILOA Criterion Statement Campus-level assessment information relating to assessment plans, processes, and results are available to and proactively shared with a variety of on-campus and offcampus stakeholders. Short Description Assessment plans, processes, and results Evidence of use of assessment results from Stakeholders from both inside and outside the institution are regularly provided with Linkages of decision all levels of campus provided to evidence that institutional decision-making is appropriately guided by assessment making to assessment stakeholders results from multiple levels of campus assessment. results Communication of changes made as a result of assessment evidence from all levels of campus Changes made as a result of assessment results are communicated to both internal Implemented changes and external campus audiences, including students. Communications include information on evidence supporting the need for change. Communication of outcomes from changes Changes made as a result of assessment results are monitored and evaluated. made as a result of assessment evidence Outcomes from changes are communicated to both internal and external campus from all levels of campus audiences, including students. Communications include information on evidence used to evaluate the change. Distribution and sharing of campus-level results of student assessment Campus-level assessment results are shared proactively with faculty and staff on campus in ways that facilitate their discussion and application (e. g. , not simply made available as a report on a website). Presentation format and accessibility of campus assessment information Campus-level assessment information is presented in easily accessible formats that are understandable to a variety of audiences both on and off campus. The presentation of assessment information is intentionally designed to promote appropriate use and interpretation of results. Improvements based on changes Communication Methods

UNC Charlotte’s Proposed Framework Model Student Learning Outcomes Statements Accreditation Assessment Plans Assessment Resources Evidence of Student Learning Use of Student Learning Evidence

Report Outs

Next Steps: Submission of SLO Plans and Reports • Some programs lack documentation on: – The annual faculty review of student learning outcomes assessment data – Changes made based on evidence of student learning – Impact of the prior year’s changes on student learning

Follow-Up to Assessment Retreat • Encourage departments to devote at least one spring department meeting to the review of assessment data • Appoint a scribe, could be the College Assessment Coordinator to document the department review of assessment data • Encourage departments to document evidence-based changes in SLOs Assessment Plans and Reports

Examples of Evidence-Based Changes 1. Curricular Changes: Added or revised a course(s), changed course sequence, developed a curriculum map 2. Course Revisions: Added or revised course content, changed the textbook 3. Pedagogy Changes: Revised methodology of delivering course materials, integrated technology 4. Revised Assessments: Changed or modified SLO(s), changed or modified assessment tool(s)

Examples of Evidence-Based Changes (cont. ) 5. Changed Criteria: Modified rubric criteria, increased Performance Outcome 6. Revised Policies: Changed entrance requirements 7. Professional Development: Provided faculty (or TA) with training 8. Budget Decisions: Requested additional resources

Wrap Up • All Plans and Reports are due May 31 st • Student Learning Outcome information from Plans and Reports to be posted on the website in Fall 2016 • Complete all Transparency Framework components by March 1, 2017

Questions

Sources of Campus Level Outcomes Adapted from Susan Hatfield , Ph. D. , Presentation to 2010 Written Communication Oral Communication Critical Thinking Quantitative Reasoning Information Literacy Co-curricular Outcome Program Learning Outcome Co-curricular Outcome Integrative Learning Program Learning Outcome Campus. Level Learning Outcom es Co-curricula Outcome General Education Program Learning

Publication Concerns • Program specific data such as course name and number (anonymity) • Impact of results on annual evaluations and course quality (punitive consequences) • Performance target and % of students achieving the target (perceived low expectations of students) • Others?

Voluntary System of Accountability Origins: Spellings Commission Report • “To meet the challenges of the 21 st century, higher education must change from a system primarily based on reputation to one based on performance. ” • “We urge the creation of a robust culture of accountability and transparency throughout higher education. Every one of our goals, from improving access and affordability to enhancing quality and innovation, will be more easily achieved if higher education institutions embrace and implement serious accountability measures. ” Source: A Test of Leadership: Charting the Future of U. S. Higher Education A Report of the Commission Appointed by Secretary of Education Margaret Spellings