Student Affairs Assessment Series Module 1 Assessment Basics

Student Affairs Assessment Series Module 1: Assessment Basics Written by Kris Baack, Linda Major and Megan Scherling

Module 1: Assessment Basics Presented by Kris Baack, Linda Major and Megan Scherling

Learning Outcomes Upon completion of this session, participants will be able to demonstrate a basic understanding of assessment by: 1. Explaining the value of assessment -to the individual -to the program -to the department -to the institution 2. Communicating the language of assessment 3. Describing how the cycle of assessment supports the Division of Student Affairs’ mission, vision and strategic plan 4. Showing the connection between specific program objectives, the Division’s strategic plan and the cycle of assessment

What Is Assessment?

What Is Assessment? • • • Identify student needs Measure student satisfaction Plan for student development Assess outcomes Assess student learning outside the classroom Monitor campus climate

Assessment Is Important • To evaluate efficacy of programs and services • To improve student learning and development • To measure and validate our student affairs work

Stakeholders & Audience • Impact of college is cumulative • Cognitive and affective development inextricably intertwined • Out of class activities enrich student learning Pascarella & Terenzini, 2005

History of Student Affairs Assessment • Sporadic and individual efforts to collect, analyze and use data • Omnibus Survey • PEARL collaborating partner (2008) • Loose coalition of those interested in assessment formed • Division enters into Student Voice contract (2011) • NASPA Consortium • Student Affairs Learning Domains • Student Affairs Assessment Council (2014) • Annual assessment report • SAAC website • Campus Labs contract continued • Professional Development Series • Collaboration with Academic Affairs to track cohort retention/graduation

Assessment Lingo • Benchmark- Student performance standards (the level(s) of student competence in a content area. ) An actual measurement of group performance against an established standard at defined points along the path toward the standard. Subsequent measurements of group performance use the benchmarks to measure progress toward achievement. (Gallaudet) • Cohort- A group whose progress is followed by means of measurements at different points in time. (Gallaudet) • Criteria/Standards- Performance descriptors that indicate how well students will meet expectations of what they should be able to think, know or do. They are descriptive benchmarks against which performance is judged. These criteria or standards may be described in varying gradients of success as in rubrics or in grades. Often they are stated in terms of percentages, percentiles or other quantitative measure (Gallaudet) • Evaluation- Both qualitative and quantitative descriptions of student behavior plus value judgments concerning the desirability of that behavior. Using collected information (assessments) to make informed decisions about continued instruction, programs, activities. Decisions made about assessment findings; deciding about the value of programs/program outcomes; may involve recommendations for changes. (Gallaudet)

Assessment Lingo • Indirect Assessment Methods- Assessment methods that involve perceptions of learning rather than actual demonstrations of outcome achievement (e. g. alumni surveys, employer surveys, exit interviews). (Gallaudet) • Learning Outcome- A statement of what a student should understand be able to do as a result of what he or she has learned in a course or program. (Clark) • Mean- One of several ways of representing a group with a single, typical score. It is figured by adding up all the individual scores in a group and dividing them by the number of people in the group. Can be affected by extremely low or high scores. (Gallaudet) • Measurement- Quantitative description of student learning and qualitative description of student attitude. (Gallaudet) • Median- The point on a scale that divides a group into two equal subgroups. Another way to represent a group's scores with a single, typical score. The median is not affected by low or high scores as is the mean. (Gallaudet)

Assessment Lingo • Mission- A holistic vision of the values and philosophy of a department, program, unit or institution. General education learning goals are often found in the institution’s mission statement. (Gallaudet) • Modifications- Recommended actions or changes for improving student learning, service delivery, etc. that respond to the respective measurement evaluation. (Gallaudet) • Objectives- Synonymous with outcomes. Statements that describe measurable expectations of what students should be able to think, know or do when they’ve completed a given educational program. Each statement should describe one expectation; should not bundle several into one statement. The statements must be clear and easily understood by all faculty in the area/department. (Gallaudet)

Assessment Lingo • Outcomes Assessment- The measurement of learning outcomes. OA examines student demonstrations of the results of learning. The process includes four cyclical steps: 1) teaching and learning, 2) developing questions and gathering information about student learning, 3) analyzing the information and drawing conclusions, and 4) reflecting and planning. It documents the alignment (or dissonance) between the intended learning (as stated in the outcomes) and the actual learning (as demonstrated by the student). The practice of Outcomes Assessment is collaborative and is intended to inform. Its goal is to continually improve student learning. (Clark) • Process- A generalizable method of doing something, generally involving steps or operations which are usually ordered and/or interdependent. Process can be evaluated as part of an assessment, as in the example of evaluating a student's performance during prewriting exercises leading up to the final production of an essay or paper. (Gallaudet)

Assessment Lingo • Program Assessment- Uses the department or program as the level of analysis. Can be quantitative or qualitative, formative or summative, standards-based or value added, and used for improvement or for accountability. Ideally, program goals and objectives would serve as a basis for the assessment. Example: How well can senior engineering students apply engineering concepts and skills to solve an engineering problem? This might be assessed through a capstone project, by combining performance data from multiple senior level courses, collecting ratings from internship employers, etc. If a goal is to assess value added, some comparison of the performance to newly declared majors would be included. (Carnegie Mellon) • Quantitative Methods of Assessment- Methods that rely on numerical scores or ratings. Examples: Surveys, Inventories, Institutional/departmental data, departmental/course-level exams (locally constructed, standardized, etc. ) (Gallaudet) • Qualitative Methods of Assessment- Methods that rely on descriptions rather than numbers. Examples: Ethnographic field studies, logs, journals, participant observation, and open-ended questions on interviews and surveys. (Gallaudet) • Reliability- The measure of consistency for an assessment instrument. The instrument should yield similar results over time with similar populations in similar circumstances. (Gallaudet)

Assessment Lingo • Sampling- A way to obtain information about a large group by examining a smaller, randomly chosen selection (the sample) of group members. If the sampling is conducted correctly, the results will be representative of the group as a whole. Sampling may also refer to the choice of smaller tasks or processes that will be valid for making inferences about the student's performance in a larger domain. "Matrix sampling" asks different groups to take small segments of a test; the results will reflect the ability of the larger group on a complete range of tasks. (Gallaudet) • Self-Assessment- A process in which a student engages in a systematic review of a performance, usually for the purpose of improving future performance. May involve comparison with a standard, established criteria. May involve critiquing one's own work or maybe a simple description of the performance. Reflection, self-evaluation, metacognition, are related terms. (Gallaudet) • Standards- Agreed upon values used to measure the quality of student performance, instructional methods, curriculum, etc. (Gallaudet) • Triangulation- Multiple lines of evidence pointing to the same conclusion. (Gallaudet)

Assessment Lingo • Value Added- The increase in learning that occurs during a course, program, or undergraduate education. Can either focus on the individual student (how much better a student can write, for example, at the end than at the beginning) or on a cohort of students (whether senior papers demonstrate more sophisticated writing skills-in the aggregate-than freshmen papers). To measure value-added a baseline measurement is needed for comparison. The baseline measure can be from the sample of students (longitudinal design) or from a different sample (cross-sectional). (Carnegie Mellon)

The Cycle of Assessment UNL Institutional Goals - Enrollment - Graduation - Research Division of Student Affairs Strategic Plan - Mission Learning Experiences and Services - Vision - Strategic Priorities Clearly identified learning or service outcomes Share results and use for decisions and practices Review assessment results Determine assessment method Identify, develop and administer measure(s)

Student Affairs Mission and Vision Statement We create an environment where students feel they matter. Mission Statement We foster social and learning environments that enhance students' lives, preparing them for lifelong success.

Strategic Plan | 2014 -2019 STRATEGIC INITIATIVE 1: Foster a campus culture that promotes student leadership, wellness and personal and social responsibility. Goal 1. Cultivate a campus climate where social justice and cultural respect are seamlessly integrated in each department within the Division of Student Affairs. Goal 2. Promote a campus-wide culture grounded in a multi-dimensional model of wellness: physical, environmental, occupational, spiritual, emotional, social and intellectual.

Strategic Plan | 2014 -2019 STRATEGIC INTIATIVE 2: Establish a transparent system of accountability that advances opportunities for continuous improvement Goal 1. Utilize division-wide and departmental assessment data to improve existing programs and services, respond more quickly to student needs, and/or eliminate as necessary.

Strategic Plan | 2014 -2019 STRATEGIC INITIATIVE 3: Secure and manage the human, financial and physical resources necessary to support our mission. Goal 1. Continuous development and project management of major new and proposed construction and renovation projects in such a manner as to ensure successful completion on time and within budget. Goal 2. Explore creative new sources of revenue to enhance the quality of programs, services and facilities necessary to retain and graduate current and future students. Goal 3. Develop, implement and institutionalize a transparent, accountable and educational system that meets the intent of the Campus Sexual Violence Elimination Act. Goal 4. Support a collegial environment that attracts exceptional employees, values holistic wellness, affords personal and professional growth opportunities, and ensures retention of an excellent workforce.

Strategic Plan | 2014 -2019 STRATEGIC INITIATIVE 4: Communicate the educational value of engagement outside the classroom and construct a common Student Affairs identity. Goal 1. Present unified data to clarify the role of Student Affairs in building the optimal academic environment. Goal 2. Achieve 100% brand adoption of the Student Affairs lockup and/or tagline on print and web materials by May 2017. Goal 3. Calculate an open and click rate as baseline for Husker Advocate newsletter during 20162017.

Increase the number of first year students who report having conversations with someone different than themselves from a mean score of 38. 2 to 41. 3 by 2017 (difference defined as race/ethnicity, socioeconomic background, religion or political views). NSSE 2013 Engagement Indicators Strategy Promote a campus-wide culture grounded in a multidimensional model of wellness: physical, environmental, occupational, spiritual, emotional, social and intellectual. Measurable Objective Foster a campus culture that promotes student leadership, wellness and personal and social responsibility. Goal We foster social and learning environment that enhance students' lives, preparing them for lifelong success. Strategic Initiative We create an environment where students feel they matter. Mission Vision Example Provide students with the skills necessary to increase the inclusivity of UNL. • DDPIO • Fortune, Mitchell, Major • Global and Cultural Engagement • Participants will apprecialte the value of interacting with diverse cultures and identities. • DDPIO Evaluation

Student Affairs Assessment Report Form http: //go. unl. edu/sa_assessment

Questions?