Assessing Academic Programs at IPFW Overview of process

Assessing Academic Programs at IPFW Overview of process, reporting structure and support for programmatic assessment under SD 15 -06 D. Kent Johnson, Ph. D Director of Assessment

Assessment Council Goals for SD 98 -22 Revision The University Assessment Council worked on revision between Nov 2014 and April 2015. The emphasis of the redesign was focused in three areas: • Design a programmatic assessment framework that provides academic units useful and actionable findings consistent with a “culture of learning” and that meets HLC assessment expectations. • Ensuring the framework has sufficient flexibility to align with and eliminate unnecessary duplication between institutional reporting requirements and reporting requirements of professional accreditors. • Provide a consistent report format that recognizes and supports distinctive assessment approaches that align with program needs and disciplinary conventions. IPFW is an Equal Opportunity/Equal Access University.

Background on SD 98 -22 Restatement • Assessment practice under existing policy was outdated (from a “best practices” perspective and in relationship to HLC Accrediting requirements • Amendments to SD 98 -22 over the years made the document unclear • Spring 2015 Assessment Council worked on a revision - SD 98 -22 reviewed to ensure it aligned with current expectations and best practice for assessment – decision made to proceed with a “restatement” to ensure clarity and limit confusion • Assessment Council completed the proposed revisions in April 2015 • Faculty Senate Approved as SD 15 -06, October, 2015. IPFW is an Equal Opportunity/Equal Access University.

HLC Assessment Expectation Statements For student learning, a commitment to assessment would mean assessment at the program level that proceeds from clear goals, involves faculty at all points in the process, and analyzes the assessment results; it would also mean that the institution improves its programs or ancillary services or other operations on the basis of those analyses. Higher Learning Commission. “The Criteria for Accreditation: Guiding Values”. 2015. Qualified faculty should also be aware of whether and how much students learn through the ongoing collection and analysis of appropriate data, because an institution should be able to demonstrate its commitment to educational achievement and improvement through ongoing assessment of student learning. Higher Learning Commission. “Determining Qualified Faculty through HLC’s Criteria for Accreditation and Assumed Practices: Guidelines for Institutions and Peer Reviewers”. October, 2015. IPFW is an Equal Opportunity/Equal Access University.

Approved Senate Document 15 -06 https: //www. ipfw. edu/dot. Asset/2869 b 623 -4 dcd-4 e 8 e-8 a 31 f 2537 b 2 eb 5 e 3. pdf IPFW is an Equal Opportunity/Equal Access University.

Part 1: Department/Program Level Assessment Activities and Results Assessing, Making Interventions, and Reassessing to Improve Programmatic Capacity for Improving Student Learning

IPFW Undergraduate Program Assessment Model

Annual Academic Program Assessment Report Outline I. II. III. IV. V. Overview of Programmatic Student Learning Outcomes (Appendix D, Section I) Curricular Maps (Institutional to Baccalaureate Framework and Programmatic to Core Courses or Specific Curricular Mileposts (Appendix D, Section II and III) Assessment Plan (Appendix D, Section IV) Assessment Results (Appendix D, Section V) Conclusions, Next Steps, and Communication (Appendix D, Section VI and VII) IPFW is an Equal Opportunity/Equal Access University.

Assessment Reporting • Sections 1, 2, and 3 should be fairly static (require minimal changes) over time • Section 4 reporting is informed by the PLAIR Model emphasizing a process of “assess-intervene-reassess” (Fulcher, Good, Coleman, and Smith, 2015). • Sections 5 and 6 draw conclusions to demonstrate the improving quality of the program, suggest how the faculty plan on continuing to improve program support of student learning, and communicate to internal and external constituents what students are learning IPFW is an Equal Opportunity/Equal Access University.

Part 2: College Level Review of Departmental Assessment Rubrics for Evaluating Academic Program Review from Appendix D of Proposed Revision to SD 98 -22

Assessment Reporting Process in Proposed Revision to SD 98 -22 Academic Program Proposed in Restatement of SD 98 -22 Academic Program Submits Academic Department Assessment Report to College Assessment Committee General Education Academic Program Prepares Assessment Summary by Course and Submits to General Education Subcommittee College Level Assessment Committee Reviews Academic Department Assessment Report General Education Committee Reviews Course Level Assessments to Prepare Academic Assessment Report Assessment Council Reviews College Level Reports and Prepares Annual Report of Assessment Progress for Each College IPFW is an Equal Opportunity/Equal Access University. Assessment Council Reviews Academic Assessment Report Using IPFW Assessment Progress Worksheet for General Education Subcommittee

Appendix D-I: Clearly Stated Student Learning Outcomes Exemplary 3 Acceptable 2 Developing 1 Clarity and specificity All SLOs are stated with clarity and specificity including precise verbs and rich descriptions of the knowledge, skills and value domains expected of students upon completing the program SLOs generally contain precise verbs, rich description of the knowledge, skills and value domains expected of students. SLOs are inconsistently defined for the program, descriptions of the knowledge, skill and value domains are present but lack consistent precision. Student-Centered All SLOs are stated in studentcentered terms (i. e. what a student should know, think, or do). Most SLOs are stated in student-centered terms. Some SLOs are stated in student-centered terms. Expectation Level SLOs exceed basic expectations established by the University and other necessary approving organizations required of the submitting unit. SLOs meet the basic expectations established by the University and other necessary approving organizations required of the submitting unit. SLOs meet only a portion of the expectations established by the University or other necessary approving organizations required of the submitting unit. IPFW is an Equal Opportunity/Equal Access University. Score

Appendix D-II: Alignment with Baccalaureate Framework IPFW Baccalaureate Framework Alignment IPFW is an Equal Opportunity/Equal Access University. Exemplary 3 Acceptable 2 Developing 1 Specific, clearly defined, studentcentered Program-Level SLOs are aligned to all foundation areas of the IPFW Baccalaureate Framework. Generally defined studentcentered Program-Level SLOs are aligned to all foundation areas of the IPFW Baccalaureate Framework. Program-Level SLOs are aligned to some foundation areas of the IPFW Baccalaureate Framework. Score

Appendix D-III: Programmatic Curriculum Map Exemplary 3 Acceptable 2 Developing 1 Common classes or learning activities are identified for all students completing the program but most SLOs are not clearly mapped to classes or activities. Curricular Map identifies expected levels of learning for some SLOs at specific points in the curriculum. Content Alignment All SLOs are mapped to common classes or learning activities expected of all students completing the program. Most SLOs are mapped to common classes or learning activities expected of all students completing the program. Student Learning Development of SLOs (Learning Benchmarks) Curricular Map clearly identifies the progression of student learning relative to all SLOs at specific points in the curriculum. Classes and/or activities engage students in the work outlined in the SLOs. Curricular Map identifies levels of expected learning relative to most SLOs at specific points in the curriculum. Classes and/or activities engage students in the work outlined by most of the SLOs. Student Engagement IPFW is an Equal Opportunity/Equal Access University. Classes and/or activities do not consistently engage students in the work outlined by most of the SLOs. Score

Appendix D-IV: Assessment Plan Exemplary 3 Acceptable 2 Developing 1 Relationship between assessments Detail is provided regarding SLO-to and SLOs -measure match. Specific items included on the assessment are linked to SLOs. The match is affirmed by faculty subject experts. Types of Measures All SLOs are assessed using at least two measures including at least one direct measure. Established Results Statements of desired results (data targets) provide useful comparisons and detailed timelines for completion. Data Collection and Design The data collection process is Integrity sound, clearly explained, and appropriately specific to be actionable. Description of how SLOs relate to assessment is general but sufficient to show alignment. Description of how SLOs relate to assessment is incomplete or too general to provide sufficient information for use in determining progress toward SLO. Most SLOs are either assessed using only indirect measures or are not assessed. Statements of desired results are missing or unrealistic for completion. Evidence of Reliability of Measures Methods used to ensure reliability of findings are clearly explained and consistently support drawing meaningful conclusions. Methods used to ensure reliability of findings are stated and generally support drawing meaningful conclusions. IPFW is an Equal Opportunity/Equal Access University. Most SLOs are assessed using at least one direct measure. Statements of desired results provide a basic data target and a general timeline for completion. Enough information is provided to understand the data collection process with limited methodological concerns. Limited information is provided about the data collection process or includes sufficient flaws to nullify any conclusions drawn from the data. Methods to ensure reliability of findings are insufficient for drawing meaningful conclusions. Score

Appendix D-V: Reporting Results Presentation of Results Historical Results Interpretation of Results IPFW is an Equal Opportunity/Equal Access University. Exemplary 3 Acceptable 2 Results are clearly present and directly related to SLOs. Results consistently demonstrate student achievement relative to stated SLOs. Results are derived from generally accepted practices for student learning outcomes assessment. Past iterations of results are provided for most assessments to provide context for current results. Results are present and related to SLOs. Results generally demonstrate student achievement relative to stated SLOs. Results are derived from generally accepted practices for student learning outcomes assessment. Past iterations of results are provided for the majority of assessments to provide context for current results. Interpretations of results are reasonable given the SLOs, desired levels of student learning and methodology employed. Multiple faculty interpreted the results. including an interpretation of how classes/activities might have affected the results. Developing 1 Results are provided but do not clearly relate to SLOs. Results inconsistently demonstrate student achievement relative to stated SLOs. Use of generally accepted practices for student learning outcomes assessment is unclear. Limited or no iterations of prior results are provided. Interpretation of results does not adequately refer to stated SLOs or identify expectations for student learning relative to SLOs. The interpretation does not include multiple faculty. Score

Appendix D-VI: Report Dissemination and Collaboration Documents and results are shared with faculty Documents and results are shared with other stakeholders IPFW is an Equal Opportunity/Equal Access University. Exemplary 3 Acceptable 2 Developing 1 Information is routinely provided to all faculty with multiple opportunities for collaboration to build meaningful future plans. Information is routinely provided to stakeholders (beyond faculty) with multiple opportunities for collaboration to build meaningful future plans. Information is provided to all faculty through an effective mode and with sufficient detail to be meaningful. Information is shared with stakeholders (beyond faculty) through an effective mode and with sufficient detail to be meaningful. Information is not distributed to all faculty or provides insufficient detail to be meaningful. Information is not distributed to stakeholders (beyond faculty) or provides insufficient detail to be meaningful. Score

Sample College Level Committee Report to an Academic Program (Appendix 1 A) Program Name: Exemplary 3 Acceptable 2 Developing 1 Clarity and specificity All SLOs are stated with clarity and specificity including precise verbs and rich descriptions of the knowledge, skills and value domains expected of students upon completing the program SLOs generally contain precise verbs, rich description of the knowledge, skills and value domains expected of students. SLOs are inconsistently defined for the program, descriptions of the knowledge, skill and value domains are present but lack consistent precision. Student-Centered All SLOs are stated in studentcentered terms (i. e. what a student should know, think, or do). Most SLOs are stated in student-centered terms. Some SLOs are stated in student-centered terms. Expectation Level SLOs exceed basic expectations SLOs meet the basic expectations SLOs meet only a portion of the established by the University and expectations established by the 2 other necessary approving University or other necessary organizations required of the approving organizations required submitting unit. of the submitting unit. Overall the quality of Student Learning Outcomes is very good. However, not all SLO’s were stated in student centered terms. For example… Recommendation IPFW is an Equal Opportunity/Equal Access University. Score 3 2 In addition, the expected level of learning for Programmatic SLO 1 and SLO 3 were stated in the Bloom lexicon at the “understand” level. Given the expected progression of student learning, we would expect students to advance to a level where they can apply and analyze. For example…

College Level Report Template Assessment Report Section 1: Summary of findings detailing scores of all academic departments/programs of the college. Section 2: Summary of recommendations made to academic departments/programs based on their assessment findings. Section 3: Summary of results of changes made or actions taken as a result of prior year findings including results of student learning and a summary of impact (positive or negative). Section 4: Conclusions providing an overall evaluation of assessment in the College and a description of changes in process planned to improve the quality of student learning. IPFW is an Equal Opportunity/Equal Access University.

Example of College Level Reporting: Summary of Findings Across Departments Clarity and specificity IPFW is an Equal Opportunity/Equal Access University. Exemplary 3 Acceptable 2 Developing 1 All SLOs are stated with clarity and specificity including precise verbs and rich descriptions of the knowledge, skills and value domains expected of students upon completing the program SLOs generally contain precise verbs, rich description of the knowledge, skills and value domains expected of students. SLOs are inconsistently defined for the program, descriptions of the knowledge, skill and value domains are present but lack consistent precision. Score Department 1: Department 2: Department 3: Department 4: Average Score: 3 2 1 3 2. 25

Part 3: Supporting Programmatic Assessment The Office of Assessment, IPFW Assessment Academy, and Assessment Council as the Departmental Support Team

Overview of Institutional Assessment Team • Office of Assessment – Coordinates and provides support for assessment activities • The Assessment Council – Shared governance focused group of faculty charged with recommending policy and providing oversight of assessment process. • The IPFW Assessment Academy Leadership Team – Faculty group charged with creating and supporting learning cohorts, programs, workshops, and other activities focused on developing and improving assessment at IPFW is an Equal Opportunity/Equal Access University.

Support Available to Academic Departments • IPFW Assessment Academy – Cohort based community to support departments through a full cycle of assessment • 2015 -16 Workshop Series – Delivered in general sessions and available as Department/Academic Program Focused Series “on-demand” and “just-intime” based on department/program needs • Blackboard Course – Supports the Workshop Series and can be “stand-alone” • Assessment Director – Provides ongoing and “on-demand” support, resources, and leadership for programmatic and course level assessment IPFW is an Equal Opportunity/Equal Access University.

Proposed Support - Technology • Assessment Council recommended software to support assessment activities in the Spring 2015 Semester • The recommendation included a call to discover and implement a comprehensive solution that supports multiple reporting requirements including all the elements of academic program review • Three vendors have been identified and proposals are being solicited IPFW is an Equal Opportunity/Equal Access University.