OutcomeBased Education Assess plan deliver and assess again

Outcome-Based Education Assess, plan, deliver, and assess again (improve!) for quality assurance OBE/WA Workshop 24 October 2021 Slide # 1 Vijay K. Arora Professor of Electrical Engineering and Engineering Management Wilkes University Wilkes-Barre, PA 18766, USA E-Mail: vijay. arora@wilkes. edu Web: web. wilkes. edu/vijay. arora Distinguished Visiting Professor Fakulti Kejuruteraan Elektrik Universiti Teknologi Malaysia

Experiential Learning Tell me, and I forget OBE/WA Workshop 24 October 2021 Slide # 2 Teach me, and I may remember Involve me and I learn Ben Franklin

Bloom’s Taxonomy-Learning at Higher Levels OBE/WA Workshop 24 October 2021 Slide # 3

Verbs for Intellectual Activity 1. Knowledge: arrange, define, duplicate, label, list, memorize, name, order, recognize, relate, recall, repeat, reproduce, state. 2. Comprehension: classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, restate, review, select, translate. OBE/WA 3. Application: apply, choose, demonstrate, dramatize, Workshop employ, illustrate, interpret, operate, practice, schedule, sketch, solve, use, write. 4. Analysis: analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, discriminate, 24 October 2021 distinguish, examine, experiment, question, test. 5. Synthesis: arrange, assemble, collect, compose, construct, create, design, develop, formulate, manage, organize, plan, prepare, propose, set up, write. Slide # 6. Evaluation: appraise, argue, assess, attach, choose 4 compare, defend estimate, judge, predict, rate, core, select, support, value, evaluate.

Washington Accord-Signatories Ø Accreditation Board for Engineering and Technology (ABET) Ø Accreditation Board for Engineering Education of Korea (ABEEK) Ø Board of Engineers Malaysia Ø Canadian Engineering Accreditation Board OBE/WA Ø Engineering Council of South Africa (ECSA) Workshop Ø Engineering Council UK (ECUK) Ø Engineers Australia Ø Engineers Ireland 24 October 2021 Ø Hong Kong Institution of Engineers (HKIE) Ø Institute of Engineering Education, Taiwan (IEET) Ø Institution of Engineers Singapore (IES) Ø Institution of Professional Engineers New Zealand Slide # (IPENZ) 5 Ø Japan Accreditation Board for Engineering Education (JABEE)

Outcomes Assessment Defined OBE/WA Workshop 24 October 2021 Slide # 6 The systematic gathering, interpreting, and use of information for the purposes of improving student learning. Emphasis is on results (outcomes) rather than inputs and processes. Document that learning has occurred Guiding Questions v. What do you want your students to KNOW? v. What do you want your students to BE ABLE TO DO? v. What do you want your students to VALUE?

Liberal Education for a Changing World OBE/WA Workshop 24 October 2021 Slide # 7 Relevant to the lives and career of students, preparing them for a broad range of careers, as well as for lifelong learning involving both formal programs and hands-on experience. Attractive so that the excitement and intellectual content of discipline will attract highly talented students with a wide variety of background and career interest. Connected to the needs and issues of the broader community through integrated activities with other parts of the educational system, industry, and government. Source: American Society for Engineering Education

Example: Attribute Gap Analysis Stevens Institute of Technology (Institute) and State Higher Education Executive Office (SHEEO) of New Jersey OBE/WA Workshop 24 October 2021 Slide # 8 Courtesy: Peter Koen, Stevens Institute of Technology

Ultimate Goal of Outcomes Assessment l. Commission on Higher Education (1996) –“examine and enhance institutional effectiveness” l. Two Tier Approach EDUCATIONAL EFFECTIVENESS OBE/WA Workshop 24 October 2021 Slide # 9 v. Improve teaching and learning v. Contribute to the personal development of students INSTITUTIONAL CONTEXT v. Ensure institutional improvement v. Facilitate accountability

The State of the Assessment Process Assessment is faculty driven and focused on a student-centered approach to teaching and learning OBE/WA Workshop 24 October 2021 Slide # 10 OUTCOMES ASSESSMENT allows us to determine the QUALITY of a University education on OUR terms v. Promotes improvement v. Supports success stories

Assessment at Various Levels Institutional Level OBE/WA Workshop Department Level 24 October 2021 Classroom Level Slide # 11

Assessment Hierarchy 1. Vision speaks to the upside possibility for program or major–A journey towards continued excellence 2. Mission frames responsibility 3. Objectives specifies responsibility (alternative paths) Goal is a destination guided by vision Outcome is a specific target (mileage covered) OBE/WA Workshop 4. Assessment Instruments & Metrics measure performance in meeting outcomes; the dashboard, if you will 5. Program Improvement Process uses performance 24 October 2021 assessment to effect positive change: Gauge success of prior decisions and actions Identify performance improvement needs Slide # Identify actions needed to correct/improve performance 12 Implement actions.

Developing Missions and Strategies Mission statements tell an organization where it is going OBE/WA Workshop 24 October 2021 Slide # 13 The Strategy tells the organization how to get there

Mission - where are you going? OBE/WA Workshop 24 October 2021 Slide # 14 þOrganization’s purpose for being þAnswers ‘What do we provide society? ’ þProvides boundaries and focus (managing constraints)

Mission Basis Ø Develops shared values and vision Ø Meets four basic needs (as stated by Steve Covey): OBE/WA Workshop 24 October 2021 Slide # 15 v v economic social (relational) psychological (emotional) and contributory Ø Leads strategic path from mission statement and reflects vision and values Ø Reflects the ecological realities (the environment) Ø Avoids producing graduates with obsolete skills and attitudes. Ø Aligns with the constitution of university/country Ø Values people (the top line) and stays solvent ( the bottom line)

Fed. Ex OBE/WA Workshop 24 October 2021 Slide # 16 Fed. Ex is committed to our People-Service-Profit philosophy. We will produce outstanding financial returns by providing total reliable, competitively superior, global air-ground transportation of high priority goods and documents that require rapid, time-certain delivery. Equally important, positive control of each package will be maintained using real time electronic tracking and tracing systems. A complete record of each shipment and delivery will be presented with our request for payment. We will be helpful, courteous, and professional to each other and the public. We will strive to have a completely satisfied customer at the end of each transaction.

Merck OBE/WA Workshop 24 October 2021 Slide # 17 The mission of Merck is to provide society with superior products and services - innovations and solutions that improve the quality of life and satisfy customer needs - to provide employees with meaningful work and advancement opportunities and investors with a superior rate of return

Arnold Palmer Hospital OBE/WA Workshop 24 October 2021 Slide # 18 Arnold Palmer Hospital is a healing environment providing family-centred care with compassion, comfort and respect… when it matters the most.

Wilkes University Mission OBE/WA Workshop 24 October 2021 Slide # 19 To educate our students for lifelong success and fulfillment in a constantly evolving and multicultural world through a commitment to individualized attention, exceptional teaching, and academic excellence.

Wilkes Vision OBE/WA Workshop 24 October 2021 Slide # 20 To be a nationally recognized independent university where intense personal engagement in exceptional academic and professional programs cultivates in every student a lifelong commitment to civic responsibility, ethical leadership, and openness to cultural differences.

Wilkes Values OBE/WA Workshop 24 October 2021 Slide # 21 As a university community, we work together with understanding, respect and integrity, guided by these principles: v Students are our defining purpose v Lifelong learning is our passion v Mentorship is our guiding behavior v Community engagement is our civic imperative v Financial stewardship is our bedrock responsibility

Factors Affecting Mission Philosophy and Values OBE/WA Workshop Profitability and Growth Environment Mission 24 October 2021 Slide # 22 Customers Public Image Benefit to Society

Strategic Process Organization’s Mission Functional Area Missions OBE/WA Workshop 24 October 2021 Slide # 23 Marketing Operations Finance/ Accounting

Outcomes Assessment Approaches OBE/WA Workshop 24 October 2021 Slide # 24 Top-Down Approach: What graduates need to know and be able to do? Life-Cycle Approach: Process of continuous improvement based on mission and benchmarks. Efficiency Enhancement Approach: Total quality management and accountability. Interdisciplinary Approach: Isolation to integration. Criteria Development and Assessment Approach: Success and Benchmarking.

What Outcomes are Assessed? OUTCOME: results achieved by students following instruction v. Cognitive (Knowledge) Outcomes OBE/WA Workshop èWhat do students know? v. Skill Outcomes èWhat can students do? 24 October 2021 v. Attitudes & Values (Affective) Outcomes èDiversity, personal identity, motivation v. Behavioral Outcomes Slide # 25 èPersistence, post-graduation activities

Models of Curriculum Development OBE/WA Workshop 24 October 2021 Slide # 26 Source: G. D. Catalino, Journal of Engineering Education, pp 11 -14, January 1996.

Learning and Teaching Ø The OBE Criteria for Accrediting Programs is based upon what students learn in the courses as opposed to what they are presented in a curriculum Ø The educational objectives and outcomes assessment OBE/WA techniques are used to determine the degree to which Workshop program goals and objectives are being attained 24 October 2021 Slide # 27 Ø The assessment, in turn, is used in an ongoing process of improving student learning through enhancements to the program Ø Each accreditation board has its own set of accreditation criteria

Level of Implementation Questions must be asked? Ø Are Program Educational Objectives established and maintained? Ø Are Constituents involved in helping set program objectives and in evaluating the level to which they are being achieved? (stakeholder analysis 0 OBE/WA Workshop Ø Are the required Processes operational? Ø Is Outcomes Assessment being practiced? Ø Are the Results of outcomes and the various processes 24 October 2021 being used to improve programs? Ø Does the assessment reach Assurance that objectives are met? Slide # Ø Is an overall System in place to meet the accreditation 28 requirement ?

Focus on Students Ø Describe how students are evaluated, advised, and monitored in a manner consistent with program objectives. OBE/WA Workshop 24 October 2021 Slide # 29 Ø Prepare the graduation requirements planning and check sheet Ø Describe the processes and procedures used to enforce policies for the acceptance of transfer students and provide evidence that the processes and procedures are working. Ø Describe the procedures used to validate credit for courses taken elsewhere and provide evidence that the procedures are working.

Program Objectives OBE/WA Workshop 24 October 2021 Slide # 30 Ø The Program Objectives are the statements that describe the expected accomplishments of graduates during the first few years after graduation. Ø The audiences for objective statements are all stakeholders such as faculty, students, employers, advisory boards and student sponsors Ø Discuss in detail the educational objectives, the process by which these objectives are determined and evaluated Ø How the program ensures these objectives are met

Program Objectives- contd What needs to be done? Ø List the Program Educational Objectives and show they are consistent with the mission of the institution and the accreditation criteria. Ø Identify the significant constituencies of the program. OBE/WA Workshop Ø Describe the processes used to establish and review the Program Educational Objectives and the extent to which the program’s various constituencies are involved in these processes. 24 October 2021 Ø Describe how the program curriculum and processes ensure achievement of the Program Educational Objectives. Provide documentation that demonstrates Slide # that the processes are working. 31

SUMMARY Degree of Specificity OBE/WA Workshop 24 October 2021 Low Mission Statement Goals Objectives Slide # 32 High

ASEE Database and Sample http: //www. asee. org/conferences/v 2 Search. cfm http: //sg. wilkes. edu/ABET/prgrpt OBE/WA Workshop www. abet. org www. washingtonaccord. org 24 October 2021 Slide # 33

Bringing it All Together… The department must agree on the mission, goals, and objectives OBE/WA Workshop 24 October 2021 Slide # 34 Achieving consensus requires compromise and communication Focus on process, not product

Your Task OBE/WA Workshop 24 October 2021 Slide # 35 Develop an assessment plan for your department/university v. You get the information you need to answer the questions that are important for your department/university v. Development of full plan will take time and sharing of resources within the university and with outside constituencies v. Assessment results may become part of the effectiveness and efficiency improvement for knowledge engineering and management Demonstrate an effective process of improvement

Constituency Input/feedback OBE/WA Workshop 24 October 2021 Slide # 36 ACTIVITY FREQUENCY CONSTITUENCY Faculty Meetings Capstone Design Projects Professional Societies Senior Student Survey Student Advisory Group Program Assessment Alumni Survey Ind Adv. Board Survey Co-Op Program Academic Program Rev. Accreditation Visit As needed Annually Annually Three Years Two Years Five Years Six Years Faculty Students/Faculty Industry/Students Administration/Faculty Alumni Industry Administration/Faculty Students/ Faculty/Profession

Program Outcomes and Assessment Ø Are the established Program Outcomes based on the Program Educational Objectives (Matrix and Explanation)? OBE/WA Workshop 24 October 2021 Slide # 37 Ø Describe how these Program Outcomes relate to the Program Educational Objectives (Matrix and Explanation) Ø Describe how the Program Outcomes chosen by the program encompass and relate to the outcome requirements of Generic Attributes (Matrix and Explanation) Ø Provide qualitative and quantitative data gathered on a regular basis that are used to assess the quality of achievement of the outcomes and your analysis of those assessment results

Program Outcomes and Assessment – contd. Ø Describe the process by which the assessment results are applied to further develop and improve the program. Document these changes Provide qualitative and quantitative data used to support these changes. OBE/WA Workshop Ø Describe the materials, including student work and other tangible materials, that will be available for review during the visit to demonstrate achievement of the Program Outcomes and Assessment. 24 October 2021 Slide # 38 Ø The programs are encouraged to organize these materials on the basis of outcomes, rather than on a course-by-course basis.

General Education Component OBE/WA Workshop 24 October 2021 Slide # 39 Describe how the program curriculum devotes adequate attention and time to the General Education component, which includes v. Mathematics and basic sciences v. Behavioral sciences v. Liberal arts. v. Prerequisites /Deficiencies v. Extra-curricular activities

METRIC ØMATRIX OF PROGRAM OUTCOMES OBE/WA Workshop ØMATRIX OF PROGRAM OBJECTIVES COMPLIANCE WITH ESTABLISHED CRITERIA ØMATRIX OF COURSE COVERAGE FOR PROGRAM OUTCOMES 24 October 2021 Slide # 40 ØDEVELOP COMPETENCIES ON THE BASIS OF THE ABOVE

Stakeholder Surveys OBE/WA Workshop 24 October 2021 Ø Senior Exit Survey (operational for couple of years) Ø Alumni Survey (in process) Ø Advising Feedback (in operation since last year) Ø IAB Survey (in process) Ø Placement Survey Data (in process) Ø Graduation Requirements Check Sheet (operational) Ø Active Faculty Participation in Course Improvements, Review of Outcomes, Objectives Development on the basis of Feedback (all - need to be done) Ø Outcomes Assessment individually by Course Ø Objectives Assessment individually by Course Slide # 41

Work to be done after surveys OBE/WA Workshop 24 October 2021 Slide # 42 Ø Course Improvement and Review Process Ø Program Objectives Review by Faculty on the Basis of Program Objectives Assessment Data Ø Alumni Response Assessment, Level of Alumni Satisfaction With Competencies, Objectives and the Program Ø Advisory Boards Response Assessment Ø Target Areas for Improvement

Improvement Process We have to practice this following process once at least some data are in place Mission Educational Objectives OBE/WA Workshop Faculty 24 October 2021 Slide # 43 Course Coordinators Program Outcomes Objectives Curriculum Instruments Outcomes Courses Metrics Evaluation and Assessment Data Constituencies

Faculty Documentation ØDiscuss the adequacy of the size of the faculty ØDescribe the extent and quality of faculty involvement in interactions with students, in advising, in service, in professional development, and in interactions with industry. OBE/WA ØDiscuss the competence of the faculty members to cover Workshop all of the curricular areas of the program and draw conclusions in that regard. ØDescribe the education, diversity of backgrounds, 24 October 2021 engineering experience, teaching experience, ability to communicate, enthusiasm for developing a more effective program, level of scholarship, participation in Slide # professional societies, and registration/licensure as 44 Professional Engineers of the faculty members.

Facilities Documentation Ø Discuss the adequacy of facilities Ø Provide information concerning facilities such as classrooms, laboratories, and computing and information infrastructures that engineering students OBE/WA and faculty are expected to use in meeting the Workshop requirements of the program. 24 October 2021 Slide # 45 Ø Identify the opportunities students have to learn the use of modern engineering tools, including identification of the important tools and the depth of the student experience.

Institutional Support Ø Discuss the adequacy of institutional support, financial resources, and constructive leadership necessary to achieve program objectives Ø Describe the processes used to determine the budget for the program. OBE/WA Ø Describe the adequacy of faculty professional Workshop development and how it is planned and funded. Ø Describe a plan and sufficiency of resources to acquire, maintain, and operate facilities and equipment required 24 October 2021 to achieve program objectives. Ø Discuss the adequacy of support personnel and institutional services necessary to achieve program Slide # objectives. 46

Appendix I - Additional Program Information OBE/WA Workshop A. Tabular Data for Program v Table I-1. Basic level Curriculum v Table I-2. Course and Section Size Summary v Table I-3. Faculty Workload Summary v Table I-4. Faculty Analysis v Table I-5. Support Expenditures B. Course Syllabi 24 October 2021 Slide # 47 C. Faculty Curriculum Vitae

Appendix II - Institutional Profile A. Background Information Relative to the Institution OBE/WA Workshop 24 October 2021 Slide # 48 v v v v General Information Type of Control Regional or Institutional Accreditation Faculty and Students Mission Institutional Support Units Non-academic Support Units B. Background Information Relative to the Engineering Unit v Engineering Educational Unit v Programs Offered and Degrees Granted v Information Regarding Administrators

Institutional Profile – contd. . Ø Supporting Academic Departments, Engineering Finances, Engineering Personnel and Policies, Engineering Enrollment and Degree Data, Definition of Credit Unit, Admission and Graduation Requirements, Basic Programs OBE/WA Workshop 24 October 2021 Slide # 49 C. Faculty v Attracting and Retaining High Quality Faculty, Faculty Professional Activities, Office Support, Adequacy of Resources, Administrative Leadership D. Library v Library Staffing, Collection and Budget, Electronic Information E. Classroom Equipment

Life-Long Education If you are planning for a year, sow rice OBE/WA Workshop 24 October 2021 Slide # 50 If you are planning for a decade, plant a tree If you are planning for a lifetime, educate a person