Program Assessment Professional Development Days August 25 th

Program Assessment Professional Development Days August 25 th and 26 th 2009

Workshop Learning Outcomes • • Describe the purpose of program assessment Outline the steps in program assessment Write measurable program learning outcomes Choose program assessment delivery points Your worksheets will be collected at the end and returned to you next week.

What is the Purpose? Program “assessment is a framework for focusing faculty attention on student learning and for providing meaningful discussions of program objectives, curricular organization, pedagogy, and student development. ” Mary Allen, Assessing Academic Programs in Higher Education

The Goal is Improvement Program assessment concentrates on the group of students in a program. To improve: Courses Programs Services Planning Budgeting

Last but not least - Accreditation By 2012, the ACCJC expects us to: • Have SLOs for all courses, programs and the institution. • Have assessed SLOs at all levels • Have reported and discussed the results and used them to guide decision-making and improvements.

Assessment Levels Institutional Program Course • Institutional outcomes should be reflected in the programs. • Program outcomes should be reflected in the courses.

What is a program? It’s up to the college to decide how to group activities for assessment purposes. Some common ones: • Sequence of courses leading to Degree or Certificate • General Education • Basic Skills • Library, Learning Assistance Center, etc. • Special sequences for groups of students like PACE, Honors, Extension • Student services (not addressed here)

By our normal way of defining “program”, there are over 100!

How can we get it all done? One step at a time! 1. Identify assessable programs 2. Develop program learning outcomes 3. Develop an assessment plan 4. Collect assessment data, analyze and report 5. Use results to improve the program This workshop concentrates on 1, 2 and 3

Identify Assessable Programs S T E P 1 For efficiency, create groupings of programs with similar goals and courses: – CSU General Education + Graduation Plan A + Graduation Plan B + IGETC + Liberal Studies Degree + ? – Business + Real Estate + Accounting – Industrial Engineering + Electronic Engineering + Engineering Technology + ? – Is Political Science a program?

Identify Programs S T E P 1 Set up a focus group to work on the next steps. It could include: • Division chair • Faculty point person • Full time and adjunct faculty • Students

Worksheet and Discussion S T E P 1 Take a few minutes to answer the Step 1 questions on your worksheet.

2. Write Program SLOs S T E P 2 A. Review LAHC institutional SLOs B. Look at course SLOs in your program C. Search the Web for outcomes to “borrow” D. Describe the ideal graduate E. Check draft SLOs for measurability F. Build course to program matrix

Institutional SLOs (ISLO) S T E P 2 A 5 ISLOs (handouts and campus web site). Modify them to be specific to your program. They don’t ALL have to fit but most will. This makes it easier for us to conduct institutional assessment!

ISLO #1 Communication S T E P 2 A Use language and non-verbal modes of expression appropriate to the audience and purpose. Example SLO for a Child Development program: Teach and care for young children using effective communication and relationship skills.

ISLO #2 Cognition S T E P 2 A Use critical thinking skills to analyze, synthesize and evaluate ideas and information. Example SLO for a Math program: Estimate and check answers to mathematical problems in order to determine reasonableness, identify alternatives, and select optimal results.

ISLO #3 Information Competency S T E P 2 A Utilize research skills necessary to achieve educational, and professional objectives. Example SLO for a History program: Use primary and secondary sources to construct sophisticated, persuasive, and logical interpretations of historical problems and events

ISLO #4 Social Responsibility S T E P 2 A Demonstrate sensitivity to and respect for others and participate actively in group decision making. Example SLO for a Computer Science program: Analyze the global impact of computing on individuals, organizations, and society.

ISLO #5 Personal Development S T E P 2 A Demonstrate self-management, maturity, and growth through practices that promote physical, mental and emotional well-being. Example SLO for a Human Services program: Identify their personal values and articulate how their value systems inform and influence their professional practices.

Look at the course SLOs S T E P 2 B • Find repeating – Words – Activities – Ideas • Course assessment forms are on the campus web site (Faculty/Staff, Student Learning Outcomes/Assessment, by department)

Course SLO Word Cloud History program word cloud created at wordle. net

Search the Web S T E P 2 C • Other institutions Use www. google. com or www. bing. com to search. Search: history program learning outcomes Example: University of Maryland http: //www. ugst. umd. edu/core/Learning. Outcome. htm • Professional Organizations – Go to the site and search for learning outcomes ACM (Computer Science) ABET (Engineering) MAA (Math) Warning: you will find bad SLOs and good ones.

The Ideal Graduate S T E P 2 D • Look at program review – Mission and goals • Advisory committee members • Transfer institution contacts – Call or email and ask what the students transferring in need to have – What do they find lacking?

The Ideal Graduate S T E P 2 D Search the Internet or ask Bob Richards for help locating data. For students studying computers: According to Kelly IT Resources, there a number of technical skills currently in demand, including Java 2 Platform, Enterprise Edition (J 2 EE), Visual Java++, Web. Sphere, ERP, and CRM. However, the IT worker who wants to get ahead needs more than just technical savvy. In a survey conducted by the Journal of Computer Information Systems, results indicated that attributes including, “interpersonal and personal skills/traits, such as team skills, communication skills, critical/creative thinking skills, and personal motivation” are often valued higher than the traditional technical requirements of a job. From “IN-DEMAND IT SKILL SETS: WHAT EMPLOYERS WANT, WHAT WORKERS NEED” Published by Kelly IT Resources at http: //www. kellyit. com. pr/res/content/us/kit/en/docs/kitr_indemand_skillsets. pdf

Check: Measurability S T E P 2 E • Use active and specific terms. See Bloom’s taxonomy (in handouts) • Don’t use words like “understand” – go for higher level skills • The meaning of a word can be defined more clearly in the scoring rubric. For example, “sophisticated” from the history SLO.

Check: Simplicity S T E P 2 E • Keep the number down to around 5 • Have more if your outcomes are determined by an outside accrediting body like Nursing. • Concentrate on the most important skills or knowledge the students will come away from the program with and what they will produce to show you that they have mastered those skills.

Worksheet and Discussion S T E P 2 E Take a few minutes to answer the Step 2 questions on your worksheet.

Check Alignment with Courses S T E P 2 F Curriculum Alignment Matrix Course PSLO #1 100 I 120 200 PSLO #2 PSLO #3 PSLO #4 PSLO #5 I I P PSLO #6 P P P 204 P 300 P 329 D 400 P P P D 480 490 D D D A matrix ensures that student have been exposed to the outcome in the program. Can point out problems! D From Assessing Academic Programs in Higher Education by Mary J. Allen p. 43

Program Assessment Form S T E P 2 – Blank form is a document on the campus Assessment Web site – Download and type in your program SLOs with Word – Send document to Division Chair who will forward to Dean and Assessment Coordinator for Web publication. – Program Learning Outcomes are also to be published in the college catalog and all other program materials.

Develop Assessment Plan S T E P 3 A. B. C. D. Select delivery points for assessment Select tool(s) Set timeline Create assessment and rubric

Delivery Point S T E P 3 A Is there a course or set of courses all majors would take before graduating? If not, is there a course many majors take? Give a survey to identify who is a major. See Accounting program example on handouts.

Delivery Point S T E P 3 A If there isn’t a terminal course, • Can you reach students as they are graduating to give out an assessment? – One campus had a free lunch for students who filed petitions • Can you have students build a portfolio as they progress through the program?

Worksheet and Discussion S T E P 3 A Take a few minutes to answer the Step 3 questions on your worksheet.

Choose Assessment Tool(s) S T E P 3 B • Direct Assessment • Indirect Assessment Experts recommend more than one type of data collection method!

Choose Assessment Tool(s) S T E P 3 B Direct methods prompt students to represent of demonstrate their learning or produce work so that observers can assess how well students’ texts or responses fit expectations Includes: Standardized tests Locally developed tests and assignments Projects Presentations Portfolios See handouts

Choose Assessment Tool(s) S T E P 3 B Indirect methods capture students’ perceptions of their learning and the educational environment that supports that learning. – Student surveys – Factbook information Definitions of direct and indirect from Assessing for Learning by Peggy Maki p. 88

Set Timeline S T E P 3 C – Which semester/year will each SLO be assessed during? – The general recommendation is to measure one per year to give time for careful review of results and integration into planning – Send the timeline to the Division Chair and to the Assessment Coordinator

Create Assessment and Rubric S T E P 3 D – Develop the tool (test questions, assignment, survey, etc. ) – Create the scoring guide or rubric – Get input from all of the faculty who will be involved.

For Future Workshops? • Designing assessments and rubrics • Collecting assessment data, analyzing and reporting • Using results to improve the program You tell us!

Thank You For Attending! • Please turn in: q. The Worksheet q. The Assessment Needs Survey q. The evaluation form • Presentation will be available on the campus assessment web pages. • Questions? – Lora Lane x 4178 or campus email – Dean Humphreys x 4025 or campus email