MPI Mission Perception Inventory Institutional Characteristics and Student

MPI Mission Perception Inventory Institutional Characteristics and Student Perception of Mission: What Makes a Difference? Ellen Boylan, Ph. D. ● Marywood University NEAIR 36 th Annual Conference ● Baltimore, Maryland ● Nov. 7 – 10 , 2009

What’s in a mission? 2

scope of research Purpose Develop an instrument to measure student perception of institutional mission. Test instrument reliability. Uncover constructs (factors). Observe constructs longitudinally. 3 GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

scope of research Purpose Develop an instrument Test instrument reliability. Research Questions 1. Is the Mission Perception Inventory (MPI) a valid and reliable measure of student perception of institutional mission? 2. What are the factors in the MPI? 3. Do the factors recur in repeated administrations of the revised MPI? 4. Are the factors equally reliable over time? Uncover constructs. Observe constructs longitudinally. Develop a prediction 5. Can a school’s performance be predicted? equation. 4

¡ leaders of public and private institutions alike are about spirituality thinking these days, as the data suggest that's what their students are thinking about, too (Inside Higher Ed, 2009). background connection between institutional programs and student learning environment (Pascarella, 2001). ¡ There is strong ¡ …institutions influence levels of engagement on campus as a result of structural features, programs, policies, and organizational culture (Kuh et al. , 2005). 5

design Select the National Survey of Student Engagement (NSSE) as the vehicle for inserting research questions. Assemble a NSSE consortium to jointly engage in research to explore student perception of mission. Develop question items; administer them to the consortium as a NSSE attachment. Test the questions: reliability, factor analysis, and correlation analysis. Repeat annually. 6 GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

derivation of the MPI and subscales Sense Of Mission Administer Consortium mission questions (20 items) Mission Perception Inventory Respect for Diversity (MPI) (~19 items) Individual Values Spiritual Practice 7

• The mission of this institution is widely understood by students. • Social and personal development is an important part of the mission. • • sense of mission (10 items =. 90) Ethical and spiritual development of students is important. This institution offers opportunities for volunteering and community service. • This institution offers opportunities for developing leadership skills. • • There are opportunities for students to their religious commitment. strengthen This institution’s religious heritage is evident. Professors here discuss the ethical implications of what is being studied. As a result of my experience here, I am more aware of my own personal values. • The mission of this institution is reflected in course offerings. 8

respect for diversity (5 items =. 878) The faculty, staff, and students here… • respect different religions respect different races and cultures ……………………………………… • • Students feel free to express individual spirituality. • Different sexual orientations are accepted. • The environment encourages appreciation of diversity. 9

administration The NSSE survey with attached Mission Perception Inventory (MPI) questions has been administered 182 times to close to 50, 000 first-year and senior students at 112 unique institutions across the United States every year since 2004. 10 GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Consortia institutions and respondents by year Respondents Institutions First Year Senior 2004 15 2, 000 1, 827 2005 16 1, 279 1, 332 2006 24 2, 684 2, 854 2007 36 4, 533 4, 331 Catholic 2008 35 4520 5063 Indepen’t 2008 19 6805 6970 145* 21, 821 22, 377 *(112 unique institutions) 11

trations of the is in m ad ed at pe re in r cu re s or Do the fact ntory (MPI) ve In n io pt ce er P on si is M d se vi re 12

? e m i t r e v o e l b a i l e r y l l a u q e s r o t c a Are the f 2004 2005 2006 2007 2008 Catholic 2008 Indept cronbach cronbach Sense of Mission . 87 . 88 . 90 . 91 . 88 Respect for Diversity . 85 . 84 . 87 . 86 . 88 . 87 Individual Actions . 67 n/a n/a n/a Religious Practice/Spirituality . 62 . 55 . 54 . 62 . 64 MPI scale . 88 . 89 . 90 . 91 . 90 Subscales 13

(MPI) Mission Perception Inventory Report 14 GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Teagle Foundation Research Continues 2009… • Develop the Mission Engagement Index. • • • Obtain NSSE 2009 consortia data Test using reliability analysis to produce factors Distribute MPI reports Compare consortia results Conduct regression analysis to create the new index Use the Mission Engagement Index (MEI) to assess performance. 15 GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

NSSE 2008 Benchmark Statistics by Consortia 16

ribe sc de x e d In t en m ge ga En on Does the Missi ct fe af at th s le ab ri va g on am s ip causal relationsh mission perception? • Dependent variable (Institution score) Mission Perception Inventory • Sense of Mission scale • Respect for Diversity scale • • Independent variables (2008 data; need 15 cases per variable*) • Selectivity 15 institutions • Enrollment 30 institutions • Urbanicity 45 institutions • Resident % 60 institutions • Female % 75 institutions Another? 90 provide institutions *…a recommended ratio of • subjects to IVs of at least 15 to 1 will a reliable regression equation (Stevens, 17 1992).

Selecting IVs for regression analysis 18

participating institutions by region Is there sufficient variability? 2008 = 2 consortia, 54 institutions 19

“urbanicity” of participating institutions 2008 Is there sufficient variability? 20

MPI 2008 Scales: Comparison of means by institution type 21

Correlation of regression variables with MPI mean 22

Deriving coefficients for the Mission Engagement Index (MEI) 23

Correlation of regression variables with Mission scale mean 24

Predictive Equation* Institution Predicted MPI Score = (Beta 1)*(Value of “setting”) (-0. 414)*(1, 2, or 3) (Beta 2)*(Value of “institution type”)+ (0. 345)*(1 or 2) Constant (3. 687) *Mortenson, T. (1997). Actual Vs Predicted institutional graduation rates for 1100 Colleges and universities. Opportunity, 58. 25

Proposed Mission Engagement Index ¡ ¡ The MEI will compare actual versus predicted scores on mission constructs. Progress on mission effectiveness can be assessed by comparing MEI outcomes to institutional goals. 26

Sample Mission Engagement Index (MEI) by Institution Type and Setting 27

¡ For statistical regression, cross validation with a second sample is highly recommended (Tabachnick, p. 153). No rest for the weary… 28

References Inside Higher Education (2009). Spiritual accountability. Retrieved from http: //www. insidehighered. com/layout/set/pri. . . assessment/01/02/2007/News Kuh, D. G. , Kinzie, J. , Schuh, J. H. , and Whitt, E. J. (2005). Never let it rest: lessons about student success from high-performing colleges and universities. Change, 37(4), 44 -51. Mortenson, T. (1997). Actual Vs predicted institutional graduation rates for 1100 colleges and universities. Opportunity, 58. Pacarella, E. T. (2001). Identifying excellence in undergraduate education. Change, 33(3), 18 -27. Stevens, J. P. (1992). Applied multivariate statistics for the social sciences (2 nd edition). Hillsdale, New Jersey: Erlbaum. Tabachnick, B. G. & Fidel, L. S. (1996). Using multivariate statics, third edition. New York, NY: Harper. Collins. 29

Discussion: 30

Institutional Characteristics & Student Perception of Mission: What Makes a Difference? Ellen Boylan, Ph. D. Director of Institutional Research and Assessment eboylan@marywood. edu Office of Planning and Institutional Research http: //cwis. marywood. edu/instresearch/activity. stm Marywood University Supported by a grant from http: //www. teaglefoundation. org/grantmaking/grantees/assessmentmethods. aspx 31

32

MPI Mission Perception Inventory Institutional Characteristics and Student Perception of Mission: What Makes a Difference? Ellen Boylan, Ph. D. ● Marywood University NEAIR 36 th Annual Conference ● Baltimore, Maryland ● Nov. 7 – 10 , 2009