Applying Bayesian Belief Networks to the Examination of

Applying Bayesian Belief Networks to the Examination of Student Outcomes Xiaohong Li, Graduate Research Asst. Rita Caso, Director Sam Houston State University Office of Institutional Research & Assessment

Outline § Purpose of the Study § Why Study Freshman Outcomes? § Why Bayesian Networks § Method § Example Inferences § Conclusions TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 2

Purpose § Apply Bayesian Belief Network(BBN) techniques to examine student outcomes for the purpose of identifying families of factors associated with students’ college success at Sam Houston State University (SHSU) § Identify what factors impact retention and graduation for First Time Freshmen (FTF) v Retention and Graduation rates: key performance indicators § Providing management information, analyzing and interpreting these data for using in planning and policy decisions TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 3

Why Study Freshman Outcomes? § To determine if we are providing the best environment & experiences to promote success for our diverse freshman population § To make tailored improvements in the learning environment and the learning experiences we offer in order to maximize successful outcomes for all students across preparation backgrounds, needs, learning styles and life-styles § To satisfy external accountability requirements TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 4

Why Study Freshman Outcomes? § University Stakeholders who need detailed insights into the conditions and combinations of factors that influence new student success: v Enrollment Management v Enrichment and Support Programs v Student Services v Academic Department TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 5

Why Bayesian Network? § Graphical Model with an Associated set of Probability Tables v Learn causal relationships easily v Better understand the problem domain and predict the consequences v Flexible and robust recommendation strategies TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 6

About Bayesian Networks § Definitions of Basic Terms: v Independent Ø v Conditional probability Ø v Ø B The probability of event C occurring, given that event A has already occurred: P(C|A) D E is independent of A and B given D E and F are conditionally independent of each other, given D Causal Theory Ø v A Conditional Independence Ø v Event A does not affect the probability of B occurring: P( A, B) = P(A) * P(B) A or B can cause D to occur Node: variable Ø Leaf Node: no outcome depends on them (E, F) Ø Root Node: do not depend on any outcome (A, B) E TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 F 7

About Bayesian Networks § “A graphical model that encodes probabilistic relationships among variables of interest” § Named “Bayes” after Reverend Thomas Bayes, a British theologian and mathematician who wrote down a basic law of probability v Bayes Rule Smoking TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 Cancer 8

About Bayesian Networks § Bayesian Networks Contain: v A Network Structure: Ø Directed, acyclic (non-circular) graph Ø Encodes a set of conditional independence and dependence information about variables v Probability Ø Probability distributions associated with each variable Ø Represented in the data and computed from the data TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 9

About Bayesian Networks § Example of Bayesian Network v Example Data below is Invented FAID Full/Part FAID Yes No Full/Part Full 0. 4 0. 6 0. 11 0. 99 FAID Yes No 0. 2 0. 8 Retention Full/Part Full Part FAID Yes No TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 Retention Yes No 0. 95 0. 05 0. 8 0. 2 0. 9 0. 1 0. 99 0. 01 10

Method § Data Processing v Data Source: Ø Institutional Research & Assessment Office data files from which Fall FTF cohorts for 2000 through 2006 were extracted v Working Data File Ø Merge extracted FTF Cohort data into aggregated data file Ø Records=13542, variables =216 Ø Dependent variables - retention rate & graduation rate computed from enrollment and graduation variables in working data file Ø Discretization – transform continuous variables into categorical variables TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 11

Method § Developing Bayesian Belief Network (BBN) Model by using a computer application program called Netica. TM 3. 25 v Selection Ø of Variables Input variables selected from commonly used in SHSU IRA Office studies of freshman outcomes Ø Variable selection reinforced by variables used in ‘ Data Mining with Bayesian Belief networks to Examine Retention and Graduation at a Public University’ by P. Edamatsu, D. Jankovic and Pokrajac, presented at AIR 2007 Forum TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 12

Data Description Name Label Type Value 1 Year Retention Discrete 2 Admitted_Hschol. Grad Year Admitted-Graduation Year Discretized 4 College of Students’ Enrollment Discrete 7 Ethnicity Discrete 6 Gender Discrete 2 I_O In-state(I)/Out of State (O) Discrete 2 F_T Full or Part Time Discrete 2 BKLC Bearkat Learning-Community Cohort Discrete 2 PBSP Probation or Suspension Discrete 6 ONOFF Whether or not student lives on campus Discrete 2 FAID Financial Aid Discrete 2 HSrank Rank in High School Discretized 5 SAT_Total SAT Total Score Discretized 6 GPA End of Semester GPA Discretized 8 Graduated_6 yrs 6 year Graduation Discrete 2 TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 13

Method § Assumptions in the model Structure v Graduation and Retention (Dependent Variables) are “leaf nodes” v Gender, Ethnicity, Full/Part, Probation & Suspension (PBSP) are “root nodes” and are independent of each other. TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 14

Method • Building the Model Structure v In order to specify the relationships between the selected variables from PRIOR information, I took inspiration from: Ø Structure used by Edamatsu, D. Jankovic and Pokrajac in their study Ø Knowledge about variables related to dependent outcome variables from other SHSU IRA Office studies Ø Knowledge about relationships between pairs of variables from correlation matrices that included all selected variables TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 15

Structure Encoded with Data Probability TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 16

Main Results § Posteriori Analysis v Students’ gender determines students’ college choice and high school rank v Ethnicity influences students’ college choice. v 1 year retention rate and 6 year graduation rate directly depend on GPA and students’ probation or suspension status v Students’ in-state or out–of-state status and ethnicity related to how many years after high school graduation students applied to the university v Students living on campus perform a little bit better than those living off campus TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 17

Results Pertaining to Gender TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 18

Results Pertaining to Gender TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 19

Results Pertaining to Gender § There is no significant difference in graduation rate and retention rate between males and females. More females’ high school ranks are above the 1 st Q (from the top) than males v Females Ø v Males Ø Tend to study majors in college of Art & Sciences and Humanities & Social Sciences Tend to study majors in college of Art & Sciences and Business Administration. TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 20

Results Pertaining to Ethnicity TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 21

Results Pertaining to Ethnicity TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 22

Results Pertaining to Ethnicity TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 23

Results Pertaining to Ethnicity TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 24

Results Pertaining to Ethnicity Ø No significant difference in graduation rate and retention rate among ethnicities Ø Native Americans are less likely (86. 7%) to attend university within 1 year after high school compare to other ethnicities (around 95%), and 91% are in-state students, while 99% of other ethnicities are in-state. Ø 46. 6% of White Americans enrolled in college of Arts and Sciences, compare to 39% of other ethnicities. Ø 94% of African Americans live on campus, compare to 75% - 86% of other ethnicities. • TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 25

Results Pertaining to GPA TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 26

Results Pertaining to GPA TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 27

Results Pertaining to GPA TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 28

Results Pertaining to GPA § Bearkat Learning Community students have a higher probability of having a higher GPA § Students with low GPA (below 2) v Have only 27% graduation rate and 55% 1 year retention rate § Students with higher GPA (2 to 2. 5) v Have 43% graduation rate and 75% retention rate § Students with highest GPA (above 3. 75) v Have 70% graduation rate and 85% retention rate TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 29

Results Pertaining to Probation and Suspension TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 30

Results Pertaining to Probation and Suspension TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 31

Results Pertaining to Probation and Suspension TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 32

In-State/Out-of-State Status TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 33

In-State/Out-of-State Status TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 34

On / Off Campus Living TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 35

On /Off Campus Living TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 36

Results Pertaining to Probation and Suspension, In or out of State and Living on or off Campus § Students on probation or suspended in the first year v Have only 22% graduation rate and 45% retention rate § Good standing students v Have 53% graduation rate and 76% retention rate. § Out-of-state students are less likely (87%) to attend university within 1 year after high school, compared to in-state students (95%). § There are no GPA distribution differences between in-state students and out-of-state students § Students living on campus have a slightly higher GPA, retention rate and graduation rate. TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 37

Conclusion § Bayesian Belief Networks are good tools for analyzing institutional research data § BBN is a powerful methodology for graphically demonstrating probability theory and can provide good references for university administration § Users could have difficulty using BBN if they do not have sufficient data or theory base to provide prior probabilities. This is particularly problematic when exploring a previously unknown network § The validity and reliability of prior beliefs used in Bayesian inference processing are critical. If this prior knowledge is not reliable, then the Bayesian network is not useful TX Association of Institutional Research (TAIR) 2008 Conference, 2/5 -7/08 38

Bibliography 1. P. Edamatsu, D. Jankovic and Pokrajac, Data Mining with Bayesian 2. David Heckerman, A Tutorial on Learning with Bayesian Networks, 3. Bruce G. Marcot, What Are “Bayesian Belief Network Models? ”, 2005 4. Castillo, E. , J. M. Gutierrez and A. S. Hadi Expert Systems and Probabilistic Network Models. Springer Verlag, 1997 5. Jie Cheng, Russell Greiner, Learning Bayesian Belief Network Classifiers: Algorithms and System 1995 Belief networks to Examine Retention and Graduation at a Public University, presented at AIR 2007 Forum 1997 39

Questions? 40