Explaining models for predicting atrisk students Look under

Explaining models for predicting at-risk students – Look under the bonnet Jakub Kocvara, Dr. Martin Hlosta, Prof. Zdenek Zdrahal

What is OU Analyse? • Creating a predictive model that identifies students at risk of failing the module as early as possible so that OU intervention is efficient and meaningful. • Tutor facing dashboard (Early Alert Indicators) • Supporting around 250 modules at the moment

How does a machine learning model work? Current presentation demographic factors previous study results online activity Training data Previous presentation demographic factors previous study results online activity assignment result Machine Learning algorithm Predicted assignment result Prediction data Model Predicted outcome

What are some properties of a model? Training speed – training can be a very expensive operation Interpretability how much do we understand the inner workings of a model Interpretability Predictive power – e. g. accuracy often connected to complexity Complexity

Interpretable models Last assignment score > 60 Last week online activity Linear model, decision trees, … Forum Posts > 5 Not Submit Last assignment score Submit Online activity last week = Yes Not Submit

”Black box” models More complex models can be more accurate, but are inherently harder to interpret Large decision trees, ensemble methods, neural networks, …

Why is intepretability important? Trust – providing tutors with reasoning behind model’s decisions “If the users do not trust a model or a prediction, they will not use it” – Ribeiro, et al. (2016) Error checking – assessing validity of the model by analysing suspicious predictions Can point to incomplete or wrongly chosen training dataset Ethical and legal reasons – GDPR’s “right to explain”

Explaining predictions in OU Analyse Explanations generated by an independent Naïve Bayes model Some intrinsic disadvantages Expecting independent variables Explanations are usable only when Naïve Bayes agrees with the main model

The importance of variables in the model

LIME – Local Interpretable Model-agnostic Explanations Takes an individual prediction and tweaks individual features to create new instances Weighs them by proximity to the original Tries to train a simple model (i. e. linear regression), that approximates the black-box model well in the vicinity of the original prediction Evaluates which feature values confirm or contradict black-box model’s hypothesis

LIME – Husky vs. wolf anecdote

Future of OU Analyse Work in progress: Incorporating LIME module into the predictions pipeline Goal is to have every student’s prediction explained and showing on tutor’s dashboard This would aid tutors in while deciding whether to act on the basis of a prediction or not Increased trust in our models would lead to increased adoption and ultimately retention

Sources "Why Should I Trust You? ": Explaining the Predictions of Any Classifier (2016) – Ribeiro, Singh, Guestrin Interpretable machine learning (2019) – Cristoph Molnar A Unified Approach to Interpreting Model Predictions (2017) – Lundberg, Lee