ML for Nonverbal Behavior Analysis in Schizophrenia Talia

ML for Non-verbal Behavior Analysis in Schizophrenia Talia Tron Prof. Daphna Weinshall, Prof. Abraham Peled Py. Con 2018

Once upon a time… Ph. D in computational neuroscience

How to save the world? Psychiatric Diagnosis I feel very depressed lately Sounds like you suffer from depression

Focus on Schizophrenia Positive signs Non-verbal Behavior Negative signs

Schizophrenia classification 3 D Two Phase SVM based Classifier BUT 88% Accuracy 85% Recall

Schizophrenia classification • We ignore: • Other clinical conditions • Clinical subpopulations • Continuous monitoring • Non communicative • There are better signs

How to become clinically relevant? • Black box Communication • Schizophrenia vs. Control Single patient over time clinical sub-population • One time Continuous Monitoring • Context Natural context

How to become clinically relevant? • Black box Communication • Schizophrenia vs. Control Single patient over time clinical sub-population • One time Continuous Monitoring • Context Natural context

Facial expression in Schizophrenia Control Patient Ø Flat affect Ø Facial dynamics Ø Facial clusters Ø Emotional content time

Facial expression in Schizophrenia • What is your name? Elsa. Flat affect • How are you today? Sad. Inappropriate affect

Facial expression in Schizophrenia

Facial expression in Schizophrenia • No inappropriate affect when taking flatness into account (Analysis of variance)

Not enough!! • Black box Communication • Schizophrenia vs. Control Single patient clinical sub-population • One time Continuous Monitoring • Context Natural natural context

Not enough!! • Black box Communication • Schizophrenia vs. Control Single patient clinical sub-population • One time Continuous Monitoring • Context Natural natural context

Continuous motor monitoring • 25 Patients monitored for about a month • 50 Hz Accelerometers • Basic Measures - number of steps, energy square, energy variance (dynamics)

Continuous motor monitoring • Detect abnormal behaviors (ARIMA model) ŷt = μ + ϕ 1 yt-1 +…+ ϕp yt-p - θ 1 et-1 -…- θqet-q p = number of autoregressive terms q = Number of lagged forecast errors

Continuous motor monitoring • Detect abnormal behaviors (ARIMA model) • Compare with clinical conditions & drug usage Symptom severity Lithium Look at the WIDER picture

Not enough!! • Black box Communication • Schizophrenia vs. Control Single patient clinical sub-population • One time Continuous Monitoring • Context Natural natural context

Continuous motor monitoring • Look on clinical sub-populations (Patient X Day) Negative Factor Negative Baseline Positive Factor

Continuous motor monitoring • Look on clinical sub-populations • Take context into account (regular daily activities)

Continuous motor monitoring • Look on clinical sub-populations • Take context into account (regular daily activities) • Communicate results Negative Baseline Positive

Continuous motor monitoring • Look on clinical sub-populations • Take context into account (regular daily activities) • Communicate results • Regression & Classification (Precision = 0. 757, Recall = 0. 748 )

Enough? • Black box Communication • Schizophrenia vs. Control Single patient clinical sub-population • One time Continuous Monitoring • Context Natural natural context

Enough? • Black box Communication • Schizophrenia vs. Control Single patient clinical sub-population • One time Continuous Monitoring • Context Natural natural context • Beyond basic measures

Beyond basic motor measures • Convert continuous signal to “motor words” Acceleration DEDCBABCDEDBABCDEDBA Time

Beyond basic motor measu • Convert continuous signal to “motor words” NLP! YAY!

Beyond basic motor measu • Convert continuous signal to “motor words” • LDA (Topic modeling)

Beyond basic motor measures • Convert continuous signal to “motor words” • LDA (Topic modeling) How diverse is patient’s motor activity How steady is it in similar context How similar is it to other patients

Beyond basic motor measures

There’s still a lot to do… • Automatic extraction of objective, quantitative and clinically relevant behavioral measures. • Improve the reliability of psychiatric diagnosis • Allow better patients’ characterization which may assist both research and treatment. • Can not yet replace a human care giver. • Far from production • Privacy and voluntary cooperation must be taken into account.

AND THEY LIVED HAPPILY EVER AFTER Data Scientist @ Intuit

Acknowledgments • Prof. Daphna Weinshall • Prof. Abraham Peled M. D. • Mikhail Bazhmin M. D • Prof. Alexander Grinshpoon M. D. • Dr. Yehezkel S. Resheff • Elena Dahan M. D. SHA’AR MENSAHE Mental Health Center

Q&A