BIRS 2016 Opening the analysis black box Improving
BIRS 2016: Opening the analysis black box: Improving robustness and interpretation Matthew Brown, Ph. D University of Alberta, Canada
Overview 1. About us 2. Preprocessing quality assurance 3. Interpretation of group vs. individual differences 4. Trial type f. MRI signatures
Dept. Psychiatry Dept. Computing Science Computational Psychiatry Group
Clinical decision-making • Diagnosis – What disease? • Prognosis – Predict patient response to treatment options
What are we detecting? • 10 psychosis patients, 10 controls, f. MRI • Highly diagnostic Fourier power distribution from voxels IN THE EYES • Eye movement disturbances in psychosis
ADHD-200 and ABIDE datasets • • n=1000 approx. ADHD patients or autism patients Structural MRI, resting state f. MRI Simple diagnosis – Classify patients vs. controls – Accuracy 50 -70% in various papers • Some papers reported higher 75%+ accuracy BUT cherry-picking sites?
ADHD-200 Global Competition Validation Accuracy (%) • Best-performing algorithm, but did not win • Used only non-imaging features: – Age, gender, handedness, IQ, site of scan – 3 -classification (ADHD-c, ADHD-i, control) – 63% hold-out accuracy (vs. 54% chance) Using non-imaging features Chance accuracy Brown et al. 2012
Histogram of oriented gradient (HOG) features Image from Ghiassian et al. under review. Also see Dalal and Triggs 2005. IEEE Computer Society Conference on. vol. 1. IEEE, p. 886– 893.
ADHD-200 and ABIDE datasets • Ghiassian et al. under review • State of the art (as of 1. 5 years ago) • 2 -classification (patients vs. controls) Chance Non-imaging + Structural MRI Non-imaging + Functional MRI ADHD-200 55% 69% 70% ABIDE 51% 60% 64% 65%
Overview 1. About us 2. Preprocessing quality assurance 3. Interpretation of group vs. individual differences 4. Trial type f. MRI signatures
Registration failure Subject 15 Standard preprocessing methods failed for 1 of 21 subjects. Fixed ->
Inter-site variability ADHD-200 Subjects Projected onto PCA component space PCA Component 2 Each colour is a different scanning site. Even with standard normalization procedures, inter-site structure remains in the data. PCA Component 1 Sen et al. in preparation
Overview 1. About us 2. Preprocessing quality assurance 3. Interpretation of group vs. individual differences 4. Trial type f. MRI signatures
Clinical research One goal: Associate disease with biological features Healthy Huntington’s Image from Wikipedia
ADHD-200 resting state f. MRI functional connectivity analysis ICA Brown et al. 2012
ADHD patients vs. controls Brown et al. 2012 “Default mode” network Patients vs. controls “Desired” simple interpretation: “Patients are different from controls. This difference tells us something about the disease. ”
Group vs. individual differences Patients Controls Brown et al. 2012 Statistically significant group differences, but substantial overlap between individual patients and controls.
Interpretation Patients Controls Brown et al. 2012 • Simple interpretation “patients are different from controls” • Overlap precludes simple interpretation • Yet many papers provide precisely and only the simple interpretation
Overview 1. About us 2. Preprocessing quality assurance 3. Interpretation of group vs. individual differences 4. Trial type f. MRI signatures
Black box analysis Analysis Software
General linear model regression Model voxel i’s timecourse Model matrix for trial type k
Two different models for hemodynamic response function SPM canonical model Finite impulse response model
Check deconvolved timecourses SPM canonical model Finite impulse response model, same region Basically agree on shape (but not statistical differences in this case)
Check deconvolved timecourses SPM canonical model Finite impulse response model, same region Noise in deconvolved timecourses
Another example SPM canonical model Finite impulse response model, same region Noise in deconvolved timecourses
GLM analysis • Check deconvolved timecourses • What is the model fitting – Noise vs. signal • Model selection – regularization
Summary Quality check everything Visualization Intermediate steps and final results Particularly important for non-technical end-users
Acknowledgements People: Azad, Benoit, Dursun, Ghiassian, Greenshaw, Greiner, Juhas, Purdon, Ramasubbu, Rish, Sen, Silverstone Funding: AICML, AIHS, CIHR, Norlien Foundation, AHS, AMHB, UAlberta Questions?
Invitation Continue informing other researchers about analysis pitfalls and caveats. Questions?
Title
- Slides: 30
