MASTER OF SCIENCE IN DATA SCIENCE Deep Learning

MASTER OF SCIENCE IN DATA SCIENCE Deep Learning in Medical Physics: When small is too small? Miguel Romero, Yannet Interian Ph. D, Gilmer Valdes Ph. D, and Timothy Solberg Ph. D *Was partially supported by the Wicklow AI and Medicine Research Initiative at the Data Institute.

SECTIONS ● ● ● MOTIVATION &PROBLEM EXPERIMENTS RESOURCES &TECHNIQUES RESULTS DISCUSSION

SECTIONS ● ● ● MOTIVATION &PROBLEM EXPERIMENTS RESOURCES &TECHNIQUES RESULTS DISCUSSION

Motivation Diagnosis Diabetic Retinopathy MNA MASTER OF NONPROFIT ADMINISTRATION

Question Is life that easy? = Can we do the same in our images and problems? MNA MASTER OF NONPROFIT ADMINISTRATION

Performance as a function of the # data available MNA MASTER OF NONPROFIT ADMINISTRATION

Performance as a function of the # data available Medical Physics MNA MASTER OF NONPROFIT ADMINISTRATION

Performance as a function of the # data available DL < Ridge ? ! MNA MASTER OF NONPROFIT ADMINISTRATION

Our problem: When small data is too small? MNA MASTER OF NONPROFIT ADMINISTRATION

Our problem: When small data is too small? ● ● MNA MASTER OF NONPROFIT ADMINISTRATION Complexity? Amount of noise?

SECTIONS ● ● ● MOTIVATION &PROBLEM EXPERIMENTS RESOURCES &TECHNIQUES RESULTS DISCUSSION

#1 Set of Tasks: "With real labels" MNA MASTER OF NONPROFIT ADMINISTRATION

#1 Set of Tasks: "With real labels" MNA MASTER OF NONPROFIT ADMINISTRATION

#1 Set of Tasks: "With real labels" ? Pneumonia MNA MASTER OF NONPROFIT ADMINISTRATION

#1 Set of Tasks: "With real labels" 50 50 MNA MASTER OF NONPROFIT ADMINISTRATION 2000 # of samples 1600

#2 Set of Tasks: "With synthetic labels" MNA MASTER OF NONPROFIT ADMINISTRATION

#2 Set of Tasks: "With synthetic labels" Original Dataset [0/1] MNA MASTER OF NONPROFIT ADMINISTRATION

#2 Set of Tasks: "With synthetic labels" Original Dataset Model X f [0/1] MNA MASTER OF NONPROFIT ADMINISTRATION

#2 Set of Tasks: "With synthetic labels" Original Dataset Model X f Synthetic Dataset [0/1] Threshold selection & Hard labels creation MNA MASTER OF NONPROFIT ADMINISTRATION

#2 Set of Tasks: "With synthetic labels" Original Dataset Model X f Synthetic Dataset [0/1] Threshold selection & Hard labels creation MNA MASTER OF NONPROFIT ADMINISTRATION

#2 Set of Tasks: "With synthetic labels" Original Dataset Model X f Synthetic Dataset [0/1] Threshold selection & Hard labels creation MNA MASTER OF NONPROFIT ADMINISTRATION Model Y g

#2 Set of Tasks: "With synthetic labels" Original Dataset Model X f Synthetic Dataset Model Y g [0/1] Threshold selection & Hard labels creation # of samples MNA MASTER OF NONPROFIT ADMINISTRATION

SECTIONS ● ● ● MOTIVATION &PROBLEM EXPERIMENTS RESOURCES &TECHNIQUES RESULTS DISCUSSION

Data & algorithms Datasets ● NIH Chest X-rays → detect 14 diseases ● RSNA Bone Age → Detect the age (under/over) median Deep Learning ● Res. Net 18 → 18 layers → 11 M parameters ● Res. Net 34 → 34 layers → 21 M parameters ● Dense. Net 121 ● MURA → Abnormality detection ● Image. Net → 121 layers → 9 M parameters ● Custom CNN → 7 -9 layers → Image classification MNA MASTER OF NONPROFIT ADMINISTRATION ML/Statistical models ● Logistic Regression → 90 parameters ● Ridge → 90 parameters ● Lasso → 90 parameters ● Elastic Net → 90 parameters ● Random Forest → variable # parameters

Rough idea Datasets ● NIH Chest X-rays → detect 14 diseases ● RSNA Bone Age → Detect the age (under/over) median Deep Learning ● Res. Net 18 → 18 layers → 11 M parameters ● Res. Net 34 → 34 layers → 21 M parameters ● Dense. Net 121 ● MURA → Abnormality detection ● Image. Net → 121 layers → 9 M parameters ● Custom CNN → 7 -9 layers ML/Statistical models ● Logistic Regression → 90 parameters ● Ridge → 90 parameters ● Lasso → 90 parameters ● Elastic Net → 90 parameters ● Random Forest → variable # parameters → Image classification 4 data-sets ≃ M parameters MNA MASTER OF NONPROFIT ADMINISTRATION ≃ H parameters

DL techniques: ● Data Augmentation ● Transfer learning from Image. Net & MURA. ○ CNN as feature extractor ○ Fine tuning the CNN - all at once ○ Fine tuning the CNN - progressive unfreezing + differential learning rates ● Training approaches (some). ○ Regular lr policy ○ One-cycle cosine with annealing lr policy and momentum ○ Adam optimizer ● Testing Time data Augmentation (TTA) MNA MASTER OF NONPROFIT ADMINISTRATION

Rough idea ● Data Augmentation Multiple ways of doing DL ● Transfer learning from Image. Net & MURA. ○ CNN as feature extractor ○ Fine tuning the CNN - all at once ○ Fine tuning the CNN - progressive unfreezing + differential learning rates ● Training approaches (some). ○ Regular lr policy ○ One-cycle cosine with annealing lr policy and momentum ○ Adam optimizer ● Testing Time data Augmentation (TTA) MNA MASTER OF NONPROFIT ADMINISTRATION

SECTIONS ● ● ● MOTIVATION &PROBLEM EXPERIMENTS RESOURCES &TECHNIQUES RESULTS DISCUSSION

Real labels - Approach 1 DL Pneumonia 14 diseases MNA MASTER OF NONPROFIT ADMINISTRATION Linear Model

Improves previous approach Pneumonia 14 diseases MNA MASTER OF NONPROFIT ADMINISTRATION

Not that good in perspective Pneumonia 14 diseases MNA MASTER OF NONPROFIT ADMINISTRATION

SECTIONS ● ● ● MOTIVATION &PROBLEM EXPERIMENTS RESOURCES &TECHNIQUES RESULTS DISCUSSION

Discussion ● DL ≠ Magic. ● Production level models requires large amounts of rich data. ● DL seems to improve traditional approaches "earlier" in the binary balanced task. ● If trained appropriately, improves traditional approaches with small data but not small enough for most medical phyisics problems. MNA MASTER OF NONPROFIT ADMINISTRATION

THANK YOU, QUESTIONS ?