AI for Medicine Lecture 17 Logistic Regression Part
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-14.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-16.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-17.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-19.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-20.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-21.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-22.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-23.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-24.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-25.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-26.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-27.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-28.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-29.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-30.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-31.jpg "A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: Testing • Predicted Class [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-33.jpg "A Concrete Example: Testing • Predicted Class [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: Testing • Predicted Class [1, 1, 1, 0, 1, 1] 1](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-34.jpg "A Concrete Example: Testing • Predicted Class [1, 1, 1, 0, 1, 1] 1")
![A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-35.jpg "A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331")
![A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-36.jpg "A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331")
![A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331](https://slidetodoc.com/presentation_image_h2/b0799aa97bfca77b47e5035b2a24776d/image-37.jpg "A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331")
- Slides: 42
AI for Medicine Lecture 17: Logistic Regression- Part II March 29, 2021 Mohammad Hammoud Carnegie Mellon University in Qatar
Today… • Last Monday’s Session: • Logistic Regression – Part I • Today’s Session: • Logistic Regression- Part II • Announcement: • Assignment 2 grades are out
Outline Logistic Regression Classification Hypothesis Function Cost Function Gradient Descent
Learning a Logistic Regression Model • ≣
Gradient Descent For Logistic Regression • Partial derivative
Gradient Descent For Logistic Regression • The final formula after applying partial derivatives
Inference After Learning •
A Concrete Example: The Training Phase • and vaccine the of nigeria y Email a 1 1 0 1 1 1 Email b 0 0 1 1 0 0 Email c 0 1 1 0 0 1 Email d 1 0 0 Email e 1 0 1 1 Email f 1 0 1 1 0 0 A Training Dataset
A Concrete Example: The Training Phase • and vaccine the of nigeria y Email a 1 1 0 1 1 1 Email b 0 0 1 1 0 0 Email c 0 1 1 0 0 1 Email d 1 0 0 Email e 1 0 1 1 Email f 1 0 1 1 0 0 1 entails that a word (i. e. , “and”) is present in an email (i. e. , “Email a”)
A Concrete Example: The Training Phase • and vaccine the of nigeria y Email a 1 1 0 1 1 1 Email b 0 0 1 1 0 0 Email c 0 1 1 0 0 1 Email d 1 0 0 Email e 1 0 1 1 Email f 1 0 1 1 0 0 0 entails that a word (i. e. , “and”) is abscent in an email (i. e. , “Email b”)
A Concrete Example: The Training Phase • y Email a 1 1 0 1 1 1 Email b 0 0 1 1 0 0 Email c 0 1 1 0 0 1 Email d 1 0 0 Email e 1 0 1 1 Email f 1 0 1 1 0 0
A Concrete Example: The Training Phase • The feature vector y Email a 1 1 1 0 1 1 1 Email b 1 0 0 1 1 0 0 Email c 1 0 1 1 0 0 1 Email d 1 1 0 0 Email e 1 1 0 1 1 Email f 1 1 0 0 To account for the intercept
Recap: Gradient Descent For Logistic Regression • First, let us calculate this factor for every example in our training dataset
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0. 5 [1, 0, 1, 1, 0, 0] 1 -0. 5 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
Recap: Gradient Descent For Logistic Regression •
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 [1, 0, 0, 1, 1, 0] 0 [1, 0, 1, 1, 0, 0] 1 [1, 1, 0, 0, 1, 0] 0 [1, 1, 0, 1] 1 [1, 1, 0, 1, 1, 0] 0
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 [1, 0, 0, 1, 1, 0] 0 [1, 0, 1, 1, 0, 0] 1 [1, 1, 0, 0, 1, 0] 0 [1, 1, 0, 1] 1 [1, 1, 0, 1, 1, 0] 0
Recap: Gradient Descent For Logistic Regression •
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0. 5 [1, 0, 1, 1, 0, 0] 1 -0. 5 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0. 5 [1, 0, 1, 1, 0, 0] 1 -0. 5 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0. 5 [1, 0, 1, 1, 0, 0] 1 -0. 5 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0 [1, 0, 1, 1, 0, 0] 1 0 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0 [1, 0, 1, 1, 0, 0] 1 0 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0 [1, 0, 1, 1, 0, 0] 1 0 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0 [1, 0, 1, 1, 0, 0] 1 -0. 5 [1, 1, 0, 0, 1, 0] 0 0 [1, 1, 0, 1] 1 0 [1, 1, 0, 1, 1, 0] 0 0
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0 [1, 0, 1, 1, 0, 0] 1 0 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 0 [1, 0, 0, 1, 1, 0] 0 0. 5 [1, 0, 1, 1, 0, 0] 1 -0. 5 [1, 1, 0, 0, 1, 0] 0 0 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0 [1, 0, 1, 1, 0, 0] 1 0 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0. 5 [1, 0, 1, 1, 0, 0] 1 0 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 0 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 -0. 5 [1, 0, 0, 1, 1, 0] 0 0 [1, 0, 1, 1, 0, 0] 1 0 [1, 1, 0, 0, 1, 0] 0 0. 5 [1, 1, 0, 1] 1 -0. 5 [1, 1, 0, 1, 1, 0] 0 0. 5
A Concrete Example: The Training Phase • [1, 1, 1, 0, 1, 1] 1 [1, 0, 0, 1, 1, 0] 0 [1, 0, 1, 1, 0, 0] 1 [1, 1, 0, 0, 1, 0] 0 [1, 1, 0, 1] 1 [1, 1, 0, 1, 1, 0] 0 -0. 5 0 0 0 -0. 5 0
A Concrete Example: Testing •
A Concrete Example: Testing • Predicted Class [1, 1, 1, 0, 1, 1] 1 0. 622459331 [1, 0, 0, 1, 1, 0] 0 0. 377540669 [1, 0, 1, 1, 0, 0] 1 0. 622459331 [1, 1, 0, 0, 1, 0] 0 0. 377540669 [1, 1, 0, 1] 1 0. 622459331 [1, 1, 0, 1, 1, 0] 0 0. 377540669
A Concrete Example: Testing • Predicted Class [1, 1, 1, 0, 1, 1] 1 0. 622459331 [1, 0, 0, 1, 1, 0] 0 0. 377540669 [1, 0, 1, 1, 0, 0] 1 0. 622459331 [1, 1, 0, 0, 1, 0] 0 0. 377540669 [1, 1, 0, 1] 1 0. 622459331 [1, 1, 0, 1, 1, 0] 0 0. 377540669
A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331 [1, 0, 0, 1, 1, 0] 0 0. 377540669 [1, 0, 1, 1, 0, 0] 1 0. 622459331 [1, 1, 0, 0, 1, 0] 0 0. 377540669 [1, 1, 0, 1] 1 0. 622459331 [1, 1, 0, 1, 1, 0] 0 0. 377540669
A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331 1 [1, 0, 0, 1, 1, 0] 0 0. 377540669 0 [1, 0, 1, 1, 0, 0] 1 0. 622459331 1 [1, 1, 0, 0, 1, 0] 0 0. 377540669 0 [1, 1, 0, 1] 1 0. 622459331 1 [1, 1, 0, 1, 1, 0] 0 0. 377540669 0
A Concrete Example: Testing • [1, 1, 1, 0, 1, 1] 1 0. 622459331 1 [1, 0, 0, 1, 1, 0] 0 0. 377540669 0 [1, 0, 1, 1, 0, 0] 1 0. 622459331 [1, 1, 0, 0, 1, 0] 0 0. 377540669 NO 1 Mispredictions! [1, 1, 0, 1] 1 0. 622459331 1 [1, 1, 0, 1, 1, 0] 0 0. 377540669 0 0
A Concrete Example: Inference • y Email a 1 1 1 0 1 1 1 Email b 1 0 0 1 1 0 0 Email c 1 0 1 1 0 0 1 Email d 1 1 0 0 Email e 1 1 0 1 1 Email f 1 1 0 0 Email k 1 0 0 1 ? Our Training Dataset
A Concrete Example: Inference • y Email a 1 1 1 0 1 1 1 Email b 1 0 0 1 1 0 0 Email c 1 0 1 1 0 0 1 Email d 1 1 0 0 Email e 1 1 0 1 1 Email f 1 1 0 0 Email k 1 0 0 1 ?
A Concrete Example: Inference • Class 1 (i. e. , Spam)
A Concrete Example: Inference • y Email a 1 1 1 0 1 1 1 Email b 1 0 0 1 1 0 0 Email c 1 0 1 1 0 0 1 Email d 1 1 0 0 Email e 1 1 0 1 1 Email f 1 1 0 0 Email k 1 0 0 1 1 Somehow interesting since it considered “vaccine” and “nigeria” indicative of spam!
Next Monday’s Class… • Deep Learning- Part I