Probability for Machine Learning Foundations of Algorithms and
Probability for Machine Learning Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 1
Probabilistic Machine Learning • Not all machine learning models are probabilistic • … but most of them have probabilistic interpretations • Predictions need to have associated confidence • Confidence = probability • Arguments for probabilistic approach • • Complete framework for Machine Learning Makes assumptions explicit Recovers most non-probabilistic models as special cases Modular: Easily extensible Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 2
References • “Introduction to Probability Models”, Sheldon Ross • “Introduction to Probability and Statistics for Engineers and Scientists”, Sheldon Ross • “Introduction To Probability”, Dimitri P. Bertsekas, John N. Tsitsiklis Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 3
Basics • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 4
Random Variables • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 5
Discrete Random Variables • Probability mass function Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 6
Example distributions: Discrete • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 7
Continuous Random Variables • Probability density function Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 8
Example density functions • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 9
Random Variables • Cumulative distribution function Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 10
Moments • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 11
Random Vectors and Joint Distributions • Discrete Random Vector • Joint pmf • Continuous Random Vector • Joint pdf Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 12
Example multi-variate distributions • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 13
Random Vectors and Joint Distributions • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 14
Conditional Probability • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 15
Conditional Probability • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 16
Independence and Conditional Independence • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 17
Covariance • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 18
Central Limit Theorem • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 19
Notions from Information Theory • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 20
Jensen’s Inequality • Foundations of Algorithms and Machine Learning (CS 60020), IIT KGP, 2017: Indrajit Bhattacharya 21
- Slides: 21