Generative Models for probabilistic inference Michael Stewart Remember

Generative Models for probabilistic inference Michael Stewart

Remember the Joint Distribution?

What about very large/complex models?

"Generative" Modeling Implement probability theory in computer science to infer a joint distribution Bayesian prior -> posterior provides learning opportunity Sampling methods are their own field of study Applications in neuroscience, machine learning, biology [1] Requires a model and sampling method. . .

"Generate" examples

Define a Model This is a broad procedure; what is the goal? Bio: a. observe f. MRI data, infer latent locality function in brain b. observe genome data, infer latent gene relationships AI: a. observe words, infer latent topics or semantic information

How to sample Rejection sampling: no Usually a kind of Gibbs/MCMC

Some implementations: a. [2] BLOG • [3] Church • [4] Python modules provided by Tom Haines

Recommended reading A. Daud, J. Li, L. Zhou, and F. Muhammad, "Knowledge discovery through directed probabilistic topic models: a survey, " Frontiers of Computer Science in China, vol. 4, no. 2, pp. 280 -301, Jun. 2010. [Online]. Available: http: //dx. doi. org/10. 1007/s 11704 -009 -0062 -y M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, 2007. [Online]. Available: http: //www. worldcat. org/isbn/1410615340 N. Goodman, J. Tenenbaum, T. O'Donnell, and the Church Working Group. Probabilistic Models of Cognition http: //projects. csail. mit. edu/church/wiki/Probabilistic_Models_of_Cognition Coin examples: Church Learning as Conditional Inference

References [1] A. Venkataraman, Y. Rathi, M. Kubicki, C. -F. Westin, and P. Golland, "Joint generative model for f. MRI/DWI and its application to population studies. " Medical Image Computing and Computer-Assisted Intervention, vol. 13, no. Pt 1, pp. 191 -199, 2010. [Online]. Available: http: //www. pubmedcentral. nih. gov/articlerender. fcgi? artid=3056120&tool=pmcentrez&rend ertype=abstract [2] B. Milch, B. Marthi, S. Russell, D. Sontag, D. L. Ong, and A. Kolobov, "Blog: Probabilistic models with unknown objects, " in In IJCAI, 2005, pp. 1352 -1359. [Online]. Available: http: //citeseerx. ist. psu. edu/viewdoc/summary? doi=10. 1. 1. 116. 2131 [3] N. D. Goodman, V. K. Mansinghka, D. Roy, K. Bonawitz, and J. B. Tenenbaum, "Church: a language for generative models, " in Uncertainty in Artificial Intelligence, 2008. [Online]. Available: http: //web. mit. edu/droy/www/papers/Goo. Man. Roy. Bon. Ten. UAI 2008. pdf [4] http: //code. google. com/p/haines/ plate notation example: M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, 2007. [Online]. Available: http: //www. worldcat. org/isbn/1410615340