Teaching an Agent by Playing a Multimodal Memory

Teaching an Agent by Playing a Multimodal Memory Game: Challenges for Machine Learners and Human Teachers AAAI 2009 Spring Symposium: Agents that Learn from Human Teachers March 23 -25, 2009, Stanford University Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and Engineering Cognitive Science, Brain Science, and Bioinformatics Seoul National University, Seoul 151 -744, Korea btzhang@cse. snu. ac. kr http: //bi. snu. ac. kr/

Talk Outline l Multimodal Memory Game (MMG) l Challenges for Machine Learners l Challenges for Human Teachers l Toward Self-teaching Cognitive Agents 2 © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/

Toward Human-Level Machine Learning: Multimodal Memory Game (MMG) But, I'm getting married tomorrow Well, maybe I am. . . I keep thinking about you. But, I'mwondering getting married And if we tomorrow made a mistake giving up so fast. Well, I am. . . Are youmaybe thinking about me? I keep thinking about But if you are, call me you. tonight. And I'm wondering if we made a mistake giving up so fast. Are you thinking about me? But if you are, call me tonight. Text Hint Image Hint Sound Image-to-Text Generator (I 2 T) Machine Learner Text-to-Image Generator (T 2 I) © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/

Text Generation Game (from Image) Image Sound Text I 2 T Learning by Viewing

Image Generation Game (from Text) Image Sound Hint I 2 T Learning by Viewing

Characteristics of MMG Game l l l l Interactive Multimodal Long-lasting Hard to learn Scalable data Humans as teachers Difficulty controllable Learning by imitation (viewing and watching) 6 © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/

Three Approaches l Learning Architecture ¨ Model l Learning Strategies ¨ Algorithms l Teaching

Methods of Machine Learning l Symbolic Learning l ¨ ¨ ¨ Version Space Learning ¨ Case-Based Learning l Neural (Connectionist) Learning ¨ Multilayer Perceptrons ¨ Self-Organizing Maps ¨ Hopfield Networks l Evolutionary Learning ¨ ¨ Evolution Strategies Evolutionary Programming Genetic Algorithms Genetic Programming Probabilistic Learning l Bayesian Networks Boltzmann Machines Hidden Markov Models Deep Belief Networks Hypernetworks Other Machine Learning Methods ¨ ¨ ¨ Reinforcement Learning Decision Trees Boosting Algorithms Kernel Methods (SVM) PCA, ICA, LDA etc.

Learning with Hypernetworks x 1 x 2 [Zhang, DNA 12 -2006] x 15 x

How to Learn from Image-Text Pairs 10 © 2009, SNU Biointelligence Lab, http: //bi.

How to Generate Image from Text 11 © 2009, SNU Biointelligence Lab, http: //bi.

Image-to-Text Translation Results Query Matching & Completion Answer I don't know what happened I don't know what happened There's a a kitty in … in my guitar case There's a kitty in my guitar case Maybe there's something I … I get pregnant Maybe there's something I can do to make sure I get pregnant 12 © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/

Text-to-Image Translation Results Matching & Completion Query Answer I don't know what happened Take a look at this There's a kitty in my guitar case Maybe there's something I can do to make sure I get pregnant 13 © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/

Further Challenges

Challenges for Machine Learners l l l l l Incremental learning Online learning Fast update One-shot learning Predictive learning Memory capacity Selective attention Active learning Context-awareness Persistency l Concept drift l Multisensory integration l 15 © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/

Challenges for Human Teachers l l l l l Getting feedback Sequencing examples Identifying the weak points Choosing problems Controlling parameters Evaluating progress Estimating difficulty Generating new queries Modeling the effect of learning parameters Catching environmental change l Minimal interactions l Multimodal interaction l 16 © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/

Conclusion l Multimodal memory game (MMG) ¨ Highly-interactive lifelong learning scenario ¨ Challenges current machine learning techniques l Challenges for machine learners ¨ More attentive, active behavior ¨ Rather than parameter fitting, passive adaptation l Human partners ¨ More active role in interacting with the agents l The future: Self-teaching cognitive agents ¨ Cognitive learning agents that teach themselves = Active learning agents + cognitively-aware human teachers ¨ Design new queries and test their answers by interacting with humans 17 © 2009, SNU Biointelligence Lab, http: //bi. snu. ac. kr/