Reliable Meaningful Communication Madhu Sudan Microsoft Research August

Reliable Meaningful Communication Madhu. Sudan Microsoft Research August 6, 2015 KAIST: Reliable Meaningful Communication 1 of 27

This Talk n n n Part I: Reliable Communication n Problem and History (briefly) Part II: Recovering when errors overwhelm n Sample of my work in the area Part III: Modern challenges n Communicating amid uncertainty August 6, 2015 KAIST: Reliable Meaningful Communication 2 of 27

Part I: Reliable Communication August 6, 2015 KAIST: Reliable Meaningful Communication 3 of 27

Reliable Communication? n Problem from the 1940 s: Advent of digital age. We are not ready We are now ready Alice n Bob Communication media are always noisy n But digital information less tolerant to noise! August 6, 2015 KAIST: Reliable Meaningful Communication 4 of 27

Reliability by Repetition n August 6, 2015 KAIST: Reliable Meaningful Communication 5 of 27

Shannon’s Theory [1948] n Alice August 6, 2015 Encoder Decoder KAIST: Reliable Meaningful Communication Bob 6 of 27

Shannon’s Theorem n August 6, 2015 KAIST: Reliable Meaningful Communication 7 of 27

Shannon’s contributions n Far-reaching architecture: Alice n n Encoder Decoder Bob Profound analysis: n First (? ) use of probabilistic method. Deep Mathematical Discoveries: n Entropy, Information, Bit? August 6, 2015 KAIST: Reliable Meaningful Communication 8 of 27

Challenges post-Shannon n August 6, 2015 KAIST: Reliable Meaningful Communication 9 of 27

Progress 1950 -2010 n n Profound contributions to theory: n New coding schemes, decoding algorithms, analysis techniques … n Major fields of research: n Communication theory, Coding Theory, Information Theory. Sustained Digital Revolution: n Widespread conversion of everything to “bits” n Every storage and communication technology relies/builds on theory. n “Marriage made in heaven” [Jim Massey] August 6, 2015 KAIST: Reliable Meaningful Communication 10 of 27

Part II: Overwhelming #errors August 6, 2015 KAIST: Reliable Meaningful Communication 11 of 27

Explicit Codes: Reed-Solomon Code n August 6, 2015 KAIST: Reliable Meaningful Communication 12 of 27

Overwhelming Errors? List Decoding n August 6, 2015 KAIST: Reliable Meaningful Communication 13 of 27

Reed-Solomon List-Decoding Problem n August 6, 2015 KAIST: Reliable Meaningful Communication 14 of 27

Decoding by example + picture [S’ 96] n Algorithm idea: Find algebraic explanation of all points. n n Stare at the solution (factor the polynomial) n n August 6, 2015 KAIST: Reliable Meaningful Communication 15 of 27

Decoding by example + picture [S’ 96] n Algorithm idea: Find algebraic explanation of all points. n n Stare at the solution (factor the polynomial) n n August 6, 2015 KAIST: Reliable Meaningful Communication 16 of 27

Decoding Algorithm n August 6, 2015 KAIST: Reliable Meaningful Communication 17 of 27

Part III: Modern Challenges Communication Amid Uncertainty? August 6, 2015 KAIST: Reliable Meaningful Communication 18 of 27

New Kind of Uncertainty n Uncertainty always has been a central problem: n n n Modern complication: n n n But usually focusses on uncertainty introduced by the channel Rest of the talk: Uncertainty at the endpoints (Alice/Bob) Alice+Bob communicating using computers Huge diversity of computers/computing environments Computers as diverse as humans; likely to misinterpret communication. Alice: How should I “explain” to Bob? Bob: What did Alice mean to say? August 6, 2015 KAIST: Reliable Meaningful Communication 19 of 27

Example Problem n n Archiving data n Physical libraries have survived for 100 s of years. n Digital books have survived for five years. n Can we be sure they will survive for the next five hundred? Problem: Uncertainty of the future. n What formats/systems will prevail? n Why aren’t software systems ever constant? August 6, 2015 KAIST: Reliable Meaningful Communication 21 of 27

Challenge: n n If Decoder does not know the Encoder, how should it try to guess what it meant? Similar example: n Learning to speak a foreign language n Humans do … (? ) n n Can we understand how/why? Will we be restricted to talking to humans only? Can we learn to talk to “aliens”? Whales? Claim: n Questions can be formulated mathematically. n Solutions still being explored. August 6, 2015 KAIST: Reliable Meaningful Communication 22 of 27

Modelling uncertainty Uncertain Communication Model Classical Shannon Model A 1 B 1 A 2 A A 3 B 2 Ak August 6, 2015 Channel B B 3 New Class of Problems New challenges Needs more attention! KAIST: Reliable Meaningful Communication Bj 23 of 27

Modern questions/answers n n Communicating players share large context. n Knowledge of English, grammar, socio-political context n Or … Operating system, communication protocols, apps, compression schemes. But sharing is not perfect. n Can we retain some of the benefit of the large shared context, when sharing is imperfect? n Answer: Yes … in many cases … [ongoing work] n n New understanding of human mechanisms New reliability mechanisms coping with uncertainty! August 6, 2015 KAIST: Reliable Meaningful Communication 24 of 27

Language as compression n 1 2 August 6, 2015 KAIST: Reliable Meaningful Communication 25 of 27

Summary n Reliability in Communication n Key Engineering problem of the past century n Led to novel mathematics n Remarkable solutions n Hugely successful in theory and practice n New Era has New Challenges n Hopefully new solutions, incorporating ideas from … n n Information theory, computability/complexity, game theory, learning, evolution, linguistics … … Further enriching mathematics August 6, 2015 KAIST: Reliable Meaningful Communication 26 of 27

Thank You! August 6, 2015 KAIST: Reliable Meaningful Communication 27 of 27