Guardian Angels and Cognition Amplifiers AI 2 0

Guardian Angels and Cognition Amplifiers: AI 2. 0 Technologies in Support of Humankind Raj Reddy Carnegie Mellon University Pittsburgh, PA 15213 Sep 28, 2016

AI 2. 0: Harnessing Intelligence from Many Sources · Big Data Intelligence: · Data Driven Learning of Models and Algorithms · Cross Media Intelligence: · Social Media Intelligence Gathering · From All Language Sources · From All the Media: Text, Speech, Image and Video · Crowd Intelligence: · Global Brain: from Individual Intelligence to Collective Intelligence · Augmented Intelligence: · Human-Machine Hybrid Intelligence for Collaborative Problem Solving · UAV (Unmanned Autonomous Vehicle) Intelligence: · Intelligence from Collaborating Teams of Robots

Guardian Angels and Cognition Amplifiers Augment Intelligence of Humankind • Continuous Monitoring of Sensors on the User Smart Phone Enables Activity • • Monitoring of the “Opt-In” Subset of Humankind Activity Patterns of the User (and Others with Similar Profile) can be Used to Infer User Intentions Intention Awareness enables Creation of Personal Guardian Angels (Gats) and Cognition Amplifiers (Cogs) • • • Cogs are created by continuously monitoring multimodal sensors on modern smart phones, and discovering and interpreting user activities and intentions thru non-intrusive learning technologies. The Activity Monitoring System non-intrusively adapts and creates personalized Cogs using Cog Templates from the Platform providers and Cog developers Gats are created by monitoring Global, National and Local events and Sources of Interest and discovering and interpreting their potential impact to the individual user and notifying the user at the right time in the right medium. • Gats and Cogs enable Automation of Much of the Human Work and Daily • Personal Activities Than One-size-fits-all Apps Maxim is “Activity Monitoring” enables “Intention Awareness” enable “Gats and Cogs” enables “ 80%+ Automation”

What Gats and Cogs Are and Do • Thousands of Gats and Cogs help each person on the planet • • • including people affected by language-divide and literacy-divide for routine tasks such as purchasing, productivity, play games, and for demanding tasks such as problem solving, explanation, negotiation, and discovery • Autonomic - Always On • Humans usually not in the loop • Cogs in Cloud - Not on Smart Phone • Gigabit Wireless leads to Infinite Memory and Bandwidth? • Invisible Computing (on Body) replaces Laptops and Smart Phones • Conserve Human Attention • buy human assistance as needed thru crowd sourcing? • Multimodal communication among Cogs • Context Adaptive • Continuous Monitoring of Location, Touch, Smell, Taste, Speech, Vision • Learning algorithms (continuously adapting) • Lifelong Adaptation and Learning (Temporal) • Problem Solving (Tasks): Clarification Dialog and Confirmation • Speech Centric

AI 2. 0 Enables Guardian Angels for Humankind • A Guardian Angel is a Personal Enduring Autonomic Intelligent Agent that Discovers And Warns You About Unanticipated Events That Could Impact Your Safety, Security, and Happiness • Just-in-Time Warnings: Hurricanes, Earthquakes, Extreme Weather • Act as Coach in Health and Education Matters • Accident Alerts and Rerouting; Transport Strikes • Scarcity of Essential Resources: Food, Energy, Water etc. • Guardian Angels (Gats) are Examples of Use of AI 2. 0 Technologies in Support of Humankind. They Use and Need • Big Data Intelligence • Cross Media Intelligence • Crowd Intelligence • Augmented Intelligence • UAV Intelligence

A Separate Gat for Every Skill Family of Guardian Angels For Each Person • Personal: Guardian Angels for Personal Essentials • Health • Education • Food Security • Energy Security • Water Security • Other: Medicine/Pharmaceuticals, Shelter/Housing, Sanitation • National/Local: Guardian Angels for Necessities • Transportation • Telecom/Smartphone/Communication/Wi. Fi • Banking • Entertainment • Shopping • Global - Guardian Angels for Disasters • Earthquakes/Tsunami • Typhoons/Hurricanes/Cyclones • Fire • Flooding

Guardian Angel For Health • Monitor Life Support Parameters • Using Improvised Smart Phone • Using a Low Cost Version of Apple Watch and Fitbit • Body Media Implant or Capsule in the Future? • Analyze the Data for Tell Tale Signals • Notify and/or Warn the User of Impending Problems • Guardian Angel Service Providers For Health • Devices, Tools and Gat Apps

Guardian Angels For Education • Keystroke Activity Monitoring Exposes Student Learning Behavior • Attendance • Paying Attention or Playing games? • Learning Speed • Problem Solving Speed • Valuable Tool for Student, Parents and Educators • Timely Notifications and Warnings • Guardian Angel Service Providers For Education • Devices, Tools and Gat Apps

Guardian Angels For Emergencies Provide the Right Information to the Right People · Right Information · To the Right People · At the Right Time · In the Right Language · I the Right Medium: Voice, Video and/or Text · In the Right Level of Detail

2004 Tsunami · The 2004 Indian Ocean Earthquake Occurred at 00: 58: 53 UTC on 26 Dec 2004 · With the Epicentre Off The West Coast Of Sumatra, Indonesia. · The Shock Had a Moment Magnitude of 9. 1– 9. 3 And a Maximum Mercalli Intensity of IX (Violent). · Caused When The Indian Plate was Subducted by The Burma Plate · Triggered A Series Of Devastating Tsunamis Along The Coasts Of Most Landmasses Bordering The Indian Ocean, · Killing 230, 000– 280, 000 People in Indonesia, Sri Lanka, Thailand etc. · Inundating Coastal Communities With Waves Up To 30 Metres (100 Ft) High. · It Was One Of The Deadliest Natural Disasters In Recorded History

Tsunami 2004: Right Information to the Right People at the Right Time in the Right Language in the Right Medium (Voice, Video and/or Text) in the Right Level of Detail • Step 0: Determine the Implications of Tsunami 2004 · Step 1: Right People: Find the affected people on the Coast and their Smart Phone info. Wake up if Needed · Step 2: Right Time: In enough time to get out of Harms Way · Step 3: Right Language Synthesize a message in the his/her local Dialect · Step 4: Send Text, Audio or Video Message · Step 5: Only that “ 30 meter waves will hit in 15 minutes. Get to High Ground”. Suppress other details

Access to Right Information: Determine the Implications of Tsunami 2004 · Find and Filter the Sources of Information · · Global Experts notify National Experts who Notify State Experts who Notify Local Experts who Notify all the Local Guardian Angels · Publish/Subscribe Model · Identify Impacted Regions, Expected Time of Arrival · Send Only Notifications that are Actionable by GAs · Include all information that is needed by GA · e. g. 100 Foot Waves · Relevant Information · Many events do not affect you · Your GA needs to analyze all the relevant (local) Published Notifications to see which are of immediate threat

Determining the Right (Impacted) People · Right People: Find the affected people on the Coast and · Their Smart Phone info. · Wake up if Needed

At the Right Time · Right Time: In enough time to get out of Harms Way

What Language? Text or Audio? How Much Detail? · Step 3: Right Language Synthesize a message in the his/her local Dialect · Step 4: Send Text, Audio or Video Message · Step 5: Only that “a 35 foot wave will hit in 15 minutes. Get to High Ground”. Suppress other details

AI 2. 0 Enables Cognition Amplifiers for Humankind • A Cognition Amplifier is a Personal Enduring Autonomic Intelligent Agent that anticipates what you want to do and helps you to do it with less effort • • • Buying and selling: Transact with multiple providers Email: Filter spam, understand respond to actionable email News: Based on topic preferences, novelty, collaborative filtering Banking: Monitor bank account, Credit Cards, Pay Bills Travel: Flights, hotel, schedule disruptions, cancellations • Thousands of Cogs and Gats as Personal Assistants?

Shopping: Book Ordering. What the User Experiences User Get me a copy of the book Black Swan. I’ll get The Black Swan: Second Edition by Nassim Taleb for your Kindle Great My Shopping Assistant

EXAMPLE: Book Ordering. What Really Happens User My Dialog Agent Get me a copy of the book Black Swan. My Shopping Assistant User utters a command Dialog agent parses voice to text, decodes meaning, recognizes it as a shopping request and passes to shopping assistant

EXAMPLE: Book Ordering. What Really Happens Shopping assistant consults with other assistants to get information on user preferences and context that will help it do the job correctly. My Shopping Assistant Shopping assistant already knows shopping preferences (paper book or kindle, shipping method, payment method. recent purchases) does not need to ask another agent for those. My Web Monitor Has he searched for anything like this, or been reading any web pages about it? My Banking Assistant Books are usually $10 -50. Does he have the money right now? My GPS Monitor Library of Congress Is he home or travelling? He likes paperbacks, but I’ll put it on his kindle if he’s not near home. What books are called “Black Swan”? I want to make sure I have the right one.

EXAMPLE: Book Ordering. What Really Happens Armed with context, shopping assistant can confidently reply to user. User My Shopping Assistant I’ll get The Black Swan: Second Edition by Nassim Taleb for your Kindle Great.

EXAMPLE: Book Ordering. What Really Happens My Shopping Assistant Amazon Walmart Asks for quotes from a variety of service agents, and then selects the best one and executes the purchase.

Architecture of Cogs and Gats Publish and Subscribe • Cogs and Gats are Mobile Apps (like a shadow) for Each Person on the Planet • • Unlike APPs of today, Cogs are Mass-customized to Each Individual Designed to be Non-intrusive, Autonomic, and Device Independent Always On, Always Present and Always Working Always Learning • Enduring (life-long) • Cogs and Gats Monitor, Analyze and Learn From Experience; • • Learn From Own Experience And Experience of Others • And share knowledge with a community of Cogs and Gats Automated Discovery of Data and Information Sources • Gats and Cogs Publish and Subscribe Anonymized Data of User Activities and Experiences • Data, suitably anonymized, can be used to learn appropriate responses for every possible situation by • • Learning preferences by observing user choices, Learning by task similarity and user similarity, Learning by error correction and Simply learning thru clarification dialog ( does that mean yes? Would you care to define it? )

Technologies of Cogs and Gats • • Cogs and Gats Publish and Subscribe Service Agents are created by service providers using agent templates included in the platform • Download and Pay just as for Apps and Personalization Guardian Angels Can request and manage services on behalf of their masters. Every organization that wants to enable access to their services by Guardian Angels will create a Service Agent for this purpose. Opt-in “Waze” like Models Privacy • • Individual Other participants Legal • Can be subpoena-ed Security • Information falling into wrong hands

Publish/Subscribe Eco System of Guardian Angels and Cognition Amplifiers Global Infrastructure Platform Activity Monitoring and Intention Awareness User Guardian Angels Cloud Based User Infrastructure Platform Knowledge Source Publishers Global, National and Local Knowledge Source Distributors Re. Tweet Model Global Data. Base of Humankind

Personalization and Customization of Guardian Angels and Cognition Amplifiers Cloud Based Guardian Angel Platform Linked to User Smart Phone Learning • Learn by Watching • Learn by being taught • Learn by doing • Learn by asking others • Learn by discovery Power Mgmt Security • Multimodal authentication • Continuous authentication • Encryption Activity Monitoring Big Data Management • Process, index, story and retrieve data • Mine, Cluster, and Summarize relevant experience Multi-Sensor Integration Dialog • With humans • With other agents Autonomic Systems Self Healing • Detect errors • Resolve errors Etc. Family of Personalized Guardian Angels and Cognition Amplifiers Subscribe to National and Global Sources and Act on Relevant Information

What Cogs and Gats Are Not • One System Fits All • Designed for Mass Use • User Activated APPs • Assume Human in the loop always • Intentional Activation • Require Laptops or Smart Phones • Consume Human Attention • Texting instead of Voice and Vision • Context Insensitive • Fixed algorithms (non-changing) • Non Temporal

Potential Economic Impact of Guardian Angels and Cognition Amplifiers • Every person on the planet will be able to perform many daily habits more effectively using Gats and Cogs • Daily habits (routines, activities) include a wide spectrum from routine tasks (such as banking and travel planning) to tasks too difficult for the user • Over 80% of all human activity will done by Cogs by 2020 • 7 Billion People Market Vs 2 Billion Today • Ultimately Humans Could be 10 Times More Efficient and Effective • Global GDP is $100 Trillion • Even 10% improvement will lead to $10 T additional wealth creation • Gat and Cog Eco System Requires Platform Providers, App Providers, and Agents for Personal Customization • All of Whom Would Benefit from the Additional Wealth Creation