Using AI to Eliminate COVID Lockdowns An Agenda

Using AI to Eliminate COVID Lockdowns An Agenda for the Post-Pandemic Era Raj Reddy Carnegie Mellon University Presented at CMU and Microsoft Research Asia, Beijing, China, June 16, 2020 Keynote Speech at the 4 th World Intelligence Congress, Tianjin, China (Shorter Version) June 23, 2020

Synopsis of the Presentation • Covid-19 Pandemic has the whole world reeling in turmoil, and it will lead to unprecedented • • changes in the way we live, learn and work around the world Any solution that makes it unnecessary to have lockdown will require identification and quarantining of most of the COVID infected people. It is not necessary to be 100% accurate. 90% accurate diagnosis would reduce R 0 by a factor of 10, good enough to stop the exponential spread of the Virus. Early Detection and Isolation of the Infected may be possible using recent advances in AI and Machine Learning thru continuous monitoring of vital signs of a population • A modern smartphone and smartwatch combo can be used to monitor Vital Signs highly correlated with Covid-19, and can be used to quickly identify and isolate persons at high risk contacts within a few meter radius • Some of the vital signs will show significant change one or two days before • A Phone. Watch can measure temperature, heart rate, blood pressure, breathing rate, and oxygen levels • • automatically at regular intervals. Additional vital signs such as aches and pains, loss of appetite, loss of taste and smell, alertness and reaction times and urine output can be entered as a survey once a day A Phone. Watch can also use proximity sensing to keep track of visible manifestation of symptoms. Using sensor data on over an extended period of time, we can predict infections few days ahead of the Emergence Visible Symptoms.

COVID-19 Leads to Lockdowns Around the Globe in 2020 • Lockdowns and Quarantines is over 500 -year-old concept. be no need for lockdowns. In 21 st century, there may • BTW, Newton did his greatest work under lock down in 1665 escaping the great plague. year, he invented calculus, optics, laws of motion, and gravity. In one • We Need to Find Effective Mechanisms to Control the Exponential Spread • Given an Infection Rate of 5 per Generation, • • In about 12 Generations, everyone in US would be Infected In about 14 Generations, everyone in the World would be infected • • Not Enough Beds, Doctors, Nurses and Ventilators No Coffins and No place to Bury the Dead • Communities Are Unable to React in a Timely Manner • Overwhelming the National Healthcare Resources • Exponential Spread of COVID-19 Is Scary to Behold • Lockdown May Not Be Needed, If the Infected-Persons Can Be Identified and Isolated • Number of Deaths will Go Down by a Factor of 5 to 10, if an Infected Persons Is Isolated Quickly and Can Get Timely Medical Treatment • Even if No One Has Been Vaccinated and No Effective Contact Tracing

Different Countries Have Tried Different Strategies to Control COVID-19 • Italy, Spain and UK have Delayed Lockdowns • Have over 500 Deaths Per Million • US is lower at 324/M • Japan and Sweden had no shutdown, but focused on finding clusters of infections and attacking the underlying causes, which often prove to be overcrowded gathering spots such as nightclubs. • Norway has 1/10 as Many Deaths as Sweden Per Capita • Denmark has only 1/4 as Many Deaths Per Capita • On the other hand, India which has less than 10% of the healthcare resources (beds, doctors, nurses) compared to Sweden or Japan • must use lockdown, to control the spread and unnecessary deaths • An undisciplined population requires stronger measures Death No of Rate/Million No Hospital No Nurses/10000 Population Beds/100000 Doc/100000 0 Australia 4 384 370 1260 Brazil 153 220 1010 China 3 434 200 270 100 261 400 1030 Egypt 11 50 50 190 Finland 58 328 380 1470 France 445 598 330 1150 Germany 102 800 420 1320 4 53 90 170 556 318 400 570 Japan 7 1305 240 1220 Korea South 5 1227 240 1820 Norway 43 360 290 730 Qatar 16 250 850 Russia 36 805 400 130 South Africa 13 90 90 730 Saudi Arabia 17 260 570 4 230 550 Spain 598 297 390 620 Sweden 450 222 400 1180 Turkey 55 281 180 270 UAE 27 250 570 UK USA 585 324 254 277 280 260 820 1450 Denmark India Italy Singapore

Dealing With a Pandemic • Current Solutions to The Pandemic Problem • Restrict Face to Face Contact • Order People to Stay At Home • Prohibit Large Gatherings • Movie Theaters • Stadiums • Close Schools • Shut Down All Air, Train And Bus Travel. • Unintended Consequences • Unemployment • Economy, GDP and Income Tax Revenue down • Stranded Passengers • Universities Begin Hybrid/Remote

A Vision for The Future • No infected cases per million and deaths per million in US are second highest in the world. • If We Can Reliably Identify Non-Infected, there will be no need for shutdowns • At worst about 1% are infected and Less than 0. 1% die • What is needed is a plan to have enough Hospital Beds, Doctors and Nurses • At Least to the level of Japan and Sweden • The Remaining Uninfected 99% are also in Lockdown Unnecessarily Affecting Economy • Biomedical testing is too slow and too little (about 5% test so far) • AI and ML Have Been Able to Discover Patterns Not Obvious To Humans • Past Breakthroughs in AI Enabled by Big Data and Machine Learning • • • Language Translation • Google Translate: Any Language to Any Language • Unrehearsed Spontaneous Speech from Open Populations (USSOP): Siri, Cortana, Alexa Speech to Speech Dialog Autonomous Vehicles: CMU, Stanford, Google, Tesla · Big Data Driven AI systems can Solve Previously Unsolvable Problems · AI 2. 0: Using Previously Unavailable Sources of Data · Knowledge from Big Data, Cross Media, Crowd Intelligence, etc. • Use Data Driven Knowledge to Identify COVID Infected Persons • Biomedical Sensors on Most Smart Phones and Smart Watches May Provide the Data Needed

Objective: Use AI 2. 0 to Discover and Isolate High-Risk Individuals And Permit Non-Social Distancing of Non-Infected

Discover and Isolate COVID Infected • Recent Contact Tracing Apps Use Thermal Screening, Time and Location information (for entry into airports) • Sports events, Classrooms, Theaters, and Weddings are Still Shutdown • Instead of Testing at the Last Minute (e. g. at Entry of Airports), Using Smartphone and Smartwatch-based Sensors for Monitoring may be Helpful • Monitor Typical Signs and Symptoms of COVID-19 Present at Onset of Illness: • • Symptoms Fever (83– 99%) Shortness of breath (31– 40%) Cough (59– 82%) Fatigue (44– 70%) Loss of Appetite (40– 84%) Sputum production (28– 33%) Myalgia (11– 35%) Phone. Watch Measurements Direct: Temperature, Heart Rate, Blood Pressure Direct: Respiratory Rate/Oxygen Level Direct: Microphone Direct: Reaction Time Indirect via Survey: Appetite Indirect via Survey : Mucus Indirect via Survey : Aches and Pain • Use AI and ML techniques to Monitor “Vital Signs Data” to Identify and Isolate Infected Patients • Rapid Identification of Infected Persons using Phone. Watch based Apps • Monitor Every 4 Hours or Less

Sensors on Smart Watches

Precision Epidemiology • Precision Epidemiology is the Art and Science of Early Detection and Isolation of the Infected thru Continuous Monitoring of Vital Signs of a Population within a few meter radius and 1 to 2 Days Before the emergence visible symptoms using sensors on a phonewatch over an extended period of time • Some of the vital signs will show significant change one or two days before visible manifestation of symptoms • Trace, Test and Treat to prevent the virus from spreading and lower the mortality rate • R 0, the reproduction number, is the number of new infections estimated to stem from a single case of an infectious disease. • Exponential spread if R 0>1. 0. R 0 is 5. 0 can Lead to a Million Infections in 10 Generations • Traditionally controlling the spread meant a complete lockdown • It is in Society’s interest to try to reduce R 0 to below 1. 0 asap • An R 0 of below 1. 0 is all that is needed to stop the exponential spread • If 90% or More Infected Patients Can Be Detected and Quarantined Rapidly Using Phone. Watch Based Diagnosis, R 0 can be Reduced by a Factor 10 • an R 0 of 5 can be lowered to an R 0 of 0. 5 • Using data collected from personal Phone. Watches may be the “silver bullet” • To Quickly Identify and Isolate Persons of High Risk and Enable Precision Epidemiology • Phone. Watch sensors can monitor heart rate, breathing rate, temperature, blood pressure, and oxygen

Key Observations • A modern Phone. Watch can test for fever, heart rate, breath sounds, and low blood oxygen • levels as well as time and location information. A Phone. Watch can be used to measure all vital signs every few minutes and look for significant changes • Test for Fever, Heart Rate, Breath Sounds, and Low Blood Oxygen • This will permit isolation of such individuals in minutes and hours instead of days, and stop the • • spread quickly Key Strategy: Get R 0 below 1. 0 The goal is not to eliminate the viruses (there are too many) Key Intuition: In a next-gen contact tracing app, if we can continuously monitor for temperature, heart rate, breathing rate, and oxygen levels of everyone in a closed group, we may be able use AI deep learning technology to identify and isolate infected people • • Then the community of non-infected can go about their business without social distancing. • Arogya Setu App in India requires a Smartphone and downloaded App for entry into airports

Diagnosis from Data • Eight vital signs: Temperature, Heart Rate, Blood Pressure, Breathing Rate, Oxygen Level, Pain, Alertness, Urine Output • First five can be measured automatically every hour using a Phone. Watch • Last three are subjective and can be obtained thru a 20 -sec-daily-survey • Assuming that we have such data, we can calculate mean and standard deviation for each parameter for each person over a month and a year • Establish a normal state for each person. • When one or more parameters is 2 sigma away, then there is a cause for concern. • When all the parameters are within accepted range, then there is no need for Social Distancing of a Non-Infected Person • Asymptomatic persons contribute less than 4% to the spread. Contact tracing can be used to track down potential persons at risk, only if they do not have Phone. Watch-based monitoring system providing a daily clean bill of health.

Potential Showstoppers • Everyone in Closed Community (Schools, Sports Events, Theaters, and Travelers) Must Have A Phone. Watch • 10 Billion Phones Costing $100 Each Will be a $Trillion • US spent over $6 trillion in Covid Relief just to support US Economy, about 4% of the Global Population. • A Combination of Phone. Watch, a Modular System, is Needed • Current UI has to Be Improved. Current Systems Require Self Policing & Discipline • Temperature and Oxygen can now be measured by Holding Finger to Camera, but Not Yet Autonomic • Nex. Gen Phone. Watchs That can Measure Temperature, Heart Rate, Coughs, and Oxygen Levels Autonomically Will be Needed • Need to Add Biomedical Sensing Functionality to Contact Tracing Apps • Current Contact Tracing Apps Could Have Added Some of the Functionality But Did Not • NHS/ Apple Google Apps/ Arogya Setu App may Require a Phone. Watch, in the Future • Collecting and Archiving Data From The Entire Population on The Planet Will Require Huge Computational Resources • 1010 Times 109 Bytes per Year (3 Megabyte Per Day Person), about 10 Exabytes! • Every Phone. Watch is a Supercomputer and Has Many GB of Storage. Keep Data and Computation Local

COVID Economics Can the World Afford to Give Everyone a Phone. Watch? • World Population is 7. 8 Billion (7, 800, 000) • Cost is about $100 to $150 for a respectable phone • Total 12 Cost of providing a phone for everyone may be Under a Trillion Dollars • ($10 ) Global GDP $90 Trillion (US is 21, China 13, and India $2. 8) • Cost of Global Lockdown for two months = 90/6 =$15 Trillion • Cost of One-Phone-For-Everyone (OPFE) About 6. 7% of Global Lockdown • Cost of OPFE as a fraction US stimulus of $6 Trillion is 16% • Cost of OPFE in China is 1% of GDP ((1. 44 B*100)/$14. 4 T) • Cost of OPFE in India is 4% of GDP ((1. 38 B*100)/$3. 2 T)

Why Now? What is New? • AI and Machine Learning Have Made It Possible to Solve Previously • • Unsolvable Problems Such as Identifying Covid Positive Persons Smart Phones and Smart Watches have Sensors that we did not have 5 years ago We have the ability to Access Body Vital Signs of Everyone: • Body Temperature, Heart Rate, Blood Pressure, Breathing Rate, Oximetry • Also Crackly Breath Sounds, Cough, Headache • Complementary Measures of Time and Location • When Most People Have a Phone. Watch with Healthcare Apps, Public Health • Officials will be able to Isolate Covid Positive Persons and Free the Rest of Us from Lockdown Given the Cost of the Pandemic, Perhaps a Phone. Watch Might be Provided to Each Citizen • Should be Added to Maslow's Hierarchy of Needs • Air, Water, Food, Energy, Shelter, Education, Health etc

Action Items • Initiate Data Collection using Phone. Watches • Time and Location • Proximity Sensing • Temperature, Blood Pressure, Heart Rate, Oxygen level, Height, and Weight • Create a privacy preserving open source app for organizations to adopt to enable them to operate freely in a COVID environment. • The App should both identify and isolate High-risk individuals as well as certify No -Risk individuals • No-Risk Individuals may be permitted to operate freely without any COVID constraints such as masks and social distancing • Develop AI based Profiles and Patterns for Identification Infected • Persons with Six Sigma Accuracy. No more than 1 in a million error. Initiate Development of Next. Gen Smart Phone. Watch to Include Sensors for Biomedical Data.

Gentle Slope Systems No Knee-Jerk Reactions • Lockdown with no warning may be harmful to people least able help themselves • Migrant workers and day-laborers • Instead of a Total Shutdown for 6 to 10 weeks, gradual weekly relaxation in Non-Hotspots, and Not-Infected • Continuous Monitoring using IT as in Disaster Management for Cyclones with and Locally Empowered Authority to dynamically relax rules local would be better • Data Analytics may be OK, if Bio Analytics are too slow or too expensive • Instead of shutdown for two whole months, move to facemasks and handwashing after a week • No mask no entry strategy • Realistically, not everyone will have a smart phone. • Create solutions for Smaller Groups like a classroom, theater or stadium • Contact Tracing App may be preferred to shutting down Airtravel • Even if it leads to concerns about privacy • Use Phone. Watches for continuous monitoring

Summary • During 1918 Spanish Flu Pandemic, India had 17 million deaths • Primarily because of poor health care infrastructure. • Even today India still has fewer beds, doctors and nurses per capita than developed countries • Developing countries like India can not afford lockdowns. Every month the country is in a lockdown, the GDP will be reduced by 5% or more. • Or about 150 billion dollars, about 40% about of the annual budget of India. . • If every person in closed group such as a university, work-place or airplane has a Phone. Watch with biomedical sensors that can provide a clean bill of health via Bluetooth connection at the entry-point • Such a community of non-infected can go about as usual and not worry about social distancing. • Infected persons then be sent for further medical testing (and isolation if needed), while the rest • can proceed through a green channel to events such as sports, schools and weddings There will be no need for a lockdown.

In Conclusion… • Routine monitoring of all the people on the Planet will improve our ability to Predict and Prevent most illnesses, not just Viral Infections. Phone. Watches with Biomedical Sensors can be used to discover most illnesses • Besides infectious diseases, many chronic problems such as hearing, vision, dental, anemia, diabetes and hypertension can be continuously monitored and treated • Improving societal health and wellbeing at negligible additional costs. • Cost of providing a Phone. Watch to everyone in India (or China) would be around $140 Billion. Providing 7. 8 billion Smart Phone. Watchesto everyone around globe can eliminate lockdowns in the future. • Such a policy will cost about a trillion dollars, a small fraction of US spending of $6 trillion for Covid-19 stimulus • Violations of Privacy Must Be Handled Head-on • All data must be anonymized; access only on need-to-know-basis • Differential privacy may be used by combining data of individuals with similar profiles.

Backup Slides

Benefits of Phone. Watch Based Digital Testing • Regular Non-Intrusive Phone. Watch Based Monitoring of Vitals Signs Data, and Classifying to Identify and Isolate the Infected from The Rest Can Make Lockdowns Unnecessary Provide Additional Information To Help Isolate the Infected from Normal It is hoped that they may provide previously unidentified clues to asymptomatic patients Global Continuous Monitoring of Vital Signs Will Help Public Health Officials to Identify and Treat All Illnesses, Before Things Get Out of Control • •

Impact of Regular Monitoring on a Phone. Watch • Today, Travelers in India Can Only Enter the Airport, if They Have Downloaded the “Arogya Setu App” for Contact Sensing. With Thermal Screening and Ability to Use Proximity Sensing To Do Contact Tracing, A Traveler Can Proceed Thru a Green Channel. • Others, who may be At-Risk Must Go Red Desk for Further Testing • Travelers Must Own a Smart Phone and Download the Arogya Setu App • Any Traveler who has Authorized the Government to Use Their Time and Location Information May Also Be Willing to Authorize the Use Data From Biomedical Sensors • With Automatic Data Collection Using a Phone. Watch on a Daily Basis, any parameters that are outside the limits can trigger visit to clinic possibly eliminating the need for quarantine

Airport Scenario 2020 Current Airport Scenario • Arrive 2 hour before ETD, Wearing Face Masks and Gloves • On Exit Go to Thermal Screening Station • If Arogya Setu Not Installed on Phone, go to Counter for Install • No Airport Entry if Red Status on Arogya Setu or Covid Symptoms • Airport Entry after Verification of Ticket, Heath and Self Declaration, Baggage Receipt as SMS. • SMS Baggage Receipt • Minimum touch at security screening • Minimum social distancing and face masks within airport • Collect a safety kit - consisting of three-layered surgical masks, face shields and sanitizers • No touch entry verification • Possible Scenario with Phone. Watch Bluetooth Link • At Security Entry Point, Go to Green Channel after Bluetooth Handshake and exchange of QR code • with Boarding Pass, Self-Declaration Form, Covid-free certificate and Data exchange. • • All Others Go to Red Channel And Biomedical Checks People of High-risk but without symptoms may enter with Face Masks and Gloves • Baggage receipt SMS added to QR code • At gate, before aircraft entry, one more Bluetooth handshake of entry documents

Sensors on Smart Phones • • • • Proximity sensor Ambient light sensor Ambience white balance sensor Camera with OIS Accelerometer Gyroscope Compass Barometer NFC for Apple Pay Touch ID fingerprint scanner GPS/Glonass/Bei. Dou/Galileo/Japanese gnss Pressure touch sensitive display Face ID scanner Multiple microphones

How Do Sensors Work • Pulse can be monitored by the photodiodes; • Heart rate sensor uses infrared (IR) and visible-light LEDs and photodiodes to detect your heart rate - measured in beats per minute (BPM). Apple Watch is able check the pulse often and continuously (and the IR sensor every 10 minutes) • Blood oxygenation by placing a finger on Camera • Body temperature is by placing the watch-sensor on forehead • Sleep monitoring uses the accelerometer and gyroscope, so long as the battery is charged before going to sleep • Blood sugar levels is measurable using near-infrared spectroscopy • Step detection is measured by the accelerometer