The Future of Artificial Intelligence Stuart Russell University

The Future of (Artificial) Intelligence Stuart Russell University of California, Berkeley Université Pierre et Marie Curie

Why are we doing AI? v To create intelligent systems

Why are we doing AI? v To create intelligent systems v The more intelligent, the better

Why are we doing AI? v To create intelligent systems v v The more intelligent, the better We believe we can succeed v Limited only by ingenuity and physics

An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made if [we] work on it together for a summer. John Mc. Carthy and Claude Shannon Dartmouth Workshop Proposal, 1956

Why are we doing AI? v To create intelligent systems v v The more intelligent, the better To gain a better understanding of human intelligence

Why are we doing AI? v To create intelligent systems v The more intelligent, the better To gain a better understanding of human intelligence v To magnify those benefits that flow from it v

Anil Ananthaswamy, “I, Algorithm: A new dawn for AI, ” New Scientist, Jan 29, 2011

An industry arms race v Once performance exceeds a minimum level, small improvements are worth billions Speech v Text understanding v Object recognition v Automated vehicles v Domestic robots v

Military arms race

What if we do succeed? “The first ultraintelligent machine is the last invention that man need ever make. ” I. J. Good, 1965 v Might help us avoid war and ecological catastrophes, achieve immortality and expand throughout the universe v Success would be the biggest event in human history v

What if we do succeed? “The first ultraintelligent machine is the last invention that man need ever make. ” I. J. Good, 1965 v Might help us avoid war and ecological catastrophes, achieve immortality and expand throughout the universe v Success would be the biggest event in human history … and perhaps the last v

So, if that matters…. . Along what paths will AI evolve? v What is the (plausibly reachable) best case? Worst case? v Can we affect the future of AI? v Technical or societal solutions? v What should we do now? v

This needs serious thought If a superior alien civilization sent us email saying, “We’ll arrive in 30 -50 years”, would we just reply, “OK, call us when you get here, we’ll leave the light on”? v The AI community needs a substantial institutional commitment, reasonably soon v

Precedent: Nuclear Physics Rutherford (1933): anyone who looks for a source of power in the transformation of the atoms is talking moonshine. v Sept 12, 1933: The stoplight changed to green. Szilárd stepped off the curb. As he crossed the street time cracked open before him and he saw a way to the future, death into the world and all our woes, the shape of things to come. v Szilard (1934): patent on nuclear chain reaction; kept secret v

Precedent: Nuclear Physics Hahn et al (1939): uranium fission v Szilard and Fermi (1939): uranium chain reaction. “That night, there was very little doubt in my mind that the world was headed for grief. ” v Einstein/Szilard (1939): letter to Pres. Roosevelt urging development of nuclear weapons before Nazis v Szilard et 70 al (1945): petition to end war by inviting Japanese to A-bomb test v Emergency Committee of Atomic Scientists v

Precedent: Chemical and Biological Weapons 1925 Geneva Protocol banned use in warfare, but R&D, stockpiling continued v Long negotiations (until 1972 for biological, 1992 for chemical weapons) v 1975 (biological) and 1997 (chemical) treaties ban “development, production, acquisition, stockpiling, retention, transfer or use” v

Precedent: Genetic Engineering 1973 -4: Paul Berg stopped his own experiment to insert carcinogenic virus DNA into E. coli; prominent scientists request a moratorium on recombinant DNA experiments v Asilomar conference (1975) set up guidelines: v v Physical, biological containment; risk analysis Ban on disease/toxin organism manipulation Credited with avoiding restrictive legislation v Industry compliance via FDA controls on sales v

Precedent: Genetic Engineering NIH Recombinant DNA Advisory Committee will not approve any protocol modifying human germline v Cartagena Protocol (2003) governs trade in GMOs, enshrines precautionary principle v 2010: Pres. Commission proposes federal oversight of synthetic biology activities v 2012: 100+ NGOs call for worldwide moratorium v

The process is beginning MIRI, FHI, CSER, FLI, etc. v AAAI task force (Horvitz & Selman) v US Air Force: Test, Evaluation, Verification, and Validation for Autonomy v

The process is beginning MIRI, FHI, CSER, FLI, etc. v AAAI task force (Horvitz & Selman) v US Air Force: Test, Evaluation, Verification, and Validation for Autonomy v Today’s meeting v

Meeting Schedule 1. 00 -1. 30 -1. 50 -2. 10 -2. 30 -2. 50 -3. 10 -3. 30 -5. 00 -6. 30 -7. 00 -8. 30 -10. 30 Introductions (Russell) Robots (Veloso) Intelligence explosion (Dewey) Unintended consequences (Shanahan) Autonomous trading (Wellman) Ontology, organizations (Mallah) break Technical research agenda Organizational/socioeconomic responses walk to reception AAMAS reception, Sorbonne “Cordeliers” Dinner Le Procope, 13 rue de l'Ancienne Comédie

Technical research agenda v v Verification/validation Designing reward/utility functions v v Iterated design/simulation/verification Inductive specification from examples Ensuring compliance Theory of agents v v Subsumption, composition, distribution, cooperation, transparency, etc. Theory of non-agents v v v Pure question-answering systems Meta-non-agents Bootstrapping methods v v v Superintelligent self-verification Superintelligent boxing If you were me what utility function would you give yourself?

Organizational/socioecono mic responses v One international, professional society Keep tabs on worldwide effort levels etc. v Promulgate culture of responsibility, safety v Develop standards for risk analysis/verification v Keep decision makers and society well informed v Lobby for research funding v Legal standards for liability etc. v Conferences, conference tracks, journals v