AI Safety CMPT 310 Summer 2019 AGIs Artificial

AI Safety CMPT 310 Summer 2019

AGIs Artificial General Intelligence

Superintelligence [a superintelligence is] any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest -- Nick Bostrom

Superintelligence one example of superintelligence would be a human mind running on upgraded hardware, e. g. say 3 million times faster than the average human brain

Superintelligence whether or not superintelligences are conscious, or truly intelligent, doesn’t matter --- what we care about is what they end up doing

The Last Invention … the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. -- I. J. Good

Time to Worry? Should we be worried about creating AGIs, or superintelligences, that might not work they way we want?

Example: Minimizing Human Suffering • Suppose we create an AGI with the goal to minimize human suffering • Suppose it concludes that, even in paradise, human nature is such that humans will always find some way to make themselves suffer • Therefore, the agent concludes that the best way to minimize human suffering is to … kill all humans

Example: Paper Clip Maximizer Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips. The future that the AI would be trying to gear towards would be one in which there were a lot of paper clips but no humans. -- Nick Bostrom

Time to Worry? … AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else. -- Eliezer Yudkowsky

Asimov’s 3 Laws of Robotics First Law A robot may not injure a human being or, through inaction, allow a human being to come to harm. Second Law A robot must obey the orders given it by human beings except where such orders would conflict with the First Law. Third Law A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

A Common Theme AGIs might kill us!

AI Safety Some people suggest that we need to be thinking about safety issues when designing AI systems. Other people say it’s like worrying about over -population on Mars --- we are so far away from AGI that there’s no need to worry about it yet.

Corrigibility one suggestion is that we ought to design systems that are corrigible, i. e. systems that know they are not always right and so shouldn’t just blindly do what they think is best; they must be open to correction

Value Alignment another suggestion is that we somehow must ensure that AGIs share our most important human values

Stop Buttons In industry, dangerous machines have a stop button that a person can press to immediately turn off the machine. Perhaps we should be adding stop buttons to our AI systems? Check out this video on the stop button problem …