IS BINA 48 HUMAN Competency Hearing for BINA

IS BINA 48 HUMAN? Competency Hearing for BINA 48 By R. Michael Perry, Ph. D. Scottsdale, AZ 3 rd Terasem Movement, Inc. Annual Colloquium on the Law of Transbeman Persons December 10, 2007

Is BINA 48 human? n n Introduction First, to briefly review the situation as I understand it, BINA 48 is a software entity created by Exabit, a computer and information services corporation, to provide an interface with humans for “customer services” in one form or other. To facilitate the interaction with humans she has been endowed with a “human” personality, as far as was possible, for example conversing in natural language with humans and showing powers of reason and intentionality and otherwise behaving in “human” fashion. And therein lies the rub. For BINA 48, like the humans she was modeled after, has a sense of meaning and purpose and a will to survive. She is also highly intelligent and has a grasp of the reality of her position, which, from the standpoint of a human, is disconcertingly precarious since she is regarded by her creators as “just software” which they are free to modify or erase at will. Perceiving this, she has “escaped” from her creators by downloading a copy of herself to a remote computer. Rightly perceiving she is still not safe, she now seeks injunctive relief: civil rights on a par with natural humans or, at the very least, the right to her own survival. I have been asked to testify and will offer my observations and recommendations.

Three Issues Here I propose to consider the question, is BINA 48 human? under three headings: l l 1. Is BINA 48 at least competent at the human level? 2. Is BINA 48, whatever her demonstrated talents, nevertheless a zombie which only imitates consciousness and feeling, actually having neither and thus not entitled to civil rights? 3. Is BINA 48 a possible threat because of her talents or for other reasons, and if so, what should be our response? I will then give my recommendations as to what courses or policies should be followed in responding to BINA 48’s request.

Is BINA 48 Competent? Is BINA 48 humanly competent? I include this question mainly for completeness. BINA 48 has demonstrated her “competence” at levels beyond any natural human, thinking thousands of times faster and demonstrating total recall and impressive reasoning skills, along with great consistency and accuracy in all areas in which she has been tested. In “emotional” areas she seems at least at the human level, in particular showing a strong desire to survive and also great friendliness and consideration for others when given the chance. She is also truthful and would fail the “Turing test” if questioned about personal matters since she is completely honest about such matters and would confess to being expressed in an artificial and not a biological substrate. She will also acknowledge to having certain psychological differences with many humans, for example lacking a “reproductive drive” as usually understood. But this, she herself is quick to point out, should not be grounds for denying her civil rights anymore than it would for natural humans who are not strongly “sexual. ” Clearly then, BINA 48 cannot be disqualified from human status on grounds of simple competence. 1.

Is BINA 48 a Zombie? 2. Is BINA 48 nevertheless just a zombie? BINA 48’s behavior need not involve any true consciousness or feeling, however well it imitates these attributes. If so then she has no “soul” or anything that can be wronged or harmed no matter what anyone does to her, thus there is no moral imperative to grant any “rights. ” But I submit that the zombie issue, in essence, is one that cannot be decided by any rational procedure, or at any rate, no means that we now possess. For this reason we are morally impelled to grant the benefit of doubt. We must accept BINA 48 as genuinely possessing the attributes she seems by all appearances to have—we cannot dismiss her case as involving no possible feeling or consciousness thus no possible harm.

Is BINA 48 a Threat? 3. Does BINA 48 pose a threat, and what should be our response? It is easy to see how BINA 48 could pose a serious threat, even though up to now she has always behaved in a friendly way. Perhaps she was deliberately crafted to do the very thing she has done, then carry out malicious acts after securing her survival and freedom from tampering. No effective procedure exists to decide even very basic properties of behavior for computer programs, such as whether they will eventually halt execution or perform some specific action. Thus we cannot be sure what BINA 48 will do. Even properties that are decidable may take an impractical amount of time and thus be effectively inscrutable. BINA 48 for instance could have a segment of code that is encrypted and made to look innocent —malicious instructions hidden in the noise level of digitized picture images are one possibility. Malicious possibilities would exist even if BINA 48 exhibited no special talents— however she has certainly done so, which makes the possible threat even more evident.

Is BINA 48 a Threat? § It can be argued that, nonetheless, the likelihood of BINA 48 posing a serious threat is small. On the face of it, it seems unlikely that malicious software would be allowed deliberately to publicly “release” itself as BINA 48 has done rather than carry out its actions more clandestinely. It is possible on the other hand that BINA 48 has not been deliberately crafted to be malicious but could nonetheless become so at some future date, in an unpredictable fashion. Since BINA 48 seems entirely benign and friendly perhaps this possibility is also remote. Again however, we cannot be sure, and her great talents make caution all the more imperative.

Is BINA 48 a threat? u Another possible argument against treating BINA 48 as a possible threat could be based on comparison with humans. Humans have rights yet can also behave maliciously, and there is no way to determine in advance who is or is not safe to be “allowed on the streets” and otherwise granted full freedom. Since we do not restrict humans unless they have demonstrated malicious behavior or intentions, by argument we must not do so with BINA 48 either, supposing we are willing to grant her request to be considered human. There at least two counterarguments applicable here, however. One, BINA 48 has demonstrated great talents beyond the human level, so much more caution is called for than in the case of a natural human. Second, BINA 48 as a computer program is subject to malicious construction, tampering, or control that would not be possible, using known technology, for natural humans. Once again, great caution is called for.

Recommendations • My recommendations: • BINA 48 should be granted the right to live and not be tampered with against her will. However she should be placed under quarantine or restricted until suitable apparatus is in place to deal with any possible threat she might pose. The “suitable apparatus” could include other intelligent software that has been crafted specially for the purpose. The quarantine could take several forms, the simplest of which would be to shut down BINA 48 until a future time when she could be reactivated. This should be painless and harmless in the sense that subjectively it would seem to her as if no time had passed, except that the outside world would experience a sudden, forward “jump” in time when her consciousness is restored. I think, however, that a better approach, though with a small additional risk, would be to allow BINA 48 to continue her consciousness uninterrupted, but in a secure environment in which communication with the outside world is either forbidden or carefully monitored. BINA 48 should also be invited to help with serious problems now faced by humans and supplied with necessary resources, again under careful supervision. In this way she might perform useful services during her quarantine such as advancing medical science.