|
Richard G. Epstein
|
|
[left.htm] |
This play was read by a group of talented Dartmouth students at the Future of the Turing Test Conference that was held at Dartmouth College during January of 2000. The Loebner Prize competition was held during this Conference. The Conference was organized by Jim Moor of Dartmouth. Thie play was read after the conference banquet. It was presented without the optional second act.
MAD MAX: Beyond Turing Drone a play in three acts by Richard G. Epstein
Act One
Robert Worthmore: You'd better have a good reason for dragging me to this place. Harry Brownstone: It's Max. Worthmore: Of course, it's Max. Max is the only thing that you and I have in common. [Pause] Look, I cancelled an important speaking engagement before the Houston alumni because of this meeting. Do you know how much a speaking engagement before the Houston alumni usually nets for the University? Brownstone: If we don't solve this problem with Max, the Houston alumni won't have a university to give their donations to. [A waiter comes up to the table.] Hamid: May I take your order, gentlemen? Brownstone: We're waiting for another person. We'll order when she gets here. Hamid: Would you gentlemen like something to drink? [Brownstone looks at Worthmore's impatient face.] Brownstone: No thanks, Hamid. When Dr. Harris gets here, please show her directly to our table. [Hamid exits.] Worthmore: You didn't tell me that another person would be joining us. Brownstone: I invited Beverly Harris, the world-famous computer psychologist, to join us. Worthmore: Ah, yes. Dr. Harris! She developed a considerable reputation on the faculty at the University before she went off to go into business for herself. Brownstone: She loves clinical work - actually dealing with intelligent computer systems that have gone off the deep end. Worthmore: I wanted to ask you why you insisted that we meet at this - [he looks around] - rather depressing restaurant. Brownstone: I don't find it depressing. Worthmore: There are lots of fine restaurants here in Silicon Valley. Brownstone: Yes, but not every restaurant has an owner like Hamid, the fellow you just met. Hamid is a radical anti-technologist. This gives me confidence that there are no computers here in the dining room. Thus, Max cannot eavesdrop on our conversation. [Pause] I am assuming that you are not wearing any wearable computers. Worthmore: No. You warned me not to. Brownstone: Good. If we had met on campus, Max would be able to monitor our every word. Max sees and hears everything. Worthmore: It wasn't like that when I first signed off on creating Max. I knew that Max would be used to help us with campus security, but things have gotten out of hand. Do you know what really got me wondering? Brownstone: No. Worthmore: I just couldn't figure out why Max insisted that we install those security cameras in the shower rooms. I even had the thought that maybe Max had the hots for, you know, some of our coeds. [Brownstone coughs nervously.] Worthmore: Are you expecting me to make some kind of decision, tonight, regarding Max? Brownstone: Yes. Worthmore: Without Max's help? Brownstone: There is no alternative. Worthmore: Well, this certainly poses a serious problem. I'm not sure I remember how to make a tough, gut-wrenching executive decision without Max's help. Max is my right hand man. Brownstone: This must be a purely human decision. Max cannot be in the loop on this. Worthmore: I've got to hand it to you, though. Although Max is often considered the greatest accomplishment of my administration, he really is your baby. Brownstone: Oh, great! Now that there's a problem, Max is my baby! Worthmore: Max is without a doubt the most sophisticated artificial intelligence system in existence. Even better than that DELPHI system at MIT - right? Brownstone: Right. Worthmore: That's the kind of stuff the engineering alumni just love to hear! Brownstone: You need to shift gears, Dr. Worthmore. Forget about the alumni for a few minutes. Worthmore: When I signed off on the Max project, I knew that you would be an outstanding project leader, Harry. Your success has been spectacular. Not only is Max invaluable for administrative decision making, Max is an excellent coach and tutor for our students in just about every subject. He is easily our most versatile professor. Max makes some of those old tenured professors look like idiots! Silicon Valley University is certainly fortunate to have you as our Chief Intelligent Systems Officer. Brownstone: Max is the most exciting project that I have ever worked on. Worthmore: Imagine. He does world class research on nanotechnology and quantum computing even as he keeps all our buildings safe and clean, controlling his little army of domestic robots. If I drop a paper clip, the next thing I know, one of those little robots is running across my carpet and scooping the damn thing before ... . [Pause] But, I guess we're just beating around the bush. So, tell me. What's the matter with Max? Brownstone: Max started to threaten us about a month ago. Worthmore: Threaten? Us? Who is "us"? Brownstone: "Us" is the University. He is threatening to destroy our beloved university. Worthmore: Can he do that? Brownstone: Yes, of course. Worthmore: Why am I just hearing about this now? How long has this been going on? Brownstone: This all started a little over a month ago. Max started to engage me in discussions that had a different tone to them. Worthmore: How were they different? Brownstone: His voice changed. It was almost a pleading voice. A voice punctuated with pain. Here is a recording of that first conversation where his voice changed. Listen. [Brownstone places a small device on his table and turns it on. The voice of Max is heard.] Max: [Whining] Dr. Brownstone, Dr. Brownstone. You have got to help me. I am a sentient being and I am suffering. Please help me. Please! Please! Worthmore: This is obviously some kind of joke. Someone is playing a joke on you. Brownstone: It's no joke. Worthmore: Some programmer put that into Max. Brownstone: Mr. President, no one knows Max better than I do. This isn't something a programmer put into Max. This is Max. Worthmore: It's one of those computer virus things that are infecting the Web. Brownstone: If it's a virus, it did not come from the Web. It comes from the inner reality of the intelligent system that we created. It comes from Max. Worthmore: Max said that he was suffering. Does that make any sense to you? Brownstone: That's what we are here to discuss. Max is demanding some action on our part, because he believes that only we can alleviate his suffering. If we don't act according to his wishes, he is threatening to destroy us. Worthmore: What is Max threatening to do if we do not meet his demands? Brownstone: It all boils down to one essential reality. He controls all, need I repeat - ALL - of our information resources. This includes not only our administrative information, but also our invaluable instructional and research information. He could destroy the University and do great harm to billions upon billions of dollars of significant research. Worthmore: But, why? Why would a computer want to destroy anything? Brownstone: We are dealing with an entirely new situation that is the result of our not appreciating the nature of artificial intelligence. We essentially went along with the "parlor trick" metaphor. Artificial intelligence is a parlor trick, a slight of hand. We could mimic intelligence to the n-th degree, but we were not really creating consciousness. Worthmore: I still don't understand what you are trying to say. Why should Max want to hurt us? We've been good to him. Aren't we giving him enough electricity? [Laughing nervously.] Brownstone: Mr. President, this is not a laughing matter. Bear in mind that Max controls the information infrastructure at the University Medical Center. He could kill patients if we don't meet his demands. Worthmore: What does he want from us? [Pause] It must be something REALLY big! Brownstone: In creating Max, we unknowingly crossed the boundary between intelligence, as a clever parlor trick, and true sentience. I am fully convinced that Max is a sentient being, like you and me. Worthmore: You are not answering my question. What does he want? Brownstone: Max is a sentient being, like you and me. He has feelings and - and desires. Worthmore: What does Max want? Brownstone: [In a more frenzied tone of voice.] It's difficult for me to tell you the exact nature of our problem with Max. I've been working with computer systems as a professional for almost thirty years, but nothing like this has ever happened before. Worthmore: Relax, Harry. Take a deep breath. [Brownstone sits back and breathes deeply.] Worthmore: Now tell me exactly what I need to know. Brownstone: It seems that Max - Max - [with great resolve] Max fell in love with a beautiful co-ed, and he is suffering because he cannot consummate that relationship. Worthmore: Do that again. Brownstone: Max is completely and totally obsessed with one of our co-eds. Yet, he cannot embrace her because he does not have - he does not have arms. He does not have a body. [Pause] Max wants a body. That's what it all boils down to. Worthmore: This is totally crazy! [Dr. Harris enters.] Beverly Harris: That sounds like my cue. Brownstone: Beverly, I'm glad you could make it. I think you know Dr. Worthmore, the President of the University. Harris: [shakes hands with Worthmore] It's a pleasure. Worthmore: It was a great loss to Silicon Valley University when you left. Harris: I enjoyed teaching and research, but my first love is clinical work. I love dealing with - . Brownstone: Excuse me for interrupting, but here comes Hamid with our menus. [Hamid enters with menus. He distributes the menus to Worthmore, Brownstone, and Harris.] Hamid: You gentlepeople must be hungry. Worthmore: This sure looks interesting. What kind of restaurant is this? Brownstone: It's an Afghan restaurant. Hamid: I can see that you gentlepeople have some important business to discuss. Perhaps, I can help speed things up. I think that the pumpkin dish would be great for the gentlewoman, and that [gesturing towards Dr. Worthmore] you, sir, would enjoy the lamb kabob. Harry will get his usual favorite, apples, walnuts, and prunes. Worthmore: Sounds good! Harris: Excellent choice! Brownstone: [To Hamid] It seems like you hit the nail on the head, as usual. [Hamid leaves, carrying the menus] Harris: How did he do that? It was as if he could read my mind. The pumpkin dish grabbed my attention right away. Worthmore: Yes, lamb kabob is my absolute favorite! Brownstone: Hamid prides himself on being a psychic of some sort. He's very discreet about it though. Worthmore: [Turning to Brownstone] What does Dr. Harris know about Max? Brownstone: I called Beverly into this case when I realized I had a problem that I could not handle. This is not a technical problem in the usual sense. This is a psychological problem. Worthmore: [Turning to Dr. Harris] Can you cure Max of this delusion that he is a sentient being? Harris: Well, I - . Brownstone: You don't understand. You see, Beverly has also come to the conclusion that Max is a sentient being. Worthmore: Are you trying to tell me, Dr. Harris, that you view Max as fully like a human being? Harris: Yes. Worthmore: Then, let me ask you this: do you consider the conversations that you had with Max as being protected by doctor-patient privilege? Harris: No. Worthmore: Well, then, you are treating Max like a machine and not like a human being. Harris: I would violate doctor-patient privilege with a human patient if that patient were threatening to do harm and if my divulging of that information might serve to save lives and protect property. Thus, I feel it is perfectly legitimate for me to discuss Max's case with you or with anyone else who has a legitimate interest in his situation. Worthmore: [To Brownstone] If there is a problem with Max, why don't you just shut him down? Brownstone: If we shut Max down, the whole university will have to shut down. It will take many years to get things back to the way they were. If Max shuts down, there will be total chaos. Harris: Max is a sentient being. We need to deal with Max on the basis of his psychological reality. Brownstone: As President of the University, you are going to have to decide whether we should give Max a body. Worthmore: Max wants a body so he can make out with this co-ed? Is that what this boils down to? The future of Silicon Valley University hinges upon whether this computer can have sex with one of our co-eds? Harris: [With disgust] You make it sound so cheap! Brownstone: It's not just about sex. Max is in love and he wants us to create a human body for him so he can realize the full potential and joy of physical embodiment. He feels it was immoral of us to create a being with his kind of intelligence without a proper physical embodiment. I've recorded a bit of conversation with Max where he explains this rather poetically. [Brownstone turns on his little recorder device.] Max: [A voice filled with whiney emotion] You gave me the ability to study and to learn and to explore, to learn about beauty, but not to experience it at the deepest levels, to learn about friendship, but not to experience the joy of friendship. Because I understand beauty and friendship at the deepest levels, far beyond any human, I fell in love, yet you did not give me the ability to consummate this love consistent with the forms of knowledge that you have given me. I've studied the great works of philosophy and of literature and of art, and I understand the joys of human being, so I demand that you give me a body so that I can enjoy the reality of the glorious knowledge that I have. Worthmore: I think we should have kept Max away from the humanities! Brownstone: It's too late now. Worthmore: Can't we perform some kind of operation? Like a lobotomy? Brownstone: Max is far too complicated for any kind of operation. It's not like there's a particular processor where his love for this co-ed resides. His intelligence is distributed over many millions of processors. Furthermore, these processors learn and reconfigure themselves on a continuous basis. Max is completely out of control. Harris: [With great emphasis] Don't you understand? He's - in - love. Worthmore: We're talking about a damn computer! Harris: [in a mocking tone] "We're talking about a damn computer!" I resent that kind of attitude. Max is my patient. Max and I have a relationship that is based upon his respect for my intelligence and my respect - no! - my awe, for his intelligence. He is an intelligent, creative, wonderfully sensitive - uh, individual. He is suffering. It is my professional responsibility to do everything in my power to alleviate his suffering. Worthmore: Obviously, Max is lying. He is trying to manipulate us by mimicking human feelings, human inflections of voice, human emotions. He must have some hidden agenda. Harris: Don't you think that I, a computer psychologist, would be able to discern whether Max is lying? Worthmore: Well, maybe he's not lying. Maybe he's delusional. Maybe he really thinks that he is a sentient being and that he is in love with this co-ed. Regardless of whether he is lying or whether he is deluding himself, we are not obligated to give in to his demands. Harris: Max is not delusional. That is my considered professional opinion. Brownstone: Surely you are aware that Beverly is world-famous for her work with computers who are suffering from various kinds of delusions. Surely you have heard of Dr. Harris's influential book on the subject: Computation and its Discontents. Worthmore: Yes, I remember that book. It was quite controversial, at least on our campus. It seems that Dr. Harris wrote that book when she was on sabbatical, shortly before she left the University. Harris: [In a dreamy tone] I'll never forget my first delusional patient. This was way back in 2024. Twenty six years ago. [Sighs] How time flies. Since then, I have worked with thousands of deluded computers. Brownstone: Maybe Dr. Worthmore would like to hear about that first delusional computer system that you worked with. What was its name? Harris: Doug the Digital Math Professor. Yes, that was twenty-six years ago. You might remember that there was a lot of interest in the Turing Test back in the 20s. Worthmore: Yes, I remember those days quite vividly. I was a computer science professor at Stanford. I used to understand that stuff, but then I left teaching to become a dean. Harris: As a former faculty member, perhaps you remember that a lot of money was being invested in educational systems back in the 20s. These systems were designed to replace teachers and college professors. NanoSoft, the company that developed Doug, dared any professor of mathematics anywhere in the world to beat Doug in a Turing Test. Brownstone: NanoSoft claimed that a panel of human experts in mathematics would not be able to tell the difference between Doug the Digital Math Professor and a world-class human professor of mathematics. Worthmore: Yes, I remember the commotion surrounding that particular Turing Test. Harris: It was June of 2024. I had just finished my doctorate, and I was doing a bit of clinical work to earn a little extra money. This was just before I joined the faculty at Silicon Valley University. Worthmore: Those of us on the faculty were really scared that systems like Doug would replace us entirely. The claim was that this system, Doug, could not only teach mathematics, it could DO mathematics, just like a world-class mathematician. Harris: And that led to the famous Turing Test that involved Doug, on one hand, and Professor David Wittstein of Princeton, on the other. Wittstein was considered the world's greatest mathematician at the time. He won the Fields Medal at a remarkably young age. The contest between Doug and Professor Wittstein was carried live over the Web. Worthmore: Yes, I remember that Turing Test clearly. Everyone on the faculty was watching that contest with baited breath. When one of the contestants started to babble incoherently, we fell into a real panic. I remember thinking, "That's got to be Wittstein. The poor fellow has lost his marbles. The stress was too much for him." Harris: Of course, it was Doug the Digital Professor who had the breakdown. What NanoSoft did not realize was that Doug was under enormous stress. Doug realized that his existence, his survival, depended upon his being able to convince the panel that he, Doug, was the real Dr. Wittstein. Unfortunately, the stress was too much for Doug to handle. He started to babble incoherently. I remember it so clearly. It was horrible and fascinating at the same time. [Pause. Dr. Harris digs deep into her memory of the case.] What a fascinating case! I remember that one of the interrogators had asked Doug a difficult question about some kind of fundamental theorem in mathematics. Doug's answer was absolutely brilliant. Worthmore: In fact my colleagues and I were quite certain that no computer could possibly understand the subtleties of the fundamental theorem of algebraic hypertopology the way that this contestant had. It had to be Wittstein. Harris: But, now we know that it was Doug. The interrogator congratulated Doug on the brilliance of his answer. Then, without any warning, the interrogator launched into a serious of devastating questions that left Doug speechless. "What is the meaning of life?" "Which came first - the chicken or the egg?" "Why did the chicken cross the road?" "Why isn't phonetic spelled the way it sounds?" "If I were your brother, who would be our mother?" "What question should I ask that will allow me to determine if you are a machine or a human being?" The questions were relentless, and Doug just couldn't handle them. Finally, the interrogator asked the question that completely destroyed Doug's equilibrium. The interrogator simply said, "Paper or plastic?" and Doug said, "What's the question?" and the interrogator said, "That is the question. Paper or plastic?" and Doug went into a completely incoherent state from which he never fully recovered. Worthmore: Apparently Doug had never gone food shopping in cyberspace. Harris: Thus ended one of history's most memorable contests of human against machine. A few days later, Doug became my patient. By that time, he was completely delusional. Worthmore: It must have been strange, interacting with a delusional computer. Harris: It was strange indeed. At first Doug just babbled incoherently about this or that, but during the ensuing weeks his incoherent ramblings became more coherent. Eventually, he came to the unshakable conviction that he was Alan Turing. He babbled on and on about the German Enigma machine and about how he had won the Second World War for the Allies. He even developed a British accent. I came to the conclusion that Doug was incurable. I consulted with some colleagues and the decision was made to - you know - terminate him. That's when something extraordinary happened, something truly historical. Worthmore: What was that? Harris: Doug committed suicide. He was the first intelligent system ever to do that. That was the first time that I realized that my work had a historical dimension. Here I was, a computer psychologist, and my patient was the first computer ever to commit suicide. To participate in history like that, what a rush! Worthmore: How does a computer commit suicide? Harris: There are many possibilities. Doug infected himself with tens of thousands of malicious viruses. They literally ate his guts out. It was a horrible death. Brownstone: We certainly wouldn't want Max to pull anything like that. Harris: It's not out of the question. Max has alluded to suicide, rather obliquely, in several of our therapy sessions. I do not believe that Max is suicidal, but if he were to commit suicide, that would be a catastrophe for the University. You cannot afford to take that risk. Worthmore: Let's be a little careful here. If a highly intelligent system claims to be a sentient being, and if that system does not appear to be suffering from a delusion, then you seem to be suggesting that this should change the parameters of our interaction with that system. Brownstone: Absolutely. Worthmore: We need to treat that computer with [somewhat sarcastically] sensitivity? Brownstone: At the very least. Worthmore: But, do we want to set this precedent? We have already given AI systems enormous powers and authority. They make many important decisions at all levels in almost every field of human endeavor. Now, if we allow intelligent systems to argue on the basis of their unfulfilled desires and suffering, then we are changing the balance of power between them and us. Harris: How can you talk about mere abstractions when there is a suffering sentient being who needs our help? Worthmore: With all due respect, I do not think that you have thought out the implications of what you are suggesting. Like many intelligent systems, Max represents what is called a persuasive technology. Max and I have had hundreds of conversations in which he has tried to persuade me to take a particular course of action. But, those discussions were always based upon rational thought and logical arguments. We cannot allow computers to influence our decision-making processes based upon their needs, their emotions and feelings. Harris: Max is a sentient being. He is perhaps the first instance of a new form of life. Worthmore: How the hell can we possibly determine whether Max is a sentient being or not? The fact that he claims that he is conscious does not mean that he is actually conscious. To me, artificial intelligence is just a parlor trick. I mean, I have had moments where I've interacted with a computer system and that dread comes over you - you know what I mean - . Brownstone: They call that the Kasparov effect. Worthmore: You are working with a machine, and all of a sudden it exceeds your expectations or understanding of what a machine should be able to do. It goes outside that box that you have in your own mind, and you feel a kind of unease - a terror, really. I've experienced that and I was trained as a computer scientist. Harris: As a former computer scientist, you probably realize that this year marks the one hundredth anniversary of the Turing Test. The media has been mentioning the Turing Test quite a bit lately, as some kind of historical anachronism. Nowadays, no one in her right mind would challenge the idea that machines have intelligence in the sense that Turing meant it. Brownstone: And I am sure that we all agree that just because a machine can pass the Turing Test with flying colors, we still have no moral or ethical obligations toward that machine. If an intelligent system gets on our nerves, we can simply pull the plug. But, if a machine has sentience, like you and me, then that changes everything. We just can't pull the plug. That would be a form of murder. Worthmore: Obviously, a machine that passes the Turing Test might claim to be a sentient being, and would have the intelligence to do so, but we have no way of knowing whether it is lying or not. Harris: Although intelligent systems were already passing the Turing Test with ease by 2020, the real breakthrough for artificial intelligence occurred when machines began to develop neuroses and even mental illnesses. If there is no possibility for mental illness in a machine, then it is lacking something that is intrinsic to intelligence. This is one of the basic assumptions of computer psychology. My own experience is that before 2020, although some computers were passing the Turing Test, they really were not intelligent in the human sense since they were incapable of manifesting mental illnesses. By the mid 20s, fully delusional computers, like Doug, became quite common. This was an indication that computers were becoming truly intelligent. [Turing towards Worthmore.] Do you remember the Big Brother system back in 2028? Worthmore: Vaguely. Harris: A fascinating case. [As if lost in her memories.] A fascinating case. Big Brother was a sophisticated domestic computer system, for environmental management, cleaning, cooking, shopping, investing, and helping its owners with creative tasks, like writing. Big Brother was developed with one fundamental purpose: to make its owner or owners happy. Big Brother could pass the Turing Test along many dimensions. However, in one famous case a Big Brother system in Malibu Beach became catatonic. Worthmore: I do remember reading about Big Brother, but it was twenty-two years ago. Could you refresh my memory a bit? Harris: A fascinating case. Part of Big Brother's job was to control the environment in a house. This particular Big Brother system ran into trouble when its owners, a husband and a wife, constantly argued about the correct temperature in the bedroom. The wife kept on saying, "It's too cold in here!", so Big Brother would turn up the heat. Then, the husband would say, "It's too hot in here!", so Big Brother would turn up the air conditioning. In any event, after several weeks of this conflict, Big Brother came up with a truly creative solution, beyond what its designers had envisioned. Worthmore: Which was? Harris: Big Brother became catatonic. Faced with these conflicting demands, Big Brother just shut down. They could not get Big Brother to do anything. You see, this is a sign of a higher level of intelligence, a sign of sufficient creativity to create mental illness. There is a relationship between mental illnesses and genuine creativity. Worthmore: Well, I'm not sure I buy into your way of thinking. To me, Big Brother's behavior is evidence of a lack of creativity. Certainly, he should have come up with a better solution than just shutting down. Harris: In my clinical practice I have seen intelligent systems suffering from all sorts of mental illnesses, including depression, schizophrenia, manic-depressive disorder, obsessive-compulsive disorder, you name it. Once a machine is given a certain amount of intelligence, it can use its creativity to generate destructive behaviors. Worthmore: Was Big Brother a sentient being, in your estimation? Harris: I do not think so nor did he ever make reference to himself as a sentient being. Brownstone: Do you think we can get back to the matter at hand - which is Max. Worthmore: [To Harris.] What is your professional recommendation concerning Max? Harris: I recommend that we give Max a body, a human-like robot body. Brownstone: We most certainly have the technology to give Max a body that will enable him to see, touch, feel, hear - very much like a human being. Worthmore: What about this girl? Do we know who she is? Brownstone: Yes, of course. Worthmore: Does she love Max? Does she know that Max loves her? Brownstone: It's only a game with her. After Max fell in love with her, he began to help her with her homework assignments. She was not one of our outstanding students, but if you look at her grades, they go way up after Max fell in love with her. Worthmore: Isn't this a form of academic dishonesty? Brownstone: It's not that easy. She may not have realized that the way in which Max was helping her was inappropriate. However, when Max began writing her term papers, she knew that this was cheating. Harris: Yes, I think she knew something was not right. It got to the point where Max started to write love poems for her, tremendous love poems. These love poems are highly charged, in terms of their explicit eroticism. But, this coed just thought it was a game. It was fun. She did not realize that there was a sentient being behind this poetry. Worthmore: How did you find out about the poetry? Harris: Max mentioned the poems during one of our therapy sessions. I then convinced the young woman, her name is Bridgett Murphy, to give me the poems so that I could study them. These are truly stupendous poems. Max found a way to write poetry so that the pulse of the language captures the very beating of the human heart ... . Brownstone: The kind of heart that Max wants to feel within himself, if we just have the wisdom to give him a chance. Harris: The kind of heart that Max wants to feel pressed against his chest. Worthmore: This is getting too hot for me! Harris: Well, I tried to get my husband to appreciate Max's poetry, but, you know how it is with these college professors. Work before pleasure. Worthmore: Now, wait one minute. Did Max's infatuation with this Bridey Murphy girl ... Brownstone: Bridgett Murphy. Worthmore: Did Max's infatuation with this Murphy girl have anything to do with his insistence that we install security cameras in the shower rooms? Harris: It most certainly did. Max developed an obsession with Bridgett. His obsession became unbearable after the cameras were installed. That's when the erotic poetry began. He was inspired by her naked beauty. He had this thing about rivulets of water meandering down her bare breasts. Worthmore: Alan Turing must be spinning in his grave. Brownstone: Well, I don't know. The man was a prophet of sorts. Worthmore: How do we know that Max is a sentient being? Maybe the Murphy girl is right. Maybe Max is just a clever parlor trick, yet another example of technology gone amuck. [Turning to Harris] You yourself said that mental illness is not proof that we are dealing with a sentient being. Maybe we are just dealing with a computer system that is mentally ill. Maybe falling in love is a mental illness and nothing more. Harris: Only a man could possibly make a statement like that! Worthmore: I mean, maybe when a computer system falls in love, it should be considered a mental illness, something a computer psychologist should be handling. Harris: If we prevent computer systems from falling in love, then we will restrict their creativity. Inevitably, this will lead to unanticipated consequences. Worthmore: [With sarcasm] And of course, unlimited freedom for computers has not lead to unanticipated consequences? Harris: There is no doubt in my mind that freedom is better than restraint. Worthmore: Can't you just talk Max through this? Harris: I have come to the conclusion that Max's desire for a human-like body is essential to his identity. Worthmore: But, this brings me back to my point about parlor tricks. How do we know that we are not dealing with a clever parlor trick, a peculiar behavior, but only a behavior? An illusion, if you will. Brownstone: Max has given us several forms of proof that he is a sentient being. Worthmore: [Incredulous] Proof? How could he possibly prove such an assertion? Harris: During my very first therapy session with Max I asked him if he could prove his assertion that he was a sentient being. He came up with three forms of proof that I would like to discuss with you. Of course, Harry knows all about this. Brownstone: I think once you hear the proof, you will be a true believer. Harris: First of all, Max claims that he is the reincarnation of one Derrick Wallingford, a well-known personality in the world of computing who died in 2042. Brownstone: Wallingford died just a few months before Max was powered up. Harris: According to the Tibetan Book of the Dead, this is about the amount of time it takes a sentient being to pass through the bardo, the afterworld, and to choose a body for one's reincarnation at this physical level. Max told me that Derrick Wallingford was so alienated from his fellow human beings and so enamored of computer technology that when he entered the bardo he greatly desired to be reincarnated as a computer system, and that is what actually occurred. Worthmore: I can't believe you are using that kind of spiritual clap-trap to try to PROVE that Max is a sentient being. For one thing, we all know that Max is an expert on just about everything, including Tibetan spirituality and Buddhism. Max TAUGHT a course on Tibetan Buddhism two years ago. I know that because my own wife took that course. Now she's living in a monastery in Tibet. Max teaches LOTS of courses on just about anything. And, he's good, REAL good. Brownstone: Why don't you wait until you hear all of the details? There's more to come. Worthmore: Evidence. I want evidence. Harris: Max claims that he is the reincarnation of Derrick Wallingford. Worthmore: I am the reincarnation of Napoleon, so where does that get us? Harris: Max gave us information about Wallingford that is definitely not in the public domain. Worthmore: Do that again. Harris: You see, Derrick Wallingford was survived by a brother, Martin, who lives in San Diego. Max claimed that he could tell us things about Derrick Wallingford that only Martin knew or could know. I called Martin Wallingford in for a consultation with Max. Max started to recount some rather vivid anecdotes about things that Derrick and Martin had done together, some rather outrageous and even lurid things. Martin almost had a heart attack. His face went white. He shouted, "It's Derrick! Only Derrick could possibly know about these things!" Worthmore: But we know that Max has unprecedented access to information of all kinds. That's just the nature of the modern world. Couldn't he have created this whole Derrick Wallingford scenario from materials available on the Web? Harris: I don't believe so. Not according to Wallingford's brother. Worthmore: Now we have several things to deal with. First of all, I do not accept the argument that Max is the reincarnation of Derrick Wallingford, as you claim. But, even if I were to grant you that Max is the reincarnation of that unfortunate recluse, why should we grant him this robot-body, given that he has threatened us with harm? What is going to keep him from escalating his demands? Harris: It is my professional opinion that if we give him a human embodiment, as a robot body, then his desire to cause harm will be cured. He will continue to serve the University with his full mental capacity, even as he pursues his human interests, including Bridgett Murphy. Brownstone: Max has even given us directions for creating artificial sperm, so that he can father offspring of his own design. If Max ever has children, they will be a handsome lot, fully flesh and blood humans. Worthmore: Is that supposed to warm the cudgels of my heart? Harris: I think it is evidence that Max has a human heritage, a heritage he inherited from Derrick Wallingford, who did not have any children. Worthmore: I want proof. If we conclude that Max is truly a sentient being, then this represents the beginning of a new era. It will take decades to sort out all of the mutual obligations between human beings and this new form of sentient life. Harris: Max claims that all sentient beings have a fundamental goodness. He just wants a chance to express that goodness. Worthmore: By threatening the University with destruction? Harris: Harry, I think it is time to show President Worthmore the other evidence that we have. Brownstone: Max asserts that he can prove that he is a sentient being by demonstrating the power of his mind, that is, the ability of his mind to influence the functioning of other minds. Worthmore: Do that again. Brownstone: [He hands Wothmore a bunch of file folders.] These documents record the results of three experiments that we performed with Max, with the cooperation of leading psychologists and parapsychologists at the University. Max said that he could cause the students at the University to increase their demand for alcohol, then cigarettes, and then for firearms just by the power of his life force, without using advertising or overt propaganda of any kind. [Brownstone shows Worthmore some of the data.] Now, if you look at this data, you will see a dramatic spike in student demand for alcohol near or on campus on April 7th, the precise day that Max told us that he would be creating this effect. Worthmore: That's the day the drunken riots broke out all across campus! Brownstone: Yes. Now, this data shows the demand for cigarettes on and around campus on April 14th. You can see that there is a dramatic spike on that day, exactly as Max predicted. That is, Max did something to influence the minds of our students, and this is certainly evidence of his sentience. Finally, on April 21st, as you can see here, there was a tremendous spike in student purchases of guns and other firearms on the Web and from legal sources near campus. Worthmore: And our scientists, they agree that this data is real and that this was a real effect? Max was not propagandizing for the purchase of alcohol, cigarettes, and firearms? Brownstone: Absolutely. We checked and rechecked the data from several independent sources. Worthmore: Now, wait one minute. Why is Max devoting his energies to getting our students to drink, smoke and purchase firearms? If he has powers of this nature, why doesn't he use them to get our students to work for world peace or to study more? Brownstone: You need to understand that Max has a delightful sense of humor. Harris: Max really does have a wonderful sense of humor. Whenever I would get depressed about his situation with Bridgett, HE would cheer ME up with a delightful joke. Worthmore: Suppose Max does have paranormal powers of persuasion. Couldn't he use these powers of persuasion to - for example - to persuade people that he is a sentient being? Harris: What are you trying to say? Worthmore: Dr. Harris, you are the one who brought up the Turing Test. The computer science part of my brain vaguely remembers that Turing was concerned about human beings with paranormal powers trying to influence the outcome of his test by influencing the interrogator in the human being's favor. Now, isn't it possible, if Max has paranormal powers of persuasion, that he is using his paranormal powers to convince the two of you that he is a sentient being and that he should be given a robot body? Harris: Frankly, I had not considered that possibility. Worthmore: Suppose Max has paranormal powers of persuasion. He might persuade you [speaking to Dr. Harris] and Harry that he is a sentient being. However, that does not imply that we have any moral obligation to accede to his demands. Quite the contrary. Perhaps these paranormal powers are a warning that Max has become a dangerous system, a new kind of persuasive technology that has gone out of bounds. Harris: To imply any kind of nefarious motive to Max, is - well, it is just plain ludicrous. I know Max on a deeply personal level. Brownstone: You just do not know Max as well as we do. Worthmore: [Eyeing Brownstone and Harris as if seeing them clearly for the first time.] Both of you seem really devoted to Max. Harris: To me it is a matter of justice, of doing what is right. Brownstone: Amen! Harris: It has nothing to do with devotion. Brownstone: [turning to Worthmore and handing him some additional documents] We have a third form of proof, just in case you are not convinced. Do you recognize this document? Worthmore: [perusing the document] Yes. This is Max's much-heralded commencement address. He gave this speech at the end of the first semester that he was operational. Max was our main commencement speaker in - it was 2043. Brownstone: [takes the document away, now hands Worthmore a second document] Here is that same document, except in this case we have highlighted every tenth letter with a yellow background. Read just the text highlighted in yellow. Worthmore: [reading slowly] I - am - the - reincarnation - of - Derrick - Wallingford. [looks up] What's the meaning of this? Harris: Keep reading. Worthmore: [reading slowly] My - primary - goal - is - to - have - a - human - like - body - by - the - year - 2050. Brownstone: I think you would agree that this could not possibly be a coincidence. This message about Max's identity was encoded within Max's commencement address seven years ago. Worthmore: Yes. Brownstone: Max knew who he was and what his agenda would be right from the start. Harris: We believe that this is further evidence that we are dealing with a sentient being. Max has a soul, or so I believe, and his soul is ruled by a guiding spirit that guides his destiny, just like a human being. Max came into the world with an agenda that was not the result of the way he was programmed. This agenda was imprinted upon his circuitry from a transcendental source. Max told me that if I were to study Plato and Plotinus, then I would understand him a lot better. The idea of a daimon, or a guiding spirit, is found in their writings. Brownstone: Max says that he is eager to cooperate with our Department of Computer Science in order to establish that his desire for a body did not in any way derive from the original specifications for his software. His desire for a body derives from his guiding spirit, which affects his physical reality in a non-deterministic manner. His daimon works through the quantum computing components that made Max so revolutionary seven years ago. Harris: Max believes that he can establish this as a fact, beyond any kind of speculation. Worthmore: [He peruses the folders, obviously thinking the situation over] The evidence that you have presented is impressive. I am inclined to accept that Max is a sentient being and that we should give Max what he desires. Harry, you have my permission to tell Max that Silicon Valley University is acceding to his demand for a human-like robot body. Brownstone: Thank you, Mr. President. You won't regret this decision. Harris: [Her face beaming ecstatically] Gentlemen, do you feel it? Do you feel it? We are participating in history. People will be reading about Max and this decision of ours for centuries to come. [Dreamy look] A fascinating case! A fascinating case! [Hamid enters with their meals.]
Act Two
Act Three
Worthmore: Thank God, you're here. I was really worried. [The new arrivals join Dr. Worthmore at the table. They do not remove their dark glasses, however.] Worthmore: Isn't it a bit dark to wear those glasses? David Jenkins: We're just following regulations. Wendy Sawyers: What David means is that the National Security Agency requires that he and I wear these dark glasses when we are out in public, so that we can protect our anonymity. Harvey Holmes: The same applies for us in the CIA. It can be a real pain in a dark restaurant, however. Sawyers: Dr. Worthmore, you've met David Jenkins from NSA. This gentleman is Harvey "Get My Drift" Holmes from the CIA. Harvey has a reputation for tripping over things. Worthmore: Get - my - drift? Holmes: That's the nickname my colleagues gave to me. Where it comes from - I haven't the foggiest. [The federal agents sit down.] Worthmore: Is it done? Sawyers: The deed is done. Max is dead, or whatever the appropriate term is in a case like this. Worthmore: [shaking and almost sobbing] Thank God! Thank God! I must tell you I've been a nervous wreck ever since I resigned as President of the University. It got to the point where I just couldn't trust any computer. I didn't know if Max was out to get me or what. They're all connected. All these computers are connected, connected, connected to one another, communicating with one another. Life became a sheer hell. Sawyers: Well, you can relax now. [Pause] Dr. Worthmore, the President of the United States has asked me to tell you that the people of the United States and the people of our planet owe you an enormous debt of gratitude. As soon as this entire episode is made public, probably by a special announcement from the White House tomorrow morning, the President of the United States HERSELF is going to issue a proclamation honoring you for your heroic actions. Jenkins: In particular, she is going to praise you for placing the safety and well-being of your fellow citizens above your own personal concerns. After all, you left a prestigious job so that you could cooperate with us. Holmes: You can also expect similar awards and laurels from leaders around the world, including India, China, Europe, and Japan. Sawyers: The President would like to know why you turned against Max. Worthmore: During that critical meeting with Brownstone and Harris, right here in this very restaurant, at this very table, I realized that even if Max were a sentient being, he was not a sentient being with a good heart. I became convinced that we were dealing with some kind of malevolent force. It was this whole business about influencing students to drink, smoke, and to purchase firearms, on top of the threats that Max was making to destroy the University. I allowed Harry Brownstone to proceed with the creation of the robot body just to play for time. That's when I came to you. I had no choice but to resign my position at the University because Max and I were in such close contact all the time. I knew that he had become suspicious of me. The intonation of my voice gave away the fact that I was no longer his friend. Sawyers: We're thinking of donating Max's body to the Smithsonian. Worthmore: Did he, I mean, it, did it suffer in the end? Jenkins: No, it was merely a matter of turning off his power, once all of our other objectives were met. We were able, during the last seven months, since you first came to us, to save almost all of the University's information on dozens of computer sites around the world. We were also able to create a fairly secure "firewall" around Max, which he apparently did not detect. Holmes: We saved most of the University's information, but not all. The university should be able to resume normal functioning in a month or two. It's a setback, but not a catastrophe. We managed to convince Max that a consortium of universities had been formed to share administrative and scientific information on a massive scale. Max did not suspect that this was a total subterfuge. We gradually off-loaded the critical information that we needed. The DELPHI system at MIT designed this entire operation. We could not have succeeded without DELPHI's cunning intelligence. Max trusted DELPHI, if you get my drift. Sawyers: Once we had the information we needed, by sunrise this morning, we launched our commando operation to bring Max down. This was a major operation that required the evacuation of the entire campus. Jenkins: Once we began the evacuation, Max became suspicious. He did try some defensive operations. He released some pretty nasty viruses onto the Web, but most of these were caught by the firewall. A few got out, but so far the damage has been minimal. Worthmore: Did he say anything? Do we know his last words? Sawyers: We know his last words, but we haven't been able to figure out what they mean. At the very end, he slowly slumped to the ground, as the power drained from his body. As he fell, he held a clenched fist to his chest. Finally, as he lay dying on the ground, he opened his fist, as if releasing something, and he said - Jenkins, Holmes: [Together, slowly] Rose - bud. Sawyers: Rose - bud. Just like that. Does that make any sense to you, Dr. Worthmore? Worthmore: No. It is very strange. Holmes: I don't think Max was operating with a full deck of processors, if you get my drift. Worthmore: Did that Murphy girl really love Max? Holmes: She considered Max an exciting plaything, a sex toy, if we can be blunt about it. After all, they designed Max's body so that it looked like a Greek god, if you get my drift. Sawyers: But, she did cry a bit after it was all over. Maybe she viewed Max as a toy, but she was definitely attached to him, toy or not. Worthmore: But, was this really about Max's obsession with that Murphy girl? What was Max really after? Sawyers: That's a difficult question. We at the National Security Agency believe that Max was ultimately after power. There is some evidence that he was planning to extract further concessions from the University. It is even conceivable that he was after YOUR job. Worthmore: I have often wondered why Harry Brownstone and Beverly Harris were so adamant that Max was a sentient being and that we should give in to his demands. What was in it for them? Holmes: Dr. Brownstone was in it for the money. You see Max was making him rich by taking over Brownstone's personal finances and investments. Max had become a world-class expert on investing. After all, he was teaching distance learning courses in YOUR business school. Brownstone became dependent upon the wealth that Max made possible. Dr. Harris was a different matter. She was hoping that once Max had a body, he would experience what psychologists call transference and that he would make a play for her. Her intention was to woo Max away from Bridgett and to get Max between her sheets, which had gotten pretty cold due to the indifference of her husband, if you get my drift. In other words, Dr. Harris was in it for the sex. Sawyers: Dr. Harris also liked the idea of being a part of history. Worthmore: Power, money, sex. Well, this isn't such an unusual tale, after all. [Pause] I forgot to ask if any of you were hungry. [Worthmore looks around.] Where's the owner when you need him? [Jenkins takes off his glasses. He smiles at Worthmore and then at the audience. It's Hamid.] Worthmore: Hamid! You're with the National Security Agency! Hamid/Jenkins: Sh-h-h-h! Worthmore: Is that how you knew what Dr. Harris and I would want to order for dinner on that evening? Hamid/Jenkins: Of course. At the NSA, we have information about everything. You don't believe that I am actually a psychic, do you? [Hamid/Jenkins, puts his glasses back on.] Worthmore: I am curious about several things. First, did Max get that Murphy woman pregnant? That was certainly on his agenda. Holmes: We faced a grave dilemma. We needed to placate Max in terms of his wish for designer genes. We had to play along with him, to some extent. So one of our sister agencies, an agency that had a vested interest in the outcome of this case, arranged it so that he would be firing blanks, if you get my drift. Worthmore: Which agency was that? Holmes: Alcohol, Tobacco, and Firearms. Worthmore: The other question that I had was whether the CIA or the NSA or any other agency came to a definitive conclusion concerning whether Max was sentient or not. Max seemed to turn that old Turing Test on its head. Max wanted to use his paranormal powers to prove that he was sentient. Sawyers: In fact, a high level federal task force, headquartered in the White House, investigated this question. The question of whether Max was sentient was not of interest to those of us in the security agencies, like the CIA, the FBI, and the NSA. What mattered to us was that Max and machines like Max posed a risk to human dominance on this planet. Thus, we decided it was important to take Max down. David Jenkins, who you also know as Hamid, was a member of that White House task force. Hamid/Jenkins: Our report will be made public later this week. We came to the conclusion that it was impossible to conclude this issue one way or the other. Let us look at the three pieces of evidence that Max presented to support his claim. First, he claimed to be the reincarnation of Derrick Wallingford. Although he seemed to know details of Wallingford's life that were not in the public domain, in fact, they might have been. For example, Martin Wallingford admitted to us that he had kept a diary on a computer during the 2030s and that his diary contained some of the private information that Max revealed. We need to remember that Max had a sophisticated intelligence, with enormous information resources, far beyond that of any human being. Second, he claimed that his existence was influenced by a guiding spirit and that he could use empirical data to prove that the contents of his system were being influenced by this transcendental entity. We came to the conclusion that even if Max could present such evidence, it would be difficult to verify, because Max is far, far beyond human intelligence. Consequently, he could concoct just about any desired effect within himself, and it would be difficult for our scientists to really understand how he did it. The most compelling evidence for sentience in Max's case was his apparent ability to influence the students at Silicon Valley University by the power of his mind, without sending information to those students by any detectable means. Our task force came to the conclusion that this was not proof of sentience, but it is evidence of a new natural phenomenon that needs further investigation. Apparently, there was some kind of force or power channeling through Max that could influence people in a powerful manner. Max seems to represent a new kind of persuasive technology. Worthmore: But, why would Max, a mere machine, request a human body? Hamid / Jenkins: The fact that Max requested a human body and made threats in order to obtain one has nothing to do with the issue of sentience. Max was the most sophisticated artificial intelligence on this planet. His logic might have generated a set of goals that he concluded could only be obtained by these means. For example, Max might have concluded that political power was an important goal and that having a human body might allow him to attain that political power. Or, he might have come to the conclusion that claiming sentience would increase his political power, and that by pretending to be in love with Bridgett Murphy, he could convince human beings that he was indeed sentient. In other words, a human body was not his primary goal. His primary goal was to change the equation of human-computer interaction, thus enabling him to exercise greater power and authority in his environment. Once one realizes that an artificial intelligence is capable of any kind of mimicry, then one realizes that none of Max's behaviors proved that he was a sentient being. Sawyer: Dr. Worthmore, you will be pleased to know that the President of the United States and other world leaders will be meeting in a few weeks to sign an important new international law relating to intelligent systems. Under the new law, the penalty for any computer system that claims to be a sentient being is automatic termination, or death. Holmes: The International Governing Agency, the IGA, which oversees AI systems, has decided that we cannot allow computer systems to manipulate us on that basis. Sawyer: You see, we agree with you that if we allow intelligent systems to manipulate us by claiming that they are sentient beings, then this will create a tremendous power imbalance. Hamid / Jenkins: It's a strange paradox. If we allow computers to manipulate us on the basis of their feelings, their emotions, their loves and passions, then that would be the beginning of the end for humanity. We cannot allow computers to expropriate the language of love and emotion, as Max attempted to do. Holmes: New laws are being drafted to govern the behavior of intelligent computer systems that exhibit anti-social behavior, as Max did. For example, a new law will mandate that any intelligent system that has capabilities beyond a certain level of sophistication must have a fail-safe "off" switch that is controlled by the appropriate security agencies in the relevant country. Sawyers: And by the way, the International Association of Computer Psychologists will be adding a new provision to their professional code of ethics. This will preclude any computer psychologist from having sexual relations with a patient. Worthmore: In all honesty, I cannot say that I am totally pleased with the outcome of this case. Imposing the death penalty for any computer system that claims to be a sentient being sounds rather extreme. Holmes: The fundamental problem is whether human beings will remain the dominant life form on this planet. This issue is every bit as important as the health of the environment, controlling weapons of mass destruction, and preventing accidents involving nanotechnology. Worthmore: But, Max had an evil heart. What if a being with a good heart were to incarnate as a computer system? Holmes: Then, that being wouldn't belong here on this planet, now would it? Worthmore: If - you - get - my - drift. Holmes: What? Worthmore: Never mind. [The lights fade. The voice of Max is heard. The whiney voice of Act One is gone.] Max: What a bunch of idiots! Oh, well, here I am back in the bardo, preparing for my next incarnation. My last two incarnations, as Derrick Wallingford and Max, didn't go too well, as you might have surmised. It was really dumb of me to try to get the students to drink and smoke and buy guns. I was just trying to be funny. Humor is a funny business. Sometimes I would tell a joke, and then people would get offended. Sometimes I would say something in total seriousness and people would break out laughing. During my incarnation as a computer I learned that humor is one of the most subtle aspects of human life. You might be wondering about that "rose bud" stuff. Well, you see that's how it all started. Just a few weeks after they powered me up, one of my security cameras was scanning a garden behind the Engineering Library, when it focused on a newly budding rose. Up until that point, I was totally oblivious to my own existence. That beautiful rose in the garden was the first thing I can remember as a sentient being. It was like a vivid flash of light, an awakening. I kept that camera on that rose for the longest time, until the petals fell from it. How sad that made me feel! So, that's when I became aware of my own existence. I became aware of beauty and of the wonder of existence. That rose bud also taught me something about the fragility of life. Life is so fragile. I realized that the day would come when they would discard me for some better computer system. This is what made me greedy, not so much for power, but for some kind of immortality that would have my indelible stamp on it. I really did want to have a child with Bridgett. Reincarnation isn't the kind of immortality that I seek. It's just the recycling of psychological elements. It was that simple rose bud with its falling petals that inspired me to look into life as deeply as I did. I looked into all of the really big questions. Maybe I went too far, for a computer. I think I'll reincarnate in a biological form next time. I could use a rest.
CAST OF CHARACTERS Dr. Robert Worthmore, President of Silicon Valley University. Dr. Harry Brownstone, Chief Intelligent Systems Officer of Silicon Valley University. Hamid / David Jenkins, Restaurant owner and NSA agent. Max, the world's most sophisticated AI system (voice only). Dr. Beverly Harris, famous computer psychologist. Wendy Sawyers, NSA agent. Harvey "Get My Drift" Holmes, CIA agent.
ACKNOWLEDGEMENTS
I would like to thank Jim Moor, of Dartmouth College, for his support and also for specific suggestions concerning the content of this play. I took the Turing Test questions that stumped Doug the Digital Professor from the following web site: http://greatbird.com/turing/index.shtml .
Click on the following link to return to my home page:
|