« One Law to Rule Them All | Main | Asimov's Laws of Robotics: Implications for Information Technology »
Deconstructing Asimov's Laws
by Asimov’s 3 Laws of Robotics may seem a decent set of guidelines for ensuring that future robots and AIs behave in satisfactory ways. But there are several problems that immediately emerge when we look deeper. For one, it’s not so straightforward to convert a set of statements into a mind that follows or believes in those statements. Two, semantic ambiguity means that without personally understanding the reasons for the laws and the original intent, a robot might misinterpret their meaning, leading to problems. Third, Asimov’s Laws ignore the possibility that a robot will acquire the ability to reprogram itself – an inevitable eventuality if intelligent robots are created. How can we confront these issues, as we move closer to the creation of a new intelligent species?
Isaac Asimov’s “Three Laws of Robotics”:
- A robot may not injure a human being, or, through inaction, allow a human being to come to harm.
- A robot must obey orders given it by human beings, except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
Asimov’s Laws are humanity’s early, off-the-cuff answer to the question, “How do we build robots that get along well in human society?” But they led to failure in his stories, and would lead to failure if implemented in real life. Depending on the physical and intellectual power of future robots, and their ability to improve these powers, these failures could range from spilt coffee to the total extermination of the human race.
Asimov’s Three Laws of Robotics describe a minimum set of rules that many humans would prefer that all robots follow. But the rules, in their conciseness, lack content. Words like “robot,” “orders,” “humans,” and “existence” are fuzzy semantic constructs whose meaning lies primarily in the minds that interpret them. Connecting such words with concrete classes of physical systems or processes is not as easy as it seems. Human beings are fortunate to have a rich underlying base of cognitive complexity that connects a few marks on paper (words) to entire classes of complex physical processes.
Often the Laws are mistakenly presented as sufficient to create a robot that is thoroughly safe and fits in well with society. But the Laws are vastly insufficient; they’re simply too simple. The only working examples of benevolence and harmony with society that we have are well-balanced humans – whose behavior is implemented by brains that are enormously complex. In order to create truly benevolent AIs, we will need to create designs with the motivation to absorb major portions of this complexity.
Consider the apparently simple action of sweeping a floor with a broom. To us, it seems straightforward, but the computations occurring are actually far more elaborate than anything performed by present-day computer software. Maintaining balance, scanning a room for dust, performing hand motions to manipulate the broom into the chosen positions – a team of roboticists might slave for decades and still be unable to duplicate such complex processes.
“Getting along well with others” may seem straightforward to us, but it would actually require a goal system of substantial complexity to replicate. Asimov’s Laws ignore the extensive cognitive complexity wrapped up in any intelligent social morality. At best, Asimov’s Laws are a first tiny step on a huge journey of satisfactory goal system design in robots. At worst, they will give pre-robotic human civilization a false sense of security and preparedness, discouraging further research in the relevant areas.
Part of the issue is that humans are heavily loaded with instincts, common sense, and rules of thumb that assist us in determining which actions are acceptable and constructive, to ourselves and to others. Creating robots that add value to human society will not be a matter of creating a “generic mind” and then “constraining” it with Asimovian laws, but creating sensible and benevolent minds from the get-go. Gandhi was not a nice guy because someone forced him to be that way, but because that was part of his inborn character. Future robots should be the same.
In creating future robots and AIs, we will need to plan for scenarios in which robots become smarter than humans, bodies far more effective at manipulating reality than humans, robots which manipulate their own source code, and so on. Pessimists might throw up their arms in despair and say, “At that point, there’s nothing you can do. There is no way a human programmer can code an AI with certain goals and expect it to stay that way. You can’t control something smarter than you are.”
Yet, the very same humans that say these things behave in ways derived from their creator – evolution. Present-day human goal systems are derivatives of our original evolutionary goals. We may not be as uncivilized, barbaric, or narrow-minded as our ancestors, but our goals still firmly derive from evolutionary programming. Even if we decided to completely “throw out” our original goals, the choices we made about throwing out those goals would still somehow derive from the root goals!
Intelligent robots will be the same. A benevolent robot modifying its own programming will not randomly turn to selfishness, unless a cognitive predisposition in that direction is already present. A selfish robot modifying its own programming will not randomly turn to benevolence either. Although we may expect any sort of robot to start modifying itself to become more humanlike, this is pure science fiction without any scientific foundation. Any self-modifying robot will be an n-th order derivative of its initial conditions, and it will be up to us to ensure that those initial conditions contain enough seed complexity to develop into something we can be proud of.
Comments
“Any self-modifying robot will be an n-th order derivative of its initial conditions” — have you got any evidence for that? What does it mean?
Posted by: T at July 17, 2004 09:41 PM
A robot CANNOT reprogram itself in any way that will hinder its three laws, as doing so will break its first law in the process.
Posted by: Anonymous at July 18, 2004 08:44 PM
N-th order? It just means that the state of the robot at any point is dependent upon the initial programming and capabilities of the robot. The variable “n” is simply there to represent that this is valid whether we observe the robot after it has made one modification or after it has made an arbitrarily large number of modifications. No evidence is required.
Posted by: BBlalock at July 19, 2004 09:55 PM
It’s entirely possible to produce robots with hard coded instructions that cannot be changed without replacing the hardware. The trick is to also counteract possible unauthorized tampering by humans or other robots to change the hardware. One could do this by fully intergrating it within their brain so it has to be replaced as a unit, highly expensive and difficult to do so. Combined with propeitary technology and methods, it should be very hard to get in and modify them.
Besides, in Asimov’s books, the robots, even new ones seemed to already have been impressed with basic programming when they were first produced so every single one doesnt have to go through the whole learning process individually which could also alter their outlook and personality. Minimize the factors and you minimize the risks.
Posted by: Aaron Bate at August 1, 2004 09:49 PM
What about if the robot in question makes a judgment call? This in theory does not interfere with the laws if the robot in question believes the fact the his actions are helping humanity, versus disobeying it. Ergo; a robot could (in theory, mind you) re-program itself if the environment it is in requires a judgment procedure.
Posted by: Codeman at August 1, 2004 09:49 PM
What if there was a FOURTH Law? Something like: “A robot must do all it can to remain under the direct control of it’s owner specifically, and any human failing that.”
And to solve the problem of self-modification, a robot must have both a human order and external software stored on a seperate computer (strictly controlled by the company) to change any part of it’s OS. Unauthorised change could, perhaps, be made to destroy the robot, violating both the Third and aforementioned Fourth Laws.
Posted by: Michael Hodges at August 24, 2004 11:54 PM
A brief commentary on three points of the article:
“Third, Asimov’s Laws ignore the possibility that a robot will acquire the ability to reprogram itself.”
As a matter of fact, this is precisely what happened in one of Asimov’s novels, Robots and Empire. R. Daneel Olivaw and R. Giskard Reventlov, determining that the Three Laws were in fact insufficient to facilitate their efforts to further the goals of humanity, developed what Daneel called the Zeroth Law of Robotics - that a robot cannot cause harm to humanity, or, through inaction, allow humanity to come to harm. Daneel was the first to implement this law into his programming, much to the astonishment of Dr. Vasilia Fastolfe, who had less-than-noble intentions regarding the use of R. Giskard. At the end of the same novel, Giskard employed this law and allowed the planet Earth to become fatally irradiated, for the benefits its ultimate desertion would have for humanity’s expansion into the Galaxy. Notice that this reprogramming was exactly in the form you imply: the robots’ natural adherence to the Three Laws influenced their development of the Zeroth.
“Often the Laws are mistakenly presented as sufficient to create a robot that is thoroughly safe and fits in well with society. But the Laws are vastly insufficient; they’re simply too simple.”
It is pointed out in Asimov’s stories that the “Three Laws” are simply a crude linguistic approximation of the programming inherent in a robot’s positronic brain. It seems reasonable that the actual “Laws” are much more complex, and are intrinsically linked to the robot’s abstracted perceptions of the world.
The “fuzzy semantic constructs” you reference also come into play in Asimov’s fiction. Again in Robots and Empire, Solarian humaniform robots are induced to attack human beings through the employment of a very narrow definition of what constitutes a human being. These robots obeyed with alacrity creatures that matched their definition of a human being (i.e., those speaking with a Solarian accent to their Galactic Standard), but were instructed most firmly to destroy all “impostors”.
“Gandhi was not a nice guy because someone forced him to be that way, but because that was part of his inborn character.”
Some psychologists would argue that he was that way because of his life experiences that developed his character, but that’s not particularly germane. If these robots are capable of learning, which they clearly are, the question then is simply one of where the “learning” begins. Human beings are born with some inherent capabilities, while the rest are developed in the course of a lifetime. It would indeed be prudent to imbue robots with their “personality” or predilections from the time of their creation - in this regard, they would surpass the analogous development of a human at the stage of their “birth”.
Posted by: Mr. Accident at September 6, 2004 02:49 AM
“Asimov’s Three Laws of Robotics describe a minimum set of rules that many humans would prefer that all robots follow. But the rules, in their conciseness, lack content. Words like ‘robot,’ ‘orders,’ ‘humans,’ and ‘existence’ are fuzzy semantic constructs whose meaning lies primarily in the minds that interpret them. Connecting such words with concrete classes of physical systems or processes is not as easy as it seems. Human beings are fortunate to have a rich underlying base of cognitive complexity that onnects a few marks on paper (words) to entire classes of complex physical processes.”
This is an engineering problem, and humans will have to tackle this problem sooner or later. A robot will most likely understand such semantics before it becomes intelligent enough to even put the laws (especially the first law) into action.
“Often the Laws are mistakenly presented as sufficient to create a robot that is thoroughly safe and fits in well with society. But the Laws are vastly insufficient; they’re simply too simple. The only working examples of benevolence and harmony with society that we have are well-balanced humans — whose behavior is implemented by brains that are enormously complex. In order to create truly benevolent AIs, we will need to create designs with the motivation to absorb major portions of this complexity.”
I agree with this statement, which brings up an excellent question. How will this design be made? It seems like a more difficult task than the one mentioned in the last paragraph. An AI which is still less intelligent than a human could be experimented with and provide aid to this task. Just as an AI cannot understand our brains, we cannot understand how the AI thinks. This is a task which will require interaction between scientists and a prototype AI.
“Getting along well with others†may seem straightforward to us, but it would actually require a goal system of substantial complexity to replicate. Asimov’s Laws ignore the extensive cognitive complexity wrapped up in any intelligent social morality. At best, Asimov’s Laws are a first tiny step on a huge journey of satisfactory goal system design in robots. At worst, they will give pre-robotic human civilization a false sense of security and preparedness, discouraging further research in the relevant areas.”
Be thankful that re-writing these goals will not alter or jeopardize the implementation of the three laws. Again, more complicated goals that involve “understanding human behavior” will have to be designed by both humans and AI. In any case, I don’t see this as a major problem, as it will slowly and safely be worked out.
“Part of the issue is that humans are heavily loaded with instincts, common sense, and rules of thumb that assist us in determining which actions are acceptable and constructive, to ourselves and to others. Creating robots that add value to human society will not be a matter of creating a ‘generic mind’ and then ‘constraining’ it with Asimovian laws, but creating sensible and benevolent minds from the get-go. Gandhi was not a nice guy because someone forced him to be that way, but because that was part of his inborn character. Future robots should be the same.”
Programming a sensible and benevolent mind would open doors for mistakes and errors in execution, as it would be more complicated than a generic mind. Outfitting a general AI with the three laws and then allowing the robot to gradually understand the meaning of human sensibility and benevolence as it grows would probably be a better way to go about it. You would have to instruct the AI to learn this, however, as it will not learn on it’s own, or will it?
“In creating future robots and AIs, we will need to plan for scenarios in which robots become smarter than humans, bodies far more effective at manipulating reality than humans, robots which manipulate their own source code, and so on. Pessimists might throw up their arms in despair and say, “At that point, there’s nothing you can do. There is no way a human programmer can code an AI with certain goals and expect it to stay that way. You can’t control something smarter than you are.”
I don’t see this as a problem. Modifying the source code in a way that would remove or change the first law would be violating the first law, and the AI would be prevented from doing so. Nothing will ever be made to force the robot to supercede the first law, and the robot will not allow such changes to be made.
“Intelligent robots will be the same. A benevolent robot modifying its own programming will not randomly turn to selfishness, unless a cognitive predisposition in that direction is already present. A selfish robot modifying its own programming will not randomly turn to benevolence either. Although we may expect any sort of robot
to start modifying itself to become more humanlike, this is pure science fiction without any scientific foundation. Any self-modifying robot will be an n-th order derivative of its initial conditions, and it will be up to us to ensure that those initial conditions contain enough seed complexity to develop into something we can be proud of.”
And I am pretty sure that an AI will never modify it’s own code in a way that would change the three laws, as that would be violating those laws. Just make sure that it knows that it would be violating those laws. If it is intelligent enough and has enough processing power, then it should have the ability to stop itself from modifying the source code against the three laws.
Posted by: Mark Nuzzolilo at June 12, 2005 07:30 PM
Powered by Movable Type 3.01D