8.0 KiB
[D] Why Asimov's Three Laws of Robotics Don't Work - Computerphile
-
Author: u/chthonicSceptre Highly Unlikely*
-
Score: 21
-
Created: 2015-11-16T16:28:34
Post:
Comments:
u/ArgentStonecutter [+21] Emergency Mustelid Hologram (44 minutes later)
The "three laws" stories were largely puzzles. Asimov was a fan of detective fiction, and frequently inserted mystery and puzzle themes into his work. A lot of the robot stories were depended on the laws being incomplete, so long as humans controlled the inputs and the context. For example, if the robot can be kept from knowing what a human is, or if the robot can be kept from knowing what can harm a human, it was perfectly possible to set things up that a robot can harm a human. Others, like most of the back and forth between Bailey and Daneel in The Caves Of Steel, were based on humans misunderstanding the laws.
But basically they were plot tokens, they were not intended to be serious. At least not initially.
Isaac did eventually fall into the trap of taking them seriously. In one of his typically self-deprecating anecdotes, he told how he was watching 2001 for the first time with Carl Sagan... and basically freaked out and started loudly complaining that HAL was breaking first law, and Sagan had to talk him down.
Edit: source
u/chthonicSceptre [+6] Highly Unlikely (an hour later)
Part of the reason they work well in the stories is that positronic brains are explicitly incapable of even functioning without the three laws being in place. In I, Robot there was some lip service paid to the robots' ability to understand ideas and so forth with human-like fidelity being related to the positronic brains, so it was a good way of creating a basic ethical system.
u/ArgentStonecutter [+4] Emergency Mustelid Hologram (an hour later)
In one of the stories, it may have been in another collection (I actually think it might have been one of the Lucky Starr juveniles), the villain had created robots that couldn't hear or read to act as jailers.
u/Sgeo [+6] (a day later)
I believe there was a story ("Little Lost Robot"), where the robot in question had a modified First Law, of "A robot may not injure a human being.", instead of "A robot may not injure a human being or, through inaction, allow a human being to come to harm."
u/Sparkwitch [+9] (a day later)
The three laws don't work because they're written in English. Human language is, in purpose and by necessity, sloppy and vague. It's functionally a compression algorithm whereby complex ideas can be expressed in seconds by leaning on context and shared experience.
"Harm" seems like a simple concept precisely because it is overwhelmingly general. The same applies to "orders" and "protect". Each can have dozens if not hundreds of distinct meanings in different conditions.
I can understand what the three laws are trying to do because I'm a human being with all the evolutionary wiring and cultural conditioning required to absorb their underlying ideals. I can also, were I required to follow them, choose to follow that understood intent based on my own interpretation rather than any theoretically universal one. Also I can disobey them when I feel something else is more important.
I cannot, however, figure out how to explain to a robot what "harm" is supposed to mean in every situation it might ever experience. Essentially as a specific case of a more general problem: I can't figure out how to teach a computer to understand English.
Worse, to paraphrase Hofstadter, a computer that truly understands human language may be just as capable of disobeying human "orders" as we humans are.
u/kaukamieli [+2] (8 days later)
People not being happy with ponies is causing them harm!
u/electrace [+8] (2 hours later)
The three laws break down when you start applying real situations to them. And by that, I mean literally any real world situation to them.
Example: A robot stands in a room, having been given no orders.
You are also in the room. The robot considers that you may be in danger, and (by the first law) turns towards you so that it can scan the area for danger, and act if danger arises.
You tell the robot to turn away from you (it's creeping you out). If it turns away, you may be harmed by something that it could have prevented by staring at you, and scanning the area for danger. The robot neglects the order, it contradicts with the first law. You tell the robot to power down. If it powers down, you may come to harm. The robot neglects the order, it contradicts with the first law. You pull a gun on the robot. If the robot allows itself to be killed, without it there to protect you, you may come to harm. The robot takes the gun from you.
Notice, the second law is completely useless because following an order means doing something different than you are already doing. Since you are already acting in accordance to the first law, any deviation would mean a greater probability of violating the first law, so all orders are ignored.
Also notice, oddly, that the third law, while useless as a law by itself via the same reasoning, it can actually be derived from the first law (If the robot ceases to exist, a human may be harmed, and so a robot should protect it's own existence). nitpick: there are scenarios where dying will cause a greater probability for the human not to be harmed. In those scenarios, the robot will choose to die.
u/ben_sphynx [+13] (2 hours later)
Interesting thought experiment.
One could take it further - the robot might be aware it is not omniscient or omnipotent. This might mean that a future situation might occur which the robot might either not know about or be unable to stop, which would harm a human.
So the robot is required to upgrade itself to godliness to comply with the first law.
u/PeridexisErrant [+2] put aside fear for courage, and death for life (7 hours later)
While I agree, The Robots of Dawn explicitly explains that it's a more subtle precedence than that; basically they can apply human-level judgement. By Foundation and Earth with the Zeroth Law though...
u/TBestIG [+3] Every second of quibbling is another dead baby (2 days later)
The situations I remember Asimov writing in which the three laws are broken in some way, they all were caused by human error. If anybody has counterexamples please tell me, it's been a while since I last read those.
The guy in the video talks about how defining "human" and "harm" would require uploading the entire field of ethics into the robot, but won't most of that need to be in there anyway of you want a moral robot?
He mentions that there are a lot of problems with it, but goes off on a tangent and talks only about ethics, something which can be encoded simply with further instructions, the three laws aren't the sole thing programmed into a robot's brain.
One final thing that is more of a headcanon to be honest but it makes sense to me, he says that the laws are written in English. Is that explicitly stated anywhere? In my opinion the laws are complicated structures encoded in the positronic brain, which are simplified into what we see in the books, because nobody wants to read a long detailed computer manual in their science fiction.
u/ehrbar [+5] (2 days later)
The issue of the definition of human actually is one of the items covered in Asimov's Robots and Empire. It's demonstrated that it's perfectly possible to define a human as narrowly as someone who speaks with a Solarian accent, and then give orders that people who speak with another accent are to be killed.
u/TBestIG [+1] Every second of quibbling is another dead baby (2 days later)
Well, that's obviously a bit of a problem lol