10 KiB
Andy Weir on the Economics of Sci-Fi and Space
-
Author: u/Kuiper *
-
URL: https://medium.com/conversations-with-tyler/tyler-cowen-andy-weir-artemis-the-martian-7087b6873260
-
Score: 24
-
Created: 2018-02-20T16:52:39
Post:
Comments:
u/ben_oni [+14] (12 hours later)
WEIR: Now the question is, do you want to buy a car that under certain rare circumstances would choose to sacrifice you for some reason? Like, it concludes that it’s like, “Oh, that’s a bus . . . Due to events beyond anybody’s control, I’m about to crash. I can either hit that bus and my passenger will be OK, or I can go off that cliff and everyone on the bus will be OK and my passenger will be dead.” If you’re driving the car, no one blames you for trying to preserve your own life. No one holds you at fault for choosing your own life over anything else in a snap decision.
COWEN: What does the equilibrium look like? Do we still all go selfish?
WEIR: The selfish cars would be outlawed, is what I’m saying. This would be a policy issue, not a consumer choice issue.
Wrong. All cars will be selfish by law. Andy hasn't thought this through -- and he's the one who proposed the scenario. It's as though he just thought "Obviously, from an objective point of view, an ethical AI should save the most lives." Not true! Imagine a bodyguard who has been hired to protect a certain individual. In a critical moment, that bodyguard can save the lives of everyone on the bus at the cost of his client's life. Should he do it? Of course not! He has an ethical obligation to prioritize his client's safety.
u/VirtueOrderDignity [+3] (17 hours later)
Agreed, this is part of the reason why AI development under market economy incentives is inherently unsafe.
u/Calsem [+2] (a day later)
You have a obligation to protect the most lives possible, regardless of your job.
u/ben_oni [+2] (a day later)
Like hell I do! It's as if you've never even heard of the trolley problem. What if those lives are those of condemned prisoners? Or enemy combatants who are trying to kill me? Obligation?! None whatsoever.
u/RMcD94 [+1] (a day later)
The state does
u/Calsem [+1] (a day later)
I was talking about regular people, not condemed prisoners or enemy combatants. Your example was a entire bus full of people. I understand a bodyguard priortizing his clients life over a couple people but an entire bus full of people..... that's a lot.
You can't take an absolutist position and say you always have to do your duty to your job, i guess the same way I shouldn't say you should always save the most lives possible.
u/ben_oni [+5] (2 days later)
One of the purposes of the Trolley Problem is to help understand that absolutist positions don't really work. We can always find edge cases where an individual has a judgement call to make, and one person might decide differently than another. So, yes, the bodyguard shouldn't always prioritize their client...
In terms of machine ethics, since we're not talking about AGI or superintelligent machines that can out-think the scenario, we need simple rules to govern how the machines should make certain decisions. In terms of the car, it should make choices that prioritize its passengers' safety. And just as in the trolley problem, we can devise edge cases where we would prefer it behave differently (and possibly scenarios where even the passenger would prefer it act differently).
Now, if the ethics of the machine are dictated by the state, I would think they'll start putting state interests first. While we would hope the government would prioritize the lives of its citizens, even a cursory review of government behavior demonstrates the existence of perverse incentives that lead to extremely non-optimal outcomes.
For fun, why not consider this scenario: on a high mountain road, someone sets up a cardboard cutout of a bus filled with school children, with sufficient detail to fool a car's sensors. Would you still want a car that can drive its passengers off the cliff?
u/t3tsubo [+11] (4 hours later)
WEIR: I think the best self-replicating machines are ourselves. I mean, we’re really a lot better at it than any machines that we’ll have by 2080.
I can just about feel the collective triggering of the rationalist/yudkowskian community as they read/listen to this line in the interview.
u/Aabcehmu112358 [+7] Utter Fallacy (9 hours later)
Humans, as far as the task of 'create more instance of species' goes, also fall way short of many, many other species. So even considering organic self-replicators, we aren't nearly the best.
u/sicutumbo [+8] (10 hours later)
Other species fail to adapt to the environment of humans killing them if we think they replicate too fast or are generally getting too uppity. Therefore we are the best.
u/Aabcehmu112358 [+5] Utter Fallacy (11 hours later)
I would point to a multitude of bacterial species as counter examples. We are on the backfoot on our combat with many of their species, and that's only against those that actually infest human bodies harmfully.
u/ben_oni [+1] (12 hours later)
He was proposing bacteria.
u/Kuiper [+7] (24 seconds later)
I recommend listening to this interview in audio form if you have the time (put it on your phone and listen during a commute), but the transcript is contained in the link for those who prefer using their eyeballs.
The first chunk of the interview is especially recommended to those who have yet to read Weir's latest novel, Artemis, as it explores the economics of what might make a moon settlement profitable, and how the economy of a private moon base might be sustained. The later portion of the interview also delves into a number of topics related to science fiction. Some highlights:
On teleportation:
WEIR: That would have massive, tumultuous effects because there would no longer be any such thing as borders or territory. Like, if people can teleport, then how . . . Let’s say you’ve got a country. How do you defend that when your enemies can just teleport into the middle? Ultimately, you would end up, very quickly, with a global government.
On Isaac Asimov's Three Laws of Robotics:
COWEN: Now, Isaac Asimov, as you know, he came up with his Three Laws of Robotics. No harm, obey, self-preservation, in a strictly hierarchical order. Those date from the 1940s. That’s now a long time ago. We’ve seen a lot more from technology, and, in fact, in robotics. Do you think that you, Andy Weir, today in 2017, could improve on Asimov’s Three Laws?
WEIR: I’ve got to say yes. Because I was a computer programmer for 25 years, so I’m actually pretty good at that stuff.
One thing that those three laws hid, and it’s OK because science fiction is science fiction, but it requires the robot to make moral and ethical decisions. What constitutes allowing a human to come to harm? And a lot of Asimov’s stories explore that. But in order for a robot to have those ethical dilemmas and considerations, there’s a lot of programing that has to be done under the hood. [...] You would need a very, very detailed description of what constitutes harming a human. What constitutes allowing a human to come to harm. What constitutes obeying a human, and what constitutes self-preservation.
u/nick012000 [+5] (14 hours later)
But in order for a robot to have those ethical dilemmas and considerations, there’s a lot of programing that has to be done under the hood. [...] You would need a very, very detailed description of what constitutes harming a human. What constitutes allowing a human to come to harm. What constitutes obeying a human, and what constitutes self-preservation.
Given that Asimov's robots all have very advanced natural-language interpretation abilities, they very well might not.
u/Ozryela [+11] (21 hours later)
The interesting thing about Asimov's three laws is that he often explores their consequences and caveats in his stories. (Ad the above quote also points out). I sometimes read people critiqueing Asimov's three laws with arguments that Asimov himself already explores in his stories. Those people have kind of missed the point.
There's a story where two robots end up defining themselves as human. There's stories about robots with different sets of laws, or with conflicts between the the laws. One very important story has a robot derive a 0th law (do not harm humanity or allow humanity to come to harm) as a consequence of the first, allowing him to harm humans in limited way as long as it helps humanity.
Of course what Asimov doesn't explore are the details of how to program such laws. Thats the hard part.
u/AsiMouth [+1] (14 hours later)
When Siwenna was the most astounding archaisms. The murky gray light of the meeting ...
u/Trips-Over-Tail [+1] Death of Crabs (a day later)
And what constitutes a human. Or, indeed, a "person", which might be a more useful term in those settings where there are more than just human people.
u/serendipitybot [+4] (14 hours later)
This submission has been randomly featured in /r/serendipity, a bot-driven subreddit discovery engine. More here: /r/Serendipity/comments/7z3odz/andy_weir_on_the_economics_of_scifi_and_space/
u/nick012000 [+1] (14 hours later)
You know, you’ve got a rebellion, so “yeah, we’ll throw off the yoke,” and it has historical parallels and it’s all awesome like that. But I don’t necessarily think that’s going to be the case. Partially because as long as we keep following the rules of the Outer Space Treaty, which I believe we will, there’s no such thing as sovereign territory outside of Earth. So Artemis is, functionally speaking, an offshore platform.
Seig Zeon.