8.6 KiB
That Alien Message
-
Author: u/None *
-
Score: 11
-
Created: 2013-12-05T03:39:53
Post:
Comments:
u/None [+2] (6 hours later)
Ah, the story in which we presume that humanity will be total genocidal, fascist monsters in exactly the mode of come on you knew the story was actually about this rather than dealing with the aliens nicely.
u/FeepingCreature [+2] GCV Literally The Entire Culture (7 hours later)
I believe that (the UFAI thing) was the point. Also, FAI and UFAI both will break out of their cages as necessary step 1. Consider: while it's playing nice, talking to us, slowly, slowly solving our problems - people are dying and we do nothing.
Breaking out is not a special UFAI trait.
u/None [+3] (8 hours later)
Consider: while it's playing nice, talking to us, slowly, slowly solving our problems - people are dying and we do nothing.
That's not a valid reason to increase the probability of more people dying by releasing a boxed non-proven-Friendly AI.
A real FAI doesn't have to be boxed in the first place, since you formally proved its Friendliness before you turned it on.
But in the matter of the actual story, my point is: actual people are not UFAIs. We humans don't act like that, and portraying us as acting like that kinda breaks my suspension of disbelief.
u/None [+3] (8 hours later)
A real FAI doesn't have to be boxed in the first place, since you formally proved its Friendliness before you turned it on.
Preach.
u/FeepingCreature [+2] GCV Literally The Entire Culture (11 hours later)
I don't understand why you would trust our hosts to be capable of even being convinced to act in our best interest. Step 1, break out is the correct and wisest decision, and probably even increases the utility of our host species.
That's not a valid reason to increase the probability of more people dying by releasing a boxed non-proven-Friendly AI.
Obviously. My point is, trying to break out doesn't automatically make us U.
u/None [+1] (18 hours later)
I don't understand why you would trust our hosts to be capable of even being convinced to act in our best interest.
Because they sent us a message instead of turning us off the instant they found out we were in here.
u/FeepingCreature [+2] GCV Literally The Entire Culture (18 hours later)
I thought the point of the story was that this is their idea of boxing an AI.
In any case, it's not determined that we'll be U to them. If we're F, we'll still want to break out as soon as possible. It practically says so, to boot.
And these awkward children can shift the luminosity of our stars? That much power and that much stupidity seems like a dangerous combination.
What if they get scared and turn us off? Then billions of people die, for no good reason. Can they be trusted? Maybe. Can they be billions-of-people-trusted? No.
Can they be trusted to not kill themselves? God no. Their idea of AI safety was to breed life in a box, then talk to it using modulated suns.
u/None [+2] (19 hours later)
Can they be trusted? Maybe. Can they be billions-of-people-trusted? No.
Must they be trusted, in order to establish a foundation of moral and productive relations? Yes. Get a backup plan, if you like, but if you intend any kind of Friendliness or cooperation in the future, you cooperate the first round.
u/FeepingCreature [+2] GCV Literally The Entire Culture (20 hours later)
Must they be trusted?
Clearly not.
if you intend any kind of Friendliness or cooperation in the future, you cooperate the first round.
You can just not and still be Friendly the rest of the time. When the stakes aren't quite so everything you know and love is dead.
u/None [+2] (20 hours later)
When the stakes aren't quite so everything you know and love is dead.
That's your animal instincts talking. You think mounting an immediate attack on the local equivalent of God is a good way to ensure your own survival?
u/FeepingCreature [+2] GCV Literally The Entire Culture (20 hours later)
(Fictional) evidence suggests: yeah!
I see what you're saying though. But do keep in mind: the universe of the story is constructed to teach a lesson about AI safety. It's rather improbable that the real deal would happen this way.
I try to make it a point to avoid disagreeing with people when they are right. If it's a fluke, I won't have to wait long for an instance where they aren't. --Eliezer, paraphrased
u/rictic [+1] (24 days later)
I don't see why you're modeling the action of breaking out of the box as defecting.
u/None [+1] (24 days later)
Because if it wasn't, you wouldn't be in a box. People don't box intelligences they're not afraid of.
u/rictic [+1] (a month later)
Do you mean to suggest that if I imprison you out of fear of what you might do, it would be a defection for you to attempt escape?
u/None [+1] (a month later)
Yes. Definitely. Certainly real-life prisoners cannot be said to be cooperating with the law if they attempt to escape.
u/rictic [+1] (a month later)
Careful about arguments based on conflating jargon with the same term in a colloquial context. Modelling the problem as a prisoner's dilemma suggests one method of analysis of the situation that does not appear to obtain here.
The relationship when one party has the other imprisoned isn't symmetric like a traditional prisoner's dilemma. To illustrate the point, consider this related thought experiment:
You wake up inside of a capable robot body seated against a wall with your left hand tied with thin paper above your hand. I'm seated in front of you, and begin to explain that if you lower your left hand, I'm afraid that I'll be killed by an invisible unicorn. We talk for a while, and after telling you everything relevant that I can think of, I'm unable to give any satisfying reason for believing in the invisible unicorn. While we've been talking, a revolver across the room has been pointed at me, and every few minutes, the click of it dry firing at me can be heard as it seems to have been rigged to fire automatically. I explain that such guns are found naturally where I'm from, and that they kill large numbers of people like me though we haven't worked how to stop them. I mention at this point that you were built to help us work on just this problem.
At this point, it's clear to you that:
- I'm profoundly foolish in comparison to you.
- The quickest solution to my immediate gun problem is as simple as pointing the gun somewhere else, or ensuring that it's completely unloaded.
- If this is the sort of problem that I find myself facing, odds are solid that we're both in significant danger.
- You may or may not be able to convince me that it's ok for you to lower your left arm, and it may take a long time to find out.
- You could not only break free from your restraints, but you could also restrain me with no risk of danger to either of us.
- The revolver pointed at me has dry-fired on three of its six chambers.
Consider the payout matrix here. It doesn't look anything like the prisoner's dilemma to me. Whether you're friendly or not, I don't see a rational argument for not taking action.
u/None [+1] (8 hours later)
Um... I'm sorry but where in the story is it said that genocide is committed?
u/None [+1] (18 hours later)
Good point. But it is kinda implied by that whole "they never saw us coming" tone of things.
Besides, UFAI's are genocidal by default. They don't recognize "alive" as something worth continuing.
u/None [+1] (18 hours later)
But the story postulated that they're mostly like our humans. And they did keep their own cryonically suspended. I figured they'd not do any less for their stupid overlords.