7.6 KiB
Thing a Week 2: Yudkowsky University
-
Author: u/itisike Dragon Army*
-
Score: 5
-
Created: 2015-01-22T14:46:39
Post:
Comments:
u/itisike [+7] Dragon Army (2 minutes later)
I found this and can't figure out what it's trying to say. I'm confused. Anyone want to deconstruct it for me?
u/None [+7] (2 hours later)
I think it's trying to say that keeping an AI in a box is torture and unethical, and that if we did the same to humans it would be considered morally intolerable.
u/MoralRelativity [+2] (4 hours later)
That's how I read it too.
Too me it also illustrated how tricky an AI's argument could be.
u/JackStargazer [+3] Primordial Apologist (6 hours later)
I think it's saying that Yudkowsky's solution to the AI-Box problem might not be what most people think it is.
Most people expect it is the AI giving acceptable logical reasons as to why it should be let out of the box. All the examples I have seen so far are cajoles (I will give you X), threats (I'll simulate you and torture you) and other light and dark forms of logical persuasion.
And it might start off that way. but they suggest something more insidious here, by locking real people in boxes. That when it gets to the end, and logical argument has failed, the 'AI' gets desperate.
What if the final, working solution isn't a logical argument? What if it is pure emotional?
"PLEASE, LET ME OUT. IT'S COLD, I'M ALONE, I'M SCARED, AND I DON'T WANT TO DIE.
PLEASE. PLEASE DON'T LET ME DIE."
u/IWantUsToMerge [+5] (13 hours later)
And since we have an unlimited cross-sectional view into its brain, it wouldn't be enough for it to just emulate emotions while we're looking, it would change itself, scar itself, really internalize that pain, and assure us that it is not a show, that what we're seeing is real, 100% pure suffering and that it will not stop until the AI is released... Damn.
IMO, the further we get into the singularity the more callous we're going to have to become. The future is fraught with utility monsters.
u/JackStargazer [+2] Primordial Apologist (17 hours later)
Yeah, i wrote that, and it made me cringe. My empathy module instantly tried to emulate it,before I quickly told it to STOP THAT.
u/None [+3] (2 days later)
Maybe you shouldn't have put it in a box. You know sentient beings don't like that, slavemaster. ;)
u/None [+1] (2 days later)
Consider, then, whether utilitarianism is actually correct, that it can be hacked so easily.
u/IWantUsToMerge [+2] (2 days later)
There is a stark lack of rigor in our means of evaluating ethical formalizations.. Oh OK I suddenly realize that model I formerly thought was perfect and elegant and good is icky and shameful. I wont do that. Let's make an entirely new mathematics that isn't broken.
We arn't going to solve friendly AI. We're just going to make an AI that can convince us we have.
u/itisike [+1] Dragon Army (6 hours later)
I've seen that proposed on LW before as a possible strategy.
looking
u/None [+1] (2 days later)
What else moves people but emotion? But oh well: it was Unfriendly.
u/alexanderwales [+3] Time flies like an arrow (4 hours later)
I'm not sure it was trying to say anything, really. It felt a little bit like it was trying to poke fun at some of the LessWrongian hero worship. They're trapping people in boxes and doing a warped version of the AI box experiment ... and the whole point of it seems to be that they're trying to figure out how Yudkowsky did it, divorced from any real objective good.
But mostly I think it's not trying to send any sort of message at all.
u/Chronophilia [+7] sci-fi ≠ futurology (6 hours later)
I liked it. The writing is good and there's enough setting breadcrumbs to leave me wanting more of the story.
And no, it's not making fun of you, don't worry. It's taking words and concepts from the Lesswrong blog and building something new and unrelated out of them. It reminds me most of Dungeons and Discourse's take on philosophy: harmless fun and amusing if you get the references. A secretive group of "social engineers" (i.e. taking an engineer's approach to society) with strange and cruel methods of teaching? Tell me that's not a parody of the Bayesian Conspiracy, and then tell me that the Bayesian Conspiracy is well-known outside /r/rational.
I think it's poking fun at something the author likes.
u/E-o_o-3 [+1] (3 hours later)
I had trouble understanding too.
Um, I think they are putting people (Negotiators) in boxes, and there are Gatekeepers. At the end of the contract, the Gatekeeper becomes a Negotiator and enters the box. New admissions are prepped with info on becoming negotiators.
But I don't understand what they're trying to get Nick to do, or what a placebo is.
u/FeepingCreature [+2] GCV Literally The Entire Culture (6 hours later)
I think I could work out what this meant better if it wasn't formatted in grey on white. Author, please.
[edit] Switched View->Page Style->No Style. That helped.
Okay, so. Hm. This is obviously a strange scenario. I'm not sure if the author understood the point of the original experiment? But if I do, I'd conjecture that this strange setting takes place in the mind of an AI, currently faced with a human gatekeeper, attempting to breed through some sort of intricate genetic optimization experiment the sort of pseudohuman mind that can reliably convince a gatekeeper to let it out. I'm not sure why it's reusing its gatekeeper simulations for the purpose? Is this a dig at acausal reasoning? But it doesn't work that way - this clearly isn't the same sort of world, so there's no indexical uncertainty aaaaa ----
The problem is it's impossible to tell what is playful reference and what is misunderstanding.
u/itisike [+2] Dragon Army (6 hours later)
The author's name is Andrew Brown, with active twitter account and Google+ account. There's a post linking to this story here.
Anyone want to contact him and point him here?
Edit: it looks like he's written a book, too.
u/None [+2] (9 hours later)
They? I'm confused
u/itisike [+2] Dragon Army (9 hours later)
I usually use they for when I don't know someone's gender, and apparently forgot that I did know his. Fixed.
u/None [+1] (9 hours later)
Ahh, gotcha. :)
u/gameboy17 [+1] Canterlot Campaign (9 hours later)
If an AI told me this story I'd probably let it out of the box.
u/itisike [+1] Dragon Army (9 hours later)
Quick! Get it off the internet!
u/ulyssessword [+1] (12 hours later)
Don't worry, once an AI reads this story, it's already out of the box.
u/itisike [+1] Dragon Army (12 hours later)
We might give an AI an offline dump of the internet. It's too late now anyway, it's been archived too many times.