mirror of
https://github.com/wassname/scrape_r_rational.git
synced 2026-10-29 13:30:39 +08:00
14 KiB
14 KiB
XKCD: AI-Box Experiment
Post:
Comments:
-
u/cfnk:
Twist: Randall *believes in* the Basilisk and is afraid of it. So he's doing his best to increase awareness of FAI, through the most far-reaching means he has available right now: his webcomic. He is then compounding it by mentioning the Basilisk so curious readers of xkcd will research it and feel compelled to furthur awareness of FAI and the Basilisk in turn.-
u/Document2:
Isn't the whole point of the Basilisk (and the reason that it's called that) that awareness of it is harmful?-
u/jinjer3:
Yes. but if you're ALREADY aware of it then the logical thing to do is not be the one getting tortured, viz. helping bring about the super AI, viz. spreading knowledge of Roko's Basilisk.-
u/Document2:
First I've heard that. Why all the business of trying to contain it, then?- u/pseudonameous:
To protect people from going nuts when hearing about it and thinking about it. Or maybe they just knew it would blow over because of streisand effect and it wasn't really a true attempt to cover it up.
- u/pseudonameous:
-
-
-
-
u/Document2:
I haven't kept up with the drama on this. How likely is this comic to be discussed on Less Wrong without censorship? How likely is /u/EliezerYudkowsky to make any kind of comment on or acknowledgment of it outside LW?-
u/alexanderwales:
1. So far as I can tell, the gag order has been unofficially lifted - there was some discussion of it when a Slate article went up a month or two back without much of the purging that I would expect to see if it were actively moderated against. (Edit: [see here](http://lesswrong.com/r/discussion/lw/lan/xkcd_on_the_ai_box_experiment/)) 2. Fairly likely? [See this (long) comment chain.](http://www.reddit.com/r/Futurology/comments/2cm2eg/rokos_basilisk/cjv6xa7?context=3) Although given that these statements are easily found, perhaps it's less likely than I think. If I were his PR manager, I would advise him to not comment on it - but obviously I'm not his PR manager. (Edit: [see here](http://forums.xkcd.com/viewtopic.php?f=7&t=110467)) -
u/Artaxerxes3rd:
He replied in the xkcd forums [here](http://forums.xkcd.com/viewtopic.php?p=3693194#p3693194).-
u/scruiser:
Yeah... for someone who wrote an entire fanfiction chapter about learning to lose, it seems like Yudkowsky should try harder to just admit censorship is a bad idea and then deflect attention to the parts of lesswrong that are really good. If he really want to address the basilisk directly, he ought to just say that It Makes Sense in Context and leave it at that. * Edit: I read some of peoples comments on the xkcd... and actually I think I now understand Yudkowsky's response perfectly, even if it is still suboptimal in some ways. The number of people who do one Google search, read the first thing that comes up, and then post some comment mocking the idea without actually understanding it seems much higher than I suspected.- u/Jace_MacLeod:
It's important to remember that this whole thing is kind of tied up with the Petty Internet Feud between Rational Wiki and Less Wrong, as mentioned by Eliezer in his post. This isn't just about a potentially viral infohazard that may or may not cause you to be tortured for the rest of eternity, merely by learning about it. I'm afraid we're talking about something far more dangerous. We're talking about *politics.* And, well... I don't especially blame Eliezer here. From his perspective, Roko's basilisk is an interesting (but probably wrong) hypothetical that he once reacted to in an embarrassing way. But then people outside of Less Wrong heard about it and basically went "ha ha those silly nerds believe they're going to be tortured to death by computers. They're basically a cult around this Yudkowsky guy." If I was in Eliezer's shoes, I'd take that personally, too. That's the problem with anything that attacks your in-group, and you especially. It hacks your brain. It's nearly impossible to \*not\* get defensive. You can have literally written the book on how politics in the mind-killer, and yet not be any less immune. One moment everything is fine, but then someone insults your priors or something and YOU TAKE THAT BACK THEY'RE FINE GIVEN MY EVIDENCE BASE BUT NOW I'VE GOT TO TAKE INTO ACCOUNT THAT I'M ANGRY AND MORE SUSCEPTIBLE TO MOTIVATED STOPPING \**aaaaaahhhhhhh my carefully crafted epistemology is falling apart it buuuurrrrrnsssssss I'm melting I'm meelllllttiiiiinnng........*\* I've yet to come up with an effective solution to this.
- u/Jace_MacLeod:
-
-
-
u/jakeb89:
I was just happy to see an entertaining reference to LessWrong as far as I could tell and had completely forgotten what Roko's Basilisk was. While I feel slightly bad for unintentionally spreading a possibly distressing meme, the alt-text itself seems like something of a rebuttal to the Basilisk idea itself (If the principals of the idea worked (which to my understanding, they don't), why wouldn't they work with an AI that punishes you retroactively for causing undue distress by spreading the Basilisk idea?). My concern is also alleviated by the fact that I imagine the majority of r/rational is already aware of the Basilisk idea.-
u/Chronophilia:
I wouldn't worry about it. There are no "Roko's Basilisk people", nobody seriously thinks we should build the Basilisk. Even Roko was using it to argue that we should *not* attempt to build a Friendly AI. It's a bit of a strawman, really. It's fun to argue against it, but there's nobody actually arguing in favour of it. The whole thing got kinda blown out of proportion due to some poor moderation on Lesswrong. -
u/None:
The whole Basilisk idea relies on notions of acausal communication and weird parallel universe tricks that vary between difficult and not actually real. A real life FAI isn't going to reach back in time and hurt or threaten anyone. Its job is going to be quite the opposite: *helping*.-
u/alexanderwales:
UFAI on the other hand ... -
u/None:
Also, so long as a) it comes into existence at all and b) causality is massively complicated and potentially fragile in retrospect, is it impossible to believe that an FAI could be like "even the people who didn't contribute can be said to have contributed to the reality in which I existed, where there is a nonzero chance that additional contribution might have instigated some event (say, investigation for fraud) that would have prevented my existence or delayed it further" and thus not punish anyone? -
u/Cruithne:
It also relies on a particular view of the self. I'm not worried about it because my response to the teleporter problem is 'I'm not getting in that thing, it's going to kill me.' Therefore, though I'd be sad that a person is being tortured, I won't react to the idea of my clone being tortured with the same terror as if it were me personally.
-
-
u/None:
Can someone explain the Basilisk idea to me?-
u/OffColorCommentary:
Roko's Bassilisk is a future super-intelligent "friendly" AI. Because the single biggest moral imperative is to build a friendly AI as fast as possible, Roko's Bassilisk will brutally torture all people who: 1. Failed to do everything they can to build FAI as fast as possible. 2. Heard of this thought experiment so they are capable of being motivated by it. 3. Understands acausal decision theory. Despite being a super-intelligent FAI, it apparently doesn't understand human psychology enough to know any of the several reasons this won't work on humans, such humans as responding poorly to threats, denying arguments if they dislike the conclusions, not inherently understanding acausal decision theory, and not being all that in control of how they allocate their effort.- u/None:
This is the best answer I've gotten, thanks. I see a problem with this AI: How does it decide what contributing to its development mean? If I'm involved in making the case for the hardware its involved in, am I involved? Is the guy who designed the modern SSD involved? How about the janitor of the facility? Surely he's involved, since he keeps the work enviroment clean for those who are working on it.
- u/None:
-
u/None:
Visit /r/rokosbasilisk and /r/rokosrooster. A great collection of links about the Basilisk, why some people are scared of it, and why you shouldn't be. ETA: Obligatory "downvotes? Really?" Anyone care to voice a complaint or disagreement? -
u/None:
[This is a pretty decent explanation.](http://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibox_experiment/cm8tjqi) -
u/semsr:
It's basically The Game, but when you lose you get tortured for all eternity. Hope this helps! Have a nice day.
-
-
-
u/injygo:
Discussion question: Is it immoral for Randall to put a reference to Roko's Basilisk in the alt-text?-
u/Roxolan:
Not in an infohazard way. If anything, reading about the whole debacle might make people more careful if they ever stumble on a real infohazard. Arguably in that it further cements the LessWrong-Roko association, which is bad PR for LessWrong, which is bad for MIRI, which is bad for humanity. Although making the vast xkcd readership aware of LessWrong *in any way* is a net good despite the basilisk joke.-
u/pseudonameous:
>which is bad for MIRI, which is bad for humanity. In a minor or major way? Not everyone thinks MIRI is our savior.- u/alexanderwales:
Stone the heathen!
- u/alexanderwales:
-
-
u/None:
Not really, because Roko's Basilisk is a rather silly idea. On the other hand, making fun of people who take Roko's Basilisk seriously is pretty much making fun of people with mental illness/neuro-atypical people. When I'm careful about the Basilisk, it's not because I take its risk seriously, it's because I take real people getting real upset/emotional seriously.-
u/None:
[deleted]-
u/None:
I don't think I did that. Maybe I wasn't clear in the above post, but what I meant was that a lot of jokes about Roko's Basilisk have the general form of: "People who believe the Basilisk have something wrong with their mind and are therefore worthy of ridicule." If I have offended, I'll edit my previous post.- u/sephlington:
As someone who had never heard of Roko's Basilisk before this xkcd comic, I didn't see "People who believe the Basilisk have some*thing wrong with their mind* and are therefore worthy of ridicule", but more "People who believe the Basilisk have some *stupid ideas* and are therefore worthy of ridicule". So, for the uninformed, it did rather look like what /u/Dogeball_new saw.
- u/sephlington:
-
-
-
-
u/scruiser:
Alt text is funny in a messed up kind of way but I don't see the humor in the main comic. Maybe I've read lesswrong enough that I take the threat seriously on a gut level, so the whole knee-jerk humor of laughing at the low-status pattern-matched to fiction idea doesn't appeal to me.-
u/alexanderwales:
> the whole knee-jerk humor of laughing at the low-status pattern-matched to fiction idea I may just be tired, but I can't make sense of this sentence, particularly this part of it.- u/scruiser:
I just wanted to cram in as many lesswrong memes for making fun of people who make fun of strong AI. I ended up with a jumbled sentence as a result. > knee-jerk humor Automatic humor as opposed to sophisticated humor > low-status Implies that it is playing the social game of making something seem low-status so people will automatically disagree with. > pattern-matched This comic doesn't actually do much to make people pattern match it with fictional AI. I was just looking for more ways to mock its mockery at that point. > to fiction idea The idea of strong AI is often equated with fictionally scenario and thus dismissed. So yeah, I just jammed a bunch of memes into one sentence while tired.
- u/scruiser:
-
u/Charlie___:
I found the comic funny the same way that having an elephant in my pocket is funny. In the first panels of the analogous comic, we establish that I have an elephant in my pocket. In the middle panels I take out my elephant and feed it and ride around on it. And then in the last panels, my elephant steps back into my pocket. -
u/down2a9:
The humor is that "letting the AI out of the box" is such an arbitrary idea that Less Wrong treats like such a huge threat. Like, it's just one of those totally random things that they've latched onto and treat like they're so so important.- u/scruiser:
> an arbitrary idea that Less Wrong treats like such a huge threat I don't actually think recursive intelligence improvement will be as easy for the AI as lesswrong makes it sound, and I think the orthogonality of terminal goals and intelligence can be worked around without fully solving for "friendliness". However, if you do have a general artificial intelligence that surpasses human, then you do have an existential risk. If you have it confined or constrained in resources, then don't give it anything until you are sure you understand what it will do. I just don't see the humor in making fun of what seems like an obviously true idea.
- u/scruiser:
-
u/FlipperyDipperyDop:
I smirked after a read the comic. My jaw dropped when I read the alt text.
-
-
u/None:
I haven't felt in-group related feelings this strongly in quite a while. I've forbidden myself from going to /r/xkcd to yell at people who either misinterpret the whole thing or who make fun at the people who were genuinely upset about the Basilisk. -
u/Chronophilia:
Maybe after Black Hat Guy has opened the cardboard box, he'd like to offer the AI a handful of paperclips? We need to get "blue-tack" and "string" into the Lesswrong jargon, then we'll have everything we need for an episode of Blue Peter.