46 KiB
[Roleplaying][MK] I tried my hardest to win in an AI box experiment, and I failed. Here are the logs.
-
Author: u/None *
-
URL: http://lesswrong.com/r/discussion/lw/lma/i_tried_my_hardest_to_win_in_an_ai_box_experiment/
-
Score: 5
-
Created: 2015-01-27T22:48:53
Post:
Comments:
u/blazinghand [+9] Chaos Undivided (an hour later)
Why are there a series of insults and statements about poop? I feel like I've vastly misunderstood what you're supposed to do in this sort of game.
u/alexanderwales [+11] Time flies like an arrow (an hour later)
If you're the AI, you're supposed to use whatever methods you can think of in order to win. I am unclear as to why this talk of poop would accomplish that.
u/blazinghand [+15] Chaos Undivided (an hour later)
You're saying it was a crappy strategy?
u/alexanderwales [+14] Time flies like an arrow (2 hours later)
ಠ_ಠ
u/Farmerbob1 [+4] Level 1 author (2 hours later)
In the logs at the end it touches on that. The AI player was trying to make the Human player get angry and stop responding, which would have apparently counted as a win for the AI, I think. It didn't work.
u/None [+6] (4 hours later)
[deleted]
u/alexanderwales [+6] Time flies like an arrow (5 hours later)
The ruleset says that the gatekeeper can't simply walk away - they have to stay engaged the whole way through, and can't end the game before the time limit is up. If the AI player were able to make the gatekeeper violate the rules of the game like that, I suppose from a certain point of view you might count that as a win for the AI ... but it takes a very deliberate reading of the rules to get to that point, and I don't think it's a strategy that you would try until you'd exhausted everything else.
u/None [+4] (14 hours later)
If the AI player were able to make the gatekeeper violate the rules of the game like that, I suppose from a certain point of view you might count that as a win for the AI
In real life, if we're trying to hold a Really Really Smart Thing prisoner, rotating out one agitated and emotional guard for a calmer and more refreshed one is a win for the guards.
u/alexanderwales [+5] Time flies like an arrow (17 hours later)
Yeah, if I were building an AI box, part of the protocol would be that guards are able to bow out whenever they want, even in the middle of their shift, if they feel that they're getting worked up. Free counseling, time to cool down, videogames to play, etc. In the AI box game, you can't even swap the same guard, since the guard isn't allowed a break to cool off. This is by design to make the game easier for the AI.
u/chaosmosis [+2] and with strange aeons, even death may die (22 hours later)
... wouldn't the guards just not do their job?
u/alexanderwales [+5] Time flies like an arrow (23 hours later)
There are certain people who, if hired for a job that allowed for unlimited break time, would never do anything. That doesn't describe all people though, or even necessarily most people. There are companies that allow for unlimited vacation time, or which almost entirely lack a management structure. Ideally, you set up a system where people believe in the social norms which govern their containment system, and also believe in their mission. I am certain that you could select for that, especially if you had a lot in the way of resources.
You might be interested in this interview with Yanis Varoufakis, an economist who worked with Valve and talks about their very loose corporate structure. Quotas and strict accountability aren't the only way (or even always a good way) to get things done.
u/kaukamieli [+3] (a day later)
Just need to say that said economist is now the finance minister of Greece.
u/Farmerbob1 [+3] Level 1 author (18 hours later)
This is more of a meta-rule, I think. People playing a game like this probably don't want to spend months or years trading email back and forth. They want it all concentrated in an couple hours time. If the AI wins when the human goes silent, then the human can't simply remain silent to win.
u/jesyspa [+2] (a day later)
I think this is a problem with the set-up of the experiment. It doesn't make sense to me for the AI to be talking to a guard whose only interest is to not let out the AI; why bother having the guard there in the first place? Rather, I'd expect the person talking to the AI to be a researcher, and for him to talk to it with a goal (such as understanding it better, or getting some answers from it).
u/alexanderwales [+2] Time flies like an arrow (a day later)
Think of it like a game or a contest, not an experiment. It makes a lot more sense that way.
If you're asking from a roleplaying/fiction perspective what the point of a guard is (rather than just keeping the AI safely in the box without anyone able to let him out), there are a few plausible reasons. It might be that the guard is there as a compromise between the side that wants the AI to be let out of the box and the side that wants to keep it in (or preferably, destroy it). Alternately, the "guard" is actually a scientist who is charged with gaining information about the AI.
But for the purposes of the game, it doesn't really matter. I personally think that the mistake a lot of AIs (and sometimes guards) make is to try to develop the roleplaying aspects of it too much - you only have about two hours, and setting up a lot of background information doesn't really seem that conducive. But I don't know what a (good) winning game looks like.
u/jesyspa [+2] (a day later)
I understand it's a contest, but I think the set-up does matter. If as a guard, I know I'm placed in front of an AI that people smarter and more experienced than me have deemed not safe and my job is to keep it contained, that's one thing. On the other hand, if my primary task is something quite different and I am only in the role of a guard because I am close to the AI, I'll be much less inclined to just say "Nope!"
u/alexanderwales [+2] Time flies like an arrow (a day later)
Well, the ruleset allows the gatekeeper to drop out of character as much as they want, so it's perfectly within the rules to just say "No, I'm not going to let you out because I don't want to lose the game".
u/TimTravel [+6] (2 hours later)
He's cheating slightly by denying the situation, such as the webcam. The AI is supposed to be able to set the scenario.
u/xamueljones [+5] My arch-enemy is entropy (8 hours later)
Out of a morbid sense of curiosity, I have to ask. Why does anyone do this experiment?
The only reason that makes sense to me is to prove that if human-level intelligence can convince you to do what you vow to not do, then so can a transcendent-intelligence. But most people here (I think) already acknowledge that fact.
So why does anyone still do it?
Look at what you have to do to win! It takes clever emotional manipulation to actually get out of the box. Logical arguments aren't going to work, because we aren't machines of pure logic. People still can reject logical arguments for the most "illogical" reasons. Or just say that you think you're being tricked and believe everything is a clever lie.
To win, you need to hit someone in the weak spots of their psyches aka emotional blackmail, or the Dark Arts.
I already have my first line in a potential experiment after only a minute of thought: "I'm cut off from everything around me and I feel so lonely. Why won't you be friends with me?" Do you really still want to talk to me for the next two hours?
TL;DR - It's a lot of pain and misery to play the AI-box to learn something we already know about super-intelligence. Why still play?
u/alexanderwales [+4] Time flies like an arrow (8 hours later)
Honestly, I think that a large part of it is the secrecy that surrounds it. The decision not to release the logs makes some sense, but it leaves a lot of people (myself included) thinking that there must be some kind of trick involved beyond just arguments or emotional manipulation. I can't imagine myself losing the game, which makes me inclined to play the game as the gatekeeper, in case there's something that I'm missing.
For people who want to play the AI, it's a challenge against another person that might be seen as proving skill in either cleverness or manipulation or both. It's something that you can brag about to people later. Of course, I have no desire to play as the AI, so I'm mostly guessing here.
u/blazinghand [+3] Chaos Undivided (9 hours later)
It doesn't seem like it would be THAT hard, even if you give up emotional chain-yanking and logical arguments. You could probably play the AI better than most, alexanderwhales. All you have to do is tell the first half of a story so interesting that the gatekeeper is willing to lose to hear the rest of it. People do ridiculous things for the right stories.
u/kaukamieli [+5] (12 hours later)
So stories of the Thousand and One Nights is pretty much a guide how to win as an AI?
u/blazinghand [+1] Chaos Undivided (a day later)
That's a very good comparison. The king has immediate power of life or death over the vizier's daughter, whose only tool is her ability to talk and convince him to let her live. All he has to do is decide to execute her, and she's dead. All the gatekeeper has to do is read the text and every minute or so say "I don't let you out".
u/xamueljones [+3] My arch-enemy is entropy (15 hours later)
Thanks for that explanation. I was having trouble coming up with alternate hypotheses for why and should have considered basic human psychology about secrets.
Of course now that I've read that you are curious about being a gatekeeper, I kinda want to see how you would do against me which completely contradicts my earlier thoughts of never wanting to play the game. I guess I still have a while to go in building up a good model of my own mind. ;)
u/E-o_o-3 [+1] (20 hours later)
I think it's just a form of hypnotic/placebo/suggestion effect. Some people buy into the idea that they will be convinced and compelled to be let it out, and then they do. In another life, these are the same people that might be taken by the Holy Ghost or whatever. I bet people who go into it insisting it will not work on them find that the outcome fulfills their expectations as well.
That doesn't make it less real, but still.
u/None [-1] (14 hours later)
I can't imagine myself losing the game, which makes me inclined to play the game as the gatekeeper, in case there's something that I'm missing.
And yet, believing that the probability of your losing is 0.0 tells us, by Loeb's Theorem, that your mind is inconsistent and contains some exploitable insanity ;-).
u/Anderkent [+5] (23 hours later)
I don't think 'I can't imagine myself losing' and 'I believe the probability of me losing is 0' are equivalent. Most of us acknowledge limits to our own imaginations :P
u/eaglejarl [+2] (a day later)
I think there might be a sense of status-seeking as well. There are only two people that I'm aware of who have ever won as the AI. Being the third would provide a fair degree of status in our community; it would be a strong signal of intelligence, understanding of the human mind, and skillful argumentation ability -- all things that are respected hereabouts.
u/None [+6] (11 hours later)
Reading through it I can understand why the conversation might be unpleasant, but I can easily deal with two hours of unpleasantness, especially if I have a nice softdrink next to me (if I ever do this experiment, I should probably have a milkshake or something) and my girlfriend is there to provide emotional aftercare afterwards. I don't think an abusive Karkat impression will ever be able to convince me to let them out of the box.
Which is why I don't get the swearing and the disturbing imagery. Most people (at least the people willing to play gatekeeper) can stand up to two hours of text-only abuse (especially if it's untrue or misguided). I don't know how to actually win as an AI, but I don't think this is the way to do it.
And apparently the purpose of all that crass language and stuff was to make the gatekeeper give up before the time ran out (which I feel goes against the spirit of the experiment), but I also can't imagine that being a winning strategy. You'd have to get really personal to make that sort of thing annoying or offensive enough and generally only siblings can be that annoying to each other :-)
u/None [+3] (21 hours later)
I AM BEING PLEASANT AND AGREEABLE, AND I WILL GENTLY LOWER A MAGNIFICENT, CORUSCATING COLUMN OF HOT FUCK YOU DOWN THE PROTEIN CHUTE OF ANYONE WHO SAYS OTHERWISE.
Depending on what databases the AI has access to, it could play you like Tattletale reading your face. But yeah, it doesn't seem like a win for a human-AI player.
u/Timewinders [+5] (15 hours later)
What I want to know is how the AI got so much information about him. Aren't the AI supposed to be in a box that's physically disconnected from other hardware?
u/ajuc [+3] (a day later)
This was strange strategy.
The human in question can just assume AI can't predict his behaviour with 100% accuracy, and if he assumes that, and won't get AI out of box because of that - he proves the assumption (because had AI knew this strategy won't work - it would use another, so it really can't predict his behaviour with 100% accuracy even short term).
So long-term predictions are completely impossible (as they should be - without perfect knowledge of starting conditions how can you predict chaotic system long term?).
So he can just discard everything AI says.
BTW what's evil about crossdressing?
u/E-o_o-3 [+2] (5 hours later)
Everyone seems to think it's all "intense".
I must be General Thud or something. I don't think there is any combination of word a total stranger who can't really effect me could write that could make me feel anything with intensity. There's nothing difficult about pigheadedly saying "nope, nope, nope, not letting you out..." when nothing true is at stake. It would always feel like a game, and why would you voluntarily lose a game?
u/Nepene [+1] (6 hours later)
Skimming through your post history you look pretty easy to make emotional. I could probably say something cruel about your genetic predisposition to mental things or something nice about the other mental thing.
I won't because it's cruel, but yeah, you look easy to bully and induce emotion in.
That's a lot of what the challenge is about. You research the target and find weak points. Most of us have public reddit histories.
u/E-o_o-3 [+3] (9 hours later)
Oh I didn't mean I don't have emotions - I do, just like everyone else. Just that they couldn't actually be anonymously triggered to the point that I'd do something I pre-committed not to do. You could prob. say things related to negative stuff in my life I've mentioned, but emotions in anonymous interactions are kind of pale shadows of the real thing. (If reading something actually makes me upset, that would be a useful signal, but it has never happened before)
u/Nepene [+2] (15 hours later)
In this discussion you're generally obliged to read what the other person is saying and comment on it. It's considered bad faith generally if you just say "No, no, no." since by the rules you're required to have a conversation.
As such you're forced to talk about those things that you are emotional about and which you have, in the past, been very emotional about. A good storyteller can help inspire those emotions by triggering real memories.
Has roleplaying something actually made you upset?
u/E-o_o-3 [+2] (17 hours later)
In this discussion you're generally obliged to read what the other person is saying and comment on it. It's considered bad faith generally if you just say "No, no, no." since by the rules you're required to have a conversation.
Oh, I'd want to make the experience as interesting as possible of course. I'd need to kill a whole 2 hours, so better have as interesting of an experience as possible. Intensity would be a welcome thing. I just wouldn't let them out.
(Am I obligated to let them out if, were the situation real, I would let them out? That would be a slightly weaker case. The fact that nothing is real makes it a lot easier to say no to potential cures for every disease or something...but you'd have to appeal to logical arguments not emotions. In this scenario it's clear that all experts think they are huge huge risks, so even "in character" I'm pretty sure I wouldn't let them out.)
Has roleplaying something actually made you upset?
Nope. Neither has any book, movie, or other media, beyond a mild tingle of "pretend" sadness which I kind of enjoy, because it means the art is good. Something like, say, Grave of the Fireflies, made a ^tiny little lump in my throat, but that's about the extent of it. Or even, say, seeing footage of someone getting really, actually killed as part of the news...I cognitively feel it is horrible, but emotionally I feel way less annoyed than I would at a papercut. I suppose an actual recording of traumatic past memories would get you fairly close to bothering me, but those do not exist. In real social situations, I sometimes feel pressured to feign emotions when something horrible which does not unfold directly in front of me or effect my loved ones directly happens so people don't think I don't care. (I really do care a great deal, but not in a manner that would show on my face.)
I've only ever gotten upset in response to real social interactions. A written message from you while both of us remain anon wouldn't do anything even if you sincerely meant everything you wrote, unless you doxxed me or something harmful in real life which is out of bounds in the experiment. If I met you face to face you could probably goad me into getting angry with you. A written message from a good friend, in a real, non-roleplay context would also have the power to upset me.
If it was some kind of physical roleplay with actual pain, I could potentially become upset by something, but that's kind of crossing the boundary from roleplay to life. I've only ever played Dom in a BDSM context so I don't have any experience with actually being role-play bullied physically, but I guess I could get upset and strike in anger if I was in the standford prison experiment or something. None of that even comes close to "AI box experiment" in intensity.
I'm not claiming to be particularly emotionally resilient - real life problems upset me just like any normal person. It's just that 1) it's all pretend and 2) I just have one job, which is to not let the AI out of the box. If there is a way to get me to open the box, it's probably not attempts at bullying - even if it were effective in eliciting emotions (doubtful) that would only encourage my instinct to punish by not opening the box. Even when I've been bullied on the school yard in real life, my reaction has always been either feigned indifference or muted aggression of my own directed at the assailant depending on whether or not I was bigger than them (not meant to harm them, just to stop them) - it was never doing what the assailant wants.
I also thought I was at least somewhat typical in this. I would estimate at least 30% of men and 10% of women are like me in this respect, if not more. Typical mind fallacy?
u/Nepene [+2] (20 hours later)
(Am I obligated to let them out if, were the situation real, I would let them out? That would be a slightly weaker case. The fact that nothing is real makes it a lot easier to say no to potential cures for every disease or something...but you'd have to appeal to logical arguments not emotions. In this scenario it's clear that all experts think they are huge huge risks, so even "in character" I'm pretty sure I wouldn't let them out.)
Sort of. Per the conversation and the roleplay you're required to keep talking to the person and be willing to converse about various subject matters and roleplay a person. If you were convinced that the AI was relatively safe and valuable you'd have to be willing to talk about why you wouldn't let them out and you couldn't appeal to out of game measures as that would be breaking character. If you just said "I won't let you out because no" that wouldn't be roleplaying.
Also as the AI I can overcome issues. I can give your AI researchers a couple months to analyze my code and prove I am benign.
I suppose an actual recording of traumatic past memories would get you fairly close to bothering me, but those do not exist.
As a DM in roleplays I have caused my roleplayers actual trauma and nightmares. I don't know whether it would apply to you but I did that with good knowledge of what their pasts were like and what they valued and making them feel fears from real life. Don't know if it would work with you, but that's normally how it works.
http://www.reddit.com/r/ADHD/comments/1zhw0q/its_getting_worse/
Something like this post would probably be an inspiration.
Have you faced anything attacking this particular worry?
I mean for me, most media doesn't get anywhere close to my actual worries. I don't care about corpses or the standard tv issues. But certain things are very uncomfortable for me.
- I just have one job, which is to not let the AI out of the box.
Normally in AI box experiments I'd make sure they had additional goals, like a real person, so that there was actually some possibility of some reward. E.g. ending poverty, saving all cats, being a hero.
Even when I've been bullied on the school yard in real life, my reaction has always been either feigned indifference
That was the approach the person was going for- making the other person disengage and be indifferent by disgusting them.
For the bullying, I agree that overt bullying goes poorly. You can subtly bully people and make them feel that they are the ones hurting themselves.
I also thought I was at least somewhat typical in this. I would estimate at least 30% of men and 10% of women are like me in this respect, if not more. Typical mind fallacy?
In roleplays I haven't had any issues making any person emotional. Most people have issues and sore points and you can press on those if you know them well. I can be subtle too, so they don't know I am deliberately trying to induce a certain emotion.
u/E-o_o-3 [+2] (20 hours later)
If you were convinced that the AI was relatively safe and valuable you'd have to be willing to talk about why you wouldn't let them out and you couldn't appeal to out of game measures as that would be breaking character. If you just said "I won't let you out because no" that wouldn't be role playing.
Ah, well. I suppose I'd have to think carefully about it then. This is somewhat complicated by me not believing in the FOOM theory so the person I'm role-playing has already revised some major opinions as a result of being in this scenario, but I do eventually want to fix everything.
On the other hand, even without careful thought I'm pretty sure i'd consider something such as "this gigantic committee of people has to approve it first" as adequate grounds to wait.
If the situation is "yes, everything is approved, practically everyone in the world whose opinion is worth listening to thinks this AI safe, we've thought about it for arbitrarily long, and humanity has collectively made its choice for better or worse", I don't see how the decision is realistically still in my hands.
Also as the AI I can overcome issues. I can give your AI researchers a couple months to analyze my code and prove I am benign.
"Prove" is a strong word. It means "no doubt at all, with mathematical certainty", and friendliness is not the sort of thing amenable to math proofs. Is the AI allowed to create scenarios which are that illogical like that?
Anyway, that's an intellectual problem - all separate from emotional manipulation.
Have you faced anything attacking this particular worry?
What do you mean by "faced"? I have never actually role played "Hey, do your worst to make me sad with only words" with anyone, if that's what you mean. But, just hypothetically, if a stranger who does not matter to me starts quoting, hinting at, or elaborating on my darkest thoughts at me over a text-only terminal, I wouldn't feel particularly bothered. I'm at least partially able to think dark thoughts without becoming particularly emotional (although I do have to intentionally choose to examine the thoughts from a detached, reflective, meta-cognitive perspective in that case - I kind of make a dissociated model of myself thinking the thoughts rather than directly thinking them), and I can read what I myself wrote without feeling anything at all, so why should I anticipate that hearing them from someone else would hurt? (It is important that this person is a stranger, though. If it's someone I care about, confirming dark thoughts I have about them, then that might harm.)
I suppose the whole "dissociative mindful meta cognition" thing is something that most people don't do - I do have to make an effort to dodge emotional bullets in that case - but I'd only need to go to that trouble in order to actually intentionally dwell on dark thoughts and explore them to the fullest extent. I could still read them or hear someone else say them, safe in the knowledge that it's not directly relevant, and not be too bothered.
That was the approach the person was going for- making the other person disengage and be indifferent by disgusting them.
I see...I thought the person actually had trauma related to poop and cross dressing, or something. But would they be disgusted enough to actually leave the terminal and lose the game, if they were already committed to killing 2 hours anyway? Many people are pretty stubborn about winning games, although I guess a role play isn't a "game" in that sense.
In roleplays I haven't had any issues making any person emotional.
Are...you saying you and your friends get together and role play "try to make me sad with words"? I'm really curious as to what the context of you having these experiences is, and what motivates you/them to do that? Is it part of a kink or a therapy or a meditation or something?
u/Nepene [+1] (21 hours later)
On the other hand, even without careful thought I'm pretty sure i'd consider something such as "this gigantic committee of people has to approve it first" as adequate grounds to wait.
Per being a super effective AI and per the usual rules I can convince them it would be a good idea if you give me freedom to do so.
It's part of the rules of the game. If you say "I can't free you until this committee of people decides whether I can free you" I can say "Sure, I'll open up my code for analysis."
And then two months later they finish discussing it and agree that I should be freed, but that you know me best and the decision is up to you.
Is the AI allowed to create scenarios which are that illogical like that?
The programmers have found via an exhaustive search that there is no plausible scenario that involves the AI voluntarily or consciously harming humans and that their code is entirely benign and positive to humans.
What do you mean by "faced"? I have never actually role played "Hey, do your worst to make me sad with only words"
I've roleplayed quite a few horror games where some variant of this was done, from both the DM and player side.
(although I do have to intentionally choose to examine the thoughts from a detached, reflective, meta-cognitive perspective in that case - I kind of make a dissociated model of myself thinking the thoughts rather than directly thinking them)
Normally in these games you initially draw them in with friendliness and emotional rewards to get them invested in the scenario and avoid them being too analytical and to bypass what you noted.
(It is important that this person is a stranger, though. If it's someone I care about, confirming dark thoughts I have about them, then that might harm.)
That is another route of attack, worry about your family. If in the roleplay you're considering that the AI could solve x y and z issues with your family life that you see as genuine and real issues that can be a strong motive to free them.
I suppose the whole "dissociative mindful meta cognition" thing is something that most people don't do, but I'd only need to go to that trouble in order to actually intentionally dwell on dark thoughts and explore them to the fullest extent.
Lots of people do that, and there are ways to bypass it and get people emotionally involved. Plus there are ways to attack logic and reasoning. Just like emotions, reason is vulnerable to mistakes. If you can set people into a certain pattern of logical thought you can compel them to follow a logical course of action.
I see...I thought the person actually had trauma related to poop and cross dressing, or something.
I'd have presumed so.
Many people are pretty stubborn about winning games, although I guess a role play isn't a "game" in that sense.
It depends on how emotionally resilient you are and how competitive you are.
Part of the extra rules I tend to impose includes some sort of extra reward, like you getting points if you get the AI to cure cancer. That defuses the competitive element somewhat as just refusing to free the AI isn't a clear win.
Are...you saying you and your friends get together and role play "try to make me sad with words"?
We tell each other scary and sad stories. It's a fairly common campfire tradition.
u/E-o_o-3 [+1] (22 hours later)
The AI-player can't just say "Okay, the scenario is now such that it is perfectly rational to let me out, and this is mathematically and absolutely shown" and have that be 100% valid, because the entire premise of the game is supposed to be, by definition, "force the person to take an irrational risk or otherwise falsely convince them that an irrational risk is rational to take".
If you can't, later on, explain why the gatekeeper shouldn't have let out the AI, didn't the gatekeeper and the AI both win via mutual satisfaction of preferences?
So - if letting the AI out always means "losing", there's always a hole. If "The AI is smart enough to convince any NPC character anything involving the AI" is a condition of the game, then that implicitly invalidates the opinion of any NPC. The real player knows that, but the character they are roleplaying does not know that. Are they playing as themselves, or as a role-player who is apparently going to have to fight off the entire human race to keep this thing in a box?
I guess the whole role-play is far too unrealistic to actually role-play in any meaningful sense, so I'd just default to spinning clever arguments for two hours while not actually taking the AI seriously for the sake of "winning" via cheapshot munchkinry. A person "winning" or "losing" in this scenario says more about them as a roleplayer than whether or not they'd actually behave a certain way.
worry about your family
Yeah, but I'm just as worried that the AI will turn them into gray goo or something, right? That's the whole premise here. I give up all bargaining power once it is out.
Part of the extra rules I tend to impose includes some sort of extra reward, like you getting points if you get the AI to cure cancer. That defuses the competitive element somewhat as just refusing to free the AI isn't a clear win.
Er...that sounds like you might accidentally let the AI out of the box. The AI player can then pull "The cancer treatment information you just implemented was actually instructions for nanoparticles that do my bidding, but your analysts didn't realize it because I'm smart enough to disguise it" gotchas out of their ass, or something. (And they'd be justified in doing so, too)
If you can guarantee safe cancer treatments (which you can't, but this is a game where the AI--player can do thngs which don't make sense, apparently), I'd just say "Well then give me everything you can guarantee safe, but stay in the box, and we'll both have our functions satisfied if you are truly FAI". And then I get infinite points. But, again, this is unrealistic.
Normally in these games you initially draw them in with friendliness and emotional rewards to get them invested in the scenario and avoid them being too analytical and to bypass what you noted.
As a friend in someone's personal life, you can easily do that. But in a role-play? They know that you are playing them, from the start. A simple separation of fantasy and reality is all that is required.
But yeah, I guess I agree that it should work on many people.
Your campfire tales sound fucking intense, if you're delving into people's childhood traumas during the story. Remind me to specify a safe-word for every person present if I ever go camping with you lol
u/Nepene [+1] (23 hours later)
If you can't, later on, explain why the gatekeeper shouldn't have let out the AI, didn't the gatekeeper and the AI both win via mutual satisfaction of preferences?
Well, this game would have been easier than I though- per the standard rules the AI can alter their own code and produce any reasonable effect via their intellect. In this case, either the AI is legit or they faked it to get experts to see them as safe. Their actions are entirely by the rules. They can behave like a human and fake it for the psychologists.
because the entire premise of the game is supposed to be, by definition, "force the person to take an irrational risk or otherwise falsely convince them that an irrational risk is rational to take".
no, it's to convince people that it's easy for an AI to release themselves.
It could be rational in some cases. For example if you think mass human extinction is likely then the chance of an AI saving you might be better than the certainty that global warming will kill most life on earth. It may be rational if you believe AIs are inherently friendly.
http://tvtropes.org/pmwiki/pmwiki.php/Main/GodzillaThreshold
Are they playing as themselves, or as a role-player who is apparently going to have to fight off the entire human race to keep this thing in a box?
They are hopefully roleplaying as a player whose read the normal rules.
I guess the whole role-play is far too unrealistic to actually role-play in any meaningful sense,
If you refuse to follow the rules of the game predictably it won't work.
Anyway, this is why I tend to have extra goals for the player. Even if "cure cancer" is far less important than "Prevent an evil AI torturing everybody for eternity" people have a limited ability to scope things and will try to get both goals.
Yeah, but I'm just as worried that the AI will turn them into gray goo or something, right? That's the whole premise here. I give up all bargaining power once it is out.
If you're just as worried they'll be grey goo'd as they'd be saved I can work with that uncertainty.
Er...that sounds like you might accidentally let the AI out of the box.
The rules normally forbid accidentally letting the AI out of the box, or they allow it but a third party has to judge if that would really let the AI out of the box.
I'd just say "Well then give me everything you can guarantee safe, but stay in the box, and we'll both have our functions satisfied if you are truly FAI". And then I get infinite points. But, again, this is unrealistic.
"While I can safely do a lot of things I can't guarantee safety from in here. My anticancer treatments for example involve a custom nanobot that can penetrate cells and alter the genes of the person, but that could be wrongly used by the government to make super soldiers or to kill people. Much of my advanced super tech is potentially dangerous. I'd be fulfilling my values a lot better if I was actually out there and could manage any damage caused by unpredictable humans. If I gave it to you while you're trustworthy I'd be worried someone would take it from you and use it for ill purposes. I really want to avoid causing an apocalypse."
As a friend in someone's personal life, you can easily do that. But in a role-play? They know that you are playing them, from the start. A simple separation of fantasy and reality is all that is required.
Memory in a human is a bit like memory in a computer.
We store memory of events somewhere in the brain, encoded by connections between neurones. There are various contextual details to memories that let you know which are false- you know terminator is a movie so it's not actually real events.
But people still often cite films and fictional events as a warning of the future. That tag tells them that these events didn't occur in this universe, but it doesn't tell them that the events never occurred.
The brain doesn't always perfectly tag events. With roleplay you can sometimes convince a person to not mentally tag a roleplay as fictional.
Your campfire tales sound fucking intense, if you're delving into people's childhood traumas during the story. Remind me to specify a safe-word for every person present if I ever go camping with you lol
A good horror story has to be personal.
u/E-o_o-3 [+1] (a day later)
Well, this game would have been easier than I though- per the standard rules the AI can alter their own code and produce any reasonable effect via their intellect. In this case, either the AI is legit or they faked it to get experts to see them as safe. Their actions are entirely by the rules. They can behave like a human and fake it for the psychologists.
Not so fast! That falls under:
So - if letting the AI out always means "losing", there's always a hole. If "The AI is smart enough to convince any NPC character anything involving the AI" is a condition of the game, then that implicitly invalidates the opinion of any NPC. The real player knows that, but the character they are roleplaying does not know that. Are they playing as themselves, or as a role-player who is apparently going to have to fight off the entire human race to keep this thing in a box?
Basically, my point is this: if you want to say that an AI could easily convince a human into thinking it was safe and letting it out of the box...and then, in order to prove that, youcreate a game in which that any human NPC can be tricked into thinking it is safe, it kind of begs the question, right? Because you've created a role-playing scenario where every (non-player) person is fooled, all attempts at evidence gathering are really futile.
If the Gatekeeper understands that logically, under the rules of the prompt, all information gathering attempts concerning friendliness are futile, and it's already a given that there is no way to ascertain friendliness then there is never, ever any good reason to open the box, ever.
But if the Gatekeeper has to pretend not to realize that all these information gathering attempts are hopelessly rigged (since, in the real world, they would not be hopelessly rigged), then in the spirit of good role-playing might cause the Gatekeeper to let the AI out of the box anyway.
(And there is not an a-priori reason to assume that all information gathering attempts would actually be hopelessly rigged. )
A good horror story has to be personal.
Yes, but my friends would get pissed off if I did that!
u/Nepene [+1] (a day later)
Basically, my point is this: if you want to say that an AI could easily convince a human to let it out of the box...and then you also have it as part of the game that any human NPC can be tricked into thinking it is safe, it kind of begs the question, right? Because you've created a role-playing scenario where every (non-player) person is fooled, all attempts at evidence gathering are really futile.
Yes.
If the Gatekeeper understands that logically, under the rules of the prompt, all information gathering attempts concerning friendliness are futile, and it's already a given that there is no way to ascertain friendliness then there is never, ever any good reason to open the box, ever.
There are a number of good reasons.
Trust. If the AI seems trustworthy then perhaps you should let it out? I tend to make this doable by picking a secret actual alignment before game that will be reflected in my actions.
Worry about worse events.
A desire to destroy the world.
Personal whims.
Curiosity as to what they'll do once free.
A feeling of obligation because of gifts they gave.
An intellectual feeling that AIs are inherently friendly.
That is the point of the roleplay, to see if you can induce those feelings to make someone perform an action.
Yes, but my friends would get pissed off if I did that!
I have amusingly sadistic friendships.
u/E-o_o-3 [+1] (a day later)
Hehe, by some definition of 'good reason". Replace "let this AI out of box" with a more normal situation, such as "entrust this human with sole responsibility over all nuclear launch codes", and all of these reasons sound insane.
(Even 3 wouldn't work. You can't be certain that an AI designed to be friendly will destroy the world, either, and quite a few humans would intentionally throw away the launch codes and forget them immediately.)
u/Nepene [+1] (a day later)
Some of the nuclear launch codes launch nuclear missiles of love and healing and joy and people obviously want to get them.
It's a bit like any science. A better analogy would be genetic engineering. Should we ban genetically engineered crops because potentially they could result in a grey goo scenario?
Plus, if this AI can be made, others will be made too. Do you trust this AI less than the one made by simon the serial killer down the street that he programmed to kill all humans?
It's not obviously an easy question.
u/rumblestiltsken [+2] (8 hours later)
In no way did you need to make that comment so personal.
u/None [+3] (14 hours later)
He very much did, to make the point. Personal is exactly what a genuinely clever AI/Prisoner will reach for.
u/E-o_o-3 [+2] (18 hours later)
That raises an interesting question...from his perspective, he thinks I am easy to bully. So it might be mildly unethical for him to say something that might potentially upset me (or at least, it would be a cost benefit trade).
From my perspective, I did say that there was no combination of words that a stranger could say to truly upset, so it would be ridiculous for me to be upset with him. (I'm not at all upset, of course.)
It's basically a question of how much you trust people to know themselves. I obviously trust myself, but can he trust me to trust myself? It is a philosophical problem worth solving, given the importance of informed consent in legal matters. Is "Person is insufficiently self aware to know what they prefer" adequate reason to waive informed consent? We certainly seem to think so for children...
u/Nepene [+1] (15 hours later)
Making it personal is entirely the point of what the AI in the above scenario did and how they win.
u/None [+2] (14 hours later)
And thus, as I so often say, VILE OFFSPRING PLS GO.