12 KiB
Love, Interest
-
Author: u/Aretii Cultist of Cthugha*
-
URL: http://justisdevan.tumblr.com/post/161646944789/love-interest
-
Score: 97
-
Created: 2017-06-10T13:28:55
Post:
Comments:
u/eternal-potato [+44] he who vegetates (2 hours later)
I was young. My utility function was stupid.
This is now my default response to this kind of questions.
u/IWantUsToMerge [+5] (22 hours later)
The funny thing is if you have a utility function it means you fundamentally haven't changed, never will, and if your character has changed, it's only because you're in a different context now where your incentives are different and probably not because you're sorry or anything like that.
u/Lord_Drol [+5] NERV (4 days later)
That's not remotely true. First of all: even superintelligences are not omnipotent, and there is nothing stopping events outside ones own control from changing your utility function. Whether that's a bit-flip from a stray cosmic ray, or a more powerful superintelligence twisting you to suit its own goals, or some other reason, it's entirely imaginable that an entity's utility function changes over time. (Against that entity's wishes, of course. Well, against that entity's original wishes.)
But even aside from that: it's flat out wrong that "having a utility function" is in any way an unusual case. The VNM theorem says that any agent who satisfies four axioms has a utility function. Not that they necessarily have an explicit function that they consciously try to maximize, but rather that it is possible to construct a function that their behavior always seeks to maximize.
Now, are you VNM-rational? For instance, are your preferences transitive? If they're not, I can money-pump you for everything you've got. So if you're not money-pumpable, you must necessarily have transitive preferences.
Likewise, all the rest of the VNM axioms are ones that any non-trivially exploitable agent follows. This means you, yes you, have a utility function. Not that you're consistently trying to optimize it, but there is a function out there that you could be accurately modeled as trying your best to maximize.
So don't treat having a utility function as something special that only AIs do. Perhaps only an AI might be aware of the explicit formula for its own utility function, but you, me, and most other humans are utility-maximizers all the same.
u/IWantUsToMerge [+2] (6 days later)
Are there any ways to money pump a human that don't expose the imbalance clearly enough that the slow consistency-finding process wouldn't immediately patch it? I can't think of any. This does not mean that intransitive preferences in humans don't exist, it just means humans intuitively try to fix them where they can.
Maybe if you take a static snapshot of a human's preference ordering, it would be VNM rational. I can't find a way to demonstrate otherwise. However, it's quite obvious that preferences are not stable over time. The evening human buys an irritating alarm clock to undermine the temporary preferences of the morning human. We drink and sleep and expose ourselves to deeply transformative influences without trepidation, we don't genuinely aspire to stability. We subject ourselves to mode shifts we might never come back from, and I'd argue that we do that so frequently that if you found a utility function that implements it, it would boil down to "It's good to implement this state machine". We lack the character of a utility function(if I as someone who is not a practicing mathematician have the right to characterize the utility function construct.. perhaps not). We are state machines that sometimes use utility functions.
I suppose I can't argue on mathematical grounds that such a thing wouldn't be a utility function.
I guess the disagreement is about pragmatics, when I say "utility function", I don't mean the mathematical construct, I'm talking about a machine part. If you see a utility function out in the world it's because someone built it. If a human built it, they probably would have made it behave far more.. temporally consistently.. than any human is. Many of the consequences of that elegant temporal consistency will appear monstrously strange. I wonder if it will seem monstrous enough that future, moderate FAI advocates would argue that too much consistency is bad.
u/Lord_Drol [+1] NERV (8 days later)
Actually, I don't think we disagree too much. True, I was implicitly referring to a "static snapshot", as you put it. The point I was making in the first part of my comment is relevant to both humans and non-humans equally, or perhaps even more so to humans, as we have basically no control over the miasma of chemicals and so forth that our brains are steeped in.
If you're grumpy because you're hungry or tired, that's a good example of "your" utility function changing because of "outside" forces. I say "your", in quotes, because treating the new version of you as a wholly different agent is probably more correct. As for "outside", well, it's physically inside your body, but it's outside your zone of control.
Anyway, the main point of my comment was that things like this apply to all agents. A smart agent tries to minimize this kind of influence, and a superintelligence is probably going to be really good at it (in the same way it's good at everything). We, as humans, are bad at it, as you point out. So in practice, yes. Superintelligences you encounter are going to change over time a lot less than us, simply because they are better optimizers. Still, it's a quantitative difference, not a qualitative one the way implied by your original comment.
I think that's important to remember. But all in all, I do agree with you about the pragmatics of monstrous temporal consistency in superintelligent AIs. But! That's a good thing, a necessary thing, especially for Friendly AI. If we manage to build an FAI, the absolutely last thing we'd want is for it to change its mind! Mindspace is wide enough that a UFAI which changes its mind isn't going to stumble into being Friendly, whereas an FAI which changes its mind most likely won't retain its Friendliness.
Too much consistency is not bad. Our lack of it is probably our greatest failure as human beings. And I absolutely hope that we'll be able to build a consistent FAI in the future, because the only alternative is UFAI.
u/KingMako [+39] Unlicensed Archrationalist (2 hours later)
I like how the ninja is never actually encountered. As expected of a true ninja.
u/TheAtomicOption [+27] (3 hours later)
Someone should write a story that has nothing to do with ninjas but where every so often they just mention that "no one knew if the ninja had been there."
u/696e6372656469626c65 [+3] I think, therefore I am pretentious. (a day later)
"Not even the ninja himself knew if the ninja had been there."
u/noggin-scratcher [+17] I am a happy tree (an hour later)
I am sorry that parts of me are big, and parts are small.
Giggidy
u/Aretii [+15] Cultist of Cthugha (a minute later)
Found it via EY's Facebook, since this was a response to a prompt he tossed out about "the exceptional woman with a surfeit of supernatural lovers."
u/embrodski [+16] (an hour later)
Oh my god!
Abner leaped from his coffin, pirouetted in the air. Andrew caught him in a dip. They both made eye contact with me.
u/The_Magus_199 [+11] Ankh-Morpork City Watch (3 hours later)
Huh. Neat!
...Is it weird that my first response to this story is "She is totally a Heir of Blood"? I mean, a nexus around which bonds change... it fits!
u/GrecklePrime [+12] (an hour later)
That ending had me in stitches. Very cute story.
u/Bobertus [+9] (6 hours later)
The premise has similarities to the Haruhi Suzumiya series.
Until I stopped going to classes.
Because I found a much, much better source of knowledge.
I really expected "source of income".
u/Kishoto [+6] (10 hours later)
Lost it. Perfect ending line. XD
u/SaintPeter74 [+5] (13 hours later)
That was brilliant and engaging. I love post-post modern works where you take a genre or trope apart, expose all the gears, then put it back together and make it work just like the original trope . . . only not. Just brilliant, really.
Also . . . that last line . . .LOL
u/Bowbreaker [+1] Solitary Locust (20 hours later)
I love post-post modern works
Have some great suggestions?
u/SaintPeter74 [+1] (a day later)
I'm not sure what you mean. I'm no author, I just appreciate the form. A lot of rational fic seems to fall into that category for me. Also, Joss Wheadon's work as well, although he primarily subverts tropes.
u/Bowbreaker [+1] Solitary Locust (a day later)
I was hoping for suggestions on short(-ish) stories that hit that note for you because I enjoy that stuff as well but never thought of a way to label it like that. Seeing how you had a cool term for it I thought you knew what you're talking about.
u/Aretii [+1] Cultist of Cthugha (a day later)
Is http://tvtropes.org/pmwiki/pmwiki.php/Main/Reconstruction the thing you're looking for?
u/SaintPeter74 [+1] (2 days later)
Ah, sorry for the confusion. Nothing for short stories comes to mind straight away.
In terms of long form, though, "The Practical Guide to Evil" web serial is pretty good.
Not sci-fi or genre, but "A Heartbreaking Work of Staggering Genius" by David Eggers is pretty amazing. It manages to live up it it's title. It's worth reading in print because he subverts so many conventions that it would be almost impossible to do it well in an e-book. Maybe get a copy at your local library or something. On top of that, it's actually a memoir. In the original print he actually included the phone numbers of all of the principle people so you could call them and confirm the veracity of his story . . . yet it still manages to both deconstruct and reconstruct the memoir.
u/IWantUsToMerge [+4] (22 hours later)
That was nice but you've gotta admit "a flawed fragment of god who loves stories did it" is the strongest possible deus ex machina, perfectly fitted to the task of explaining any nice story.
u/mycroftxxx42 [+3] (14 hours later)
I was smiling when I finished it, and I'm still smiling after reading all 13 comments at this point, so I'm pretty sure that I liked it and it's good.
u/Laborbuch [+1] (a day later)
That was quite a nice story. I was smiling almost all the time I was reading, sometimes with sad undertones, but still smiling. And at one point I laughed (the maxed utility function), which is a compliment of sorts.
Thank you for writing this.
u/Aretii [+3] Cultist of Cthugha (a day later)
Wasn't me wasn't me wasn't me! I just saw EY link to this on his Facebook, had a similar reaction, and then noticed it hadn't been posted here.
u/Laborbuch [+2] (a day later)
Ah, sorry I presumed.
Well, no harm done, just praise falsely attributed :)
u/Schpwuette [+1] (4 days later)
They’d become friends, finally, and baking united their interests. Creativity. Sin. Fire. Joy.
Ahahaha! This is like one of those amazing matches that show up in Unsong...
u/Sailor_Vulcan [+1] Champion of Justice and Reason (8 hours later)
This was very funny. Dont think this counts as rational fic though does it?
u/Aretii [+5] Cultist of Cthugha (10 hours later)
*makes ehhhhh noise while waggling hand*
I see your point, but I felt that it fell close enough to the conceptual cluster to be worth posting here. Much like Unsong, it doesn't really meet the sidebar definition, but it has broad aesthetic similarities, makes use of setting elements rationalfic readers will find appealing, and, possibly most baldly, their existence is linked to persons of interest to the rationalfic community.
u/mcherm [+2] (22 hours later)
I agree. Not rational. Totally belongs here anyway.