25 KiB
How to Distinguish Between a Person and an Optimizing Process
-
Author: u/xamueljones My arch-enemy is entropy*
-
URL: https://www.reddit.com/r/rational/comments/2z7p1o/how_to_distinguish_between_a_person_and_an/
-
Score: 18
-
Created: 2015-03-16T08:18:36
Post:
I'm working on a story where a teacher is teaching her class about the mechanics behind time travel. Time travel in this universe only comes in the form of Stable Time Loops with only one timeline. The current lecture is about how certain Time Loops are selected over other potential Time Loops.
One student has raised the alternate hypothesis that the selection of time loops could be evidence of an intelligent mind rather than an Optimization Process like evolution. Currently the teacher has told the class to discuss reasons for each side.
I already have some good points for why the selection of time loops looks like an Optimization Process thanks to the LessWrong sequence on Evolution, although more tips would be helpful. However, I don't have a good definition for person-hood (or more specifically sapience) which is something that should be present in this world of AIs (not explicitly touched on in this story though). Basically what qualities does a person have, that an automatic process lack?
TL;DR - Given a list of actions, is the agent a person or is it a mechanical optimization process?
EDIT: I changed the TL;DR to be more clear on what I'm looking for.
Comments:
u/None [+44] (5 hours later)
Consult this handy flowchart:
Is it good at its job?
Yes.
Optimizing process.
No.
Person.
u/xamueljones [+8] My arch-enemy is entropy (8 hours later)
Heh. That's funny. I'll include this as a joke from the class clown.
u/None [+17] (42 minutes later)
In this case, "person" is a subtype of "optimizing process", namely, the subtype that happens to be implemented as a real-time, online learning algorithm embedded in the biological or post-biological psychological baggage of a human being, including things like internally distinct emotional valences and consciousness.
u/xamueljones [+2] My arch-enemy is entropy (8 hours later)
Still....how would one be able to distinguish the two solely from a given list of actions? It can be hard to read things like emotions and change in motives from anything other than face-to-face communication.
u/Kishoto [+2] (16 hours later)
As far as I can tell, at a sufficient level of complexity, there won't be a difference. That's the only thing that makes humans human. Complexity. Specifically related to our cognition and sentience.
u/DataPacRat [+6] Amateur Immortalist (11 hours later)
I can't offer a good theory, but in /practice/, the usual criteria for "personhood" that I've been using is something I call the "Trader's Definition": Can I bargain with it? Can I offer to exchange a banana for some backrubs, or some programming for some playtime? This has the advantage that anything that fails the definition, I have the incentive to treat /as if/ it weren't a person, while anything that passes, I have the incentive to treat it /as if/ it were.
For your Time Loops case, if the thing that selects time loops can be negotiated with, then whether it's an Optimization Process or not, then on a practical matter, there's good reason to treat it as if it has an intelligent mind; while if it pursues its agenda without regard to any such offers, then treating it as a non-sapient Optimization Process seems workable.
In short, to decide on which particular dividing line to use, first you should pick what difference the dividing line will make to your actions.
u/TimTravel [+6] (6 hours later)
u/xamueljones [+6] My arch-enemy is entropy (8 hours later)
Wow. You hit the main points of what I'm going for with my version of time travel. Paradoxical universes which are unstable time loops are excluded and their prior probabilities of occurring are "squeezed" into other potential time loops until a stable time loop reaches a higher probability over all other unstable time loops. After that is another step that I want to keep secret for now, because I'm not sure if it would make sense with the rest of the rules.
Still is this a case of great minds thinking alike, or the fact that this is the only sane way (to me) to do stable time loops?
u/TimTravel [+1] (9 hours later)
Iterative single timeline is another simple one but it prevents self-causing events. You simply overwrite the past timeline, or fork a new one. There are major ethical issues in this that are usually ignored.
Another one I came up with involves two dimensions of time, an infinite discrete sequence of timelines with a few added restrictions to ensure convergence. A finite number of timelines have weird stuff in them but the limit provably always exists given the restrictions.
Most attempts of defining it don't actually work but the reasons they don't are subtle. A lot of people fall in the trap of saying "Oh, I got a prophesy to do x, so I'd better do it or something bad will happen!" or "I'd better do it because it's inevitable.", which is not wrong, but if you have a policy of not cooperating with "bad" prophesies then it makes bad prophesies less likely to happen.
u/itisike [+1] Dragon Army (15 hours later)
A simple model which makes cooperating with prophecies rational is where prophecies are made at random and aways come true. So cooperating with one means that the universe doesn't need to manipulate anything to make it come true, which may lead to even worse outcomes while not cooperating won't stop it.
u/TimTravel [+1] (a day later)
That model doesn't fully describe the probability of timelines.
u/itisike [+1] Dragon Army (a day later)
If there's only one timeline, probability isn't part of the territory.
u/TimTravel [+1] (a day later)
If they aren't counterfactual-invariant then they aren't the laws of physics.
u/itisike [+1] Dragon Army (a day later)
Could you rephrase that? I don't see where you're going with that.
u/TimTravel [+1] (a day later)
There has to be such thing as "what if they did this instead", whether the fictional laws are deterministic or probabilistic.
u/itisike [+2] Dragon Army (a day later)
Thanks. So if "this instead" means something that wouldn't make the prophecy come true, then the universe I described would force it to come true, by manipulating something. If something isn't well-defined, you could say that the minimum amount of manipulation required is what happens, for some definition of amount, perhaps involving energy.
u/TimTravel [+1] (a day later)
I'm not clear on the probabilistic mechanism for selecting the way in which paradoxes are averted, and I have absolutely no idea how to define such a mechanism in a deterministic way. The simplest probabilistic method is simply conditioning the (prior) distribution of timelines on there being zero paradoxes, which has the implications I described. I'm not sure what the implications would be of selecting the minimum/maximum energy or minimum/maximum entropy timeline in which no paradoxes happen, but that would be interesting.
u/itisike [+2] Dragon Army (a day later)
What isn't deterministic about my method? Just rank all universes where the prophecy comes true according to some criteria and choose the first.
Your assumption is that the prophecies are causally connected to the future, in a retro-causality manner, but that isn't required for them to always come true.
u/TimTravel [+1] (a day later)
It is deterministic but I have no idea what the implications would be for the set of counterfactual timelines and whether the rules would make cooperation with "bad" prophesies rational or not.
u/itisike [+2] Dragon Army (a day later)
It would mean that prophecies happen regardless of what you plan on doing, so pre-committing not to cooperate with ones you don't like won't lower the probability of them happening.
u/TimTravel [+1] (a day later)
Precommitting might still work. I don't know what it would do to the set of possible timelines. Selecting by energy or entropy might still have the same implications or it might have unpredictable results. I have no idea what it would mean for computationally-unbounded rational behavior in such a universe. It's not clear to me that your selection criteria evade rational precommitment strategies.
u/itisike [+2] Dragon Army (a day later)
If a precommitment is causally unrelated to the mechanism that creates prophecies, then it shouldn't be helpful.
Imagine prophecies as lottery numbers; what kind of pre-commitment would make it more likely to win the lottery in a singleverse? If none, why would one help to get a better prophecy?
u/TimTravel [+1] (a day later)
If the selection mechanism avoids timelines in which you precommit then precommitment is useful. I find it hard to imagine how it might not do so.
u/itisike [+2] Dragon Army (a day later)
It might avoid those timelines, but only after it fixes the prophecy. So it would need to make the prophecy come true even with your precommitment, or prevent you from precommiting. The point here is that it might not be able to stop the prophecy.
u/TimTravel [+1] (a day later)
Oh, I think I get it.
(Probabilistic version) First it fixes a random prophesy (somehow) based on a prior distribution of possible futures, then it conditions random events on that prophesy being eventually fulfilled. That would work. It's not completely clear how you would define the distribution of possible prophesies though.
u/itisike [+2] Dragon Army (a day later)
First it fixes a random prophesy (somehow) based on a prior distribution of possible futures, then it conditions random events on that prophesy being eventually fulfilled.
I'm not even doing that. I'm choosing random properties apart from any simulation of the future. If you used anything from the future, you are acausally interacting with it, and TDT strategies of pre-committing come into play.
I'm thinking of just "choose a random prophecy, perhaps using some cosmic Markov chain, and then make it come true". So absolutely no connection to future timelines, so commitments will not help.
u/TimTravel [+2] (a day later)
I agree completely that this prevents strategies of precommitment from being useful. The only problem is that it doesn't fit my intuition of the prophesy having anything to do with the future. It's more like destiny than prophesy. The Gods decide something is going to happen, and they choose to tell somebody about it, then it happens.
u/itisike [+2] Dragon Army (a day later)
How would you tell whether you're in that world or not? I do think it's possible to have evidence for mine over yours, in which case stories with characters that go along with bad prophecies can be rational.
u/TimTravel [+1] (a day later)
If prophesies of unlikely events are made, it's weak evidence of your model. If people who precommit against all prophesies have the same amount of prophesies told about them as people who do not that would be strong evidence.
u/Chronophilia [+1] sci-fi ≠ futurology (17 hours later)
Any kind of branching or parallel timeline can create the appearance of stable time loops, by having a self-causing event that happens in several "consecutive" timelines with few or no changes. It's very hard for the characters to tell the difference, if time travel is rare and expensive enough that doing controlled experiments isn't an option.
Theoretically, you could use this to retcon a single-stable-timeline setting into a multiple-branching-timeline setting, though I've never seen that done well so it's probably not a good idea.
u/xamueljones [+1] My arch-enemy is entropy (17 hours later)
Actually this is meant to be set in a world where time travel is common enough that the entire story is planned around a series of lectures in a graduate level course on Temporal Mechanics (albeit a course only given in the top, top schools). Therefore, it occurs often enough that people are able to investigate using controlled experiments.
I'm writing the story this way, because I really want to focus on the meat of the story which is investigation of the rules behind time travel without getting distracted by some poorly-written plot. This story is simply to make it easy on me to do all of the world-building in a fun manner and if I ever decide to write a plot using the time travel, then it will be set in the same world. Basically build a sandbox to play around in now, and reap benefits later.
Before people ask the obvious question about why haven't someone created a super-long time loop putting them in control of the entire human history, it's because the length of the time loop is logarithmic with respect to probability of it occurring, or in simple English, loops longer than a few days are about as likely to happen as you winning the lottery and then getting struck by lightening immediately afterwards.
u/Dykster [+5] (8 hours later)
Have a nice comment on chessmasters vs. chess AI: http://www.reddit.com/r/rational/comments/2z5ooe/d_goddamn_do_i_hate_prophecies/cpg6uxs
u/xamueljones [+2] My arch-enemy is entropy (9 hours later)
Okay, this seems useful. Does the unknown agent work by prophecy (via a meta-time) or does it work by a distinct 'style' or strategy? There are subtle ways to guess at both.
Thanks.
u/chaosmosis [+2] and with strange aeons, even death may die (8 hours later)
A person would be able to make leaps from hills and valleys, rather than being forced to use gradual change.
u/DCarrier [+3] (20 hours later)
I'd say the opposite. A person, or any form of computational process with limited resources, will only be able to look for solutions in a small portion of the available space and will do things like make gradual changes. A law of physics that randomly selects one of many solutions to a set of equations can pick out any of all possible solutions.
u/EliezerYudkowsky [+8] Godric Gryffindor (a day later)
Affirm. Natural selection and human intelligence will both have complicated signatures coming from the way they compute things. A law of physics that selects stable possibilities should give rise to stable loops that are (if you end up in a random one) just randomly selected from among all stable loops; there should be no visible signature of how the loop might've been reached, aside from "It's a random stable one."
u/chaosmosis [+2] and with strange aeons, even death may die (a day later)
Who said we were talking about a randomized selection, or law of physics? OP wants to talk about a situation in which a person can be distinguished from an optimizing process, and a situation where a time loop is chosen at random doesn't seem like optimization to me. Presumably all of the time loops have some quality in common which makes the idea of a designer appealing, but which also might make their stability more probable - survival of the fittest.
Suppose time loops can be triggered and there's a set of potential stable time loops set up and one of these loops is optimized along the criteria purported to be the designer's motivations, but requires non-incremental change to reach. Perhaps the time loops tend to maximize the color purple, and a situation is established where the best way for the time line to maximize purple is to first create several other colors in a particular sequence. If the purplemost time loop is the one selected, that's evidence for a designer rather than a gradual optimizing process.
There are non-gradual optimizing processes too, of course. But those reliably can't be distinguished from humans because humans are a subset of them, so I assumed they were irrelevant to what OP wanted to discuss.
u/xamueljones [+2] My arch-enemy is entropy (a day later)
Thanks for your post. What you are detailing explains the arguments for an optimization process that I am planning on for my characters to use, only with the selection criteria being simplicity and the number of future time loops generated (with evidence to back it up).
u/xamueljones [+2] My arch-enemy is entropy (8 hours later)
That seems to be the main difference if I look at it from the theoretical perspective, but I don't quite get how it would be applicable in practice. Maybe by seeing if there are any sudden shifts in patterns to imply a person, and absence of these 'shifts' is weak evidence of an optimization process?
Thanks for helping.
u/PeridexisErrant [+2] put aside fear for courage, and death for life (15 hours later)
This is somewhat complicated by a distinction in what constitutes an "optimisation process". /u/chaosmosis distinguishes local optimisers (eg evolution), but lumps more complex algorithms in with people.
For example: assume a two-dimensional search space with too many points to sample them all, and an unknown utility function over the dimensions. Evolution would pick a random point, then sample adjacent cells and move to the maximal cell (then repeat). A person might systematically sample the space, then try to understand the function and then focus on sampling high-utility areas.
A better dumb optimising algorithm (dumb as in, "I could write this" and "not domain-specific") would be to start sampling in a fractal pattern, and alternate a new fractal-point with local searches from points-above-mean according their proportional goodness.
A smart optimising algorithm would have a superhuman ability to reason about the underlying function, and look like a hyper-competent human (or one with inside information).
u/eaglejarl [+2] (16 hours later)
Evolution would pick a random point, then sample adjacent cells and move to the maximal cell (then repeat).
Which leads to local maxima. Not sure that's relevant here, but it's a point of difference.
u/xamueljones [+2] My arch-enemy is entropy (17 hours later)
Local maxima in the context of a time loop is simply the first possible stable time loop the Optimizer finds which is closest (through hill-climbing) to the original timeline when the time traveler gained access to a time machine without any time traveling assistance (think HPMOR's Final Exam where the Time Turner didn't come into play unless there was a timeline where Harry got access without help).
If it was a better Optimizer, then it would find the global maxima which is a timeline with no time loops, since the Optimizer would be influencing probability to prevent time travel from ever developing. Yes, the Optimizer is selecting for as few time loops as possible.
u/chaosmosis [+1] and with strange aeons, even death may die (a day later)
Yes, the Optimizer is selecting for as few time loops as possible.
I'd just like to point out that minimizers don't really exist in nature, except in service to maximizers. I was seeing your situation as paralleling evolution, but now I'm not sure whether that holds up. It's (more or less) tautologically true that genes which replicate more often become more common, but it's not tautologically true that genes which prevent others from replicating become more common. So it's difficult for me to imagine why an Optimizer such as this might exist.
u/xamueljones [+2] My arch-enemy is entropy (a day later)
I'm slightly confused by your post, but I'll try answering anyway.
The Optimizer is meant to be an inherent feature of the Time Loops where a Stable Time Loop is selected out of a pool of potential Stable Time Loops and the selection criteria is simplicity.
Now the twist here, which I probably stupidly left out of my explanation, is that present Time Loops can be affected by future potential Time Loops. Not only does the Optimizer select a Time Loop if it doesn't have paradoxes, it also checks if that Time Loop will in the future lead to Time Loops which are guaranteed to be unstable. This basically never happens in nature because a guaranteed paradox is nearly impossible to set up, but I have certain experiments in mind to illustrate this property.
The side-effect is that if there is a choice between a simple Stable Time Loop which leads to a very large number of complex Stable Time Loops or a more complex Stable Time Loop which leads to fewer simple Stable Time Loops, then the Optimizer will select for the second choice. But there is a constraint on the complexity of the Stable Time Loop to be allowed (probably scales exponentially). The Optimizer is trading off between the complexity and how common Stable Time Loops can be.
As a result, the earliest time travel researchers were plagued by improbabilities and lethal dangers, because what they would discover were assured of influencing future time loops. As knowledge of time travel spread, the probability manipulations became less extreme since an arbitrary person time looping became less likely to influence how other people approached future time loops.
u/xamueljones [+1] My arch-enemy is entropy (17 hours later)
Okay, thanks for clearing it up for me. I'm not confident in my ability to actually write (and understand) the consequences behind a good optimization process, but your distinction between local optimizers and people is a good one and really useful. I'm intending the Optimization Process behind the time loops to be really simple in terms of complexity, but if I do this properly, it should appear more sophisticated than it really is due to how it can be influenced by people's actions and even their intentions for the future.
u/OffColorCommentary [+2] (a day later)
This is only true of gradient ascent optimizers and similar. There are plenty of non-person optimization processes capable of sudden change. All of the ones we generally deal with in AI are smarter than gradient ascent, but there also exist stupid ones:
State best = random(StateSpace) while (?) // However long we're allowed to execute State proposed = random(StateSpace) if (u(proposed) > u(best)) best = proposed return best;
u/MadScientist14159 [+2] WIP: Sodium Hypochlorite (Rational Bleach) Eventually. Maybe. (12 hours later)
if optimizing for a small number of coherent goals:
optimizing process elif optimizing for a large number of sometimes conflicting goals: person else: alien
u/serge_cell [+1] (a day later)
Necessary (but not sufficient) property of person is internal representation of itself. That is if we have some input 'In' most of learning work like producing representation of In : Rep(state)*In and optimize some min by state Cost(Rep(state) * In, state) .
Person have internal representation of itself RCost(repitself, state) and minimize min by (state,repitself) RCost(Rep(state) * In ,repitself, state) + ||Cost(Rep(state) * oldIn, state) - RCost(Rep(state) * oldIn ,rep, state)|| and use it as initial state for optimization Cost(Rep(state) * In, state)
Just joking :))
u/OffColorCommentary [+1] (a day later)
There are optimization processes dumber than people and optimization processes smarter than people. There are more complicated ones, and simpler ones too. None of these properties are useful in figuring out your question.
People are a very small target within the space of optimization processes. You can look for them the same way you can always look for people; by trying to empathize with them and seeing how poorly that goes. You can make humans angry, or make them laugh. There are possible non-person optimization processes with a sense of humor (the infinite improbability drive might be one), but there are not very many of them.
Note that we as a species took a very long time to realize that weather wasn't a person, and that's not even an optimization problem. We're pretty good at projecting emotions onto things that don't have them.