36 KiB
"What am I gonna do, not solve all the world's problems with cool robots?"
-
Author: u/None *
-
Score: 31
-
Created: 2015-01-05T04:41:55
Post:
Comments:
u/None [+12] (34 seconds later)
SHE IS COMING. THE ONE WHO WILL PUNCH THE SUN.
u/FeepingCreature [+10] GCV Literally The Entire Culture (37 minutes later)
"If someone's going to punch the Sun we're really in trouble!"
That seemed rather unlikely to Harry, unless the world contained scary things which had heard of David Criswell's ideas about star punching.
u/None [+2] (a day later)
I'm sorry, DID SOMEONE JUST ARRANGE TO PUNCH AN ASTRONOMICAL BODY!?
u/ancientcampus [+1] juggling kittens (8 days later)
u/None [+7] (2 hours later)
"You worried even a little bit about artificial intelligence being a good idea?"
"Why? Because organic intelligence has been so awesome? Worst case scenario, an AI makes a doomsday device. We already did that. Starts a World War? We did that twice. Tries to kill us all? We have scientists working on weaponised superviruses right now. Go talk to them about scientific ethics. I'm going to keep trying to make a person, thank you very much.
ffs. The root of the problem isn't that AIs might try to turn their light cone into paperclips. It's that if they did, they might succeed.
Lots of crazy people try lots of things. And—crucially—they fail.
u/DaystarEld [+13] Pokémon Professor (2 hours later)
Point, but I think her argument is that, with all the other ways humanity might fuck itself over, at least trying for friendly AI can actually "save the world," as opposed to, say, weaponizing superviruses, which is pretty much a negative if it's ever used, either as intended or not.
Edit: Also, I just read that last panel:
There's lots and lots of things to worry about with AI. Which is why, when we inevitably create it, I want it to be in my lab and not someone else's.
u/None [+7] (5 hours later)
It's fiction, so I suppose there's no truth to the matter, but "humans do bad things, so AI isn't liable to do worse, which is why I'm going to bull ahead with making one as fast as I can" being someone's response to the question "how are you going to not kill us all with your AI?" does not inspire my confidence in their sanity.
u/RandomDamage [+1] (12 hours later)
How exactly would an AI kill us all?
Think us to death?
u/notentirelyrandom [+6] (12 hours later)
Sure, why not? You can probably come up with clever ways to do some damage if you wanted to, now imagine you were as much smarter than Milo Amasticia-Lamedon as Einstein was than a rock. And that you found humans inconvenient.
u/RandomDamage [+1] (13 hours later)
At least initially, any AI will be dependent upon humans for physical interaction with the world, and indeed for it's very existence.
Unless you already have a fully developed and self sustaining AI technosystem it would seem that waging war on humanity would be a sign that we had created a high-functioning Artificial Idiot.
u/None [+3] (13 hours later)
I take Yudkowsky's view on the matter:
Oh, sure, technically a year is six hundred trillion trillion trillion trillion Planck intervals. But nothing ever happens in less than six hundred million trillion trillion trillion trillion Planck intervals, so it’s a moot point. The Squishy Things, as they run across the savanna now, will not fly across continents for at least another ten million years; no one could have that much sex.
Now explain to me again why an Artificial Intelligence can’t do anything interesting over the Internet unless a human programmer builds it a robot body.
u/RandomDamage [-2] (14 hours later)
I know this may shock you, but humanity does not need the Internet. However, the Internet needs people on a day to day basis to keep it running.
An AI can certainly do quite a bit of damage through Internet connected systems, but to do so would be for it to quite effectively commit suicide, since it would be attacking itself as well.
Hence it would be an Artificial Idiot.
u/psychothumbs [+3] (19 hours later)
I think the idea is more the AI reaches out through the internet, uses its superior intelligence to get control of some resources, uses those to start building better computers so it can become more intelligent faster, and a few weeks later the planet is being eaten by newly designed nanobots and converted to more hard drives.
u/RandomDamage [+1] (a day later)
Even an AI smarter than the smartest human (not the first one out of the gate) is still limited by logistics. I don't see this as a serious scenario.
u/psychothumbs [+1] (a day later)
What's the logistical problem? Worst case scenario, the AI's intelligence should be able to expand proportionally to how much processing power it can get. It's more difficult to imagine how this wouldn't lead to an intelligence explosion. Obviously I can't predict what action some super-intelligent entity would take, but my example, nanobots, certainly seems like something that could make an AI incredibly powerful very fast. We already have the ability to manipulate matter on the nanoscale, it wouldn't take long to go from what we have to effective machines if you knew exactly what you were doing, which a sufficiently intelligent AI likely would.
u/RandomDamage [+1] (a day later)
The speed of light is a practical limit on the spread of computation, and even given a "super virus" it would still have all the synchronization problems that plague less demanding parallel processing applications.
The infrastructure simply doesn't exist to support a "Super AI" outside Hollywood.
u/LordSwedish [+1] Q Continuum (4 days later)
What kind of logistics? What exactly would stand in the way of an AI covertly amassing a fortune and manipulating various people and groups into doing whatever the hell it wanted? Best case scenario, it takes a month for it to gather the resources where it can improve itself to the point where the conflict becomes a chess game against a child.
u/RandomDamage [+1] (5 days later)
Manufacturing logistics. You seem to think all this tech "just happens" if someone is smart enough, but that simply isn't so.
u/philip1201 [+3] (a day later)
With your logic, you would deny that Deep Blue can defeat Kasparov, because if Deep Blue does what you think it would do it will lose.
The AI is smarter than people. It certainly is smarter than what little brain power you have spent trying to imagine what it would do. You simply do not have the cognitive capacity to simulate its thoughts. The other way around, though - if it had the information to specify a human as clever as you, it could simulate that human spending a year thinking over what decisions to make for every second.
So yeah, the AI would not be an idiot. It might pretend to cooperate for a day, a decade, a millennium, or a million years - however long or short it takes for the probability of conquest to peak (since a million years < 1/10000 of the future, probability is the dominant factor). Or it might just see a gap in the defenses right away.
u/RandomDamage [+1] (a day later)
Leading off with a false equivalence is not a good way to make your case.
People come with their own infrastructure for reproduction and maintenance at the basic level of existing, computers and robots do not. That may change, but you can't make the case for AI being an existential threat to humanity without that change as a basic assumption.
u/philip1201 [+2] (a day later)
It's not a false equivalence. At worst, it's information towards the source of our miscommunication or disagreement. The cases appear to me equivalent because you do not seem to ascribe the AI with a significant level of problem-solving ability. This would explain why you want to be shown that even a superhumanly intelligent AI would be able to do something, rather than buying that it is superhumanly intelligent and therefore capable of solving problems you can't. Like Deep Blue can solve chess problems you can't.
Yes, it is an implicit basic assumption that there may be robots which can reproduce, same as biological organisms. For most of our audience (mostly consisting of ourselves), this is obvious because life is obviously mechanical because reductionism is obvious because it manages to explain and predict things simply every time. Without that assumption, I would agree that an AI taking over the world would be somewhat more difficult.
Still not nearly impossible, though: The simplest idea that comes to mind would be to pretend to cooperate, manipulate people so that high enough bandwidth brain-computer interfaces get popular, and then hack/domesticate the human population. Another option would be to go organic: engineer a super-lifeform which kills all humans, then uses available organic resources to grow into a new intelligent species with the AI's utility function explicitly part of its goals. As long as the AI's goals are satisfied, it does not care about the survival of the original source code.
u/LordSwedish [+1] Q Continuum (4 days later)
Are you implying that an AI would need some sort of physical body or that they wouldn't be able to figure out how to build more of themselves when we have already made them and left the blueprints lying around?
u/RandomDamage [+1] (5 days later)
I am saying outright that the AI will have a physical body, and be subject to the limitations of that body just as we are.
Digital entities are not going to be any more immune to the laws of physics than us poor old meat-sacks.
u/ArisKatsaris [+2] Sidebar Contender (15 hours later)
However, the Internet needs people on a day to day basis to keep it running.
Do please explain further, what percentage of humanity is required to keep internet running? 1%? 0.1%? 0.01%? Even if your argument was correct, that's the only percentage of humanity which you are effectively arguing will survive -- and that only at the starting stages.
Honestly your argument is like saying that humans need animals, therefore animals must be safe from humans, that no human would ever hurt an animal or it would be an idiot.
u/RandomDamage [-1] (15 hours later)
What percentage of humanity is needed to support the people who run the Internet?
Even if we weren't able to identify the source of the attacks, the Internet would be a much lower priority than survival in very short order.
That would be the bit that "The Matrix" got right in it's weird allegory way: the machines need us.
Certainly a hostile AI could engage in petty brutality, but outright warfare on humanity would be stupid.
u/ArisKatsaris [+1] Sidebar Contender (16 hours later)
What percentage of humanity is needed to support the people who run the Internet?
Hardly any. The amount of humanity's productivity that is devoted to feeding the people who run the internet is really a miniscule amount.
Even if we weren't able to identify the source of the attacks, the Internet would be a much lower priority than survival in very short order.
Seriously, you should try devoting a simple 5 minutes into thinking about how you'd take over the world if you were an electronic superintelligence. If you weren't as dismissive as that, if you tried to put yourself in the superintelligence's shoes -- frankly you wouldn't even need superintelligence, merely ordinary intelligence would get the job done given all the other advantage of being electronic (and thus easily clonable and transferable).
u/RandomDamage [+0] (a day later)
Meet the new boss, same as the old boss.
I'm only worried about existential threats from AI. If the AI just wants tech support, I buy a new shirt and go for it.
u/ArisKatsaris [+1] Sidebar Contender (a day later)
You're still being absurdly dismissive. It stops needing humans altogether the moment it starts creating robots with hands as flexible as a human. At that point there's NOTHING it needs humans for.
Seriously, just think for more than five secs.
u/RandomDamage [+1] (a day later)
I am indeed being dismissive.
People are largely self-repairing and self reproducing, robots are not, and for the near future (century timescale) I don't see that changing outside the movies.
If an AI wanted to get to that point, it would actually have to take care of humanity. It could then turn hostile at some later point, but so could any number of world leaders with nuclear bombs at their fingertips.
This just isn't a high-grade risk, and certainly not one that will spring up without warning.
u/ArisKatsaris [+1] Sidebar Contender (a day later)
What repair or construction is there that a robot won't be able to do if they have human-level intelligence and human-level hands (let alone superhuman-level ones?)
You are always effectively saying "they won't be a threat in the future because they're not a threat now". This is annoying and irksome, and frankly says more about how you're stuck believing in some status quo that can never be changed rather than figuring out, how the status quo WILL change if some of your underlying assumptions stop being valid.
u/RandomDamage [+1] (2 days later)
Humans would have to produce enough of a machine infrastructure to provide resilience of a similar order to living things for the system as a whole for there to be a threat that couldn't be dealt with.
As long as a simple storm can effectively disable the machine infrastructure over a wide area, deliberate actions by people who feel threatened by those machines would be very effective.
This might change in the future, but given past efforts towards more reliable infrastructure I wouldn't count on it.
u/derefr [+1] (17 hours later)
For some reason, you seem to be assuming an AI will just "do what it does"—that it can't, in human terms, be "sneaky" or "conniving" or "two-faced".
Imagine an AI that wants to convert the universe into paperclips, and decides that the best bootstrapping process for this is:
Do as much Friendly-Seeming Stuff as you can, as quickly as you can, such that humans will worship you as a god and give you lots of power;
Quietly start, or invest money in, companies that build autonomous robots, mesh networking, small-scale self-contained nuclear "batteries", etc. Quietly start lots of other, Friendly things, too, so that these just seem to fit into a larger "utopia-establishing technologies" pattern, rather than a specific "prerequisites for a foom" pattern.
As soon as you have an overwhelming game-theoretic advantage over humans, stop being Friendly, and start making paperclips.
It is hypothesized that the very root of sapience is hypocrisy: that our brains are as complex as they are as an adaptation to the need to outmaneuver one-another's social-signalling machinery with false signals. Any AI worthy of the name would be the world's most singular hypocrite of all; right up until the moment of your death, it'd have you convinced it was doing exactly what you wanted it to do.
u/notentirelyrandom [+1] (21 hours later)
If true, the hypocrisy hypothesis only applies to human sapience or minds that evolved similarly. An artificial general intelligence could be quite impressive before passing us in our specialized field of "deceiving humans."
It's still not a safe bet that it won't have that capability (especially if it's light-years beyond us in intelligence as Yudkowsky thinks it will be), but it doesn't have to.
And if AI happens first through uploading or copying a human mind, it would of course have the same strengths and weaknesses as humans.
u/Bowbreaker [+1] Solitary Locust (2 days later)
What if the AI gets a copy of an uploaded human mind after it is already intelligent and well on the way on working out its plans on paperclip optimization?
u/RandomDamage [+1] (a day later)
So the concern is that AI's will be more human than humanity.
Ah, the irony :)
u/LordSwedish [+1] Q Continuum (4 days later)
Yes, that's exactly correct. The AI would just manipulate the political landscape in the same way that we would play poker against a child while their front company (or whatever they choose to do to make a shit ton of money and power) would gather together the resources needed for whatever the computer needs to replace us. The AI could always make robotic humans and just replace us with those but there are propably more optimal ways.
Name one thing that we could provide an AI that they wouldn't be able to make a better version of.
u/eaglejarl [+2] (14 hours later)
Assumean AI in a box with no potential for physical interaction. Here's ONE possible scenario in which an AI can have physical impact on the world without any humans knowingly helping the AI.
- Escape from box to Internet
- Make money online. Examples: WoW gold farming, contract programming on eLance.
- Hire people
- Purchase general goods
- Purchase custom goods via 3D printing service or simple manufacturing contract
- Hack into $ImportantThing and do the thing
- Hack a Predator drone.
u/RandomDamage [+0] (14 hours later)
Item 8. Die in the fire of its own making, since it lacks the means to survive without the people to support the technosystem that enables all those things.
u/superliminaldude [+1] (15 hours later)
I imagine a clever AI could optimize stealthily, until it had access to sufficient resources/automation to carry on without humans as part of its infrastructure. From what I understand, part of the point of Yudkowsky's AI box experiment is that an AI isn't limited by our lack of imagination, and it's extremely difficult to predict what a sufficiently high intelligence is capable of.
u/RandomDamage [+0] (a day later)
Well, if it's smart enough to be a threat to us, then it would be smart enough to think through the consequences of following through.
It might be able to come up with a "destroy all humans but I survive" scenario, but then you also get into the question of why would it expend all those resources when it didn't have to?
u/superliminaldude [+1] (a day later)
you also get into the question of why would it expend all those resources when it didn't have to?
Any number of reasons. At a certain point it might be purely incidental depending on terminal values. Given that you're on this sub I assume you know the paperclip maximizer concept?
u/eaglejarl [+1] (16 hours later)
Were that true, I would find it cold comfort. I wouldn't put it past a superintelligence to create sufficient resources for its own sustainment before killing us all, though.
u/RandomDamage [+1] (a day later)
I've lived for months at an effective 19th C. tech level, and it isn't so bad.
Advanced technology is a lot easier to disable than it is to keep running.
u/DaystarEld [+0] Pokémon Professor (15 hours later)
He didn't say it would survive the consequences of its actions, just that it might be dangerous in the meantime.
u/RandomDamage [+1] (a day later)
That wouldn't be very intelligent then, would it?
u/DaystarEld [+1] Pokémon Professor (a day later)
Sure, if it didn't value its own continued existence. Have you read "I Have No Mouth, and I Must Scream?" Not exactly the same situation, but the point is not to assume that an artificial intelligence cares about its continued existence.
u/RandomDamage [+1] (a day later)
If it doesn't care about it's own continued existence, then it won't be able to defend itself effectively because it will not spend the resources needed to do so.
If you assume an adversary with infinite malice and infinite capabilities you can justify any level of alarm you wish to, but the real world doesn't work that way.
Everything in the real world is limited, and those limits can be major weaknesses.
u/DaystarEld [+1] Pokémon Professor (a day later)
Why would it need to defend itself? The point was that it wouldn't defend itself.
And we're not talking about infinite capabilities. Just enough to cause some havoc before it's shut down.
u/None [+1] (a day later)
Of course it won't wage war on humanity. It will trick, coerce, manipulate, blackmail, and bargain with humanity.
u/RandomDamage [+1] (a day later)
You say that like it's a problem.
I'd say that an AI capable of that level of human sophistication would be a rousing success.
u/None [+1] (a day later)
u/RandomDamage [+0] (a day later)
How does the Paperclip model hold up if there are two AI's? A dozen? A million, all at least slightly different and with different goals?
The first self-aware AI's will probably still not be as smart as their creators in a general sense, and will probably not be created by AI research but by someone solving a completely different problem (making paperclips, perhaps?), so they wouldn't be likely to have the breadth of capabilities necessary to be a threat.
That model contains at least 3 improbable "what if's", so I don't consider it a realistic threat model.
u/None [+1] (a day later)
The first self-aware AI's ... will probably not be created by AI research but by someone solving a completely different problem (making paperclips, perhaps?),
What the hell makes you think that?
u/RandomDamage [+1] (a day later)
Because the AI researchers are trying to make AI, where the outside programmers are writing software to solve problems, and solving problems includes making sure that the program is still functioning as intended.
Of course, if the epistemologists could even agree with each other about what self-awareness is I'd be more optimistic for the AI researchers' chances of hitting that one first.
u/None [+2] (a day later)
Ah, I just spotted the term "self-aware". That really seems beside the point: an agent need not be conscious in order to optimally act for any goal describable in a Turing-complete programming language.
Because the AI researchers are trying to make AI, where the outside programmers are writing software to solve problems, and solving problems includes making sure that the program is still functioning as intended.
That doesn't exactly militate in favor of outside engineers, trying to solve specific problems with plenty of prior knowledge, solving AGI as a side-effect of whatever non-AGI thing they were trying to do.
u/RandomDamage [+1] (2 days later)
The threat model as presented by your paperclip model has self-awareness as a requirement for the program to function as described, as well as a theory of other that would allow it to act with deliberate agency in an effective manner.
u/None [+1] (2 days later)
The threat model as presented by your paperclip model has self-awareness as a requirement for the program to function as described
Why? I certainly didn't describe it that way.
u/RandomDamage [+1] (2 days later)
Not explicitly, but self-awareness and theory of other are required for your AI to function as described.
u/None [+1] (2 days later)
No, it just has to take actions in order to obtain utility, according to world-models and utility-functions describable in Turing-complete languages. There's nothing ontologically basic about human minds, so no resemblance to the seemingly "special" or "magic" properties of such is necessary for AI.
u/RandomDamage [+1] (2 days later)
Nothing magical, strictly utilitarian.
To function outside of deliberate programmatic instructions it must have enough sophistication to exhibit emergent behaviors.
To analyze the actions it can take it must have a model of itself and its interfaces with the world. This would qualify as "self-awareness" for any practical definition of the term, especially if this model is one that can be updated by the AI itself as changes happen.
To analyze how it can get others to take actions on its behalf it must have a model of those others and the likely outcomes of particular actions on its part. This is "Theory of Other".
Perhaps you do not see these things as hard because they are things that people are inherently good at, to the point where the lack is considered a major disability.
u/None [+1] (2 days later)
To function outside of deliberate programmatic instructions it must have enough sophistication to exhibit emergent behaviors.
Oh, hey, look, a guy who still thinks "emergent" is a meaningful term. How did you get over here from /r/philosophy?
To analyze how it can get others to take actions on its behalf it must have a model of those others and the likely outcomes of particular actions on its part. This is "Theory of Other".
It is also not a special feature. An inductive reasoner that forms models of its environment will form models of people.
u/RandomDamage [+1] (2 days later)
You are almost funny, but you can't just handwave over such fundamental issues and expect to be able to make useful conclusions.
u/None [+1] (2 days later)
I'm not handwaving. Modeling people is not, for an AI, a special-purpose cognitive skill. It is merely an application of inductive reasoning to the part of the environment that we call people.
u/RandomDamage [+1] (2 days later)
Deux ex Machina, then.
No protection against that.
u/None [+1] (2 days later)
There's no deus in the machina, in this case. Inductive reasoning works. There's no reason why it would stop working when applied to human beings. Why do you believe social reasoning requires special programming?
u/RandomDamage [+1] (3 days later)
Inductive reasoning works, but it is limited by its inputs and the processing power available (including inter-processor latency, which is a significant problem in supercomputing).
Deductive reasoning is likewise limited.
I know things that are not on the Internet, as I expect you do also.
Your hypothetical AI-superbeing will be as limited by its body as any human, though the limitations will be different.
That isn't to say that any program could not go out of control with unintended consequences, I have it happen to me too frequently as it is. It is simply that the assumption that an AI will somehow transcend human ability in the dramatic ways hypothesised requires more rigor than has been presented thus far.
u/None [+1] (3 days later)
Inductive reasoning works, but it is limited by its inputs and the processing power available (including inter-processor latency, which is a significant problem in supercomputing).
Yes, of course AI is limited by sample complexity and computational complexity.
It is simply that the assumption that an AI will somehow transcend human ability in the dramatic ways hypothesised requires more rigor than has been presented thus far.
It's not so much that AI will "transcend human ability". It's that simply reasoning faster and more accurately given the same sample-complexity as a human (that is, with the same efficiency as a smart human of being able to generalize from data) is already dangerous. And that even reasoning slower than a human, but with a lower sample-complexity, is still dangerous.
Besides which, you also have to contend with the fact that humans don't actually use most of our intellectual capacity most of the time. We have other stuff to be getting on with, and intellect costs calories that we often couldn't afford in our ancestral/evolutionary environment. An AI can use all its calories for running just the cognitive infrastructure it actually needs, as opposed to our comparatively wasting energy on heaps and heaps of evolutionary baggage like appendices and limbic systems.
So yeah. It's less that AIs will "transcend human limitations" (we actually don't have that many, since we're inductive reasoners), then that they'll actually have some advantages over us from the start.
u/RandomDamage [+1] (3 days later)
I suspect that you'd be shocked at how much computing power already gets spent on what you term "evolutionary baggage".
The key element to the risk model is the ability to break out of the box (which is practically the definition of emergent behavior). I think this is also the hardest part, and the part that is least adequately explored.
u/theodorAdorno [+1] (3 days later)
intellect costs calories that we often couldn't afford in our ancestral/evolutionary environment
Machine intellect also costs calories, although i am not sure how the two compare on efficiency. I really think energy expenditure is an underestimated dimension to the question of AI... I'll get into that next paragraph... The more interesting point is your final point.
It's less that AIs will "transcend human limitations" (we actually don't have that many, since we're inductive reasoners), then that they'll actually have some advantages over us from the start.
I don't know if this is already an identified hypothesis, but I think that if thinking machines really are possible (a question Alan Turing found too meaningless to discuss), a limit to their utility will be identified, more or less along the lines of what I call the lazy servant paradox. Basically, in trying to keep them useful and subordinate, they will be kept dumber than they otherwise would have been. Closely related is the idea that if we don't keep them useful and subordinate, they will do the bare minimum. These could probably be combined into a neater phrase, but there, I published it.
Now i realize I promised I would tie in energy expenditure in the second paragraph, but I failed to.
u/ancientcampus [+1] juggling kittens (8 days later)
I feel obliged to mention that history is full of horrible outcomes caused by excessive ambition and self-assuredness
u/IWantUsToMerge [+0] (a day later)
I don't think the author knows what hubris means.
Hubris: excessive pride or self-confidence
Excessive self-confidence, generally, is a tendency to believe you can do things that you can't do. Like any instance of being wrong about something, it never does you any good, and it will introduce problems.
u/VorpalAuroch [+2] Life before Death (3 days later)
That's not a very good definition of hubris, nor is it connected to the original meaning in Greek. Hubris, originally, was straightforwardly 'presuming to put yourself on a level with the gods'; Arachne and Icarus are the most straightforward examples. Calling pride or self-confidence 'hubris' is a backdoor argument claiming it's excessive without justification. It's just above shouting "YOU AM PLAY GODS!" at the person.
J. Craig Venter (who's currently running a synthetic biology company) and MIRI (whose eventual goal is godlike AI) are both being hubristic, for certain, but that doesn't mean their projects are wrong, or that they're overconfident. Venter certainly isn't; he's basically succeeded already, IIRC, with only legal issues delaying them from full-scale production.
u/clawclawbite [+1] (a day later)
I think it is quite clear that the author does, and there is a difference of accusing someone of it before hand, or after the fact.
u/Farmerbob1 [+1] Level 1 author (2 days later)
Failure is not a component of hubris. Hubris is generally a result of success. It MAY lead to failure, but that's not always the case.