24 KiB
Welcome to Life: the singularity, ruined by lawyers
-
Author: u/xamueljones My arch-enemy is entropy*
-
Score: 72
-
Created: 2017-07-28T17:07:13
Post:
Comments:
u/Frommerman [+15] (an hour later)
Everyone here should watch everything else Tom Scott has produced as well. He's one of my mainstays.
u/DTravers [+3] (23 hours later)
I disagree, I think his "things you didn't know" stuff is, while interesting, far too short in each video.
u/Frommerman [+3] (23 hours later)
He has other things as well. If you want a mix of highbrow, lowbrow, and British humor, I highly recommend Citation Needed.
u/JackStargazer [+12] Primordial Apologist (12 hours later)
I've seen this video before, and as a lawyer I feel I should be mildly offended by the title.
I think this is more the Singularity ruined by Capitalism than lawyers.
Still not as bad as Accellerando though.
u/MineDogger [+6] (an hour later)
Well, that was interesting.
Now I guess I need to figure out how to pirate my own brain...?
Shit... How many terabytes is that gonna take?
u/Empiricist_or_not [+6] Aspiring polite Hegemonizing swarm (10 hours later)
I think the going pre-optimization estimate is 20 petabytes or peta-flops, which is why the Google GPU pod got posted in here. good guesstimates might be here but it's my ethanol night so you only get half assed googles from me.
u/MineDogger [+1] (11 hours later)
"My anus! I don't have that kind of storage capacity!" He exclaimed... flabbergasted
u/IAMSCR00T [+1] (12 hours later)
They did it... Those bastards, they finally did it...
u/CeruleanTresses [+1] (12 hours later)
Too late, they already deleted the criminal thought patterns that contain your inclination to pirate your brain.
u/xamueljones [+12] My arch-enemy is entropy (4 minutes later)
A horrifyingly realistic depiction of the singularity shown in a youtube video.
The publishers of this video will not be held responsible for any issues due to nightmares or underlying medical conditions aggravated by the viewing of this video.
Pregnant women and children under the age of 4 should not watch.
u/TheConstipatedPepsi [+37] (14 minutes later)
You find this realistic? I think any civilisation capable of not only full-brain emulation, but also of searching the emulation for specific content would automatically be able to create a superintelligence in a very short time, at which point things become either a lot worse or a lot better than what this video depicts. It's very hard for me to imagine a scenario in which we have the capabilities implied by the video, but don't almost immediately go far beyond them.
u/clockworktf2 [+15] (47 minutes later)
Very well said, most depictions of civilization at a very high but not yet incomprehensibly high level for a long time are unrealistic for basically this reason.
u/Amonwilde [+8] (2 hours later)
Not sure what you're saying. Would superintelligences abolish capitalism? Would superintelligences just delete stored consciences? I don't think mentioning a superintelligence is any more relevant here than saying, "Well, civilization will probably have collapsed by then." Well, it might have, but so what?
I think the blue sky nature of imagining the far future should make you more, rather than less, tolerant of scenarios like these. In this scenario, maybe there are superintelligences, and maybe they just don't give a shit about uploaded minds. Or maybe they exist, but have executive function deficits that basically mean that humans still call all the shots, but just can do more--kind of like the "A.I." of today. Or maybe they're just modified emulated human minds (like in Saturn's Children) and humans can also add processors to their own minds, so it's a still relatively level playing field. Not hard to think of scenarios where A.I. exists and doesn't really matter, or where uploading is possible but creating A.I. is not. In fact, I'd say the exponential, Elon Musk branded A.I. of our nightmares seems pretty damn unlikely.
u/TheConstipatedPepsi [+12] (3 hours later)
Would superintelligences abolish capitalism?
Well, yeah, unless for some reason capitalism is built-in the utility function of the super-AI. We may be thinking of different things when saying "superintelligence", I think you may be thinking of something like Hal, or at least a really clever computer who knows everything on the internet, I'm thinking of a planet-sized quantum computer who tries to optimise the matter configuration of the universe according to some formally specified utility function.
Would superintelligences just delete stored consciences?
Yes, the human simulations are running on matter that the super-AI could re-purpose for its own goals, if the superAI goals are not well-aligned with ours, it will delete the stored simulations.
My point was that the technology level implied by the video is inherently unstable, a civilisation which is capable of full-brain emulation, yet doesn't advance any further for long enough that the scenario in the video arises is very weird. Full-brain emulation almost trivially produces superAI by simply emulating many AI researcher minds and running them much faster than real-time.
u/Empiricist_or_not [+4] Aspiring polite Hegemonizing swarm (11 hours later)
Paging /u/Datapackrat who I think has a very reasonable transition state between the two written in his SI series of an upload, but I do not seem to have a current bookmark.
@Datapackrat, hey you've thought about this a lot more than I as evidenced by your work, would you please weigh in and link to the series if it's still on docs, or better yet, if finished
u/DataPacRat [+3] Amateur Immortalist (a day later)
The still-incomplete story is at Extracted. Fair warning, I have some significant rewrites in mind for the second half.
As for weighing in, I'll cheat a little and paste a comment I recently posted to a private group:
-----8<-----
Working with analogy on the level of a person from a century or so again being presented with today's fashion and being able to file most of it into "clown suits" but being unable to differentiate finer details...
I currently have a suspicion that early superintelligences will appear to current-day baseline humans as being "well-run corporations". One platform such WRCs might run on might be clans of em-clones who have good enough self-recognition mechanisms to be able to trust with high certainty that their copies possess highly similar values; but that seems likely to be one of the less efficient possible forms, eventually to be shuffled off to niche economic niches (such as, say, having enough concentrated bits of sapience to be able to put up with being spread across areas with significant light-speed lags).
It also occurs to me that the transition from transhuman superintelligence to posthuman superintelligence will mirror AlphaGo's recent progression, as the WRC or WRCs change from making surprising and particularly good tactical choices (which result in baseline humans exclaiming "What a clever strategy! It opens whole new fields of approaching these problems!") to making choices that are incomprehensible even in retrospect (outside of meta-analyses such as "I don't know why that WRC did X, Y, or Z, but I confidently predict that WRC is going to end up better off after having done them than if it hadn't").
Conclusion: When I get back to 'Extracted', I'm going to need to rewrite the behaviour of the em-companies (including the one hosting our protagonist) to match this framework.
And now, laundry!
("Before enlightenment: chop wood, carry water. After enlightenment: chop wood, carry water.")
(To reach enlightenment: go through "The Mind Illuminated: A Complete Meditation Guide to Integrating Buddhist Wisdom and Brain Science", recently mentioned in one of Yudkowsky's facebook posts.)
----->8-----
u/Amonwilde [+3] (4 hours later)
Even in a universe in which problems are being solved by duplicated and networked human intelligences, does that preclude a subscription service (probably low rent) targeted at legacy meatbags? I agree that this would be unstable in the sense that, eventually, you'd have a computronium scenario in which all intelligence became emulated, but it seems at least possible that there would be a period between being able to upload a mind and turning all matter into computronium. Could even be a long period, or even indefinite in some scenarios, such as if it turns out that we don't need matter for increased processing (admittedly unlikely) or that there's some advantage in keeping wetware around (likely, given how much legacy tech is sitting around my office, let alone the rest of the world). And, honestly, would you be shocked if capitalism were part of an A.I.'s utlity function? It's the first thing we'd have them do, and once they were competing with one another they would have to continue using money for the same reeasons that we do. (i.e., we use money because we use money).
u/None [+3] (22 hours later)
Well, yeah, unless for some reason capitalism is built-in the utility function of the super-AI.
Aaaaaand that premise is going in my back pocket for the next time I want to write Fully Automated Gay Space Luxury Communist TTGL.
u/TheConstipatedPepsi [+2] (23 hours later)
Honestly, capitalism is such an abstract idea that it would be extraordinarily difficult to formally specify it in a utility function, I suspect that anyone who can make an AI whose goal is something like "preserve capitalism" can just as easily make an AI whose goal is "do whatever I ought to tell you to do".
u/None [+1] (23 hours later)
I don't think we really know yet which of those English statements is packing more bits of information content.
u/696e6372656469626c65 [+1] I think, therefore I am pretentious. (2 days later)
Between "ought" and "capitalism" (which are the two terms doing the majority of the work in those two sentences) being more complicated, I'm going with "ought".
u/None [+1] (2 days later)
That's only because you understand ought less than capitalism -- or so we think. Capitalism includes a whole lot of ought when it comes down to it.
u/None [+8] (3 hours later)
I think any civilisation capable of not only full-brain emulation, but also of searching the emulation for specific content would automatically be able to create a superintelligence in a very short time,
But they don't want to. At least by my observations, most people in power want to keep our setting, so to speak, exactly the same, for as long a time as possible, whatever technology gets developed.
The biggest reason that a malign superintelligence would develop IRL - at this point, with many experts being aware of possible danger and agreeing that safety is a meaningful concern - is that someone goes rogue and uses a powerful AI to fight the system. The people who are already well-integrated into the system have every incentive to stop anything from happening: they can always shrug and say that humanity is still in control, where by humanity they of course mean themselves.
Sorry, Professor Quirrell, but "you fools, you'll destroy us all" is actually just a rallying cry for clamping down on anyone capable of powerful magic to make sure that the Malfoys' political chess-game isn't disrupted.
u/TheConstipatedPepsi [+7] (12 hours later)
But they don't want to. At least by my observations, most people in power want to keep our setting, so to speak, exactly the same, for as long a time as possible, whatever technology gets developed.
This works if you ignore all the people in power at Google, Facebook, Amazon, Microsoft, etc. who are all desperately dumping money into AI capabilities research. They do want to build super-AI, every civilisation which still has problems to solve wants to build a super-AI which listens to them, because that's basically equivalent to solving every problem. So unless the civilisation implied by the video does not have any more problems, they would still be trying to build ever more capable problem-solving agents.
The biggest reason that a malign superintelligence would develop IRL - at this point, with many experts being aware of possible danger and agreeing that safety is a meaningful concern - is that someone goes rogue and uses a powerful AI to fight the system.
I think you underestimate both the number of experts who don't take superAI concerns seriously and the difficulty increase in building a safe superAI vs. a typical superAI.
u/None [+4] (23 hours later)
This works if you ignore all the people in power at Google, Facebook, Amazon, Microsoft, etc. who are all desperately dumping money into AI capabilities research.
I guess we perceive different things there, and I'm curious to know where you're coming from. From my background and based on my experience, it looks like they basically want to build "AI" into all their products, by which they largely mean supervised statistical learning with deep convnets. In order to do so, they continually overhype the achievements of supervised deep convnets, and more so for unsupervised and reinforcement learning.
They do want to build super-AI
Have they said so?
every civilisation which still has problems to solve wants to build a super-AI which listens to them, because that's basically equivalent to solving every problem.
I don't hear all of civilization saying they want to build a super-AI to solve their problems.
So unless the civilisation implied by the video does not have any more problems, they would still be trying to build ever more capable problem-solving agents.
Or they'd find it a cheaper use of resources to solve their problems the old-fashioned way, as even a superintelligence eventually must.
I think you underestimate both the number of experts who don't take superAI concerns seriously
I might. Have we got numbers?
the difficulty increase in building a safe superAI vs. a typical superAI.
I think we may have a difference of view here. I think there's actually a fairly large gap between the simplest possible "general" intelligence (in the sense of being able to learn any task, given human-scale amounts of data and CPU power, and a cost function specifying the task), the simplest naturally "world-optimizing" general intelligence, and the simplest self-improving world-optimizing general intelligence.
That could just be my personal views and background talking, but it seems to me, just based on what I know, that you need to solve a fresh, fundamental technical problem to ascend each of those steps. At each level before the last, you can stop, hype your shit up, get billions in investment money, and make a long, successful career out of not having a super-AI.
In fact, I think many of the qualitative, important problems rest between the first step (simplest apparently-general intelligence) and the second step (simplest world-optimizing general intelligence). That's actually why I think Singularity hype over deep learning is so thoroughly misguided: not only have people pointed out the flaws in deep learning as machine learning, they've also been spending a long time pointing out its flaws as a theory of how to build a mind able to grip the world and squeeze its own timeline into the small spaces it wants to visit.
Deep learning is going to generate some very interesting industrial applications, including general, trainable-for-anything statistical learners. These will not have the kind of capabilities that someone like MIRI or the AI safety community care about in terms of general intelligence: the ability to use multiple ontologies as appropriate, a utility function with fixed intentional/aboutness content, ability to model itself as embedded in an environment, etc.
u/TheConstipatedPepsi [+1] (a day later)
I guess we perceive different things there, and I'm curious to know where you're coming from. From my background and based on my experience, it looks like they basically want to build "AI" into all their products, by which they largely mean supervised statistical learning with deep convnets. In order to do so, they continually overhype the achievements of supervised deep convnets, and more so for unsupervised and reinforcement learning.
While it is true that they want to build narrow AI into all their products, which at this point is really just deep convnets or RNNs, I don't think that they need to overhype unsupervised learning and RL to do that. Most of the exciting (imo) research is in GANs and deep RL, which don't have that many lucrative applications right now. Deepmind's mission statement is "1.Solve Intelligence 2. Solve Everything else", this is really a company focused on building general Intelligence. When I hear people like Eric Schmidt and Zuckerberg talk about AI, I really think that they believe in its long-term potential, not just in the immediate applications.
Have they said so?
The only reference I can think of is this video in which a bunch of influential people are asked (in the first 3 minutes) whether they would want super-AI to exist at some point. That's not quite equivalent to wanting to build it, so I should probably say that I think everyone ought to want to build a safe super-AI.
I don't hear all of civilization saying they want to build a super-AI to solve their problems.
Fair enough, but that's probably because civilisation is stupid.
Or they'd find it a cheaper use of resources to solve their problems the old-fashioned way, as even a superintelligence eventually must.
I disagree with this, a super-AI would undoubtedly be more resource-efficient than humanity
I might. Have we got numbers?
The most prominent researcher I'm thinking of is Yann LeCun, who takes concerns about social inequality caused by AI seriously, but is not convinced by super-AI worries. The only semi-relevant survey I found was this one, look at section 3.4 on the predicted impacts of human-level AI, a substantial portion of people think human-level AI will have a very good impact.
I think we may have a difference of view here. I think there's actually a fairly large gap between the simplest possible "general" intelligence (in the sense of being able to learn any task, given human-scale amounts of data and CPU power, and a cost function specifying the task), the simplest naturally "world-optimizing" general intelligence, and the simplest self-improving world-optimizing general intelligence.
I kind of agree with the distinction between the first and second, but not with the one between the second and third. Given that you have an AI which can optimise the world, it should also trivially be able to optimise its source code, given that it is part of the world, just imagine a humanoid robot taking actions in the world, one of those actions is to sit at a desk looking at its own source code and doing AI research. I actually think that self-improving AI in some sense will appear long before we have any kind of general AI, there's a relatively small step between something like this and an RL algorithm capable of modifying its own internals.
u/None [+1] (a day later)
Given that you have an AI which can optimise the world, it should also trivially be able to optimise its source code, given that it is part of the world,
The jump there is between a Cartesian dualist modeling of the world, and a "naturalized induction" system. A Cartesian dualist system given access to its own source code will usually muck itself up.
I think it's a short path from second to third, but I'm not sure how short. I can't quite tell whether self-modeling in ordinary causal-model terms is enough to really achieve Vingean reflection, or whether you need to include something explicitly Vingean in the self-modeling to get it right.
u/None [+5] (22 hours later)
Sorry, Professor Quirrell, but "you fools, you'll destroy us all" is actually just a rallying cry for clamping down on anyone capable of powerful magic to make sure that the Malfoys' political chess-game isn't disrupted.
Upon review, this actually explains a lot about Voldemort's actions in HPMoR. He spent a long time trying to maneuver politics and institutions to handle what he considered his long-term self-interest, which sometimes even lined up with the world's long-term interest. This turned out to be incredibly difficult to do with subtlety and precision rather than with massive blunt objects.
So he decides to make a blunt object. Fuck it, he says, I'll make a persona who's a mad, evil villain right out of all the plays and stories. He can give himself an overinflated name, stupid levels of braggadocio, and a mutated face. He can run a stupid fucking cult, preach a made-up ideology that panders to some easy prejudices, and his colorful supervillain lifestyle will ride on the backs of the wealthy and powerful.
And it works. It works better than anything he's ever tried before. The wannabe cultists come to him, in droves! They listen! They fear him! They do what he says! He doesn't need subtlety or precision when he's playing a supervillain. He can literally just kill anyone he wants, for any reason or no reason, and his followers will rationalize to themselves that they need to do better.
So he retires. He becomes the mask, not because it's part of some complicated scheme, but because it's just more fun and works better than all those other things he tried.
Wow. I feel like that's a really appreciable character motivation, actually. The guy basically just got tired of doing things the most difficult way possible and decided to go get an easier job. Loads of people do that.
u/xamueljones [+2] My arch-enemy is entropy (an hour later)
True, I was mostly writing that comment as if it was an ad as a scaremonger to get more viewers at a local movie theater.
u/The_Dar [+1] (19 hours later)
If that sort of copy right law mentioned in the video still exists, then I can imagine this scenario happening in some countries across the world since they all agree to mind there own business (like North Korea and China's Firewall).
Edit: I mean if the AI were expected to solve bureaucracy and tons of other issues forever "almost immediately", wouldn't it be reason for the programmers to be extremely cautious? I think the transition won't be that smooth if we're not to be doomed.
u/None [+1] (3 hours later)
That's it. You fuckers had one job. Now come the legions in gold power armor, the eldritch sigils, the labor shortages, the mass depopulation, and the piece-by-piece tearing apart of this power-mongering sadistic mockery of a civilization. And then, the laughing. The long, long time of laughing.
You had one job. All you had to do was give a shit, and solve your problems yourselves like capable Spirals.
u/Chevron [+2] there is no antimemetics division (10 hours later)
Huh, the whole time I was thinking "This sounds like a more annoying version of Tom Scott's voice", guess his voice was just annoying to me today.
u/hankyusa [+1] Sunshine Regiment (20 hours later)
I think annoyance was an intended response to this video.