16 KiB
Dude, you broke the Future! [META]
-
Author: u/Magodo Ankh-Morpork City Watch*
-
URL: https://media.ccc.de/v/34c3-9270-dude_you_broke_the_future#t=431
-
Score: 11
-
Created: 2017-12-30T14:34:48
Post:
Comments:
u/Sailor_Vulcan [+17] Champion of Justice and Reason (an hour later)
btw, this guy is a science fiction author and not an artificial intelligence expert, and he's making arguments based on narrative rather than on actual reason and evidence. I don't know if the quality of his reasoning improves at all later in the video, I stopped watching during his ignorant spiel about how transhumanism is (in his opinion) a religion. Was very disappointing.
u/nicholaslaux [+2] (2 hours later)
I'm a bit confused by your point - how would someone being an "expert in ai" make them more (or less) qualified to talk about philosophical goals?
I largely didn't see any unreasonable jumps of logic in what he was saying, even being someone who by default likely believes what he's dismissing. Narrative analysis is a valid form of analysis, and it seems hard to argue that transhumanism isn't "stories of how you want the future to be".
u/Veedrac [+18] (3 hours later)
Only 10 minutes in but he's already made a bunch of unfounded claims, and clearly doesn't understand the good arguments for AI risk. It feels like the rant of someone who has filter bubble that only shows him the most controversial claims, and he generalises from there.
His argument so far is basically
If something walks like a duck and quacks like a duck, it's probably a duck. And if it looks like a religion, it's probably a religion.
I don't see much evidence for human-like self-directed artificial intelligences coming along any time soon, and a fair bit of evidence that nobody except some freaks in cognitive science departments even want it. I mean, if we invented an AI that was like a human mind, it would do the AI equivalent of sitting on the sofa munching some popcorn and watching the Super Bowl all day. It wouldn't be much use to us.
What we're getting instead is self-optimising tools that defy human comprehension, but are not any more like our kind of intelligence than a Boeing 737 is like a seagull. Boeing 737s and seagulls both fly. Boeing 737s don't lay eggs and shit everywhere. So I'm going to wash my hands of the singularity as a useful explanatory model of the future without further ado.
I'm not sure this needs explicit debunking, but if anyone disagrees I'll be happy to do so.
E: More transcriptions from the 15m mark.
Now, Elon Musk, who I believe you've all heard of, has an obsessive fear of one particular hazard of artificial intelligence, which he conceives of as being a piece of software which functions like a brain in a box, namely, the paperclip maximiser.
A paperclip maximiser is a term of art for a goal-seeking AI that has a single priority, for example, maximising the number of paperclips in the universe. The paperclip maximiser is able to improve itself in pursuit of its goal, but has no ability to vary its goal, so will ultimately attempt to convert all the metallic elements in the solar system into paperclips, even if this is obviously detrimental to the well-being of the humans who set it this goal.
Unfortunately I don't think Musk is paying enough attention. Consider his own company; Tesla isn't a paperclip maximiser, it's a battery maximiser. After all, an electric car is a battery with wheels and seats. SpaceX is an orbital payload maximiser, driving down the cost of space launches in order to encourage more sales for the service it provides.
Solar City is a photovoltaic panel maximiser, and so on. All of three of Musk's very own slow AIs are based on a architecture designed to maximise return on shareholder investment, even if by doing so they cook the planet the shareholders have to live on, or turn the entire thing into solar panels. But hey, if you're Elon Musk that's OK: you're going to retire on Mars anyway.
By the way, I'm ragging on Must in this talk simply because he's the current opinionated tech billionaire who thinks that disrupting a couple of industries entitles him to make headlines. If this was 2007 and my focus was slightly different, I'd be ragging on Steve Jobs, and if my target was 1997, my target would be Bill Gates. Don't take it personally Elon.
u/TotesMessenger [+11] (8 days later)
I'm a bot, bleep, bloop. Someone has linked to this thread from another place on reddit:
- [/r/enoughmuskspam] "Is Musk not allowed to try to save the world?"
^(If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads.) ^(Info ^/ ^Contact)
u/Revisional_Sin [+8] (6 hours later)
All of three of Musk's very own slow AIs are based on a architecture designed to maximise return on shareholder investment, even if by doing so they cook the planet the shareholders have to live on, or turn the entire thing into solar panels.
And this make Stross LESS worried about AI!? Or is Musk not allowed to try to save the world?
Arglglhbbh
u/nicholaslaux [+4] (4 hours later)
Is the debunking that anyone in transhumanism thinks that there would be a human-like AI rather than the self-optimizing tool-like AIs?
I agree that portraying a paperclip maximizer as a "brain in a box" is probably inaccurate, but I thought the rest of the talk where he discusses companies as real-life versions of paperclip maximizers seemed to encompass the risk that people like Musk are warning explicitly about.
I also acknowledge my own political biases, which are largely similar to Stross's and are clearly visible in this talk, are avoided being mentioned, such as every time he discusses Cambridge Analytica, but doesn't mention Civis Analytics (the company that spun out from the Obama campaign's analytics arm, which I'm sure did many similar things) and the like.
But the underlying point of "companies are slow-AI that already have some of the risks we're discussing, and technology is making the fast-AI concerns even more real for them" seems definitely relevant to the discussion and not obviously wrong to me. If there is something more clearly wrong, I'd love to be pointed in the direction of something elaborating why, so I can update my own thoughts of the subject, though.
u/Veedrac [+10] (6 hours later)
It’s hard to call any one issue with his comments on superintelligence definitive, and there are people who think brain uploads are an easier problem than bona fide artificial intelligence, so I would hesitate to point to that issue.
To jump to your next point, in my understanding Musk is worried about AI as “our biggest existential threat”, and seems to have have a very Yudkowskian view of it, rather than view it as a social hazard.
There is a fundamental discontinuity in superintelligence that, if it acts like traditional AI risk advocates suspect, makes any attempt to analogise them to corporations unhelpful. A corporation with misaligned incentives attempts to subvert its political safeguards, whereas a superintelligence with misaligned incentives converts the Earth to computronium.
I don’t consider the argument that “companies are slow-AI” particularly interesting because it serves him as nothing but a label. He never uses it to take insights learned from AI research and apply it to his problem, something you can test by hypothesizing the same talk without using this term and noting that it is basically the same.
Note that whereas my objection to the “companies are slow-AI” analogy is that I don’t feel it is helpful, my objection to his comments on superintelligence are that his claims are simply wrong. I don’t think you’ve asked for clarification here, so I’ll leave it as that, but the offer is still open.
u/696e6372656469626c65 [+4] I think, therefore I am pretentious. (12 hours later)
I actually think there's some merit to the "companies as slow AI" view. Companies are slow AI. The key word there, however, is "slow". Companies behave exactly how you'd expect unaligned AGIs to behave, except much more slowly. One thing that AGIs do is self-improve--that is to say, they create more intelligent versions of themselves. And in fact, companies do this all the time--they're constantly augmenting their capabilities using machine learning techniques. The reason they haven't turned the Earth into computronium yet isn't because they're not "Unfriendly AI"--it's because they're slow Unfriendly AI. The idea that some unsuspecting entrepreneur will one day create a company that creates a genuine, bona-fide artificial superintelligence that converts the planet into computronium is completely consistent with the view that the company itself was a nascent superintelligence--one that eventually self-improved to the point where it destroyed the world. This is, of course, not a reason to worry less about AI alignment, but to worry more.
u/Veedrac [+3] (12 hours later)
Validity and usefulness are two different metrics; I was arguing about the latter. The label has no explanatory power, and it won't change anybody's preexisting beliefs.
u/Izeinwinter [+1] (3 days later)
.. About AIs? Perhaps not, but it is a very useful lens through which to analyse a heck of a lot of problems with the way our economy is structured. Corporate governance becomes a blindingly obvious serious problem once you catch on.
u/Veedrac [+2] (3 days later)
How would you convince someone that corporations act like slow AI if they aren't already convinced that corporate governance is a serious problem? I don't see a path where this tells you anything you didn't already know.
u/thrawnca [+1] Carbon-based biped (a day later)
Doc Future includes an AGI that.
u/CCC_037 [+3] (22 hours later)
there are people who think brain uploads are an easier problem than bona fide artificial intelligence
I think it may well be an easier problem. Developing a bona fide artificial intelligence involves first figuring out what intelligence is, and that's a very hard problem.
Brain uploading merely involves some sort of scanner significantly better than any scanner we can currently create and some sort of simulation software on a computer that's extremely powerful by today's standards. The actual AI itself is then merely copied directly. It won't be easy, but I think it will be less hard than developing true AI.
u/crivtox [+2] Closed Time Loop Enthusiast (a day later)
But in the process of discovering how to scan and upload people and to optimize the software you learn about how minds work in general, which is really useful for true ai.And anyway whether it actually happens first its not that relevant, it's not like you can't make ai after you can upload people and have it destroy the world anyway, uploads are more difficult to make smarter than something better designed.
u/CCC_037 [+1] (a day later)
Yes, brain uploads will be very useful for figuring out how minds work.
u/696e6372656469626c65 [+11] I think, therefore I am pretentious. (4 hours later)
Narrative analysis is a valid form of analysis
When talking about narratives, yes. When talking about real life, not so much.
u/nicholaslaux [+2] (5 hours later)
Depends on what the analysis itself is, and what parts of real life you're discussing. Humans think and communicate in stories, and transhumanism seems very much to be "stories about how we should want our future to go".
The rest of his talk doesn't seem to be relying on narrative analysis to discuss other real life events, unless I missed something obvious. However, with three distinct people seeming to lean that way, I'm wondering if there is a bias that I have that's stopping me from seeing something.
u/Magodo [+3] Ankh-Morpork City Watch (a minute later)
Apologies for the slightly misplaced timeprint. Rewind by 2 minutes
u/Sailor_Vulcan [+1] Champion of Justice and Reason (an hour later)
this video is not buffering properly. is there anywhere else I can watch it?
u/Sailor_Vulcan [+1] Champion of Justice and Reason (an hour later)
nvm i found it
u/Magodo [+3] Ankh-Morpork City Watch (an hour later)
I should have posted this one in the first place
u/-main [+1] (3 days later)
There's also a transcript on his blog, for people who prefer text to video:
http://www.antipope.org/charlie/blog-static/2018/01/dude-you-broke-the-future.html
u/varrant [+1] (an hour later)
There is something interesting about the speed of things in regard to best practices. We've written a lot of code this year, that is likely going to be replaced rather than rewritten, because tech moves forward, which means all the time we've spent on doing it right, so it'll be easy to get into in five years, is essentially wasted.
Not that we're going to abandon best practices, because you never know, but it's somewhat hilarious to think about how much effort we've wasted doing something right that no one will ever notice.
u/nicholaslaux [+2] (5 hours later)
I think a lot less code than you might expect gets replaced, especially when best practices are used. Probably dependent upon the industry that you're looking at, but many industries are very slow to replace code, because starting over from scratch tends to be a very good way to reintroduce a lot of old bugs.
Relevant very old blog post: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/
u/None [+1] (19 hours later)
Isn't it much easier to replace code that's easy-to-read and well documented? And isn't it easier to replace or refit systems composed of well-documented components interacting cleanly in documented ways?
u/Anderkent [+6] (22 hours later)
Yes. This is why good code gets replaced while bad code lives on :p
u/CCC_037 [+1] (22 hours later)
And isn't it easier to replace or refit systems composed of well-documented components interacting cleanly in documented ways?
Significantly. If you have to use a packet sniffer to figure out how the code you're supposed to be replacing is interacting with some other bit of code, then someone has slipped up somewhere.
u/None [+1] (5 days later)
Ok, who tried to Report this out of existence? Fess up!
u/None [+0] (4 hours later)
[removed]
u/-main [+1] (3 days later)
Well, it's not fiction.
Should probably have other tags instead though, even if he does indirectly reference the rationalist community and adjacent communities.