75 KiB
What, in your opinion, is the most threatening existential risk we currently face?
-
Author: u/696e6372656469626c65 I think, therefore I am pretentious.*
-
URL: https://www.reddit.com/r/rational/comments/2oub3w/what_in_your_opinion_is_the_most_threatening/
-
Score: 14
-
Created: 2014-12-10T06:28:48
Post:
Exactly what it says on the tin. I'm interested in people's opinions here. I imagine that there will be a disproportionate amount of people citing AI, just because, well, this is /r/rational, but I'm fairly confident there will be a significant minority with something else in mind. (Frankly, if there isn't, I'd be worried about echo-chamber issues, even if AI actually is the most threatening thing out there, because really, the likelihood that practically everyone on a subreddit would come to the same conclusion independently is just too small.) I should point out that I myself am uncertain on this topic; AI certainly does seem threatening, but depending on your time estimate before it arrives, it's possible there might be more immediate problems we're facing. I'm just not sure. So... comments?
Comments:
u/xamueljones [+12] My arch-enemy is entropy (3 hours later)
"What's the status of the project?"
"Not very good. I don't understand why all of the civilizations keep self-destructing before they reach Type I."
"Are they self-destructing in the same way?"
"That's the annoying part. I anticipated certain failure points such as the environmental, technological, and biological disasters, but they keep finding a different way to destroy themselves. The latest one failed due to poor social engineering. They ended up making everyone happy and in love with each other. But without any ambition, they stagnated and their birth rates fall so fast that they went extinct within a generation. It's enough to wear on my patience."
"Maybe the problem isn't with your methods, but your efficiency."
"Sir?"
"Instead of trying to anticipate everything, let them learn how to anticipate the future and notice potential failures instead of having them 'naturally' evolve into an advanced society."
"Hmm, that could work. If I let them do the work for me, they would be able to notice and prevent the problems occurring at the micro-scale, allowing me to focus on managing the macro-time scale. That will require teaching them rather advanced skills of thought at an unusually early stage. Maybe if I nudge one of their scientists into disguising the lessons as one of their popular works of literature....."
u/Kishoto [+2] (7 hours later)
Just so we're clear, is this a reference to the Bible?
u/xamueljones [+8] My arch-enemy is entropy (7 hours later)
No. I just guessed that since so many people on this subreddit had read or heard of Harry Potter and the Methods of Rationality, people would get the reference of learning about rationality skills through a rewrite of a popular book. But I probably made it too vague.
The story is meant to say that we need to anticipate a lot of possible ways we can fail, to avoid extinction.
u/None [+3] (8 hours later)
The reference was pretty obvious.
u/Empiricist_or_not [+1] Aspiring polite Hegemonizing swarm (13 hours later)
What's the difference if there is an outside creator?
u/None [+1] (13 hours later)
Huh? Outside creator of what?
u/Empiricist_or_not [+1] Aspiring polite Hegemonizing swarm (13 hours later)
I replied at the wrong level Outside creator Re: reference from the bible.
u/None [+0] (a day later)
Well, to be fair, isn't the less wrong guy (the one who wrote the harry potter fanfic) more of a blogger than a scientist (not that the two are mutually exclusive; plenty of established scientists have blogs)?
The Bible's many authors weren't scientists either, but it's a lot more popular, and therefore more likely a priori to be the subject of an offhand reference. Though perhaps someone like Asimov or Sagan would be even likelier.
u/None [+1] (2 days later)
Well, to be fair, isn't the less wrong guy (the one who wrote the harry potter fanfic) more of a blogger than a scientist (not that the two are mutually exclusive; plenty of established scientists have blogs)?
I'd honestly have to check what /u/EliezerYudkowsky's publication record is at this point.
u/alexanderwales [+17] Time flies like an arrow (an hour later)
Rogue nanotechnology, super disease (engineered or evolved), and nuclear armageddon all get my vote over AI. The thing to worry about with AI isn't just that AI gets developed, it's that:
- AI gets developed
- It's able to recursively self-improve
- It goes rogue
- In a way that we can't stop
I don't even know which of those points to be most skeptical about. When people talk about being afraid of artificial intelligence being developed, it's always the nightmare scenario of an artificial intelligence so advanced that it can talk people into anything or spin up new technologies on the fly. I'm not saying that I don't take it seriously, because I do. But in terms of the actual probabilities ...
Well, look at the other scenarios. As an engineering challenge, self-replicating nanotechnology is tough, but I don't think it's as tough as hard AI. Same goes for engineering a bacteria or virus capable killing large swaths of humanity to the point where we can't bounce back. And nuclear armageddon only needs the wrong person (or people) to get into power.
Maybe I'm just disillusioned by having read The Singularity is Near at an impressionable age, which had some dates that made the future seem a lot nearer than it really was, or maybe I've just worked on too many software projects to have much confidence in one so vastly exceeding expectations.
u/coriolinus [+4] (6 hours later)
What, precisely, would you consider the difference between self-replicating nanotech and genetically engineered bacteria to be?
Grey goo is a classic SF fear, but it doesn't seem particularly plausible; in the end, these things need to power themselves somehow, which significantly reduces the stuff they can realistically eat / self-replicate from. There needs to be an energetic chemical pathway from raw materials to more of themselves, and bacteria have already taken up all the easy examples of those niches.
u/eaglejarl [+5] (9 hours later)
Grey goo doesn't have to eat everything, though. It only needs to eat enough of the energy reserves that we can't sustain sufficient energy-harvesting to maintain civilization. Nanotechnology that ate fossil fuels would be an example.
u/FaceDeer [+1] (18 hours later)
There's an awful lot of alternatives to fossil fuels, though. If it all went poof tomorrow it'd be a major speedbump for our civilization, but hardly a brick wall.
u/alexanderwales [+2] Time flies like an arrow (a day later)
Basically what /u/eaglejarl said. I don't think it's out of the realm of possibility that a man-made nanobot (or engineered bacteria for that matter) would be more effective than something cobbled together by evolution. And they only really need to be good at deconstructing one specific thing - an engineered bacteria that was good at eating through trees would be nearly as deadly to humanity as one that could eat through anything, because it could cause a total ecosystem collapse.
u/comport [+14] (2 hours later)
Here are my current fears, in order of already happening to far fetched:
Economic Ecological Apocalypse
Food instability driven by changing weather patterns, water shortages exacerbated by groundwater pollution and hydroelectric projects, the collapse of industries that rely on cheap consistently available oil, and climate-enabled health crises gradually worsen until all of the small factors combine and the global economy collapses in a flashpoint, marked by superpower proxy wars for control of [rivers, pipelines, shale gas] and civil unrest with an isolationist anti-intellectual theme. This may not be an extinction event (for humans anyway), but it won't be a lot of fun.Global Thermonuclear War / The Fallout Premise
A world ending nuclear war seems far fetched right now, but it would probably only take a generation or so of a government evolving in the wrong direction before it might be willing to threaten its resource rivals (after all, that's all it took to go from the cold war to current global politics). Actually, Putin would only have to be a little bit more insane for us to be there already.Runaway Climate Change
I haven't seen a lot of evidence that runaway climate change can be triggered by a 2C rise, but who thinks that global warming will be restricted to that in the following decades and centuries? If the most pessimistic of climate scientists are right we could end up Venusing the Earth over the next few centuries. I'm not urgently afraid of this, but we're getting into less likely territory.Solar Flare
A big solar flare could knock out unshielded technology on earth. Humanity would probably survive, but I'm not sure civilization could. This could probably be inserted into the list more accurately by looking at how often they happen compared to asteroid impacts, but this is in fear order, not probability order.Asteroid Impact
An oldie, but a goodie. Not particularly exciting or glamorous, but we know it happens, it's happened before, and even if we see the asteroid with humanity's name on it years in advance there's no guarentee we'd be able to generate the technology (or even, a cracked and jaded part of me thinks, the political will) to tractor it off course.Cronus Apocalypse
I'm lumping all of the extinction events where one of humanity's technological children kills its creator, either by bug or malice. Terminator, Matrix, Grey Goo and Paperclip all own less of my fear space than even an asteroid impact - not only would we have to make a lot of mistakes to get there, but we're going into it with our eyes so wide open. There are already a lot of smart people thinking about how to avoid this.Exotic Physics
These are fun ones. We either accidentally generate strange matter or are hit by a roaming quark star, and the whole Earth is converted to strangelets over a period of time. Maybe strangelet production in colliders is the great filter? Fun. Also - a vacuum metastability event. Not necessarily possible or likely, but the untimely end of the entire universe has to get an honorable mention.
u/None [+6] (7 hours later)
I basically agree with your whole list, but there's one thing to note: the Apocalypse Level of the event. FHI, for instance, has a tendency to, in my personal opinion, underrate "Econ-Eco Apocalypse" merely on grounds that it would not actually completely annihilate humanity or technological civilization. I consider this a problematic assessment, because I think there is a chance it would destroy technological civilization, setting us back to a permanent low-productivity, low-energy, nigh-Malthusian existence.
I'm lumping all of the extinction events where one of humanity's technological children kills its creator, either by bug or malice. Terminator, Matrix, Grey Goo and Paperclip all own less of my fear space than even an asteroid impact - not only would we have to make a lot of mistakes to get there, but we're going into it with our eyes so wide open. There are already a lot of smart people thinking about how to avoid this.
The big thing that reduces my worry about UFAI is that even a recursively self-improved superintelligence does have sample complexity and computational complexity bounds it cannot exceed, and the first AGI agents will not be recursively self-improved superintelligences. They won't be able to self-improve without first gathering enough data and performing enough processing on it (call it "education") to form an accurate, naturalistic model of the world that includes itself and includes the necessary understanding to code an improved agent.
That phase will take time and data, lots of it, during which we humans will still have the advantage and probably be able, if the makers have bothered to take decent precautions in the first place (see: entire sub-field of Corrigibility, currently in its infancy), to shut the damn thing off, by force if necessary.
Which isn't to say there's no risk. It's to say that the risk is more on the order of massive radiation spills than on the "INSTA-KILL" level.
u/eaglejarl [+5] (9 hours later)
One additional thing that worries me about civilization collapse: we've used up a lot of the easily available fossil fuel and other resources. Going from our current state to "unlimited" energy (fusion, lots of fission, space based solar) is quite doable. If we had to start technology over from scratch, would we be able to get across the leap from steam to any of those methods, or have we / will we have used up the intermediate energy sources?
u/None [+6] (9 hours later)
And that is exactly what worries me when I think about these issues. Fuck, at the moment I just wish we had the balls as a species and a geopolitical world to actually build out nuclear energy, both for decarbonization and for energy-scarcity reasons and to give us a firm foundation for developing more advanced technologies without the fear of resource collapse.
u/eaglejarl [+2] (10 hours later)
I read an interesting white paper on thorium reactors. If true, it would be the holy grail of energy supply.
u/None [+9] (10 hours later)
Sweet Cthulhu, there was a functioning prototype running safely and efficiently at net energy gain! And those ABSOLUTE FUCKTARDS abandoned the line of research because it was too nonviolent! God fucking damnit!
u/autowikibot [+1] (10 hours later)
#####
######
#### Section 1. Background and brief history of article Thorium-based nuclear power:
After World War II, uranium-based nuclear reactors were built to produce electricity. These were similar to the reactor designs that produced material for nuclear weapons. During that period, the U.S. government also built an experimental molten salt reactor using U-233 fuel, the fissile material created by bombarding thorium with neutrons. The reactor, built at Oak Ridge National Laboratory, operated critical for roughly 15000 hours from 1965 to 1969. In 1968, Nobel laureate and discoverer of Plutonium, Glenn Seaborg, publicly announced to the Atomic Energy Commission, of which he was chairman, that the thorium-based reactor had been successfully developed and tested:
^Interesting: ^Sustainable ^development ^| ^Individual ^and ^political ^action ^on ^climate ^change ^| ^Thorium ^Energy ^Alliance ^| ^Liquid ^fluoride ^thorium ^reactor
^Parent ^commenter ^can [^toggle ^NSFW](/message/compose?to=autowikibot&subject=AutoWikibot NSFW toggle&message=%2Btoggle-nsfw+cmqw7r1) ^or [^delete](/message/compose?to=autowikibot&subject=AutoWikibot Deletion&message=%2Bdelete+cmqw7r1)^. ^Will ^also ^delete ^on ^comment ^score ^of ^-1 ^or ^less. ^| ^(FAQs) ^| ^Mods ^| ^Magic ^Words
u/Empiricist_or_not [+5] Aspiring polite Hegemonizing swarm (13 hours later)
There are two strikes against Thorium reactors in the united states:
It is a metal salt reactor. The united states is a traditionalist mentality in nuclear power due to the cultural horror bias that forces American nuclear power to be very conservative and risk adverse beyond our strong and sensible planning, testing, and design risk aversions. The point being modern U.S. Nuclear power basically is entirely descendant from the legacy of Rickover and proven pressurized water reactors with certain negative feedback coefficients. This culture [basing on myself] shows associates metal salt reactors with contamination due to some of the problems Russians had with their metal salt reactors, and well a thorium reactor is a thorium fluoride reactor, which screams corrosion to me before I look much deeper into the chemistry.
It's a breeder reactor: Look at the treaties on nuclear proliferation. It's pretty hard to differentiate safe fuel producing reactors from producing weapons fissabels, it's basically a difference of concentrations, but the regulation questions makes it very hard and risky on the bureaucrat size, and in an industry where you have to pay government inspectors at ~300 USD a hour to check your designs you have a hard time going with a design that might risk concern against international treaties.
u/None [+1] (10 hours later)
Unfortunately, that's been said about fusion, too, for a very long time. How soon is a functional, net-energy prototype going to be operational, and what are the radiation risks?
("He said, instead of going and looking it up for himself after a long day of work...")
u/eaglejarl [+5] (14 hours later)
They built one already and it worked, but it was killed off in favor of uranium reactors that produced plutonium for bombs.
Thorium reactors produce 2-3 orders of magnitude less waste than uranium reactors and the waste cools off in a few hundred years instead of tens of thousands.
u/None [+1] (a day later)
Excuse the flippant reaction on this subreddit, but... FFFFFFFUUUUUUUUUUUU
u/eaglejarl [+1] (a day later)
blink
What's that for?
u/None [+2] (a day later)
it was killed off in favor of uranium reactors that produced plutonium for bombs.
u/AmeteurOpinions [+1] Finally, everyone was working together. (17 hours later)
u/iemfi [+2] (11 hours later)
But there's still plenty of coal. Also hydro, wind, and geothermal seem easy enough to access with limited technology. And retaining and passing on knowledge seems like it would be a priority for survivors.
u/eaglejarl [+3] (14 hours later)
Yep, agreed on all points. If we were to collapse now, I would have no fear of our ability to restart. But what if we collapse in a century or two, when the easily accessible coal has been exhausted? Hydro and wind are only useful in certain places...
[google]
Hm, actually, hydro seems to have pretty good energy supply -- a quick search says that China produced 721 Tw/hr in 2010. Maybe that would be enough to restart.
Cool, thanks.
u/azripah [+2] (21 hours later)
Energy wouldn't be too bad, really. Solar power generators date to the 19^th century and don't really require any materials that couldn't be obtained with... traditional, let's say, methods.
http://upload.wikimedia.org/wikipedia/commons/6/66/Mouchot1878x.jpg
It's the materials that are derived from petrochemicals that would be annoying to do without. Without even going in to all the materials surrounding you made from plastic, think of all the fertilizer and people supported by that fertilizer that is currently derived from fossil fuel resources. Feeding everyone currently on the planet while lacking the benefits of the green revolution would be taxing to say the least.
u/None [+1] (7 days later)
[deleted]
u/None [+1] (8 days later)
I'm pretty sure one of us is telepathically stealing thoughts from the other.
One of the following questions can be answered from information you have available to you, and the other requires telepathic contact: What is the name of my Stand, and who is best pony?
So, what advice would you give to yourself 10 years ago?
This is going to get pretty rambly. Theme tune go!
Are you actually 15 years old? This matters. LW is already a club of relatively similar people, so we probably are very similar people, but the kinds of things I should say to you vary dramatically based on how old you are, what sort of life situation you're in, and what kind of upbringing you had.
For instance, if I really had to talk to myself 10 years ago, I'd say: I fully understand how screwed-up you feel and how fucked-up the world looks to you, but you really need to take a major upgrade to your self-discipline, your work-ethic, and your ability to fool people you don't like into getting along with you professionally, because you are wasting our precious time. I mean, I got into university at age 17-18, and got in at a pretty good institution, with honors, but there's a hell of a lot I didn't do because I had far too much of a sense of signaling, an overly cynical view of people, an overly simplistic and cynical view of the world, and very little sense of goals besides "try to get by and have some fun while I'm at it."
I came out a pretty ok, intelligent, decently-off person at age 21, but was burned out at 22 and depressed later into 22 and for the early part of 23, made a decision at 23 that kinda messed me up pretty badly in retrospect for 23-25, and God only knows what I'm going to retroactively realize was a total fuck-up next.
So... rambly advice to a younger person who's the general sort of person I used to be...
Most people's models of the world are faaaar too simplistic, as a result of which they believe in stupid bullshit like fate and inevitability. Not only is going beyond the impossible and kicking logic to the curb an awesome way to live, but additionally, the more specific and in-depth knowledge you gain of fields, people, events, everything, the more you realize that history is mostly governed by sheer entropy. Things don't happen because they have to happen, but often because they were just the most likely thing to happen against a background of general unmanaged chance.
It follows that your ability to increase the orderliness of the world, to optimize for outcomes you want, increases super-linearly with marginal increases or improvements in your tools: knowledge, wealth, friends, social skills, privilege of birth, all of it. In this sense, Spiral Power is entirely real. Most people believe the increase is linear: this is because most people on the planet start from such a phenomenally underprivileged place in life that they have to exert large amounts of their optimization power just to stay alive, they've got such a large constant factor to overcome that they never reach the portion of the curve where the returns become superlinear. But if you actually understand what's happening and can do something about it, anything at all, you can usually overpower the sheer, unoptimized entropy that normally runs things. Of course, beware unintended consequences, but that's how reasoning and decision-making always work out...
Almost no-one is evil, almost everything is broken, but most people worship evil gods. They don't think they worship evil gods, they often don't think they worship anything at all, but by and large, they devote their emotions and actions to things that they would not, on reflection, actually live with -- or even teach themselves to stop caring about what actually happens. The Right Thing is the one for which you can reply to "And then what?" with "MISSION. FUCKING. ACCOMPLISHED." And unfortunately, trying to tell most people that their gods are evil will simply land you with accusations of having something wrong with you, so you can't often talk about this stuff without caging the whole conversation first.
There are far more wonderful things and people in Heaven and Earth than are dreamed of in your philosophy. As you get older and find your way to more people who're your kind of people, you are going to find that you're not nearly as alone in the world as you think you are. But then again, you might be older than strictly 10 years younger than me, and hopefully had a much better childhood than I did. Hopefully you've already got the friends and loved ones that took me a long time to get.
If you don't already have enough friends and loved ones, a good criterion is to find people around whom you don't have to hide your powerlevels. You're going to be perpetually bitter and resentful at the world until you stop trying to be normal and go be yourself with someone, anyway. "Being yourself" is not actually about signaling "honestly" and somehow magically getting friends; it's actually about the fact that some things about you are just terminally valued and you don't want to change them, when you consider everything, so fuck it, just live that way and make the friends who like you that way.
Blah blah stay away from drugs, debt, and cults; learn a lot of math and science in school; exercise and eat vegetables blah blah
Now, if we're going to continue this, instead of my rambling nonspecifically, we should make a Life Advice Thread on /r/LessWrongLounge and you should tell me your general life circumstances and goals and such, the better to
exploit you to my own endstell you things that actually help you get what you want.
u/autourbanbot [+1] (8 days later)
Here's the Urban Dictionary definition of Hide Your Power Level :
What (actual) geeks have to do in situations where their nerd knowledge and/or abilities (with computers, video games, internet, et cetera) would come in handy, but using them would reveal to everyone that they are a geek. The phrase itself is taken from Dragon Ball Z. Also extends to anime fans who hide their interest to avoid being labeled as a weeaboo, or weeaboos in a social setting who actually manage to shut the fuck up for once.
1. They had a video game tournament at my college last week that even included Starcraft. Despite how well I know I would have done, I chose to hide my power level in order to keep up appearances. Hide your power level.
2. I was walking through town with friends when we happened upon what appeared to be a cosplayer gathering. Though I could name every character, when my buddy asked what the shit those guys were doing, I hid my power level and told him I had no idea.
^(about) ^| ^(flag for glitch) ^| ^(Summon: urbanbot, what is something?)
u/xkcd_transcriber [+1] (8 days later)
Title: Constructive
Title-text: And what about all the people who won't be able to join the community because they're terrible at making helpful and constructive co-- ... oh.
Stats: This comic has been referenced 106 times, representing 0.2393% of referenced xkcds.
^xkcd.com ^| ^xkcd sub ^| ^Problems/Bugs? ^| ^Statistics ^| ^Stop Replying ^| ^Delete
u/Topher876 [+5] (16 hours later)
I'm not sure if it's what scares me the most but the thing that makes me most angry is a the possibility of an anti-intellectual movement taking over. There are a lot of ways Society could fall apart but to think we could make a conscious decision to abandon our progress just makes me shudder
u/DangerouslyUnstable [+4] (an hour later)
Asteroid strike. We don't do enough monitoring and haven't put enough money into doing something even if we see it. Nukes won't work, especially if we see it late which we probably will.
u/iemfi [+3] (7 hours later)
Well, I'll be the first to say AI by a big margin. The rest either won't result in extinction, are incredibly unlikely, or both. They also have the advantage of being obviously bad. Like if we detected an asteroid on a collision course we wouldn't be worrying about whether it was harmful, we would throw everything at it. We've also had the tech to wreck the Earth for some time and haven't done so. So I don't see why nanotech, super bugs etc, would change that.
u/Oh_Hi_Mark_ [+3] (12 hours later)
Simulation shutdown. There are probably things we could do to cause or avert it, and we'll never know what they were until it's too late.
u/Nepene [+2] (7 hours later)
A solar flare. It would utterly disrupt everything electronic, happens on a regular basis, and is too expensive to cheaply deal with.
An asteroid strike we can cheaply deal with with a gigaton nuke. We program AIs and nanotech, we're probably going to program safeguards.
u/eaglejarl [+2] (9 hours later)
I'm honestly not sure: are we able to build a gigaton nuke? Largest ever was only 50 megatons.
Even if we can, how would we get it there? We currently have nothing capable of launching to beyond LEO. We do still have the plans for a Saturn V -- I think; I've read that they were lost -- but could we build, test, launch, and have it arrive in time?
If the asteroid came from the inner system (e.g. an Apollo), we quite possibly wouldn't see it until it hit us, as the sun would be behind it. Even if it comes from the outer system and were spotted with fifty or a hundred years on the clock, I would be worried about people procrastinating and / or arguing about who should put in how much funding etc that we never actually deal with it in time.
Nuking the asteroid head-on would likely not help -- at most it would break it into smaller chunks with the same mass and they would still hit us. If they were small enough then maybe they would all burn up, but I question the ability of mankind to build and deploy a weapon capable of vaporizing or thoroughly decomposing a 10km chunk of nickel-iron.
If we hit it at an angle then maybe we could deflect it enough that it would miss us, but that would rely on having detected it early enough, which isn't a certainty.
What safeguards would we build on a fast-takeoff AI that would ensure our safety?
What safeguards would we build on nanotech that would ensure our safety?
Assuming such safeguards exist, once nanotechnology / AI is real, it will become cheap enough to be accessible to construction by small groups and corporations. With enough labs building it, someone will be careless or crazy or stupid or evil enough not to incorporate the safeguards.
u/Nepene [+2] (12 hours later)
http://en.wikipedia.org/wiki/Asteroid_impact_avoidance
Following the 1994 Shoemaker-levy 9 comet impacts with Jupiter, Edward Teller proposed to a collective of U.S. and Russian ex-Cold War weapons designers in a 1995 planetary defense workshop meeting at Lawrence Livermore National Laboratory (LLNL), that they collaborate to design a 1 gigaton nuclear explosive device, which would be equivalent to the kinetic energy of a 1 km diameter asteroid. This 1 Gt device would weigh about 25-30 tons being light enough to be lifted on the Energia rocket and it could be used to instantaneously vaporize a 1 km asteroid, divert the paths of extinction event class asteroids (greater than 10 km in diameter) within a few months of short notice, while with 1 year notice, at an interception location no closer than Jupiter, it would also be capable of dealing with the even rarer short period comets which can come out of the Kuiper belt and transit past Earth orbit within 2 years.
You can basically scale up nuclear bombs as much as you like, they just add more weight and are less effective than many smaller bombs at killing cities. As noted, you can do this with far less warning than for most weapons. You'd probably use existing crafts as much as possible, maybe build extra engines if necessary.
All the heat you dump into an asteroid vaporizes the material and is ejected. It causes the asteroid to fly off in a different direction.
What safeguards would we build on a fast-takeoff AI that would ensure our safety?
You'd probably program them with a certain set of ethics and desires that excluded mass death of humans, as appropriate for the application.
What safeguards would we build on nanotech that would ensure our safety?
A kill switch, need for some rare resources to grow, variants of that.
Assuming such safeguards exist, once nanotechnology / AI is real, it will become cheap enough to be accessible to construction by small groups and corporations. With enough labs building it, someone will be careless or crazy or stupid or evil enough not to incorporate the safeguards.
For the most part I'd imagine people would use standardized ones like microsoft or linux or apple, and ones that would resist any insane AIs.
For nanotech, it's fairly cheap to kill it, not as big an issue. Flamethrower kills pretty much any nanotech, emp.
u/eaglejarl [+1] (14 hours later)
Ah, cool. I didn't know about Teller proposition. Thanks for the pointer.
I think you're being a bit casual about AI and nanotech. An AI would be, by definition, self-modifying. Even if you could program in a "certain set of ethics and desires that excluded mass death of humans" how would you ensure that those retained intact across multiple iterations of self-modification. Also, an AI does not need to have "exterminate, exterminate" as its utility function in order to be an extinction risk. Paperclippers (or the equivalent) are a far more probable threat.
Killing nanotech depends on how much of it has generated and where before people become aware and start fighting back. I don't know enough about the subject to speak authoritatively, but "point a flamethrower at it" sounds a bit too casual for my comfort.
u/Nepene [+1] (14 hours later)
You're welcome.
An AI would be, by definition, self-modifying.
Like a computer. Core files would presumably be off limits, so their behavior would mostly be the same. They could run new programs and such, but not modify their underlying key files or hardware. I don't imagine most would want an uncontrollable AI, and it likely wouldn't be very effective.
Also, an AI does not need to have "exterminate, exterminate" as its utility function in order to be an extinction risk. Paperclippers (or the equivalent) are a far more probable threat.
I'd imagine there would be military grade AIs protecting society, a paper clipper likely wouldn't be well evolved to defeat potential threats.
Killing nanotech depends on how much of it has generated and where before people become aware and start fighting back.
It needs appropriate resources like anything. It's not very smart. It can grow like a disease, by infecting new things.
I'd imagine that by the time it was easy to weaponize people would have a variety of defensive nanotech things to oppose a spread. There'd be accidents, but it's not that hard to stop it.
I don't know enough about the subject to speak authoritatively, but "point a flamethrower at it" sounds a bit too casual for my comfort.
Low volume means a high surface area to volume ratio which makes it easy to cook the insides.
u/None [+1] (a day later)
Also, an AI does not need to have "exterminate, exterminate" as its utility function in order to be an extinction risk.
Well yeah, but Daleks are freaking adorable. Don't go telling me I can't have an army of them! Or at least one to guard my house!
u/eaglejarl [+1] (a day later)
Ok, fair enough. Really can't argue with you on this one. ;>
u/Farmerbob1 [+1] Level 1 author (12 hours later)
There is a way that we could fairly trivially deal with almost any size asteroid, by using remote fuel methods to deliver a rocket and fuel to them in order to simply push them a bit so they miss Earth, and then either capture them or push them into a solar intersecting orbit.
It would actually be fairly trivial, and we could do it with existing technology, with minimal engineering. One of the biggest hang-ups we have in space is this painful adherence to wanting to carry all of the fuel required for a mission as a single lump, when we've had the technology for decades that would allow us to remotely fuel vessels in space, outside of orbit, much like in-flight refueling.
The rocket equation tells us we need hugely stupid amounts of fuel to generate lots of delta-v, but the space science community doesn't seem to have picked up on the fact that carrying all the fuel at once is no longer necessary, which in turn throws everything we know about high delta-v missions on it's ear.
u/eaglejarl [+1] (14 hours later)
Perhaps. That's not what Nepene was advocating, though.
Also, this:
If the asteroid came from the inner system (e.g. an Apollo), we quite possibly wouldn't see it until it hit us, as the sun would be behind it.
u/Nepene [+1] (a day later)
We can deal with asteroids, yes. This is dependent on three factors.
When do we spot them? The later we spot them the less well slow but cheap methods work. Your method may be slow.
How big is it? Something really big is going to need a lot of fuel to deal with it or really big bombs.
What do we have available to hit them with? If a specialized craft has to be built this is less likely to happen.
u/Farmerbob1 [+1] Level 1 author (a day later)
Going too much into depth about how we can deal with asteroids in this thread would probably get us loaded with lots of negative karma, so we probably want to keep it brief.
Yes, we have to know they are there to do anything about them, and the farther away they are when we see them, the better.
The larger the detection envelope is, the larger the reaction envelope will be.
The larger the reaction envelope is, the less force will be required to push the asteroid off of a collision trajectory.
The larger the asteroid is, the more fuel will be required to deal with it, but again, the reaction envelope is important. The sooner we act, the less force will be required.
With our current technology, and no space industry, the simplest solution would probably be to rapidly manufacture small, simple rockets to be carried into space by larger rockets. The large rockets then unload their small rocket cargo and the small rockets could then be launched from Earth orbit and carry nuclear warheads to the asteroid.
If humanity ever bothers to actually start a real space industry, our options for dealing with asteroids become far more attractive. In fact, with a strong near-Earth industrial capacity, humanity would probably be more than happy to discover meteors on intercept orbits with Earth, because we could capture them with minimal fuel costs as compared to trying to haul them out of the asteroid belt.
u/Nepene [+1] (2 days later)
People like gorey details, it's fine.
http://www.space.com/19988-asteroid-detection-nasa-private-industry.html
But the numbers get worse as the asteroids get smaller. Scientists have detected less than 30 percent of the 4,700 or so 330-footers (100 m) that come uncomfortably close at some point in their orbits. Such space rocks could destroy an area the size of a state if they slammed into Earth.
We don't yet have the best of detection abilities
With our current technology, and no space industry, the simplest solution would probably be to rapidly manufacture small, simple rockets to be carried into space by larger rockets.
Manufacturing rockets doesn't tend to be a rapid thing. We'd probably do best to adapt existing ones.
u/Farmerbob1 [+1] Level 1 author (2 days later)
Adapting or manufacturing shrug someone's making missiles somewhere in the world right now. We probably don't have anything off the shelf that will work. I know of no small missiles. Though ISS supply rocket upper stages might work.
u/Nepene [+2] (2 days later)
It takes a lot more force to get into space and out of the earth's gravitational field than it does to throw a missile around at a small speed. We'd probably be using Russian rockets. They have a number of useful ones with a lot of lifting power.
u/Farmerbob1 [+1] Level 1 author (2 days later)
I think we talked past each other a bit there. You could use the Russian rocket to carry the little missiles into space. It takes a lot less delta-v to get out of orbit than it does to get off Earth.
u/Nepene [+1] (3 days later)
Ah yeah, probably. I wonder if anyone has done an analysis of whether missiles work well in space- I know several plans like yours, so presumably someone would have done the maths.
u/Farmerbob1 [+1] Level 1 author (3 days later)
It's definitely feasible. The rocket equation can show us that with just a few quick calculations. Whether it's already been organized on standby, or can be made ready in short order, I have no idea.
u/Nepene [+1] (3 days later)
I'd imagine there could be some issues with shielding electrical components from cosmic radiation or the vacuum of space. It'd be worth working out if the missiles would go off correctly. Space is harsh.
u/ianyboo [+2] (8 hours later)
I worry about the small things like the economic upset from Walmart (or whoever) automating its workforce, other companies having to do the same and then massive unemployment, Ferguson style riots, a complete loss of control in america and the rest of the world collapsing with us.
Sort of a Elysium style apocalypse. Not necessarily an extinction event, but enough of a slowdown that we remain a single planet species for an extra hundred years and catch a rogue asteroid for our stupidity.
u/None [+1] (14 hours later)
Beyond a Jobless Recovery speculates on this problem and how to deal with it as a society.
The most promising option is basic income, possibly in conjunction with reduced work weeks, eliminating or reducing minimum wage, and earlier retirement.
The most likely option, I think, is homelessness, reduced lifespans, rioting, and imprisonment, with a sprinkling of charity.
u/None [+1] (a day later)
The one downside is that such a situation can't actually last very long: progress in computation will eventually reach the stage of AGI, so it's not like the world will suffer in some post-scarcity oppression era for too many decades.
u/Chronophilia [+2] sci-fi ≠ futurology (9 hours later)
Nuclear war, certainly.
It seems like all cataclysms have a sweet spot of technological development where they could wipe out the human species. Well, perhaps a "bitter spot" is a better name for it.
Earthquakes or disease are too late - we've been dealing with those for our civilisation's entire history, and we've got good enough infrastructure to handle everything that's been thrown at us so far. Diseases aren't getting more advanced, but healthcare is. Arguably, a species-killing epidemic would have an easier time spreading across nations and continents, since we've got aeroplanes now... but I think it's safe to say that we're better equipped to deal with pandemics than we ever were before.
Nanotechnology and AI are too early - the technology to create them simply doesn't exist, and I find it unlikely that anyone working on any top-secret project has beaten the cutting edge by enough to change that.
Nukes work, they're here now, and the only reason we haven't already been wiped out is that the people with the keys are under a lot of pressure to not use them. If the apocalypse arrived tomorrow, I'd wager that it would be a nuclear one.
u/AmeteurOpinions [+3] Finally, everyone was working together. (10 hours later)
Diseases aren't getting more advanced
Read up on the misuse of antibiotics. We're slowly making strains of bacteria immune to the most common and affordable treatments, which will be a rather expensive problem within our lifetimes.
u/Chronophilia [+2] sci-fi ≠ futurology (10 hours later)
Oh yeah, I forgot about that. Still, my point stands - we've only had antibiotics since 1928, so anything that hasn't killed us in the last million years is unlikely to have become an existential threat in the last 84.
u/notmy2ndopinion [+5] Concent of Saunt Edhar (21 hours later)
I disagree -- to put it colloquially ala Ian Malcolm: "Life finds a way."
You are dismissing illness as the MAJOR THREAT because we've invented some medications? Antibiotics, pesticides and any other wiggly-killer will become outdated as high-replication rates put a selective pressure seeking novel mutations for survival.
We are guiding the hand of evolution.
Frankly, an existential threat I worry about is the next terraforming microorganism that becomes ubiquitous. Precursors to chloroplasts flooded Earth with potent oxidants and poisoned the air, causing massive extinction with OXYGEN. Who know what sort of changes could terraform our planet in terrible ways that we can't escape, even if we come up with gravity equations to float to Saturn? (think Blight from Interstellar, except it is infectious and removes the only source of sustenance and air for a species of consumers, reliant on OTHER creatures for energy.
u/Chronophilia [+1] sci-fi ≠ futurology (21 hours later)
You are dismissing illness as the MAJOR THREAT because we've invented some medications? Antibiotics, pesticides and any other wiggly-killer will become outdated as high-replication rates put a selective pressure seeking novel mutations for survival.
Precisely. Reckless usage of antibiotics will cause new strains of antibiotic-resistant bacteria to emerge... and then we'll only be back where we were before antibiotics were invented. Slightly better off, even, since antibiotic-resistant strains are often less robust in other ways.
And if super-powered insta-kill pandemics were possible, we'd have seen them before, and our species would never have survived to this point. And they'd have still happened if we didn't use antibiotics. Germs don't gain more XP for defeating our countermeasures, they become adapted to the environment they find themselves in (i.e. ones with antibiotics) and less adapted to environments that they're not in (ones without antibiotics).
Compare and contrast with weapons designed by human ingenuity to defeat human ingenuity. MAD has worked so far, but all it takes to break it is one General Ripper to go insane at the right place and the right time. And I don't care for plans that require a large number of unknown people to always act in the sane and sensible manner. People are crazy.
u/Nepene [+2] (a day later)
Our current civilization, due to its interdependence, may be more vulnerable to a disease with a 33-66% death rate in the populace. People who survive may not know how to farm.
u/None [+1] (a day later)
After antibiotics, we'll use bacteriophages. Life finds a way.
u/trifith [+1] Man plans, god laughs. Like the ant and the grasshopper. (12 hours later)
Nukes are the biggest threat at the moment.
The good news is that most of the people who can launch nukes seem to understand that the people they are launching at probably already have a nuke targeted at them. MAD is crazy, but it works.
u/ajuc [+2] (10 hours later)
Biotechnology developing so well, that people will have bioprinters on their desktops. Someone inevitably will produce plague to end all plagues and that will be it.
u/None [+1] (14 hours later)
If by "people" you mean "microbio labs", sure. If you mean private individuals, well, why would I have a bioprinter on my desktop? To extrude exotic flavors of food paste? If I get horribly burnt and want to make my own skin grafts because I'm a DYI type person? It doesn't make sense to me.
u/alexanderwales [+3] Time flies like an arrow (15 hours later)
Depends on the costs, really. If bioprinters can be purchased for thousands of dollars, or even hundreds of thousands of dollars, then that puts them within reach of private individuals, doomsday cults, or terrorist organizations. "People will have bioprinters on their desktops" is a somewhat hyperbolic way of putting it, but low level biofabrication getting cheap is a real threat if the engineering and production side of things gets widespread.
u/ajuc [+1] (15 hours later)
Want to produce some new fashionable drug for the price of milk and sugar? No problem - just download it and hit "print".
All the cool kids in school have elven ears and eyes seeing in darkness. It's just one custom-tailored virus away.
Want to get fit, but don't have time? We know this site where they sell "hasten my metabolism" drugs files.
Why deal with public healthcare and pay monopolist companies, if you can just print everything at home. Never again you will need to go to drugstore.
u/None [+1] (15 hours later)
Want to produce some new fashionable drug for the price of milk and sugar? No problem - just download it and hit "print".
A chemical extruder for arbitrary chemicals would certainly be useful, especially if its input compounds were available as a utility -- if I'm low on boron, the public utility organization (or Amazon Slime) will include it in my weekly delivery. But it's a bit of a distance from simple compounds like cleaning solutions to full working cells.
All the cool kids in school have elven ears and eyes seeing in darkness. It's just one custom-tailored virus away.
That would be a pretty tricky virus to engineer.
Want to get fit, but don't have time? We know this site where they sell "hasten my metabolism" drugs files.
Which is potentially dangerous and should only be done with a prescription.
Never again you will need to go to drugstore.
Sure, thanks to the new drone-based courier system and online ordering.
You're mainly showing why these bioprinters should be (and almost certainly will be) strictly regulated. Of course, with the existential risk issue, that means your plague creator needs to be a microbiologist to access the bioprinter -- which she already needed to be in order to engineer the plague.
u/ajuc [+1] (15 hours later)
Yes I obviously agree that they should be regulated. After all I consider them one path to apocalypse. I just think regulating such things in high-tech future with internet and good 3d printers will be impossible.
u/trifith [+1] Man plans, god laughs. Like the ant and the grasshopper. (15 hours later)
Recreational drug synthesis.
Hell, home pharmacology run off of open source instruction sets could be a thing. Look at 3d printing.
u/Vermora [+2] (16 hours later)
Deranged, determined individuals with a lot of power.
It is currently possible for one person, working independently and alone, to build or purchase a weapon (perhaps a bomb or a gun) and use it to kill a few people, if they are intelligent and determined enough.
As technology develops, the amount of raw physical damage that a person is capable of dealing increases, while humans are not generally getting physically tougher.
It is conceivable (although not necessarily feasible) that in the far future, science and technology will advance to the point that a small group of highly deranged, capable individuals would be capable of dealing catastrophic damage to the human race as a whole.
u/FaceDeer [+3] (18 hours later)
Important to consider in this scenario, though, is the fact that increased technology also allows for people to build new defenses against these sorts of things. If biotech reaches the point where a deranged individual can download a plague kit off of the internet and mix up a horrifying new pandemic, that same technology allows everyone else to download a vaccination kit to neutralize it.
u/MoralRelativity [+1] (15 hours later)
Asteroid or comet impact.
u/OffColorCommentary [+1] (15 hours later)
I think rogue AI is the single biggest risk, as in most likely to actually happen and least likely for us to survive.
Climate collapse seems even more certain, but the brunt of it is rather far out, and it won't be sudden. Even though we as a species seem pretty inept at preventing it, we'll have plenty of time to adapt and deal with it, even if all the actual solutions suck.
Nuclear war is more imminent - it's more likely to happen this year than a runaway AI is - and plenty lethal, but it doesn't seem that likely to actually happen in the long run. Maybe it's more likely this year than it has been in a long time, but we got past the cold war. I think the warnings actually got through to leaders on this one.
Things I see as non-issues:
Antibiotics failing - It will be incredibly awful for people with weak immune systems, and hospital mortality rates will skyrocket, but we won't all die from this. A really big problem we should be concerned about, but not an existential risk.
Asteroid - Vanishingly unlikely during human timescales.
Grey Goo - Runaway self-replicators are an especially bad lab accident, not a world-ending disaster. Nanobots need specific materials, and will slow or stop as they run out of their proper environment. Growing past a puddle will require specialized structures, which we have no reason to program in (unicellular plants didn't make it onto land). Also, despite the stock saying, the worst-case growth rate is quadratic (surface area of a sphere), not exponential.
u/E-o_o-3 [+1] (15 hours later)
My vote goes to things from space damaging the planet or its orbit. If it ends up being our own fault though, it will probably involve big explosives.
That said, there's lots of other horrible things which could go wrong that don't necessarily end humanity forever but do kill most of it - like, say, ecological collapse.
u/scruiser [+1] CYOA (17 hours later)
You use the term "currently" I think AI will start being an existential risk around the time we start getting near human AGI (which I expect will be done with first scan/uploads of human brains). Once we are at the point, then it jumps up to a huge existential risk, but before that point its not a "current" concern. Also, another issue is the difficulty of estimating how close we are to human level AGI. So basically we could go decades without any risk of AGI, and then the first near human AGI is developed, and then the FOOM happens 'quickly'. Its still something we should plan for, and we probably want to solve many of the related problems (i.e. Friendliness) well ahead of time, but I think it is still a difficulty to estimate amount of time away and thus not current.
u/None [+1] (a day later)
(which I expect will be done with first scan/uploads of human brains)
Not to start a massive debate, but that is incorrect as a simple matter of fact. The labs I follow, and DeepMind, and MIRI, are all closer to de novo AGI than anyone is to uploading humans.
u/scruiser [+2] CYOA (a day later)
Not really to start a debate, but could you just give your reasons here? Deepmind is really impressive, but I would still expect them to have a lot of work to do to get to human level AGI. I don't think we know that just extending Deepminds's techniques to a large enough neural network will result in a complete intelligence. MIRI on the other hand... I can see how their work might be useful in the future, but don't think they've yet come up with anything that is even implementable on finite computing resources in finite time.
For mind uploading, the progress has been slow, but I expect it to pick rapidly as the goal gets closer. Right now, the Blue Brain project has simulated the cortical column of a rat. This technique has been shown to be successful, it is just a matter of extending it to more types of neurons, and more neurons. It is still decades of research for doing this with a human, but it will just be a matter of time. I've read estimates claiming 2023 for a human brain. Even allowing for significantly more time, I would still expect them to be working on a human brain by 2030-2040.
If anything, I expect efforts like Deepmind to benefit from the study of biological neurons and neural networks. Principles found in biological neural networks feed into artificial neural network research.
So just tell me why you think DeepMind or MIRI will succeed by 2030-2040. Also, it might be helpful in future discussions if we differentiate how much biological inspiration to artificial neural networks means they are no longer de novo AGI.
u/None [+2] (a day later)
Deepmind is really impressive, but I would still expect them to have a lot of work to do to get to human level AGI.
Because you have AGI loooong before you have human-level AGI. Neural Turing Machines can learn functions of specified complexity (that is, the complexity is a parameter of the learning model, over which the hypothesis space is indexed) from input and output examples already. The question is how you make these highly general learning models efficient, both computationally and in terms of (as I noted elsewhere) sample complexity, and then how you specify the tasks you actually want them to perform, and then coupling the solutions to those problems into "agents" that actually run autonomously or semi-autonomously to perform tasks without human interference.
In my view, something like what MIRI wants to do is a very advanced task within a larger, more general field. "Build an FAI" is more advanced than "build a paperclipper", but a paperclipper isn't actually the stupidest sort of AGI you can build. The stupidest sort of AGI you can build won't even engage in generalized world-optimization. The "general" part is the set of learnable hypotheses being "all Turing-computable functions" or "all Turing-semicomputable environments".
Turning those very general learning models into world-optimizing autonomous decision agents is actually a distinct task from simply making algorithms that can learn very general functions or environments.
You might say that this doesn't sound like AGI very much. That's my point: in order to actually build fully general reasoning machines, we have to be able to dissolve "AGI agent" just like anything else. And it turns out that once we dissolve it, we find that it's mostly easier not to build autonomous world-optimizers than to build them -- you just leave out the part that maximizes a utility function over world-states and replace it with a fairly normal algorithm of the stimuli-reaction mold.
So just tell me why you think DeepMind or MIRI will succeed by 2030-2040. Also, it might be helpful in future discussions if we differentiate how much biological inspiration to artificial neural networks means they are no longer de novo AGI.
Artificial neural networks are, in my books, always de novo AGI. They're not realistic models of the actual brain in their current form -- but they are useful.
u/scruiser [+2] CYOA (a day later)
Artificial neural networks are, in my books, always de novo AGI. They're not realistic models of the actual brain in their current form -- but they are useful.
I think this is the key to our disagreement. I expect attempts at modeling animal and human brains to lead to discoveries that can be directly applied to artificial neural networks. So our first AGI wouldn't be an upload per se, but rather an artificial neural network directly influenced by animal or human neural models.
Anyway I think I understand your view better now so we can have more productive exchanges on this subreddit in the future.
u/FaceDeer [+1] (18 hours later)
What do you mean by "existential"? End of civilization, end of humanity, or end of life on Earth in general?
There is currently nothing that I consider a plausible existential threat toward life on Earth or even to humanity as a species. We're extremely resilient and extremely widespread, we'd easily survive any of the mass extinction events that Earth has experienced in the past half billion years or so. It would take an unprecedented cosmic event to make Earth uninhabitable, an out-of-nowhere gigantic impactor for example. We can't make it sufficiently bad via climate change or nuclear war for humanity to be at risk.
As for ending our civilization, a full-scale nuclear war seems like the most likely plausible way to do it in the near term. I don't consider climate change that big a threat because it happens slowly enough that our civilization will adapt, even if it degrades us somewhat.
I don't consider AI to be a near term threat either. Long term, maybe, but only insofar as it will render humanity increasingly obsolescent rather than having a computer wake up one day and decide to Destroy All Humans. It's certainly not something we currently face, though.
u/BekenBoundaryDispute [+1] (20 hours later)
I'm thinking that most of the conventional anthropogenic scenarios lead to about the same place: a resource-depletion foom that leads to proto-industrial technological stasis.
- It could just happen on its own; consumption is always going to overtake production if you don't get into space, and it will at the very least radically change the nature of modern economics as we know it (which will make solving the problem harder).
- Nuclear wars might very well be "survivable", in the sense that in spite of most of the world dying, a significant population could probably scrounge together micronutrients/food whatever for a few decades (it's far from unprecedented, especially with intelligent actors). By the time anthropogenic x-risk becomes a problem again, a foom is just that more likely.
- Pandemics are similar to nuclear war. Even a particularly virulent one would still leave founder populations (e.g. North Sentinel Island, the highlands of Papua New Guinea, possibly paranoid survivalist compounds and large ships if the incubation is short enough).
- A somewhat strange, but remotely plausible possibility is a economy-smashing demographic or political crisis. This isn't just stock "Eurabia" rambling, there are lots of potential vectors. Japan is barely holding together very deeply ingrained social structures. I have no idea what's going to happen if Putin dies (or slowly goes senile). Mexico could go up in (even more) flames. China's rapid development will slow down, and has a ceiling due to the inherent constraints of an authoritarian culture on ability to train hackers and academics. The Quiverfull movement could catch on somewhere. All of these only take one or two 9/11-scale political upheavels to trigger.
- Climate change, anthropogenic or otherwise, is always on the table, but I think the timescales and effects involved make this unlikely to occur before one of the other outcomes.
Both the transhuman program and space expansion need a lot of financial and political capital and raw materials. This might not be feasible with the resources and ideologies that would be left after one of the preceding events.
What comes after this depends on the specific circumstances, but given that it will probably settle on an eventual steady state of manual-labor-intensive agriculture, it does not take much imagination to see how it could be terrifying.
u/None [+1] (a day later)
This isn't just stock "Eurabia" rambling
There's also, you know, Da3esh. They appear to have averted the whole "Eurabia" deal by starting what is quite nearly a third World War in the Middle East. It's only our luck that the rest of the world is already tired of dealing with our region's shit.
Of course, in a weird way, Da3esh's war in the Middle East has the upside consequences that the rest of the world is finally making real moves to get off oil, since they're finally seeing that it's too damn volatile and entangles them in a nasty piece of work of a region.
u/Cruithne [+1] Light Sith epistemologist (23 hours later)
Infertility. It's rising quickly, and within a few decades we could see an underpopulation crisis. The population will almost definitely be larger than it is now, of course, but growth is the norm. We can adapt to more people much better than we can adapt to fewer.
u/eaglejarl [+2] (a day later)
It's not literal infertility though, right? People aren't losing the biological ability to have children, it's just that people in developed nations are choosing not to have them.
So long as it is a choice, I suspect it will balance out. If nothing else, we'll start seeing governments paying people to have kids -- the opposite of China's only-one policy.
u/Farmerbob1 [+1] Level 1 author (a day later)
Doesn't Germany already do this? Or is that an exaggeration?
u/None [+2] (a day later)
Only if you count child benefit payments between 184 and 215€ per child as paying parents to have children. There are however nice systems like parents time (parents can stay at home to raise a child for a limited amount of time while continuing to receive a percentage of their former wages) in place to support families.
u/Cruithne [+1] Light Sith epistemologist (a day later)
It is literal infertility, unfortunately.
u/eaglejarl [+2] (a day later)
Huh, I hadn't been aware of that.
A very quick google says that it's on the rise in the UK and Canada, but declining in the US. I've also heard that parts of Europe were having falling population, but I thought that was just by people's choice.
Do you have any idea what's causing it?
u/xamueljones [+1] My arch-enemy is entropy (a day later)
There are a lot of possible reasons from all of the chemicals we constantly put in our food, to just simply people having more trouble because they are waiting until their mid-thirties to reproduce, to some genetic mutations, to another million possible reasons why. No one has any clear idea (or hasn't told the rest of us yet) and I feel that's what most people are more concerned about.
I have to say though, the first thing I thought of when I first heard about this was, 'Thank goodness! We can actually get our population under control before a Malthusian catastrophe occurs.'
u/MadScientist14159 [+1] WIP: Sodium Hypochlorite (Rational Bleach) Eventually. Maybe. (2 days later)
Apathy towards the major problems we already have.
u/ZankerH [+1] Remove Kebab (3 days later)
Disregarding gnon on our leftward march. The decline that's happened so far has already been explained away and accepted as normal, and it isn't obvious to me at all whether or when the institutions orchestrating the leftward march will stop to look at reality and rationally assess the consequences of their actions.
And yes, I'm equating the collapse of classical western civilisation with an existential risk to humanity, of which it's a good enough first-order approximation.
u/mhd-hbd [+1] Writes 'The World is Your Oyster, The Universe is Your Namesake' (20 days later)
I see three: Pandemics, Meteor strikes, Economical collapse due to the 'bot revolution, and UFAI coming after the 'bot revolution.
For reference: 'Bot revolution.
Economical collapse is not an existential risk per-se, but I could see it being managed poorly and amplifying the current trends of inequality instead of equalizing society and moving us towards a post-scarcity society.
Essentially what is depicted in Manna.
With the rise of advanced approximate learning algorithms, I don't think UFAI is far off. It scares me shitless, tbh, and I hope we can notice it being evil and kill it with nukes before it gets too smart.
u/krakonfour [+1] (5 hours later)
Overpopulation, specifically, overpopulated developing countries.
Food requirements will explode, wealth distribution fails to address new needs and the starving masses soon get fed up with their fat masters indulging on corn-fed beef instead of feeding 10 people with that corn.
u/leplen [+2] (7 hours later)
And what do they do about it? A large group of people struggling to feed themselves in Africa don't have the resources to invade North America. Starving people also don't have children.
There have been hungry people since forever. It's not clear how they suddenly constitute an existential risk.
u/krakonfour [-2] (8 hours later)
This sort of mentality creates and perpetuates the problem, ie I'll do whatever I want as long as they can't do anything about it.
u/eaglejarl [+7] (9 hours later)
That may or may not be true, but it doesn't address the question: how is overpopulation-induced-starvation an extinction risk?
u/leplen [+1] (a day later)
I consider global poverty an important problem and effective altruism in general to be one of the more important uses of my time and energy. Existential risk is not synonymous with important, there are many important things that are not existential risks.
u/None [+3] (14 hours later)
An exploding population means growing food requirements. On the other hand, insufficient nutrition means pregnancy is much more dangerous, which should drive down the population even if rations are just barely enough to keep people alive in normal conditions.
And if rations are below what keeps people alive in normal conditions, people start dying in droves. Problem solved, albeit in a nasty way.
u/trifith [+1] Man plans, god laughs. Like the ant and the grasshopper. (12 hours later)
Population cannot grow past the food supply without encouraging a reduction in population. It's a self-balancing system. It cannot reach the level of existential risk.
People have also been predicting overpopulation for centuries, and so far, it hasn't happened. We're well beyond the levels of "dangerous overpopulation" made by the first predictions.
u/None [+1] (a day later)
Of course, we're also currently well beyond the Earth's sustainable carrying capacity for our species.
u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (16 hours later)
Heart disease.
u/JosephLeee [+1] (21 hours later)
Recreational drugs.
u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (21 hours later)
Orders of magnitude less risk. Or were you suggesting a solution to ease the stress, a known cause of heart disease, from knowing how likely it is you will die of heart disease?
In which case, agreed.
u/libertarian_reddit [+0] (15 hours later)
G-g-g-ghosts!
u/Farmerbob1 [+0] Level 1 author (12 hours later)
Can I paint this with broad strokes and just say humanity? We're too unpredictable and irrational as a society for me to confidently say what I think is the most dangerous threat we pose to ourselves.
I see lots of good ideas in here, but I can't choose which one! I feel like a kid in a candy shop with a dollar bill, looking from jar to jar, unable to make a decision.