36 KiB
The Age of Em, A Book about Living in a Future where Uploading Tech is Easy
-
Author: u/xamueljones My arch-enemy is entropy*
-
URL: http://ageofem.com/
-
Score: 6
-
Created: 2016-04-17T22:09:01
Post:
Comments:
u/xamueljones [+5] My arch-enemy is entropy (2 minutes later)
So here's the website for Robin Hanson's new book. I'm just letting people here know about it so they get the chance to pre-order it before May 1st.
For those of you who don't remember him, he's one of the people whose name you'd frequently hear whenever you read about Yudkowsky's professional work.
Here's a link to The Hanson-Yudkowsky AI-Foom Debate.
It's not rational fiction, but it's a (text)book I think a lot of people here would be interested in checking out.
u/Lugnut1206 [+3] (3 hours later)
i'm pretty sure this isn't moral
you're making copies of real people and enslaving millions of them so the real-real people don't have to work
u/xamueljones [+3] My arch-enemy is entropy (7 hours later)
Just pointing out that I'm not the author of the book (because you using the pronoun "you're") and he's not saying that people should be behaving the same way as an em.
However he does seem to...gloss over the more disturbing aspects, as if he's saying that we shouldn't judge them. So I sort of agree with you, but I'm planning on buying and reading the book to see if there's better reasons/phrasing which support his arguments.
u/Lugnut1206 [+2] (8 hours later)
i was using you're in terms of "hypothetically, you (the reader) are a person who is implementing these ideas"
i'll also admit i didn't make it more than halfway down the page so um
yeah, i'm sure it's an interesting book though
u/BadGoyWithAGun [+2] (17 hours later)
However he does seem to...gloss over the more disturbing aspects, as if he's saying that we shouldn't judge them.
That's not what he's doing. If you've ever heard him speak (especially in front of more left-leaning audiences), he always points out that he's in the predicting business, not the judging business. In other words, he lays out what he considers to be the likely consequences of ems, regardless of what people may think of their moral implications
The thing is, as far as I can tell his prediction given ems is thus: humanity being succeeded by trillions of copies of increasingly dumbed-down emulations of a few thousand previously-living humans living on subsistence wages in a hyperspeed race to the bottom.
u/Empiricist_or_not [+2] Aspiring polite Hegemonizing swarm (2 days later)
calling /u/DocFuture because this topic is depressing, a highly predicted failure state that his (fictional) work is actively confronting, in hope he has something, positive please, to contribute.
u/whywhisperwhy [+1] (5 days later)
Really? I haven't read past the first Doc Future, and that's a pretty significant jump of themes. Might have to give the next few books a try.
u/Empiricist_or_not [+1] Aspiring polite Hegemonizing swarm (5 days later)
It's a significant theme in the 3rd book. Not the whole plot but an interesting one. It has a AGI and the jury is still out on how realistic it is for a world that has Flicker in it.
u/DocFuture [+1] (6 days later)
Sorry, haven't been on reddit for a bit. This is one of the (many) bad failure modes in the futures Doc sees--and one of the reasons he hasn't been pushing cybernetics or uploading tech this time. In my fictional world, and plausibly in ours, profit motive + corporations + global externality evasion + uploading tech = humans lose. The Grs'thnk have strong restrictions on externalities and AI, and are still worried. DASI is just starting to put together infrastructure to sidestep the problem, while Black Swan distracts everyone with the more flamboyant and obvious task of killing off the most problematical of the existing corporations to buy time and change the incentive landscape.
u/MugaSofer [+1] (13 hours later)
Pretty much everyone agrees with this except for Robin Hanson, who insists on defending the idea that poor ems will be basically happy-ish, and something something total utilitarianism, and also GOOD LUCK STOPPING IT.
Also, I don't think he anticipates there being any "real people" once they're massively outcompeted by ems.
u/klassekatze [+3] (5 days later)
Maybe I'm missing something, but isn't it true that there are humans who legitimately enjoy a given task? If they can morally exist, then constructing or copying a human mind that shares that quality shouldn't be immoral (unless you start going into strange territories where only "naturally" achieved this or that is moral). There are also humans right now who wouldn't care about being copied, or even about dying (deletion analogue). So unless their existence is wrong, making more minds like them shouldn't be wrong either.
u/whywhisperwhy [+1] (5 days later)
unless you start going into strange territories where only "naturally" achieved this or that is moral
Anything other than copying volunteers of people seems unethical to me unless we start bringing utilitarianism into this. I would think brainwashing or otherwise conditioning someone (assuming this is what you meant by "naturally") in a way that effectively makes them a slave would be just as controversial as constructing someone to assume a slave role.
u/klassekatze [+3] (5 days later)
Copying non-volunteers doesn't even enter the picture because there are thousands that would volunteer and you only need a few or even one to make unlimited workers.
I don't really see the difference between copying a volunteer that likes work and copying a volunteer that doesn't and then making that copy like work.
I feel compelled to mention here this is something I would be perfectly comfortable doing to a copy of myself. I don't see it in the same context as brainwashing a pre-existing person because the copy to be modified has never existed as a distinct individual and in fact would never exist if I didn't start them up post-edit. If this is immoral then I have to ask, are created nonhuman AI workers immoral too? If it is about level of sentience or humanlikeness, are you using a dartboard to pick where you are drawing your line?
Furthermore, most people must work to achieve their terminal values, among them usually survival. If it is within your power when creating a new intelligence to make one that derives satisfaction from necessary work, but you instead make it suffer because that is human 'factory default', doesn't that make you a "malicious god"?
But suppose for the sake of argument we declare that any modification of the mind is immoral, even with the consent of the mind in question. This is a world of seven billion. There is almost certainly someone who is an ideal worker who desires no recompense already in existence through no manipulation on our part. Copy them with their agreement and we're done.
If that too is immoral, then I think we have long, long passed the point where it is more about forcing your terminal values, morals, and beliefs on others (by which I mean the volunteers) and not really about helping anyone at all.
u/whywhisperwhy [+1] (5 days later)
First, I agree with you that uploading or copying volunteers is moral. And if a volunteer agrees to be altered, I suppose I would accept that as well (although it gets questionable again if the person is unwell in some way, economic duress, etc.). So the only thing we seem to disagree on is involuntary creation/modification of minds with the intention to enslave. Practically speaking, I don't think that's a flaw in Robin Hanson's projection.
If this is immoral then I have to ask, are created nonhuman AI workers immoral too?
Right, I think creating any sentient being with the intention of enslaving it is immoral. I'll preemptively agree that the level of sentience intelligence/self-awareness is problematic difficult to pin down, though.
Furthermore, most people must work to achieve their terminal values, among them usually survival. If it is within your power when creating a new intelligence to make one that derives satisfaction from necessary work, but you instead make it suffer because that is human 'factory default', doesn't that make you a "malicious god"?
I'd also agree that if you're going to create a slave, it's better to make them derive satisfaction from their role, otherwise as you say, it's cruel.
I feel compelled to mention here this is something I would be perfectly comfortable doing to a copy of myself.
I would use the exact opposite (not about being modified necessarily, but being created with a utility function that makes me a slave).
u/klassekatze [+3] (5 days later)
Typically, when one thinks of a slave, they think of someone who is forced to do something, whether they want to or not. Someone that has no choice. I would argue that if the sentient is allowed to not work, then it isn't a slave - even if it is designed to want to work, and is thus unlikely to use that right.
I think it is important to have the same rules for minds that arise without your interference as you do for those that are created.
If creating a mind designed to only want to work is wrong, it seems to me that is the same as saying that a naturally arisen person (i.e. such as a person made of meat you had nothing to do with) that only wants to work is wrong. It is only a step further into the real world and, say, BDSM is wrong. Although I guess the analogy is weak because working improves things and BDSM doesn't.
You might have some deeper reasoning behind treating creations differently. If not, let me propose an ironic analogy: "A white person who wants to work is not a slave. A black person who wants to work is a slave." Unfortunately, the analogy is a bit off-- differently colored people are actually able to be told apart. A theoretical perfect edit/creation that only wants to work and an upload of a theoretical perfect worker would not be.
Food for thought.
EDIT: The TLDR is: I don't think that a person devoted to working is wrong. (If anything, they are less wrong [heh] than people who find work painful.)
I absolutely do not think "it just is" is an acceptable argument for why creating such a person is wrong if you acknowledge their existence by other means is not wrong.
If you have some other argument, of course, I would love to hear it.
u/whywhisperwhy [+1] (6 days later)
First, if your utility function guarantees that your happiness is strongly tied to working, do you really have a choice whether to work or not?
Also, my argument is not "it just is," it's based on the principle that intelligent beings should have self-determination. As I said, in general I would not want to be created with a utility function locked such that my purpose is to work and serve people, even if it made me happy. I do see what you're saying that if some people naturally developed into that attitude, I wouldn't see anything wrong with that, but the difference is that they had some choice in the matter. That having been said, the more I think about it the more I think it's not "wrong" per se, just not what I would prefer (if this were CMV you'd be receiving a delta right now).
I think the optimal situation would be that an artificial intelligence be treated like any human and at least be given some measure of choice in what it decides to pursue (for example, if it decides to pursue physics research then great, recruit it for that, if it decided to become interest in social welfare, also great) and also some ability to change its utility function. Obviously that has to be tempered with practicality (we don't want it to become genocidal and we also want it to be productive), though.
Edit: Out of curiosity, do you think these AGI or modified uploaded humans should be property?
u/klassekatze [+1] (6 days later)
Out of curiosity, do you think these AGI or modified uploaded humans should be property?
I don't think any intelligence should be property. But what I'm thinking there is probably not what you are thinking.
If you own something, you can do anything you want to it. If you don't, you can't - you need the permission of the owner, in this case the intelligence.
I do think you shouldn't change an existing intelligence against its will. You should not attempt to force it to do things that it doesn't want to do. If it will let you change its utility function, then I see nothing wrong with that.
I think that creating a new intelligence or editing a copy with a utility function and running it isn't wrong either, regardless of what that function is. It doesn't exist yet-- it is just a potential, a file - a saved copy of klassekatze.
Once you have started it running, then I consider it a person - and you don't own it. If it wants to do anything you say, though, that is fine.
The first thing that comes to mind here is what you said earlier:
I would use the exact opposite (not about being modified necessarily, but being created with a utility function that makes me a slave).
This issue doesn't come up because unless you literally give your mind away, that would be theft, to put it lightly.
I think my position works pretty well both morally and pragmatically. Unless I've missed something, it doesn't allow any of the nasty slavery that has ever come up in human history nor anything I can think of. It doesn't allow "skynet" either: in the event you do it wrong and a created AGI doesn't actually want to do whatever, it is legally protected and doesn't have that as motivation to cause problems. The same applies to edited human minds.
u/MugaSofer [+1] (6 days later)
Consider a psychopath. They see nothing wrong with killing. Nevertheless, them killing people is wrong.
More generally, there's the issue of competition - if (as Hanson argues) regular and indeed irregular humans will be entirely replaced with the most efficient version of ems, then that means experiences we view as valuable - love, art, a sense of wonder and awe, and so on - will be eradicated in favour of only actions that are economically productive.
EDIT: Scott Alexander argues this second point somewhat in Meditations on Moloch, while the first point is kind of argued in the Fun Theory sequence on LessWrong (although I believe it was removed when they were adapted to book format.)
u/klassekatze [+1] (6 days later)
Consider a psychopath. They see nothing wrong with killing. Nevertheless, them killing people is wrong.
Absolutely. However, I'm going to go out on a limb here and say "manufacturing criminals" isn't what you meant. I don't see what your analogy has to do with creating people who want to do perfectly legal and productive things.
(as Hanson argues) regular and indeed irregular humans will be entirely replaced with the most efficient version of ems, then that means experiences we view as valuable - love, art, a sense of wonder and awe, and so on - will be eradicated in favour of only actions that are economically productive.
If Hanson argues that, then I think he is wrong - I haven't read the book, mind you. That assumes scarcity. I instead assume that the ability to create infinite workers who want only to work will forever guarantee the existence of those experiences, by providing more resources than we have now, and the existing stock people will still exist. "Normal" people certainly aren't going to all die in a world with uploading, much less stop reproducing. The workaholic ems aren't going to murder them for not working, bar some incredibly poor editing on the part of their manufacturer.
I'll flatly say I can't imagine that scenario happening - it doesn't make sense unless a lot of powerful and simultaneously stupid people all agreed to aggressively own all the matter and energy forever and choke every sentient in red tape until they turn blue from before uploads even took off.
u/MugaSofer [+2] (6 days later)
I don't see what your analogy has to do with creating people who want to do perfectly legal and productive things.
Not everything that's legal and productive is good. Hopefully, it's not necessarily bad, but that still leaves a lot of other stuff.
To pick a silly-but-standard analogy, if half the population became obsessed with paperclips, they'd create a bunch of paperclips for no good reason. Paperclips aren't evil, as long as you ignore people getting buried in paperclip-slides or what have you; they're just kind of pointless.
Similarly, if most of humanity was replaced with workaholics - not "normal range of human variance" workaholics, beings that can't conceive of anything else - they'd spend a lot of time doing boring, pointless work. Not for some greater goal, either - presumably, most of the economy would consist of stuff they need to do the pointless, boring work of making it.
(And, of course, they'd make the laws, which means "legal and productive things" could mean "grinding up your fellow workers to make hammers", because they're all psychopaths.)
(as Hanson argues) ...
Hanson believes that everyone will become as efficient workers as possible.
He argues we should still be happy for them/they won't be that different from us, Scott Alexander argues otherwise.
Hanson argues that we currently exist in a very unusual time period; the ability to produce workers has fallen far behind the production of resources those workers require, leading to vast amounts of excess resources. (A period he poetically calls "the Dreamtime".) Computers, on the other hand, can be produced in exactly the amounts and to exactly the specifications required for any project; meaning there's an overwhelming incentive to do so for any given project, as long as the project is maximally profitable.
That assumes scarcity. I instead assume that the ability to create infinite workers who want only to work will forever guarantee the existence of those experiences, by providing more resources than we have now, and the existing stock people will still exist.
Modern workers produce much more, per person-hour, than they did before. Most people hate work. Yet working hours have not reduced. Why?
Because how much you "need" to work is based on how much other people work. That's what competition is.
...murder them for not working...
Yes, Hanson believes people will be "murdered for not working" - specifically, they will run out of resources to support themselves, whether physical resources or processing power on some server rack somewhere. And then they (or their batteries) will starve.
This is a real thing that actually happens, so, you know ...
Compared to the overwhelming advantage of an organism designed specifically for producing profit, why would anyone give resources to a human? They have nothing to trade.
If the answer is "altruism", consider whether altruists are likely to have higher profit margins than non-altruists.
u/klassekatze [+1] (6 days later)
Compared to the overwhelming advantage of an organism designed specifically for producing profit, why would anyone give resources to a human?
I'm assuming that any mind capable of editing itself is also capable of forking, and being a moderately intelligent sort would edit the fork instead. Avoiding troublesome potential human editing error like 1% more evil and all that. Some of these forks will naturally retain "loyalty", others might be hardcoded for it. Either way, they provide resources for the original.
Yes, Hanson believes people will be "murdered for not working" - specifically, they will run out of resources to support themselves, whether physical resources or processing power on some server rack somewhere. And then they (or their batteries) will starve.
I'm not saying there won't be somebody somewhere starving to death, but that isn't mass extinction.
There would have to be a pretty stellar series of fuckups for enough minds-in-charge on Earth to be so divergent from human norms that they are indifferent to mass murder just so they can have physical resources they have no use for.
Off the top of my head, I could get a boat, strap a waterwheel to it as a generator and drop anchor in the middle of the ocean. The only reason I'd be justifiably bothered is if the worker ems have used up the rest of the ocean. At that point obviously technology has leveled up, so move my boat to the asteroid belt or something.
Modern workers produce much more, per person-hour, than they did before. Most people hate work. Yet working hours have not reduced. Why? Because how much you "need" to work is based on how much other people work. That's what competition is.
Yeah, no. People still work as much because they want modern stuff. If you are willing to live at a lower level, you can work way, way less. Food is cheaper than it ever was; you could live for a week at fast food prices off an hours labor in the US. If you own land you owe only taxes. Land with moving water is surprisingly cheap.
Anyway, the only other reason I can see is some sort of holocaust-esque "they came for the x, and we said nothing" spiel before they finally repossess the server racks of the last original humans and that just isn't likely to go unnoticed. And still requires all the big bosses to have done brain surgery on themselves rather than using forks and employees to get the same result, leaving human-normal minds in charge that would not go "Repo Man" on all the original humans for a stretch of matter they don't need when they own 3/4ths the planet or whatever.
u/MugaSofer [+2] (6 days later)
I'm assuming that any mind capable of editing itself is also capable of forking, and being a moderately intelligent sort would edit the fork instead. Avoiding troublesome potential human editing error like 1% more evil and all that. Some of these forks will naturally retain "loyalty", others might be hardcoded for it. Either way, they provide resources for the original.
I'm not sure I follow. You seem to be suggesting that creating a more profitable worker would be an accident we'd prevent against, but it seems obvious to me that it would be some people's end-goal and that those people would go on to be more successful.
There would have to be a pretty stellar series of fuckups for enough minds-in-charge on Earth to be so divergent from human norms that they are indifferent to mass murder just so they can have physical resources they have no use for.
...
...
... you literally live in the First World. That is the world you're living in right now.
(Also, they have a use for physical resources: expanding their businesses by purchasing equipment and hiring ems. Obviously, this won't appeal to everyone, but "luckily" we're dealing with simulations who have been selected to maximize exactly that trait.)
People still work as much because they want modern stuff.
Hmm, maybe. It would certainly be surprising if that wasn't a factor, given all we've created.
On the other hand, glancing at the stats, it looks like food has steadily gotten cheaper (in terms of percentage of household income), while housing and medical care have stayed roughly the same, and transport costs have gone from nil to almost fully replacing the gains from food costs. That sounds like people working harder (via commuting) to replace any gains made while positional goods stay the same.
I could get a boat, strap a waterwheel to it as a generator and drop anchor in the middle of the ocean. The only reason I'd be justifiably bothered is if the worker ems have used up the rest of the ocean. At that point obviously technology has leveled up, so move my boat to the asteroid belt or something.
How are you going to move to the asteroid belt if you have no money and live on a boat?
u/klassekatze [+1] (6 days later)
On the other hand, glancing at the stats, it looks like food has steadily gotten cheaper (in terms of percentage of household income), while housing and medical care have stayed roughly the same, and transport costs have gone from nil to almost fully replacing the gains from food costs. That sounds like people working harder (via commuting) to replace any gains made while positional goods stay the same.
I'll admit I see the pattern and I'm no expert in this matter.
I'm not sure I follow. You seem to be suggesting that creating a more profitable worker would be an accident we'd prevent against, but it seems obvious to me that it would be some people's end-goal and that those people would go on to be more successful.
No. By 1% more evil, I was referring to editing your own utility function being dangerous in general, because humans make programming mistakes - as in literal "I live only to count paper clips" when you meant to be a successful paper clip manufacturer in order to pay for a space yacht. Rather, you make the edit to a fork, and leave the original running indefinitely. Particularly when you are optimizing for a task but don't want to risk messing other stuff up - and human civilization is the product of avoiding work. Changing that risks fixing what isn't broke, and thus breaking it. Furthermore humans are uncomfortable with changing their minds.
How are you going to move to the asteroid belt if you have no money and live on a boat?
This could be a problem, but a world where they actually use up the ocean is alien to our own. Human emulation obviates the need to limit ourselves to biological clockspeeds, and even the most pessimistic estimation of that multiplier allows workaholic ems to solve all design problems. (Crunching the numbers allow in excess of 1000x on the more extreme end.) It isn't unreasonable to believe that at that point there would be viable designs for boat-portable assemblers, and every other technology necessary to build a spacecraft - with or without the workaholics declaring ownership of the oceans permission.
A bit more speculative than I wanted to go, but so is them coming after my boat.
(Also, they have a use for physical resources: expanding their businesses by purchasing equipment and hiring ems. Obviously, this won't appeal to everyone, but "luckily" we're dealing with simulations who have been selected to maximize exactly that trait.)
I think we are operating from different positions here, mainly the idea that the commander-in-chief of a given power bloc must be a modified human simulation rather than a unmodified legacy of the prior era.
I also don't think anyone will hire ems at all - there is no reason not to manufacture them willing to work for no pay. Ethics of this I debated in another part of this thread, but the pragmatic aspect seems undeniable. You can't posit a world where ems can legally claim the ocean in the name of profit yet corporations failed to legally make unpaid obedient workers in the name of profit. Well you can, but I think it is improbable. (Also the unpaid worker corp wins the profit competition.) If you accept this position, it is no longer a competition between worker ems and originals - it is a competition between worker ems and worker ems in service to an original. Sufficiently large numbers of ems and the inefficiency of that one becomes irrelevant.
The real question I have is how Hanson's book and/or this very conversation doesn't make this a self-defeating prophecy. If we can predict this quiet eradication of original humanity, then surely people would do the same when it actually starts happening. Then we would pass possibly unfair laws ensuring original human supremacy of the matter and if they were violated would use force to maintain that control while we still had greater power that trumped their optimizations. The morality of this is debatable, but that it is typical human nature is less so. Space-Steve-Jobs or w/e isn't stupid, and he doesn't need super optimizations to skim a report from his own worker ems that says there are less original humans every year and then investigate.
As an aside on the whole only-have-a-boat. With the increasing practical power of individuals with the successors to primitive 3d printing, the ability for individuals to do things other than roll over and die also increases. Short of a draconian police state from early on in favor of the workaholics - in charge, not simply in service to original humans - this would inevitably lead to physical violence.
To repeat: the only advantage the purist workaholics have is they have 100% workaholics and the originals can have 100% - 1 mind in charge. I don't think this guarantees them victory in any battlefield, physical or economic.
I've probably circled around a few times here but eh. The important points are covered either way. Again, there may be a huge number of people getting the short stick, but I don't believe originals would be eliminated as a whole, because they had an unfair advantage from the start, and knew exactly what was happening when they invented worker ems. After all, they read Hanson's book.
u/MugaSofer [+1] (7 days later)
I think Hanson (and I know Scott) is basically a technological determinist - any regulations to prevent this technology will inevitably fail, because people who don't regulate will do better. I disagree with this in principle, but it's definitely concerning (see the utter failure of regulators to prevent piracy, for example.)
The "rich CEO ends up funding humanity's runtime for the next billion years outcome isn't ideal, but it's certainly one of the better outcomes. On the other hand, I'm concerned that a CEO-em might inevitably do better than a company limited by an ethical master, especially given their improved micromanagement etc? Which I assume (can't recall them addressing it) would be their counterargument.
In defence of paying ems, our current economy kind of depends on people having money to allocate resources properly (and most societies that kept slaves let them have personal property, with many allowing them to buy their own freedom if they saved up.) Still, there are other solutions. I don't think it's integral to the concept (and I think Hanson usually says "use".)
boat-portable assemblers
Ah, you're missing my point there. Even if ems invent boat-portable spaceship manufacturies, you have no money (and the speed of the economy means any money you had before is probably worthless by now), and your boat is worthless because they're vacuuming up the oceans. You have nothing to give and nothing to trade.
Of course, this is a highly specific and unlikely example, and the whole concept is reasonably suspect - it's not clear that applying the emulation technology to it's own designers wouldn't produce an Eliezer-style foom, for example.
u/klassekatze [+1] (8 days later)
Even if ems invent boat-portable spaceship manufacturies, you have no money (and the speed of the economy means any money you had before is probably worthless by now), and your boat is worthless because they're vacuuming up the oceans. You have nothing to give and nothing to trade.
This seems to be predicated on the idea that a modified em is so much better than you that it is impossible for you (or your loyal fork) to design an assemble before the em hegemony takes over the world and officially declares they own all matter and vacuums up the ocean out from under that boat you likely didn't have time to buy. Which is possible, but if its true I expect the world to simply belong to the first em which was created by Chinese or Koreans or whoever cared the least about the risks and is probably a faulty upload and thus more irrational than human minds are usually. So it puts a mystery module in some design its creators can't afford to not use if they can even perceive the module and that module starts assembling super UAVs and nanomachines son and you can see where this goes.
I don't necessarily believe in Eliezer-style foom but I do think that if you are running at a thousand-to-one (which is easily believable going by the brains "clockspeed" vs electronics) that, well. I could put in a month and some change sidereal and have equivalent to 100 years design work on say, that assembler.
Civilization as we know it is built on the assumption there is things you can't do yourself. Given our current progress, the extra time ems have suggests it is reasonable personal manufacturing technology was developed before ems were in any position to essentially take over the world (I'm pretty sure that's a prerequisite to vacuuming up the ocean by any name) if only because the alpha test ems were locked in a lab.
And the inventors of the technology are not Mass Effect Cerberus.So you don't need money because you aren't asking for permission in the first place, because most humans don't ask permission when the alternative is clearly death - this seems self evident. I also assumed you were an upload yourself otherwise you'd need food, so you do have some measure of time to design your manufacturing bootstrapping plan if not the spacecraft, and don't need meat life support with all that entails.
Anyway. I guess what I'm saying is that if we assume regulations will fail if non-regulators can do better, then the world will belong to whoever rushes out a duct taped unstable em first and blindly prints their shiny blueprints nobody understands (because any trojans have had hundreds of years of subtlety applied). And Hanson's, or any other, world will only exist insofar as that singular mind allows it.
If we can avoid that? I'm willing to bet we can avoid the far slower threat of ems turning the planet into manufacturing materials - certainly to the extent of unmodified human uploads finding a haven or escaping the planet, at worst.
u/None [+0] (6 days later)
"Compared to the overwhelming advantage of an organism designed specifically for producing profit, why would anyone give resources to a human? They have nothing to trade."
I think the clear answer to this is that if they won't give, then you have to take. If race-to-the-bottom ems are the next big enemy of humanity, humanity needs to get back to the roots of what it does best. Hatred, war, and genocide.
u/Lugnut1206 [+1] (17 hours later)
Sounds like he might be failing to multiply.