14 KiB
From the script for SUCKER OF BLOOD
-
Author: u/DataPacRat Amateur Immortalist*
-
URL: https://www.facebook.com/yudkowsky/posts/10154358025619228
-
Score: 30
-
Created: 2016-07-09T07:03:35
Post:
Comments:
u/alexanderwales [+27] Time flies like an arrow (15 minutes later)
I think the intended lesson here is that we should be less cautious about the use of experimental and untested technologies which might have unforeseen consequences.
^/s
u/EliezerYudkowsky [+9] Godric Gryffindor (22 hours later)
Anything that doesn't wipe out the planet could indeed, perhaps, do with a bit less fear relative to modern culture.
u/None [+10] (a day later)
The problem there is that for most relevant definitions of "wipe out the planet", there are a whole lot of things that could wipe out the planet. In fact, at this point, you basically have to redefine downwards to "wipe out the planet worse than it's already being wiped-out right now".
Gene drives to eliminate a non-native species not tightly integrated into the food web that also gives human beings deadly parasites are an intervention that probably will be achieved. But the ethical-review process does actually take all that fucking time for a reason: DDT was once considered an obvious intervention with no possible downsides, as has been, in certain communities, "build a superintelligent AI and ask it what to do".
In short, the supervillain isn't as much of an ironic hero as he thinks he is, but for your choice to use the "WHY DIDN'T YOU SCIENCE AT IT" trope instead of the "There are gates you do not open, there are seals you do not unseal" trope which you normally use.
u/EliezerYudkowsky [+11] Godric Gryffindor (a day later)
I normally deal with things that one-shot humanity. DDT did not kill off humanity, and while that's true you can play the game multiple times and look at net winnings. Not building a ton of nuclear plants was such an awful mistake as to overshadow a lot of things and give all caution a justifiably horrible reputation to this day, and a lot of earlier "risks" turned out very favorably. If something had gone wrong and caused two hundred million annual cases of malaria, it would have been literally the worst technological disaster in history. Something going wrong with mosquitos is incredibly unlikely to top it.
u/None [+5] (a day later)
Not building a ton of nuclear plants was such an awful mistake as to overshadow a lot of things and give all caution a justifiably horrible reputation to this day
Oh come the fuck on. Not building a ton of nuclear plants was not about caution. It was about the public and the decision-making apparatus being terminally misinformed and also bribed by existing energy interests, paid not to understand the relevant science.
If something had gone wrong and caused two hundred million annual cases of malaria, it would have been literally the worst technological disaster in history. Something going wrong with mosquitos is incredibly unlikely to top it.
Luckily, we are running live experiments every day in what happens when you basically destroy Nature in large portion, for any reason or no reason. Currently, it's not going well, to the tune of trillions of dollars per year in actually-existing damage.
On the other hand, it's worth stating that my model here is what you wrote in the Facebook post: a four-year delay in deploying interventions against malaria, during which we remain able to use other existing interventions against malaria. Under those conditions, I side with caution and with not making even more environmental scientists, ecologists, etc start a bout of alcoholism to cope with how fucked we all are because we disregarded the substrate of our daily lives. Under the condition you seem to now be implying, that "caution" means the well-tested, well-understood intervention against malaria never gets deployed because people are too goddamn technophobic, I agree that some helpful terrorism would be helpful.
The question is the actual causal structure. I'm normally pretty much suffering radical uncertainty as to how many scientifically effective technologies or medical interventions have never been deployed because the cultural tide just said no, getting past obvious examples like nuclear energy. I can just about plausibly imagine that we could have had fusion power and thorium-salt fission if we'd actually fucking invested in them.
"We could have destroyed malaria but the bioethics board was too cautious" is in conspiracy-theory territory for me: low prior and too noisy a likelihood to investigate.
u/MuonManLaserJab [+3] (2 days later)
It was about the public and the decision-making apparatus being terminally misinformed and also bribed by existing energy interests, paid not to understand the relevant science.
And what arguments were used to misinform by these interests, other than appeals to caution (e.g. the single word "Chernobyl")?
u/None [-4] (a day later)
Oh, and then there's the issue where you seem to be assigning quite a low value to nonhuman life, but that may be forced on you by your commitment to utilitarianism. Hmm.
u/None [+2] (a day later)
[deleted]
u/MuonManLaserJab [+1] (2 days later)
I'm not the one you're talking to, but I assign value to nonhuman life according to:
- How useful/enjoyable/pretty it is to humans,
and
- The degree to which it is "person-like," or better, in terms of intelligence and compassion.
u/None [+1] (a day later)
Well I don't know. Eliezer definitely seems to assign low to no value to nonhuman life. It comes up when he's asked about vegetarianism, too. I'm unsure what the freck I'm supposed to do with it. He seems to think nonhumans aren't conscious and can't suffer?
u/Brightlinger [+2] (a day later)
Wild animal suffering is generally regarded as a Very Hard Problem by people who think it's a problem at all. For mosquitoes in particular, it seems very plausible that the best solution is in fact extermination; mosquitoes do not exactly lead long happy lives.
u/space_fountain [+5] (14 hours later)
It's worth mentioning that in the real world there is also examples of mosquitoes engineered to be resistant to Malaria and the corresponding gene drive to spread the trait. There's still some dangers there, but I'm very much in the camp of this stuff needing to be used in the very near future. If I was in charge of one of those labs, I don't know that I could or would refrain from releasing them.
The best argument I can come up with against that type of unilateral action is that these kind of things exhibit exponentiation growth. I looked up the equation in an earlier comment which I'll link below, but basically the number of individuals with the trait you start with makes a huge difference to the amount of time it takes the trait to basically take over the population. My worry is if by only releasing a few individuals you exposed malaria to the trait, but gave it time to evolve some sort of counter before the protection had spread widely enough. It's similar to why we don't just give people a little bit of antibiotic.
My earlier comment about this along with the discussion people had back then.
u/OutOfNiceUsernames [+8] fear of last pages (32 minutes later)
Are you familiar with the meme that goes, "Fuck this shit I'm out"?
Urgh.
u/Roxolan [+5] Head of antimemetiWalmart senior assistant manager (17 hours later)
Different tastes; it made me laugh.
u/None [+3] (12 hours later)
[removed]
u/OutOfNiceUsernames [-2] fear of last pages (13 hours later)
u/rhaps0dy4 [+1] (38 minutes later)
Yeah that probably shouldn't be in the actual movie.
u/lvwolb [+4] (2 days later)
I dunno. The way the proposed gene drive for mosquito extinction works makes me very uncomfortable, on a technical level.
The problem is that the proposed approach behaves like a "endo-virus" with two ways of propagation: (1) Transmission along germ line. Simple, all kinds of virii do this. (2) Cross-chromosomal infection: One chromosome is infected (from the father), and then infects the other one. The proposed payload is basically "become male during embryogenesis", which will drive the target mosquito to extinction.
Why is this scary? Because we put in all the required mechanisms for infections to occur, minus a protein shell for the virus. Hence, "endo-virus". The cross-chromosomal infection is the "gene-drive" thing, and it is a pretty universal construction that can easily evolve to carry a different payload, or infect a different species. The different payload is just a copy-paste away, as well as the different target site for other species. The latter thing is the amazing property of CRISPR, as opposed to usual gene-binding proteins.
Now, no protein shell for the virus is built in, so in principle no other insect should become infected. But this appears to be a really flimsy wall against catastrophe: killing off some vital pollinator? More apocalyptic, but also more unlikely: The novel way of virus design beeing picked up by evolution.
To repeat: The proposed attack does NOT need extremely rare horizontal gene transfer to occur to infect a different species. Instead, it only needs the different species to take up genetic material floating around; the machinery for building this into the chromosome is shipped along the proposed attack.
How would a "safe" gene-drive (A) look like that does not release the innovation of CRISPR-based virii into the wild?
(1) A parent with one or two versions of (A) produces a toxin and a corresponding anti-toxin. (2) Gametes (unfertilized eggs or sperm) that carry (A) produce the anti-toxin and survive to be fertilized / fertilize. (3) Gametes without (A) do not produce the anti-toxin and get killed by the toxin produced by the parent.
Since the "bad" gametes get killed and recycled quickly, the fitness load is pretty limited.
This is entirely sufficient as a gene-drive, no scary novel Rube-Goldberg way of virus transmission needed.
u/symmetry81 [+3] (6 hours later)
I noticed I'm sort of confused about whether the technique would work. Part of me is thinking that genes that each child has exactly one father and one mother, so if some gene tends to increase the proportion of one at the expense of the other and it becomes relatively common then the alleles that allow for daughters will have more reproductive success and the gene should reach equilibrium at a low incidence.
On the other hand in some species, such as humans, the gender of offspring is fundamentally controlled by the male's contribution so maybe we should only be analyzing the spread of this sort of sex fixing within the male population?
On the third hand, if this sort of disaster was possible I'd expect it to happen in nature and maybe I'd have heard about it?
But on the gripping hand the people who are proposing this probably know more about population genetics than I do and have run actual sims.
u/alexanderwales [+6] Time flies like an arrow (14 hours later)
You can read the paper in Nature here:
However, the higher homing rates observed for CRISPRh at AGAP007280, combined with the milder fertility reduction observed in heterozygous females indicate that this construct could spread through a population, at least initially, and impose a reproductive load on the population as it does so, fulfilling one of the major requirements for a functional gene drive measure for vector control (Fig. 2f). To investigate the ability of the CRISPRh allele to spread at the AGAP007280 locus, caged populations were initiated with CRISPRh/+ and WT individuals at equal frequency and monitored over several generations. Consistent with the modeling predictions we observed a progressive increase in the frequency of individuals positive for the CRISPRh allele from 50% to 75.1% over four generations (Fig. 2g). Such a reproductive load will impose a strong selection pressure for resistant alleles, some of which will be generated by the gene drive system itself through NHEJ or MMEJ repair of endonuclease-induced chromosome breaks, as we previously showed molecularly (Supplementary Fig. 4). The longer term dynamics will depend on the efficiency of spreading on the one hand and the fitness cost of mutations arising at the cleavage site on the other hand.
Basically, the simulations say that it will spread (to a degree) and the actual caged study suggests it will spread (to a degree).
u/alexanderwales [+3] Time flies like an arrow (2 days later)
This link has some really interesting reading on possible problem areas for this technique.
u/Sailor_Vulcan [+2] Champion of Justice and Reason (12 hours later)
I WANT TO SEE THIS MOVIE