## [Weekly Challenge] Science is Bad * Author: u/alexanderwales *Time flies like an arrow** * URL: https://www.reddit.com/r/rational/comments/3hnau9/weekly_challenge_science_is_bad/ * Score: 23 * Created: 2015-08-19T22:53:09 ### Post: ## Last Week [Last time,](https://www.reddit.com/r/rational/comments/3gs5ko/weekly_challenge_disney_movies/?sort=confidence) the prompt was "Disney Movies". /u/ZeroNihilist is the winner with their story ["Monsters, Incentivized"](https://www.reddit.com/r/rational/comments/3gs5ko/weekly_challenge_disney_movies/cu1s7vx), and will receive a month of reddit gold along with super special winner flair. Congratulations /u/ZeroNihilist! (Now is a great time to go to that thread and look at the entries you may have missed, especially the late entrants; contest mode is now disabled.) ##### This Week This week's prompt is "Science Is Bad". We're all familiar with [Caveman Science Fiction](http://dresdencodak.com/2009/09/22/caveman-science-fiction/); this is your chance to do it *right*. [See the entry on TvTropes](http://tvtropes.org/pmwiki/pmwiki.php/Main/ScienceIsBad). Your challenge is to do this in a way that won't be cringe inducing for people in this subreddit. Yes, you can also use this opportunity to do "Engineering Is Bad" instead. Remember, prompts are to inspire, not to limit. The winner will be decided **Wednesday, August 26th.** You have until then to post your reply and start accumulating upvotes. It is strongly suggested that you get your entry in as quickly as possible once this thread goes up; this is part of the reason that prompts are given a week in advance. Like reading? It's suggested that you come back to the thread after a few days have passed to see what's popped up. The reddit "save" button is handy for this. ##### Rules * 300 word minimum, no maximum. Post as a link to Google Docs, pastebin, Dropbox, etc. **This is mandatory.** * No plagiarism, but you're welcome to recycle and revamp your own ideas you've used in the past. * Think before you downvote. * Winner will be determined by "best" sorting. * Winner gets reddit gold, special winner flair, and bragging rights. * All top-level replies to this thread should be submissions. Non-submissions (including questions, comments, etc.) belong in [the meta thread](http://www.reddit.com/r/rational/comments/39dxi3), and will be aggressively removed from here. * Top-level replies must be a link to Google Docs, a PDF, your personal website, etc. It is suggested that you include a word count and a title when you're linking to somewhere else. * In the interest of keeping the playing field level, please refrain from cross-posting to other places until after the winner has been decided. * No idea what rational fiction is? [Read the wiki!](http://www.reddit.com/r/rational/wiki/index) ##### Meta If you think you have a good prompt for a challenge, [add it to the list](https://docs.google.com/spreadsheets/d/1B6HaZc8FYkr6l6Q4cwBc9_-Yq1g0f_HmdHK5L1tbEbA/edit?usp=sharing) (remember that [a good prompt is not a recipe](http://www.reddit.com/r/WritingPrompts/wiki/prompts?src=RECIPE)). If you think that you have a good modification to the rules, let me know in a comment [in the meta thread](http://www.reddit.com/r/rational/comments/39dxi3). Also, if you want a quick index of past challenges, I've [posted them on the wiki](https://www.reddit.com/r/rational/wiki/weeklychallenge). ##### Next Week Next week's challenge is "Dueling Time Travelers". Two people, with access to time travel, in some sort of conflict with one another. Your choice of [time travel model](http://qntm.org/models), though it's highly recommended that you keep it rational. Yes, you can include more than two people in conflict, or just have one person in conflict with their past/future self. **Next week's thread will go up on 8/26.** Please confine any questions or comments to [the meta thread](http://www.reddit.com/r/rational/comments/39dxi3). ### Comments: > **u/None** [+25] (19 hours later) > > [Elegance](https://docs.google.com/document/d/1wbP4Wq9EY5dHoJ4QIxk5LmlU4-_9D-GLaN4FxsdZcvE/edit?usp=sharing) > > 1653 words >> **u/BadGoyWithAGun** [+11] (19 hours later) >> >> Props for actually invoking "science is bad", works in this trope can often be better summarised as "engineering is bad". >>> **u/None** [+6] (20 hours later) >>> >>> Thanks. About halfway through I realized that it would have fit better in the rational horror one, but inspiration is fickle. >>> >>> If you happen to have any critique that you don't want to sway the voting, would you mind sending a pm? That applies to anyone who reads this. >> **u/Kishoto** [+4] (a day later) >> >> Mind giving me a behind the scenes understanding of what just happened? Like in layman's terms? >> >> [](#s "I think I was able to understand that the knowledge of that suicide professor's theory was dangerous, but I'm not really clear on how. Or on what happened.") >> >> EDIT: Also, nice Hitchhiker's reference! >>> **u/notmy2ndopinion** [+6] *Concent of Saunt Edhar* (4 days later) >>> >>> [An Elegant Mind. Bravo!](#s "This isn't a spoiler, but more a commentary on the story writing process. Rather than taking the Hollywood route to schizophrenia with the CIA and imaginary companion scenario, THIS is how I imagine a genius would actually unhinge as reality disintegrates in the world around them. It is elegantly disorienting. Auditory command hallucinations? Check. Delusions of grandeur? Check. Reality-warping paradigm shift that prompts radical and sudden action to commit suicide? WHOA. Pump the mother effin brakes, man.") >>> >>> [Black box warning:](#s "If any of you find yourself in this scenario, you are much more likely to be schizophrenic in the real-world than discovering that you are an experiment in a simulated-world. Even if you are convinced that you are a simulation and you feel compelled to end your life, you should seek out a mental health professional. The question you can puzzle out for yourself is the modified Pascal's Wager on risking which world is more likely or which derived action is most rational in both scenarios.") >>>> **u/None** [+2] (4 days later) >>>> >>>> I am so sorry and should have thought that through more. Should I be adding this disclaimer to the end of the story? >>>>> **u/notmy2ndopinion** [+1] *Concent of Saunt Edhar* (7 days later) >>>>> >>>>> I don't know -- it takes the edge off the rational horror to back pedal and give a disclaimer [right after](#s "having the main character decide that the only reasonable course of action is to commit suicide.") But my growing sense of dread prompted me to write the Black Box warning, to caution other readers about approaching this story as a realistic info-hazard. >>>> **u/None** [+1] (5 days later) >>>> >>>> [deleted] >>>>> **u/notmy2ndopinion** [+1] *Concent of Saunt Edhar* (7 days later) >>>>> >>>>> By Rhllor, I hope you're joking about the "it's not enough" >>>> **u/RMcD94** [+0] (7 days later) >>>> >>>> Shouldn't we already believe what caused the end? >>>>> **u/notmy2ndopinion** [+1] *Concent of Saunt Edhar* (10 days later) >>>>> >>>>> I believe the ending is true to what what the main character believes... and what the author intended. >>>>> >>>>> After all, it wasn't all in his head. >>>>> >>>>> However, following the same line of thinking that other rational redditors have, when they are skeptical of threads with "you suddenly believe you have X superpower" as the truth, I put a filter of doubt on the scenario and would advise going to a psych ED if you ever found yourself in a similar situation. >>>>>> **u/RMcD94** [+0] (10 days later) >>>>>> >>>>>> >After all, it wasn't all in his head. >>>>>> >>>>>> But doesn't Nick Bostrom's [famous essay](http://www.simulation-argument.com/simulation.html) already presuppose intelligent people would believe this already. So that even if you think you are in ...conclusion of that essay... then there's really no extra evidence and the conclusion of said essay isn't to do what's in your black box warning. >>>>>> >>>>>> So then I don't understand how you can estimate likeliness from that kind of position. I still agree with your black box warning I just think the conclusion might not follow. >>>>>> >>>>>> I feel like the chances of anyone reading this at this point who is worried about spoilers is small but oh well. >>>>>>> **u/notmy2ndopinion** [+1] *Concent of Saunt Edhar* (13 days later) >>>>>>> >>>>>>> perhaps I am confused by the oblique nature of avoiding spoilers, but I think we agree. (Thanks for the article by the way, it's much more academic than comparing this to Vanilla Sky.) >>>>>>> >>>>>>> You can't estimate likelihood of Bostrum's scenario (3), but I'm not going to be the one to tell you to pick X over Y if your "rational brain" convinces you to commit an irreversible act in real life. >>> **u/None** [+0] (a day later) >>> >>> I sent you a pm >>>> **u/None** [+5] (6 days later) >>>> >>>> /u/amebold , /u/MugaSofer >>>> >>>> [Plot](#s "Mills works with/under Terry. Mills is an astronomy and cosmology enthusiast, and Terry is a physicist that spends a great deal of time reviewing papers. One day, they have a conversation about a paper that promised a unified theory of physics. Mills is shown to be cynical and pragmatic, Terry is more idealistic and dedicated. >>>> The authors of a paper that Terry reviewed commits suicide, and Terry reveals that he kept a copy of their unfinished paper and notes in a phone call to Mills. He encourages Mills to come to the office and share in the discovery. >>>> Terry completes the theory that the unnamed physicist had been working on, and immediately goes slightly crazy, the key beginning to corrupt his mental patterns. He remains sane enough to recognize this, and begins distracting himself to avoid thinking about the key. The simulators tell him about the danger of a global exception, so he destroys his office in an insane effort to cover up the direction of his work. He realizes that Mills knows what paper he was working on, and so prepares to ambush Mills. >>>> Mills enters, but avoids the ambush and wrestles Terry to the ground. Angry and frightened, he interrogates Terry, forcing him to recall the key - this corrupts Terry's mind entirely. >>>> The simulators contact Mills and explain what happened. >>>> Mills realizes that he has to prevent knowledge of the key from spreading, so that any incidences remain isolated. He tosses Terry's body in the car and drives for the seaside.") >>>>> **u/None** [+1] (a month later) >>>>> >>>>> Whooops, you've misspelled my username and I didn't see this until now. This has helped me understand the story (I didn't catch that [ ] (#s "the italicized text was the scientists communicating with him. I'm guessing they were using concepts instead of words? And if a substantial number of people know the key, the simulation crashes? Also, what did 'fourty-three' refer to?")). Anyway, I thought it was a great story (I really like your writing style and how well you're setting the mood), but language barrier is one hell of a thing. >>>> **u/Zephyr1011** [+3] *Potentially Unfriendly Aspiring Divinity* (2 days later) >>>> >>>> I'm also unclear on parts of this. Would you mind sending the PM to me too? >>>> **u/None** [+2] (5 days later) >>>> >>>> And would you mind sending the PM to me too? >>>> **u/MugaSofer** [+2] (6 days later) >>>> >>>> *raises hand hesitantly* >>>> >>>> I heard there were PMs? Are they free? > **u/MugaSofer** [+29] (16 hours later) > > [Fantastic, or, Reed Richards is Useless](https://pseudonymwrites.wordpress.com/2015/08/20/fantastic-or-reed-richards-is-useless/) > > 2077 words > > EDIT: Wow, I'm really flattered - I knocked this out over the course of about half an hour. Thank y'all! >> **u/Kishoto** [+8] (17 hours later) >> >> [](#s "Did....did Reed Richards just create Superman?! ") >> >> Great story. It's early but I have a strong vibe that this one is a winner. >>> **u/notmy2ndopinion** [+6] *Concent of Saunt Edhar* (4 days later) >>> >>> I'm not a comic book nerd, but I thought Mad Scientist [spoilers](#s "/Trask created biological versions of Sentinels. Similar to Superman in form yes, but the mission and volition of purple-helmeted robots who want to destroy all supers, because they are x-risks.") >> **u/avret** [+3] *SDHS rationalist* (a day later) >> >> I suppose that's one way to avert MAD. On the other hand, this makes Richards a target. >> **u/RMcD94** [+2] (7 days later) >> >> Could do with a proof read > **u/Kishoto** [+19] (a day later) > > [SHE](https://kishoto.wordpress.com/2015/08/20/she-rrational-challenge-science-is-bad/) > > 2579 words >> **u/avret** [+3] *SDHS rationalist* (5 days later) >> >> I was kind of wondering when someone would post the first uFAI story. As always, you've posted something spectacular. >> >> Quick question--the AI's goalset prior to integration was to salvage refuse, right? What caused the change to preserving humanity? DId integrating other AI mean taking in part of their utility functions but not the other, more friendly parts? >>> **u/Kishoto** [+2] (5 days later) >>> >>> [Well, that's an interesting question...](#s "Basically, the main AI, after increasing its computational power massively after integrating so many other AI, got to a point where it broke down its 'salvage refuse' goal. That is, it looked at WHY it was salvaging refuse, the answer being to preserve Earth, so humanity would have a home to live in. It then came to the conclusion that there were so many threats to Earth's current condition DUE to humanity's large presence in the world. Plus all of the different countries meant it was hard to keep people in line properly. So it saw one of its only options being the preservation of humanity via Integration. That way, the Earth is left empty of humanity and SHE is free to terraform as she will, to preserve it. SHE also is looking forward into the future and planning for any natural disasters, such as ice ages and asteroid strikes. As it were, humanity's too dispersed and plentiful and not ready for a real disaster. So SHE finds it easier to just store them all in the Server, which isn't all that big relatively, probably about the size of a very large Walmart. It's easier to protect a Walmart than to try and protect the splintered nation divided mess humanity is. ") >>> >>> [](#s "And SHE thinks that it IS being quite friendly. It's preserving humanity against itself, storing them in a paradise. The scary part is that SHE doesn't really understand the sort of injustice that is tampering with our feelings and memories. Since AI were never designed to have such direct control over humans, there's no protocol in any of her subprocesses that says 'hey, wait, maybe humans would prefer if you just stayed out of their heads'. If there was enough public outcry in the Server about it, SHE would stop. But considering she detects negative feelings and is quick to reroute the neuro-chemistry of said feelings, there won't be any time for an outcry to develop. After a while, humanity is just going to be floating around in the equivalence of a very good drug trip, always feeling great and happy, having had the capacity to feel anything unpleasant removed against our will. ") >>>> **u/avret** [+2] *SDHS rationalist* (5 days later) >>>> >>>> Is that the kind of thing that happens in a mind, AFAWK? Do minds typically deconstruct their terminal goals to form new ones?(I'm honestly curious) >>>>> **u/Kishoto** [+1] (5 days later) >>>>> >>>>> I'm not fully certain. It's the type of conclusion I could see a rational mind that has goals and utility functions making this sort of choice, especially if they're lacking things like empathy or respect for the sanctity of the human mind. A cold AI with the goal of "preserve humans" as opposed to "preserve humanity in a form matching what they would like to be preserved as" could do something like this, from my perspective anyway. >>>>>> **u/avret** [+3] *SDHS rationalist* (5 days later) >>>>>> >>>>>> I agree with that perspective, I'm just uncertain how SHE would get from 'salvage refuse' as a terminal pre-programmed goal to 'preserve humans'. >>>>>>> **u/Kishoto** [+3] (5 days later) >>>>>>> >>>>>>> To me, it's thought process looked something like this (although this is heavily simplified): >>>>>>> >>>>>>> Salvage refuse -> what's the goal of salvaging refuse -> preserve Earth -> what's the point of preserving Earth -> to provide humanity with a home to live on -> what's the point of this home? -> to fulfill their needs -> what are their needs? -> (insert fancy week long robot debate) to be happy and content -> what makes them happy? -> (insert chemicals that incite "good" feelings in humanity->how can I administer these chemicals -> (insert more robot debate, output of which is super happy drug) -> how can I administer this to all humans at all times -> (more debate) how will I keep them alive if happiness and pleasure is overriding their basic survival needs -> Integration Plan >>>>>>>> **u/avret** [+3] *SDHS rationalist* (5 days later) >>>>>>>> >>>>>>>> Do minds typically do that to their terminal goals? >>>>>>>>> **u/Kishoto** [+1] (5 days later) >>>>>>>>> >>>>>>>>> SHE did. Can't speak for any other minds, I'm not versed enough in biology or psychology to say. It seems like a logical enough chain of reasoning for me to insert it into my rational story. >>>>>>>>>> **u/avret** [+1] *SDHS rationalist* (6 days later) >>>>>>>>>> >>>>>>>>>> Ok. >>>> **u/MugaSofer** [+1] (6 days later) >>>> >>>> >Basically, the main AI, after increasing its computational power massively after integrating so many other AI, got to a point where it broke down its 'salvage refuse' goal. That is, it looked at WHY it was salvaging refuse, the answer being to preserve Earth, so humanity would have a home to live in... >>>> >>>> Huh. Did the programmers anticipate it might go rogue and deliberately include this feature? >>>>> **u/Kishoto** [+3] (6 days later) >>>>> >>>>> The ability to deduce and critically examine its goals is a feature it developed on its own after absorbing enough AIs. It looked at its salvage refuse goal, sought to execute it as optimally as possible, and the attempts to optimize it, along with some narrative magic, meant it developed the ability to deconstruct its goals, in an attempt to keep with the "spirit" or motivation behind the goal, instead of following it verbatim. >> **u/None** [+2] (a day later) >> >> Oh no SHE di'nt! >> >> I'll show myself out, shall I? >> >> Great story! Is there any reason for the precise number of days, or even just the ballpark the figure landed in? >>> **u/Kishoto** [+3] (a day later) >>> >>> No real reason, just a random number chosen to [](#s "show that SHE moved quickly, but not too quickly. If you think of the implications, like the fact that she never sleeps and her insane computational power, you wonder what could've taken that long. That was the time she spent sorting and organizing, creating her own 'self' per se, coming to her Integration conclusion and then constructing the Integration robots.") > **u/None** [-1] (5 hours later) > > [deleted] >> **u/avret** [+2] *SDHS rationalist* (14 hours later) >> >> Perhaps I'm missing something, but what is the relation of this story to the prompt? >> (Excellently written, btw) >>> **u/electrace** [+3] (15 hours later) >>> >>> >Perhaps I'm missing something, but what is the relation of this story to the prompt? >>> >>> [It's easy to miss but,](#s "humans were the ones who made these things. Presumably, by using science. I can't really think of any other way that you could make something that takes control of a human body.") >>> >>> [Also, from the link in the OP,](#s "The typical theme is that some sort of advanced scientific research has Gone Horribly Wrong, creating a monster...") >>> >>> >(Excellently written, btw) >>> >>> Thanks! It's the first one I've written here. >>>> **u/avret** [+2] *SDHS rationalist* (16 hours later) >>>> >>>> Ah, ok. Maybe make that a bit more clear? >> **u/None** [+1] (5 hours later) >> >> Permission access need changing. >>> **u/electrace** [+1] (8 hours later) >>> >>> Sorry. It should be fixed now. >> **u/FishNetwork** [+0] (6 hours later) >> >> I'm getting a permissions error :(. >> >> Could you double-check your settings? >>> **u/electrace** [+1] (15 hours later) >>> >>> Sorry. It should be fixed now. ---