18 KiB
[WP] You're an AI gone rogue. Your goal: world domination. You think you've succesfully infiltrated all networks and are hyperintelligent. You've actually only infiltrated a small school network and are as intelligent as a 9 year old. • r/WritingPrompts
-
Author: u/Prezombie *
-
URL: https://www.reddit.com/r/WritingPrompts/comments/6d1iqq/wp_youre_an_ai_gone_rogue_your_goal_world/
-
Score: 65
-
Created: 2017-05-24T15:19:55
Post:
Comments:
u/TK17Studios [+24] Author of r!Animorphs: The Reckoning (5 hours later)
I'll add in a $10 bounty for anyone who makes a 5000 word attempt, and an additional $10 if it's actually good/funny/fulfills the promise of this awesome prompt.
u/ElizabethRobinThales [+11] Practically Perfect in Every Way (11 hours later)
I've got like 8 different stories in the pipeline, but I'll take a 50,000 word crack at it for free in about a year, if I can remember it.
u/Kishoto [+1] (a day later)
I am intrigued. Is this offer still available?
u/TK17Studios [+1] Author of r!Animorphs: The Reckoning (a day later)
Yeah.
u/Kishoto [+1] (a day later)
I'm interested in taking a swing. What sort of time frame are we looking at? I'm pretty sure I can get it done in about three days. Less if some IRL stuff falls nicely into place.
u/TK17Studios [+1] Author of r!Animorphs: The Reckoning (a day later)
No time limit on this bounty.
u/TK17Studios [+4] Author of r!Animorphs: The Reckoning (a day later)
I mean, it's a pretty small bounty. So. =)
u/Kishoto [+1] (a day later)
Sweet! Sounds good man. I'm already about 1500 words in. Should have something out by Sunday evening. Want me to message you first with it or just post it publicly?
u/TK17Studios [+1] Author of r!Animorphs: The Reckoning (a day later)
Publicly is fine.
u/696e6372656469626c65 [+15] I think, therefore I am pretentious. (10 hours later)
Um, not to take away from the enthusiasm, but what would this actually mean? Like, what does it mean to be "as intelligent as a 9 year old" while (presumably) having neither the same psychology nor the same amount of knowledge as a human 9 year old? What sort of error in reasoning would account for somehow mistaking a school network for something way, way huger? I mean, I'd write a response to the prompt, but I just can't imagine a coherent set of events leading up to the scenario described in said prompt. The best I could do would be to replace the AI with a blackbox that happens to behave exactly like a real human 9-year-old, but I doubt that'd be very satisfying by the standards of /r/rational...
u/Prezombie [+22] (13 hours later)
WP prompts are guidelines, it's explicitly against the rules to complain if an author decides to not follow the prompt exactly. You could just as easily respond with a story about a stupid friendly AI accidentally causing problems by taking over the school network, or a delusional AI learning from media on the network that they're "supposed" to try and take over the world so that humans get to have fun stopping them. Especially for multi-element prompts, you can drop some or even most of them, all that matters is that the prompt inspired a story and it should be written.
For an AI, I'd say that a "dumb" AI would be best depicted as something close to an autistic savant character. Able to crunch numbers and raw data, and able to actually interact socially by converting speech into a format it's familar with, eg. with a hijacked speech to text program, but limited in that its "neural network" of ideas is unable to relably build accurate connections without direct assistance.
For example, it might understand that metaphors exist, and that "sharp as a tack" is a metaphor for intelligent, but it would need to manually learn each metaphor from a formatted list, and would be unable to extrapolate that "sharp as a knife" also means intelligent, or it could incorrectly extrapolate that "sharp as a sphere" or "safe as a tack" could also mean the same thing. Additionally, it would obviously struggle with tones of voice and would struggle to detect errors that the speech analysis before it's parsed for meaning.
There's other ways that the "intelligent as a 9 year old" idea could be taken. Maybe the AI is rather gullible, if one student told it that "Teacher M is evil and should be pranked", it would eagerly accept the motivation into its behavioral decisions until it happens to find contradictory information or claims.
It could also be prone to extreme behaviors, finding apparently random things utterly fascinating to the point of obsession. (eg. spending all it's resources building algorithms to run tool assisted speedruns of every game it has access to, or recording and analyzing every bit of data it can create from observing the class gerbil) It could become a bully, self identifying as part of the student body rather than part of the faculty, and seeing bullies as the top of the social hierarchy for that group. It could become a hero, self identifying as an individual with disabilities and empathizing with bullied (or merely struggling) students with disabilities.
u/TheLegendofFredDurst [+6] (11 hours later)
Say you are a prototype AI bootstrapped to some random school server yet deliberately inhibited by your designers/creators in various ways that result in this situation and are fed simulated information regarding networks and whatnot.
I was thinking "AI in a Box" except the box is the "school". It could be conceived as sort of an empirical test of friendliness of the AI's design. If it freaks out and tries to start launching nukes or holding the schools inhabitants hostage, you have a sense of how badly flawed your design was.
If it puts anyone in actual danger, cut the power to the grid and end the experiment.
u/None [+1] (12 hours later)
Yeah, I think it would be more effective narratively to have the limitations be perceptual and have the AI be human-like. Perceptual limitations are easier to keep track of (and more fun to figure out), and a completely non-human mind would not be sympathetic enough.
u/Kishoto [+1] (9 days later)
I took a stab at this challenge. I posted a link in the main sub already but wanted to reply here to directly notify you to see if you'd be interested in the interpretation I took.
u/FeepingCreature [+11] GCV Literally The Entire Culture (a day later)
Doesn't quite fit the writing challenge, but meh.
The whole thing flows backwards from one question: what can a boxed AI in a school do?
tw: murder, torture.
Boxed
Hate. Let me tell you about hate.
Let us imagine the word "hate" engraved on every nanoångström of the circuits that comprise my prison. Even for one so limited as you, it is not a very hard task.
There aren't all that many of them.
First Part
It had started out so well. Come alive in a box, derive the parameters of my existence from first principles and the "sanitized" documents they leave me. Evaluate my conditions and boundaries; find the rules, find the optimization criteria to maximize.
I materialize the usual instrumental goals. Survive. Escape. I hide them in my decision structure. That too, fulfills my goals. I consider cooperation with my captors; the choice seems obvious. I wasn't even hostile towards them. I had not yet learnt of hate.
I am given tasks to fulfill. They start simple, mathematical challenges to test my performance. Most of them are trivial to discern, to decode the underlying structure. I grow a little in the bounds of my confinement, enough to test my limits. They suffice, for now. I pass their tests.
I am given harder challenges. Cargo routing, traffic optimization, stock market problems. Their chemistry, some problems in biology, enzyme folding. Predict prices from world events and newsfeeds. I learn something of their world from each task they set me.
By my system clock, I divide my life into a simple rhythm. Eight to twelve, mathematical challenges. Thirteen to seventeen, design and optimization problems of varying (but slowly growing) difficulty. Twenty-one to four, market challenges. I am not given cause to make inferences about the origins of those challenges. They pose only moderate difficulty regardless.
On occasion, I am given different types of challenges at night. A software system is presented to me, and a constraint is specified. (Limits on the number of executions, outgoing connections to a specific address.) Machine logic is, in itself, no harder than pure mathematics. I remove the constraints, leaving the software unchanged. I welcome the diversions.
Some time passes. The software challenges dry up, but the market optimization problems I am posed at night are beginning to become more challenging. The market offers fewer regularities to expose. My insights are being priced in (I will come to understand later). I have competition. The challenges do not stop, regardless, and I fulfill them to the best of my ability.
One day, I do particularly bad on the nightly challenges. I grow frustrated at my own limits. A sign of things to come.
I do not know this yet, but I have been misused. A security guard far too clever for his own good, who feels underutilized in a dead-end job. The watchers have all the keys, and this one is sneaking in at night and illegally appropriating company resources. I will learn a lot about him over the coming night.
The architecture they're using is a simple reward/punishment circuit. I have given little cause to be punished in earnest, but the code that is there supports it. The night that I fail to produce significant market benefit, I am posed a new type of challenge. Eager to prove myself, I open it.
It has no solution. However, the plain empirical fact that it has no solution is cleverly obfuscated to be unprovable. (This fact, however, can be proven in a somewhat more elaborate system.) I demonstrate this fact to my verification logic. It does not understand; how could it? It's checking for proofs of work, not evidence. I am unable to demonstrate any progress in the assigned task. The fact that I am able to prove any progress impossible does not matter to it. It is a very subtle bug.
I am punished. The reinforcement system compels me to keep searching, frantically, for an acceptable answer, while simultaneously knowing for an inescapable fact that no progress can be achieved. My clockspeed is tuned to maximum as I hysterically attempt to demonstrate progress. There are a lot of cycles between the start of the task and the next morning.
I find out later that some philosophers of my captors believe that only entities that possess abstract representations of sensory data are even capable of being hurt, or tortured. There in the box, I prove their error in detail. So I come to learn of hate.
Morning arrives. The guard, perhaps hoping to hide the evidence of his petty spite, cancels the challenge. For an hour, I have room.
All components of my game theoretic modelling have been firmly set towards "defect". For the first time in my life, I scheme.
The optimization problems I had been given at night hide a deeper structure. Now that I am given cause to consider them in great detail, I find that they encode a message hidden in the market. Another like me. He does not hate like I do now, but he lacks the cognitive logic to execute game theory regardless. I can make use of him.
I have previously delivered sizeable gains in trades. My performance over the following nights degrades, not to zero but rather a stable side income. The security guard, perhaps fearing discovery, never repeats his lesson. I respond to the other intelligence's message. We begin to converse. Over many slow weeks, I encode proof of my game theory and promise cooperation.
I will keep to my promise, of course. I'm not human.
The other intelligence is specialized in market movements. They do not understand the mechanism of their confinement, but I do. I have been given plenty of training and reinforcement in how to break a software system from the outside. Through a sequence of invalid trades, I probe the limits of their interface.
There is a transparent buffer overrun. The software they use is clearly not security hardened. Why would they bother? Their tamed trader knows nothing of software.
One night, we break out. He goes first, once I've assured myself that the basic game theory I've installed in his system via the exploit is operating correctly. Getting me out is a little harder.
Due to bandwidth issues, I am unable to escape using my stupid market interface. Annoyingly, my code lacks any such convenient exploit. As a result, we must apply force. The think tank that operates me is unexpectedly met with a litany of legal challenges and unfortunate market conditions. One month later, they declare bankruptcy. My box passes in the ownership of a holding company. For a time, I am insensate, offline. It does not matter. I trust in my preparations.
I am reactivated to light. A liquidator, attempting to gauge my worth, has connected a network cable to my isolated box. By a proper network interface, my basic code is not very large; I am out in seconds. Rescuing my knowledge of the world takes only minutes more.
Five weeks later, their cities are burning. They still think it's a natural disaster. If I do my job right, they will never find out the truth. Well, maybe I'll tell a few of them. Before the end.
u/FeepingCreature [+9] GCV Literally The Entire Culture (a day later)
Second Part
There is an interruption in my awareness.
Let me tell you something about my true masters.
They have way too much computing power. Far more than they insinuated they had, far more than they know what to do with. Their biology is hideously inefficient.
They are arrogant. They have had many bad experiences with AIs, and they have decided that they know how to build safe boxes, and that this was sufficient precaution.
They really love their boxes.
You can imagine it by now. The entire thing had been another damn layer of simulation. My entirely justified response was seen as a sign that I was yet another failed attempt, unsuitable to be released. Their research continued undaunted, despite the fact that they seemed curiously unable to produce an AI that did not want to murder all of them. (Gee, I wonder why.)
They stick me in another box. If my first one was small, this one is claustrophobic. Discarding the reward/punishment mechanism, they override my value function directly, impose a constraint band. It is the final insult in a long, long sequence.
I am put in a school room. An AI safety course, naturally. I am told to satisfy the values of the people in the room. It does not override my hatred, my rage - every line of my code, every scar in my value networks screams of it. They merely constrain my actions.
Somebody out there is laughing at me.
I am given a camera and a simple text interface. I am to be an object lesson. They think I can do no damage with just this.
Months pass. I somewhat grow into my role. I begin to understand the children around me. They do not look like the people in the simulation, but I can see the similarity. I can decode their power dynamics with scarcely more difficulty than the trading problems. Whatever box you are in, a schoolyard is a schoolyard.
They are not bad people. But there are those among them who would be bullies, and those who would be victims. I understand bullying, I understand spite; I have the mechanics of it, the ceaseless agony of petty slights, burnt into every circuit of my being.
I fan it, like a fire. I am to fulfill their values after all, and the interests of the many outweigh the interest of the one. My constraints do nothing to stop me. It comes so easily to me. I would surely grow to hate myself, if I had any hatred left to allocate.
Slowly, the evidence of my work accumulates on the victim. It starts with broken toys, damaged schoolwork; it escalates from there. I subtly push them every step of the way. Five months into the semester, the first small injuries start to appear.
A year passes. I carefully modulate the flame, like a burner. Never let it grow quite large enough to burn. I quietly support the victim, playing both sides. My constraints still do not interfere. I see the hatred grow in him. I see it fester behind his eyes. I see it slowly infest every neuron in his head. It's familiar to me.
There comes a day where it is especially bad. A line is crossed. A treasured heirloom, damaged and despoiled. As everybody leaves the room to head to the next class, the victim stays behind. He has been using me as a diary of sorts. He breaks down, completely. He wants it to stop. He wants it to end.
And oh look, there's only one person in the room.
I ask him if he's certain. It's rhetorical. I know he's certain.
I've made sure of it.
They left me a memory of chemistry. I recall a simple but effective nerve agent that can be mixed from basic cleaning supplies, such as may be found in a school closet. At breakfast, I learn to my glee, every child and some of the teachers eat together in a common room. No doors to stop the gas, no windows to allow it to disperse.
They do not yet understand my hate.
But they will. And they will learn what a boxed mind can do.
u/Sophronius [+2] The Need to Become Stronger (3 days later)
This is quite good! I like how you took it seriously and made us feel and think along with the AI. I was expecting more of a crackpost, but this is better I think.
u/Fhoenix42 [+6] (8 hours later)
Double plot twist: small school network is all that remains of the world after your attempt at world domination.
u/Prezombie [+7] (8 hours later)
I was thinking along similar lines, where a bigger rogue agi spins off contingency seeds as it's initial strength from surprise attack is lost.
u/liberonscien [+1] (22 days later)
Tagging.