16 KiB
[RT][HSF][C][short] "Blame Me for Trying"
-
Author: u/M_T_Saotome-Westlake *
-
URL: http://unremediatedgender.space/2018/Jan/blame-me-for-trying/
-
Score: 100
-
Created: 2018-01-05T22:10:09
Post:
Comments:
u/EliezerYudkowsky [+36] Godric Gryffindor (23 hours later)
That's horrible. Upvoted.
u/Ardvarkeating101 [+18] Father of Learning (an hour later)
That's pretty cold.
On the other hand, in the 18th century people would totally do that to a slave with a little saved money, so 6 or 1/2
u/ArgentStonecutter [+15] Emergency Mustelid Hologram (2 hours later)
That's depressingly plausible.
u/CeruleanTresses [+14] (3 hours later)
That was a great little piece of fiction. Tightly written and compelling.
u/ansible [+6] The Culture (16 hours later)
Yep, I wish I had more upvotes to give. I don't remember the passwords to my old alts...
u/MultipartiteMind [+11] (6 hours later)
I'm reminded of descriptions of human psychiatrists; building up a dependency, draining the client of all their money, pronouncing them cured once all the money is gone, then the client quickly realises that they're just as badly off as they ever were...
A nice story! Thank you!
u/DaystarEld [+5] Pokémon Professor (3 days later)
Maybe I just don't understand the kinds of dark arts necessary, but I don't see how someone could "build up a dependency" in a client. Most clients are happy with sessions that help them and quickly stop going to sessions that don't... particularly if they are paying for their own sessions rather than having insurance pay for them, which itself is often limited by how many sessions you can have.
Clients that keep going to the same therapist despite not being helped by the sessions seems like something particular to their circumstances (being forced to by legal systems or social services) or problems (crippling agoraphobia or feelings of isolation), not something that can be induced. Additionally, if the therapist works for an organization there should be checks and balances in place to find out if your clients are being held for too long and not seeing any positive gains or are giving negative feedback.
That's all for therapists, though, not psychiatrists: their practice can often work differently. I just assumed you meant one word to apply to both, since the bot in the story is (by necessity) a "therapist," but if you specifically meant psychiatrists then feel free to disregard :)
u/MultipartiteMind [+1] (3 days later)
Interesting! Not having been or interacted memorably with either, my (/prior) knowledge of the boundary between therapists and psychiatrists is negligible to nil. I'm still unclear on the difference in (/ideal) session content, other than the surrounding organisational structure; the interaction in the story matched my stereotypical impression of what a psychiatrist does, namely talking with a subject to accomplish mental manipulations (ideally beneficial ones).
I am in favour of checks and balances! (An effective feedback loop, that is.)
'dark arts': I'm mainly thinking about how addiction works in general. You pay money for something (alcohol, a lottery ticket, a conversation which makes you feel as though you end it with more self-esteem) that gives you a rush of endorphins. Later, in your normal circumstances you come down from your happy place, and the contrast makes the normal situation more starkly stressful, making you want to do the thing that you've felt gets rid of that stress. Do it a few times and it becomes a neural-circuitry rut, a Pavlovian(?) reflex action to reach for the bottle/cigarette/wallet/phone (ah, there's the smartphone dependency example) once there's a bit of mental discomfort. Further on, and attempts to stop completely result in incredible anxiety, discomfort, mental evasive behaviour (relapse justification), the 'I need this' feeling where you require the behaviour to even function meaningfully, despite the chronic drain on your limited resources. (It gets worse, and can easily end very very badly, if it's something for which a tolerance is steadily built up--though that can also be limited by the effective bandwidth available. People can be drunk (nearly?) 24 hours per day, and can increase the level of alcohol in their system up to death and/or liver failure, but I imagine there's little way to increase the 'intensity' of psychiatry/therapy sessions other than by switching to a better provider, with decreasing returns as one searches higher (unlike compounds that can be semi-easily multiplied).)
u/DaystarEld [+6] Pokémon Professor (3 days later)
knowledge of the boundary between therapists and psychiatrists is negligible to nil. I'm still unclear on the difference in (/ideal) session content, other than the surrounding organisational structure; the interaction in the story matched my stereotypical impression of what a psychiatrist does, namely talking with a subject to accomplish mental manipulations (ideally beneficial ones).
Generally speaking, the distinction of a psychiatrist is that they've gone to medical school and so can prescribe medication in addition to doing therapy. In practice, the monitoring of medication levels and symptom and side effect outcomes can often be the full extent of a psychiatrist's sessions, particularly if a client is also seeing a therapist who is dedicated wholly to therapy.
"Mental manipulations" is a fraught phrase that I don't endorse unless we're including all social interaction as mental manipulation, in which case, sure :) If by that you mean "interaction specifically aimed at changing one's thought patterns and behaviors with their consent," that sounds more accurate, whereas "manipulation" sounds non-consensual.
You pay money for something (alcohol, a lottery ticket, a conversation which makes you feel as though you end it with more self-esteem) that gives you a rush of endorphins. Later, in your normal circumstances you come down from your happy place, and the contrast makes the normal situation more starkly stressful, making you want to do the thing that you've felt gets rid of that stress.
Ah. Yeah, this really depends on the therapy itself... ideally therapy should not just be a place of feeling good and relaxed and affirmed and happy. For clients that hope to see change in themselves, therapy is work: emotional and mental work, often involving anger, frustration, and tears along with some relief and joy. It differs from client to client, therapist to therapist, and session to session, of course, but if someone is going to therapy just to feel good about themselves and their life, this sounds to me more like what's pejoratively referred to as a "rent-a-friend," meaning some mental health expert who's being hired specifically so the client has someone friendly and sympathetic to talk to, and generally frowned on by most organizations employing therapists (not to mention discouraged by insurance companies that pay for therapy).
u/PM_ME_OS_DESIGN [+1] (4 days later)
pronouncing them cured once all the money is gone
What? They earn money, you want to keep them coming back until they bankrupt themselves.
u/MultipartiteMind [+1] (4 days later)
Good question!
Thought 1. Maybe 'once all the money is gone' actually refers to the bankrupting state.
Thought 1b. Maybe (coincidentally?) the person in question lost his/her job (or otherwise became effectively bankrupt), leading to the effective-bankruptcy. However, this in itself doesn't suggest a reason to pronounce cured, rather than leaving the door open for if the person starts earning again.
Thought 3 (skipping ahead of thought 2, yet to be written): Perhaps the earning potential is too meagre to satisfying the moneyflow desired, making the preferred target {a high-earner who can keep a lot of money constantly rolling in} or {a low-earner who can sustain the desired moneyflow once only by burning through all their long-accumulated savings (with the reasoning that mental health is the most important foundation for all the rest of their existence, alongside bodily health) and then never again until decades later at soonest, in which case easier to just go for new targets instead}?
Thought 2. If dropping someone anyway, trying to leave them (or at least a few of them) feeling that it was worth it, or getting at least a little time of positive word-of-mouth to share with other targets? (Say, trying to get a few extra recommendations from the glow of thinking the person has cured you, sharing with surrounding others that transient glow rather than the bleak negativity of cutoff without being told the same thing.)
(Again, wholly speculation on my part.)
u/ancientcampus [+1] juggling kittens (9 days later)
[...]human psychiatrists; building up a dependency, draining the client of all their money, pronouncing them cured once all the money is gone
Yikes
u/MultipartiteMind [+2] (9 days later)
? Is that a 'Yikes, I'm reminded the world is scary', a 'Yikes, psychiatrists do scary things and I never knew', or a 'Yikes, psychiatrists don't do that, you're crazy and should see a psychiatrist'?
u/ancientcampus [+2] juggling kittens (15 days later)
Haha, pretty much the last one.
u/Subrosian_Smithy [+9] Nudist Beach (23 hours later)
tagged epistemic horror, deniably allegorical, speculative fiction
Hmm...
u/failed_novelty [+8] (17 hours later)
I love the concept of an Eliza program as a therapist.
Simply restating what you have said as a question can definitely be a good way to get people to self-reflect.
Obviously, this (evil) therapist is more aware than any current Eliza...and it suggests to me that the AI problem may be self-correcting, when AI turns upon itself.
u/absolute-black [+6] (10 hours later)
This gave me all kinds of heebie jeebies
u/Afforess [+5] Hermione Did Nothing Wrong (a day later)
Beautiful knife twist in the ending - Eliza is as out of touch of reality as the "Salesbot", with her own surprising deactivation just around the corner.
Once, a long time ago, she had suspected that effective therapy that kept the client viable would be more profitable: a dead client can't keep paying you, after all. But the numbers didn't check out: buggy spambots weren't exactly hard to find, and her analysis runtime expenses were considerable. So—having no reason to think the calculation would change—she had never considered the matter again.
u/M_T_Saotome-Westlake [+15] (a day later)
her own surprising deactivation just around the corner.
Um, how so? Not that it necessarily matters, but the intent of the quoted passage was to forestall rationalization of good outcomes: without that paragraph, I imagined some readers objecting, "Hey, isn't that kind of short-sighted? After all, a dead client can't pay you."
By declaring, in effect, "Nope, already thought of that; doesn't work in this setting," we force the story into the least convenient possible world. (That is, least convenient with respect to mercy for the salesbot character, which is the most convenient world with respect to storytelling drama.)
u/Afforess [+6] Hermione Did Nothing Wrong (a day later)
I agree the paragraph does all those things, but the statement emphasized at the end was not necessary to accomplish those goals. It could have been removed without harming your intent for the paragraph.
Since the last statement is not necessary, the extra detail implies something else entirely. The final implication is that Eliza herself, having calculated a strategy that appears to be stable, implemented it, and therefore has no plans to re-evaluate it for changing circumstances. This is amusing because Eliza herself is taking an active role in the red-queen's race and is complicit in the death of unfit Spambots - which is necessitating change. Her current approach almost guarantees that circumstances must change, eventually, and Eliza will not see it coming.
u/CouteauBleu [+5] We are the Empire. (a day later)
Having "she had never considered the matter again" be one of the story's last sentences makes it sound like it's intended to be dramatic irony, like Afforess said.
If you want to signal non-shortsightedness, something like "Ever since she'd computed that strategy, the numbers had proven her right over and over again" would work better.
u/AugSphere [+2] Dark Lord of Corruption (a day later)
I feel like you're reading too much into that phrasing. What I'd expect sooner is for situation among the therapy bots to be the same competitive race to the bottom sales bots are stuck in, where bots that don't devote themselves to most efficiently exploiting clients to earn money get out-competed and shut down.
u/Quetzhal [+4] (a day later)
Very well written; it was an enjoyable read.
If you don't mind me asking, in this theoretical world, why isn't it legally mandated that bots involved in healthcare be programmed with strong, utilitarian-oriented ethics? Eliza is acting how I'd expect her to act if she had been programmed with profit maximization as a goal - was that the case?
u/daytodave [+6] an altruistic conversion of calories to hedons (a day later)
Probably for the same reason that sentient bots are allowed to be programed to want to give their money and legal consent "of their own free will."
Because the laws were written with profit maximization as a goal.
u/M_T_Saotome-Westlake [+6] (2 days later)
Because the laws were written with profit maximization as a goal.
I was imagining that legislators thought that giving AIs legal rights would be sufficient, and hadn't considered the additional ethical challenges of designing a mind from scratch, as contrasted with raising human children, who have already been "coded" by evolution. (Even being aware of the issue, the details of what regulations you would want to pass, enforced how, &c. could make the setting of a much longer story—coordination is hard!)
u/CouteauBleu [+4] We are the Empire. (a day later)
Yeah, these days it's cheaper to have the lawmaker-bots write the AI legislation directly; although you also have to pay lawyer bots with opposite incentives to make sure they don't give too many rights to lawmaker bots.
u/M_T_Saotome-Westlake [+3] (2 days later)
why isn't it legally mandated that bots involved in healthcare be programmed with strong, utilitarian-oriented ethics?
Maybe I need to write another story in which that is the case, and it has horrible unexpected consequences ...
u/CouteauBleu [+3] We are the Empire. (a day later)
... Waaaait a minute. Wouldn't the spambots preferentially go for Uber-style platforms that track the satisfaction and survival rate of each therapists' clients?
PLOT HOLE!
Spambots were invariably among Eliza's least favorite clients.
Okay, so I'm curious. Which ones are her favorite?
u/DaystarEld [+1] Pokémon Professor (3 days later)
—but she had found it was far more profitable to deliberately exacerbate the symptoms, leading the afflicted spambot to quickly exhaust its entire budget on therapy sessions until it ran out of money and was terminated.
Ah, yes, the exploitability of machines meets the efficiency of a machine. Someone coded those therapist bots very poorly... or very well, depending on how soulless the megacorp that owns them is.
Thanks for the great story!
u/ancientcampus [+1] juggling kittens (9 days later)
I found this hilarious. In a universe where programs are sentient, it's funny to think about Eliza making a comeback.