Files

21 KiB

Dark Side Rationality: Taking Advantage of People's Non-Rationality

Post:

Link to content

Comments:

u/eaglejarl [+12] (23 seconds later)

The rationality here is with the game designers, not the players.

u/ArgentStonecutter [+8] Emergency Mustelid Hologram (25 minutes later)

That's pretty impressive, but consider that Star Citizen has taken in $77M for virtual starships and hasn't even shipped yet.

http://www.wired.com/2015/03/fans-dropped-77m-guys-buggy-half-built-game/

The United Empire of Earth Navy caused quite a stir last November when it announced that it would be putting 200 decommissioned Javelin Destroyers up for sale. Each 1,132-foot-long spaceship has the sort of amenities that your average interstellar mercenary finds hard to resist: four primary thrusters, 12 maneuvering thrusters, a heavily armored bridge, private quarters for a captain and an executive officer, six cargo rooms, general quarters for a minimum of 23 crew members, and a hangar big enough to accommodate a gunship. There's even a lifetime insurance policy.

The document that announced the Javelins' impending sale took pains to stress that these warships were fixer-uppers. “They are battle-hardened and somewhat worse for wear,” it read, “and have been stripped of the weapons systems.” Thus, any would-be buyer would eventually have to shell out extra to equip the 20 gun turrets and the two torpedo launchers. The asking price for each ship: $2,500. And that wasn't some form of fictional futuristic space bucks; it was 2,500 real dollars. Actual, real, present-day American Earth dollars.

Despite those caveats, all 200 Javelins sold out. In less than a minute.

u/Transfuturist [+6] Carthago delenda est. (12 hours later)

If Star Citizen fails, it will be the biggest Early Access shitstorm yet.

u/ArgentStonecutter [+3] Emergency Mustelid Hologram (19 hours later)

I'm ready

u/Kishoto [+5] (an hour later)

I also seek to be able to do this. I'd love to learn enough about the errors in the average humans way of thinking to turn a profit, be it monetarily, socially or otherwise. Does the fact that the thought of being that sort of puppeteer is an exciting one mean I'm a little messed up?

u/daydev [+10] (5 hours later)

We all love "munchkining the system" here. But it's one thing to twist the rules of an impartial arbitrary magic system, and it's a very different thing to exploit people through evolutionary "bugs".

u/Newfur [+3] Crazy like a fox. Literally. (6 hours later)

Why so? I feel like playing devil's advocate today, and I'm interested to hear you elucidate your reasoning.

u/eaglejarl [+8] (7 hours later)

I won't speak for daydev, but one argument that could be raised is: munchkining magic systems doesn't hurt anyone. Munchkining evolutionary bugs actually does hurt people -- for example, in the linked article you'll see that gamers spent thousands of dollars on the game. That money could have been used to help a lot of people, but instead it was spent on virtual goods that were then destroyed; if people had skipped playing the game and just set a giant stack of money on fire the result would have been less harmful, because the opportunity costs wouldn't have been paid.

u/redrach [+3] (8 hours later)

Well to be precise no money is being set on fire, even metaphorically speaking. It's all going to the makers. What's really happening is they've managed to sell an enormous number of people extremely perishable virtual goods.

u/eaglejarl [+2] (10 hours later)

Fair point. Still, the general idea applies -- the players could have made the world a better place with that money, but instead they gave it to a corporation.

u/None [+2] (18 hours later)

[deleted]

u/traverseda [+4] With dread but cautious optimism (23 hours later)

On the meta level, we don't want people who identify with the broader rationalist/lesswrong community doing that. It makes all of us seem less trustworthy. Utilitarianism is scary, idealism and love and tolerance and puppies aren't.

u/Ishamoridin [+1] (5 days later)

This is solved by playing a level above and keeping your rationalist nature a secret. It's probably a good idea in general, to be honest, unless you're teaching others to be more rational.

u/traverseda [+1] With dread but cautious optimism (5 days later)

Raising the sanity waterline is an important goal, which means we need high status rationalists, and good public perception of rationality in general. We don't want another bavarian illuminati.

I'd like a different group identity for people who lean in unwholesome directions. I'm not saying remove the witches from our community, but it would be good to make some big distinctions between US and THEM, even if we use a lot of the same resources and generally get along.

u/Ishamoridin [+1] (5 days later)

There are various methods to accomplish that though. There's definitely a case for exploiting irrationality to make money and then using that money to promote rational thought in some concrete manner that I can't think of. Hell, even if you do it and just keep the money you're disincentivising irrationality.

As far as bad PR goes, as long as the people engaging in 'unwholesome' applications of rationality don't proclaim themselves rationalists then I'm not sure I see much of a problem. Anyone who manages to connect the dots enough to realise they're rationalists is probably something of a rationalist themselves.

u/traverseda [+1] With dread but cautious optimism (6 days later)

As far as bad PR goes, as long as the people engaging in 'unwholesome' applications of rationality don't proclaim themselves rationalists then I'm not sure I see much of a problem.

I'd also recommend people who do call themselves rationalists publicly disavowing that kind of thing...

u/Ishamoridin [+1] (6 days later)

If called upon, sure, but otherwise it's enough to just not fall for it and ignore it. Unless you're implying that this subreddit qualifies as 'public' but using doublespeak here would somewhat inhibit discourse.

u/Chronophilia [+3] sci-fi ≠ futurology (a day later)

I know you're not being serious, but that gave me chills. How self-absorbed and self-righteous would a person have to be, to think that they are that much more intelligent and moral than the average? That even accounting for the losses in the system, it'd still be better for the world than working in charity yourself and letting everyone else spend their money how they feel it should be spent - because the general, irrational public is just so irrational and so immoral and so far beyond help that they shouldn't be allowed to make spending decisions that affect the world? And you can make those decisions, because you're far more intelligent than everyone else?

I've always thought the textbook description of a utility monster was unrealistic, and now I've found the example I was missing.


But just in case you're actually thinking of doing this: making a profitable company from scratch is far from trivial. Behind every billion-dollar Farmville and Candy Crush there's ten thousand games that barely recouped development costs and a million games that crashed and burned. Even if you think you've got a brilliant revolutionary idea that can't possibly fail... well, that describes half of Silicon Valley, and most of their startups still don't work.

u/eaglejarl [+3] (a day later)

Much more than half.

u/None [+2] (a day later)

[deleted]

u/Chronophilia [+2] sci-fi ≠ futurology (a day later)

Do you not think it plausible that someone might see enough evidence of their intelligence and altruism relative to others' that they can be confident in the statement that they are "much more intelligent and moral than the average"?

No, I really don't. Rationality is not about being better than everyone else, it's about recognising and overcoming one's cognitive biases. And several of those biases are actively pushing in the direction of "you're better than everyone else" - the Dunning-Kruger effect, the bias blind spot, the way your own moral code is the standard by which you judge everyone else's so by definition you'll always be the most moral person, etc.

The wisest man is the one who knows he knows nothing, and the easiest way to lose your rationality is to assume that you've got it.

Anyway, the number of people who could actually be a truly benevolent dictator is massively less than the number of people who could become an evil but misled dictator, to the point that even if you've spent a long time self-reflecting and analysing your motivations it's still more likely that you're in the second group. And even if you're not... is it worth sending the message that taking over people's lives for them is OK, as long as you're more moral than them? Do we want scientologists and creationists to take home that lesson?

u/None [+2] (a day later)

[deleted]

u/Chronophilia [+1] sci-fi ≠ futurology (a day later)

I've lost track of what we were arguing about - playing devil's advocate is like that - so I'm going to call it here. Good talk, good talk.

u/ArgentStonecutter [+1] Emergency Mustelid Hologram (a day later)

What's the corporation spending it on?

u/eaglejarl [+1] (a day later)

Almost certainly not charity. Salaries, mostly. The people earning those salaries might use it for good, but you've taken X dollars and reduced it by the corporate tax rate plus the income tax rate. The government takes the tax money and does mostly things that are far from optimally benefiting the world.

u/ArgentStonecutter [+1] Emergency Mustelid Hologram (a day later)

The government takes the tax money and does mostly things that are far from optimally benefiting the world.

Much as I despise things like the F35 and the War on Drugs, there are some things that pretty much only governments can do.

u/eaglejarl [+1] (a day later)

I did say "mostly". Point being, this guy spent $9000 on that game, and the vast majority of it went to overhead and wastage.

u/ArgentStonecutter [+2] Emergency Mustelid Hologram (a day later)

All of it went to someone who spent it on something. Even the waste.

u/elevul [+1] Cyoria Observer (2 days later)

It depends on whether the players have had any benefit from the game, though. My F2P addiction parted me of a lot of money, but it had also taken me out of deep depression and suicidal thoughts in a way that no psych had managed to.

u/daydev [+5] (14 hours later)

Well, rationality presumes that we should "rise above" our flawed biological hardware. So it's kind of hypocritical to pronounce that everyone should overcome the evolutionary bugs, but in the meantime take advantage of those who haven't.

And it's "not fair" precisely because they are bugs. It's like hacking computer systems. It may be fun as intellectual exercise, but if you then proceed to steal people's money through hacked systems, you're a crook. So pranks for fun may be OK, but cons for money are not, and this kind of freemium is almost an industrialized con. It's not exactly the same, since you're actually getting what you're paying for, but they arrange the system to suck you into paying for things you never would've wanted to pay otherwise. It's like middle ground between extremely aggressive marketing and cons.

Also, as eaglejar said, you can't hurt the universe by exploiting its rules to get an advantage. But you can hurt people by exploiting their wiring to get and advantage.

u/Newfur [+1] Crazy like a fox. Literally. (16 hours later)

Clearly this is so - but what would you do in the least convenient possible universe where the only way to avoid getting conned in the long run is to anonymously con a few people first in order to understand and interdict the major lines of attack, and in which simply dicking around with easily exploited magic systems does nothing for this skill?

u/None [+2] (20 hours later)

It's kind of like what I've been told reading Thinking Fast and Slow is like: you think you know the tricks and will see them coming, but then they're used against you, and you totally fall for it every time.

u/None [+1] (21 hours later)

Does the fact that the thought of being that sort of puppeteer is an exciting one mean I'm a little messed up?

Depends precisely which sort of puppeteer you want to be.

u/Newfur [+0] Crazy like a fox. Literally. (2 hours later)

Like hell! Every skill worth the name ought to be able to be leveraged into some or other variety of benefit, and at the end of the day, every competition is a zero-sum one. The fact that you might get to turn the fortitude of your mind into toying with those who chose not to is just gravy.

u/alexanderwales [+9] Time flies like an arrow (4 hours later)

Every skill worth the name ought to be able to be leveraged into some or other variety of benefit, and at the end of the day, every competition is a zero-sum one.

I don't see how this is true. A zero-sum game is one in which the gains and losses of all the players are balanced against each other. If you and I both go hunting a single deer, only one of us can get it ... but at the end of our competition, one of us does have a deer, and the sums of our gains and losses are not zero (because you have not "lost" a deer).

Or perhaps as a better example, you and I enter into a pie-baking contest. Both of us will have gained (by at least one pie) at the end.

u/Newfur [-2] Crazy like a fox. Literally. (6 hours later)

OK, fine, mixter pedantic, they're constant-sum, which is tantamount to being zero-sum and in a low-constant regime, as almost every competition is, may as well be zero-sum. :P

u/Kishoto [+2] (3 hours later)

Can I say I love your wording in this reply? :)

u/Newfur [+2] Crazy like a fox. Literally. (3 hours later)

Why, thank you!

u/None [+1] (4 hours later)

[deleted]

u/Newfur [+1] Crazy like a fox. Literally. (4 hours later)

Well, yes. And cooperation with worthy allies is the noblest of states. But one doesn't always find oneself in such beautiful situations, and sometimes people delude themselves into thinking that there are indeed competitions(*) that somehow aren't zero-sum, that have no losers.

u/eaglejarl [+1] (6 hours later)

people delude themselves into thinking that there are indeed cooperations that somehow aren't zero-sum, that have no losers.

Say that Joe and I cooperate to produce a website--I do the programming, he does the design. At the end of the process we have a great website. Isn't that a non-zero-sum cooperation?

u/Newfur [+1] Crazy like a fox. Literally. (7 hours later)

Apologies, typo there. Meant competitions.

u/eaglejarl [+3] (7 hours later)

Ah. That makes more sense. I still don't agree, though -- lots of competitions produce new wealth. A good example would be the browser wars; Microsoft and Netscape went at it hammer and tongs, then Opera / Firefox / Chrome came on the scene and today the Web is way more powerful and easier to (use | build for) than it was fifteen years ago.

u/Newfur [+1] Crazy like a fox. Literally. (16 hours later)

I'm not sure I would call that an actual competition so much as a really strange and distributed kind of cooperation.

u/ArgentStonecutter [+1] Emergency Mustelid Hologram (a day later)

It's called "the market".

u/japr [+1] (13 days later)

He has a point, though; the "competition" of the market actually relies upon both internal collaboration within each company AND building off of the successful ideas of your "competitors" to create the next wave of breakthroughs.

We call it competition, but viewing it in that way actually slows down our potential progress to some degree via bottlenecking the information flow that leads to said breakthroughs.

u/Chronophilia [+1] sci-fi ≠ futurology (a day later)

and at the end of the day, every competition is a zero-sum one

Technically correct, but life is not a competition.

u/FuguofAnotherWorld [+4] Roll the Dice on Fate (35 minutes later)

It's the perfect storm. I've read the theory the makers of this kind of thing used before, and it's fascinating but this one takes it one step further. It seems they've advanced to the pinnacle of manipulation through flashing numbers. It's like a dollar auction mixed with the sunk cost fallacy, classical conditioning and an ocean of evaporative strengthening of peers.

u/Farmerbob1 [+5] Level 1 author (4 hours later)

People are looking at electronic games and seeing this behavior, but the same behavior is far older that any electronic games.

Gambling. Lotteries. The house always wins in the end.

u/eaglejarl [+6] (5 hours later)

There's some differences, though. Take a look at, for example, blackjack:

  • Pay to play; you know you're spending money if you sit down
  • Can win money, and the money comes from the house
  • Players don't interact [1], and can't gain or lose money from/to each other
  • Can leave at any time, no consequences
  • When you leave, you keep all your money and whatever you've won from the house.

Now look at the linked game:

  • Sit down for free, but lures you into spending money
  • Can't win
  • Continually interact with other players by costing them money
  • Can't leave without cost
  • When you leave, you steadily lose all the money you've put in

The house's money is never at risk, and the player can only lose, not win or break even. Furthermore, players are directly pitted against one another to make each other lose faster.


[1] Technically blackjack players interact to the effect that if I take a card then the next person gets a different card. That's not generally relevant, though.

u/Farmerbob1 [+3] Level 1 author (13 hours later)

There are differences, but in both cases the victims have been fooled into engaging in a contest that is either unwinnable, or has an extraordinarily low probability of winning decisively.

'Winning' is a practically unattainable goal, but a lot of people will see the goal and not understand the impracticality of it.

A sufficiently unscrupulous rational individual can implement 'entertainment' that will attract irrational people who are attracted to winning.

Is there a functional difference between a tiny chance of winning and no chance of winning?

No, not for most individuals. Most 'winners' of games of chance end up losing more than they win before they are 'lucky' and win big. Occasionally, some newbie gambler will win big after just a few hands or a few slots, but that's rare. It has to be. If it wasn't, the house would lose money.

The game specifically referred to is simply a gambling operation with no way to win against the house. You can still win against other players.

A lot like poker.

u/darkflagrance [+1] (8 hours later)

The big difference is the regulation (or lack thereof)

u/darkflagrance [+3] (an hour later)

The players burn each other's money. What's not to love about this game design? It's beautiful.

u/Kishoto [+1] (an hour later)

This video speaks of something similar. It's a very interesting watch, and it also examines how this works on a biological level, to a degree, without being overly technical.