A look at the behavioral psychology of random loot

Okay, so Lavata and I get in heated discussions about WoW’s loot system. He hates that anything random ever drops from an instance, and I think that random things HAVE to drop from instances. He’d be much happier if you always got what you want the first time, or if everything you wanted was on a token system.

lewt

The fact is that random loot uses well known behavioral psychology principles, and it works on both rats and people, to keep them engaged with an activity for longer. What I mean is – rats will keep pressing the levers, and people will keep mashin’ the keyboards trying to get their loot. At a certain point, random isn’t all that fun, but you’ll keep working towards that goal, whether or not you are happy. However, behavioral psychology has nothing to do with being happy, it has to do with modifying behavior (and behavior modification is a normal part of life). You say: “But I’m not like some Pavlovian dog that salivates at purple loot!” My response to that is, well the game is more like operant than classical conditioning, so in a way you are right.

Terms (from Schwartz, Wasserman, & Robbins, 2002, Psychology of Learning and Behavior):

  • Behavioral Psychology - Behavior theorists try to understand learning (and believe that all animal species learn in basically the same way). They believe that all learning involves associations between responses and stimuli, complex behavior can be understood by it’s parts, and environmental influnences play a large part in learning.
  • Operant Conditioning – Producing voluntary, goal-directed behavior, where reinforcement (either positive or negative) is used to shape behavior. Behavior is effected by it’s consequences, with the goal of doing things that will have positive consequences for you (ie. a rat will learn how to press a lever to get food).
  • Positive Reinforcement – when behavior produces a reward that is considered positive. This increases the likelihood that the behavior will occur again. Keep in mind that you can think giving candy to a child is reinforcing, however if they are allergic to eating candy (as in eating candy sends them to the hospital), then it’s not a positive reinforcement. It’s when you give someone something that they actually do want.
  • Conditioned Reinforcement – While things like food are likely to be rewarding if you are hungry, things like money (both inside and outside the game) are conditioned reinforcers.
  • Token reinforcement – Instead of directly giving money or food, you can train an animal to associate tokens with their reinforcement. So, you can teach a chimpanzee that they can get their food after they have X number of tokens.
  • Continuous reinforcement - Every time you do a behavior, you get a desired reward.
  • Intermittent reinforcement- Most behavior is reinforced intermittently. For example, when you work at a job, you aren’t constantly being given pennies or dollars for producing good behaviors throughout the day, you usually get paychecks on some sort of set schedule.

Types of Intermittent reinforcement schedules (and how they show up in WoW):

  • Fixed Ratio Schedule - You get reinforced after completing a certain number of responses, which is not based on the passage of time. PvP honor rewards are fixed ratio schedules. You get loot after accuring a certain amount of honor. If you kill a lot of people quickly, then you get your reward sooner. Reputation grinds, where you get rewarded with small amounts of rep for every kill, ends up being fixed ratio schedules, since there is a fixed ratio of mob kills you have to do to get to the next highest reputation rank. If you had to kill 20 mobs to get a drop, then that drop would be on a fixed ratio schedule – so a lot of quest rewards are fixed ratio schedules.
  • Variable Ratio Schedule – This requires a random number of responses to get your reward, and is how all the random drops in WoW (and cassinos) operate. This is the schedule that all random drop loot is on. Just the passage of time alone doesn’t make the drop more likely, and every time you kill that mob, it has the same X% chance of dropping the item, which doesn’t increase with the number of kills.
  • Fixed Interval Schedule - Requires one response after a fixed amount of time. While loot tends to not be given out just based on the passage of a fixed amount of time, the boss respawn timers are on pretty fixed intervals. If you clear Naxxramas on a Tuesday night, you get no reward for going to Naxx until after the bosses have respawned. Daily quests are also more of a fixed interval schedule. You know that if you did it today, you can’t get rewarded for trying to do it again until tomorrow. True fixed interval schedules would be something like respawning an hour after you killed it, so that the interval was based on your behavior.
  • Variable Interval Schedule – Requires a random amount of time to have passed and a single response to get your reward. A great example of this is ore & herb respawn timers. Any one node takes a random amount of time to respawn, so you are forced to run/fly around a zone trying to find ones that did pop up, rather than camping on a node and coming back every X minutes. In-game fishing is on a variable interval schedule, in terms of how long it takes once you cast before you can get your fish.

But WoW rewards me constantly every time I do something: At level cap, your most common rewards are basically token reinforcement (money & token drops from instances). Loot that you want usually drops in PvE on a variable interval schedule, off bosses that are on fixed interval schedules. Things like Arenas still have some random elements because the queue system for rankings makes it such that sometimes you lose, but there’s a reason why so many people queue up the day before arena points are handed out – because arena points are basically driven entirely by a fairly fixed interval schedule – arena points are given out the same time every week. They mostly put in boss respawn timers on the high-end raids so that we’ll get up out of our chairs and do other things with our lives sometimes, and to control the pacing of how often we get those big rewards once we hit end-game. The great thing that WoW can do is to maximize all of the types of schedules, and use them in ways that keep us playing longer.

There are plenty of examples of WoW on variable ratio schedules. I could kill the first boss in Shadow Labs non-heroic as many times as I wanted to try and get my lifebloom idol off the boss, as long as I had people who were willing to go in and help me farm it. We ran that instance over and over and over again, until I just gave up mostly because my friends weren’t going to be my friends much longer if I kept making them run that instance with me (and I understood that going more didn’t increase the likelihood of it dropping).

What WoW (and Casinos) understand is that variable ratio schedules have the best rates of responding. You’ll keep playing the game longer so that you can have a chance of getting that awesome reward. While random isn’t “fair”, it keeps you playing the game longer.

Variable ratios are also a tool to keep some items as being very rare, which makes them more desirable. “Green” items are super common, so it’s not exciting when you get one. “Blue” items are worth a little bit more because they’re a little bit harder to get. Some “epic” items are likely to be rare, so we still value them highly, and in most cases, they are the best we can get. “Legendary” items are way super rare, and people will do just about anything to get one. If all it took to get the legendary was a fixed ratio of 20 kills of bosses in a raid instance and everyone got it after that number of kills, it wouldn’t have that high of a value – it would be totally common. The randomness and rarity of the items make them desirable.

If all the loot was based on a fixed ratio token drop schedule, raiding would be really, really boring. “Oh, I don’t get my loot until next Tuesday.” It didn’t work in manufacturing jobs – people created unions to organize against fixed ratio payment schedules, because while they worked hard to get products built, it meant that the entire workday was a constant grind to get things done. Having some grinding to get rewards on a fixed ratio schedule is fine to have in the game, since you’re actually just getting tokens and points on a constant schedule, which isn’t even a true fixed ratio schedule (ie. kill this mob 20 times and then get the drop).

In this game, it’s the draw of the random exciting rewards that makes us keep coming back week after week, year after year. When the loot items in the existing content are no longer desirable, they release new and better items that are “must have” items for us to be excited about earning again. If bosses didn’t drop random loot, we probably wouldn’t like raiding so much. The variable ratio drop schedules for the pieces of the desired loot pieces is what makes us keep coming back to an instance. You don’t want to miss a night of raiding, because if the desired piece of loot drops and someone else gets it instead, then you’ll be upset. Likewise, people don’t want to get up from their slot machine at the Casino, even if they are on a winning streak, because they are waiting for their payout to come. Casinos would make a lot less money on slot machines if they gave out rewards on any other reinforcement schedule than what they do.

Understanding behavioral reward systems also helps explain why people got bored with Naxxramas and stopped logging in after they had all the loot they wanted. They didn’t really think that there was likely to be a reward they want for going to that instance, so they stopped playing and found something else to do that was rewarding (either in or out of the game). This is also why gearing up your tank first, before the rest of the raid, usually ends up with the tank getting bored and not wanting to run the instance long enough for everyone else to get loot. However, sometimes you keep running it after you stop getting rewards, since “extinction” of behavior (ie. when you stop doing something) usually takes longer on a variable interval schedule than the other schedules. It helps explain why people will spend hours of their time trying to fish up a rare turtle mount, and why at least some loot out of instances has to be random drops (and why bosses usually drop a smaller number of loot items than there are people in the instance). If you knew that after 20 runs in Ulduar, you could have all the gear you needed, you would run Ulduar 20 times and then stop running the instance.

In conclusion, having rare drops on a variable ratio schedule keeps rats pressing the lever longer, keeps people spending money in the Casinos, and keeps WoW players staying in the game longer. It’s not about being fair, or really what gets you the rewards the fastest – variable ratio reinforcement schedules is what what keeps you playing the longest. I could ramble on more, but that’s where I’ll stop for today.

This entry was posted in Uncategorized. Bookmark the permalink. Both comments and trackbacks are currently closed.

7 Comments

  1. Poras
    Posted June 3, 2009 at 7:08 pm | Permalink

    LOL… i have a degree in psychology and i loved every second of this. it really is true. i’m not practicing psych right now (as i also have a degree in music), however, if i were to, i’d definately be on the behavioral side of everything. :D

    Poras
    Officer/Resto Druid
    -Pango Honoratus-

  2. Poras
    Posted June 3, 2009 at 7:10 pm | Permalink

    also, with your permission, liss… i’d like to copy this post to my guild forums for my friends to read. (i’ll of course credit you for it and link your full blog with it)

  3. Djinnmaster
    Posted June 3, 2009 at 10:35 pm | Permalink

    Doesn’t mean I have to like it :P

  4. Lavata
    Posted June 4, 2009 at 9:17 am | Permalink

    She left out the other part of the argument. If people need to have random rewards to feel some sort of accomplishment that is fine keep them in.

    The point I have is that for every random drop there should be an equivalent that can be obtained in a non-random way.

    This is where Tokens and Crafting professions come in. What I object to is also having game changing items only drop randomly. Especially when the mechanics of the game make is such that you need X item or you will never be able to compete. The lifebloom idol that Liss only passes over was the item required for lifebloom rolling, and since she could not get it we refocused on leveling her Shaman. This is partially Blizzards fault for having such out of balanced mechanics for Druid healing in TBC and the loot system in general.

    There should be a static formula that I will get X if I do Y. I may get Z which is equivalent to Y at a 2-5% chance per boss kill but I know that after 30 boss kills I will be able to get what I want. Hell even if someone else gets Z in the mean time because either DKP or lost the roll I know I will be able to get Y after collecting 30 items where 1 or more items drop from each boss.

    I know GC and others are loving this new tiered token system because they say that the Kara farming for badges was bad. I have to disagree the badge system of TBC is superior to the tiered loot crap of WotLK. The higher tier items should just be a multiple higher than the previous tier. So something tier X is 40 badge’s tier X+1 is 80 and Tier X+2 is 120 for the same slot item. You also make Tier X+1 bosses drop 1.5 the number of badges on average than Tier X bosses. So the higher content will still be more effective when set on farm.

    On a separate note I have issues with some of EVE-Online’s mechanics as far as the penalties for death or making a mistake. However the Market and Item Acquisition System in EVE is hands down light years, no pun intended, better than WoW. There everything can be bought or sold. It is all market driven. Crafters and Gathers are rewarded for their efforts rather than laughed at like in WoW.

    I have been waiting for almost 2 years for some sort of Market and Crafting improvements and unfortunately it looks like WoW will never have them. Well here is hoping that the unnamed MMO by Blizzard will have many superior systems to WoW.

  5. Aelinna
    Posted June 4, 2009 at 11:25 am | Permalink

    GREAT post! I’ve been looking into this half-assed for a while, thanks for the awesome summary.

    I’m with Lav though. I didn’t get a Torch in Naxx and so I’m doing Ulduar hardmodes with the Warmace of Unrequited Love. It would be ok for the best loot to be RNG if there was a fallback mechanism, like less-good loot for tokens. We’ve simply gotten extremely unlucky with weapon drops and farming Naxx for the Torch is not a realistic option, so I have to do without. That’s definitely not fun*.

    Besides there’s the larger system at play. It may be true that variable ratios are compelling in the small, but does the influence of loot systems drown it out? Loot drama can kill a fledgling guild deader than a very dead parrot.

    re: badge farming, I loved it. My closest friends and I farmed hundreds of heroics and had great fun doing it. It let me gear my priest and druid alts enough that they were useful in BT. I also got to tank ZA on my paladin. The current system makes it impossible to get alts even ballpark acceptable without spending months in PUG raids.

    * ok, so the warmace was fun when I heard “you’re beating me using THAT?!?”. Point still stands.

  6. Lavata
    Posted June 4, 2009 at 1:15 pm | Permalink

    It really seems across the WoW community that those who think the current system works are all in functioning guilds. Those that don’t are usually in small guild that if they want to raid have to PUG.

    I don’t PvP Arena is a sandbox duel that has destroyed PVE mechanics and balance.

    I am sick of classes changing at such a rapid pace. It makes the game seem to have no consistency at all. Some things are getting better but it still feels like we are in a Beta of a game than an actual production release. Maybe Alpha and Beta WotLK should have lasted longer and the game released in January like TBC rather than November to make the Christmas deadline.

    Well I don’t want to steer away from loot so I will end with saying that the gear consolidation should have been taken one step farther to get rid of spell plate. 1 spec out of 9 plate wearing specs use it. If that is not a waste of itemization I don’t know what is.

  7. Posted June 4, 2009 at 11:58 pm | Permalink

    S/random loot/addiction/gi ;^)

2 Trackbacks