Tuesday, May 16, 2006

The Pigeon and The Seed

[Game Design] [Round Table]

The majority of game design proceeds from a simple assumption - that the worth of a game can be measured by how fun it is, and all other things being equal, a game that is fun will make more money than one that is not.

Except that's not really true, is it?

Surely it's only necessary to make the first few hours of a game fun, so that the player will become invested. After that, you can milk play out of the player for a period of time limited only by the effectiveness of your reward schedule. Building a reward schedule is in many cases cheaper and easier than building fun. And for a very succcessful MMO, that leads to a very important (and very easy) business decision: if you have to choose between your players having fun, or your company making more money, which do you pick?

No prizes for guessing.

But what on Earth am I talking about? I'm talking about behavioural science.

Around 1957, American behavioural scientist BF Skinner was conducting a series of interesting experiments on pigeons. In these trials, pigeons were placed in cages. Within the cage was a disc, which could be pecked and functioned like a lever or button. The pigeons were subjected to one of five different reward schedules.

* Some pigeons were on a continuous reinforcement schedule. When this group pecked the disc, they were rewarded with seed. Every peck immediately produced seed.
* Some were on a fixed ratio schedule. This group would get seed regularly after a certain amount of presses. For example, every tenth peck on the disc would produce seed.
* Some were on a variable ratio schedule. This group would get seed after random numbers of pecks of the disc. That is to say, on average every tenth peck would produce seed, but the specific number from one reward to the next might vary from five to nine to thirteen, and so forth.
* Some were on a fixed interval schedule. Seed would become available through pecking the disc every, say, five minutes. It didn't matter how often the disc was pressed - it would only deliver seed once in any five minute period.
* Lastly, some pigeons were on a variable interval schedule. Seed would become available on average every five minutes, but the specific amount of time between rewards becoming available might vary from two minutes to ten minutes.

Four interesting relationships are indicated by experiments of this sort.

1) A pigeon on a fixed ratio schedule will pause significantly (stop pecking) after each reward before starting the number of presses necessary to achieve the next reward. This pause is not present in variable schedules - the activity is continuous.

2) A pigeon on a continuous reinforcement schedule whose seed supply is turned off will very quickly stop pecking the disc. But a pigeon on an interval or ratio reward schedule, particularly a variable one, takes a very very long time to stop pecking the disc. The less frequent the rewards originally were, the longer it takes the pigeon to stop pecking.

3) No output/reward equation is present in this behaviour; on schedules were rewards come extremely infrequently, a pigeon may actually peck itself to death trying to achieve the reward.

4) Once the pigeons had learned the relevant behaviours, their inclination to peck the disc was to some extent unrelated to whether they were actually hungry.

So what does this mean? It probably means that ratio and interval reward schemes, particularly a variable ratio reward scheme, are able to:
a) produce repeated and prolonged simple behaviour
b) produce behaviour which is not in a subject's best interest
c) motivate subjects to continue seeking rewards after the rewards cease to become relevant or useful to the subject, and
d) motivate subjects to continue seeking rewards for a long time after rewards stop being available.

Which leads me to my point: when was the last time you went grinding for loot?

It seems that fun isn't necessary to make players continue playing a game long after reaching the point of diminishing returns in terms of original content. In fact, fun isn't even the best way to make players continue playing. It's entirely possible it may even be detrimental to the aim of keeping players hooked. So I ask again - if you are developing a massively multiplayer online game, and you're going to be charging players for access by the month, what's going to be your design goal?

(Apologies to Skinner and to behavioural scientists for my horrible layman's version of this area of study.)
Further reading:
John Hopson's article on Gamasutra, Behavioural Game Design.
Chris Bateman's post on Only a Game, Designing Rewards in Games.
More on BF Skinner and operant conditioning is available here.
This is my post for the May Round Table of Bloggers.


Jason O said...

Of course this is why I quit playing MMO's. It wasn't fun and only rewarded the obsessive-compulsive players.

It was just one step on my journey to playing only single-player games.

GregT said...

At the moment I'm playing Perplexcity (an alternate reality game - see my post of that name) and I've been reacquainted with exactly how much unique ongoing content can be provided on a modest budget (and with a developing narrative, no less) when you don't have quite so much need for coding and debugging. I think there's a whole bunch of lessons in this for the industry generally.

Julia B said...

Weirdly, I am one of those pigeons that can sit for hours as long as you give me enough seed to stay alive. Hence my adoration of pokies.

However, there has to be something more than a goal to reach. Look at Bards Tale for the PS2. Plenty of goals, but who cared?

Fun is an elusive element, but surely the best games are the ones that simply make the grind interesting, either with cool music (ie. Final Fantasy fight music, or even the dreaded pokies) or seeing your progression on a leader board (Perplex City).

GregT said...

Bard's Tale had plenty of goals? Were you playing the same one I was? The irony of Bard's Tale was that by doing away with the inventory system, removing the whole "shop for new equipment" mechanic, instituting linear story progression (alternate endings notwithstanding), it basically created a game based around grind, without actually leaving in any of the mini-goals and reward schemes that make the grind bearable. It had fixed ratio levelling, it didn't stagger reward schemes (so that when you achieve one reward you're already halfway to another, differen reward), and it offered no meaningful character customisation, alternative incentives, or incentive for exploration. It's a near perfect example of how NOT to do goal and reward schemes.

Julia B said...

If by goals you mean quests, there seemed to be plenty. There were mini-goals such as barrel bashing and generic talk to this person, defeat this monster, talk to this person again for reward.

It was just generally annoying.

GregT said...

No, by goals I mean (to take an example), the way that every Grand Theft Auto game since three has worked. You have an overarching goal (finish the story). You have incremental subgoal steps along the way (finish the missions). Fine, Bard's Tale has that.

But you also have an alternate overarching goal (100% completion). You have alternate explicit goals such as finishing the taxi/ambulance/delivery etc missions, finding all the secret packages. You have hidden goals, such as finding the secret rooms and bulletproof vehicles. You have room for the player to develop their own goals, such as car collecting or pulling off amusing stunts. The tools are present in the game for the player to create their own play; they can, to varying extents, make the game into the sort of game they like playing.

Bard's Tale - it's an action RPG. If you don't like levelling up and killing monsters, well, you bought the wrong game, cos that's all you'll be doing. You can try and do other things, but the game will either plain not support it, prevent you from it, or discourage you from it. And it'll be dull.

What you're talking about aren't really goals, they're just mechanics for delivering rewards. The rewards are predictable, they don't change the game experience, they're largely the same reward every time, and there's not a lot of incentive to see the next reward (because it'll be pretty much the same as the last one). It's not multiplayer, so there's no one to boast to; and every player playing the game will find all the same rewards, more or less, so you can't even talk about the "cool thing" you found that someone else might not have.

Jonno B said...

So in terms of world of warcraft, could the schedules work like this?

continuous reinforcement - see a barrel, break it, get the 5 gold pieces, repeat

fixed ratio - levelling up your character

variable ratio - the gold pieces that's spilled out after killing an enemy

I'm trying to work out where fixed interval and variable interval would be used. Are there special quests which can only be entered at special times in the game?