One thing that game companies have a lot of is data. They record tons of ‘big’ data using their various analytics. However, this only tells the company what people have done (where did they go? What did they click on? How often do they log in? How much money are we making?). This big analytics data says nothing about how people feel (Why did they do something? Did they enjoy what they did? What changes would make people happier?). The use of usability testing to generate data can help break up echo-chambers that are likely to form within companies or within feedback forums.
To understand the motivation of players and to understand the ‘why’ in player behavior, game companies need to do actual psychology testing of their game via surveys and lab testing. The New York Times recently ran an article on the importance of “small” data (titled “How not to drown in numbers”). In it, the article highlights the fact that Facebook not only measures clicks, but asks people “why?” via surveys.
With regards to video game companies, one that has been very successful in using ‘small’ data is Riot Games, who produce League of Legends. Part of the success driven by League of Legends comes from maximizing enjoyment via using user data to decide how to make future changes in the game. They pilot potential future game content with the help of psychologists. The most obvious use of data by Riot is their player behavior team that has significantly reduced negative player behavior in the game, which they continue to develop today. Their older GDC talk about how their ‘small’ data collection improved the game can be found here. Riot’s data collection taught us that it was possible to reduce negative player behavior, even if it wasn’t possible to completely eliminate negative behavior.
The interesting part of Riot is that they extended this value of data collection into every aspect of the game’s design. This includes surveys about the amount of money people were willing to pay for services, such as new skins (or what types of new skins players would want). If something appears in the game, it is likely a combination of designer’s ideas and people who tested the potential impacts of those ideas before they were implemented.
Riot’s approach differs significantly from companies who design primarily based on their own developer’s guts with little or no integration of surveys or other usability data. Usability testing can help to see whether or not people are having fun. There is no way to measure fun other than to actually interact with users via usability testing (whether survey-based or observation of players). The best design is user-centered design, rather than trusting that other people will use a product in the way that designers intend. Design needs to care about usability and player experiences. For people interested in usability testing, Carol Barnum has a book titled “Usability Testing Essentials: Ready, Set… Test!” there are also plenty of other resources to learn about how to measure player behavior and enjoyment.
Regardless of the methods, ignoring the experiences of players can result in designing without knowing what users will do in the future, or how players feel. While it is unlikely that any designer can make everyone happy all the time, understanding players’ thoughts, motivations, and goals can improve the design of games. User testing allows for understanding how people will react to things that are not yet released out in the wild. While tools like feedback forums are helpful, the use of actual targeted user testing and survey data gives much better samples and avoids the echo-chamber problem that can happen inside offices or discussion forums. Frustrating players by making them feel like their thoughts and feelings don’t matter ends up hurting companies. Interacting with users via actual experiments and surveys can help to make sure that data being used to inform game design in meaningful ways. In the rise of big data via analytics after the game’s release, we can’t forget the importance of measuring whether or not people are having fun now, or could be having MORE fun in the future. A little psychology driving user experience testing can go a long way in improving game design. I just wish more game companies invested in psychologists as a core part of their design teams.