Blog Archives

Breaking the echo-chamber: usability and data driven design

One thing that game companies have a lot of is data. They record tons of ‘big’ data using their various analytics. However, this only tells the company what people have done (where did they go? What did they click on? How often do they log in? How much money are we making?). This big analytics data says nothing about how people feel (Why did they do something? Did they enjoy what they did? What changes would make people happier?). The use of usability testing to generate data can help break up echo-chambers that are likely to form within companies or within feedback forums.

To understand the motivation of players and to understand the ‘why’ in player behavior, game companies need to do actual psychology testing of their game via surveys and lab testing. The New York Times recently ran an article on the importance of “small” data (titled “How not to drown in numbers”). In it, the article highlights the fact that Facebook not only measures clicks, but asks people “why?” via surveys.

With regards to video game companies, one that has been very successful in using ‘small’ data is Riot Games, who produce League of Legends. Part of the success driven by League of Legends comes from maximizing enjoyment via using user data to decide how to make future changes in the game. They pilot potential future game content with the help of psychologists. The most obvious use of data by Riot is their player behavior team that has significantly reduced negative player behavior in the game, which they continue to develop today. Their older GDC talk about how their ‘small’ data collection improved the game can be found here. Riot’s data collection taught us that it was possible to reduce negative player behavior, even if it wasn’t possible to completely eliminate negative behavior.

The interesting part of Riot is that they extended this value of data collection into every aspect of the game’s design. This includes surveys about the amount of money people were willing to pay for services, such as new skins (or what types of new skins players would want). If something appears in the game, it is likely a combination of designer’s ideas and people who tested the potential impacts of those ideas before they were implemented.

Riot’s approach differs significantly from companies who design primarily based on their own developer’s guts with little or no integration of surveys or other usability data. Usability testing can help to see whether or not people are having fun. There is no way to measure fun other than to actually interact with users via usability testing  (whether survey-based or observation of players). The best design is user-centered design, rather than trusting that other people will use a product in the way that designers intend. Design needs to care about usability and player experiences. For people interested in usability testing, Carol Barnum has a book titled “Usability Testing Essentials: Ready, Set… Test!” there are also plenty of other resources to learn about how to measure player behavior and enjoyment.

Regardless of the methods, ignoring the experiences of players can result in designing without knowing what users will do in the future, or how players feel.  While it is unlikely that any designer can make everyone happy all the time, understanding players’ thoughts, motivations, and goals can improve the design of games. User testing allows for understanding how people will react to things that are not yet released out in the wild. While tools like feedback forums are helpful, the use of actual targeted user testing and survey data gives much better samples and avoids the echo-chamber problem that can happen inside offices or discussion forums. Frustrating players by making them feel like their thoughts and feelings don’t matter ends up hurting companies. Interacting with users via actual experiments and surveys can help to make sure that data being used to inform game design in meaningful ways. In the rise of big data via analytics after the game’s release, we can’t forget the importance of measuring whether or not people are having fun now, or could be having MORE fun in the future. A little psychology driving user experience testing can go a long way in improving game design. I just wish more game companies invested in psychologists as a core part of their design teams.

Posted in Research on video games, Written By Lissanna

Druid travel form survey results

Hi druids! I have analyzed the data from 466 responses to the druid travel form survey. Below is the summary of the findings. This link will also take you directly to the Google analytics summary page (This includes the full 615 responses that were completed before I turned the survey off). Note that the Google analytics summary page dynamically updates and reflect the current results with more participants than what was used for the summary here. There isn’t a difference overall in the trends between the below analysis and the updated Google analytics.

Overall summary and design suggestions

  • While responses to the change are overall negative, there are some people who like the change (or like at least some aspects of the change). There was not a single consensus all druids agreed upon.
  • In general, there is less of a problem with ground travel and swim form being merged (other than general bugs with the transition out of the water). This was the aspect people rated most positively in both the multiple-choice and open-ended questions.
  • People instead most often report disliking having ground and flight travel forms merged. While glyph of stag addresses this problem for 13% of people responding, there are also others who either don’t want to use the glyph or don’t have their particular problem solved by the glyph (this included 9% of people who didn’t know about the glyph). For example, people who want to use the Cheetah glyph will never see their Cheetah form in places with flight enabled, since there’s no way to both split flight form from ground form AND use the cheetah glyph.
  • People also report a lot of problems where they fall to their deaths more often when in flight form (e.g., zoning, logging out and back in, or entering combat shifts them out of flight form), that increases their dislike of the merge.
  • It might make the most sense to split off flight form from the merged form permanently, rather than only via the glyph. Then, the Glyph of Stag should split ground and swim form, so that people stop drowning their friends when ground form steps over puddles when using Mount Form.

See the complete analysis below.

Sample Demographics

  • Druid Mains = 80% of sample; Druid alts = 20% of sample.
  • Level 90 druids = 95% of sample. Remaining 5% were below level 90.
  • 82% of druids have used travel form “sometimes” or “often”.
  • 33% of druids are currently using Glyph of Stag.
  • Travel form is most often used for solo questing, exploring the world, and herb/ore farming.

Travel form usage: 80% for solo questing, 76% for exploring the world.

 Summary of Open Ended Responses

What druids like about the change

  • The thing people most frequently report liking about the change is shifting in and out of aquatic form automatically (from either ground, air, or both).
  • Some druids also like the reduced button bloat, no longer having to use macros to do the same thing. In addition, some report ease of use or convenience as something they like about the change.
  • The pie chart below shows the responses after I coded them for frequency:

What druids like about the change to travel form.

What druids dislike about the change

  • Druids most often report dislikes related to loss of control, flexibility, fun, or choice related to their shapeshift forms.
  • In particular, druids often reported a dislike of having the ground travel form and flight form merged. This was particularly problematic for people who had been previously using the cheetah glyph, as cheetah form can’t be used in flight-enabled areas. In general, the glyph of stag is not suitable for addressing the concerns related to merging ground and flight form together, especially due to the exclusivity with other glyphs, and the feeling of “wasting” a glyph spot.
  • People listed many other types of dislikes, including problems where the glyph of stag causes riders on the back of stags to be dismounted when shifting into water. Some people also just generally disliked the change entirely.

What druids dislike about the change to travel form

Multiple-Choice responses

  • Only 22% of druids said that they liked the changes made to travel form (13% neutral, and 63% dislike). In addition 72% suggested they liked the old separated forms better.
  • Overall, 75% of druids feel that the new travel form prevents them from doing things they would like to do.
  • In addition, 67% suggested they had concerns about control over their shapeshift buttons. Only 13% of people said that glyph of stag addressed these concerns.
  • However, 65% of druids said that having land travel form shift into swim form when they went in the water was helpful.
  • 55% of druids find the new travel form frustrating to use, but only 28% find the new form confusing to use (thus, the frustration likely stems from other problems, rather than from confusion related to the change).
  • Please go directly to the Google Analytic link to see the pie charts for each multiple-choice question.

Percent of people who like the new travel form designThanks everyone for your responses! My summary of the bug reports and problem open-ended question (which can be seen in the analytics summary page) will be submitted elsewhere to save space in this post.

Posted in Research on video games, Uncategorized

Druid Travel Form Survey – CLOSED

Update: Thank you everyone who took the time to complete the survey. After 615 responses, I have now de-activated the survey and closed it to future responses. Screenshots of the questions are provided below for archiving purposes and being able to interpret the survey results. You can view the results and interpretation from the first 466 responses here.




Posted in Research on video games, Uncategorized

A quick note

Hi all,

A quick update about me: I have been traveling the last week for work. I attended a work-related conference for the Society for Research in Child Development this last week in Seattle, Washington. At the conference, I presented some preliminary results from the Autism face processing study that was included as part of the research I described in my crowdfunding campaign last fall. The data I presented was some of the functional MRI brain data from the baseline (before the intervention) time point of the intervention study, since we are still working on completing the follow-up assessments from the intervention this month, with lots of other exciting research things happening this Summer/Fall. However, this means I may be a little slow in posting blog content until the semester ends in a few weeks. Sorry for falling behind in my blog posting! I will try to get a post up actually related to druids “soon”.

Looking for guest bloggers! Are you a druid that enjoys writing about WOW? Well, I’m opening back up my “voices from the community” series to guest bloggers this summer. If you have an idea for a World of Warcraft Druid-related post that you would like to write and have posted on Restokin, feel free to send me an e-mail: lissanna70 at Please tell me about yourself (info such as how long you have been playing, an armory link, links to any previous blogs/youtube/etc if you have them), and what druid-related post ideas you are interested in possibly writing.

Resto Roundtable at the Team Waffle Podcast: The Team Waffle Podcast recently had a resto druid roundtable. The lineup included Jarre, Hamlet, Sodah, Jasyla, and of course Arielle as the host/moderator.

Posted in Research on video games, Restoration Healing Trees, Written By Lissanna


Featured Blogs