Testing games for their usability is a relatively new field. It required a change in focus, from "Can it be used?" to "Can it be used, and is it fun?". This change in focus meant previously recognised methods had to be adjusted and entirely new ones created to enable user experience to effectively evaluate games and highlight usability issues.
So - there are now many methods available to assess game usability; however many of these methods struggle with regards to social games.
Before we go further it's best to define 'social' games. When it is used through this article, it refers to games that can be played between friends online (for example those played through the medium of Facebook).
Social games are the perfect game to receive game usability treatment for two reasons:
- They are often played by a much wider audience than typical games, meaning the focus on ease of use needs to be much sharper.
- There's no 'barrier to entry'. The games are usually free, and the player has lots to choose from. If they can't pick up a game quickly they'll take their time elsewhere.
This means developers have to take extra care to ensure that the game is understood and easily played by anyone - they can't rely on previously learnt "game knowledge", or that players will push their way through problems.
The issues with testing social game usability
The interaction required by social games doesn't lend itself easily to testing. Specifically:
- The 'social' aspect, interacting with friends. This is very hard to reproduce in a testing environment
- The time delay encouraged by games, e.g. Cityville's 'energy' (needed to do almost any development), naturally enforces breaks in play... and keeps you coming back. How do you take this into account in testing?
There are methods available to tell you whether players are able to play, such as can they play through the tutorial and beyond. However most methods struggle to reproduce the 'natural' playing experience or gather data on the social play. Essentially it's easy to assess the 'game', but much harder to assess the 'social' aspect of a social game.
Below are a summary of some of the more popular game usability techniques, and some tips to help adjusting them to assess a social game usability.
- Expert evaluation
- 1-on-1 user testing
- Usability group playtesting
- Diary studies
- Customer feedback/Forum
- Focus groups
- Analytics analysis
Expert evaluation of a social games usability
An expert evaluation is an assessment by 1 or more usability/user experience consultants using their knowledge and previous experience to look for usability issues.
Expert evaluations are still one of the more common methods used to assess game usability - they're quick and inexpensive. They have a lot to offer a developer, and are a great method to kick-start a larger game usability project, removing all the 'low hanging fruit' before looking further up the tree.
Ensure any expert evaluation of a social game also includes an assessment of the 'social' aspect of play. This can be low level, such as how easy it is for players to invite friends to join, or higher level, looking at the whole experience on offer for players and their interaction with and between friends.
One reservation around an expert evaluation is that the whole process can be completed without any contact with users. This means you miss a great opportunity to see your players interact with your products, and as a result developing a greater understanding of your players and their needs, motivations and desires. This is a real value add to user research as this understanding can be used directly to improve the current/future games.
1-on-1 user testing of a social game
This involves participants fitting your user profile playing the game with a facilitator observing and supporting. Relevant questions can be asked as players play/encounter issues to discover the player's understanding and mindset.
1-on-1 user testing of a game leads to lots of quality in-depth information. This is because the reasoning behind player behaviour can be investigated and examined in much greater detail than through almost any other method available.
Testing with a social game is very similar to game usability testing for all games, but there are various pieces of finesse that can be added:
- A member of the assessment team sitting online elsewhere can be the player's "friend" enabling the 'add friend' mechanic to be assessed, as well as social interaction elements (an alternative option could be to run 2 test sessions at once, allowing both players to interact with each other)
- Consider disabling the 'energy' mechanic, to allow the player to keep playing longer than they would be able to do normally
- Alternatively speed the recovery period, to ensure the player is able to return to playing after a mini-break (potentially spent completing a questionnaire about the game)
- Or consider setting up a second account the player can log into after 'exhausting' their first account. This could be at a much more advanced level, allowing you to assess the introduction to the game, as well as a much more advanced position
The biggest issues with 1-on-1 user testing is that it's very labour intensive, and as a result off-putting to many cash-strapped developers. Another potential issue is the cognitive load when playing a particularly busy game often leaves players little time/mental capacity to talk during play. This means the technique needs to be adjusted depending on how much effort is required to play.
Game usability playtesting of a social game
Game usability playtesting is the term I use to describe multiple people playing in a room together with 1 or 2 facilitators supporting all of them. This allows the game to be played by more people than through 1-on-1 user testing, at around the same cost.
However this breadth comes at a loss in fidelity, as the facilitator is stretched over more people, meaning the depth of information is much lower. You may observe a usability issue occurring, but without being able to interrogate the player, you may not understand why it's occurring. The understanding of your players and their behaviour is not as complete as it could be when compared to other methods.
A real strength of game usability playtesting a social game is that the players are able to play with each other, meaning the social interaction aspects of the game can be tested by players. However this is a very artificial manner to attempt to play a social game - with all the 'friends' in the same room.
A further issue with group playtesting, especially if you plan to use verbal feedback during the session is "Groupthink". This is where players are potentially influenced by other player's responses. The risk can be reduced by asking participants to complete written questionnaires, rather than giving verbal feedback, but other players will still influence each other (for example through body language, or involuntary exclamations "that sucks!", "that's so cool!").
Diary study to assess a social game
During a diary study players are asked to keep a record of their play with a game. What this record contains is open - it can be defined by you. This could include:
- What they played (games or levels)
- How long
- What they liked/disliked/found confusing
- How they felt
- Who they played with
- Anything else you want to find out!
Diary studies are an interesting method to develop a great understanding of your players; it really gives you a window into their lives and into their playing habits. And results in a vast amount of data for your use. This method, as 1-on-1 user testing allows you to develop a real understanding of your players and their needs, motivations and desires, and their play habits, as the diary can record play over an extended period.
A real advantage in using diary studies when assessing a social game is that they are particularly well suited. They allow data to be gathered over this extended period, fitting around the players natural play schedule, meaning play doesn't need to be squeezed into 1 quick play/assessment session.
One of the biggest disadvantages to diary studies is the large amount of time and effort required to conduct them, meaning they are one of the more expensive methods available. A diary study is best used when followed up with interviews of the participants, to clarify contents, adding more to the cost. A further disadvantage is the extra time required to gather data, and the large attrition (drop out) rate of participants that occurs as a result.
Finally diary studies are reliant on users 'self-reporting' their behaviour. This comes with potential risks, and gains. When discovering about user behaviour, this is particularly useful, when looking to find usability issues, observing player behaviour to be a much more powerful approach.
Using customer feedback to improve a social game
Using feedback from players is a well recognised and often used technique to improve a game - particularly social games, as they often develop over an extended period after release. It can be a real goldmine of ideas, particularly around possible future developments in social interaction.
It's also a very inexpensive contact point between you and your players, enabling to learn about them so much more (their needs, expectations, motivations, behaviours). It should never be ignored if it's available.
As diary studies, customer feedback relies on 'self reporting' behaviour, meaning this is less efficient at looking for usability issues. You couldn't use player feedback to get an idea of, for example, what issues they encountered in the tutorial.
Another issue with customer feedback is that you are only listening to players motivated enough to get in contact. This may not include those players you're trying to improve the game for. Very few people who are baffled by a games tutorial will proceed to go online and tell the developer (but they may well warn their friends away).
Focus group assessment of a social game
Focus group assessment of a game is often used as a gauge to assess player acceptance. It is less well suited to finding usability issues, as (just like customer feedback) how people play is very different to how people claim to play.
Once more, it's a cost effective tool to give you access to and contact with lots of potential players, enabling you to develop a greater understanding of them. Don't overlook it for this reason, but it's simply not suitable to find usability issues.
Focus groups also rely on 'self-reporting' behaviour, meaning they are not best suited to find specific usability issues.
Analytics analysis to improve a social game
Analytical data is being used more and more to aid game balancing, but analytics are also a crucial source of information for user experience consultants. With effective page tracking individual issue pages in a process, or whole problem processes are easily identified.
Remember analytics data cannot identify why people are dropping out, just where. This can allow you to very quickly pinpoint what pages and processes need assessment using the other methods mentioned in this article, but cannot be used as a standalone method.
In order for analytics analysis to be effective you must ensure to approach the data with specific questions in mind. Without these questions it becomes extremely difficult to prevent yourself from drowning in data. Low-level sample questions include:
- Where in the sign up/tutorial do players drop out to a significant degree?
- Are players able to proceed through the checkout?
- What pages have the largest attrition (drop out rates)?
- How many players 'use' your social interaction facilities?
Choosing an appropriate method
Selecting an appropriate method to assess your game is tough, it's even hard when looking to assess a social game. When assessing a social game's usability selecting the appropriate method is made even more important, due to the special restrictions social games place on players.
With so many methods available, all of which will "do a job" to a greater or lesser extent, how do you pick between them? Think about your aims, what you need to learn, what you want to find out and your previous knowledge, what you already know. These two factors can help you narrow down significantly the methods and help you find which method will be best suited for you.
Conclusions
Social games offer a particular challenge when assessing their games usability. With the social interaction possible in a social game, assessing the usability can create some real issues. Methods are available, but many need to be adjusted to get full value from their use.
