Game User Research Presentation

Last week I had the privilege of listening to Daniel Natapov, the User Research PM from Ubisoft Toronto. Mr. Natapov specifically deals with how players relate to the game and different evaluations of game user research. He discussed different types of methods he uses in his daily work environment, his previous work experience, what types of user research methods he uses and which ones he never dealt with. He also discussed problems that occur during the research stage, what game he is currently working on, some facts that people do not expect in his line of work and tips for going into the working environment. Overall, he was very helpful guest speaker and I learned a lot from his words.

There are eight game user research methods that a person can look at when trying to determine how game players interact with the game. Behavioural observation allows a person to watch somebody play the game while you take notes; it determines what happened but not how or why. When conducting a behavioural observation a person must design a scenario to be play tested, write a script for the user researcher, think about what behaviours to expect, test the script in pilot run-through, define the target demographic, run the play testing session, end the session and finally schedule breaks so that the test subjects do not become tired or become fatigued. Think-aloud protocol is when players describe their actions as they play, the observer does not prompt or correct them while they play the game. Whatever the player says is recorded and then replayed at a later time to be analyzed. Interviews are usually structured and they validate the playtest goals, usually the game tester will play the game and the interviewer would ask questions that revolve around what they need to determine. Heuristic evaluation is using a game user research expert to play a game and evaluate it. This allows for an expert critique and it is similar to a game review. Focus groups are groups of players that are gathered into a large group and they discuss their opinions, beliefs and attitudes amongst themselves with a moderator to instigate discussion. This is an easy and fast way to receive feedback. Questionnaires are a set amount of questions the player must answer either before and/or after they have played the game. It determines the average of the entire group; however, it is less personal and can lead to misleading statistics. Gameplay metrics uses data about the behaviour of players in game environments. It takes game data such as player movement, firing weapons and interface interaction and determines whether or not a player is comfortable with their in game surroundings. 

Figure 1 - Focus Group being conducted

 

Daniel Natapov has used many of these research methods however he finds that focus groups are a good method because it is an inexpensive and fast way to collect data. Daniel depicted that when creating a focus group area it must be very open so that when the game testers arrive they do not feel uncomfortable. Also the moderator must be very relaxed and tell them not to be agitated, the moderator must make them feel like it is a safe place where they can share their opinions without feeling embarrassed. He also had some experience with think-aloud protocol, he stated that it is useful when a player is playing a game without distraction; it allows them to feel more intertwined with the game environment. Something I found really interesting was that he never really looked at heuristics while evaluating his games; he said that because there are no standard heuristics a person could say it is not reputable and thus, not worth analyzing. 

Figure 2 - Splinter Cell Blacklist

Daniel Natapov has been recently been working on Splinter Cell Blacklist. He could not speak too much about it but he said that he has done much research on the game attributes such as in game bugs and colour schemes in the game. One thing that really stood out was that art and other assets are constantly changing during focus group research, it’s not always the actual code errors. I always thought that when game testers were playing the game their only focal point was bugs in the game not how aesthetically pleasing it looked. 

Figure 3 - First Impressions Matter

Some interesting facts that he discussed during his presentation was that some problems that occur during the research time is unrelated to the people that are playing the game. One major factor was that the game was unable to run due to programming issues, this meant that he could not even run the game for people to discuss it. He also stated that game user research analysts have to be able to identify information quickly and process it into workable data. You must be able to interoperate what the focus group says and use it to benefit the game. Mr. Natapov worked for Microsoft before coming to Ubisoft and one thing he found very different is that working at Microsoft meant having a larger group of people to analyze things together. At Ubisoft it is a smaller group so there are less people to critique the data. In larger groups there are more experts, this allows more ideas to be formulated and it creates less of a bias opinion. 

Tips to Consider

Before leaving Daniel Natapov gave some tips for transitioning from school to the working world. He said to always wear a suit to your interview because first impressions do count and you must have very good social skills. During the interview or while talking to a game developer outside of work, you must always be seen as a sociable person that can get the job done.

Questions to Consider?

A question I would like to ask Daniel is how he became a GUR expert? And was it something he wanted to do right from the start?

What GUR method is best when looking at a fighter game versus a JRPG, what makes them different?

How could I make my focus group moderators less bias, could I use heuristics as a base to create some type of standard?

References

Figure 1 - http://newyorknatives.files.wordpress.com/2011/05/focus-group-on-students-mobility221.jpg

Figure 2 - http://media-a.shopto.net/ShopToMedia/images/screenshots/XB2TO42/A.jpg

Figure 3 - http://educationismylife.com/wp-content/uploads/2013/02/Good_first_impression-412x360.jpg