Feedback-based gameplay metrics: measuring player experience via automatic visual analysis

ABSTRACT

Using gameplay metrics to articulate player interaction within game systems has received increased interest in game studies. The value of gameplay metrics comes from a desire to empirically validate over a decade of theorization of player experience and knowledge of games as ludic systems. Taking gameplay metrics beyond formalized user testing (i.e. with the aim of improving a product) allows researchers the freedom of examining any commercially available game without the need to have access to the game's source code. This paper offers a new methodology to obtain data on player behavior, achieved through analyzing video and audio streams. Game interface features are being analyzed automatically, which are indicative of player behavior and gameplay events. This paper outlines the development of this methodology and its application to research that seeks to understand the nature of engagement and player motivations.

References

1. Aarseth, E. (1997) Cybertext: Perspectives on Ergodic Literature. Baltimore and London: Johns Hopkins University Press.
2. Aarseth, E. (2001) Computer Game Studies, Year One. In Game Studies, Volume 1, issue 1.
3. Aarseth, E. (2006). Mapping the Madness: A Games Analysis Framework. Paper presented at the From Games to Gaming Research Seminar, Gothenburg University.
4. Anderson, C. A. (2004). An update on the effects of playing violent video games. Journal of Adolescence, 27 (2004), pp. 113--122.
5. Araújo, M., Roque, L. (2009) Modeling Games with Petri Nets. In Breaking New Ground: Innovation in Games, Play, Practice and Theory, Proceedings of DiGRA 2009 (London, UK)
6. Bartle, R. (1996) Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDs. The Journal of Virtual Environments, Vol. 1, No. 1.
7. Bay, H., Ess, A., Tuytelaars, T., Gool, L. V. (2008) SURF: Speeded Up Robust Features, Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346--359, 2008
8. Bracken, C. C., Skalski, P. (2009) Telepresence and Video Games: The Impact of Image Quality. PsychNology Journal, 7(1), 101--112.
9. Canossa, A., Drachen, A., Sørensen, J. R. M.( 2001). Arrrgghh!!! -- Blending Quantitative and Qualitative Methods to Detect Player Frustration. In Proceedings of FDG 2011, ACM. (Bordeaux, France)
10. Chen, D., Shearer, K., Bourlard, H. (2001) Video OCR for Sport Video Annotation and Retrieval. In Proceedings of the 8th IEEE International Conference on Mechatronics and Machine Vision in Practice, p. 57--62
11. Copier, M. (2003) The Other Game Researcher: Participating in and watching the construction of boundaries in Game Studies, in M. Copier & J. Raessens (Eds.) Level Up: Digital games research conference, Utrecht: University of Utrecht Press.
12. Drachen, A., Canossa, A. (2009) Towards Gameplay Analysis via Gameplay Metrics. In Proceedings of the 13th MindTrek 2009 (Tampere, Finland), ACM, 202--209. DOI:10.1145/1621841.1621878
13. Drachen, A., Canossa, A.(2009) Analyzing Spatial user Behavior in Computer Games using Geographic Information Systems. In Proceedings of the 13th MindTrek 2009 (Tampere, Finland), ACM, 182--189. DOI:10.1145/1621841.1621875
14. Drachen, A.; Canossa, A., Yannakakis, G. (2009) Player Modeling using Self-Organization in Tomb Raider: Underworld. In Proceedings of IEEE Computational Intelligence in Games (CIG) 2009 (Milan, Italy), IEEE Publishers, pp. 1--8
15. Epp, C., Lippold, M., Mandryk, R. L. 2011. Identifying Emotional States Using Keystroke Dynamics. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI 2011), Vancouver, BC, Canada. 715--724. DOI=10.1145/1978942.1979046.
16. Fagerholt, E., Lorentzon, M. (2009) Beyond the HUD. Master of Science Thesis, Department of Computer Science and Engineering, Chalmers University of Technology, (Göteborg, Sweden).
17. Ferguson, C. J. (2007). The good, the bad and the ugly: A meta-analytical review of positive and negative effects of violent video games. Psychiatric Quarterly, 78, 309--316.
18. Gee, J. P. (2003) What video games have to teach us about learning and literacy. Comput. Entertain. 1, 1 (October 2003), 20--20.
19. Gieryn, T. F. (2002) Boundaries in Science, in S. Jasanoff (Ed.) Handbook of Science and Technology Studies, London: Sage.
20. Grimshaw, M., Schott, G. (2007) Situating Gaming as a Sonic Experience: The Acoustic Ecology of First-Person Shooters. In Situated Play, Proceedings of DiGRA 2007 Conference. (Tokyo, Japan).
21. Huang, C. L, Liao, B. Y. (2001) A robust scene-change detection method for video segmentation. Circuits and Systems for Video Technology, IEEE Transactions on, vol.11, no.12, pp.1281--1288, Dec 2001
22. Johansen, S. A., Noergaard M. and Rau J. (2008). Can eye tracking boost usability evaluation of computer games? Paper presented at: CHI 2008, Evaluating User Experiences in Games
23. Joosten, E., Lankveld, G. V, Spronck, P. (2010) Colors and Emotions in Video Games. 11th International Conference on Intelligent Games and Simulation GAME-ON 2010 (ed. Aladdin Ayesh), pp. 61--65. EUROSIS
24. Juul, J. (2009) Fear of Failing? The Many Meanings of Difficulty in Video Games. From Mark J. P. Wolf & Bernard Perron (eds.): The Video Game Theory Reader 2. New York: Routledge 2009. pp. 237--252.
25. Kafai, Y. B. (2006). Playing and making games for learning. Games and Culture, 1(1), 36--40. Sage Publications.
26. Kim, J. H., Gunn, D. V., Schuh, E., Phillips, B. C., Pagulayan, R. J, Wixon, D. (2008) Tracking Real-Time User Experience (TRUE): A comprehensive instrumentation solution for complex systems. In CHI 2008 Proceedings, Data Collection, ACM. (Florence, Italy)
27. Laurel, B. (1991). Computers as Theatre. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.
28. Mirza-Babaei, P., McAllister, G. (2011) Biometric Storyboards: visualizing meaningful gameplay events. BBI Workshop CHI 2011. ACM. (Vancouver, Canada)
29. Murray, J. H. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace. The Free Press, New York, NY, USA.
30. Nacke, L. E.. 2009. Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment. Doctoral Dissertation Series No. 2009:04, Blekinge Institute of Technology. School of Computing.
31. Nacke, L., Drachen, A., Korhonen, H., Kuikkaniemi, K., Niesenhaus, J., van den Hoogen, W. Poels, K., IJsselsteijn, W., de Kort, Y. (2009) DiGRA Panel: Playability and Player Experience Research. September 1, 2009. DiGRA 2009: Breaking New Ground: Innovation in Games, Play, Practice and Theory, Brunel University, West London, UK.
32. Nacke, L. E., Drachen, A. and Goebel, S. (2010). Methods for Evaluating Gameplay Experience in a Serious Gaming Context. International Journal of Computer Science in Sport, vol. 9 (2) (International Association of Computer Science in Sports).
33. Nacke, L., Grimshaw, M. (2011) Player-game Interaction Through Affective Sound. In Game Sound Technology and Player Interaction: Concepts and Developments, ISBN 978--1616928285. Chapter 13. 264--285. Hershey, PA, USA: IGI Global Publishing.
34. Paleari, M., Chellali, R., and Huet, B. (2010). Bimodal Emotion Recognition. In proceedings ICSR'10, International Conference on Social Robotics, November 23--24, 2010, Singapore.
35. Plutchik, R. (2001). The nature of emotions. American Scientist, 89, 344--350.
36. Ruch, A. W. (2010) Videogame Interface: Artefacts and Tropes. In Videogame Cultures and the Future of Interactive Entertainment Global Conference (2nd: 2010) (7--9 July 2010: Oxford, UK)
37. Saraceno, C., Leonardi, R. (1997) Audio as a support to scene change detection and characterization of video sequences. Acoustics, Speech, and Signal Processing, 1997. ICASSP-97., 1997 IEEE International Conference on, vol.4, no., pp.2597--2600 vol.4, 21--24 Apr 1997
38. Skalski, P., Whitbred, R. (2010) Image versus Sound: A Comparison of Formal Feature Effects on Presence and Video Game Enjoyment. PsychNology Journal, 8(1), 67--84.
39. Tychsen, A., Canossa, A. (2008) Defining Personas in Games Using Metrics. In FuturePlay Proceedings, ACM. (Toronto, Canada).
40. Ye, Q., Huang, Q., Jiang, S., Liu, Y., Gao, W. (2005) Jersey number detection in sports video for athlete identification. In Visual Communications and Image Processing 2005. Edited by Li, Shipeng; Pereira, Fernando; Shum, Heung-Yeung; Tescher, Andrew G. Proceedings of the SPIE, Volume 5960, pp. 1599--1606

Index Terms

• Applied computing

  • Computers in other domains

    • Personal computers and PC applications

      • Computer games

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Graphics processors

• Human-centered computing

  • Human computer interaction (HCI)

• Information systems

  • Information systems applications

    • Multimedia information systems

      • Massively multiplayer online games

• Software and its engineering

  • Software organization and properties

    • Contextual software domains

      • Virtual worlds software

        • Interactive games