Player retention in league of legends: a study using survival analysis

ABSTRACT

Multi-player online esports games are designed for extended durations of play, requiring substantial experience to master. Furthermore, esports game revenues are increasingly driven by in-game purchases. For esports companies, the trends in players leaving their games therefore not only provide information about potential problems in the user experience, but also impacts revenue. Being able to predict when players are about to leave the game - churn prediction - is therefore an important solution for companies in the rapidly growing esports sector, as this allows them to take action to remedy churn problems.

The objective of the work presented here is to understand the impact of specific behavioral characteristics on the likelihood of a player continuing to play the esports title League of Legends. Here, a solution to the problem is presented based on the application of survival analysis, using Mixed Effects Cox Regression, to predict player churn. Survival Analysis forms a useful approach for the churn prediction problem as it provides rates as well as an assessment of the characteristics of players who are at risk of leaving the game. Hazard rates are also presented for the leading indicators, with results showing that duration between matches played is a strong indicator of potential churn.

References

1. Zoheb Borbora, Jaideep Srivastava, Kuo-Wei Hsu, and Dmitri Williams. 2011. Churn prediction in mmorpgs using player motivation theories and an ensemble approach. In Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE Third International Conference on Social Computing (SocialCom), 2011 IEEE Third International Conference on. IEEE, 157--164.
2. Ionut Brandusoiu and Gavril Toderean. 2013. Churn prediction modeling in mobile telecommunications industry using decision trees. Journal of Computer Science and Control Systems 6, 1 (2013), 14.
3. James Brightman. 2016. League of Legends generates 150m a month - SuperData. http://www.gamesindustry.biz/articles/2016-06-09-league-of-legends-generates-usd150m-a-month-superdata. (2016).
4. Chris Chambers, Wu-chang Feng, Sambit Sahu, and Debanjan Saha. 2005. Measurement-based characterization of a collection of on-line games. In Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement. USENIX Association, 1--1.
5. Kristof Coussement and Koen W De Bock. 2013. Customer churn prediction in the online gambling industry: The beneficial effect of ensemble learning. Journal of Business Research 66, 9 (2013), 1629--1636.
6. David R Cox. 1992. Regression models and life-tables. In Breakthroughs in statistics. Springer, 527--541.
7. Wu-chang Feng, David Brandt, and Debanjan Saha. 2007. A long-term study of a popular MMORPG. In Proceedings of the 6th ACM SIGCOMM Workshop on Network and System Support for Games. ACM, 19--24.
8. John Hadden, Ashutosh Tiwari, Rajkumar Roy, and Dymitr Ruta. 2007. Computer assisted customer churn management: State-of-the-art and future trends. Computers & Operations Research 34, 10 (2007), 2902--2917.
9. Fabian Hadiji, Rafet Sifa, Anders Drachen, Christian Thurau, Kristian Kersting, and Christian Bauckhage. 2014. Predicting player churn in the wild. In Computational intelligence and games (CIG), 2014 IEEE conference on. IEEE, 1--8.
10. David W Hosmer Jr and Stanley Lemeshow. 1999. Applied survival analysis: regression modelling of time to event data (1999). Eur Orthodontic Soc (1999), 561--2.
11. Shin-Yuan Hung, David C Yen, and Hsiu-Yu Wang. 2006. Applying data mining to telecom churn management. Expert Systems with Applications 31, 3 (2006), 515--524.
12. Deborah Kaminski and Cheryl Geisler. 2012. Survival analysis of faculty retention in science and engineering by gender. Science 335, 6070 (2012), 864--866.
13. Edward L Kaplan and Paul Meier. 1958. Nonparametric estimation from incomplete observations. Journal of the American statistical association 53, 282 (1958), 457--481.
14. Jaya Kawale, Aditya Pal, and Jaideep Srivastava. 2009. Churn prediction in MMORPGs: A social influence based approach. In Computational Science and Engineering, 2009. CSE'09. International Conference on, Vol. 4. IEEE, 423--428.
15. P Kollar. 2017. The Past, Present and Future of League of Legends Studio Riot Games, Polygon Platform, 2016. http://www.polygon.com/2016/9/13/12891656/the-past-present-and-future-of-league-of-legends-studio-riot-games. (2017).
16. Junxiang Lu. 2002. Predicting customer churn in the telecommunications industry---An application of survival analysis modeling using SAS. SAS User Group International (SUGI27) Online Proceedings (2002), 114--27.
17. M. Milosevic, N. Zivic, and I. Andjelkovic. 2017. Early churn prediction with personalized targeting in mobile social games. Expert Systems with Applications (2017).
18. Guangli Nie, Wei Rowe, Lingling Zhang, Yingjie Tian, and Yong Shi. 2011. Credit card churn forecasting by logistic regression and decision tree. Expert Systems with Applications 38, 12 (2011), 15273--15285.
19. Noppon Prakannoppakun and Sukree Sinthupinyo. 2016. Skill rating method in multiplayer online battle arena. In Electronics, Computers and Artificial Intelligence (ECAI), 2016 8th International Conference on. IEEE, 1--6.
20. Fred Reichheld. 2001. Prescription for cutting costs. Bain & Company. Boston: Harvard Business School Publishing (2001). http://www.bain.com/IMages/BB_Prescription_cutting_costs.pdf
21. François Rioult, Jean-Philippe Métivier, Boris Helleu, Nicolas Scelles, and Christophe Durand. 2014. Mining tracks of competitive video games. AASRI Procedia 8 (2014), 82--87.
22. Samuli Ripatti and Juni Palmgren. 2000. Estimation of multivariate frailty models using penalized partial likelihood. Biometrics 56, 4 (2000), 1016--1022.
23. Julian Runge, Peng Gao, Florent Garcin, and Boi Faltings. 2014. Churn prediction for high-value players in casual social games. In Computational Intelligence and Games (CIG), 2014 IEEE Conference on. IEEE, 1--8.
24. David Schoenfeld. 1982. Partial residuals for the proportional hazards regression model. Biometrika 69, 1 (1982), 239--241.
25. Matthias Schubert, Anders Drachen, and Tobias Mahlmann. 2016. Esports analytics through encounter detection. In Proceedings of the MIT Sloan Sports Analytics Conference.
26. Rafet Sifa, Christian Bauckhage, and Anders Drachen. 2014. The Playtime Principle: Large-scale cross-games interest modeling. In Computational Intelligence and Games (CIG), 2014 IEEE Conference on. IEEE, 1--8.
27. R. Sifa, F. Hadiji, J. Runge, A. Drachen, K. Kersting, and C. Bauckhage. 2015. Predicting Purchase Decisions in Mobile Free-to-Play Games. In Proc. of AAAI AIIDE.
28. Adam Summerville, Michael Cook, and Ben Steenhuisen. 2016. Draft-Analysis of the Ancients: Predicting Draft Picks in DotA 2 using Machine Learning. In Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference.
29. Superdata. 2016. 2016 MMO and MOBA Games Market. (2016). https://www.superdataresearch.com/market-data/mmo-market/
30. Superdata. 2016. Year in Review, December 2016. (2016). https://www.superdataresearch.com/market-data/market-brief-year-in-review/
31. Superdata Research. 2017. European Esports Conference Brief (http://strivesponsorship.com/wp-content/uploads/2017/04/Superdata-2017-esports-market-brief.pdf). (2017). http://strivesponsorship.com/wp-content/uploads/2017/04/Superdata-2017-esports-market-brief.pdf
32. Pin-Yun Tarng, Kuan-Ta Chen, and Polly Huang. 2009. On prophesying online gamer departure. In Proceedings of the 8th Annual Workshop on Network and Systems Support for Games. IEEE Press, 16.
33. Thanasis Vafeiadis, Konstantinos I Diamantaras, George Sarigiannidis, and K Ch Chatzisavvas. 2015. A comparison of machine learning techniques for customer churn prediction. Simulation Modelling Practice and Theory 55 (2015), 1--9.
34. Markus Viljanen, Antti Airola, Jukka Heikkonen, and Tapio Pahikkala. 2017. Playtime Measurement with Survival Analysis. arXiv preprint arXiv:1701.02359 (2017).
35. Markus Viljanen, Antti Airola, Anne-Maarit Majanoja, Jukka Heikkonen, and Tapio Pahikkala. 2017. Measuring Player Retention and Monetization using the Mean Cumulative Function. arXiv preprint arXiv:1709.06737 (2017).
36. Markus Viljanen, Antti Airola, Tapio Pahikkala, and Jukka Heikkonen. 2016. Modelling user retention in mobile games. In Computational Intelligence and Games (CIG), 2016 IEEE Conference on. IEEE, 1--8.
37. Huiwen Wang, Bang Xia, and Zhe Chen. 2015. Cultural Difference on Team Performance Between Chinese and Americans in Multiplayer Online Battle Arena Games. Springer International Publishing, Cham, 374--383.
38. M. Wu, S. Xiong, and H. Iida. 2016. Fairness mechanism in multiplayer online battle arena games. In 2016 3rd International Conference on Systems and Informatics (ICSAI). 387--392.
39. Hanting Xie, Sam Devlin, Daniel Kudenko, and Peter Cowling. 2015. Predicting Player Disengagement and First Purchase with Event-frequency Based Data Representation. In Proc. of CIG.
40. Hong Zhang. 2008. Customer retention in the financial industry: An application of survival analysis. Ph.D. Dissertation. Purdue University.

Index Terms

• Applied computing

  • Computers in other domains

    • Personal computers and PC applications

      • Computer games

• Information systems

  • Information systems applications

    • Data mining

• Mathematics of computing

  • Probability and statistics

    • Statistical paradigms

      • Survival analysis