What Makes a Strong Team?: Using Collective Intelligence to Predict Team Performance in League of Legends

ABSTRACT

Recent research has demonstrated that (a) groups can be characterized by a collective intelligence (CI) factor that measures their ability to perform together on a wide range of different tasks, and (b) this factor can predict groups' performance on other tasks in the future. The current study examines whether these results translate into the world of teams in competitive online video games where self-organized, time-pressured, and intense collaboration occurs purely online. In this study of teams playing the online game League of Legends, we find that CI does, indeed, predict the competitive performance of teams controlling for the amount of time played as a team. We also find that CI is positively correlated with the presence of a female team member and with the team members' average social perceptiveness. Finally, unlike in prior studies, tacit coordination in this setting plays a larger role than verbal communication.

References

1. Ishani Aggarwal, Anita Williams Woolley, Christopher F. Chabris, and Thomas W. Malone. 2015. Cognitive diversity, collective intelligence, and learning in teams. Proceedings of Collective Intelligence 2015.
2. Muhammad Aurangzeb Ahmad, Cuihua Shen, Jaideep Srivastava, and Noshir Contractor. 2014. On the problem of predicting real world characteristics from virtual worlds. In Predicting Real World Behaviors from Virtual World Data, Muhammad Aurangzeb Ahmad, Cuihua Shen, Jaideep Srivastava and Noshir Contractor (eds.). Springer, 1--18.
3. Ali E Akgün and Gary S Lynn. 2002. Antecedents and consequences of team stability on new product development performance. Journal of Engineering and Technology Management 19, 3-4: 263--286.
4. Linda Argote. 1982. Input uncertainty and organizational coordination in hospital emergency units. Administrative Science Quarterly 27: 420--434.
5. William Sims Bainbridge. 2007. The scientific research potential of virtual worlds. Science 317, 5837: 472--476.
6. Simon Baron-Cohen, Sally Wheelwright, Jacqueline Hill, Yogini Raste, and Ian Plumb. 2001. The "Reading the Mind in the Eyes" Test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. The Journal of Child Psychology and Psychiatry and Allied Disciplines 42, 2: 241--251.
7. Grace A. Benefield, Cuihua Shen, and Alex Leavitt. 2016. Virtual team networks: How group social capital affects team success in a Massively Multiplayer Online Game. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, ACM, 679--690.
8. Joël Billieux, Martial Van der Linden, Sophia Achab, et al. 2013. Why do you play World of Warcraft? An in-depth exploration of self-reported motivations to play online and in-game behaviours in the virtual world of Azeroth. Computers in Human Behavior 29, 1: 103--109.
9. Christopher F. Chabris. 2007. Cognitive and neurobiological mechanisms of the law of general intelligence. In Integrating the Mind: Domain General versus Domain specific Processes in Higher Cognition, M. J Roberts (ed.). Psychology Press, Hove, UK, 449--491.
10. Chien-Hsun Chen, Chuen-Tsai Sun, and Jilung Hsieh. 2008. Player guild dynamics and evolution in massively multiplayer online games. CyberPsychology & Behavior 11, 3: 293--301.
11. Dongseong Choi and Jinwoo Kim. 2004. Why people continue to play online games: In search of critical design factors to increase customer loyalty to online contents. CyberPsychology & Behavior 7, 1: 11--24.
12. Laura Dabbish and Robert Kraut. 2008. Awareness displays and social motivation for coordinating communication. Information Systems Research 19, 2: 221--238.
13. Ian J. Deary. 2012. Intelligence. Annual Review of Psychology 63: 17.1--17.30.
14. Dennis Devine and Jennifer Philips. 2001. Do smarter teams do better: A meta-analysis of cognitive ability and team performance. Small Group Research 32, 5: 507--532.
15. Paul Dourish and Victoria Bellotti. 1992. Awareness and coordination in shared workspaces. Proceedings of the 1992 ACM Conference on Computer-supported Cooperative Work, ACM, 107--114.
16. Amy Edmondson. 1999. Psychological safety and learning behavior in work teams. Administrative Science Quarterly. 44, 2: 350--383.
17. David Engel, Anita Williams Woolley, Ishani Aggarwal, et al. 2015. Collective intelligence in online collaboration emerges in different contexts and cultures. CHI '15 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
18. David Engel, Anita Williams Woolley, Lisa X. Jing, Christopher F. Chabris, and Thomas W. Malone. 2014. Reading the mind in the eyes or reading between the lines? Theory of Mind predicts collective intelligence equally well online and face-to-face. PLoS ONE 9, 12.
19. Simon Ferrari. 2014. From generative to conventional play: MOBA and League of Legends. Proceedings of DiGRA. 2013. Retrieved from http://www.digra.org/wp-content/uploads/digitallibrary/paper_230_formattingfixed.pdf
20. Darren Gergle, Robert E. Kraut, and Susan R. Fussell. 2013. Using visual information for grounding and awareness in collaborative tasks. Human-Computer Interaction 28, 1: 1--39.
21. Cristina B. Gibson and Susan G. Cohen. 2003. Virtual teams that work: Creating conditions for virtual team effectiveness. Jossey-Bass, San Francisco.
22. Cristina B Gibson and Jennifer L. Gibbs. 2006. Unpacking the concept of virtuality: The effects of geographic dispersion, electronic dependence, dynamic structure, and national diversity on team innovation. Administrative Science Quarterly 51, 3: 451.
23. Martine Haas and Mark Mortensen. 2016. The secrets of great teamwork. Harvard Business Review, June: 70--76.
24. John Halloran, Geraldine Fitzpatrick, Yvonne Rogers, and Paul Marshall. 2004. Does it matter if you don't know who's talking?: Multiplayer gaming with Voiceover IP. CHI '04 Extended Abstracts on Human Factors in Computing Systems, ACM, 1215--1218.
25. Andrew F. Hayes. 2013. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. Guilford Press.
26. Yun Huang, Mengxiao Zhu, Jing Wang, et al. 2009. The formation of task-oriented groups: Exploring combat activities in online games. International Conference on Computational Science and Engineering, 2009. CSE '09, 122--127.
27. David Huffaker, Jing Wang, Jeffrey Treem, et al. 2009. The social behaviors of experts in massive multiplayer online role-playing games. International Conference on Computational Science and Engineering, 2009. CSE '09, 326--331.
28. Sirkka L. Jarvenpaa, Kathleen Knoll, and Dorothy E. Leidner. 1998. Is anybody out there? Antecedents of trust in global virtual teams. Journal of Management Information Systems 14, 4: 29--64.
29. Adam S. Kahn, Cuihua Shen, Li Lu, et al. 2015. The Trojan Player Typology: A cross-genre, cross-cultural, behaviorally validated scale of video game play motivations. Computers in Human Behavior 49: 354--361.
30. Bradley L. Kirkman, Benson Rosen, Cristina B. Gibson, Paul E. Tesluk, and Simon O. McPherson. 2002. Five challenges to virtual team success: Lessons from Sabre, Inc. Academy of Management Executive 16, 3: 67--79.
31. Robert E. Kraut, Darren Gergle, and Susan R. Fussell. 2002. The use of visual information in shared visual spaces: Informing the development of virtual copresence. Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work, ACM, 31--40.
32. James R. Larson. 2009. In search of synergy in small group performance. Psychology Press, NY, NY.
33. Alex Leavitt, Brian C. Keegan, and Joshua Clark. 2016. Ping to win?: Non-verbal communication and team performance in competitive online multiplayer games. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ACM, 4337--4350.
34. K. Lewis, M. Belliveau, B. Herndon, and J. Keller. 2007. Group cognition, membership change, and performance: Investigating the benefits and detriments of collective knowledge. Organizational Behavior and Human Decision Processes 103: 159--178.
35. Ann Majchrzak, Sirkka L. Jarvenpaa, and Andrea B. Hollingshead. 2007. Coordinating expertise among emergent groups responding to disasters. Organization Science 18, 1: 147--161.
36. Joseph E. McGrath. 1984. Groups: Interaction and Performance. Prentice-Hall, Englewood Cliffs, NJ.
37. Jingbo Meng, Dmitri Williams, and Cuihua Shen. 2015. Channels matter: Multimodal connectedness, types of co-players and social capital for Multiplayer Online Battle Arena gamers. Computers in Human Behavior 52: 190--199.
38. Philip H. Mirvis. 1998. Practice improvisation. Organization Science 9, 5: 586--592.
39. Erik E. Noftle and Richard W. Robins. 2007. Personality predictors of academic outcomes: Big five correlates of GPA and SAT scores. Journal of Personality and Social Psychology 93, 1: 116--130.
40. G. P. Pisano, R. M. J. Bohmer, and A. C. Edmondson. 2001. Organizational differences in rates of learning: Evidence from the adoption of minimally invasive cardiac surgery. Management Science. 47, 6: 752--768.
41. Rabindra A. Ratan, Nicholas Taylor, Jameson Hogan, Tracy Kennedy, and Dmitri Williams. 2015. Stand by your man: An examination of gender disparity in League of Legends. Games and Culture: 1555412014567228.
42. Byron Reeves, Thomas W. Malone, and Tony O'Driscoll. 2008. Leadership's online labs. Harvard Business Review 86, 5: 58--66.
43. Dennis M. Rousseau. 1985. Issues of level in organizational research: Multi-level and cross-level perspectives. In Research in Organizational Behavior, L. L Cummings and B. Staw (eds.). JAI, Greenwich, CT, 1--37.
44. Keith Sawyer. 1992. Improvisational creativity: An analysis of jazz performance. Creativity Research Journal 5, 3: 253--263.
45. Cuihua Shen, Peter Monge, and Dmitri Williams. 2014. The evolution of social ties online: A longitudinal study in a massively multiplayer online game. Journal of the Association for Information Science and Technology 65, 10: 2127--2137.
46. Cuihua Shen, Rabindra Ratan, Y. Dora Cai, and Alex Leavitt. 2016. Do men advance faster than women? Debunking the gender performance gap in two massively multiplayer online games. Journal of Computer-Mediated Communication: n/a-n/a.
47. Kenneth B. Shores, Yilin He, Kristina L. Swanenburg, Robert Kraut, and John Riedl. 2014. The identification of deviance and its impact on retention in a multiplayer game. Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, ACM, 1356--1365.
48. Charles Spearman. 1904. General intelligence, objectively determined and measured. American Journal of Psychology 15, 2: 201--293.
49. Roderick I. Swaab, Michael Schaerer, Eric M. Anicich, Richard Ronay, and Adam D. Galinsky. 2014. The toomuch-talent effect team interdependence determines when more talent Is too much or not enough. Psychological Science 25, 8: 1581--1591.
50. Zachary O. Toups, Jessica Hammer, William A. Hamilton, Ahmad Jarrah, William Graves, and Oliver Garretson. 2014. A framework for cooperative communication game mechanics from grounded theory. Proceedings of the First ACM SIGCHI Annual Symposium on Computer-human Interaction in Play, ACM, 257--266.
51. John B. Van Huyck, Raymond C. Battalio, and Richard O. Beil. 1990. Tacit coordination games, strategic uncertainty, and coordination failure. The American Economic Review 80, 1: 234--248.
52. Ruth Wageman, J. Richard Hackman, and Erin Lehman. 2005. Team diagnostic survey. Journal of Applied Behavioral Science 41, 4: 373--398.
53. Dmitri Williams. 2010. The mapping principle, and a research framework for virtual worlds. Communication Theory (10503293) 20, 4: 451--470.
54. Dmitri Williams, Scott E. Caplan, and Li Xiong. Can you hear me now? The impact of voice in an online gaming community. 33, 4: 427--449.
55. Gwen M. Wittenbaum, Garold Stasser, and Carol J. Merry. 1996. Tacit coordination in anticipation of small group task completion. Journal of Experimental Social Psychology 32, 2: 129--152.
56. Anita Williams Woolley, Ishani Aggarwal, and Thomas W. Malone. 2015. Collective intelligence and group performance. Current Directions in Psychological Science 24, 6: 420--424.
57. A. W. Woolley, C. F. Chabris, A. Pentland, N. Hashmi, and Thomas W. Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. Science 330, 6004: 686--688.
58. Nick Yee, Jeremy N. Bailenson, Mark Urbanek, Francis Chang, and Dan Merget. 2007. The unbearable likeness of being digital: The persistence of nonverbal social norms in online virtual environments. CyberPsychology & Behavior 10, 1: 115--121.
59. Honor Begging - League of Legends Community. Retrieved May 27, 2016 from http://forums.na.leagueoflegends.com/board/showthrea d.php-t=2662411

Index Terms

• Human-centered computing

  • Collaborative and social computing