Abstract
This paper replicates, extends, and refutes conclusions made in a study published in PLoS ONE ("Even Good Bots Fight"), which claimed to identify substantial levels of conflict between automated software agents (or bots) in Wikipedia using purely quantitative methods. By applying an integrative mixed-methods approach drawing on trace ethnography, we place these alleged cases of bot-bot conflict into context and arrive at a better understanding of these interactions. We found that overwhelmingly, the interactions previously characterized as problematic instances of conflict are typically better characterized as routine, productive, even collaborative work. These results challenge past work and show the importance of qualitative/quantitative collaboration. In our paper, we present quantitative metrics and qualitative heuristics for operationalizing bot-bot conflict. We give thick descriptions of kinds of events that present as bot-bot reverts, helping distinguish conflict from non-conflict. We computationally classify these kinds of events through patterns in edit summaries. By interpreting found/trace data in the socio-technical contexts in which people give that data meaning, we gain more from quantitative measurements, drawing deeper understandings about the governance of algorithmic systems in Wikipedia. We have also released our data collection, processing, and analysis pipeline, to facilitate computational reproducibility of our findings and to help other researchers interested in conducting similar mixed-method scholarship in other platforms and contexts.
- Phil Agre. 1997. Beyond the mirror world: Privacy and the representational practices of computing. Technology and Privacy: The New Landscape. The MIT Press, Cambridge. Google ScholarDigital Library
- Solon Barocas, Sophie Hood, and Malte Ziewitz. 2013. Governing Algorithms: A Provocation Piece. SSRN (2013).Google Scholar
- Jeremy Birnholtz, Colin Fitzpatrick, Mark Handel, and Jed R. Brubaker. 2014. Identity, Identification and Identifiability: The Language of Self-presentation on a Location-based Mobile Dating App. In Proc MobileHCI 2014. ACM, 3--12. Google ScholarDigital Library
- Jorge Luis Borges. 2015 {1946}. On Exactitude in Science. Collected Fictions. Penguin. https://kwarc.info/teaching/TDM/Borges.pdfGoogle Scholar
- Geoffrey Bowker. 2005. Memory Practices in the Sciences. MIT Press, Cambridge, Mass.Google Scholar
- Geoffrey Bowker and Susan Leigh Star. 1999. Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge. Google ScholarDigital Library
- Jed R. Brubaker and Gillian R. Hayes. 2011. SELECT * FROM USER: infrastructure and socio-technical representation Proc CSCW 2011. ACM Press, New York, 369.Google Scholar
- Moira Burke and Robert Kraut. 2008. Mopping up Proceedings of the ACM 2008 conference on Computer supported cooperative work - CSCW '08. ACM Press, New York, 27.Google Scholar
- Brian Butler, Elisabeth Joyce, Jacqueline Pike, Elisabeth Joyce, and Brian Butler. 2008. Don't look now, but we've created a bureaucracy: the nature and roles of policies and rules in wikipedia. In Proc CHI 2008. ACM, Florence, 1101--1110. lot2.org Google ScholarDigital Library
- Taha Yasseri, Robert Sumi, András Rung, András Kornai, and János Kertész. 2012. Dynamics of Conflicts in Wikipedia. PLoS ONE, Vol. 7, 6 (jun. 2012), e38869. 1932--6203Google ScholarCross Ref
- J. Yates and W.J. Orlikowski. 1992. Genres of organizational communication: a structurational approach to studying communication and media. Academy of Management Review Vol. 17 (1992), 299--326.Google ScholarCross Ref
Index Terms
- Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of 'Even Good Bots Fight'
Recommendations
The work of sustaining order in wikipedia: the banning of a vandal
CSCW '10: Proceedings of the 2010 ACM conference on Computer supported cooperative workIn this paper, we examine the social roles of software tools in the English-language Wikipedia, specifically focusing on autonomous editing programs and assisted editing tools. This qualitative research builds on recent research in which we ...
Genre, knowledge and digital code in web-based communities: an integrated theoretical framework for shaping digital discursive spaces
Emerging digital discursive spaces, such as wikis, offer new opportunities for knowledge communication. However, participants join such spaces through the lenses of their established discursive practices. These practices, however, interact with the code – ...
When the levee breaks: without bots, what happens to Wikipedia's quality control processes?
WikiSym '13: Proceedings of the 9th International Symposium on Open CollaborationIn the first half of 2011, ClueBot NG -- one of the most prolific counter-vandalism bots in the English-language Wikipedia -- went down for four distinct periods, each period of downtime lasting from days to weeks. In this paper, we use these periods of ...
Comments