Yan Xia
Haiyi Zhu
Skip Abstract Section
Abstract
Toxicity in online discussions has been an intriguing phenomenon and an important problem. In this paper, we seek to better understand toxicity dynamics in online discussions via a case study on Reddit that explores the antecedents and consequences of toxicity in text. We inspected two dimensions of toxicity: language toxicity, i.e. how toxic the text itself is; and toxicity elicitation, i.e. how much toxicity it elicits in its response. Through regression analyses on Reddit comments, we found that both author propensity and toxicity in discussion context were strong positive antecedents of language toxicity; meanwhile, language toxicity significantly increased the volume and user evaluation of the discussion in some sub-communities, while toxicity elicitation showed mixed effects. We then discuss how our results help understand and regulate toxicity in online discussions by interpreting the complicated triggers and outcomes of toxicity.
- Ashley A Anderson, Dominique Brossard, Dietram A Scheufele, Michael A Xenos, and Peter Ladwig. 2014. The "nasty effect:" Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, Vol. 19, 3 (2014), 373--387.Google Scholar
Digital Library
- Ashley A Anderson, Sara K Yeo, Dominique Brossard, Dietram A Scheufele, and Michael A Xenos. 2016. Toxic talk: How online incivility can undermine perceptions of media. International Journal of Public Opinion Research, Vol. 30, 1 (2016), 156--168.Google ScholarCross Ref
- Lynne M Andersson and Christine M Pearson. 1999. Tit for tat? The spiraling effect of incivility in the workplace. Academy of management review, Vol. 24, 3 (1999), 452--471.Google Scholar
- Ofer Arazy, Lisa Yeo, and Oded Nov. 2013. Stay on the Wikipedia task: When task-related disagreements slip into personal and procedural conflicts. Journal of the American Society for Information Science and Technology, Vol. 64, 8 (2013), 1634--1648.Google ScholarCross Ref
- Paul Baker. 2001. Moral panic and alternative identity construction in Usenet. Journal of Computer-Mediated Communication, Vol. 7, 1 (2001), JCMC711.Google Scholar
- Sigal G Barsade. 2002. The ripple effect: Emotional contagion and its influence on group behavior. Administrative Science Quarterly, Vol. 47, 4 (2002), 644--675.Google ScholarCross Ref
- Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. arXiv preprint arXiv:2001.08435 (2020).Google Scholar
- Robert J Bies and Thomas M Tripp. 1996. Beyond distrust. Trust in organizations (1996), 246--260.Google Scholar
- Galen V Bodenhausen, Lori A Sheppard, and Geoffrey P Kramer. 1994. Negative affect and social judgment: The differential impact of anger and sadness. European Journal of social psychology, Vol. 24, 1 (1994), 45--62.Google Scholar
- Porismita Borah. 2014. Does it matter where you read the news story? Interaction of incivility and news frames in the political blogosphere. Communication Research, Vol. 41, 6 (2014), 809--827.Google ScholarCross Ref
- Erin E Buckels, Paul D Trapnell, and Delroy L Paulhus. 2014. Trolls just want to have fun. Personality and individual Differences, Vol. 67 (2014), 97--102.Google Scholar
- Damon Centola. 2010. The spread of behavior in an online social network experiment. science, Vol. 329, 5996 (2010), 1194--1197.Google Scholar
- Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone can become a troll: Causes of trolling behavior in online discussions. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing. 1217--1230.Google ScholarDigital Library
- Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2015. Antisocial behavior in online discussion communities. In Ninth International AAAI Conference on Web and Social Media.Google Scholar
- Anna Chmiel, Pawel Sobkowicz, Julian Sienkiewicz, Georgios Paltoglou, Kevan Buckley, Mike Thelwall, and Janusz A Hołyst. 2011. Negative emotions boost user activity at BBC forum. Physica A: statistical mechanics and its applications, Vol. 390, 16 (2011), 2936--2944.Google Scholar
- Kevin Coe, Kate Kenski, and Stephen A Rains. 2014. Online and uncivil? Patterns and determinants of incivility in newspaper website comments. Journal of Communication, Vol. 64, 4 (2014), 658--679.Google ScholarCross Ref
- Lorenzo Coviello, Yunkyu Sohn, Adam DI Kramer, Cameron Marlow, Massimo Franceschetti, Nicholas A Christakis, and James H Fowler. 2014. Detecting emotional contagion in massive social networks. PloS one, Vol. 9, 3 (2014).Google Scholar
- Yogesh Dahiya, Partha Talukdar, et almbox. 2016. Discovering response-eliciting factors in social question answering: A reddit inspired study. In Tenth International AAAI Conference on Web and Social Media.Google Scholar
- Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Eleventh international aaai conference on web and social media.Google Scholar
- Munmun De Choudhury and Sushovan De. 2014. Mental health discourse on reddit: Self-disclosure, social support, and anonymity. In Eighth International AAAI Conference on Weblogs and Social Media.Google ScholarCross Ref
- Joseph de Rivera and Carmen Grinkis. 1986. Emotions as social relationships. Motivation and emotion, Vol. 10, 4 (1986), 351--369.Google Scholar
- Maeve Duggan. 2014. Online harassment .Pew Research Center.Google Scholar
- James H Fowler and Nicholas A Christakis. 2008. Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years in the Framingham Heart Study. Bmj, Vol. 337 (2008).Google ScholarCross Ref
- James H Fowler and Nicholas A Christakis. 2010. Cooperative behavior cascades in human social networks. Proceedings of the National Academy of Sciences, Vol. 107, 12 (2010), 5334--5338.Google ScholarCross Ref
- Nico H Frijda and Batja Mesquita. 1994. The social roles and functions of emotions. (1994).Google Scholar
- Iginio Gagliardone, Danit Gal, Thiago Alves, and Gabriela Martinez. 2015. Countering online hate speech .Unesco Publishing.Google Scholar
- Björn Gamb"ack and Utpal Kumar Sikdar. 2017. Using convolutional neural networks to classify hate-speech. In Proceedings of the First Workshop on Abusive Language Online. 85--90.Google Scholar
- Himaanshu Gauba, Pradeep Kumar, Partha Pratim Roy, Priyanka Singh, Debi Prosad Dogra, and Balasubramanian Raman. 2017. Prediction of advertisement preference by fusing EEG response and sentiment analysis. Neural Networks, Vol. 92 (2017), 77--88.Google ScholarCross Ref
- Jamie Guillory, Jason Spiegel, Molly Drislane, Benjamin Weiss, Walter Donner, and Jeffrey Hancock. 2011. Upset now?: emotion contagion in distributed groups. In Proceedings of the SIGCHI conference on human factors in computing systems. 745--748.Google ScholarDigital Library
- Jeffrey T Hancock, Kailyn Gee, Kevin Ciaccio, and Jennifer Mae-Hwah Lin. 2008. I'm sad you're sad: emotional contagion in CMC. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. 295--298.Google ScholarDigital Library
- Claire Hardaker. 2010. Trolling in asynchronous computer-mediated communication: From user discussions to academic definitions. Journal of politeness research, Vol. 6, 2 (2010), 215--242.Google ScholarCross Ref
- Claire Hardaker. 2013. "Uh....not to be nitpicky,,,,,but... the past tense of drag is dragged, not drug.": An overview of trolling strategies. Journal of Language Aggression and Conflict, Vol. 1, 1 (2013), 58--86.Google ScholarCross Ref
- Ali Hasan, Sana Moin, Ahmad Karim, and Shahaboddin Shamshirband. 2018. Machine learning-based sentiment analysis for twitter accounts. Mathematical and Computational Applications, Vol. 23, 1 (2018), 11.Google ScholarCross Ref
- Jared B Hawkins, John S Brownstein, Gaurav Tuli, Tessa Runels, Katherine Broecker, Elaine O Nsoesie, David J McIver, Ronen Rozenblum, Adam Wright, Florence T Bourgeois, et almbox. 2016. Measuring patient-perceived quality of care in US hospitals using Twitter. BMJ Qual Saf, Vol. 25, 6 (2016), 404--413.Google ScholarCross Ref
- Jack Hessel and Lillian Lee. 2019. Something's Brewing! Early Prediction of Controversy-causing Posts from Discussion Features. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 1648--1659.Google ScholarCross Ref
- Jack Hessel, Lillian Lee, and David Mimno. 2017. Cats and captions vs. creators and the clock: Comparing multimodal content to context in predicting relative popularity. In Proceedings of the 26th International Conference on World Wide Web. 927--936.Google ScholarDigital Library
- Benjamin D Horne, Sibel Adali, and Sujoy Sikdar. 2017. Identifying the social signals that drive online discussions: A case study of reddit communities. In 2017 26th International Conference on Computer Communication and Networks (ICCCN). IEEE, 1--9.Google ScholarCross Ref
- Hossein Hosseini, Sreeram Kannan, Baosen Zhang, and Radha Poovendran. 2017. Deceiving Google's Perspective API Built for Detecting Toxic Comments. arXiv preprint arXiv:1702.08138 (2017).Google Scholar
- Hyunseo Hwang, Youngju Kim, and Catherine U Huh. 2014. Seeing is believing: Effects of uncivil online debate on political polarization and expectations of deliberation. Journal of Broadcasting & Electronic Media, Vol. 58, 4 (2014), 621--633.Google ScholarCross Ref
- Aaron Jaech, Victoria Zayats, Hao Fang, Mari Ostendorf, and Hannaneh Hajishirzi. 2015. Talking to the crowd: What do people react to in online discussions?. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2026--2031.Google ScholarCross Ref
- Joseph M Kayany. 1998. Contexts of uninhibited online behavior: Flaming in social newsgroups on Usenet. Journal of the American Society for Information Science, Vol. 49, 12 (1998), 1135--1141.Google ScholarDigital Library
- Jihie Kim and Jaebong Yoo. 2012. Role of sentiment in message propagation: Reply vs. retweet behavior in political communication. In Social Informatics (SocialInformatics), 2012 International Conference on. IEEE, 131--136.Google ScholarDigital Library
- Adam DI Kramer. 2012. The spread of emotion via Facebook. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 767--770.Google ScholarDigital Library
- Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, Vol. 111, 24 (2014), 8788--8790.Google ScholarCross Ref
- Robert E Kraut and Paul Resnick. 2012. Building successful online communities: Evidence-based social design .Mit Press.Google ScholarDigital Library
- Travis Kriplean, Jonathan Morgan, Deen Freelon, Alan Borning, and Lance Bennett. 2012. Supporting reflective public thought with considerit. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. 265--274.Google ScholarDigital Library
- Haewoon Kwak and Jeremy Blackburn. 2014. Linguistic analysis of toxic behavior in an online video game. In International Conference on Social Informatics. Springer, 209--217.Google Scholar
- Haewoon Kwak, Jeremy Blackburn, and Seungyeop Han. 2015. Exploring cyberbullying and other toxic behavior in team competition online games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 3739--3748.Google ScholarDigital Library
- Cliff Lampe and Paul Resnick. 2004. Slash (dot) and burn: distributed moderation in a large online conversation space. In Proceedings of the SIGCHI conference on Human factors in computing systems. 543--550.Google ScholarDigital Library
- Noam Lapidot-Lefler and Azy Barak. 2012. Effects of anonymity, invisibility, and lack of eye-contact on toxic online disinhibition. Computers in human behavior, Vol. 28, 2 (2012), 434--443.Google ScholarDigital Library
- Hangwoo Lee. 2005. Behavioral strategies for dealing with flaming in an online forum. The Sociological Quarterly, Vol. 46, 2 (2005), 385--403.Google ScholarCross Ref
- So-Hyun Lee and Hee-Woong Kim. 2015. Why people post benevolent and malicious comments online. Commun. ACM, Vol. 58, 11 (2015), 74--79.Google ScholarDigital Library
- Jennifer S Lerner, Julie H Goldberg, and Philip E Tetlock. 1998. Sober second thought: The effects of accountability, anger, and authoritarianism on attributions of responsibility. Personality and Social Psychology Bulletin, Vol. 24, 6 (1998), 563--574.Google ScholarCross Ref
- Steven Loria, P Keen, M Honnibal, R Yankovsky, D Karesh, E Dempsey, et almbox. 2014. Textblob: simplified text processing. Secondary TextBlob: Simplified Text Processing (2014).Google Scholar
- Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web. 145--153.Google ScholarDigital Library
- Patrick B O'sullivan and Andrew J Flanagin. 2003. Reconceptualizing 'flaming' and other problematic messages. New Media & Society, Vol. 5, 1 (2003), 69--94.Google ScholarCross Ref
- Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and trends in information retrieval, Vol. 2, 1--2 (2008), 1--135.Google Scholar
- Deokgun Park, Simranjit Sachar, Nicholas Diakopoulos, and Niklas Elmqvist. 2016. Supporting comment moderators in identifying high quality online news comments. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1114--1125.Google ScholarDigital Library
- John Pavlopoulos, Prodromos Malakasiotis, and Ion Androutsopoulos. 2017. Deeper attention to abusive user content moderation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 1125--1135.Google ScholarCross Ref
- Fabian Prochazka, Patrick Weber, and Wolfgang Schweiger. 2018. Effects of civility and reasoning in user comments on perceived journalistic quality. Journalism Studies, Vol. 19, 1 (2018), 62--78.Google ScholarCross Ref
- Adrian Raine. 2002. Annotation: The role of prefrontal deficits, low autonomic arousal, and early health factors in the development of antisocial and aggressive behavior in children. Journal of Child Psychology and Psychiatry, Vol. 43, 4 (2002), 417--434.Google ScholarCross Ref
- reddit contributors. 2020 a. Reddiquette | Reddit Help. https://www.reddithelp.com/en/categories/reddit-101/reddit-basics/reddiquette [Online; accessed 16-May-2020].Google Scholar
- reddit contributors. 2020 b. What is karma? | Reddit Help. https://www.reddithelp.com/en/categories/reddit-101/reddit-basics/what-karma [Online; accessed 16-May-2020].Google Scholar
- Leonie Rösner and Nicole C Kr"amer. 2016. Verbal venting in the social web: Effects of anonymity and group norms on aggressive language use in online comments. Social MediaGoogle Scholar
- Society, Vol. 2, 3 (2016).Google Scholar
- Leonie Rösner, Stephan Winter, and Nicole C Kr"amer. 2016. Dangerous minds? Effects of uncivil online comments on aggressive cognitions, emotions, and behavior. Computers in Human Behavior, Vol. 58 (2016), 461--470.Google ScholarDigital Library
- Jeffrey Z Rubin, Dean G Pruitt, and Sung Hee Kim. 1994. Social conflict: Escalation, stalemate, and settlement .Mcgraw-Hill Book Company.Google Scholar
- Haji Mohammad Saleem, Kelly P Dillon, Susan Benesch, and Derek Ruths. 2017. A web of hate: Tackling hateful speech in online social spaces. arXiv preprint arXiv:1709.10159 (2017).Google Scholar
- Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1668--1678.Google ScholarCross Ref
- Pnina Shachaf and Noriko Hara. 2010. Beyond vandalism: Wikipedia trolls. Journal of Information Science, Vol. 36, 3 (2010), 357--370.Google ScholarDigital Library
- John Suler. 2004. The online disinhibition effect. Cyberpsychology & behavior, Vol. 7, 3 (2004), 321--326.Google Scholar
- Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th international conference on world wide web. 613--624.Google ScholarDigital Library
- Wikimedia Support & Safety Team. 2015. Harassment Survey 2015. https://upload.wikimedia.org/wikipedia/commons/5/52/Harassment_Survey_2015_-_Results_Report.pdf [Online; accessed 4-April-2019].Google Scholar
- Jessica Vitak, Kalyani Chadha, Linda Steiner, and Zahra Ashktorab. 2017. Identifying women's experiences with and strategies for mitigating negative effects of online harassment. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 1231--1245.Google ScholarDigital Library
- William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In Proceedings of the second workshop on language in social media. 19--26.Google ScholarDigital Library
- Tim Weninger, Xihao Avi Zhu, and Jiawei Han. 2013. An exploration of discussion threads in social news sites: A case study of the reddit community. In 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013). IEEE, 579--583.Google ScholarDigital Library
- Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web. 1391--1399.Google ScholarDigital Library
- Guang Xiang, Bin Fan, Ling Wang, Jason Hong, and Carolyn Rose. 2012. Detecting offensive tweets via topical feature discovery over a large scale twitter corpus. In Proceedings of the 21st ACM international conference on Information and knowledge management. 1980--1984.Google ScholarDigital Library
- Dawei Yin, Zhenzhen Xue, Liangjie Hong, Brian D Davison, April Kontostathis, and Lynne Edwards. 2009. Detection of harassment on web 2.0. Proceedings of the Content Analysis in the WEB, Vol. 2 (2009), 1--7.Google Scholar
- Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. Conversations Gone Awry: Detecting Early Signs of Conversational Failure. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1350--1361.Google ScholarCross Ref
- Yabing Zhao, Xun Xu, and Mingshu Wang. 2019. Predicting overall customer satisfaction: Big data evidence from hotel online textual reviews. International Journal of Hospitality Management, Vol. 76 (2019), 111--121.Google ScholarCross Ref
- Marc Ziegele, Timo Breiner, and Oliver Quiring. 2014. What creates interactivity in online news discussions? An exploratory analysis of discussion factors in user comments on news items. Journal of Communication, Vol. 64, 6 (2014), 1111--1138.Google ScholarCross Ref
Index Terms
- Exploring Antecedents and Consequences of Toxicity in Online Discussions: A Case Study on Reddit
Recommendations
Investigating Toxicity Across Multiple Reddit Communities, Users, and Moderators
WWW '20: Companion Proceedings of the Web Conference 2020Online platforms like Reddit enable users to build communities and converse about diverse topics and interests. However, with the increasing number of users that post disturbing comments containing profanity, harassment, and hate speech, otherwise known ...
Are These Comments Triggering? Predicting Triggers of Toxicity in Online Discussions
WWW '20: Proceedings of The Web Conference 2020Understanding the causes or triggers of toxicity adds a new dimension to the prevention of toxic behavior in online discussions. In this research, we define toxicity triggers in online discussions as a non-toxic comment that lead to toxic replies. Then, ...
Detecting Toxicity Triggers in Online Discussions
HT '19: Proceedings of the 30th ACM Conference on Hypertext and Social MediaDespite the considerable interest in the detection of toxic comments, there has been little research investigating the causes -- i.e., triggers -- of toxicity. In this work, we first propose a formal definition of triggers of toxicity in online ...
Comments