Paula Fortuna
Skip Abstract Section
Abstract
The scientific study of hate speech, from a computer science point of view, is recent. This survey organizes and describes the current state of the field, providing a structured overview of previous approaches, including core algorithms, methods, and main features used. This work also discusses the complexity of the concept of hate speech, defined in many platforms and contexts, and provides a unifying definition. This area has an unquestionable potential for societal impact, particularly in online communities and digital media platforms. The development and systematization of shared resources, such as guidelines, annotated datasets in multiple languages, and algorithms, is a crucial step in advancing the automatic detection of hate speech.
- ACL. 2017. ALW1: 1st workshop on abusive language online. Retrieved from https://sites.google.com/site/abusivelanguageworkshop2017/home.Google Scholar
- Swati Agarwal and Ashish Sureka. 2015. Using KNN and SVM based one-class classifier for detecting online radicalization on Twitter. In Proceedings of the International Conference on Distributed Computing and Internet Technology. Springer, 431--442. Google ScholarDigital Library
- Swati Agarwal and Ashish Sureka. 2017. Characterizing linguistic attributes for automatic classification of intent based racist/radicalized posts on tumblr micro-blogging website. arXiv Preprint arXiv:1701.04931 (2017).Google Scholar
- Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 759--760. Google ScholarDigital Library
- Tanvi Banerjee, Amir H. Yazdavar, Andrew Hampton, Hemant Purohit, Valerie L. Shalin, and Amit P. Sheth. Identifying pragmatic functions in social media indicative of gender-based violence beliefs. Manuscript Submitted for Publication.Google Scholar
- Jamie Bartlett, Richard Norrie, Sofia Patel, Rebekka Rumpel, and Simon Wibberley. 2014. Misogyny on Twitter. Technical Report. Demos.Google Scholar
- Peter Burnap and Matthew L. Williams. 2014. Hate speech, machine classification and statistical modelling of information flows on Twitter: Interpretation and communication for policy decision making. In Proceedings of the Conference on the Internet, Policy 8 Politics. 1--18.Google Scholar
- Pete Burnap and Matthew L. Williams. 2015. Cyber hate speech on Twitter: An application of machine classification and statistical modeling for policy and decision making. Policy Internet 7, 2 (2015), 223--242.Google ScholarCross Ref
- Pete Burnap and Matthew L. Williams. 2016. Us and them: Identifying cyber hate on Twitter across multiple protected characteristics. EPJ Data Sci. 5, 1 (2016), 11.Google ScholarCross Ref
- Ying Chen. 2011. Detecting Offensive Language in Social Medias for Protection of Adolescent Online Safety. Ph.D. Dissertation. The Pennsylvania State University.Google Scholar
- CHI2017. 2017. 2017 Workshop on online harassment. Retrieved from http://social.umd.edu/woh/.Google Scholar
- CLiPS. 2016. HADES. Retrieved from https://github.com/clips/hades.Google Scholar
- CONTACT. 2017. Interdisciplinary conference on Hate speech. Definitions, Interpretations and Practices. Retrieved from https://sites.google.com/site/abusivelanguageworkshop2017/home.Google Scholar
- Keith Cortis and Siegfried Handschuh. 2015. Analysis of cyberbullying tweets in trending world events. In Proceedings of the 15th International Conference on Knowledge Technologies and Data-driven Business. ACM, 7. Google ScholarDigital Library
- CrowdFlower. 2017. Data for everyone. Retrieved from https://www.crowdflower.com/data-for-everyone/.Google Scholar
- Maral Dadvar, Franciska de Jong, Roeland Ordelman, and Dolf Trieschnigg. 2012. Improved cyberbullying detection using gender information. In Proceedings of the 12th Dutch-Belgian Information Retrieval Workshop. University of Ghent, 23--25.Google Scholar
- Ali A. Dashti, Ali A. Al-Kandari, and Hamed H. Al-Abdullah. 2015. The influence of sectarian and tribal discourse in newspapers readers’ online comments about freedom of expression, censorship and national unity in Kuwait. Telemat. Informat. 32, 2 (2015), 245--253. Google ScholarDigital Library
- Thomas Davidson. 2017. Automated hate speech detection and the problem of offensive language. Retrieved from https://github.com/t-davidson/hate-speech-and-offensive-language.Google Scholar
- Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. arXiv Preprint arXiv:1703.04009 (2017).Google Scholar
- Marie-Catherine De Marneffe and Christopher D. Manning. 2008. Stanford Typed Dependencies Manual. Technical report, Stanford University.Google Scholar
- Guy De Pauw, Ben Verhoeven, Bart Desmet, and Els Lefever. 2016. First workshop on text analytics for cybersecurity and online safety (TA-COS 2016). In Proceedings of the 1st Workshop on Text Analytics for Cybersecurity and Online Safety (TACOS’16), collocated with the 10th International Conference on Language Resources and Evaluation (LREC’16). European Language Resources Association.Google Scholar
- Fabio Del Vigna, Andrea Cimino, Felice Dell’Orletta, Marinella Petrocchi, and Maurizio Tesconi. 2017. Hate me, hate me not: Hate speech detection on Facebook. In Proceedings of the 1st Italian Conference on Cybersecurity. 86--95.Google Scholar
- Cambridge Dictionary. 2017. Profanity. Retrieved from https://dictionary.cambridge.org/dictionary/english/profanity.Google Scholar
- Karthik Dinakar, Roi Reichart, and Henry Lieberman. 2011. Modeling the detection of textual cyberbullying. Soc. Mobile Web 11, 02 (2011).Google Scholar
- Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate speech detection with comment embeddings. In Proceedings of the 24th International Conference on World Wide Web. ACM, 29--30. Google ScholarDigital Library
- Sara Douglass, Sheena Mirpuri, Devin English, and Tiffany Yip. 2016. They were just making jokes: Ethnic/racial teasing and discrimination among adolescents. Cultur. Divers. Ethnic Minor. Psychol. 22, 1 (2016), 69.Google ScholarCross Ref
- Eyspahn. 2016. Online hate speech modeling using Python and reddit comment data. Retrieved from https://github.com/eyspahn/OnlineHateSpeech_PyLadiesSea.Google Scholar
- Facebook. 2013. What does Facebook consider to be hate speech? Retrieved from https://www.facebook.com/help/135402139904490.Google Scholar
- FBI. 2015. 2015 hate crime statistics. Retrieved from https://ucr.fbi.gov/hate-crime/.Google Scholar
- Fabio Giblietto and Yenn Lee. 2015. To be or not to be Charlie: Twitter hashtags as a discourse and counter-discourse in the aftermath of the 2015 Charlie Hebdo shooting in France. In Proceedings of the 4th Workshop on Making Sense of Microposts (#Microposts’14).Google Scholar
- Njagi Dennis Gitari, Zhang Zuping, Hanyurwimfura Damien, and Jun Long. 2015. A lexicon-based approach for hate speech detection. Int. J. Multimedia Ubiq. Eng. 10, 4 (2015), 215--230.Google ScholarCross Ref
- Edel Greevy. 2004. Automatic Text Categorisation of Racist Webpages. Ph.D. Dissertation. Dublin City University.Google Scholar
- Edel Greevy and Alan F. Smeaton. 2004. Classifying racist texts using a support vector machine. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 468--469. Google ScholarDigital Library
- Stanford NLP Group. 2017. The Stanford NLP Group. Retrieved from http://nlp.stanford.edu/.Google Scholar
- Radhouane Guermazi, Mohamed Hammami, and Abdelmajid Ben Hamadou. 2007. Using a semi-automatic keyword dictionary for improving violent Web site filtering. In Proceedings of the 3rd International IEEE Conference on Signal-Image Technologies and Internet-Based System (SITIS’07). IEEE, 337--344. Google ScholarDigital Library
- Yannis Haralambous and Philippe Lenca. 2014. Text classification using association rules, dependency pruning and hyperonymization. arXiv Preprint arXiv:1407.7357 (2014). Google ScholarDigital Library
- Reporting Hate. 2017. Hate speech conference I.H.D.I.P. Retrieved from http://reportinghate.eu/contact2017/.Google Scholar
- No hate speech movement. 2017. No hate speech movement. Retrieved from https://www.nohatespeechmovement.org/.Google Scholar
- Hatebase. 2017. Hatebase. Retrieved from https://www.hatebase.org/.Google Scholar
- Alex Hern. 2016. Facebook, YouTube, Twitter, and Microsoft sign EU hate speech code. Retrieved from https://www.theguardian.com/technology/2016/may/31/facebook-youtube-twitter-microsoft-eu-hate-speech-code.Google Scholar
- Sarah Hewitt, Thanassis Tiropanis, and Christian Bokhove. 2016. The problem of identifying misogynist language on Twitter (and other online social spaces). In Proceedings of the 8th ACM Conference on Web Science. ACM, 333--335. Google ScholarDigital Library
- ILGA. 2016. Hate crime and hate speech. Retrieved from http://www.ilga-europe.org/what-we-do/our-advocacy-work/hate-crime-hate-speech.Google Scholar
- Jigsaw. 2017. Perspective API. Retrieved from https://www.perspectiveapi.com/.Google Scholar
- Kaggle. 2013. Detecting insults in social commentary. Retrieved from https://www.kaggle.com/c/detecting-insults-in-social-commentary/data.Google Scholar
- Panos Kompatsiaris. 2016. Whitewashing the nation: Racist jokes and the construction of the african “other” in Greek popular cinema. Soc. Ident. 23, 3 (2016), 360--375.Google ScholarCross Ref
- Ivana Kottasová. 2017. Europe says Twitter is failing to remove hate speech. Retrieved from http://money.cnn.com/2017/06/01/technology/twitter-facebook-hate-speech-europe/index.html.Google Scholar
- Till Krause and Hannes Grassegger. 2016. Facebook’s secret rules of deletion. Retrieved from http://international.sueddeutsche.de/post/154543271930/facebooks-secret-rules-of-deletion.Google Scholar
- Klaus Krippendorff. 2004. Content Analysis: An Introduction to Its Methodology. Sage.Google Scholar
- Giselinde Kuipers and Barbara van der Ent. 2016. The seriousness of ethnic jokes: Ethnic humor and social change in The Netherlands, 1995--2012. Humor 29, 4 (2016), 605--633.Google ScholarCross Ref
- Irene Kwok and Yuzhou Wang. 2013. Locate the hate: Detecting tweets against blacks. In Proceedings of the Association for the Advancement of Artificial Intelligence. Google ScholarDigital Library
- Anti-Defamation League. 2015. The trap of masculinity: how sexism impacts boys and men. Retrieved from https://www.adl.org/sites/default/files/documents/assets/pdf/education-outreach/trap-of-masculinity.pdf.Google Scholar
- Shuhua Liu and Thomas Forss. 2014. Combining N-gram based similarity analysis with sentiment analysis in web content classification. In Proceedings of the International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management. 530--537. Google ScholarDigital Library
- Shuhua Liu and Thomas Forss. 2015. New classification models for detecting hate and violence web content. In Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K’15), Vol. 1. IEEE, 487--495. Google ScholarDigital Library
- Wilson Jeffrey Maloba. 2014. Use of Regular Expressions for Multi-lingual Detection of Hate Speech in Kenya. Ph.D. Dissertation. iLabAfrica.Google Scholar
- Lacy G. McNamee, Brittany L. Peterson, and Jorge Peña. 2010. A call to educate, participate, invoke. and indict: Understanding the communication of online hate groups. Commun. Monogr. 77, 2 (2010), 257--280.Google ScholarCross Ref
- Yashar Mehdad and Joel Tetreault. 2016. Do characters abuse more than words? In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial’16). 299--303.Google ScholarCross Ref
- B. Nandhini and J. I. Sheeba. 2015. Cyberbullying detection and classification using information retrieval algorithm. In Proceedings of the 2015 International Conference on Advanced Research in Computer Science Engineering 8 Technology (ICARCSET’15). ACM, 20. Google ScholarDigital Library
- Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 145--153. Google ScholarDigital Library
- Andre Oboler and Karen Connelly. 2014. Hate speech: A quality of service challenge. In Proceedings of the IEEE Conference on e-Learning, e-Management, and e-Services (IC3e’14). IEEE, 117--121.Google ScholarCross Ref
- United Nations Alliance of Civilizations (UNAOC). 2017. #SpreadNoHate: A global dialogue on hate speech against migrants and refugees in the media. Retrieved from https://www.unaoc.org/what-we-do/projects/hate-speech/.Google Scholar
- David Martin Powers. 2011. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. Int. J. Mach. Learn. Technol. 2, 1 (2011), 37--63.Google ScholarCross Ref
- Sheryl Prentice, Paul J. Taylor, Paul Rayson, Andrew Hoskins, and Ben O’Loughlin. 2011. Analyzing the semantic content and persuasive composition of extremist media: A case study of texts produced during the Gaza conflict. Info. Syst. Front. 13, 1 (2011), 61--73. Google ScholarDigital Library
- Elaheh Raisi and Bert Huang. 2016. Cyberbullying identification using participant-vocabulary consistency. arXiv Preprint arXiv:1606.08084 (2016).Google Scholar
- Vasu Reddy. 2002. Perverts and sodomites: Homophobia as hate speech in Africa. South. African Linguist. Appl. Lang. Studies 20, 3 (2002), 163--175.Google ScholarCross Ref
- Bjorn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils Kurowsky, and Michael Wojatzki. 2017. Measuring the reliability of hate speech annotations: The case of the european refugee crisis. arXiv Preprint arXiv:1701.08118 (2017).Google Scholar
- Anna Schmidt and Michael Wiegand. 2017. A survey on hate speech detection using natural language processing. In Proceedings of the Workshop on Natural Language Processing for Social Media (SocialNLP’17). 1.Google ScholarCross Ref
- Leandro Silva, Mainack Mondal, Denzil Correa, Fabrício Benevenuto, and Ingmar Weber. 2016. Analyzing the targets of hate in online social media. arXiv Preprint arXiv:1603.07709 (2016).Google Scholar
- Natalya Tarasova. 2016. Classification of Hate Tweets and Their Reasons using SVM. Master’s thesis. Uppsala Universitet.Google Scholar
- Neil Thompson. 2016. Anti-discriminatory Practice: Equality, Diversity and Social Justice. Palgrave Macmillan.Google ScholarCross Ref
- Annie Thorburn. 2016. Hate Speech ML. Retrieved from https://github.com/anniethorburn/Hate-Speech-ML.Google Scholar
- Stéphan Tulkens, Lisa Hilte, Elise Lodewyckx, Ben Verhoeven, and Walter Daelemans. 2016. A dictionary-based approach to racism detection in dutch social media. arXiv Preprint arXiv:1608.08738 (2016).Google Scholar
- Twitter. 2017. The Twitter Rules. Retrieved from https://support.twitter.com/articles/.Google Scholar
- UCSM. 2016. IWG hatespeech public. Retrieved from https://github.com/UCSM-DUE/.Google Scholar
- William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In Proceedings of the Second Workshop on Language in Social Media. Association for Computational Linguistics, 19--26. Google ScholarDigital Library
- Zeerak Waseem. 2016. Are you a racist or am I seeing things? Annotator influence on hate speech detection on Twitter. In Proceedings of the 1st Workshop on Natural Language Processing and Computational Social Science. 138--142.Google ScholarCross Ref
- Zeerak Waseem. 2016. Hate speech Twitter annotations. Retrieved from https://github.com/ZeerakW/hatespeech.Google Scholar
- Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 88--93.Google ScholarCross Ref
- Mike Wendling. 2015. 2015: The year that angry won the internet. Retrieved from http://www.bbc.com/news/blogs-trending-35111707.Google Scholar
- Christian Wigand and Melanie Voin. 2017. Speech by Commissioner Jourová—10 years of the EU Fundamental Rights Agency: A call to action in defence of fundamental rights, democracy and the rule of law. Retrieved from http://europa.eu/rapid/press-release_SPEECH-17-403_en.htm.Google Scholar
- Yahoo! 2017. Webscope datasets. Retrieved from https://webscope.sandbox.yahoo.com/.Google Scholar
- David Yarowsky. 1994. Decision lists for lexical ambiguity resolution: Application to accent restoration in Spanish and French. In Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 88--95. Google ScholarDigital Library
- Youtube. 2017. Hate speech. Retrieved from https://support.google.com/youtube/answer/2801939?hl=en.Google Scholar
- Shuhan Yuan, Xintao Wu, and Yang Xiang. 2016. A two phase deep learning model for identifying discrimination from tweets. In Proceedings of the International Conference on Extending Database Technology. 696--697.Google Scholar
- Matthew Zook. 2012. Mapping racist tweets in response to President Obama’s re-election. Retrieved from https://www.theguardian.com/news/datablog/2012/nov/09/mapping-racist-tweets-president-obama-reelection.Google Scholar
Index Terms
- A Survey on Automatic Detection of Hate Speech in Text
Recommendations
Hate begets Hate: A Temporal Study of Hate Speech
CSCWWith the ongoing debate on 'freedom of speech' vs. 'hate speech,' there is an urgent need to carefully understand the consequences of the inevitable culmination of the two, i.e., 'freedom of hate speech' over time. An ideal scenario to understand this ...
Hate speech detection: A solved problem? The challenging case of long tail on Twitter
Special Issue on Semantic Deep LearningIn recent years, the increasing propagation of hate speech on social media and the urgent need for effective counter-measures have drawn significant investment from governments, companies, and researchers. A large number of methods have been developed ...
A Measurement Study of Hate Speech in Social Media
HT '17: Proceedings of the 28th ACM Conference on Hypertext and Social MediaSocial media platforms provide an inexpensive communication medium that allows anyone to quickly reach millions of users. Consequently, in these platforms anyone can publish content and anyone interested in the content can obtain it, representing a ...
Comments