Spotting prejudice with nonverbal behaviours

ABSTRACT

Despite prejudice cannot be directly observed, nonverbal behaviours provide profound hints on people inclinations. In this paper, we use recent sensing technologies and machine learning techniques to automatically infer the results of psychological questionnaires frequently used to assess implicit prejudice. In particular, we recorded 32 students discussing with both white and black collaborators. Then, we identified a set of features allowing automatic extraction and measured their degree of correlation with psychological scores. Results confirmed that automated analysis of nonverbal behaviour is actually possible thus paving the way for innovative clinical tools and eventually more secure societies.

References

1. Audacity Team. 1999-2016. Audacity. (1999-2016). http://www.audacityteam.org/
2. Tobias Baur, Ionut Damian, Florian Lingenfelser, Johannes Wagner, and Elisabeth André. 2013. Human Behavior Understanding: 4th International Workshop, HBU 2013, Barcelona, Spain, October 22, 2013. Proceedings. Springer International Publishing, Cham, Chapter NovA: Automated Analysis of Nonverbal Signals in Social Interactions, 160--171. DOI: http://dx.doi.org/10.1007/978-3-319-02714-2_14
3. Stefan Benus, Frank Enos, Julia Bell Hirschberg, and Elizabeth Shriberg. 2006. Pauses in deceptive speech. Proc. ISCA 3rd International Conference on Speech Prosody.
4. A. Bharathi, A. Singh, C. Tucker, and H. Nembhard. 2016. Knowledge Discovery of Game Design Features By Mining User-Generated Feedback. Computers in Human Behavior 60 (2016), 361--371.
5. Nicola Bicocchi, Damiano Fontana, and Franco Zambonelli. 2014. A self-aware, reconfigurable architecture for context awareness. In IEEE Symposium on Computers and Communications, ISCC 2014, Funchal, Madeira, Portugal, June 23-26, 2014. 1--7.
6. Nicola Bicocchi, Matteo Lasagni, and Franco Zambonelli. 2012. Bridging vision and commonsense for multimodal situation recognition in pervasive systems. In 2012 IEEE International Conference on Pervasive Computing and Communications, Lugano, Switzerland, March 19-23, 2012. 48--56.
7. Keith Brawner and Benjamin Goldberg. 2012. Real-time monitoring of ecg and gsr signals during computer-based training. In Intelligent Tutoring Systems. Springer, 72--77.
8. Nathan Burba, Mark Bolas, David M Krum, Evan Suma, and others. 2012. Unobtrusive measurement of subtle nonverbal behaviors with the Microsoft Kinect. In Virtual Reality Short Papers and Posters (VRW), 2012 IEEE. IEEE, 1--4.
9. A. Burns, B.R. Greene, M.J. McGrath, T.J. O'Shea, B. Kuris, S.M. Ayer, F. Stroiescu, and V. Cionca. 2010. SHIMMER x2122; x2013; A Wireless Sensor Platform for Noninvasive Biomedical Research. Sensors Journal, IEEE 10, 9 (Sept 2010), 1527--1534. DOI:http://dx.doi.org/10.1109/JSEN.2010.2045498
10. John F Dovidio, Kerry Kawakami, and Samuel L Gaertner. 2002. Implicit and explicit prejudice and interracial interaction. Journal of personality and social psychology 82, 1 (2002), 62.
11. Denise Frauendorfer, Marianne Schmid Mast, Laurent Nguyen, and Daniel Gatica-Perez. 2014. Nonverbal Social Sensing in Action: Unobtrusive Recording and Extracting of Nonverbal Behavior in Social Interactions Illustrated with a Research Example. Journal of Nonverbal Behavior 38, 2 (2014), 231--245. DOI:http://dx.doi.org/10.1007/s10919-014-0173-5
12. Jonathan Gratch, Ron Artstein, Gale M Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, and others. 2014. The Distress Analysis Interview Corpus of human and computer interviews.. In LREC. 3123--3128.
13. A. G. Greenwald, D. E. McGhee, and J. L. K. Schwartz. 1998. Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology 74 (1998), 1464--1480.
14. A. G. Greenwald, B. A. Nosek, and M. R. Banaji. 2003. Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology 85 (2003), 197--206.
15. S. Handri, K. Yajima, S. Nomura, N. Ogawa, Y. Kurosawa, and Y. Fukumura. 2010. Evaluation of Student's Physiological Response Towards E-Learning Courses Material by Using GSR Sensor. In Computer and Information Science (ICIS), 2010 IEEE/ACIS 9th International Conference on. 805--810. DOI:http://dx.doi.org/10.1109/ICIS.2010.92
16. Béatrice S Hasler, Oren Salomon, Peleg Tuchman, Ady Nae O'Malley, and Doron A Friedman. 2011. Real-time Translation of Nonverbal Communication in Cross-Cultural Online Encounters. In Proc. of CMVC.
17. M.R. Hebl and Dovidio J.F. 2005. Promoting the social in the examination of social stigmas. Journal of personality and social psychology 9, 9 (2005), 156--182.
18. Jin Joo Lee, Brad Knox, and Cynthia Breazeal. 2013. Modeling the dynamics of nonverbal behavior on interpersonal trust for human-robot interactions. (2013).
19. D. Matsumoto and H. C. Hwang. 2016. The cultural bases of nonverbal communication. 77--101.
20. Dimitris Metaxas and Shaoting Zhang. 2013. A review of motion analysis methods for human nonverbal communication computing. Image and Vision Computing 31, 6 (2013), 421--433.
21. Joann M. Montepare. 2014. Nonverbal Behavior in the Digital Age: Meanings, Models, and Methods. Journal of Nonverbal Behavior 38, 3 (2014), 279--281. DOI: http://dx.doi.org/10.1007/s10919-014-0187-z
22. M. L. Patterson. 1982. A sequential functional model of nonverbal exchange. Psychological Review 89 (1982), 231--249.
23. Suzanne M Retzinger. 1995. Identifying shame and anger in discourse. The American Behavioral Scientist 38, 8 (1995), 1104.
24. G. Stratou, S. Scherer, J. Gratch, and L.-P. Morency. 2013. Automatic Nonverbal Behavior Indicators of Depression and PTSD: Exploring Gender Differences. In Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on. 147--152. DOI:http://dx.doi.org/10.1109/ACII.2013.31
25. Grace A Sullivan and Wind Goodfriend. 2013. The Effects of Controlling Nonverbal Intimacy. Journal Contents 18, 2 (2013), 32--41.
26. Andrea Stevenson Won, Jeremy N Bailenson, and Joris H Janssen. 2014a. Automatic detection of nonverbal behavior predicts learning in dyadic interactions. Affective Computing, IEEE Transactions on 5, 2 (2014), 112--125.
27. Andrea Stevenson Won, Jeremy N Bailenson, Suzanne C Stathatos, and Wenqing Dai. 2014b. Automatically detected nonverbal behavior predicts creativity in collaborating dyads. Journal of Nonverbal Behavior 38, 3 (2014), 389--408.
28. Yang Rui Zhang, Guang Huo, Jian Feng Wu, Jun Bo Yang, and Chen Pang. 2015. An Interactive Oral Training Platform Based on Kinect for EFL Learning. In Applied Mechanics and Materials, Vol. 704. Trans Tech Publ, 419--423.
29. Zhengyou Zhang. 2012. Microsoft kinect sensor and its effect. MultiMedia, IEEE 19, 2 (2012), 4--10.

Index Terms

• Computing methodologies

  • Machine learning

• Human-centered computing

  • Collaborative and social computing

    • Empirical studies in collaborative and social computing

  • Ubiquitous and mobile computing

    • Ubiquitous and mobile computing theory, concepts and paradigms

      • Ambient intelligence