ABSTRACT
Hearings of witnesses and defendants play a crucial role when reaching court trial decisions. Given the high-stake nature of trial outcomes, implementing accurate and effective computational methods to evaluate the honesty of court testimonies can offer valuable support during the decision making process. In this paper, we address the identification of deception in real-life trial data. We introduce a novel dataset consisting of videos collected from public court trials. We explore the use of verbal and non-verbal modalities to build a multimodal deception detection system that aims to discriminate between truthful and deceptive statements provided by defendants and witnesses. We achieve classification accuracies in the range of 60-75% when using a model that extracts and fuses features from the linguistic and gesture modalities. In addition, we present a human deception detection study where we evaluate the human capability of detecting deception in trial hearings. The results show that our system outperforms the human capability of identifying deceit.
- M. Aamodt and H. Custer. Who can best catch a liar? a meta-analysis of individual differences in detecting deception. Forensic Examiner, 15(1):6--11, 2006.Google Scholar
- M. Abouelenien, V. Pérez-Rosas, R. Mihalcea, and M. Burzo. Deception detection using a multimodal approach. In Proceedings of the 16th International Conference on Multimodal Interaction, ICMI '14, pages 58--65, Istanbul, Turkey, 2014. ACM. Google ScholarDigital Library
- J. Allwood, L. Cerrato, K. Jokinen, C. Navarretta, and P. Paggio. The mumin coding scheme for the annotation of feedback, turn management and sequencing phenomena. Language Resources and Evaluation, 41(3--4):273--287, 2007.Google Scholar
- A. Almela, R. Valencia-García, and P. Cantos. Seeing through deception: A computational approach to deceit detection in written communication. In Proceedings of the Workshop on Computational Approaches to Deception Detection, pages 15--22, Avignon, France, April 2012. Association for Computational Linguistics. Google ScholarDigital Library
- J. Burgoon, D. Twitchell, M. Jensen, T. Meservy, M. Adkins, J. Kruse, A. Deokar, G. Tsechpenakis, S. Lu, D. Metaxas, J. Nunamaker, and R. Younger. Detecting concealment of intent in transportation screening: A proof of concept. IEEE Transactions on Intelligent Transportation Systems, 10(1):103--112, March 2009. Google ScholarDigital Library
- L. Caso, F. Maricchiolo, M. Bonaiuto, A. Vrij, and S. Mann. The impact of deception and suspicion on different hand movements. Journal of Nonverbal Behavior, 30(1):1--19, 2006.Google ScholarCross Ref
- G. Chittaranjan and H. Hung. Are you awerewolf? detecting deceptive roles and outcomes in a conversational role-playing game. In 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pages 5334--5337, March 2010.Google ScholarCross Ref
- D. Cohen, G. Beattie, and H. Shovelton. Nonverbal indicators of deception: How iconic gestures reveal thoughts that cannot be suppressed. Semiotica, 2010(182):133--174, 2010.Google ScholarCross Ref
- B. Depaulo, B. Malone, J. Lindsay, L. Muhlenbruck, K. Charlton, and H. Cooper. Cues to deception. Psychological Bulletin, pages 74--118, 2003.Google ScholarCross Ref
- M. Derksen. Control and resistance in the psychology of lying. Theory and Psychology, 22(2):196--212, 2012.Google ScholarCross Ref
- P. Ekman. Telling Lies: Clues to Deceit in the Marketplace, Politics and Marriage. Norton, W.W. and Company, 2001.Google Scholar
- P. Ekman. Darwin, deception, and facial expression. Annals of the New York Academy of Sciences, 1000(EMOTIONS INSIDE OUT: 130 Years after Darwin's The Expression of the Emotions in Man and Animals):205--221, 2003.Google Scholar
- S. Feng, R. Banerjee, and Y. Choi. Syntactic stylometry for deception detection. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2, ACL '12, pages 171--175, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics. Google ScholarDigital Library
- T. Fornaciari and M. Poesio. Automatic deception detection in Italian court cases. Artificial Intelligence and Law, 21(3):303--340, 2013. Google ScholarDigital Library
- T. Gannon, A. Beech, and T. Ward. Risk Assessment and the Polygraph, pages 129--154. John Wiley and Sons Ltd, 2009.Google Scholar
- P. A. Granhag and M. Hartwig. A new theoretical perspective on deception detection: On the psychology of instrumental mind-reading. Psychology, Crime & Law, 14(3):189--200, 2008.Google ScholarCross Ref
- S. Gross and R. Warden. Exonerations in the USA, 1989 - 2012. Technical report, National Registry of Exonerations, 2012.Google Scholar
- R. Guadagno, B. Okdie, and S. Kruse. Dating deception: Gender, online dating, and exaggerated self-presentation. Comput. Hum. Behav., 28(2):642--647, Mar. 2012. Google ScholarDigital Library
- J. Hillman, A. Vrij, and S. Mann. Um ... they were wearing ...: The effect of deception on specific hand gestures. Legal and Criminological Psychology, 17(2):336--345, 2012.Google ScholarCross Ref
- J. Hirschberg, S. Benus, J. Brenier, F. Enos, S. Friedman, S. Gilman, C. Gir, G. Graciarena, A. Kathol, and L. Michaelis. Distinguishing deceptive from non-deceptive speech. In In Proceedings of Interspeech 2005 - Eurospeech, pages 1833--1836, 2005.Google Scholar
- S. Ho and J. M. Hollister. Guess who? an empirical study of gender deception and detection in computer-mediated communication. Proceedings of the American Society for Information Science and Technology, 50(1):1--4, 2013. Google ScholarDigital Library
- M. Jensen, T. Meservy, J. Burgoon, and J. Nunamaker. Automatic, multimodal evaluation of human interaction. Group Decision and Negotiation, 19(4):367--389, 2010.Google ScholarCross Ref
- A. N. Joinson and B. Dietz-Uhler. Explanations for the perpetration of and reactions to deception in a virtual community. Social Science Computer Review, 20(3):275--289, 2002. Google ScholarDigital Library
- J. Li, M. Ott, C. Cardie, and E. Hovy. Towards a general rule for identifying deceptive opinion spam. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, June 2014.Google ScholarCross Ref
- S. Lu, G. Tsechpenakis, D. Metaxas, M. Jensen, and J. Kruse. Blob analysis of the head and hands: A method for deception detection. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05), HICSS '05, pages 20--29, Washington, DC, USA, 2005. IEEE Computer Society. Google ScholarDigital Library
- F. Maricchiolo, A. Gnisci, and M. Bonaiuto. Coding hand gestures: A reliable taxonomy and a multi-media support. In A. Esposito, A. Esposito, A. Vinciarelli, R. Hoffmann, and V. Muller, editors, Cognitive Behavioural Systems, volume 7403 of Lecture Notes in Computer Science, pages 405--416. Springer Berlin Heidelberg, 2012. Google ScholarDigital Library
- T. Meservy, M. Jensen, J. Kruse, D. Twitchell, G. Tsechpenakis, J. Burgoon, D. Metaxas, and J. Nunamaker. Deception detection through automatic, unobtrusive analysis of nonverbal behavior. IEEE Intelligent Systems, 20(5):36--43, September 2005. Google ScholarDigital Library
- R. Mihalcea and C. Strapparava. The lie detector: Explorations in the automatic recognition of deceptive language. In Proceedings of the Association for Computational Linguistics (ACL 2009), Singapore, 2009. Google ScholarDigital Library
- M. Newman, J. Pennebaker, D. Berry, and J. Richards. Lying words: Predicting deception from linguistic styles. Personality and Social Psychology Bulletin, 29, 2003.Google Scholar
- J. Nunamaker, J. Burgoon, N. Twyman, J. Proudfoot, R. Schuetzler, and J. Giboney. Establishing a foundation for automated human credibility screening. In 2012 IEEE International Conference on Intelligence and Security Informatics (ISI), pages 202--211, June 2012.Google ScholarCross Ref
- M. Ott, Y. Choi, C. Cardie, and J. Hancock. Finding deceptive opinion spam by any stretch of the imagination. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT '11, pages 309--319, Stroudsburg, PA, USA, 2011. Association for Computational Linguistics. Google ScholarDigital Library
- M. Owayjan, A. Kashour, N. AlHaddad, M. Fadel, and G. AlSouki. The design and development of a lie detection system using facial micro-expressions. In 2012 2nd International Conference on Advances in Computational Tools for Engineering Applications (ACTEA), pages 33--38, Dec 2012.Google ScholarCross Ref
- I. Pavlidis, N. Eberhardt, and J. Levine. Human behaviour: Seeing through the face of deception. Nature, 415(6867), 2002.Google Scholar
- J. Pennebaker and M. Francis. Linguistic inquiry and word count: LIWC, 1999. Erlbaum Publishers.Google Scholar
- V. Perez-Rosas, R. Mihalcea, A. Narvaez, and M. Burzo. A multimodal dataset for deception detection. In Proceedings of the Conference on Language Resources and Evaluations (LREC 2014), Reykjavik, Iceland, May 2014.Google Scholar
- T. Pfister and M. Pietikäinen. Electronic imaging & signal processing automatic identification of facial clues to lies. SPIE Newsroom, January 2012.Google ScholarCross Ref
- S. Sumriddetchkajorn and A. Somboonkaew. Thermal analyzer enables improved lie detection in criminal-suspect interrogations. In SPIE Newsroom: Defense & Security, 2011.Google ScholarCross Ref
- Y. Tian, T. Kanade, and J. Cohn. Facial expression analysis. In Handbook of Face Recognition, pages 247--275. Springer New York, 2005.Google ScholarCross Ref
- C. Toma and J. Hancock. Reading between the lines: linguistic cues to deception in online dating profiles. In Proceedings of the 2010 ACM conference on Computer supported cooperative work, CSCW '10, pages 5--8, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- G. Tsechpenakis, D. Metaxas, M. Adkins, J. Kruse, J. Burgoon, M. Jensen, T. Meservy, D. Twitchell, A. Deokar, and J. Nunamaker. Hmm-based deception recognition from visual cues. In IEEE International Conference on Multimedia and Expo, 2005. ICME 2005, pages 824--827, July 2005.Google ScholarCross Ref
- A. Vrij. Detecting Lies and Deceit: The Psychology of Lying and the Implications for Professional Practice. Wiley series in the psychology of crime, policing and law. Wiley, 2001.Google Scholar
- D. Warkentin, M. Woodworth, J. Hancock, and N. Cormier. Warrants and deception in computer mediated communication. In Proceedings of the 2010 ACM conference on Computer supported cooperative work, pages 9--12. ACM, 2010. Google ScholarDigital Library
- Q. Xu and H. Zhao. Using deep linguistic features for finding deceptive opinion spam. In Proceedings of COLING 2012: Posters, pages 1341--1350, Mumbai, India, December 2012. The COLING 2012 Organizing Committee.Google Scholar
- M. Yancheva and F. Rudzicz. Automatic detection of deception in child-produced speech using syntactic complexity features. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 944--953, Sofia, Bulgaria, August 2013. Association for Computational Linguistics.Google Scholar
Index Terms
- Deception Detection using Real-life Trial Data
Recommendations
Deception detection using a multimodal approach
ICMI '14: Proceedings of the 16th International Conference on Multimodal InteractionIn this paper we address the automatic identification of deceit by using a multimodal approach. We collect deceptive and truthful responses using a multimodal setting where we acquire data using a microphone, a thermal camera, as well as physiological ...
Deception detection methods incorporating discourse network metrics in synchronous computer-mediated communication
The prevalence of deception in computer-mediated communication and the risk of misjudgement based on deceptive information call for effective detection methods of deception. Extant models for online deception detection rely mainly on verbal behaviours of ...
"Was It You Who Stole 500 Rubles?" - The Multimodal Deception Detection
ICMI '20 Companion: Companion Publication of the 2020 International Conference on Multimodal InteractionAutomatic deception detection is a challenging issue since human behaviors are too complex to establish any standard behavioral signs that would explicitly indicate that a person is lying. Furthermore, it is difficult to collect naturalistic datasets ...
Comments