Ying Li
Luis J. Salazar
ABSTRACT
We present the research, and product development and deployment, of Voice Analyzer' by Jobaline Inc. This is a patent pending technology that analyzes voice data and predicts human emotions elicited by the paralinguistic elements of a voice. Human voice characteristics, such as tone, complement the verbal communication. In several contexts of communication, "how" things are said is just as important as "what" is being said. This paper provides an overview of our deployed system, the raw data, the data processing steps, and the prediction algorithms we experimented with. A case study is included where, given a voice clip, our model predicts the degree in which a listener will find the voice "engaging". Our prediction results were verified through independent market research with 75% in agreement on how an average listener would feel. One application of Jobaline Voice Analyzer technology is for assisting companies to hire workers in the service industry where customers' emotional response to workers' voice may affect the service outcome. Jobaline Voice Analyzer is deployed in production as a product offer to our clients to help them identify workers who will better engage with their customers. We will also share some discoveries and lessons learned.
- Cacciatore, S., Luchinat, C., Tenori, L. 2014. Knowledge discovery by accuracy maximization. In Proc. Natl. Acad. Sci., USA, vol. 111 no. 14, 5117--5122.Google Scholar
Cross Ref
- Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., Schröder, M. 2000. FEELTRACE: an instrument for recording perceived emotion in real time. In ISCA workshop on speech and emotion, Northern Ireland, pp 19--24.Google Scholar
- Devillers, L., and Vidrascu, L. 2006. Real-life emotions detection with lexical and paralinguistic cues on human call center dialogs. INTERSPEECH.Google Scholar
- Fernandez, R. 2004. A Computational Model for the Automatic Recognition of Affect in Speech. PhD Thesis, Massachusetts Institute of Technology, Cambridge, MA. Google ScholarDigital Library
- Forsell, M. 2007. Acoustic Correlates of Perceived Emotions in Speech. Master Thesis in Speech Communication, Royal Institute of Technology. KTH.Google Scholar
- Kreiman, J., Van Lancker-Sidtis, D., and Gerratt, B.R., 2005. Perception of Voice Quality, in The Handbook of Speech Perception, Ed. Pisoni, D.B. and Remez, R.E., Blackwell Publishing, 338--362.Google Scholar
- Kreiman, J., Sidtis, D., 2011. Foundations of Voice Studies. Wiley-Blackwell.Google Scholar
- Lopatovska, I. and Arapakis, I. 2011. Theories, methods and current research on emotions in library and information science, information retrieval and human-computer interactions. Information Processing & Management. 47(4), 575--592. Google ScholarDigital Library
- Mullor, M., Salazar, L., Li, Y., and Contreras, J. (Jobaline, Inc., USA) 2015. Matching and Lead Prequalification Based on Voice Analysis. US Patent Application #14532600.Google Scholar
- Picard, R.W. 2010. Emotion research by the people, for the people. Emotion Review, Volume 2, Issue 3 (July 2010)Google ScholarCross Ref
- Polzehl, T., Moller, S., and Metze, F. 2010. Automatically assessing personality from speech. 2010 IEEE Fourth International Conference on Semantic Computing (ICSC). Google ScholarDigital Library
- Polzin, T. S., and Waibel, A. 1998. Detecting emotions in speech. Proceedings of the CMC.Google Scholar
- Quatier, T. F. 2002. Discrete-Time Speech Signal Processing: Principles and Practice. Google ScholarDigital Library
- Schuller, B., Steidl, S., Batliner, A., Burkhardt, F., Devillers, L., Müller, C. A., et al. 2010. The INTERSPEECH 2010 paralinguistic challenge. INTERSPEECH.Google Scholar
- Schuller, B. 2011. Voice and speech analysis in search of states and traits. Computer Analysis of Human Behavior, 227--253.Google Scholar
- Schuller, B., Steidl, S., Batliner, A., Nöth, E., Vinciarelli, A., Burkhardt, F., et. al. 2012. The INTERSPEECH 2012 Speaker Trait Challenge. INTERSPEECH.Google Scholar
- Weiss, B. and Burkhardt, F. 2012. Is 'not bad' good enough? Aspects of unknown voices' likability. INTERSPEECH.Google Scholar
- Zhao, S., Rudzicz, F., Carvalho, L. G., Márquez-Chin, C., and Livingstone, S. 2014. Automatic detection of expressed emotion in Parkinson's disease. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).Google Scholar
Index Terms
- Predicting Voice Elicited Emotions
Recommendations
Acoustic Measures for Real-Time Voice Coaching
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningOur voices can convey many different types of thoughts and intent; how our voices carry them is often not consciously controlled and as a consequence, unintended effects may arise that negatively impact our relationships. How we say things is as ...
Improving automotive safety by pairing driver emotion and car voice emotion
CHI EA '05: CHI '05 Extended Abstracts on Human Factors in Computing SystemsThis study examines whether characteristics of a car voice can affect driver performance and affect. In a 2 (driver emotion: happy or upset) x 2 (car voice emotion: energetic vs. subdued) experimental study, participants (N=40) had emotion induced ...
Regulating Feelings During Interpersonal Conflicts by Changing Voice Self-perception
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsEmotions play a major role in how interpersonal conflicts unfold. Although several strategies and technological approaches have been proposed for emotion regulation, they often require conscious attention and effort. This often limits their efficacy in ...
Comments