Sutjipto Arifin
ABSTRACT
Extracting video structures is important for video indexing and navigation in large digital video archives. It is usually achieved by video segmentation algorithms. Little research efforts has been invested on segmentation solutions that utilize the video's emotional content. These solutions not only have the potential of providing better performances than existing segmentation methods, but are also able to provide a more natural video segmentation with which viewers can associate with. The development of an affect-based segmentation solution faces many challenges, such as the dynamic and time evolving nature of a video's emotional content. This paper introduces a novel computation method for affect-based video segmentation. It is designed based on the Pleasure-Arousal-Dominance (P-A-D) emotion model[18], which in principle can represent a large number of emotions. This method consists of a P-A-D estimation stage and a segmentation stage. A P-A-D estimator based on the Dynamic Bayesian Networks (DBNs) is proposed for the first stage. A clustering-based algorithm that utilizes the video's P-A-D information is proposed for the second stage. Experimental results demonstrate the feasibility of the method.
- M. Bradley. Emotional memory: a dimensional analysis. Hillsdale, NJ: Lawrence Erlbaum, 1994.Google Scholar
- Celoxica. RC200/203 Manual, 2005.Google Scholar
- B. Y. Chua and G. Lu. Improved perceptual tempo detection of music. In Intl. Conf. of Multimedia Model., pages 316--321, Jan. 2005. Google ScholarDigital Library
- A. del Bimbo. Visual Information Retrieval. New York: Morgan Kaufmann, 1999. Google ScholarDigital Library
- P. Ekman. Facial expression and emotion. American Psycho., 48(4):384--392, Apr. 1993.Google ScholarCross Ref
- J. J. Gross and R. W. Levenson. Emotion elicitation using films. Cog. and Emot., 9(1):87--108, Jan. 1995.Google ScholarCross Ref
- A. Hanjalic and R. Lagendijk. Automated high level segmentation for advanced video retrieval systems. IEEE Trans. Circuits Syst. Video Technol., 9(4):580--588, June 1999. Google ScholarDigital Library
- A. Hanjalic and L. Q. Xu. Affective content representation and modeling. IEEE Trans. Multimedia, 7(1):143--154, Feb. 2005. Google ScholarDigital Library
- L. Itti, C. Koch, and E. Niebur. A model for saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell., 20(11):1254--1259, Nov. 1998. Google ScholarDigital Library
- A. Jaimes, T. Nagamine, J. Liu, K. Omura, and N. Sebe. Affective meeting video analysis. In IEEE Intl. Conf. on Multimedia and Expo, pages 1412--1415, July 2005.Google Scholar
- H. B. Kang. Analysis of scene context related with emotional events. In ACM Intl. Conf. on Multimedia. Google ScholarDigital Library
- H. B. Kang. Affective content detection using hidden markov models. In ACM Intl. Conf. on Multimedia, pages 259--262, Nov. 2003. Google ScholarDigital Library
- J. Kender and B. L. Yeo. Video scene segmentation via continuous video coherence. In IEEE Conferenceon Comput. Vis. and Pattern Recog. Google ScholarDigital Library
- P. Lang. Perspectives on Anger and Emotion, pages 109--134. Lawrence Erlbaum Associates, 1993.Google Scholar
- J. Laroche. Estimating tempo, swing and beat locations in audio recordings. In IEEE App. of Signal Process. to Audio and Acoust., pages 135--138, Oct.2001.Google Scholar
- M. Lew. Principles of Visual Information Retrieval. Springer-Verlag, Berlin, Germany, 2001. Google ScholarDigital Library
- R. Lienhart, S. Pfeiffer, and W. Effelsberg. Scene determination based on video and audio features. In Intl. Conf. of Multimedia Syst. Google ScholarDigital Library
- A. Mehrabian. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psycho., 14(4):261--292, Dec. 1996.Google ScholarCross Ref
- S. Moncrieff, C. Dorai, and S. Venkatesh. Affect computing in film through sound energy dynamics. In ACM Intl. Conf. on Multimedia, volume 9. Google ScholarDigital Library
- K. Murphy. Dynamic bayesian networks, Nov. 2002.Google Scholar
- C. E. Osgood, G. J. Suci, and P. H. Tannenbaum. The Measurement of Meaning. University of Illinois Press, 1967.Google Scholar
- M. Pantic and L. J. M. Rothkrantz. Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE, 91(9):1370--1390, Sept. 2003.Google ScholarCross Ref
- R. Picard. Affective Computing. The MIT Press, Cambridge, MA, 1997. Google ScholarDigital Library
- T. Pohle. Extraction of audio descriptors and their evaluation in music clasification tasks. Diploma thesis, University of Kaiserslautern, Kaiserslautern, Germany, Jan. 2005.Google Scholar
- Y. Rui, T. S. Huang, and S. Mehrotra. Constructing table-of-content for videos. Multimedia Systems, 7(5). Google ScholarDigital Library
- P. Shaver, J. Schwartz, D. Kirson, and G. O'Connor. Emotions in Social Psycology: Key Readings in Social Psycology, pages 26--56. Psychology Press, 2001.Google Scholar
- P. Valdez and A. Mehrabian. Effects of color on emotions. J. of Exp. Psycho., 124(4):394--409, Dec.1994.Google ScholarCross Ref
- J. Vendrig and M. Worring. Systematic evaluation of logical story unit segmentation. IEEE Trans. Multimedia, 4(4):492--499, Dec. 2002. Google ScholarDigital Library
- M. Xu, L. T. Chia, and J. Jin. Affective content analysis in comedy and horror videos by audio emotional event detection. In IEEE Intl. Conf. on Multimedia and Expo, July 2005.Google Scholar
Index Terms
- A computation method for video segmentation utilizing the pleasure-arousal-dominance emotional information
Recommendations
Affective Level Video Segmentation by Utilizing the Pleasure-Arousal-Dominance Information
In this paper, we offer an entirely new view to the problem of high level video parsing. We developed a novel computation method for affective level video segmentation. Its function was to extract emotional segments from videos. Its design was based on ...
A Novel Probabilistic Approach to Modeling the Pleasure-Arousal-Dominance Content of the Video based on "Working Memory"
ICSC '07: Proceedings of the International Conference on Semantic ComputingThe rapid growth of digital video information nowadays is making video content classification and indexing tools a necessity. Little research efforts have been invested on content classification solutions based on the emotional content of the video, ...
Comments