DANIEL G. LEE
Abstract
Television and film have become important equalization mechanisms for the dissemination and distribution of cultural materials. Closed captioning has allowed people who are deaf and hard of hearing to be included as audience members. However, some of the audio information such as music, sound effects, and speech prosody are not generally provided for in captioning. To include some of this information in closed captions, we generated graphical representations of the emotive information that is normally represented with nondialog sound. Eleven deaf and hard of hearing viewers watched two different video clips containing static and dynamic enhanced captions and compared them with conventional closed captions of the same clips. These viewers then provided verbal and written feedback regarding positive and negative aspects of the various captions. We found that hard of hearing viewers were significantly more positive about this style of captioning than deaf viewers and that some viewers believed that these augmentations were useful and enhanced their viewing experience.
- ABRAHAMIAN, S. 2003. EIA-608 and EIA-708 closed captioning. http://www.evertz.com/resources/eia_608_708_cc.pdf. Retrieved May 18, 2006.Google Scholar
- BERGLUND, A. AND JOHANSSON, P. 2004. Using speech and dialogue for interactive TV navigation. Universal Access in Information Society 3, 224--238. Google ScholarDigital Library
- CHORIANOPOULOS, K. AND SPINELLIS, D. 2006. User interface evaluation of interactive TV: A media studies perspective. Universal Access in Information Society 5, 3, 209-218. Google ScholarDigital Library
- COWIE, R AND CORNELIUS, R. R. 2003. Describing the emotional states that are expressed in speech. Speech Communication 40, 5-32. Google ScholarDigital Library
- EKMAN, P. (ED.) 1999. Basic Emotions. Wiley, New York.Google Scholar
- ERONEN, L. 2006. Five qualitative research methods to make iTV applications universally accessible. Universal Access in Information Society 5, 3, 219-238. Google ScholarDigital Library
- FELS, D.I., POLANO, L., HARVEY, T., DEGAN, S.S., AND SILVERMAN, C.H. 2001. Towards emotive captioning for interactive television, In Universal Access and Human-Ccomputer Interaction, 303-307.Google Scholar
- FIELD, G. 2000. Implementing DTV closed captions and video description. http://ncam.wgbh.org/dtv/overview/nab2000paper.html. Retrieved May 18, 2006.Google Scholar
- GOLBERG, M. 2000. How to read a film. http://ncam.wgbh.org/dtv/overview/nab2000paper.html. Retrieved May 18, 2006.Google Scholar
- JENSEMA, C. J. 1998. Viewer reaction to different television captioning speeds. American Annals of the Deaf 143, 4, 318-324.Google ScholarCross Ref
- JENSEMA, C., MCCANN, R., AND RAMSEY, S. 1996. Closed-captioned television presentation speed and vocabulary. American Annals of the Deaf 141, 4, 284-292.Google ScholarCross Ref
- JORDAN, A. B., ALBRIGHT, A., BRANNER, A., AND SULLIVAN, J. 2003. The state of closed captioning services in the United States. National Captioning Institute Foundation.Google Scholar
- JUSLIN, P. N. AND LAUKKA, P. 2003. Communication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin 129, 5, 770-814.Google ScholarCross Ref
- KINDRED, J. AND SHANNON, L. R. 2004. Making connections via instant messenger (IM): Students use of IM to maintain personal relationships. Qualitative Research Reports in Communication 5, 28-54.Google Scholar
- KODA, T. 2004. Interpretation of emotionally expressive characters in an intercultural communication. In Proceedings of the 8th International Conference on Knowledge-Based Intelligent Information & Engineering Systems (KES2004). P. van den Besselaar and S. Koizumi (eds.), 862-868.Google ScholarCross Ref
- KROHN, F. 2004. A generation approach to using emoticons as nonverbal communication. Journal of Technical Writing and Communication 34, 4, 321-328.Google ScholarCross Ref
- LEWIS, M.S.J. 2000. Television captioning: A vehicle for accessibility and literacy. In On-line Proceedings of CSUN. (Los Angeles, CA). www.dinf/org/csun_99/session0057.html. Retrieved Sept. 19, 2006.Google Scholar
- MEHRABIAN, A. 1968. Communication without words, Psychology Today 2, 4, 53-56.Google Scholar
- NIJHOLT, A. 2004. Where computers disappear, virtual humans appear. Computers & Graphics 28, 4, 467-476.Google Scholar
- ORTONY, A. AND TURNER, T. J. 1990. What's basic about basic emotions? Psychological Review 97, 3, 315-331.Google ScholarCross Ref
- OVIATT, S. 1999. Ten myths of multimodal interaction, Communications of the ACM 42, 11 (Nov), 74-81. Google ScholarDigital Library
- PLUTCHIK, R. 1980. Emotion, A Psychoevolutionary Synthesis. Harper & Row, New York.Google Scholar
- PLUTCHIK, R. 2001. The nature of emotions. American Scientist 48, 4, 344.Google Scholar
- PREECE, J. 1993. A Guide to Usability: Human Factors in Computing. Addison-Wesley, Reading, MA Google ScholarDigital Library
- PREECE, J., ROGERS, Y., AND SHARP, H. 2002. Interaction Design. Wiley, New York. Google ScholarDigital Library
- RIBRANT, G. 1999. Style parameters in film sound. http://filmsound.studienet.org/bibliography/stylepara.pdf. Retrieved March 8, 2006.Google Scholar
- ROBERTS, V. L. AND FELS, D. I. 2006. Methods for inclusion: Employing think aloud protocols in software usability studies with individuals who are deaf. International Journal of Human-Computer Interaction 64, 6, 489-501. Google ScholarDigital Library
- ROBSON, G.D. 2000. FCC adopts caption standards for digital television. www.robson.org/gary/captioning/dtvstandards-july2000.html. Retrieved April 15, 2006.Google Scholar
- ROVERS, L. AND VAN ESSEN, H. 2004. Design and evaluation of hapticons for instant messaging. In Proceedings of Eurohaptics 2004 (Munich).Google Scholar
- RUSSELL, J.A. AND MEHRABIAN, A. 1977. Evidence for a three-factor theory of emotions. Journal of Research in Personality, 11, 3, 273-294.Google ScholarCross Ref
- SCHERER, K. R. 2003. Vocal communication of emotion: A review of research paradigms. Speech Communication 40, 227-256. Google ScholarDigital Library
- SHILLING, R., ZYDA, M., AND WARDYNSKI, E. C. 2002. Introducing emotion into military simulation and videogame design: America's army: operations and VIRTE. In Proceedings of the GameOn Conference (London), 151-154.Google Scholar
- TEN BOSCH, L. 2003. Emotions, speech and the ASR framework. Speech Communication 40, 213-225. Google ScholarDigital Library
- THOMPSON, J. C. AND LLOYD, B. 2002. E-mail etiquette (netiquette). In Proceedings of the Annual Pulp and Paper Industry Technical Conference (Toronto), 111-114.Google ScholarCross Ref
- TOIVANEN, J., VAYRYNEN, E., AND SEPPANEN, T. 2004. Automatic discrimination of emotion from spoken Finnish. Language and Speech 47, 4, 383-412.Google ScholarCross Ref
- VALDEZ, P. AND MEHRABIAN, A. 1994. Effects of color on emotions. Journal of Experimental Psychology: General 123, 4, 394-409.Google ScholarCross Ref
- VITAC 2003. History of DTV captioning. http://www.vitac.com/technical/dtvhistory.htm. Retrieved May 18, 2005.Google Scholar
- WANG, H. L. AND CHEONG, L.-F. 2006. Affective understanding in film. IEEE Transactions on Circuits and Systems for Video Technology 16, 6, 689-704. Google ScholarDigital Library
- WARR, D. J. 2005. "It was fun, but we don't usually talk about these things": Analyzing sociable interaction in focus groups. Qualitative Inquiry 11, 2, 200-225.Google ScholarCross Ref
Index Terms
- Emotive captioning
Recommendations
Football for all: the quality of the live closed captioning in the Super Bowl LII
AbstractIn the U.S., new programming broadcast on television needs to be accessible to the hearing-impaired, and closed captioning is the tool most frequently used to provide such access. From all the audiovisual content available to viewers, live ...
Closed captioning quality in the information society: the case of the American newscasts reshown online
AbstractMany Americans access information through the evening newscasts shown on mainstream television. Nevertheless, the technological developments that our society has experienced in the last decade are changing the traditional information-access ...
Interactive TV narratives: Opportunities, progress, and challenges
This article is motivated by the question whether television should do more than simply offer interactive services alongside (and separately from) traditional linear programs, in the context of its dominance being seriously challenged and threatened by ...
Comments