American Sign Language natural language generation and machine translation

Abstract

Although deaf students in the U.S. and Canada are taught written English, their inability to hear spoken English results in most deaf U.S. high school graduates (18 year olds) reading at a fourth-grade (10 year old) level (Holt, 1991). Unfortunately, many deaf accessibility aids, like television closed captioning or teletype telephones, assume the user has strong English literacy skills. Many deaf people with English reading difficulty are fluent in American Sign Language (ASL); so, an English-to-ASL automated machine translation (MT) system could make information and services accessible when English text captioning is too complex or an interpreter is unavailable.

References

1. Bindiganavale, R., Schuler, W., Allbeck, J., Badler, N., Joshi, A., and Palmer, M. 2000. Dynamically Altering Agent Behaviors Using Natural Language Instructions. 4th International Conference on Autonomous Agents.
2. Holt, J. 1991. Demographic, Stanford Achievement Test - 8th Edition for Deaf and Hard of Hearing Students: Reading Comprehension Subgroup Results.
3. Huenerfauth, M. 2003. Survey and Critique of ASL Natural Language Generation and Machine Translation Systems. Technical Report MS-CIS-03-32, Computer and Information Science, University of Pennsylvania.
4. Huenerfauth, M. 2004a. A Multi-Path Architecture for Machine Translation of English Text into ASL Animation. HLT-NAACL Student Workshop.
5. Huenerfauth, M. 2004b. Spatial Representation of Classifier Predicates for Machine Translation into American Sign Language. Workshop on Representation and Processing of Signed Languages, LREC 2004.
6. Huenerfauth, M. 2004c. Spatial and Planning Models of ASL Classifier Predicates for Machine Translation. 10th International Conference on Theoretical and Methodological Issues in Machine Translation: TMI 2004, Baltimore, MD, USA.
7. Huenerfauth, M. 2005. (To appear in July.) American Sign Language Spatial Representations for an Accessible User-Interface. In 3rd International Conference on Universal Access in Human-Computer Interaction. Las Vegas, NV, USA.
8. Liddell, S. 2003. Grammar, Gesture, and Meaning in American Sign Language. UK: Cambridge Univ. Press.
9. Morford, J., and MacFarlane, J. Frequency Characteristics of ASL. Sign Language Studies, 3:2.
10. Neidle, C., Kegl, D., MacLaughlin, D., Bahan, B., and Lee, R. G. 2000. The Syntax of American Sign Language: Functional Categories and Hierarchical Structure. Cambridge, MA: The MIT Press

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Natural language interfaces

  • Interaction design

    • Interaction design process and methods

      • User centered design

• Social and professional topics

  • Professional topics

    • Computing profession

      • Assistive technologies

  • User characteristics

    • People with disabilities