Work at hand: an exploration of gesture in the context of work and everyday life to inform the design of gestural input devices

ABSTRACT

This research explores gestures used in the context of activities in the workplace and in everyday life in order to understand requirements and devise concepts for the design of gestural information appliances. A collaborative method of video interaction analysis devised to suit design explorations, the Video Card Game, was used to capture and analyse how gesture is used in the context of six different domains: the dentist's office; PDA and mobile phone use; the experimental biologist's laboratory; a city ferry service; a video cassette player repair shop; and a factory flowmeter assembly station. Findings are presented in the form of gestural themes, derived from the tradition of qualitative analysis but bearing some similarity to Alexandrian patterns. Implications for the design of gestural devices are discussed.

References

1. ALEXANDER, C., ISHIKAWA, S., SILVERSTEIN, M., JACOBSON, M., FIKSDAHL-KING, I. and ANGEL, S. (1977) A pattern language: towns, buildings, construction, New York, Oxford University Press.
2. BATES, E., (1994) Modularity, domain specificity and the development of language. In Evolution and Neurology of Language. Discussions in Neuroscience10(1-2) 136-149.
3. BOUISSAC, P. (accessed 2002) Describing gestures: boundaries, scales and perspectives, Virtual Symposium on the Multimodality of Human Communication: Theories, Problems and Applications, University of Toronto, May 3-5 2002.
4. BRERETON, M. (2001) Drawing Lessons in the Design of Tangible Media from a Study of Interactions with Mechanical Products, Proceedings of the Australian User Interface Conference, pp 92-100, 24th Australasian Computer Science Conference ACSC 2001 Held: Gold Coast, Australia, Jan 2001.
5. BRERETON, M.F. and MCGARRY, B. (2000) An Observational Study of How Objects Support Engineering Design Thinking and Communication: Implications for the design of tangible media. Proceedings of the ACM SIGCHI Conference on Computer Human Interaction, CHI 2000, pp 217-224, The Hague April 1-6th 2000. ACM Press.
6. BUUR, J., BINDER, T. and BRANDT, E. (2000), Taking Video beyond Hard Data in User Centered Design. Proceedings of the Participatory Design Conference, Nov. 00. New York
7. BUUR, J. and SOENDERGAARD, A. (2000) Video Card Game: An augmented environment for User Centred Design discussions. In Designing Augmented Reality Systems Conference, ACM Press, Elsinore
8. CHI'95 gesture Workshop: http://wex.www.media.mit.edu/people/wex/CHI95- workshop-writeup.html
9. CLARK, A. (1997) Being There: Putting Brain, Body and World Together Again, Cambridge, Mass, MIT Press
10. COUTAZ, D.J. and CROWLEY, J.L., (1995) Interpreting Human Gesture with Computer Vision. Position paper for the workshop "Gesture at the User Interface", CHI'95,
11. CRABTREE, A., HEMMINGS, T. and RODDEN, T. (2002) Pattern-based support for interactive design in domestic settings. Proc. Designing Interactive Systems Conference, 265-27, ACM Press.
12. ERICKSON, T (2000) Lingua Francas for design: sacred places and pattern languages, In Designing interactive systems Conference. New York
13. JAKOBSON, R (1990) On Language. eds WAUGH L.R and MONVILLE-BURSTON M. Cambridge, Mass.: Harvard University Press.
14. JORDAN, B. and HENDERSON, A. (1994). Interaction analysis: foundations and practice. The journal of the Learning Sciences 4 (1)
15. KEATES S, and PERRICOS C (1996) Gesture as a means of computer access. Communication Matters. 10(1)17-19
16. KRAUSS, R.M., CHEN, Y. and CHAWLA, P. (1996) Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us? In Advances in experimental social psychology 389-450. M. Zanna (ed.) Tampa: Academic Press.
17. KRAUSS, R.M., DUSHAY, R., CHEN, Y., and RAUSCHER, F. (1995). The communicative value of conversational hand gestures. Journal of Experimental Social Psychology31, 533-552.
18. KRAUSS, R.M., MORREL-SAMUELS, P. and COLASANTE, C. (1991). Do conversational hand gestures communicate? Journal of Personality and Social Psychology61, 743-754.
19. MCLUHAN, M (1964) Understanding Media: The Extensions of Man. New York: McGraw-Hill
20. MCNEILL, D. (1992) Hand and Mind: What Gestures Reveal about Thought." Chicago: University of Chicago Press
21. MCNEILL, D. (1985) So you think gestures are nonverbal? Psychological Review92:350-371.
22. NESPOULOUS, J.L. PERRON, P and LECOURS A.R., (1986) The Biological Foundations of Gestures: Motor and Semiotic Aspects. Lawrence Erlbaum Associates, Hillsdale, MJ,
23. PELACHAUD, C. and POGGI, I. (1997) Multimodal Embodied Agents ACM 0-89791-88-6/97/05
24. RAUSCHER, F.H., KRAUSS, R.M., and CHEN, Y. (1996). Gesture, speech and lexical access: The role of lexical movements in speech production. Psychological Science, 7 (4), 226-231.
25. RIZZOLATTI, G. LEONARDO F and VITTORIO G (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience 2 : 661-670
26. SPINNEY, L., (2000) BodyTalk New Scientist166(2233) 30-31
27. STRAUSS, A. L., CORBIN, J. (1998) Basics of qualitative research: techniques and procedures for developing grounded theory. Thousand Oaks, Sage Publications

Index Terms

• Computing methodologies

  • Modeling and simulation

    • Simulation types and techniques

      • Visual analytics

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens

    • Interaction paradigms

      • Natural language interfaces