A survey of design issues in spatial input

ABSTRACT

We present a survey of design issues for developing effective free-space three-dimensional (3D) user interfaces. Our survey is based upon previous work in 3D interaction, our experience in developing free-space interfaces, and our informal observations of test users. We illustrate our design issues using examples drawn from instances of 3D interfaces.

For example, our first issue suggests that users have difficulty understanding three-dimensional space. We offer a set of strategies which may help users to better perceive a 3D virtual environment, including the use of spatial references, relative gesture, two-handed interaction, multisensory feedback, physical constraints, and head tracking. We describe interfaces which employ these strategies.

Our major contribution is the synthesis of many scattered results, observations, and examples into a common framework. This framework should serve as a guide to researchers or systems builders who may not be familiar with design issues in spatial input. Where appropriate, we also try to identify areas in free-space 3D interaction which we see as likely candidates for additional research.

An extended and annotated version of the references list for this paper is available on-line through mosaic at address http://uvacs.cs.virginia.edu/~kph2q/.

References

1. 1.Adelstein, B., Johnston, E., Ellis, S., "A Testbed for Characterizing Dynamic Response of Virtual Environment Spatial Sensors," UIST'92, 15-22.
2. 2.Badler, N., Manoochehri, K., Baraff, D. "Multi- Dimensional Input Techniques and Articulated Figure Positioning by Multiple Constraints," ACM Workshop on Interactive 3D Graphics, 1986, pp. 151-170.
3. 3.T. Baudel, M. Beaudouin-Lafon, "Charade: Remote Control of Objects Using Hand Gestures," Communications of the ACM, 36 (7), 1993, 28-35.
4. 4.Bergman, L., Fuchs, H., Grant, E., "Image Rendering by Adaptive Refinement," Computer Graphics, 20 (4), 1986, pp. 29-37.
5. 5.Bier, E. A., Stone, M. C., "Snap-Dragging," Computer Graphics, 20 (4), 1986, pp. 233-240.
6. 6.Bier, E. A., "Snap-Dragging In Three Dimensions," Proc. 1990 Symposium on Interactive 3D Graphics, Computer Graphics, 24 (2), pp. 193-204.
7. 7.Bier, E., Stone, M., Pier, K., Buxton, W., DeRose, T., "Toolglass and Magic Lenses: The See-Through Interface," SIGGRAPH '93, pp. 73-80.
8. 8.Bolt, R., "Put-That-There: Voice and Gesture at the Graphics Interface," SIGG~H '80, 262-70.
9. 9.Bolt, R. A., Herranz, E., "Two-Handed Gesture in Multi-Modal Natural Dialog," UIST '92, pp. 7-13.
10. 10.Brooks, F. P. Jr., "Walkthrough--a Dynamic Graphics System for Simulating Virtual Buildings," Proc. ACM Workshop on Interactive 3D Graphics, 1986, pp. 9-21.
11. 11.Brooks, F., "Grasping Reality Through Illusion: Interactive Graphics Serving Science," CHI'88.
12. 12.Brooks, F., Ouh-Young, M., Batter, J., Kilpatrick, P., "Project GROPE--Haptic Displays for Scien-tific Visualization," Comp. Graph. 24 (4), 1990.
13. 13.Bryson, S., Levit, C., "The Virtual Wind Tunnel," IEEE CG&A, July 1992, pp. 25-34.
14. 14.Butterworth, J., Davidson, A., Hench, S., Olano, T. M., "3DM: A Three Dimensional Modeler Using a Headmounted Display," Proc. 1992 Symp. on Interactive 3D Graphics, pp. 135-138.
15. 15.Buxton, W., Myers, B., "A Study in Two-Handed Input," CHI'86, pp. 321-326.
16. 16.Card, S., Mackinlay, J., Robertson, G., "The Design Space of Input Devices," CHI'89, 117-124.
17. 17.Card, S., Robertson, G., Mackinlay, J., "The Information Visualizer, an Information Workspace," CHI'91, pp. 181-187.
18. 18.Chung, J. C., "A comparison of Head-tracked and Non-head-tracked Steering Modes in the Targeting of Radiotherapy Treatment Beams," Proc. 1992 Symp. on Interactive 3D Graphics, 193-196.
19. 19.Cohen, P., Sullivan, J., "Synergistic Use of Direct Manipulation and Natural Language," CHi'89, pp. 227-233.
20. 20.Conner, D., Snibbe, S., Herndon, K., Robbins, D., Zeleznik, R., van Dam, A., "Three-Dimensional Widgets," 1992 Symp. on Int. 3D Graph, 183-188.
21. 21.Cruz-Neira, C., Sandin, D., DeFanti, T., "Surround- Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE," SIGGRAPH '93, pp. 135-142.
22. 22.M. Deering, "High Resolution Virtual Reality," Computer Graphics, 26 (2), pp. 195-202.
23. 23.Feiner, S., Maclntyre, B., Haupt, M., Solomon, E., "Windows on the World: 2D Windows for 3D Augmented Reality," UIST'93, pp. 145-155.
24. 24.Feiner, S., Macintyre, B., Seligmann, D., "Knowlege- Based Augmented Reality," Comm. of the ACM, 36 (7), 1993, pp. 53-61.
25. 25.Feiner, S., Shamash, A., "Hybrid User Interfaces: Breeding Virtually Bigger Interfaces for Physically Smaller Computers," UIST '91, pp. 9-17.
26. 26.S.S. Fisher, M. McGreevy, J. Humphries, W. Robinett, "Virtual Interface Environment for Telepresence Applications," Oct. 1988, Proc. Human Factors Society 32nd Annual Meeting.
27. 27.Fitzmaurice, G. W., "Situated Information Spaces and Spatially Aware Palmtop Computers," Comm. of the ACM, 36 (7), 1993, pp. 39-49.
28. 28.Foley, J. D., Wallace, V., Chan, P., "The Human Factors of Computer Graphics Interaction Techniques," iEEE CG&A, Nov. 1984, pp. 13-48.
29. 29.Galyean, T. A., Hughes, J. F., "Sculpting: An Interactive Volumetric Modeling Technique," Computer Graphics, 25 (4), pp. 267-274.
30. 30.Gibson, J., The Ecological Approach to Visual Perception. Lawrence Efibaum, Hillsdale, NJ.
31. 31.Gleicher, M., "Supporting Numerical Computations in Interactive Contexts," Graphics Interface '93.
32. 32.Guiard, Y., "Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model," The Journal of Motor Behavior, 19 (4), 1987, pp. 486-517.
33. 33.Hauptmann, A. G., "Speech and Gestures for Graphic Image Manipulation," CHI'89, 241-245.
34. 34.Herndon, K., Zeleznik, R., Robbins, D., Conner, B., Snibbe, S., van Dam, A., "Interactive Shadows," UIST '92, pp. 1-6.
35. 35.Hinckley, K., Pausch, R, Goble, J., Kassell, N., "Passive Real-World Interface Props for Neurosurgical Visualization," CHI'94, 452-458.
36. 36.Iwata, H., "Artificial Reality with Force-feedback: Development of Desktop Virtual Space with Compact Master Manipulator," Computer Graphics, 24 (4), pp. 165-170.
37. 37.Jacob, R., Sibert, L., "The Perceptual Structure of Multidimensional Input Device Selection," CHI'92, pp. 211-218.
38. 38.Kabbash, P., MacKenzie, I. S., Buxton, W., "Human Performance Using Computer Input Devices in the Preferred and Non-Preferred Hands," INTERCHI'93, pp. 474-481.
39. 39.Kabbash, P., Buxton, W., Sellen, A., "Two-Handed Input in a Compound Task," CHI'94, pp. 417-423.
40. 40.Kaufman, A., Yagel, R., "Tools for Interaction in Three Dimensions," Proc. 3rd International Conf. on HCI (Boston, MA), Vol. 1, Sept. 1989, pp. 468~475.
41. 41.Krueger, M., "Environmental Technology: Making the Real World Virtual," Communications of the ACM, 36 (7), 1993, pp. 36-37.
42. 42.Liang, J., Shaw, C., Green, M., "On Temporal-Spatial Realism in the Virtual Reality Environment," UIST'91, pp. 19-25.
43. 43.Liang, J., Green, M., "JDCAD: A Highly Interactive 3D Modeling System," 3rd International Conference on CAD and Computer Graphics, Beijing, China, Aug. 1993, 217-222.
44. 44.Mackinlay, J., Card, S., Robertson, G., "Rapid Controlled Movement Through a Virtual 3D Workspace," Comp. Grap., 24 (4), 1990, 171-176.
45. 45.McKenna, M., "Interactive Viewpoint Control and Three-dimensional Operations," Proc. 1992 Symposium on Interactive 3D Graphics, pp. 53-56.
46. 46.Minsky, M., Ouh-young, M., Brooks, F. P., Behensky, M., "Feeling and Seeing: Issues in Force Display," Comp. Graph., 24 (2), 234-244.
47. 47.Nielsen, J., "Noncommand User Interfaces," Communications of the ACM, 36 (4), pp. 83-99.
48. 48.Norman D., The Design of Everyday Things. Doubleday: New York, New York, 1990.
49. 49.Ostby, E., "Describing Free-Form 3D Surfaces for Animation," Proc. ACM Workshop on interactive 3D Graphics, Oct. 1986, pp. 251-258.
50. 50.Pausch, R., "Support for Rapid Prototyping of Twoand Three-Dimensional User Interfaces," Proposal for ARPA BAA 93-42. Comp. Science Department, University of Virginia, March, 1994.
51. 51.Pausch, R., Shackelford, M. A., Proffitt, D., "A User Study Comparing Head-Mounted and Stationary Displays," Proc. IEEE Symposium on Research Frontiers in Virtual Reality, Oct. 1993.
52. 52.Pixsys Inc., 3522 22nd St., Boulder, CO 80304. [phone].
53. 53.Polhemus Navigation Sciences, Inc., P. O. Box 560, Colchester, VT 05446. [phone].
54. 54.Sachs, E., Roberts, A., Stoops, D., "3-Draw: A Tool for Designing 3D Shapes," IEEE Computer Graphics & Applications, Nov. 1991, pp. 18-26.
55. 55.Schmandt, C. M., "Spatial Input/Display Correspondence in a Stereoscopic Computer Graphic Work Station," Computer Graphics, 17 (3), 1983, pp. 253- 262.
56. 56.Sears, A.,Plaisant, C., Shneiderman, B., "A New Era for High Precision Touchscreens," in Advances in Human-Computer Interaction, Hartson, Hix, eds., Vol. 3, 1992, pp. 1-33.
57. 57.Shepard, R. N., Metzler, J., "Mental Rotation of Three- Dimensional Objects," Science, Vol. 171, 1971, pp. 701-703.
58. 58.Spaceball Technologies, Inc. [phone].
59. 59.Stoakley, R., Pausch, R., "Virtual Kit of Parts," unpublished manuscript, available through mosaic at http ://uvacs. cs. vi rg inia. edu/~ rws2v/p linth, html.
60. 60.Sturman, D., Zeltzer, D., Pieper, S., "Hands-On Interaction with Virtual Environments," UIST'89, pp. 19-24.
61. 61.I.E. Sutherland, "A Head-mounted Three Dimensional Display," Proc. the Fall Joint Computer Conference, 1968, pp. 757-764.
62. 62.Takemura, H., Tomono, A., Kayashi, Y., "An Evaluation of 3-D Object Pointing Using a Field Sequential Stereoscopic Display," Proc. Graphics Interface '88, June 1988, pp. 112-118.
63. 63.Taylor, R., Robinett, W., Chi, V., Brooks, F., Wright, W., Williams, R., Snyder, E., "The Nanomanipulator: A Virtual-Reality interface for a Scanning Tunneling Microscope," SIGGRAPH'93, pp. 127-134.
64. 64.Ware, C., "Using Hand Position for Virtual Object Placement," Visual Comp., 6 (5), 1990, 245-253.
65. 65.Ware, C., Osborne, S., "Exploration and Virtual Camera Control in Virtual Three Dimensional Environments," Comp. Graph., 24 (2), 175-183.
66. 66.Ware, C., Arthur, K., Booth, K. S., "Fish Tank Virtual Reality," INTERCHI'93, pp. 37-41.
67. 67.Wellner, P., "Interacting with Paper on the DigitalDesk," Communications of the ACM, 36 (7), 1993, pp. 87-97.
68. 68.Zhai, S., Buxton, W., Milgram, P., "The "Silk Cursor": Investigating Transparency for 3D Target Acquisition," CHI'94, pp. 459-464.
69. 69.Zimmerman, T., Lanier, J., Blanchard, C., Bryson, S., Harvill, Y., "A Hand Gesture Interface Device," CHI+GI'87, pp. 189-192.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Virtual reality

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens

    • Interaction paradigms