Robust computer vision-based detection of pinching for one and two-handed gesture input

ABSTRACT

We present a computer vision technique to detect when the user brings their thumb and forefinger together (a pinch gesture) for close-range and relatively controlled viewing circumstances. The technique avoids complex and fragile hand tracking algorithms by detecting the hole formed when the thumb and forefinger are touching; this hole is found by simple analysis of the connected components of the background segmented against the hand. Our Thumb and Fore-Finger Interface (TAFFI) demonstrates the technique for cursor control as well as map navigation using one and two-handed interactions.

References

1. Balakrishnan, R. and MacKenzie, I. S., Performance differences in the fingers, wrist, and forearm in computer input control. in Proceedings of the CHI '97 Conference on Human Factors in Computing Systems, (1997), 303--310.
2. Baudel, T. and Beaudouin-Lafon, M. Charade: remote control of objects using free-hand gestures. Communications of the ACM, 36 (7). 28--35.
3. Bowman, D., Wingrave, C., Campbell, J., Ly, V. and Rhoton, C. Novel uses of pinch gloves for virtual environment interaction techniques. Virtual Reality, 6 (3). 122--129.
4. Buxton, W., A Three-State Model of Graphical Input. in INTERACT '90, (1990), 449--456.
5. Hinckley, K., Czerwinski, M. and Sinclair, M., Interaction and modeling techniques for desktop two-handed input. in Proc. of the ACM UIST'98 Symposium on User Interface Software and Technology, (1998), 49--58.
6. Horn, B. K. P. Robot Vision. MIT Press, Cambridge, MA, 1986.
7. Krueger, M. W. Artificial Reality II. Addison-Wesley, Menlo Park, 1991.
8. Kurtenbach, G., Fitzmaurice, G., Baudel, T. and Buxton, W., The design and evaluation of a GUI paradigm based on two-hands tablets and transparency. in Proceedings of the CHI '97 Conference on Human Factors in Computing Systems, (1997).
9. MacKenzie, I. S. and Oniszczak, A., A comparison of three selection techniques for touchpads. in Proc. ACM CHI' 98 Conf. on Human Factors in Computing Systems, (1998), 336--343.
10. Malik, S. and Laszlo, J., Visual touchpad: a two-hand gestural input device. in Proceedings of the 6th International Conference on Multimodal Interfaces, (State College, PA, 2004), 289--296.
11. Quek, F., Mysliwiec, T. and Zhao, M., FingerMouse: A freehand pointing interface. in IEEE Automatic Face and Gesture Recognition, (1995), 372--377.
12. Wilson, A., PlayAnywhere: a compact tabletop computer vision system. in Proceedings of the 18th Annual ACM Symposium on User Interface Software Technology, (2005), 83--92.
13. Wilson, A. and Cutrell, E., FlowMouse: A computer vision-based pointing and gesture input device. in INTERACT '05, (Rome, Italy, 2005).

Index Terms

• Hardware

  • Communication hardware, interfaces and storage

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens