PlayAnywhere: a compact interactive tabletop projection-vision system

ABSTRACT

We introduce PlayAnywhere, a front-projected computer vision-based interactive table system which uses a new commercially available projection technology to obtain a compact, self-contained form factor. PlayAnywhere's configuration addresses installation, calibration, and portability issues that are typical of most vision-based table systems, and thereby is particularly motivated in consumer applications. PlayAnywhere also makes a number of contributions related to image processing techniques for front-projected vision-based table systems, including a shadow-based touch detection algorithm, a fast, simple visual bar code scheme tailored to projection-vision table systems, the ability to continuously track sheets of paper, and an optical flow-based algorithm for the manipulation of onscreen objects that does not rely on fragile tracking algorithms.

References

1. Ballard, D., and C. Brown Computer Vision. Prentice-Hall, 1982.
2. Barron, J., D. Fleet, S. Beauchemin, and T. Burkitt, Performance of Optical Flow Techniques. in Computer Vision and Pattern Recognition, (1992), 236--242.
3. Berard, F., The Magic Table: Computer Vision Based Augmentation of a Whiteboard for Creative Meetings. in IEEE International Conference in Computer Vision, Workshop on Projector-Camera Systems (PROCAMS'03), (2003).
4. Buxton, W. Projection-Vision Systems: Towards a Human-Centric Taxonomy Unpublished Manuscript, Toronto: Buxton Design, 2005.
5. Corso, J., D. Burschka and G. Hager, The 4D Touchpad: Unencumbered HCI with VICs. in Proceedings of the CVPR-HCI Workshop, (2003).
6. Dietz, P.H., and D. L. Leigh, DiamondTouch: A Multi-User Touch Technology. in ACM Symposium on User Interface Software and Technology, (2001), 219--226.
7. Fiala, M. ARTag Revision 1, A Fiducial Marker System Using Digital Techniques NRC Technical Report (NRC 47419), National Research Council of Canada, 2004.
8. Gonzalez, R., and R. Woods Digital Image Processing: Second Edition. Prentice-Hall, 2002.
9. Horn, B.K.P. Closed Form Solution of Absolute Orientation Using Unit Quaternions. Journal of the Optical Society, 4 (4). 629--642.
10. Kang, S.B. Radial Distortion Snakes. IEICE Transactions on Information and Systems, E84-D (12). 1603--1611.
11. Kato, H., M. Billinghurst, I. Poupyrev, K. Imamoto, K. Tachibana, Virtual Object Manipulation on a Table-Top AR Environment. in Proceedings of ISAR 2000, (2000).
12. Kim, J., S. Seitz, and M. Agrawala, Video-based Document Tracking: Unifying Your Physical and Electronic Desktops. in Proceedings of ACM Symposium on User Interface Software and Technology, (2004), 99--107.
13. Kjeldsen, R., C. Pinhanez, G. Pingali, and J. Hartman, Interacting with Steerable Projected Displays. in International Conference on Automatic Face and Gesture Recognition, (2002).
14. Klemmer, S.R., M. W. Newman, R. Farrell, M. Bilezikjian, J. A. Landay, The Designer's Output: A Tangible Interface for Collaborative Web Site Design. in ACM Symposium on User Interface Software and Technology, (2001), 1--10.
15. Koike, H., and Y. Sato, Y. Kobayashi Integrating Paper and Digital Information on EnhancedDesk: a Method for Realtime Finger Tracking on an Augmented Desk System. ACM Transactions on Computer-Human Interaction (TOCHI), 8 (4). 307--322.
16. Leibe, B., T. Starner, W. Ribarsky, Z. Wartell, D. Krum, J. Weeks, B. Singletary, and L. Hodges. Toward Spontaneous Interaction with the Perceptive Workbench. IEEE Computer Graphics and Applications, 20 (6). 54--65.
17. Letessier, J., and F. Berard, Visual Tracking of Bare Fingers for Interactive Surfaces. in ACM Symposium on User Interface Software and Technology, (2004).
18. Lowe, D. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60 (2). 91--110.
19. Malik, S., and J. Laszlo, Visual Touchpad: A Two-Handed Gestural Input Device. in Proceedings of the International Conference on Multimodal Interfaces, (2004), 289--296.
20. Matsushita, N., J. Rekimoto, HoloWall: Designing a Finger, Hand, Body and Object Sensitive Wall. in ACM Symposium on User Interface Software and Technology (UIST), (1997).
21. Newman, W., and P. Wellner, A Desk Supporting Computer-based Interaction with Paper Documents. in CHI '92, (1992).
22. Owen, C., F. Xiao, and P. Middlin, What is the Best Fiducial? in Augmented Reality Toolkit, the First IEEE International Workshop, (2002).
23. Patten, J., H. Ishii, J. Hines, G. Pangaro, Sensetable: A Wireless Object Tracking Platform for Tangible User Interfaces. in Proceedings of CHI 2001, (2001), 253--260.
24. Rekimoto, J., SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. in Proc. CHI 2002, (2002), 113--120.
25. Rekimoto, J., and M. Saitoh, Augmented Surfaces: A Spatially Continuous Workspace for Hybrid Computing Environments. in Proceedings of CHI 99, (1999), 378-385.
26. Rekimoto, J., and Y. Ayatsuka, CyberCode: Designing Augmented Reality Environments with Visual Tags. in Designing Augmented Reality Environments (DARE 2000), (2000).
27. Robinson, J., and C. Robertson, The LivePaper System: Augmenting Paper on and Enhanced Tabletop. in Computers and Graphics, (2001), 731--743.
28. Segen, J., and S. Kumar, Shadow Gestures: 3D Hand Pose Estimation Using a Single Camera. in Proceedings of Computer Vision and Pattern Recognition, (1999), 479--485.
29. Tomasi, C., A. Rafii, and I. Torunoglu Full-size Projection Keyboard for Handheld Devices. Communications of the ACM, 46 (7). 70--75.
30. Ukita, N., and M. Kidode, Wearable Virtual Tablet: Fingertip Drawing on a Portable Plane-object Using An Active-Infrared Camera. in Proceedings of Intelligent User Interfaces, (2004), 169--176.
31. Ullmer, B., H. Ishii, The metaDESK: Models and Prototypes for Tangible User Interfaces. in ACM Symposium on User Interface Software and Technology, (1997), 223--232.
32. Underkoffler, J., and H. Ishii, Illuminating Light: An Optical Design Tool with a Luminous-Tangible Interface. in Proceedings of the ACM CHI 98 Human Factors in Computing Systems Conference, (1998), 542--549.
33. Wellner, P. Interacting with Paper on the DigitalDesk. Communications of the ACM, 36 (7). 86--97.
34. Wilson, A., FlowMouse: A Computer Vision-Based Pointing and Gesture Input Device. in Interact '05 (to appear), (2005).
35. Wilson, A., TouchLight: An Imaging Touch Screen and Display for Gesture-Based Interaction. in International Conference on Multimodal Interfaces, (2004).
36. Wren, C.R., Y. A. Ivanov, Volumetric Operations with Surface Margins. in Computer Vision and Pattern Recognition: Technical Sketches, (2001).
37. Wu, M., and R. Balakrishnan, Multi-finger and Whole Hand Gestural Interaction Techniques for Multi-User Tabletop Displays. in ACM Symposium on User Interface Software and Technology, (2003), 193--202.
38. Zhang, Z., Y. Wu, Y. Shan, and S. Shafer, Visual Panel: Virtual Mouse, Keyboard and 3D Controller with an Ordinary Piece of Paper. in Workshop on Perceptual User Interfaces (PUI 2001), (2001).

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens