On-Body tangible interaction: using the body to support tangible manipulations for immersive visualization

ABSTRACT

Recent technological advancements in augmented and mixed reality devices opened up many opportunities to users to interact in their environments. However, there are many unanswered questions about the definition of these interaction paradigms. In this article, we present a new approach to interact in these environments: using the body as a physical support to constrain the movement of a multiple degrees of freedom device (3D Mouse). This mouse offers enough degrees of freedom as well as a necessary accuracy to support the interaction. The use of the body as a support for the interaction allows the user to move in his environment and avoids the inherent fatigue of mid-air interactions. Thus, we propose a new design space for tangible interactions on the body, and specifically on the forearm. To validate the feasibility of such an approach, we present an experiment aimed at establishing the range, stability and comfort of gestures performed with a multiple degrees of freedom mouse on the forearm.

References

1. Bach, B., Sicat, R., Beyer, J., Cordeil, M., Pfister, H. The Hologram in My Hand: How Effective is Interactive Exploration of 3D Visualizations in Immersive Tangible Augmented Reality? IEEE Transactions on Visualization and Computer Graphics (TVCG), 2018, IEEE VIS (InfoVis
2. Michel Beaudouin-Lafon. 2000. Instrumental interaction: an interaction model for designing post-WIMP user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '00). ACM, New York, NY, USA, 446--453.
3. Louis-Pierre Bergé, Marcos Serrano, Gary Perelman, and Emmanuel Dubois. 2014. Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services (MobileHCI '14). ACM, New York, NY, USA, 125--134.
4. Louis-Pierre Bergé, Emmanuel Dubois, and Mathieu Raynal. 2015. Design and Evaluation of an "Around the SmartPhone" Technique for 3D Manipulations on Distant Display. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction (SUI '15). ACM, New York, NY, USA, 69--78.
5. Lonni Besançon, Paul Issartel, Mehdi Ammi, and Tobias Isenberg. 2017. Mouse, Tactile, and Tangible Input for 3D Manipulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4727--4740.
6. Borg, G. Borg's Perceived Exertion and Pain Scales. Human Kinetics (1998), viii 104pp
7. R. Burgess, A. J. Falcão, T. Fernandes, R. A. Ribeiro, M. Gomes, A. Krone-Martins, and A. M. de Almeida. Selection of large-scale 3d point cloud data using gesture recognition. In Doctoral Conference on Computing, Electrical and Industrial Systems, pages 188--195. Springer, 2015
8. S. Clarke, N. Dass, and D. H. P. Chau. Naturalmotion: Exploring gesture controls for visualizing time-evolving graphs. Proceedings of IEEE VIS (poster session), 2016.
9. Maxime Cordeil, Tim Dwyer, and Christophe Hurter. 2016. Immersive solutions for future Air Traffic Control and Management. In Proceedings of the 2016 ACM Companion on Interactive Surfaces and Spaces (ISS Companion '16). ACM, New York, NY, USA, 25--31.
10. David Dobbelstein, Philipp Hock, and Enrico Rukzio. 2015. Belt: An Unobtrusive Touch Input Device for Head-worn Displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2135--2138.
11. Bernd Fröhlich and John Plate. 2000. The cubic mouse: a new device for three-dimensional input. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '00). ACM, New York, NY, USA, 526--531.
12. Harrison, C., Benko, H., and Wilson, A. D. OmniTouch: Wearable Multitouch Interaction Everywhere. InACMUIST ' 11(2011), 441
13. Chris Harrison, Desney Tan, and Dan Morris. 2010. Skinput: appropriating the body as an input surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 453--462.
14. Hinckley, K., Wigdor, D., Input Technologies and Techniques. Chap. 9 in The HCI Handbook, 3rd Edition, Taylor & Francis
15. Christophe Hurter, Benjamin Tissoires, and Stéphane Conversy. 2009. FromDaDy: Spreading Aircraft Trajectories Across Views to Support Iterative Queries. IEEE Transactions on Visualization and Computer Graphics 15, 6 (November 2009), 1017--1024.
16. Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2012. Tangible remote controllers for wall-size displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 2865--2874.
17. Thorsten Karrer, Moritz Wittenhagen, Leonhard Lichtschlag, Florian Heller, and Jan Borchers. 2011. Pinstripe: eyes-free continuous input on interactive clothing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 1313--1322.
18. T. Klein, F. Guéniat, L. Pastur, F. Vernier, and T. Isenberg. A design study of direct-touch interaction for exploratory 3d scientific visualization. In Computer Graphics Forum, volume 31, pages 1225--1234. Wiley Online Library, 2012.
19. Lexip 3D http://www.amazon.fr/Lexip-3DM-PRO-Souris-Filaire-Noir/dp/B00IZU1NE4
20. D. Lopez, L. Oehlberg, C. Doger, and T. Isenberg. Towards An Understanding of Mobile Touch Navigation in a Stereoscopic Viewing Environment for 3d Data Exploration. IEEE Transactions on Visualization and Computer Graphics, 22(5):1616--1629, May 2016
21. P. Milgram and H. Colquhoun. A taxonomy of real and virtual world display integration. Mixed reality: Merging real and virtual worlds, 1:1 -26, 1999
22. B. P. Miranda, N. J. S. Carneiro, C. G. R. dos Santos, A. A. de Freitas, J. Magalhães, B. S. Meiguins, et al. Categorizing issues in mid-airinfovis interaction. In Information Visualisation (IV), 2016 20th Inter- national Conference, pages 242--246. IEEE, 2016
23. Mistry, P., Maes, P., and Chang, L. WUW - wear Urworld. InACM CHI EA '09(2009), 4111
24. Michael Ortega and Laurence Nigay. 2009. AirMouse: Finger Gesture for 2D and 3D Interaction. In Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part II (INTERACT '09), Tom Gross, Jan Gulliksen, Paula Kotzé, Lars Oestreicher, Philippe Palanque, Raquel Oliveira Prates, and Marco Winckler (Eds.). Springer-Verlag, Berlin, Heidelberg, 214--227.
25. Gary Perelman, Marcos Serrano, Mathieu Raynal, Celia Picard, Mustapha Derras, and Emmanuel Dubois. 2015. The Roly-Poly Mouse: Designing a Rolling Input Device Unifying 2D and 3D Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 327--336.
26. David Rudi, Ioannis Giannopoulos, Peter Kiefer, Christian Peier, and Martin Raubal. 2016. Interacting with Maps on Optical Head-Mounted Displays. In Proceedings of the 2016 Symposium on Spatial User Interaction (SUI '16). ACM, New York, NY, USA, 3--12.
27. Houssem Saidi, Marcos Serrano, Pourang Irani, and Emmanuel Dubois. 2017. TDome: A Touch-Enabled 6DOF Interactive Device for Multi-Display Environments. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5892--5904.
28. Marcos Serrano, Barrett M. Ens, and Pourang P. Irani. 2014. Exploring the use of hand-to-face input for interacting with head-worn displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3181--3190.
29. Shu-Yang Lin, Chao-Huai Su, Kai-Yin Cheng, Rong-Hao Liang, Tzu-Hao Kuo, and Bing-Yu Chen. 2011. Pub - point upon body: exploring eyes-free interaction and methods on an arm. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA, 481--488.
30. Tamaki, E., Miyak, T., and Rekimoto, J. BrainyHand: AWearable Computing Device Without HMD and It'sInteraction Techniques. InAVI '10(2010), 387--388.
31. Peng Song, Wooi Boon Goh, Chi-Wing Fu, Qiang Meng, and Pheng-Ann Heng. 2011. WYSIWYF: exploring and annotating volume data with a tangible handheld device. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 1333--1342.
32. Dong-Bach Vo, Eric Lecolinet, and Yves Guiard. 2014. Belly gestures: body centric gestures on the abdomen. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI '14). ACM, New York, NY, USA, 687--696.
33. Julie Wagner, Mathieu Nancel, Sean G. Gustafson, Stephane Huot, and Wendy E. Mackay. 2013. Body-centric design space for multi-surface interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 1299--1308.
34. Martin Weigel, Tong Lu, Gilles Bailly, Antti Oulasvirta, Carmel Majidi, and Jürgen Steimle. 2015. iSkin: Flexible, Stretchable and Visually Customizable On-Body Touch Sensors for Mobile Computing. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2991--3000.
35. Shumin Zhai and Paul Milgram. 1998. Quantifying coordination in multiple DOF movement and its application to evaluating 6 DOF input devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '98), Clare-Marie Karat, Arnold Lund, Joëlle Coutaz, and John Karat (Eds.). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 320--327.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction techniques