Passive haptics based MR system for geography teaching

ABSTRACT

Passive haptics in an alternative way to provide the user with realistic haptic feedback with a low cost comparing to device-based active haptics. This paper presents a mixed reality system with passive haptics integration that supports bare-hand interaction for users to see and touch virtual objects in immersive virtual environment (IVE). With the educational potential, the purposed system is applied to geography teaching. The prototype combines feature-based visual tracking for user's head and objects in the real environment with free-hand motion detection via a depth camera, thus user can see both their hands and objects in the virtual environment. A study of virtual teaching of geography of Germany was conducted, 18 subjects participated in the formal experiments which include three different learning environments. The results showed that the participants learning in the virtual environment with passive haptics consume more time than the virtual environment with free-hand gesture in finishing geography teaching tasks which is contrary to the expected result. The reasons are discussed in detail. Furthermore, quality of memorizing geographical knowledge in the group with passive haptic feedback exceeds that in the group without.

References

1. Çavuşoğlu, M. C., Feygin, D., & Tendick, F. 2002. A critical study of the mechanical and electrical properties of the phantom haptic interface and improvements for high performance control. Presence: Teleoperators and Virtual Environments, 11(6), 555--568.
2. Ye, G., Corso, J. J., Hager, G. D., & Okamura, A. M. 2003. Vishap: Augmented reality combining haptics and vision. In Systems, Man and Cybernetics, 2003. IEEE International Conference on (Vol. 4, pp. 3425--3431). IEEE.
3. Ullrich, S., & Kuhlen, T. 2012. Haptic palpation for medical simulation in virtual environments. IEEE Transactions on Visualization and Computer Graphics, 18(4), 617--625.
4. Lindeman, R. W. 2003. Virtual contact: the continuum from purely visual to purely physical. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 47, No. 20, pp. 2103--2107). SAGE Publications.
5. Insko, B. E. 2001. Passive haptics significantly enhances virtual environments. PhD thesis, University of North Carolina at Chapel Hill.
6. Hoffman, H. G. 1998. Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments. In Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998(pp. 59--63). IEEE.
7. Lok, B., Naik, S., Whitton, M., & Brooks, F. P. 2003. Effects of handling real objects and self-avatar fidelity on cognitive task performance and sense of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 12(6), 615--628.
8. Milgram, P., & Kishino, F. 1994. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, 77(12), 1321--1329.
9. Jeon, S., & Choi, S. 2009. Haptic augmented reality: Taxonomy and an example of stiffness modulation. Presence: Teleoperators and Virtual Environments, 18(5), 387--408.
10. Inami, M., Kawakami, N., Sekiguchi, D., Yanagida, Y., Maeda, T., & Tachi, S. 2000. Visuo-haptic display using head-mounted projector. In Virtual Reality, 2000. Proceedings. IEEE (pp. 233--240). IEEE.
11. Cosco, F., Garre, C., Bruno, F., Muzzupappa, M., & Otaduy, M. A. 2013. Visuo-haptic mixed reality with unobstructed toolhand integration. IEEE Transactions on visualization and computer graphics, 19(1), 159--172.
12. Mitsou, N. C., Velanas, S. V., & Tzafestas, C. S. 2006. Visuo-haptic interface for teleoperation of mobile robot exploration tasks. In ROMAN 2006-The 15th IEEE International Symposium on Robot and Human Interactive Communication (pp. 157--163). IEEE.
13. Knoerlein, B., Székely, G., & Harders, M. 2007. Visuo-haptic collaborative augmented reality ping-pong. In Proceedings of the international conference on Advances in computer entertainment technology (pp. 91--94). ACM.
14. Eck, U., Pankratz, F., Sandor, C., Klinker, G., & Laga, H. 2015. Precise haptic device co-location for visuo-haptic augmented reality. IEEE transactions on visualization and computer graphics, 21(12), 1427--1441.
15. Lee, W., & Park, J. 2005. Augmented foam: a tangible augmented reality for product design. In Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05) (pp. 106--109). IEEE.
16. Fucentese, S. F., Rahm, S., Wieser, K., Spillmann, J., Harders, M., & Koch, P. P. 2015. Evaluation of a virtual-reality-based simulator using passive haptic feedback for knee arthroscopy. Knee Surgery, Sports Traumatology, Arthroscopy, 23(4), 1077--1085.
17. Amores, J., Benavides, X., & Shapira, L. 2016. TactileVR: Integrating Physical Toys into Learn and Play Virtual Reality Experiences. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 1--1). ACM.
18. Khademi, M., Hondori, H. M., Lopes, C. V., Dodakian, L., & Cramer, S. C. 2012. Haptic augmented reality to monitor human arm's stiffness in rehabilitation. In Biomedical Engineering and Sciences (IECBES), 2012 IEEE EMBS Conference on (pp. 892--895). IEEE.
19. Sandor, C., Uchiyama, S., & Yamamoto, H. 2007. Visuo-haptic systems: half-mirrors considered harmful. In Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07) (pp. 292--297). IEEE.
20. Kalenine, S., Pinet, L., & Gentaz, E. 2011. The visual and visuo-haptic exploration of geometrical shapes increases their recognition in preschoolers. International Journal of Behavioral Development, 35(1), 18--26.
21. Carlson, P., Peters, A., Gilbert, S. B., Vance, J. M., & Luse, A. 2015. Virtual Training: Learning Transfer of Assembly Tasks. IEEE Transactions on visualization and computer graphics, 21(6), 770--782.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User studies

    • Interaction paradigms

      • Mixed / augmented reality