NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers

ABSTRACT

We present an investigation of mechanically-actuated hand-held controllers that render the shape of virtual objects through physical shape displacement, enabling users to feel 3D surfaces, textures, and forces that match the visual rendering. We demonstrate two such controllers, NormalTouch and TextureTouch, which are tracked in 3D and produce spatially-registered haptic feedback to a user's finger. NormalTouch haptically renders object surfaces and provides force feedback using a tiltable and extrudable platform. TextureTouch renders the shape of virtual objects including detailed surface structure through a 4×4 matrix of actuated pins. By moving our controllers around while keeping their finger on the actuated platform, users obtain the impression of a much larger 3D shape by cognitively integrating output sensations over time. Our evaluation compares the effectiveness of our controllers with the two de-facto standards in Virtual Reality controllers: device vibration and visual feedback only. We find that haptic feedback significantly increases the accuracy of VR interaction, most effectively by rendering high-fidelity shape output as in the case of our controllers.

References

1. Alexander, J., Lucero, A., and Subramanian, S. 2012. Tilt displays: designing display surfaces with multi-axis tilting and actuation. In Proc. of ACM MobileHCI '12.
2. Araujo, B., Jota, R., Perumal, V., Yao, J.X., Singh, K. and Wigdor, D. Snake Charmer: Physically Enabling Virtual Objects. In Proc. of the TEI '16, 218--226.
3. Azmandian, M, Hancock, M., Benko, H., Ofek, E., and Wilson, A.D. 2016. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proc. of ACM SIGCHI '16.
4. Benali-Khoudja, M., Hafez, M., Alexandre, J.-M. and Kheddar, A. 2004. Tactile interfaces: a state-of-the-art survey. In Int. Symposium on Robotics, vol. 31.
5. Bowman, D., Kruiff, E., La Viola, J., and Poupyrev, I. 3D User Interfaces. Addison Wesley. 2004.
6. Bouzit, M., Popescu, G., Burdea, G., and Boian, R. 2002. The Rutgers Master II-ND force feedback glove. In Proc. of HAPTICS '02. 145--152.
7. Brewster, S. and Brown, L.M. 2004. Tactons: structured tactile messages for non-visual information display. In, Australasian User Interface Conference '04, 18--22.
8. Carter, T., Seah, S. A., Long, B., Drinkwater, B., Subramanian, S. 2013. Ultrahaptics: Multi-point mid-air haptic feedback for touch surfaces. In Proc. ACM UIST '13.
9. Choi, H.R., Kim, D., Chuc, N.H., Vuong, N.H.L., Koo, J., Nam, J., and Lee, Y. 2009. Development of integrated tactile display devices. In Electroactive Polymer Actuators and Devices. Proc. of SPIE Vol. 7287.
10. CyberGrasp Glove, CyberGlove Systems Inc. http://www.cyberglovesystems.com/cybergrasp/. Last accessed April 8, 2016.
11. CyberTouch Glove, CyberGlove Systems Inc. http://www.cyberglovesystems.com/cybertouch/. Last accessed April 8, 2016.
12. Follmer, S., Leithinger, D., Olwal, A., Hogge, A., and Ishii, H. 2013. inFORM: Dynamic physical affordances and constraints through shape and object actuation. In Proc. of ACM UIST '13. 417--426.
13. Haption Virtuose 6D. http://www.haption.com/site/index.php/en/products-menu-en/hardware-menu-en/virtuose-6d-menu-en. Last accessed July 29, 2016.
14. Hayward, V., Astley, O., Cruz-Hernandez, M., Grant, D., and Robles-De-La-Torre, G. 2004. Haptic interfaces and devices. Sensor Review, 24, 1. 16--29.
15. Hemmert, F., Hamann, S., Lwe, M., Wohlauf, A., Zeipelt, J., and Joost, G. 2010. Take me by the hand: Haptic compasses in mobile devices through shape change and weight shift. In Proc. of ACM NordiCHI'10.
16. Holz, C. and Baudisch, P. The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints. In Proc. CHI '10, 581--590.
17. Holz, C. and Baudisch, P. Understanding Touch. In Proc. of ACM SIGCHI '11, 2501--2510.
18. Iwata, H., Yano, H., Nakaizumi, F., and Kawamura, R. 2001. Project feelex: Adding haptic surface to graphics. In Proc. of ACM SIGGRAPH '01. 469--476.
19. Jang, S., Kim, L., Tanner, K., Ishii, H., and Follmer, S. 2016. Haptic Edge Display for Mobile Tactile Interaction. To appear in Proc. of ACM SIGCHI '16.
20. Kim, S.Y., Kyung, K.U., Park, J., and Kwon, D.S. 2007. Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator. Advanced Robotics. 21, 9. 961--981.
21. Krusteva, D., Sahoo, D., Marzo, A., Subramanian, S., and Coyle, D. 2015. Marionette: a multi-finger tilt feedback device for curvatures and haptic images perception. In Extended Abstracts of ACM SIGCHI '15.
22. Kyung, K.U., and Lee, J.Y. 2009. Ubi-Pen: A Haptic Interface with Texture and Vibrotactile Display. IEEE Comput. Graph. Appl. 29, 1. 56--64.
23. Lee, J., Dietz, P., Leigh, D., Yerazunis, W., and Hudson, S. Haptic Pen: A Tactile Feedback Stylus for Touch Displays. In Proc. of ACM UIST '04.
24. Luk, J., Pasquero, J., Little, S., MacLean, K., Levesque, V., and Hayward, V. 2006. A role for haptics in mobile interaction: initial design using a handheld tactile display prototype. In Proc. of ACM SIGCHI '06.
25. Massie, T. and Salisbury, J.K. 1994. The PHANToM Haptic Interface: A Device for Probing Virtual Objects. Dynamics and Control 1994. In Proc. of the ASME Winter Annual Meeting '94.
26. Novint Falcon, Novint Technologies Inc. http://www.novint.com/index.php/novintfalcon. Last accessed April 8, 2016.
27. Overholt, D. 2001. The MATRIX: A novel controller for musical expression. In Proc. of New Interfaces for Musical Expression (NIME '01). 1--4.
28. Prattichizzo, D., Chinello, F., Pacchierotti, C. and Malvezzi, M. 2013. Towards wearability in fingertip haptics: a 3-DoF wearable device for cutaneous force feedback. IEEE Trans. Haptics, vol. 6(4), 506--516.
29. Poupyrev, I., Nashida, T., Maruyama, S., Rekimoto, J., and Yamaji, Y. Lumen: Interactive visual and shape display for calm computing. SIGGRAPH ETech '04.
30. Rekimoto, J. 2013. Traxion: a tactile interaction device with virtual force sensation. In Proc. of ACM UIST '13.
31. Salisbury, K., Conti F., and Barbagli, F. 2004. Haptic rendering: introductory concepts. IEEE Computer Graphics and Applications, 24, 2. 24--32.
32. Sinclair, M., Pahud, M. and Benko, H. 2014. TouchMover 2.0 - 3D Touchscreen with Haptic Feedback and Haptic Texture. In Proc. of IEEE HAPTICS '14.
33. Summers, I.R., Chanter, C.M., Southall, A.L., and Brady, A.C. 2001. Results from a tactile array on the fingertip. In Proc. of Eurohaptics '01, 26--28.
34. Van der Linde, R.Q., Lammertse, P., Frederiksen, E., and Ruiterm B. 2002. The Haptic Master, a new highperformance haptic interface. Proc. of Eurohaptics '02.
35. Velazquez. R. 2010. Wearable Assistive Devices for the Blind. Chapter 17 in A. Lay-Ekuakille & S.C. Mukhopadhyay (Eds.), Wearable and Autonomous Biomedical Devices and Systems for Smart Environment: Issues and Characterization, LNEE 75, Springer.
36. Wang, Q. and Hayward, V. 2010. Biomechanically optimized distributed transducer based on lateral skin deformation. The International Journal of Robotics Research. 29, 4. 323--335.
37. Wellman, P., and Howe, R.D. 1995. Towards realistic vibrotactile display in virtual environments. In Proc. of ASME, 57, 2. 713--718.
38. Wijntjes, M. W., Sato, A., Hayward, V. and Kappers, A. M. Local surface orientation dominates haptic curvature discrimination. IEEE Haptics 2(2), 94--102.
39. Yano, H., Taniguchi, S., and Iwata, H. 2015. Shape and Friction Recognition of 3D Virtual Objects by Using 2DOF Indirect Haptic Interface. In Proc. of IEEE World Haptics Conference (WHC '15). 202--207.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Virtual reality

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Haptic devices

    • Interaction paradigms

      • Mixed / augmented reality

      • Virtual reality