RollingStone: Using Single Slip Taxel for Enhancing Active Finger Exploration with a Virtual Reality Controller

ABSTRACT

We propose using a single slip tactile pixel on virtual reality controllers to produce sensations of finger sliding and textures. When a user moves the controller on a virtual surface, we add a slip opposite to the movement, creating an illusion of a finger that is sliding on the surface, while varying the slip feedback changes lateral forces on fingertip. When coupled with hand motion the lateral forces can be used to create perceptions of artificial textures. RollingStone has been implemented as a prototype VR controller consisting of a ball-based slip display positioned under the user's fingertip. Within the slip display, a pair of motors actuates the ball, which is capable of gener- ating both short- and long-term two-degree-of-freedom slip feedback. An exploratory study was conducted to ensure that changing the relative motion between the finger and the ball could alter the perceptions conveying the properties of a tex- ture. The following two perception-based studies examined the minimum changes in speed of slip and angle of slip that are detectable by users. The results help us to design haptic patterns as well as our prototype applications. Finally, our preliminary user evaluation indicated that participants wel- comed RollingStone as a useful addition to the range of VR controllers.

References

1. Olivier Bau, Ivan Poupyrev, Ali Israr, and Chris Harrison. 2010. TeslaTouch: Electrovibration for Touch Surfaces. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, New York, NY, USA, 283--292.
2. Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. 2016. NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 717--728.
3. Mourad Bouzit, Grigore Burdea, George Popescu, and Rares Boian. 2002. The Rutgers Master II-new design force-feedback glove. IEEE/ASME Transactions on mechatronics 7, 2 (2002), 256--263.
4. Carissa J Cascio and K Sathian. 2001. Temporal cues contribute to tactile perception of roughness. Journal of Neuroscience 21, 14 (2001), 5289--5296.
5. Inrak Choi, Heather Culbertson, Mark R. Miller, Alex Olwal, and Sean Follmer. 2017. Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 119--130.
6. Inrak Choi, Elliot W Hawkes, David L Christensen, Christopher J Ploch, and Sean Follmer. 2016. Wolverine: A wearable haptic interface for grasping in virtual reality. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 986--993.
7. Inrak Choi, Eyal Ofek, Hrvoje Benko, Mike Sinclair, and Christian Holz. 2018. CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM.
8. Minwoo Choi and Gerard Jounghyun Kim. 2009. TouchBall: A Design and Evaluation of a Hand-held Trackball Based Touch-haptic Interface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 1535--1538.
9. Heather Culbertson, Samuel B. Schorr, and Allison M. Okamura. 2018. Haptics: The Present and Future of Artificial Touch Sensation. Annual Review of Control, Robotics, and Autonomous Systems 1, 1 (2018), null.
10. Heather Culbertson, Julie M Walker, and Allison M Okamura. 2016. Modeling and design of asymmetric vibrations to induce ungrounded pulling sensation through asymmetric skin displacement. In Haptics Symposium (HAPTICS), 2016 IEEE. IEEE, 27--33.
11. David Escobar-Castillejos, Julieta Noguez, Luis Neri, Alejandra Magana, and Bedrich Benes. 2016. A review of simulators with haptic devices for medical training. Journal of medical systems 40, 4 (2016), 104.
12. Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: Dynamic Physical Affordances and Constraints Through Shape and Object Actuation. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 417--426.
13. Brian T Gleeson, Scott K Horschel, and William R Provancher. 2009. Communication of direction through lateral skin stretch at the fingertip. In EuroHaptics conference, 2009 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2009. Third Joint. IEEE, 172--177.
14. CyberGrasp Glove. 2016. CyberGlove Systems Inc. (2016).
15. Xiaochi Gu, Yifei Zhang, Weize Sun, Yuanzhe Bian, Dao Zhou, and Per Ola Kristensson. 2016. Dexmo: An Inexpensive and Lightweight Mechanical Exoskeleton for Motion Capture and Force Feedback in VR. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 1991--1995.
16. Sidhant Gupta, Tim Campbell, Jeffrey R. Hightower, and Shwetak N. Patel. 2010. SqueezeBlock: Using Virtual Springs in Mobile Devices for Eyes-free Interaction. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, New York, NY, USA, 101--104.
17. Vincent Hayward and M Cruz-Hernandez. 2000. Tactile display device using distributed lateral skin stretch. In Proceedings of the haptic interfaces for virtual environment and teleoperator systems symposium, Vol. 69. ASME, 1309--1314.
18. Colin Ho, Jonathan Kim, Sachin Patil, and Ken Goldberg. 2015. The Slip-Pad: A haptic display using interleaved belts to simulate lateral and rotational slip. In World Haptics Conference (WHC), 2015 IEEE. IEEE, 189--195.
19. Mark Holliins, Richard Faldowski, Suman Rao, and Forrest Young. 1993. Perceptual dimensions of tactile surface texture: A multidimensional scaling analysis. Perception & psychophysics 54, 6 (1993), 697--705.
20. Hisayoshi Honda, Shinichi Hirai, and others. 2016. Development of a novel slip haptic display device based on the localized displacement phenomenon. IEEE Robotics and Automation Letters 1, 1 (2016), 585--592.
21. Hiroo Iwata, Hiroaki Yano, Fumitaka Nakaizumi, and Ryo Kawamura. 2001. Project FEELEX: Adding Haptic Surface to Graphics. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01). ACM, New York, NY, USA, 469--476.
22. RS Johansson and G Westling. 1984. Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Experimental brain research 56, 3 (1984), 550--564.
23. Rebecca P Khurshid, Naomi T Fitter, Elizabeth A Fedalei, and Katherine J Kuchenbecker. 2017. Effects of grip-force, contact, and acceleration feedback on a teleoperated pick-and-place task. IEEE transactions on haptics 10, 1 (2017), 40--53.
24. Johan Kildal. 2010. 3D-press: Haptic Illusion of Compliance when Pressing on a Rigid Surface. In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI '10). ACM, New York, NY, USA, Article 21, 8 pages.
25. Seung-Chan Kim, Ali Israr, and Ivan Poupyrev. 2013. Tactile Rendering of 3D Features on Touch Surfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 531--538.
26. Daniel Leithinger, Sean Follmer, Alex Olwal, and Hiroshi Ishii. 2014. Physical Telepresence: Shape Capture and Display for Embodied, Computer-mediated Remote Collaboration. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 461--470.
27. Thomas H Massie, J Kenneth Salisbury, and others. 1994. The phantom haptic interface: A device for probing virtual objects. In Proceedings of the ASME winter annual meeting, symposium on haptic interfaces for virtual environment and teleoperator systems, Vol. 55. Citeseer, 295--300.
28. Joe Mullenbach, Dan Johnson, J Edward Colgate, and Michael A Peshkin. 2012. ActivePaD surface haptic device. In Haptics Symposium (HAPTICS), 2012 IEEE. IEEE, 407--414.
29. Takaaki Nara, Masaya Takasaki, Susumu Tachi, and Toshiro Higuchi. 2000. An application of SAW to a tactile display in virtual reality. In Ultrasonics Symposium, 2000 IEEE, Vol. 1. IEEE, 1--4.
30. Ulf Norrsell and Hakan Olausson. 1994. Spatial cues serving the tactile directional sensibility of the human forearm. The Journal of physiology 478, 3 (1994), 533--540.
31. Shogo Okamoto, Hikaru Nagano, and Yoji Yamada. 2013. Psychophysical dimensions of tactile perception of textures. IEEE Transactions on Haptics 6, 1 (2013), 81--93.
32. Ayberk Ozgur, Wafa Johal, and Pierre Dillenbourg. 2016. Permanent magnet-assisted omnidirectional ball drive. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE.
33. Claudio Pacchierotti, Stephen Sinclair, Massimiliano Solazzi, Antonio Frisoli, Vincent Hayward, and Domenico Prattichizzo. 2017. Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives. IEEE transactions on haptics 10, 4 (2017), 580--600.
34. Jérôme Pasquero and Vincent Hayward. 2003. STReSS: A practical tactile display system with one millimeter spatial resolution and 700 Hz refresh rate. In Proc. Eurohaptics, Vol. 2003. 94--110.
35. Alvaro G Perez, Daniel Lobo, Francesco Chinello, Gabriel Cirio, Monica Malvezzi, José San Mart'ın, Domenico Prattichizzo, and Miguel A Otaduy. 2015. Soft finger tactile rendering for wearable haptics. In World Haptics Conference (WHC), 2015 IEEE. IEEE, 327--332.
36. Ivan Poupyrev, Tatsushi Nashida, and Makoto Okabe. 2007. Actuation and Tangible User Interfaces: The Vaucanson Duck, Robots, and Shape Displays. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07). ACM, New York, NY, USA, 205--212.
37. Domenico Prattichizzo, Francesco Chinello, Claudio Pacchierotti, and Monica Malvezzi. 2013. Towards wearability in fingertip haptics: a 3-dof wearable device for cutaneous force feedback. IEEE Transactions on Haptics 6, 4 (2013), 506--516.
38. William R Provancher and Nicholas D Sylvester. 2009. Fingerpad skin stretch increases the perception of virtual friction. IEEE Transactions on Haptics 2, 4 (2009), 212--223.
39. Jun Rekimoto. 2013. Traxion: A Tactile Interaction Device with Virtual Force Sensation. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 427--432.
40. Satoshi Saga and Ramesh Raskar. 2013. Simultaneous geometry and texture display based on lateral force for touchscreen. In World Haptics Conference (WHC), 2013. IEEE, 437--442.
41. Mark Salada, J Edward Colgate, MV Lee, and Peter Vishton. 2002a. Fingertip haptics: A novel direction in haptic display. In Proceedings of the 8th mechatronics forum international conference. University of Twente, 1211--1220.
42. Mark Salada, J Edward Colgate, Peter Vishton, and Eviatar Frankel. 2005. An experiment on tracking surface features with the sensation of slip. In Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. First Joint. IEEE, 132--137.
43. Mark Salada, Peter Vishton, J Edward Colgate, and Eviatar Frankel. 2004. Two experiments on the perception of slip at the fingertip. In Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS'04. Proceedings. 12th International Symposium on. IEEE, 146--153.
44. Mark A Salada, J Edward Colgate, Margaret V Lee, and Peter M Vishton. 2002b. Validating a novel approach to rendering fingertip contact sensations. In Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2002. HAPTICS 2002. Proceedings. 10th Symposium on. IEEE, 217--224.
45. Samuel B. Schorr and Allison M. Okamura. 2017. Fingertip Tactile Devices for Virtual Object Manipulation and Exploration. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3115--3119.
46. Alexa F. Siu, Eric J. Gonzalez, Shenli Yuan, Jason Ginsberg, Allen Zhao, and Sean Follmer. 2017. shapeShift: A Mobile Tabletop Shape Display for Tangible and Haptic Interaction. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 77--79.
47. Massimiliano Solazzi, William R Provancher, Antonio Frisoli, and Massimo Bergamasco. 2011. Design of a SMA actuated 2-DoF tactile device for displaying tangential skin displacement. In World Haptics Conference (WHC), 2011 IEEE. IEEE, 31--36.
48. Paul Strohmeier, Jesse Burstyn, Juan Pablo Carrascal, Vincent Levesque, and Roel Vertegaal. 2016. ReFlex: A Flexible Smartphone with Active Haptic Feedback for Bend Input. In Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '16). ACM, New York, NY, USA, 185--192.
49. Paul Strohmeier and Kasper Hornbæk. 2017. Generating Haptic Textures with a Vibrotactile Actuator. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4994--5005.
50. Ian R Summers, Craig M Chanter, Anna L Southall, and Alan C Brady. 2001. Results from a Tactile Array on the Fingertip. In Proceedings of Eurohaptics, Vol. 2001.
51. Novint Technologies. 2018. (2018). https://en.wikipedia.org/wiki/Novint_Technologies
52. Nikolaos G Tsagarakis, T Horne, and Darwin G Caldwell. 2005. Slip aestheasis: A portable 2d slip/skin stretch display for the fingertip. In Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2005. World Haptics 2005. First Joint. IEEE, 214--219.
53. Ramiro Velazquez, Edwige E Pissaloux, Moustapha Hafez, and Jérôme Szewczyk. 2008. Tactile rendering with shape-memory-alloy pin-matrix. IEEE Transactions on Instrumentation and Measurement 57, 5 (2008), 1051--1057.
54. Robert J. Webster, III, Todd E. Murphy, Lawton N. Verner, and Allison M. Okamura. 2005. A Novel Two-dimensional Tactile Slip Display: Design, Kinematics and Perceptual Experiments. ACM Trans. Appl. Percept. 2, 2 (April 2005), 150--165.
55. Eric Whitmire, Hrvoje Benko, Christian Holz, Eyal Ofek, and Mike Sinclair. 2018. Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM.
56. Laura Winfield, John Glassmire, J Edward Colgate, and Michael Peshkin. 2007. T-pad: Tactile pattern display through variable friction reduction. In EuroHaptics Conference, 2007 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2007. Second Joint. IEEE, 421--426.
57. Juan Wu, Na Li, Wei Liu, Guangming Song, and Jun Zhang. 2015. Experimental study on the perception characteristics of haptic texture by multidimensional scaling. IEEE transactions on haptics 8, 4 (2015), 410--420.
58. Vibol Yem, Ryuta Okazaki, and Hiroyuki Kajimoto. 2016a. FinGAR: Combination of Electrical and Mechanical Stimulation for High-fidelity Tactile Presentation. In ACM SIGGRAPH 2016 Emerging Technologies (SIGGRAPH '16). ACM, New York, NY, USA, Article 7, 2 pages.
59. Vibol Yem, Mai Shibahara, Katsunari Sato, and Hiroyuki Kajimoto. 2016b. Expression of 2DOF Fingertip Traction with 1DOF Lateral Skin Stretch. In International AsiaHaptics conference. Springer, 21--25.
60. Andre Zenner and Antonio Kruger. 2017. Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality. IEEE Transactions on Visualization and Computer Graphics 23, 4 (April 2017), 1285--1294.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Virtual reality

• Hardware

  • Communication hardware, interfaces and storage

    • Tactile and hand-based interfaces

      • Haptic devices