RotoSwype: Word-Gesture Typing using a Ring

ABSTRACT

We propose RotoSwype, a technique for word-gesture typing using the orientation of a ring worn on the index finger. RotoSwype enables one-handed text-input without encumbering the hand with a device, a desirable quality in many scenarios, including virtual or augmented reality. The method is evaluated using two arm positions: with the hand raised up with the palm parallel to the ground; and with the hand resting at the side with the palm facing the body. A five-day study finds both hand positions achieved speeds of at least 14 words-per-minute (WPM) with uncorrected error rates near 1%, outperforming previous comparable techniques.

References

1. Sunggeun Ahn, Seongkook Heo, and Geehyuk Lee. 2017. Typing on a Smartwatch for Smart Glasses. In Proc. Interact. Surfaces Spaces - ISS '17. ACM Press, New York, New York, USA, 201--209.
2. Ahmed Sabbir Arif and Ali Mazalek. 2016. A survey of text entry techniques for smartwatches. In International Conference on HumanComputer Interaction. Springer, 255--267.
3. Daniel Ashbrook, Patrick Baudisch, and Sean White. 2011. Nenya: subtle and eyes-free mobile input with a magnetically-tracked finger ring. In Proc. 2011 Annu. Conf. Hum. factors Comput. Syst. - CHI '11. ACM Press, New York, New York, USA, 2043.
4. Daniel Ashbrook, Patrick Baudisch, and Sean White. 2011. Nenya: Subtle and Eyes-free Mobile Input with a Magnetically-tracked Finger Ring. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2043--2046.
5. Bartosz Bajer, I Scott MacKenzie, and Melanie Baljko. 2012. Huffman base-4 text entry glove (H4 TEG). In 2012 16th International Symposium on Wearable Computers. IEEE, 41--47.
6. Nikola Banovic, Koji Yatani, and Khai N Truong. 2013. Escapekeyboard: A sight-free one-handed text entry method for mobile touchscreen devices. International Journal of Mobile Human Computer Interaction (IJMHCI) 5, 3 (2013), 42--61.
7. Thomas Baudel and Michel Beaudouin-Lafon. 1993. Charade: remote control of objects using free-hand gestures. Commun. ACM 36, 7 (jul 1993), 28--35.
8. Xiaojun Bi and Shumin Zhai. 2016. IJQwerty: What Difference Does One Key Change Make? Gesture Typing Keyboard Optimization Bounded by One Key Position Change from Qwerty. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 49--58.
9. Matthew N. Bonner, Jeremy T. Brudvik, Gregory D. Abowd, and W. Keith Edwards. 2010. No-Look Notes: Accessible Eyes-Free Multitouch Text Entry. Springer, Berlin, Heidelberg, 409--426.
10. Géry Casiez, Nicolas Roussel, and Daniel Vogel. 2012. 1 Filter: A Simple Speed-based Low-pass Filter for Noisy Input in Interactive Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 2527--2530.
11. Steven J Castellucci and I Scott MacKenzie. 2011. Gathering text entry metrics on android devices. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. ACM, 1507--1512.
12. Steven J Castellucci and I Scott MacKenzie. 2013. Gestural text entry using Huffman codes. In Proceedings of the International Conference on Multimedia and Human-Computer Interaction, Vol. 119. 1--8.
13. Liwei Chan, Yi-Ling Chen, Chi-Hao Hsieh, Rong-Hao Liang, and BingYu Chen. 2015. CyclopsRing: Enabling Whole-Hand and ContextAware Interactions Through a Fisheye Ring. In Proc. 28th Annu. ACM Symp. User Interface Softw. Technol. - UIST '15. ACM Press, New York, New York, USA, 549--556.
14. Liwei Chan, Yi-Ling Chen, Chi-Hao Hsieh, Rong-Hao Liang, and BingYu Chen. 2015. CyclopsRing: Enabling Whole-Hand and ContextAware Interactions Through a Fisheye Ring. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). ACM, New York, NY, USA, 549--556.
15. Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3D input using two magnetic sensors. In Proc. 26th Annu. ACM Symp. User interface Softw. Technol. - UIST '13. ACM Press, New York, New York, USA, 237--244.
16. Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3D Input Using Two Magnetic Sensors. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 237--244.
17. Xiang 'Anthony' Chen, Tovi Grossman, and George Fitzmaurice. 2014. Swipeboard. In Proc. 27th Annu. ACM Symp. User interface Softw. Technol. - UIST '14. ACM Press, New York, New York, USA, 615--620.
18. Artem Dementyev and Joseph A. Paradiso. 2014. WristFlex. In Proc. 27th Annu. ACM Symp. User interface Softw. Technol. - UIST '14. ACM Press, New York, New York, USA, 161--166.
19. Android Developers. 2018. Android NDK. (2018). https://developer. android.com/ndk/
20. Jun Gong, Zheer Xu, Qifan Guo, Teddy Seyed, Xiang 'Anthony' Chen, Xiaojun Bi, and Xing-Dong Yang. 2018. WrisText: One-handed Text Entry on Smartwatch Using Wrist Gestures. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 181, 14 pages.
21. Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-handed Continuous Smartwatch Input using Wrist Gestures. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 861--872.
22. Gabriel González, José P Molina, Arturo S García, Diego Martínez, and Pascual González. 2009. Evaluation of text input techniques in immersive virtual environments. In New Trends on Human--Computer Interaction. Springer, 109--118.
23. Mitchell Gordon, Tom Ouyang, and Shumin Zhai. 2016. WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding. In Proc. 2016 CHI Conf. Hum. Factors Comput. Syst. - CHI '16. ACM Press, New York, New York, USA, 3817--3821.
24. E. (Etienne) Grandjean and Etienne. 1988. Fitting the task to the man : a textbook of occupational ergonomics. Taylor & Francis. 363 pages. https://dl.acm.org/citation.cfm?id=76694
25. Tovi Grossman, Xiang Anthony Chen, and George Fitzmaurice. 2015. Typing on Glasses: Adapting Text Entry to Smart Eyewear. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). ACM, New York, NY, USA, 144--152.
26. Aakar Gupta, Muhammed Anwar, and Ravin Balakrishnan. 2016. Porous Interfaces for Small Screen Multitasking using Finger Identification. In Proc. 29th Annu. Symp. User Interface Softw. Technol. UIST '16. ACM Press, New York, New York, USA, 145--156.
27. Aakar Gupta and Ravin Balakrishnan. 2016. DualKey: miniature screen text entry via finger identification. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 59--70.
28. Aakar Gupta, Jiushan Yang, and Ravin Balakrishnan. 2018. Asterisk and Obelisk: Motion Codes for Passive Tagging. In Proc. 25th Annu. ACM Symp. User interface Softw. Technol. - UIST '18. ACM.
29. Chris Harrison and Scott E. Hudson. 2009. Abracadabra. In Proc. 22nd Annu. ACM Symp. User interface Softw. Technol. - UIST '09. ACM Press, New York, New York, USA, 121.
30. Chris Harrison and Scott E. Hudson. 2009. Abracadabra: Wireless, High-precision, and Unpowered Finger Input for Very Small Mobile Devices. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, New York, NY, USA, 121--124.
31. Sandra G Hart. 1986. NASA Task load Index (TLX). Volume 1.0; Paper and pencil package. (1986).
32. Juan David Hincapié-Ramos, Xiang Guo, Paymahn Moghadasian, and Pourang Irani. 2014. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid-air Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1063--1072.
33. Jonggi Hong, Seongkook Heo, Poika Isokoski, and Geehyuk Lee. 2015. SplitBoard: A simple split soft keyboard for wristwatch-sized touch screens. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 1233--1236.
34. Invensense. 2018. MPU-6050 Motion Tracking | TDK. (2018). https: //www.invensense.com/products/motion-tracking/6-axis/mpu-6050/
35. Eleanor Jones, Jason Alexander, Andreas Andreou, Pourang Irani, and Sriram Subramanian. 2010. GesText: accelerometer-based gestural text-entry systems. In Proc. 28th Int. Conf. Hum. factors Comput. Syst. CHI '10. ACM Press, New York, New York, USA, 2173.
36. Wolf Kienzle and Ken Hinckley. 2014. LightRing: always-available 2D input on any surface. In Proc. 27th Annu. ACM Symp. User interface Softw. Technol. - UIST '14. ACM Press, New York, New York, USA, 157--160.
37. Wolf Kienzle and Ken Hinckley. 2014. LightRing: Always-available 2D Input on Any Surface. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 157--160.
38. Junhyeok Kim, William Delamare, and Pourang Irani. 2018. ThumbText: Text Entry for Wearable Devices Using a Miniature Ring. In Proceedings of Graphics Interface 2018 (GI 2018). Canadian HumanComputer Communications Society / Société canadienne du dialogue humain-machine, 18 -- 25.
39. Pascal Knierim, Valentin Schwind, Anna Maria Feit, Florian Nieuwenhuizen, and Niels Henze. 2018. Physical Keyboards in Virtual Reality. In Proc. 2018 CHI Conf. Hum. Factors Comput. Syst. - CHI '18. ACM Press, New York, New York, USA, 1--9.
40. Per Ola Kristensson. 2007. Discrete and continuous shape writing for text entry and control. Ph.D. Dissertation.
41. Per-Ola Kristensson and Shumin Zhai. 2004. SHARK2: A Large Vocabulary Shorthand Writing System for Pen-based Computers. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, New York, NY, USA, 43--52.
42. Frank Chun Yat Li, Richard T. Guy, Koji Yatani, and Khai N. Truong. 2011. The 1Line Keyboard: A QWERTY Layout in a Single Line. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 461--470.
43. Jia-Wei Lin, Ping-Hsuan Han, Jiun-Yu Lee, Yang-Sheng Chen, TingWei Chang, Kuan-Wen Chen, and Yi-Ping Hung. 2017. Visualizing the Keyboard in Virtual Reality for Enhancing Immersive Experience. In ACM SIGGRAPH 2017 Posters (SIGGRAPH '17). ACM, New York, NY, USA, Article 35, 2 pages.
44. Mingyu Liu, Mathieu Nancel, and Daniel Vogel. 2015. Gunslinger: Subtle Arms-down Mid-air Interaction. In Proc. 28th Annu. ACM Symp. User Interface Softw. Technol. - UIST '15. ACM Press, New York, New York, USA, 63--71.
45. Cinema2Go LTD. 2018. MOGO Travel. (2018). https://www.cinema2go. biz/mogo-travel
46. Kent Lyons, Thad Starner, Daniel Plaisted, James Fusia, Amanda Lyons, Aaron Drew, and E. W. Looney. 2004. Twiddler typing. In Proc. 2004 Conf. Hum. factors Comput. Syst. - CHI '04. ACM Press, New York, New York, USA, 671--678.
47. I Scott MacKenzie and R William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI'03 extended abstracts on Human factors in computing systems. ACM, 754--755.
48. Anders Markussen, Mikkel Rønne Jakobsen, Kasper Hornbæk, Anders Markussen, Mikkel Rønne Jakobsen, and Kasper Hornbæk. 2014. Vulture: a mid-air word-gesture keyboard. In Proc. 32nd Annu. ACM Conf. Hum. factors Comput. Syst. - CHI '14. ACM Press, New York, New York, USA, 1073--1082.
49. Mark McGill, Daniel Boland, Roderick Murray-Smith, and Stephen Brewster. 2015. A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays. In Proc. 33rd Annu. ACM Conf. Hum. Factors Comput. Syst. - CHI '15. ACM Press, New York, New York, USA, 2143--2152.
50. Aske Mottelson, Christoffer Larsen, Mikkel Lyderik, Paul Strohmeier, and Jarrod Knibbe. 2016. Invisiboard: maximizing display and input space with a full screen text entry method for smartwatches. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, 53--59.
51. Suranga Nanayakkara, Roy Shilkrot, and Pattie Maes. 2012. EyeRing: an eye on a finger. In Proc. 2012 ACM Annu. Conf. Ext. Abstr. Hum. Factors Comput. Syst. Ext. Abstr. - CHI EA '12. ACM Press, New York, New York, USA, 1961.
52. A. Nguyen and A. Banic. 2015. 3DTouch: A wearable 3D input device for 3D applications. In 2015 IEEE Virtual Reality (VR). 55--61.
53. Tao Ni, Doug Bowman, and Chris North. 2011. AirStroke: bringing unistroke text entry to freehand gesture interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2473--2476.
54. Shahriar Nirjon, Jeremy Gummeson, Dan Gelb, and Kyu-Han Kim. 2015. TypingRing: : A Wearable Ring Platform for Text Input. In Proc. 13th Annu. Int. Conf. Mob. Syst. Appl. Serv. - MobiSys '15. ACM Press, New York, New York, USA, 227--239.
55. Masa Ogata, Yuta Sugiura, Hirotaka Osawa, and Michita Imai. 2012. iRing: : intelligent ring using infrared reflection. In Proc. 25th Annu. ACM Symp. User interface Softw. Technol. - UIST '12. ACM Press, New York, New York, USA, 131.
56. Masa Ogata, Yuta Sugiura, Hirotaka Osawa, and Michita Imai. 2012. iRing: Intelligent Ring Using Infrared Reflection. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, New York, NY, USA, 131--136.
57. Kurt Partridge, Saurav Chatterjee, Vibha Sazawal, Gaetano Borriello, and Roy Want. 2002. TiltType: accelerometer-supported text entry for very small devices. In Proc. 15th Annu. ACM Symp. User interface Softw. Technol. - UIST '02. ACM Press, New York, New York, USA, 201.
58. Mahfuz Rahman, Sean Gustafson, Pourang Irani, and Sriram Subramanian. 2009. Tilt techniques: Investigating the Dexterity of Wrist-based Input. In Proc. 27th Int. Conf. Hum. factors Comput. Syst. - CHI 09. ACM Press, New York, New York, USA, 1943.
59. Shyam Reyal, Shumin Zhai, and Per Ola Kristensson. 2015. Performance and User Experience of Touchscreen and Gesture Keyboards in a Lab Setting and in the Wild. In Proc. 33rd Annu. ACM Conf. Hum. Factors Comput. Syst. - CHI '15. ACM Press, New York, New York, USA, 679--688.
60. Vibha Sazawal, Roy Want, and Gaetano Borriello. 2002. The unigesture approach one-handed text entry for small devices. In International Conference on Mobile Human-Computer Interaction. Springer, 256--270.
61. Alexander Schick, Daniel Morlock, Christoph Amma, Tanja Schultz, and Rainer Stiefelhagen. 2012. Vision-based handwriting recognition for unrestricted text input in mid-air. In Proceedings of the 14th ACM international conference on Multimodal interaction. ACM, 217--220.
62. Shaishav Siddhpuria, Keiko Katsuragawa, James R. Wallace, and Edward Lank. 2017. Exploring At-Your-Side Gestural Interaction for Ubiquitous Environments. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17). ACM, New York, NY, USA, 1111--1122.
63. Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-based Text Entry in Virtual Reality. In Proc. 2018 CHI Conf. Hum. Factors Comput. Syst. - CHI '18. ACM Press, New York, New York, USA, 1--13.
64. Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-based Text Entry in Virtual Reality. In Proc. 2018 CHI Conf. Hum. Factors Comput. Syst. - CHI '18. ACM Press, New York, New York, USA, 1--13.
65. Tap Systems. 2018. Tap - Wearable Keyboard, Mouse & Controller. (2018). https://www.tapwithus.com/
66. Robert J Teather and I Scott MacKenzie. 2014. Position vs. velocity control for tilt-based interaction. In Proceedings of Graphics Interface 2014. Canadian Information Processing Society, 51--58.
67. Hsin-Ruey Tsai, Min-Chieh Hsiu, Jui-Chun Hsiao, Lee-Ting Huang, Mike Chen, and Yi-Ping Hung. 2016. TouchRing: subtle and alwaysavailable input using a multi-touch ring. In Proc. 18th Int. Conf. HumanComputer Interact. with Mob. Devices Serv. Adjun. - MobileHCI '16. ACM Press, New York, New York, USA, 891--898.
68. Hsin-Ruey Tsai, Min-Chieh Hsiu, Jui-Chun Hsiao, Lee-Ting Huang, Mike Chen, and Yi-Ping Hung. 2016. TouchRing: Subtle and Alwaysavailable Input Using a Multi-touch Ring. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). ACM, New York, NY, USA, 891--898.
69. Koji Tsukada and Michiaki Yasumura. 2001. Ubi-finger: Gesture input device for mobile use. In Ubicomp 2001 Informal Companion Proceedings. 11.
70. James Walker, Bochao Li, Keith Vertanen, and Scott Kuhl. 2017. Efficient Typing on a Visually Occluded Physical Keyboard. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5457--5461.
71. William S. Walmsley, W. Xavier Snelgrove, and Khai N. Truong. 2014. Disambiguation of imprecise input with one-dimensional rotational text entry. ACM Trans. Comput. Interact. 21, 1 (feb 2014), 1--40.
72. Guan Wang and Ayoung Suh. 2018. Disorder or Driver?. In Proc. 2018 CHI Conf. Hum. Factors Comput. Syst. - CHI '18. ACM Press, New York, New York, USA, 1--12.
73. Eric Whitmire, Mohit Jain, Divye Jain, Greg Nelson, Ravi Karkar, Shwetak Patel, and Mayank Goel. 2017. DigiTouch. Proc. ACM Interactive, Mobile, Wearable Ubiquitous Technol. 1, 3 (sep 2017), 1--21.
74. Daniel Wigdor and Ravin Balakrishnan. 2003. TiltText: using tilt for text input to mobile phones. In Proc. 16th Annu. ACM Symp. User interface Softw. Technol. - UIST '03. ACM Press, New York, New York, USA, 81--90.
75. John Williamson and Roderick Murray-Smith. 2005. Hex: Dynamics and probabilistic text entry. In Switching and Learning in Feedback Systems. Springer, 333--342.
76. Pui Chung Wong, Kening Zhu, and Hongbo Fu. 2018. FingerT9: Leveraging Thumb-to-finger Interaction for Same-side-hand Text Entry on Smartwatches. In Proc. 2018 CHI Conf. Hum. Factors Comput. Syst. - CHI '18. ACM Press, New York, New York, USA, 1--10.
77. Xing-Dong Yang, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice. 2012. Magic Finger: Always-available Input Through Finger Instrumentation. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, New York, NY, USA, 147--156.
78. Hui-Shyong Yeo, Xiao-Shen Phang, Steven J. Castellucci, Per Ola Kristensson, and Aaron Quigley. 2017. Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices. In Proc. 2017 CHI Conf. Hum. Factors Comput. Syst. - CHI '17. ACM Press, New York, New York, USA, 4194--4202.
79. Xin Yi, Chun Yu, Weijie Xu, Xiaojun Bi, and Yuanchun Shi. 2017. COMPASS: Rotational Keyboard on Non-Touch Smartwatches. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 705--715.
80. Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. Atk: Enabling ten-finger freehand typing in air based on 3d hand tracking data. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, 539--548.
81. Chun Yu, Yizheng Gu, Zhican Yang, Xin Yi, Hengliang Luo, and Yuanchun Shi. 2017. Tap, Dwell or Gesture?: Exploring Head-Based Text Entry Techniques for HMDs. In Proc. 2017 CHI Conf. Hum. Factors Comput. Syst. - CHI '17. ACM Press, New York, New York, USA, 4479--4488.
82. Chun Yu, Ke Sun, Mingyuan Zhong, Xincheng Li, Peijun Zhao, and Yuanchun Shi. 2016. One-Dimensional Handwriting. In Proc. 2016 CHI Conf. Hum. Factors Comput. Syst. - CHI '16. ACM Press, New York, New York, USA, 71--82.
83. Shumin Zhai and Per-Ola Kristensson. 2003. Shorthand Writing on Stylus Keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 97--104.
84. Yang Zhang and Chris Harrison. 2015. Tomo: Wearable, low-cost electrical impedance tomography for hand gesture recognition. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, 167--173.
85. Suwen Zhu, Tianyao Luo, Xiaojun Bi, and Shumin Zhai. 2018. Typing on an Invisible Keyboard. In Proc. 2018 CHI Conf. Hum. Factors Comput. Syst. - CHI '18. ACM Press, New York, New York, USA, 1--13.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)