ReType: Quick Text Editing with Keyboard and Gaze

ABSTRACT

When a user needs to reposition the cursor during text editing, this is often done using the mouse. For experienced typists especially, the switch between keyboard and mouse can slow down the keyboard editing workflow considerably. To address this we propose ReType, a new gaze-assisted positioning technique combining keyboard with gaze input based on a new 'patching' metaphor. ReType allows users to perform some common editing operations while keeping their hands on the keyboard. We present the result of two studies. A free-use study indicated that ReType enhances the user experience of text editing. ReType was liked by many participants, regardless of their typing skills. A comparative user study showed that ReType is able to match or even beat the speed of mouse-based interaction for small text edits. We conclude that the gaze-augmented user interface can make common interactions more fluent, especially for professional keyboard users.

References

1. Michael Ashmore, Andrew T Duchowski, and Garth Shoemaker. 2005. Efficient eye pointing with a fisheye lens. In Proceedings of Graphics Interface 2005. Canadian Human-Computer Communications Society, 203--210.
2. Isam Atroshi, Christina Gummesson, Ewald Ornstein, Ragnar Johnsson, and Jonas Ranstam. 2007. Carpal tunnel syndrome and keyboard use at work: A population-based study. Arthritis & Rheumatism: Official Journal of the American College of Rheumatology 56, 11 (2007), 3620--3625.
3. Renaud Blanch and Michaël Ortega. 2009. Rake cursor: improving pointing performance with concurrent input channels. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1415--1418.
4. John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability Evaluation in Industry 189, 194 (1996), 4--7.
5. Stuart K Card, William K English, and Betty J Burr. 1978. Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a CRT. Ergonomics 21, 8 (1978), 601--613.
6. Suparna Damany and Jack Bellis. 2000. It's not carpal tunnel syndrome!: RSI theory and therapy for computer professionals. Jack Bellis.
7. Franck Dernoncourt. 2014. Replacing the computer mouse. arXiv preprint arXiv:1410.5907 (2014).
8. David Fono and Roel Vertegaal. 2005. EyeWindows: evaluation of eye-controlled zooming windows for focus selection. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 151--160.
9. John Paulin Hansen, Anders Sewerin Johansen, Dan Witzner Hansen, Kenji Itoh, and Satoru Mashino. 2003. Command without a click: Dwell time typing by mouse and gaze selections. In Proceedings of Human-Computer Interaction--INTERACT. 121--128.
10. John Paulin Hansen, Kristian Tørning, Anders Sewerin Johansen, Kenji Itoh, and Hirotaka Aoki. 2004. Gaze typing compared with input by head and hand. In Proceedings of the 2004 Symposium on Eye Tracking Research & Applications. ACM, 131--138.
11. Robert Jacob and Sophie Stellmach. 2016. What you look at is what you get: gaze-based user interfaces. Interactions 23, 5 (2016), 62--65.
12. Robert JK Jacob. 1991. The use of eye movements in human-computer interaction techniques: what you look at is what you get. ACM Transactions on Information Systems (TOIS) 9, 2 (1991), 152--169.
13. Robert J Jacob and Keith S Karn. 2003. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. Mind 2, 3 (2003), 4.
14. Shaun K Kane, Jacob O Wobbrock, Mark Harniss, and Kurt L Johnson. 2008. TrueKeys: identifying and correcting typing errors for people with motor impairments. In Proceedings of the 13th International Conference on Intelligent User Interfaces. ACM, 349--352.
15. Reo Kishi and Takahiro Hayashi. 2015. Effective gazewriting with support of text copy and paste. In IEEE/ACIS 14th International Conference on Computer and Information Science (ICIS). IEEE, 125--130.
16. Per Ola Kristensson and Keith Vertanen. 2012. The potential of dwellfree eye-typing for fast assistive gaze communication. In Proceedings of the Symposium on Eye Tracking Research & Applications. ACM, 241-- 244.
17. Manu Kumar, Andreas Paepcke, and Terry Winograd. 2007. EyeExpose: Switching Applications with Your Eyes. Technical Report CSTR 2007-02. Stanford University.
18. Manu Kumar, Andreas Paepcke, and Terry Winograd. 2007. EyePoint: practical pointing and selection using gaze and keyboard. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 421--430.
19. Manu Kumar and Terry Winograd. 2007. GUIDe: gaze-enhanced UI design. In CHI'07 Extended Abstracts on Human Factors in Computing Systems. ACM, 1977--1982.
20. Manu Kumar, Terry Winograd, Andreas Paepcke, and Jeff Klingner. 2007. Gaze-enhanced user interface design. Technical Report. Stanford ReType: Quick Text Editing with Keyboard and Gaze CHI 2019, May 4--9, 2019, Glasgow, Scotland UK InfoLab.
21. Chris Lankford. 2000. Effective eye-gaze input into windows. In Proceedings of the 2000 Symposium on Eye Tracking Research & Applications. ACM, 23--27.
22. Michael YC Lin, Justin G Young, and Jack T Dennerlein. 2015. Evaluating the effect of four different pointing device designs on upper extremity posture and muscle activity during mousing tasks. Applied Ergonomics 47 (2015), 259--264.
23. Yi Liu, Chi Zhang, Chonho Lee, Bu-Sung Lee, and Alex Qiang Chen. 2015. Gazetry: Swipe text typing using gaze. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction. ACM, 192--196.
24. Christof Lutteroth, Moiz Penkar, and Gerald Weber. 2015. Gaze vs. Mouse: a fast and accurate gaze-only click alternative. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. ACM, 385--394.
25. Päivi Majaranta, Ulla-Kaija Ahola, and Oleg Spakov. 2009. Fast gaze typing with an adjustable dwell time. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 357--360.
26. Päivi Majaranta, Niina Majaranta, Gintautas Daunys, and Oleg Spakov. 2009. Text editing by gaze: static vs. dynamic menus. In Proceedings of the 5th Conference on Communication by Gaze Interaction (COGAIN). 19--24.
27. Martez E Mott, Shane Williams, Jacob O Wobbrock, and Meredith Ringel Morris. 2017. Improving dwell-based gaze typing with dynamic, cascading dwell times. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 2558--2570.
28. Eugene W. Myers. 1986. An O(ND) Difference Algorithm and Its Variations. Algorithmica 1 (1986), 251--266.
29. Emil F Pascarelli and Deborah Quilter. 1994. Repetitive strain injury: a computer user's guide. Wiley.
30. Diogo Pedrosa, Maria Da Graça Pimentel, Amy Wright, and Khai N Truong. 2015. Filteryedping: design challenges and user performance of dwell-free eye typing. ACM Transactions on Accessible Computing (TACCESS) 6, 1 (2015), 3.
31. Abdul Moiz Penkar, Christof Lutteroth, and Gerald Weber. 2012. Designing for the eye: design parameters for dwell in gaze interaction. In Proceedings of the 24th Australian Computer-Human Interaction Conference. ACM, 479--488.
32. Abdul Moiz Penkar, Christof Lutteroth, and Gerald Weber. 2013. Eyes only: navigating hypertext with gaze. In IFIP Conference on HumanComputer Interaction (INTERACT). Springer, 153--169.
33. Kari-Jouko Räihä and Oleg Spakov. 2009. Disambiguating ninja cursors with eye gaze. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1411--1414.
34. Julian Ramos, Zhen Li, Johana Rosas, Nikola Banovic, Jennifer Mankoff, and Anind Dey. 2016. Keyboard surface interaction: making the keyboard into a pointing device. arXiv preprint arXiv:1601.04029 (2016).
35. Sayan Sarcar, Prateek Panwar, and Tuhin Chakraborty. 2013. EyeK: an efficient dwell-free eye gaze-based text entry system. In Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction. ACM, 215--220.
36. Linda E Sibert and Robert JK Jacob. 2000. Evaluation of eye gaze interaction. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 281--288.
37. Dave M Stampe and Eyal M Reingold. 1995. Selection by looking: a novel computer interface and its application to psychological research. Studies in Visual Information Processing 6 (1995), 467--478.
38. Sophie Stellmach and Raimund Dachselt. 2012. Look & touch: gazesupported target acquisition. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 2981--2990.
39. Shari Trewin. 2002. An invisible keyguard. In Proceedings of the Fifth International ACM Conference on Assistive Technologies. ACM, 143--149.
40. Outi Tuisku, Päivi Majaranta, Poika Isokoski, and Kari-Jouko Räihä. 2008. Now Dasher! Dash away!: longitudinal study of fast text entry by eye gaze. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications. ACM, 19--26.
41. Boris Velichkovsky, Andreas Sprenger, and Pieter Unema. 1997. Towards gaze-mediated interaction: collecting solutions of the "Midas touch problem". In Human-Computer Interaction (INTERACT). Springer, 509--516.
42. Brian A Wandell. 1995. Foundations of vision. Sinauer Associates.
43. David J Ward and David JC MacKay. 2002. Artificial intelligence: fast hands-free writing by gaze direction. Nature 418, 6900 (2002), 838--838.
44. Colin Ware and Harutune H Mikaelian. 1987. An evaluation of an eye tracker as a device for computer input. In ACM SIGCHI Bulletin, Vol. 17. ACM, 183--188.
45. Wayne Westerman, John G Elias, and Alan Hedge. 2001. Multi-touch: a new tactile 2-d gesture interface for human-computer interaction. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 45. SAGE Publications, 632--636.
46. Victoria Woods, Sarah Hastings, Peter Buckle, and Roger Haslam. 2002. Ergonomics of using a mouse or other non-keyboard input device. Health and Safety Executive.
47. Shumin Zhai, Carlos Morimoto, and Steven Ihde. 1999. Manual and gaze input cascaded (MAGIC) pointing. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 246--253.
48. Xinyong Zhang, Xiangshi Ren, and Hongbin Zha. 2008. Improving eye cursor's stability for eye pointing tasks. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 525--534.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Pointing devices

    • Interaction techniques

      • Text input