Enabling mobile microinteractions
- Author:
- Daniel L. Ashbrook,
- Adviser:
- Thad E. Starner
Skip Abstract Section
Abstract
While much attention has been paid to the usability of desktop computers, mobile computers are quickly becoming the dominant platform. Because mobile computers may be used in nearly any situation—including while the user is actually in motion, or performing other tasks—interfaces designed for stationary use may be inappropriate, and alternative interfaces should be considered.
In this dissertation I consider the idea of microinteractions —interactions with a device that take less than four seconds to initiate and complete. Microinteractions are desirable because they may minimize interruption; that is, they allow for a tiny burst of interaction with a device so that the user can quickly return to the task at hand.
My research concentrates on methods for applying microinteractions through wrist-based interaction. I consider two modalities for this interaction: touchscreens and motion-based gestures. In the case of touchscreens, I consider the interface implications of making touchscreen watches usable with the finger, instead of the usual stylus, and investigate users' performance with a round touchscreen. For gesture-based interaction, I present a tool, MAGIC, for designing gesture-based interactive system, and detail the evaluation of the tool.
Cited By
Li J, Xu Y, Grossman T, Santosa S and Li M OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMs Proceedings of the CHI Conference on Human Factors in Computing Systems, (1-22)- Joshi N, Abtahi P, Sodhi R, Bartov N, Rushing J, Collins C, Vogel D and Glueck M Transferable Microgestures Across Hand Posture and Location Constraints: Leveraging the Middle, Ring, and Pinky Fingers Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, (1-17)
- Azim M, Rahman A and Heo S (2023). UnifiedSense: Enabling Without-Device Gesture Interactions Using Over-the-shoulder Training Between Redundant Wearable Sensors, Proceedings of the ACM on Human-Computer Interaction, 7:MHCI, (1-25), Online publication date: 11-Sep-2023.
- Azim M, Rahman A and Heo S (2023). SequenceSense, International Journal of Human-Computer Studies, 176:C, Online publication date: 1-Aug-2023.
- Sharma A, Salchow-Hömmen C, Mollyn V, Nittala A, Hedderich M, Koelle M, Seel T and Steimle J (2022). SparseIMU: Computational Design of Sparse IMU Layouts for Sensing Fine-grained Finger Microgestures, ACM Transactions on Computer-Human Interaction, 30:3, (1-40), Online publication date: 30-Jun-2023.
- Azim M, Rahman A and Heo S Over-The-Shoulder Training Between Redundant Wearable Sensors for Unified Gesture Interactions Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, (1-3)
- Herath A, Sharma A and Nittala A Utilizing Interactive Technologies to Encourage Healthy Dietary Behavior Adjunct Publication of the 24th International Conference on Human-Computer Interaction with Mobile Devices and Services, (1-5)
- Rey B, Zhu K, Perrault S, Bardot S, Neshati A and Irani P (2022). Understanding and Adapting Bezel-to-Bezel Interactions for Circular Smartwatches in Mobile and Encumbered Scenarios, Proceedings of the ACM on Human-Computer Interaction, 6:MHCI, (1-28), Online publication date: 19-Sep-2022.
- Haas G, Rietzler M, Jones M and Rukzio E Keep it Short: A Comparison of Voice Assistants’ Response Behavior Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, (1-12)
- Chaffangeon A, Goguey A and Nigay L μGlyphe: une Notation Graphique pour Décrire les Microgestes Proceedings of the 33rd Conference on l'Interaction Humain-Machine, (1-13)
- Chen C, Johnson J, Charles K, Lee A, Lifset E, Hogarth M, Moore A, Farcas E and Weibel N Understanding Barriers and Design Opportunities to Improve Healthcare and QOL for Older Adults through Voice Assistants Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility, (1-16)
- Yan X, Li Y, Huang B, Park S and Newman M User Burden of Microinteractions: An In-lab Experiment Examining User Performance and Perceived Burden Related to In-situ Self-reporting Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction, (1-14)
- Verma D, Bhalla S, Sahnan D, Shukla J and Parnami A (2021). ExpressEar, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5:3, (1-28), Online publication date: 9-Sep-2021.
- Saidi H, Dubois E and Serrano M HoloBar: Rapid Command Execution for Head-Worn AR Exploiting Around the Field-of-View Interaction Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, (1-17)
- Safin S, Maitrallin M, Fruchard B and Lecolinet E Processus d’appropriation et de mémorisation de raccourcis gestuels sur trackpad : Etude longitudinale des stratégies et usages des utilisateurs et impact d’une aide visuo-sémantique Proceedings of the 32nd Conference on l'Interaction Homme-Machine, (1-12)
- Matthies D, Weerasinghe C, Urban B and Nanayakkara S CapGlasses: Untethered Capacitive Sensing with Smart Glasses Proceedings of the Augmented Humans International Conference 2021, (121-130)
- Yan X, Raj S, Huang B, Park S and Newman M (2020). Toward Lightweight In-situ Self-reporting, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4:4, (1-22), Online publication date: 17-Dec-2020.
- Beruscha F, Mueller K and Sohnke T Eliciting tangible and gestural user interactions with and on a cooking pan Proceedings of Mensch und Computer 2020, (399-408)
- Olwal A, Starner T and Mainini G E-Textile Microinteractions: Augmenting Twist with Flick, Slide and Grasp Gestures for Soft Electronics Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, (1-13)
- Park K, Kim D, Heo S and Lee G MagTouch: Robust Finger Identification for a Smartwatch Using a Magnet Ring and a Built-in Magnetometer Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, (1-13)
- Sharma A, Roo J and Steimle J Grasping Microgestures Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, (1-13)
- Wu J, Colglazier C, Ravishankar A, Duan Y, Wang Y, Ploetz T and Starner T Seesaw Proceedings of the 2018 ACM International Symposium on Wearable Computers, (17-20)
- Hirzle T, Rixen J, Gugenheimer J and Rukzio E WatchVR Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, (1-6)
- Park J, Sim J, Kim J, Yi M and Lee U Interaction Restraint Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, (1-6)
- Parzer P, Sharma A, Vogl A, Steimle J, Olwal A and Haller M SmartSleeve Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, (565-577)
- Han T, Hasan K, Nakamura K, Gomez R and Irani P SoundCraft Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, (579-591)
- Oh U, Stearns L, Pradhan A, Froehlich J and Findlater L Investigating Microinteractions for People with Visual Impairments and the Potential Role of On-Body Interaction Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, (22-31)
- Jeong H, Kim H, Kim R, Lee U and Jeong Y (2017). Smartwatch Wearing Behavior Analysis, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1:3, (1-31), Online publication date: 11-Sep-2017.
- Kawaguchi N, Nishio N, Roggen D, Inoue S, Pirttikangas S and Van Laerhoven K 5th Int. workshop on human activity sensing corpus and applications (HASCA) Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, (530-536)
- Dobbelstein D, Haas G and Rukzio E The effects of mobility, encumbrance, and (non-)dominant hand on interaction with smartwatches Proceedings of the 2017 ACM International Symposium on Wearable Computers, (90-93)
- Brulé E, Bailly G, Serrano M, Teyssier M and Huron S Investigating the design space of smartwatches combining physical rotary inputs Proceedings of the 29th Conference on l'Interaction Homme-Machine, (13-20)
- Vatavu R (2017). Smart-Pockets, International Journal of Human-Computer Studies, 103:C, (1-21), Online publication date: 1-Jul-2017.
- Vogl A, Parzer P, Babic T, Leong J, Olwal A and Haller M StretchEBand Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, (2617-2627)
- Matthies D, Strecker B and Urban B EarFieldSensing Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, (1911-1922)
- Saidi H, Serrano M, Irani P and Dubois E TDome Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, (5892-5904)
- Parnami A, Gupta A, Reyes G, Sadana R, Li Y and Abowd G Mogeste Proceedings of the 8th Indian Conference on Human-Computer Interaction, (35-43)
- Sharma H, Toups Dugas P, Dolgov I, Kerne A and Jain A Evaluating Display Modalities Using a Mixed Reality Game Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play, (65-77)
- Reyes G, Zhang D, Ghosh S, Shah P, Wu J, Parnami A, Bercik B, Starner T, Abowd G and Edwards W Whoosh Proceedings of the 2016 ACM International Symposium on Wearable Computers, (120-127)
- Intille S, Haynes C, Maniar D, Ponnada A and Manjourides J μEMA Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, (1124-1128)
- Parnami A, Gupta A, Reyes G, Sadana R, Li Y and Abowd G Mogeste Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, (345-348)
- Kawaguchi N, Nishio N, Roggen D, Inoue S, Pirttikangas S and van Laerhoven K 4th workshop on human activity sensing corpus and applications Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, (690-695)
- Ashbrook D, Tejada C, Mehta D, Jiminez A, Muralitharam G, Gajendra S and Tallents R Bitey Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, (158-169)
- Lyons K Visual parameters impacting reaction times on smartwatches Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, (190-194)
- Kratz S and Wiese J GestureSeg Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, (61-72)
- Cannanure V, Chen X and Mankoff J Twist 'n' Knock Proceedings of the 42nd Graphics Interface Conference, (189-193)
- Jalaliniya S and Mardanbegi D EyeGrip Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, (5801-5811)
- Lafreniere B, Gutwin C, Cockburn A and Grossman T Faster Command Selection on Touchscreen Watches Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, (4663-4674)
- Dobbelstein D Unobtrusive Interaction for Wearable Computing Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, (203-207)
- Lopes P Proprioceptive Interaction Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, (223-228)
- Lyons K (2016). Smartwatch Innovation: Exploring a Watch-First Model, IEEE Pervasive Computing, 15:1, (10-13), Online publication date: 1-Jan-2016.
- Matthies D, Perrault S, Urban B and Zhao S Botential Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, (207-216)
- Dicke C and Müller J Evaluating Mid-air List Interaction for Spatial Audio Interfaces Proceedings of the 3rd ACM Symposium on Spatial User Interaction, (24-33)
- Kratz S and Back M Towards Accurate Automatic Segmentation of IMU-Tracked Motion Gestures Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, (1337-1342)
- Lopes P, Ion A, Mueller W, Hoffmann D, Jonell P and Baudisch P Proprioceptive Interaction Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, (939-948)
- Dobbelstein D, Hock P and Rukzio E Belt Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, (2135-2138)
- Vo D, Lecolinet E and Guiard Y Belly gestures Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, (687-696)
- Lyons K, Kim S, Seko S, Nguyen D, Desjardins A, Vidal M, Dobbelstein D and Rubin J Loupe Proceedings of the 27th annual ACM symposium on User interface software and technology, (351-354)
- Yoon S, Huo K and Ramani K Plex Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, (191-194)
- Profita H, Clawson J, Gilliland S, Zeagler C, Starner T, Budd J and Do E Don't mind me touching my wrist Proceedings of the 2013 International Symposium on Wearable Computers, (89-96)
- Golod I, Heidrich F, Möllering C and Ziefle M Design principles of hand gesture interfaces for microinteractions Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces, (11-20)
- Serrano M, Lecolinet E and Guiard Y Bezel-Tap gestures Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (3027-3036)
- Perrault S, Lecolinet E, Eagan J and Guiard Y Watchit Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (1451-1460)
- Gustafson S, Holz C and Baudisch P Imaginary phone Proceedings of the 24th annual ACM symposium on User interface software and technology, (283-292)
- Wolf K Microinteractions to augment manual tasks Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part IV, (414-417)
- Wolf K, Naumann A, Rohs M and Müller J Taxonomy of microinteractions Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part I, (559-575)
- Wolf K Microinteractions for supporting grasp tasks through usage of spare attentional and motor resources Proceedings of the 29th Annual European Conference on Cognitive Ergonomics, (221-224)
- Ashbrook D, Baudisch P and White S Nenya Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (2043-2046)
- Gustafson S, Bierwirth D and Baudisch P Imaginary interfaces Proceedings of the 23nd annual ACM symposium on User interface software and technology, (3-12)
Recommendations
Microinteractions to augment manual tasks
INTERACT'11: Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part IVThis paper summarizes the first nine months of progress on my Ph.D. project. The research focus of the project is on investigating microinteractions. a sub-topic of HCI and gesture research. The project will develop a framework for applications that use ...
Enabling tangible interaction on capacitive touch panels
UIST '10: Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technologyWe propose two approaches to sense tangible objects on capacitive touch screens, which are used in off-the-shelf multi-touch devices such as Apple iPad, iPhone, and 3M's multi-touch displays. We seek for the approaches that do not require modifications ...
PenShaft: Enabling Pen Shaft Detection and Interaction for Touchscreens
AH2021: 12th Augmented Human International ConferencePenShaft is a battery-free, easy to implement solution for augmenting the shaft of a capacitive pen with interactive capabilities. By applying conductive materials in a specific pattern on the pen’s shaft, we can detect when it is laid on a capacitive ...