Yuhang Che
Heather Culbertson
Skip Abstract Section
Abstract
In this article, we present a bi-directional communication scheme that facilitates interaction between a person and a mobile robot that follows the person. A person-following robot can assist people in many applications including load carrying, elder care, and emotional support. However, commercially available personal robot systems usually have limited sensing and actuation capabilities. They are not expected to function perfectly in complex environments, and human intervention is required when the robot fails. We propose to use a holdable mechatronic device to reduce the user’s effort in communication and enable natural interaction during the intervention. Our design of the holdable device consists of two parts: a haptic interface that displays touch cues to convey the robot’s failure status via asymmetric vibrations, and a command interface for teleoperating the robot follower with hand gestures. We experimentally evaluated the device and the communication strategy in two sets of user studies with a controlled environment and a physical robot follower. Results show that with the proposed method, users are able to perform their tasks better, respond to robot failure events faster, and adjust walking speed according to the robot’s limitations. We also demonstrate that users can successfully teleoperate the robot to avoid obstacles when navigating in challenging environments.
- Tomohiro Amemiya, Hideyuki Ando, and Taro Maeda. 2005. Phantom-DRAWN: Direction guidance using rapid and asymmetric acceleration weighted by nonlinearity of perception. In International Conference on Augmented Tele-Existence. 201--208. Google Scholar
Digital Library
- Tomohiro Amemiya, Hideyuki Ando, and Taro Maeda. 2005. Virtual force display: Direction guidance using asymmetric acceleration via periodic translational motion. In IEEE World Haptics Conference. 619--622. Google ScholarDigital Library
- Tomohiro Amemiya and Hiroaki Gomi. 2014. Distinct pseudo-attraction force sensation by a thumb-sized vibrator that oscillates asymmetrically. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. 88--95.Google ScholarCross Ref
- Edwin Babaians, Navid Khazaee Korghond, Alireza Ahmadi, Mojtaba Karimi, and Saeed Shiry Ghidary. 2015. Skeleton and visual tracking fusion for human following task of service robots. In RSI International Conference on Robotics and Mechatronics. 761--766.Google ScholarCross Ref
- Kim Baraka, Stephanie Rosenthal, and Manuela Veloso. 2016. Enhancing human understanding of a mobile robot’s state and actions using expressive lights. In IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 652--657.Google ScholarCross Ref
- Luca Consolini, Fabio Morbidi, Domenico Prattichizzo, and Mario Tosques. 2008. Leader--follower formation control of nonholonomic mobile robots with input constraints. Automatica 44, 5 (2008), 1343--1349. Google ScholarDigital Library
- Heather Culbertson, Juliette Unwin, and Katherine J. Kuchenbecker. 2014. Modeling and rendering realistic textures from unconstrained tool-surface interactions. IEEE Transactions on Haptics 7, 3 (2014), 381--393.Google ScholarCross Ref
- Heather Culbertson, Julie M. Walker, and Allison M. Okamura. 2016. Modeling and design of asymmetric vibrations to induce ungrounded pulling sensation through asymmetric skin displacement. In IEEE Haptics Symposium. 27--33.Google Scholar
- Heather Culbertson, Julie M. Walker, Michael Raitor, and Allison M. Okamura. 2017. WAVES: A wearable asymmetric vibration excitation system for presenting three-dimensional translation and rotation cues. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 4972--4982. Google ScholarDigital Library
- Meredyth Daneman and Philip M. Merikle. 1996. Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin 8 Review 3, 4 (1996), 422--433.Google Scholar
- Aveek K. Das, Rafael Fierro, R. Vijay Kumar, James P. Ostrowski, John Spletzer, and Camillo J. Taylor. 2002. A vision-based formation control framework. IEEE Transactions on Robotics and Automation 18, 5 (2002), 813--825.Google ScholarCross Ref
- D. B. Edwards, T. A. Bean, D. L. Odell, and M. J. Anderson. 2004. A leader-follower algorithm for multiple AUV formations. In IEEE/OES Autonomous Underwater Vehicles. 40--46.Google Scholar
- S. Garrido-Jurado, R. Mu noz Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez. 2014. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47, 6 (2014), 2280--2292. Google ScholarDigital Library
- Rachel Gockley, Jodi Forlizzi, and Reid Simmons. 2007. Natural person-following behavior for social robots. In ACM/IEEE International Conference on Human-Robot Interaction. 17--24. Google ScholarDigital Library
- H.-M. Gross, Ch Schroeter, Steffen Mueller, Michael Volkhardt, Erik Einhorn, Andreas Bley, Ch Martin, Tim Langner, and Matthias Merten. 2011. Progress in developing a socially assistive mobile home robot companion for the elderly with mild cognitive impairment. In IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2430--2437.Google ScholarCross Ref
- Wenjiang Guo, Wei Ni, I-Ming Chen, Zhong Qiang Ding, and Song Huat Yeo. 2009. Intuitive vibro-tactile feedback for human body movement guidance. In IEEE International Conference on Robotics and Biomimetics. 135--140. Google ScholarDigital Library
- Richard Hartley, Jochen Trumpf, Yuchao Dai, and Hongdong Li. 2013. Rotation averaging. International Journal of Computer Vision 103, 3 (2013), 267--305.Google ScholarCross Ref
- Nobutaka Hirai and Hiroshi Mizoguchi. 2003. Visual tracking of human back and shoulder for person following robot. In IEEE/ASME International Conference on Advanced Intelligent Mechatronics. 527--532.Google ScholarCross Ref
- Fumiaki Hoshino and Kazuyuki Morioka. 2011. Human following robot based on control of particle distribution with integrated range sensors. In IEEE/SICE International Symposium on System Integration. 212--217.Google ScholarCross Ref
- Lynette A. Jones and Nadine B. Sarter. 2008. Tactile displays: Guidance for their design and application. Human Factors 50, 1 (2008), 90--111.Google ScholarCross Ref
- Marin Kobilarov, Gaurav Sukhatme, Jeff Hyams, and Parag Batavia. 2006. People tracking and following with mobile robot using an omnidirectional camera and a laser. In IEEE International Conference on Robotics and Automation. 557--562.Google ScholarCross Ref
- T. John Koo and Shahram M. Shahruz. 2001. Formation of a group of unmanned aerial vehicles (UAVs). In American Control Conference. 69--74.Google Scholar
- Carlo Masone, Antonio Franchi, Heinrich H. Bülthoff, and Paolo Robuffo Giordano. 2012. Interactive planning of persistent trajectories for human-assisted navigation of mobile robots. In IEEE/RSJ International Conference on Intelligent Robots and Systems. 2641--2648.Google ScholarCross Ref
- C. Masone, P. R. Giordano, H. H. Bülthoff, and A. Franchi. 2014. Semi-autonomous trajectory generation for mobile robots with integral haptic shared control. In IEEE International Conference on Robotics and Automation. 6468--6475.Google Scholar
- Michael Matscheko, Alois Ferscha, Andreas Riener, and Manuel Lehner. 2010. Tactor placement in wrist worn wearables. In International Symposium on Wearable Computers. 1--8.Google ScholarCross Ref
- Kazuyuki Morioka, Joo-Ho Lee, and Hideki Hashimoto. 2004. Human-following mobile robot in a distributed intelligent sensor network. IEEE Transactions on Industrial Electronics 51, 1 (2004), 229--237.Google ScholarCross Ref
- Claudio Pacchierotti, Stephen Sinclair, Massimiliano Solazzi, Antonio Frisoli, Vincent Hayward, and Domenico Prattichizzo. 2017. Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives. IEEE Transactions on Haptics 10, 4 (2017), 580--600. Google ScholarDigital Library
- Jun Rekimoto. 2013. Traxion: A tactile interaction device with virtual force sensation. In Annual ACM Symposium on User Interface Software and Technology. 427--432. Google ScholarDigital Library
- Stephanie Rosenthal, Joydeep Biswas, and Manuela Veloso. 2010. An effective personal mobile robot agent through symbiotic human-robot interaction. In International Conference on Autonomous Agents and Multiagent Systems. 915--922. Google ScholarDigital Library
- Stephanie Rosenthal and Manuela Veloso. 2010. Using symbiotic relationships with humans to help robots overcome limitations. In Workshop for Collaborative Human/AI Control for Interactive Experiences.Google Scholar
- Stephanie Rosenthal, Manuela M. Veloso, and Anind K. Dey. 2011. Task behavior and interaction planning for a mobile service robot that occasionally requires help. In AAAI Conference on Automated Action Planning for Autonomous Mobile Robots. AAAI Press, 14--19. Google ScholarDigital Library
- Nicholas Roy, Gregory Baltus, Dieter Fox, Francine Gemperle, Jennifer Goetz, Tad Hirsch, Dimitris Margaritis, Michael Montemerlo, Joelle Pineau, Jamie Schulte, et al. 2000. Towards personal service robots for the elderly. In Workshop on Interactive Robots and Entertainment, Vol. 25. 184.Google Scholar
- Junji Satake and Jun Miura. 2009. Robust stereo-based person detection and tracking for a person following robot. In ICRA Workshop on People Detection and Tracking. 1--10.Google Scholar
- Stefano Scheggi, Marco Aggravi, and Domenico Prattichizzo. 2017. Cooperative navigation for mixed human-robot teams using haptic feedback. IEEE Transactions on Human-Machine Systems 47, 4 (2017), 462--473.Google ScholarCross Ref
- Stefano Scheggi, Francesco Chinello, and Domenico Prattichizzo. 2012. Vibrotactile haptic feedback for human-robot interaction in leader-follower tasks. In International Conference on Pervasive Technologies Related to Assistive Environments. 51:1--51:4. Google ScholarDigital Library
- Stefano Scheggi, Fabio Morbidi, and Domenico Prattichizzo. 2014. Human-robot formation control via visual and vibrotactile haptic feedback. IEEE Transactions on Haptics 7, 4 (2014), 499--511.Google ScholarCross Ref
- Fabrizio Sergi, Dino Accoto, Domenico Campolo, and Eugenio Guglielmelli. 2008. Forearm orientation guidance with a vibrotactile feedback bracelet: On the directionality of tactile motor communication. In IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics. 433--438.Google ScholarCross Ref
- Takuya Shima and Kenjiro Takemura. 2012. An ungrounded pulling force feedback device using periodical vibration-impact. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. 481--492. Google ScholarDigital Library
- Dominik Sieber, Selma Music, and Sandra Hirche. 2015. Multi-robot manipulation controlled by a human with haptic feedback. In IEEE/RSJ International Conference on Intelligent Robots and Systems. 2440--2446.Google ScholarCross Ref
- Aaron St Clair and Maja Mataric. 2015. How robot verbal feedback can improve team performance in human-robot task collaborations. In ACM/IEEE International Conference on Human--Robot Interaction. ACM, 213--220. Google ScholarDigital Library
- Atsushi Tani, Gen Endo, Edwardo F. Fukushima, Shigeo Hirose, Masatsugu Iribe, and Toshio Takubo. 2011. Study on a practical robotic follower to support home oxygen therapy patients-development and control of a mobile platform. In IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2423--2429.Google ScholarCross Ref
- Hanns W. Tappeiner, Roberta L. Klatzky, Bert Unger, and Ralphs Hollis. 2009. Good vibrations: Asymmetric vibrations for directional haptic cues. In World Haptics Conference. 285--289. Google ScholarDigital Library
- Sebastian Thrun. 2004. Toward a framework for human-robot interaction. Human--Computer Interaction 19, 1--2 (2004), 9--24. Google ScholarDigital Library
- Parris Wellman and Robert D. Howe. 1995. Towards realistic vibrotactile display in virtual environments. ASME Dynamic Systems and Control Division 57, 2 (1995), 713--718.Google Scholar
- Takashi Yoshimi, Manabu Nishiyama, Takafumi Sonoura, Hideichi Nakamoto, Seiji Tokura, Hirokazu Sato, Fumio Ozaki, Nobuto Matsuhira, and Hiroshi Mizoguchi. 2006. Development of a person following robot with vision based target detection. In IEEE/RSJ International Conference on Intelligent Robots and Systems. 5286--5291.Google ScholarCross Ref
- Woo-han Yun, Dohyung Kim, and Jaeyeon Lee. 2010. Person following with obstacle avoidance based on multi-layered mean shift and force field method. In IEEE International Conference on Systems Man and Cybernetics. 3813--3816.Google Scholar
Index Terms
- Facilitating Human-Mobile Robot Communication via Haptic Feedback and Gesture Teleoperation
Recommendations
A mobile robot with autonomous climbing and descending of stairs
Mobile robots are used to operate in urban environments, for surveillance, reconnaissance, and inspection, as well as for military operations and in hazardous environments. Some are intended for exploration of only natural terrains, but others also for ...
Biomimetic application of desert ant visual navigation for mobile robot docking with weighted landmarks
Previous work has shown that honeybees use a snapshot model to determine a local vector to find their way home. A simpler, average landmark vector model has since been proposed for biologically-inspired mobile robot homing. Previously, the authors have ...
Interactive Teleoperation Interface for Semi-autonomous Control of Robot Arms
CRV '14: Proceedings of the 2014 Canadian Conference on Computer and Robot VisionWe propose and develop an interactive semi-autonomous control of robot arms. Our system controls two interactions: (1) A user can naturally control a robot arm by a direct linkage to the arm motion from the tracked human skeleton. (2) An autonomous ...
Comments