Real-time gesture recognition for the high-level teleoperation interface of a mobile manipulator

ABSTRACT

This paper describes an inertial motion capture based arm gesture recognition system for the high-level control of a mobile manipulator. Left arm kinematic data of the user is acquired by an inertial motion capture system (Xsens MVN) in real-time and processed to extract supervisory user interface commands such as "Manipulator On/Off", "Base On/Off" and "Operation Pause/Resume" for a mobile manipulator system (KUKA youBot). Principal Component Analysis and Linear Discriminant Analysis are employed for dimension reduction and classification of the user kinematic data, respectively. The classification accuracy for the six class gesture recognition problem is 95.6 percent. In order to increase the reliability of the gesture recognition framework in real-time operation, a consensus voting scheme involving the last ten classification results is implemented. During the five-minute long teleoperation experiment, a total of 25 high-level commands were recognized correctly by the consensus voting enhanced gesture recognizer. The experimental subject stated that the user interface was easy to learn and did not require extensive mental effort to operate.

References

1. Hasanuzzaman, M., Ampornararamveth, V., Zhang, T., Bhuiyan, M. A., Shirai, Y., and Ueno, H., 2004. Real-time vision-based gesture recognition for human robot interaction. In IEEE ROBIO 2004: Proceedings of the IEEE International Conference on Robotics and Biomimetics, Shenyang, 413--418. DOI= http://dx.doi.org/10.1109/ROBIO.2004.1521814.
2. KUKA, KUKA youBot omni-directional mobile platform with arm. Retrieved December 2, 2013, from: http://youbot-store.com/youbot-store/youbots
3. Liang, R. H. and Ouhyoung, M., 1998. A real-time continuous gesture recognition system for sign language. In Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition Nara, 558--567. DOI= http://dx.doi.org/10.1109/AFGR.1998.671007.
4. Malima, A., Ozgur, E., and Cetin, M., 2006. A fast algorithm for vision-based hand gesture recognition for robot control. In 2006 IEEE 14th Signal Processing and Communications Applications, vol. 1-2, Antalya, 762--765. DOI= http://dx.doi.org/10.1109/SIU.2006.1659822.
5. Mitra, S. and Acharya, T., 2007. Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 37, 3 (May), 311--324. DOI= http://dx.doi.org/10.1109/TSMCC.2007.893280.
6. Qian, G., Guo, F., Ingalls, T., Olson, L., James, J., and Rikakis, T., 2004. A gesture-driven multimodal interactive dance system. In 2004 IEEE International Conference on Multimedia and Exp (ICME), vol. 1-3, Taipei, 1579--1582. DOI= http://dx.doi.org/10.1109/ICME.2004.1394550.
7. Ramey, A., Gonzalez-Pacheco, V., and Salichs, M. A., 2011. Integration of a low-cost RGB-D sensor in a social robot for gesture recognition. In 2011 6th ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), Lausanne, 229--230.
8. XSENS, Xsens MVN - Inertial Motion Capture. Retrieved Dec. 2, 2013, from: http://www.xsens.com/en/general/mvn

Index Terms

• Computer systems organization

  • Embedded and cyber-physical systems

    • Robotics

      • External interfaces for robotics

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms