Tesca Fitzgerald
ABSTRACT
We introduce "transfer by correction": a method for transferring a robot's tool-based task models to use unfamiliar tools. By having the robot receive corrections from a human teacher when repeating a known task with a new tool, it can learn the relationship between the two tools, allowing it to transfer additional tasks learned with the original tool to the new tool. The goal is to enable the robot to generalize its task knowledge to accommodate tool replacements and thus be more robust to changes in its environment. We demonstrate how the tool transform models learned from one episode of task corrections can be used to perform that task with >=85% of maximum performance in 83% of tool/task combinations. Furthermore, these transformations generalize to unseen tool/task combinations in 27.8% of our transfer evaluations, and up to 41% of transfer problems when the source and replacement tool share tooltip similarities. Overall, these results indicate that successful task adaptation for a new tool is dependent on the the tool's usage within that task, and that the transform model learned from interactive corrections can be generalized to other tasks providing a similar context for the new tool.
- Baris Akgun, Maya Cakmak, Karl Jiang, and Andrea L Thomaz. 2012. Keyframe-based learning from demonstration . International Journal of Social Robotics , Vol. 4, 4 (2012), 343--355.Google Scholar
Cross Ref
- Brenna D Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. 2009. A survey of robot learning from demonstration. Robotics and Autonomous Systems , Vol. 57, 5 (2009), 469--483. Google ScholarDigital Library
- Brenna D Argall, Eric L Sauser, and Aude G Billard. 2010. Tactile guidance for policy refinement and reuse. IEEE 9th International Conference on Development and Learning (ICDL). IEEE, 7--12.Google ScholarCross Ref
- Andrea Bajcsy, Dylan P Losey, Marcia K O'Malley, and Anca D Dragan. 2018. Learning from Physical Human Corrections, One Feature at a Time. In ACM/IEEE International Conference on Human-Robot Interaction. ACM, 141--149. Google ScholarDigital Library
- Patrick Beeson and Barrett Ames. 2015. TRAC-IK: An open-source library for improved solving of generic inverse kinematics. In IEEE-RAS International Conference on Humanoid Robots (Humanoids). IEEE, 928--935.Google ScholarDigital Library
- Solly Brown and Claude Sammut. 2012. Tool use and learning in robots. Encyclopedia of the Sciences of Learning. Springer, 3327--3330.Google Scholar
- Sonia Chernova and Andrea L Thomaz. 2014. Robot Learning from Human Teachers. Synthesis Lectures on Artificial Intelligence and Machine Learning , Vol. 8, 3 (2014), 1--121. Google ScholarDigital Library
- Yan Duan, Marcin Andrychowicz, Bradly Stadie, OpenAI Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba. 2017. One-shot imitation learning. In Advances in Neural Information Processing Systems . 1087--1098. Google ScholarDigital Library
- Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, and Silvio Savarese. 2018. Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision. In Robotics: Science and Systems. Pittsburgh, Pennsylvania.Google Scholar
- Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. 2017. One-shot visual imitation learning via meta-learning. arXiv preprint arXiv:1709.04905 (2017).Google Scholar
- Martin A Fischler and Robert C Bolles. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM , Vol. 24, 6 (1981), 381--395. Google ScholarDigital Library
- Tesca Fitzgerald, Ashok Goel, and Andrea Thomaz. 2018. Human-Guided Object Mapping for Task Transfer. ACM Trans. Hum.-Robot Interact. , Vol. 7, 2, Article 17 (Oct. 2018), bibinfonumpages24 pages. Google ScholarDigital Library
- Justin Fu, Sergey Levine, and Pieter Abbeel. 2016. One-shot learning of manipulation skills with online dynamics adaptation and neural network priors. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 4019--4026.Google ScholarDigital Library
- Pawel Gajewski, Paulo Ferreira, Georg Bartels, Chaozheng Wang, Frank Guerin, Bipin Indurkhya, Michael Beetz, and Bartlomiej Sniezynski. 2018. Adapting Everyday Manipulation Skills to Varied Scenarios. arXiv preprint arXiv:1803.02743 (2018).Google Scholar
- Heiko Hoffmann, Zhichao Chen, Darren Earl, Derek Mitchell, Behnam Salemi, and Jivko Sinapov. 2014. Adaptive robotic tool use under variable grasps. Robotics and Autonomous Systems , Vol. 62, 6 (2014), 833--846. Google ScholarDigital Library
- Charles C Kemp and Aaron Edsinger. 2006. Robot manipulation of human tools: Autonomous detection and control of task relevant features. In International Conference on Development and Learning (ICDL) , Vol. 42.Google Scholar
- Charles C Kemp, Aaron Edsinger, and Eduardo Torres-Jara. 2007. Challenges for robot manipulation in human environments {grand challenges of robotics}. IEEE Robotics & Automation Magazine , Vol. 14, 1 (2007), 20--29.Google ScholarCross Ref
- Taylor W Killian, Samuel Daulton, George Konidaris, and Finale Doshi-Velez. 2017. Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes. In Advances in Neural Information Processing Systems. 6250--6261. Google ScholarDigital Library
- F Landis Markley, Yang Cheng, John Lucas Crassidis, and Yaakov Oshman. 2007. Averaging quaternions. Journal of Guidance, Control, and Dynamics , Vol. 30, 4 (2007), 1193--1197.Google ScholarCross Ref
- Peter Pastor, Heiko Hoffmann, Tamim Asfour, and Stefan Schaal. 2009. Learning and generalization of motor skills by learning from demonstration. In IEEE International Conference on Robotics and Automation (ICRA). IEEE, 763--768. Google ScholarDigital Library
- Eric L Sauser, Brenna D Argall, Giorgio Metta, and Aude G Billard. 2012. Iterative learning of grasp adaptation through human corrections. Robotics and Autonomous Systems , Vol. 60, 1 (2012), 55--71. Google ScholarDigital Library
- Stefan Schaal. 2006. Dynamic movement primitives-a framework for motor control in humans and humanoid robotics. Adaptive Motion of Animals and Machines. Springer, 261--280.Google Scholar
- Jivko Sinapov and Alexadner Stoytchev. 2008. Detecting the functional similarities between tools using a hierarchical representation of outcomes. In IEEE International Conference on Development and Learning (ICDL). IEEE, 91--96.Google ScholarCross Ref
- Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, and Chelsea Finn. 2018. Universal Planning Networks. arXiv preprint arXiv:1804.00645 (2018).Google Scholar
- Matthew E Taylor and Peter Stone. 2009. Transfer learning for reinforcement learning domains: A survey. The Journal of Machine Learning Research , Vol. 10 (2009), 1633--1685. Google ScholarDigital Library
- J Traa. 2013. Least-squares Intersection of Lines. UIUC, Illinois (2013). http://cal.cs.illinois.edu/ johannes/research/LS_line_intersect.pdfGoogle Scholar
Index Terms
- Human-guided Trajectory Adaptation for Tool Transfer
Recommendations
Reinforcement learning transfer via sparse coding
AAMAS '12: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 1Although reinforcement learning (RL) has been successfully deployed in a variety of tasks, learning speed remains a fundamental problem for applying RL in complex environments. Transfer learning aims to ameliorate this shortcoming by speeding up ...
Transfer in variable-reward hierarchical reinforcement learning
Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL ...
Fuzzy transfer learning of human activities in heterogeneous feature spaces
PETRA '19: Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive EnvironmentsData-driven machine learning methods usually require large amounts of annotated data to be able to develop high performance learning systems. In practical situations, such large amounts of data are not easily obtainable. Transfer Learning evolved as one ...
Comments