Libin Liu
Michiel Van De Panne
Kangkang Yin
Skip Abstract Section
Skip Supplemental Material SectionAbstract
The difficulty of developing control strategies has been a primary bottleneck in the adoption of physics-based simulations of human motion. We present a method for learning robust feedback strategies around given motion capture clips as well as the transition paths between clips. The output is a control graph that supports real-time physics-based simulation of multiple characters, each capable of a diverse range of robust movement skills, such as walking, running, sharp turns, cartwheels, spin-kicks, and flips. The control fragments that compose the control graph are developed using guided learning. This leverages the results of open-loop sampling-based reconstruction in order to produce state-action pairs that are then transformed into a linear feedback policy for each control fragment using linear regression. Our synthesis framework allows for the development of robust controllers with a minimal amount of prior knowledge.
Supplemental Material
Available for Download
zip
liu.zip (117.1 MB)
Supplemental movie, appendix, image and software files for, Guided Learning of Control Graphs for Physics-Based Characters
- Mazen Al Borno, Martin de Lasa, and Aaron Hertzmann. 2013. Trajectory optimization for full-body movements with complex contacts. TVCG 19, 8 (2013), 1405--1414. Google Scholar
Digital Library
- Mazen Al Borno, Eugene Fiume, Aaron Hertzmann, and Martin de Lasa. 2014. Feedback control for rotational movements in feature space. Comput. Graph. Forum 33, 2 (2014), 225--233. Google ScholarDigital Library
- Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2009. Robust task-based control policies for physics-based characters. ACM Trans. Graph. 28, 5, Article 170 (Dec. 2009), 9 pages. DOI:http://dx.doi.org/ 10.1145/1618452.1618516 Google ScholarDigital Library
- Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2010. Generalized biped walking control. ACM Trans. Graph. 29, 4, Article 130 (July 2010), 9 pages. Google ScholarDigital Library
- Marco Da Silva, Yeuhi Abe, and J. Popović. 2008. Simulation of human motion data using short-horizon model-predictive control. In Comput. Graph. Forum, Vol. 27. Wiley Online Library, 371--380.Google Scholar
- Marco da Silva, Frédo Durand, and Jovan Popović. 2009. Linear Bellman combination for control of character animation. ACM Trans. Graph. 28, 3, Article 82 (July 2009), 10 pages. DOI:http://dx.doi.org/10.1145/ 1531326.1531388 Google ScholarDigital Library
- Martin de Lasa, Igor Mordatch, and Aaron Hertzmann. 2010. Feature-based locomotion controllers. ACM Trans. Graph. 29, 4, Article 131 (July 2010), 10 pages. DOI:http://dx.doi.org/ 10.1145/1778765.1781157 Google ScholarDigital Library
- Kai Ding, Libin Liu, Michiel van de Panne, and KangKang Yin. 2015. Learning reduced-order feedback policies for motion skills. In Proceedings of the 14th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’15). ACM, New York, NY, 83--92. DOI:http://dx.doi.org/ 10.1145/2786784.2786802 Google ScholarDigital Library
- Arnaud Doucet and Adam M. Johansen. 2011. A tutorial on particle filtering and smoothing: Fifteen years later. In Handbook of Nonlinear Filtering. Oxford, UK: Oxford University Press.Google Scholar
- Petros Faloutsos, Michiel van de Panne, and Demetri Terzopoulos. 2001. Composable controllers for physics-based character animation. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'01). ACM, New York, NY, USA, 251--260. DOI:http://dx.doi.org/10.1145/383259.383287 Google ScholarDigital Library
- Thomas Geijtenbeek and Nicolas Pronost. 2012. Interactive character animation using simulated physics: A state-of-the-art review. In Comput. Graph. Forum, Vol. 31. Wiley Online Library, 2492--2515. Google ScholarDigital Library
- Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible muscle-based locomotion for bipedal creatures. ACM Trans. Graph. 32, 6 (2013), 206. Google ScholarDigital Library
- Sehoon Ha, Yuting Ye, and C. Karen Liu. 2012. Falling and landing motion control for character animation. ACM Trans. Graph. 31, 6, Article 155 (Nov. 2012), 9 pages. Google ScholarDigital Library
- Hirotaka Hachiya, Jan Peters, and Masashi Sugiyama. 2009. Efficient sample reuse in EM-based policy search. In Machine Learning and Knowledge Discovery in Databases (Lecture Notes in Computer Science), Vol. 5781. Springer, Berlin, 469--484.Google Scholar
- Jessica K. Hodgins, Wayne L. Wooten, David C. Brogan, and James F. O'Brien. 1995. Animating human athletics. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'95), Susan G. Mair and Robert Cook (Eds.). ACM, New York, NY, USA, 71--78. DOI:http://dx.doi.org/10.1145/218380.218414 Google ScholarDigital Library
- Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion graphs. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’02). ACM, New York, NY, 473--482. Google ScholarDigital Library
- Taesoo Kwon and Jessica Hodgins. 2010. Control systems for human running using an inverted pendulum model and a reference motion capture sequence. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'10). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 129--138. Google ScholarDigital Library
- Jehee Lee and Kang Hoon Lee. 2006. Precomputing avatar behavior from human motion data. Graph. Models 68, 2 (2006), 158--174. Google ScholarDigital Library
- Yoonsang Lee, Sungeun Kim, and Jehee Lee. 2010a. Data-driven biped control. ACM Trans. Graph. 29, 4, Article 129 (July 2010), 8 pages. Google ScholarDigital Library
- Yongjoon Lee, Kevin Wampler, Gilbert Bernstein, Jovan Popović, and Zoran Popović. 2010b. Motion fields for interactive character locomotion. ACM Trans. Graph. 29, 6, Article 138 (Dec. 2010), 8 pages. DOI:http://dx.doi. org/10.1145/1882261.1866160 Google ScholarDigital Library
- Sergey Levine and Vladlen Koltun. 2013. Guided policy search. In Proceedings of the 30th International Conference on Machine Learning (ICML’13).Google ScholarDigital Library
- Sergey Levine and Vladlen Koltun. 2014. Learning complex neural network policies with trajectory optimization. In Proceedings of the 31st International Conference on Machine Learning (ICML’14).Google ScholarDigital Library
- Libin Liu, KangKang Yin, and Baining Guo. 2015. Improving sampling-based motion control. Comput. Graph. Forum 34, 2 (May 2015), 415--423. DOI:http://dx.doi.org/10.1111/cgf.12571 Google ScholarDigital Library
- Libin Liu, KangKang Yin, Michiel van de Panne, and Baining Guo. 2012. Terrain runner: Control, parameterization, composition, and planning for highly dynamic motions. ACM Trans. Graph. 31, 6 (2012), Article 154. Google ScholarDigital Library
- Libin Liu, KangKang Yin, Michiel van de Panne, Tianjia Shao, and Weiwei Xu. 2010. Sampling-based contact-rich motion control. ACM Trans. Graph. 29, 4 (2010), Article 128. Google ScholarDigital Library
- Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and control of skeleton-driven soft body characters. ACM Trans. Graph. 32, 6 (2013), Article 215. Google ScholarDigital Library
- Adriano Macchietto, Victor Zordan, and Christian R. Shelton. 2009. Momentum control for balance. ACM Trans. Graph. 28, 3, Article 80 (July 2009), 8 pages. DOI:http://dx.doi.org/10.1145/1531326.1531386 Google ScholarDigital Library
- Igor Mordatch, Martin de Lasa, and Aaron Hertzmann. 2010. Robust physics-based locomotion using low-dimensional planning. ACM Trans. Graph. 29, 4, Article 71 (July 2010), 8 pages. Google ScholarDigital Library
- Igor Mordatch and Emo Todorov. 2014. Combining the benefits of function approximation and trajectory optimization. In Proceedings of Robotics: Science and Systems. Berkeley, CA.Google ScholarCross Ref
- Igor Mordatch, Emanuel Todorov, and Zoran Popović. 2012. Discovery of complex behaviors through contact-invariant optimization. ACM Trans. Graph. 31, 4, Article 43 (July 2012), 8 pages. Google ScholarDigital Library
- Uldarico Muico, Yongjoon Lee, Jovan Popović, and Zoran Popović. 2009. Contact-aware nonlinear control of dynamic characters. ACM Trans. Graph. 28, 3, Article 81 (July 2009), 9 pages. DOI:http://dx.doi.org/10.1145/1531326.1531387 Google ScholarDigital Library
- Uldarico Muico, Jovan Popović, and Zoran Popović. 2011. Composite control of physically simulated characters. ACM Trans. Graph. 30, 3, Article 16 (May 2011), 11 pages. DOI:http://dx.doi.org/10.1145/ 1966394.1966395 Google ScholarDigital Library
- Xue Bin Peng, Glen Berseth, and Michiel van de Panne. 2015. Dynamic terrain traversal skills using reinforcement learning. ACM Trans. Graph. 34, 4, Article 80 (July 2015), 11 pages. DOI:http://dx.doi.org/10.1145/ 2766910 Google ScholarDigital Library
- Jan Peters and Stefan Schaal. 2007. Reinforcement learning by reward-weighted regression for operational space control. In Proceedings of the 24th International Conference on Machine Learning (ICML’07). ACM, New York, NY, 745--750. DOI:http://dx.doi.org/ 10.1145/1273496.1273590 Google ScholarDigital Library
- Jan Peters and Stefan Schaal. 2008. Reinforcement learning of motor skills with policy gradients. Neural Netw. 21, 4 (May 2008), 682--697. DOI:http://dx.doi.org/ 10.1016/j.neunet.2008.02.003 Google ScholarDigital Library
- Zoran Popović and Andrew Witkin. 1999. Physically based motion transformation. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., 11--20. Google ScholarDigital Library
- Marc H. Raibert and Jessica K. Hodgins. 1991. Animation of dynamic legged locomotion. SIGGRAPH Comput. Graph. 25, 4 (July 1991), 349--358. DOI:http://dx.doi.org/10.1145/127719.122755 Google ScholarDigital Library
- Stephane Ross, Geoffrey Gordon, and J. Andrew (Drew) Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS’11).Google Scholar
- Kwang Won Sok, Manmyung Kim, and Jehee Lee. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. 26, 3 (2007), Article 107. Google ScholarDigital Library
- Adnan Sulejmanpašić and Jovan Popović. 2005. Adaptation of performed ballistic motion. ACM Trans. Graph.24, 1 (2005), 165--179. Google ScholarDigital Library
- Jie Tan, Yuting Gu, C. Karen Liu, and Greg Turk. 2014. Learning bicycle stunts. ACM Trans. Graph. 33, 4, Article 50 (July 2014), 12 pages. Google ScholarDigital Library
- Jie Tan, C. Karen Liu, and Greg Turk. 2011. Stable proportional-derivative controllers. IEEE Comput. Graph. Appl. 31, 4 (2011), 34--44. Google ScholarDigital Library
- Yuval Tassa, Tom Erez, and Emanuel Todorov. 2012. Synthesis and stabilization of complex behaviors through online trajectory optimization. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’12). IEEE, 4906--4913.Google ScholarCross Ref
- Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-optimal character animation with continuous control. ACM Trans. Graph. 26, 3 (July 2007), Article 7. DOI:http://dx.doi.org/10.1145/1276377.1276386 Google ScholarDigital Library
- Kevin Wampler and Zoran Popović. 2009. Optimal gait and form for animal locomotion. ACM Trans. Graph. 28, 3 (2009), Article 60. Google ScholarDigital Library
- Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2009. Optimizing walking controllers. ACM Trans. Graph. 28, 5 (2009), Article 168. Google ScholarDigital Library
- Jack M. Wang, David J. Fleet, and Aaron Hertzmann. 2010. Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. 29, 4, Article 73 (July 2010), 8 pages. DOI:http://dx.doi.org/ 10.1145/1778765.1778810 Google ScholarDigital Library
- Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing locomotion controllers using biologically-based actuators and objectives. ACM Trans. Graph. 31, 4 (2012), 25. Google ScholarDigital Library
- Yuting Ye and C. Karen Liu. 2010. Optimal feedback control for character animation using an abstract model. ACM Trans. Graph. 29, 4, Article 74 (July 2010), 9 pages. DOI:http://dx.doi.org/10.1145/1778765.1778811 Google ScholarDigital Library
- KangKang Yin, Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2008. Continuation methods for adapting simulated skills. ACM Trans. Graph. 27, 3 (2008), Article 81. Google ScholarDigital Library
- KangKang Yin, Kevin Loken, and Michiel van de Panne. 2007. SIMBICON: Simple biped locomotion control. ACM Trans. Graph. 26, 3 (2007), Article 105. Google ScholarDigital Library
- Victor Zordan, David Brown, Adriano Macchietto, and KangKang Yin. 2014. Control of rotational dynamics for ground and aerial behavior. IEEE Trans. Visual. Comput. Graphics 20, 10 (Oct 2014), 1356--1366. DOI:http://dx.doi.org/ 10.1109/TVCG.2014.2330610Google ScholarCross Ref
- Victor Brian Zordan, Anna Majkowska, Bill Chiu, and Matthew Fast. 2005. Dynamic response for motion capture animation. ACM Trans. Graph. 24, 3 (July 2005), 697--701. DOI:http://dx.doi.org/10.1145/1073204.1073249 Google ScholarDigital Library
Index Terms
- Guided Learning of Control Graphs for Physics-Based Characters
Recommendations
Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning
Basketball is one of the world's most popular sports because of the agility and speed demonstrated by the players. This agility and speed makes designing controllers to realize robust control of basketball skills a challenge for physics-based character ...
Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning
Given a robust control system, physical simulation offers the potential for interactive human characters that move in realistic and responsive ways. In this article, we describe how to learn a scheduling scheme that reorders short control fragments as ...
Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning
Given a robust control system, physical simulation offers the potential for interactive human characters that move in realistic and responsive ways. In this article, we describe how to learn a scheduling scheme that reorders short control fragments as ...
Comments