Petr Kadleček
Ladislav Kavan
Skip Abstract Section
Skip Supplemental Material SectionAbstract
The human face is an anatomical system exhibiting heterogenous and anisotropic mechanical behavior. This leads to complex deformations even in a neutral facial expression due to external forces such as gravity. We start by building a volumetric model from magnetic resonance images of a neutral facial expression. To obtain data on facial deformations we capture and register 3D scans of the face with different gravity directions and with various facial expressions. Our main contribution consists in solving an inverse physics problem where we learn mechanical properties of the face from our training data (3D scans). Specifically, we learn heterogenous stiffness and prestrain (which introduces anisotropy). The generalization capability of our resulting physics-based model is tested on 3D scans. We demonstrate that our model generates predictions of facial deformations more accurately than recent related physics-based techniques.
Supplemental Material
Available for Download
zip
kadlecek.zip (77.6 MB)
Supplemental movie, appendix, image and software files for, Building Accurate Physics-based Face Models from Data
- Jascha Achenbach, Robert Brylka, Thomas Gietzen, Katja zum Hebel, Elmar Schömer, Ralf Schulze, Mario Botsch, and Ulrich Schwanecke. 2018. A multilinear model for bidirectional craniofacial reconstruction. In Proceedings of the Eurographics Workshop on Visual Computing for Biology and Medicine. Eurographics Association, 67--76. Google Scholar
Digital Library
- Oleg Alexander, Mike Rogers, William Lambeth, Jen-Yuan Chiang, Wan-Chun Ma, Chuan-Chang Wang, and Paul Debevec. 2010. The digital Emily project: Achieving a photorealistic digital actor. Computer Graphics and Applications, IEEE (2010). Google ScholarDigital Library
- Michael Bao, Matthew Cong, Stéphane Grabli, and Ronald Fedkiw. 2018. High-Quality Face Capture Using Anatomical Muscles. CoRR abs/1812.02836 (2018). arXiv:1812.02836 http://arxiv.org/abs/1812.02836Google Scholar
- GG Barbarino, M Jabareen, J Trzewik, A Nkengne, G Stamatas, and E Mazza. 2009. Development and validation of a three-dimensional finite element model of the face. Journal of biomechanical engineering 131, 4 (2009), 041006.Google ScholarCross Ref
- Vincent Barrielle, Nicolas Stoiber, and Cedric Cagniart. 2016. Blendforces, a Dynamic Framework for Facial Animation. Comp. Graph. Forum (2016). Google ScholarDigital Library
- Thabo Beeler, Bernd Bickel, Paul Beardsley, Bob Sumner, and Markus Gross. 2010. High-quality single-shot capture of facial geometry. In ACM Transactions on Graphics (ToG), Vol. 29. ACM, 40. Google ScholarDigital Library
- Thabo Beeler and Derek Bradley. 2014. Rigid stabilization of facial expressions. ACM Trans. Graph. 33, 4 (2014), 44. Google ScholarDigital Library
- Thabo Beeler, Fabian Hahn, Derek Bradley, Bernd Bickel, Paul Beardsley, Craig Gotsman, Robert W Sumner, and Markus Gross. 2011. High-quality passive facial performance capture using anchor frames. In ACM Trans. on Graphics (TOG). Google ScholarDigital Library
- Bernd Bickel, Moritz Bächer, Miguel A Otaduy, Wojciech Matusik, Hanspeter Pfister, and Markus Gross. 2009. Capture and modeling of nonlinear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (2009). Google ScholarDigital Library
- Bernd Bickel, Peter Kaufmann, Mélina Skouras, Bernhard Thomaszewski, Derek Bradley, Thabo the, Phil Jackson, Steve Marschner, Wojciech Matusik, and Markus Gross. 2012. Physical face cloning. ACM Trans. Graph. 31, 4 (2012), 118. Google ScholarDigital Library
- Volker Blanz and Thomas Vetter. 1999. A morphable model for the synthesis of 3D faces. In Proc. of the 26th annual conf. on Comp. graph. and interactive techniques. 187--194. Google ScholarDigital Library
- Silvia S Blemker, Peter M Pinsky, and Scott L Delp. 2005. A 3D model of muscle reveals the causes of nonuniform strains in the biceps brachii. Journal of biomechanics 38, 4 (2005), 657--665.Google ScholarCross Ref
- Xiang Chen, Changxi Zheng, Weiwei Xu, and Kun Zhou. 2014. An asymptotic numerical method for inverse elastic shape design. ACM Trans. Graph. 33, 4 (2014), 95. Google ScholarDigital Library
- Matthew Cong, Michael Bao, Kiran S Bhat, Ronald Fedkiw, et al. 2015. Fully automatic generation of anatomical face simulation models. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. ACM, 175--183. Google ScholarDigital Library
- Matthew Cong, Kiran S Bhat, and Ronald Fedkiw. 2016. Art-directed muscle simulation for high-end facial animation. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. 119--127. Google ScholarDigital Library
- Ali-Hamadi Dicko, Tiantian Liu, Benjamin Gilles, Ladislav Kavan, François Faure, Olivier Palombi, and Marie-Paule Cani. 2013. Anatomy transfer. ACM Trans. Graph. 32, 6 (2013), 188. Google ScholarDigital Library
- Paul Ekman and Wallace V Friesen. 1977. Facial action coding system. (1977).Google Scholar
- Benjamin J Ellis, Trevor J Lujan, Michelle S Dalton, and Jeffrey A Weiss. 2006. Medial collateral ligament insertion site and contact forces in the ACL-deficient knee. Journal of Orthopaedic Research 24, 4 (2006), 800--810.Google ScholarCross Ref
- Ye Fan, Joshua Litven, David IW Levin, and Dinesh K Pai. 2013. Eulerian-on-lagrangian simulation. ACM Transactions on Graphics (TOG) 32, 3 (2013), 22. Google ScholarDigital Library
- Ye Fan, Joshua Litven, and Dinesh K Pai. 2014. Active volumetric musculoskeletal systems. ACM Trans. Graph. (2014). Google ScholarDigital Library
- François Faure, Christian Duriez, Hervé Delingette, Jérémie Allard, Benjamin Gilles, Stéphanie Marchesseau, Hugo Talbot, Hadrien Courtecuisse, Guillaume Bousquet, Igor Peterlik, et al. 2012. Sofa: A multi-model framework for interactive physical simulation. In Soft Tissue Biomechanical Modeling for Computer Assisted Surgery. Springer, 283--321.Google Scholar
- Andriy Fedorov, Reinhard Beichel, Jayashree Kalpathy-Cramer, Julien Finet, Jean-Christophe Fillion-Robin, Sonia Pujol, Christian Bauer, Dominique Jennings, Fiona Fennessy, Milan Sonka, et al. 2012. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magnetic resonance imaging 30, 9 (2012), 1323--1341.Google Scholar
- Babarenda Gamage, P Thiranja, Vijayaraghavan Rajagopal, Matthias Ehrgott, Martyn P Nash, and Poul MF Nielsen. 2011. Identification of mechanical properties of heterogeneous soft bodies using gravity loading. International journal for numerical methods in biomedical engineering 27, 3 (2011), 391--407.Google Scholar
- Michael W Gee, Ch Förster, and WA Wall. 2010. A computational strategy for prestressing patient-specific biomechanical problems under finite deformation. International Journal for Numerical Methods in Biomedical Engineering 26, 1 (2010), 52--72.Google ScholarCross Ref
- Lianghao Han, John H Hipwell, Christine Tanner, Zeike Taylor, Thomy Mertzanidou, Jorge Cardoso, Sebastien Ourselin, and David J Hawkes. 2011. Development of patient-specific biomechanical models for predicting large breast deformation. Physics in Medicine and Biology 57, 2 (2011), 455.Google ScholarCross Ref
- Hibbitt, Karlsson, and Sorensen. 2001. ABAQUS/standard User's Manual. Vol. 1. Hibbitt, Karlsson & Sorensen.Google Scholar
- Alexandru Ichim, Ladislav Kavan, Merlin Nimier-David, and Mark Pauly. 2016. Building and Animating User-Specific Volumetric Face Rigs. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. Google ScholarDigital Library
- Alexandru Eugen Ichim, the Bouaziz, and Mark Pauly. 2015. Dynamic 3D Avatar Creation from Hand-held Video Input. ACM Trans. Graph. (2015). Google ScholarDigital Library
- Alexandru-Eugen Ichim, Petr Kadlecek, Ladislav Kavan, and Mark Pauly. 2017. Phace: Physics-based Face Modeling and Animation. ACM Trans. Graph. 36, 4 (2017). Google ScholarDigital Library
- Petr Kadlecek, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Krivanek, and Ladislav Kavan. 2016. Reconstructing Personalized Anatomical Models for Physics-based Body Animation. ACM Trans. Graph. 35, 6 (2016). Google ScholarDigital Library
- Meekyoung Kim, Gerard Pons-Moll, Sergi Pujades, Seungbae Bang, Jinwook Kim, Michael J Black, and Sung-Hee Lee. 2017. Data-driven physics for human soft tissue animation. ACM Transactions on Graphics (TOG) 36, 4 (2017), 54. Google ScholarDigital Library
- Yeara Kozlov, Derek Bradley, Moritz Bächer, Bernhard Thomaszewski, Thabo Beeler, and Markus Gross. 2017. Enriching Facial Blendshape Rigs with Physical Simulation. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 75--84. Google ScholarDigital Library
- Lana Lan, Matthew Cong, and Ronald Fedkiw. 2017. Lessons from the evolution of an anatomical facial muscle model. In Proceedings of the ACM SIGGRAPH Digital Production Symposium. ACM, 11. Google ScholarDigital Library
- Rudy J Lapeer, Paul D Gasson, and Vasudev Karri. 2011. A hyperelastic finite-element model of human skin for interactive real-time surgical simulation. IEEE Transactions on Biomedical Engineering 58, 4 (2011), 1013--1022.Google ScholarCross Ref
- Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28, 4 (2009), 99. Google ScholarDigital Library
- David IW Levin, Benjamin Gilles, Burkhard Mädler, and Dinesh K Pai. 2011. Extracting skeletal muscle fiber fields from noisy diffusion tensor data. Medical Image Analysis 15, 3 (2011), 340--353.Google ScholarCross Ref
- John P Lewis, Ken Anjyo, Taehyun Rhee, Mengjie Zhang, Frederic H Pighin, and Zhigang Deng. 2014. Practice and Theory of Blendshape Facial Models.. In Eurographics (State of the Art Reports). 199--218.Google Scholar
- Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. 2017. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics 36, 6 (Nov. 2017), 194:1--194:17. Two first authors contributed equally. Google ScholarDigital Library
- John E Lloyd, Ian Stavness, and Sidney Fels. 2012. ArtiSynth: A fast interactive biomechanical modeling toolkit combining multi-body and finite element simulation. In Soft tissue biomechanical modeling for computer assisted surgery. Springer.Google Scholar
- Vincent Luboz, Emmanuel Promayon, and Yohan Payan. 2014. Linear elastic properties of the facial soft tissues using an aspiration device: towards patient specific characterization. Annals of biomedical engineering 42, 11 (2014), 2369--2378.Google ScholarCross Ref
- Wan-Chun Ma, Yi-Hua Wang, Graham Fyffe, Bing-Yu Chen, and Paul Debevec. 2012. A blendshape model that incorporates physical interaction. Computer Animation and Virtual Worlds 23, 3-4 (2012). Google ScholarDigital Library
- Steve A Maas, Benjamin J Ellis, Gerard A Ateshian, and Jeffrey A Weiss. 2012. FEBio: finite elements for biomechanics. Journal of biomechanical engineering 134, 1 (2012), 011005.Google ScholarCross Ref
- Steve A Maas, Ahmet Erdemir, Jason P Halloran, and Jeffrey A Weiss. 2016. A general framework for application of prestrain to computational models of biological materials. journal of the mechanical behavior of biomedical materials 61 (2016), 499--510.Google Scholar
- Wouter Mollemans, Filip Schutyser, Nasser Nadjmi, Frederik Maes, and Paul Suetens. 2007. Predicting soft tissue deformations for a maxillofacial surgery planning system: from computational strategies to a complete clinical validation. Medical image analysis 11, 3 (2007), 282--301.Google Scholar
- Mohammad Ali Nazari, Pascal Perrier, Matthieu Chabanas, and Yohan Payan. 2010. Simulation of dynamic orofacial movements using a constitutive law varying with muscle activation. Computer Methods in Biomechanics and Biomedical Engineering 13, 4 (2010), 469--482.Google ScholarCross Ref
- Yves Nubar and Renato Contini. 1961. A minimal principle in biomechanics. The bulletin of mathematical biophysics 23, 4 (1961), 377--391.Google Scholar
- U Ozsoy, R Sekerci, and E Ogut. 2015. Effect of sitting, standing, and supine body positions on facial soft tissue: Detailed 3D analysis. International journal of oral and maxillofacial surgery 44, 10 (2015), 1309--1316.Google Scholar
- Dinesh K Pai, Kees van den Doel, Doug L James, Jochen Lang, John E Lloyd, Joshua L Richmond, and Som H Yau. 2001. Scanning physical interaction behavior of 3D objects. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, 87--96. Google ScholarDigital Library
- MB Rubin and SR Bodner. 2002. A three-dimensional nonlinear model for dissipative response of soft tissue. International Journal of Solids and Structures 39, 19 (2002), 5081--5099.Google ScholarCross Ref
- Shunsuke Saito, Zi-Ye Zhou, and Ladislav Kavan. 2015. Computational Bodybuilding: Anatomically-based Modeling of Human Bodies. ACM Trans. Graph. 34, 4 (2015). Google ScholarDigital Library
- Olaf Schenk and Klaus Gärtner. 2004. Solving unsymmetric sparse systems of linear equations with PARDISO. Future Generation Computer Systems 20, 3 (2004), 475--487. Google ScholarDigital Library
- Minoru Shinohara, Karim Sabra, Jean-Luc Gennisson, Mathias Fink, and Mickaél Tanter. 2010. Real-time visualization of muscle stiffness distribution with ultrasound shear wave imaging during muscle contraction. Muscle & nerve (2010).Google Scholar
- Eftychios Sifakis and Jernej Barbic. 2012. FEM simulation of 3D deformable solids: a practitioner's guide to theory, discretization and model reduction. In ACM SIGGRAPH 2012 Courses. 20. Google ScholarDigital Library
- Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. In ACM Trans. Graph., Vol. 24. 417--425. Google ScholarDigital Library
- Alex Smith, Sven Pohle, Wan-Chun Ma, Chongyang Ma, Xian-Chun Wu, Yanbing Chen, Etienne Danvoye, Jorge Jimenez, Sanjit Patel, Mike Sanders, and Cyrus A. Wilson. 2017. Emotion Challenge: Building a New Photoreal Facial Performance Pipeline for Games. In Proceedings of the ACM SIGGRAPH Digital Production Symposium (DigiPro '17). ACM, New York, NY, USA, Article 8, 2 pages. Google ScholarDigital Library
- Ian Stavness, Mohammad Ali Nazari, Cormac Flynn, Pascal Perrier, Yohan Payan, John E Lloyd, and Sidney Fels. 2014. Coupled biomech. modeling of the face, jaw, skull, tongue, and hyoid bone. In 3D Multiscale Physiological Human. Springer. Google ScholarDigital Library
- Joseph Teran, Sylvia Blemker, V Hing, and Ronald Fedkiw. 2003. Finite volume methods for the simulation of skeletal muscle. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. Eurographics Association, 68--74. Google ScholarDigital Library
- Joseph Teran, Eftychios Sifakis, Silvia S Blemker, Victor Ng-Thow-Hing, Cynthia Lau, and Ronald Fedkiw. 2005a. Creating and simulating skeletal muscle from the visible human data set. Vis. and Computer Graphics, IEEE Trans. on (2005). Google ScholarDigital Library
- Joseph Teran, Eftychios Sifakis, Geoffrey Irving, and Ronald Fedkiw. 2005b. Robust quasistatic finite elements and flesh simulation. In Proc. of the EG/SIGGRAPH Symposium on Comp. Anim. Google ScholarDigital Library
- Demetri Terzopoulos and Keith Waters. 1990. Physically-based facial modelling, analysis, and animation. Computer Animation and Virtual Worlds 1, 2 (1990), 73--80.Google Scholar
- HV Tran, F Charleux, M Rachik, Alain Ehrlacher, and MC Ho Ba Tho. 2007. In vivo characterization of the mech. properties of human skin derived from MRI and indentation techniques. Comp. methods in biomech. and biomed. eng. 10, 6 (2007), 401--407.Google Scholar
- Javier von der Pahlen, Jorge Jimenez, Etienne Danvoye, Paul Debevec, Graham Fyffe, and Oleg Alexander. 2014. Digital Ira and Beyond: Creating Real-time Photoreal Digital Actors. In ACM SIGGRAPH 2014 Courses (SIGGRAPH '14). Google ScholarDigital Library
- Andreas Wächter and Lorenz T Biegler. 2006. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming 106, 1 (2006). Google ScholarDigital Library
- Bin Wang, Longhua Wu, KangKang Yin, Uri Ascher, Libin Liu, and Hui Huang. 2015. Deformation capture and modeling of soft objects. ACM Trans. Graph. 34, 4 (2015), 94. Google ScholarDigital Library
- Jeffrey A Weiss, Bradley N Maker, and Sanjay Govindjee. 1996. Finite element implementation of incompressible, transversely isotropic hyperelasticity. Computer methods in applied mechanics and engineering 135, 1 (1996), 107--128.Google Scholar
- Tim Wu. 2013. A computational framework for modelling the biomechanics of human facial expressions. Ph.D. Dissertation. ResearchSpace@ Auckland.Google Scholar
- Wenwu Yang, Nathan Marshak, Daniel Sýkora, Srikumar Ramalingam, and Ladislav Kavan. 2018. Building Anatomically Realistic Jaw Kinematics Model from Data. CoRR abs/1805.05903 (2018). arXiv:1805.05903 http://arxiv.org/abs/1805.05903 Google ScholarDigital Library
- Seung-Hyun Yoon, John Lewis, and Taehyun Rhee. 2017. Blending Face Details: Synthesizing a Face Using Multiscale Face Models. IEEE computer graphics and applications 37, 6 (2017), 65--75.Google Scholar
- Eduard Zell, JP Lewis, Junyong Noh, Mario Botsch, et al. 2017. Facial retargeting with automatic range of motion alignment. ACM Transactions on Graphics (TOG) 36, 4 (2017), 154. Google ScholarDigital Library
- Gaspard Zoss, Derek Bradley, Pascal Bérard, and Thabo Beeler. 2018. An empirical rig for jaw animation. ACM Transactions on Graphics (TOG) 37, 4 (2018), 59. Google ScholarDigital Library
Index Terms
- Building Accurate Physics-based Face Models from Data
Recommendations
Physical face cloning
We propose a complete process for designing, simulating, and fabricating synthetic skin for an animatronics character that mimics the face of a given subject and its expressions. The process starts with measuring the elastic properties of a material ...
Face transfer with multilinear models
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersFace Transfer is a method for mapping videorecorded performances of one individual to facial animations of another. It extracts visemes (speech-related mouth articulations), expressions, and three-dimensional (3D) pose from monocular video or film ...
A Live Face Swapper
MM '16: Proceedings of the 24th ACM international conference on MultimediaIn this technical demonstration, we propose a face swapping framework, which is able to interactively change the appearance of a face in the wild to a different person/creature's face in real time on a mobile device. To realize this objective, we ...
Comments