Physical face cloning

Abstract

We propose a complete process for designing, simulating, and fabricating synthetic skin for an animatronics character that mimics the face of a given subject and its expressions. The process starts with measuring the elastic properties of a material used to manufacture synthetic soft tissue. Given these measurements we use physics-based simulation to predict the behavior of a face when it is driven by the underlying robotic actuation. Next, we capture 3D facial expressions for a given target subject. As the key component of our process, we present a novel optimization scheme that determines the shape of the synthetic skin as well as the actuation parameters that provide the best match to the target expressions. We demonstrate this computational skin design by physically cloning a real human face onto an animatronics figure.

References

1. Becker, M., and Teschner, M. 2007. Robust and efficient estimation of elasticity parameters using the linear finite element method. In SimVis, 15--28.
2. Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4 (July), 40:1--40:9.
3. Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. 30, 4 (August), 75:1--75:10.
4. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3 (July), 33:1--33:10.
5. Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (July), 89:1--89:9.
6. Bickel, B., Bächer, M., Otaduy, M. A., Lee, H. R., Pfister, H., Gross, M., and Matusik, W. 2010. Design and fabrication of materials with desired deformation behavior. ACM Trans. Graph. 29, 4 (July), 63:1--63:10.
7. Bonet, J., and Wood, R. D. 1997. Nonlinear Continuum Mechanics for Finite Element Analysis. Cambridge Uni. Press.
8. Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.
9. Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 4 (July), 41:1--41:10.
10. Bucur, D., and Buttazzo, G. 2005. Variational Methods in Shape Optimization Problems. Birkhuser Mathematics.
11. Chentanez, N., Alterovitz, R., Ritchie, D., Cho, L., Hauser, K. K., Goldberg, K., Shewchuk, J. R., and O'Brien, J. F. 2009. Interactive simulation of surgical needle insertion and steering. ACM Trans. Graph. 28, 3 (July), 88:1--88:10.
12. Clarberg, P., Jarosz, W., Akenine-Möller, T., and Jensen, H. W. 2005. Wavelet importance sampling: efficiently evaluating products of complex functions. ACM Trans. Graph. 24, 3 (Aug.), 1166--1175.
13. Dong, Y., Wang, J., Pellacini, F., Tong, X., and Guo, B. 2010. Fabricating spatially-varying subsurface scattering. ACM Trans. Graph. 29 (July), 62:1--62:10.
14. Gourret, J.-P., Thalmann, N. M., and Thalmann, D. 1989. Simulation of object and human skin deformations in a grasping task. In Comp. Graph. (Proc. SIGGRAPH), 21--30.
15. Hara, F., Akazawa, H., and Kobayashi, H. 2001. Realistic facial expressions by sma driven face robot. In Proc. of IEEE International Workshop on Robot and Human Interactive Communication, 504--511.
16. Hašan, M., Fuchs, M., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2010. Physical reproduction of materials with specified subsurface scattering. ACM Trans. Graph. 29 (July), 61:1--61:10.
17. Irving, G., Teran, J., and Fedkiw, R. 2004. Invertible finite elements for robust simulation of large deformation. In 2004 ACM SIGGRAPH/Eurographics SCA, 131--140.
18. Kauer, M., Vuskovic, V., Dual, J., Szekely, G., and Bajka, M. 2002. Inverse finite element characterization of soft tissues. Medical Image Analysis 6, 3, 257--287.
19. Kharevych, L., Mullen, P., Owhadi, H., and Desbrun, M. 2009. Numerical coarsening of inhomogeneous elastic materials. ACM Trans. Graph. 28 (July), 51:1--51:8.
20. Koch, R. M., Gross, M. H., Carls, F. R., von Büren, D. F., Fankhauser, G., and Parish, Y. I. H. 1996. Simulating facial surgery using finite element models. In Proc. of Comp. graph. and int. tech., ACM, SIGGRAPH '96, 421--428.
21. Lee, S.-H., and Terzopoulos, D. 2006. Heads up!: biomechanical modeling and neuromuscular control of the neck. In ACM SIGGRAPH 2006 Papers, ACM, 1188--1198.
22. Lee, S.-H., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28 (September), 99:1--99:17.
23. Levin, D. 1998. The approximation power of moving least-squares. Math. Comput. 67 (October), 1517--1531.
24. Li, X.-Y., Shen, C.-H., Huang, S.-S., Ju, T., and Hu, S.-M. 2010. Popup: automatic paper architectures from 3d models. ACM Trans. Graph. 29 (July), 111:1--111:9.
25. Mori, Y., and Igarashi, T. 2007. Plushie: an interactive design system for plush toys. In ACM SIGGRAPH 2007 papers, ACM, SIGGRAPH '07.
26. Nesme, M., Kry, P. G., Jeřábková, L., and Faure, F. 2009. Preserving topology and elasticity for embedded deformable models. In ACM SIGGRAPH 2009 papers, ACM, 52:1--52:9.
27. Nishio, S., Ishiguro, H., and Hagita, N. 2007. Humanoid Robots: New Developments. I-Tech, ch. Geminoid: Teleoperated Android of an Existing Person.
28. Nocedal, J., and Wright, S. J. 2000. Numerical Optimization. Springer, August.
29. Oh, J.-H., Hanson, D., Kim, W.-S., Han, I. Y., Kim, J.-Y., and Park, I.-W. 2006. Design of android type humanoid robot albert hubo. In Proc. of IEEE/RSJ Int. Conference on Intelligent Robots and Systems, 1428--1433.
30. Pai, D. K., Doel, K. v. d., James, D. L., Lang, J., Lloyd, J. E., Richmond, J. L., and Yau, S. H. 2001. Scanning physical interaction behavior of 3d objects. In Proc. of Comp. graph. and int. tech., ACM, SIGGRAPH '01, 87--96.
31. Schenk, O., and Gärtner, K. 2006. On fast factorization pivoting methods for symmetric indefinite systems. In Elec. Trans. Numer., no. 23, 158--179.
32. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. In ACM SIGGRAPH 2005, 417--425.
33. Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3.
34. Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Trans. on Vis. and Comp. Graph. 11, 3, 317--328.
35. Terzopoulos, D., and Waters, K. 1993. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. Pattern Anal. Mach. Intell. 15 (June), 569--579.
36. Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In Proc. Comp. graph. and int. tech., ACM, SIGGRAPH '87, 205--214.
37. Thoutireddy, P., and Ortiz, M. 2004. A variational r-adaption and shape-optimization method for finite-deformation elasticity. Int. J. Numer. Meth. Engng. 61, 1--21.
38. Van Gelder, A. 1998. Approximate simulation of elastic membranes by triangulated spring meshes. J. Graph. Tools 3, 2, 21--42.
39. Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., and Huang, P. 2004. High resolution acquisition, learning and transfer of dyanmic 3-d facial expressions. In Comp. Graph. Forum, 677--686.
40. Weyrich, T., Peers, P., Matusik, W., and Rusinkiewicz, S. 2009. Fabricating microgeometry for custom surface reflectance. ACM Trans. Graph. 28 (July), 32:1--32:6.
41. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3 (Aug.), 548--558.