Abstract
We present an approach for generating face animations from large image collections of the same person. Such collections, which we call photobios, are remarkable in that they summarize a person's life in photos; the photos sample the appearance of a person over changes in age, pose, facial expression, hairstyle, and other variations. Yet, browsing and exploring photobios is infeasible due to their large volume. By optimizing the quantity and order in which photos are displayed and cross dissolving between them, we can render smooth transitions between face pose (e.g., from frowning to smiling), and create moving portraits from collections of still photos. Used in this context, the cross dissolve produces a very strong motion effect; a key contribution of the paper is to explain this effect and analyze its operating range. We demonstrate results on a variety of datasets including time-lapse photography, personal photo collections, and images of celebrities downloaded from the Internet. Our approach is completely automatic and has been widely deployed as the "Face Movies" feature in Google's Picasa.
- Ahonen, T., Hadid, A., Pietikäinen, M. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28, 12 (2006), 2037--2041. Google ScholarDigital Library
- Arikan, O., Forsyth, D.A. Interactive motion generation from examples. ACM Trans. Graph. 21, 3 (2002), 483--490. Google ScholarDigital Library
- Beier, T., Neely, S. Feature-based image metamorphosis. ACM Trans. Graph. (SIGGRAPH) (1992), 35--42. Google ScholarDigital Library
- Berg, T.L., Berg, A.C., Edwards, J., Maire, M., White, R., Teh, Y.W., Learned-Miller, E., Forsyth, D.A. Names and faces in the news. In CVPR (2004), 848--854. Google ScholarDigital Library
- Bourdev, L., Brandt, J. Robust object detection via soft cascade. In CVPR (2005). Google ScholarDigital Library
- Bregler, C., Covell, M., Slaney, M. Video rewrite: Driving visual speech with audio. ACM Trans. Graph. (SIGGRAPH) (1997), 75--84. Google ScholarDigital Library
- Chen, S.E., Williams, L. View interpolation for image synthesis. ACM Trans. Graph. (SIGGRAPH) (1993), 279--288. Google ScholarDigital Library
- Dalal, N., Triggs, B. Histograms of oriented gradients for human detection. In CVPR (2005), 886--893. Google ScholarDigital Library
- Everingham, M., Sivic, J., Zisserman, A. "Hello! My name is … Buffy"---Automatic naming of characters in TV video. In Proceedings of the British Machine Vision Conference (2006).Google ScholarCross Ref
- Goldman, D.B., Gonterman, C., Curless, B., Salesin, D., Seitz, S.M. Video object annotation, navigation, and composition. In UIST (2008), 3--12. Google ScholarDigital Library
- Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49. University of Massachusetts, Amherst, 2007.Google Scholar
- Joshi, N., Szeliski, R., Kriegman, D.J. PSF estimation using sharp edge prediction. In CVPR (2008).Google Scholar
- Katz, S., Tal, A., Basri, R. Direct visibility of point sets. ACM Trans. Graph. (SIGGRAPH 2007) 26, 3 (2007). Google ScholarDigital Library
- Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., Seitz, S.M. Being John Malkovich. In ECCV (2010). Google ScholarDigital Library
- Kemelmacher-Shlizerman, I., Shechtman, E., Garg, R., Seitz, S.M. Exploring photobios. ACM Trans. Graph. 30, 4 (2011), 61:1--61:10. Google ScholarDigital Library
- Kovar, L., Gleicher, M., Pighin, F. Motion graphs. ACM Trans. Graph. (SIGGRAPH) (2002), 473--482. Google ScholarDigital Library
- Lasseter, J. Principles of traditional animation applied to 3D computer animation. ACM Trans. Graph. (SIGGRAPH) (1987), 35--44. Google ScholarDigital Library
- Levoy, M., Hanrahan, P. Light field rendering. ACM Trans. Graph. (SIGGRAPH) (1996), 31--42. Google ScholarDigital Library
- Marr, D., Hildreth, E. Theory of edge detection. Proc. R. Soc. Lond. B 207 (1980), 187--217.Google ScholarCross Ref
- Nalwa, V.S., Binford, T.O. On detecting edges. IEEE Trans. Pattern Anal. Mach. Intell. 8, (1986), 699--714. Google ScholarDigital Library
- Picasa, 2010. http://googlephotos.blogspot.com/2010/08/picasa-38-face-movies-picnik.html.Google Scholar
- Seitz, S.M., Dyer, C.R. View morphing. ACM Trans. Graph. (SIGGRAPH) (1996), 21--30. Google ScholarDigital Library
- Shashua, A. Geometry and photometry in 3D visual recognition. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA (1992). Google ScholarDigital Library
- Szeliski, R., Shum, H.Y. Creating full view panoramic image mosaics and environment maps. ACM Trans. Graph. (SIGGRAPH) (1997), 251--258. Google ScholarDigital Library
- Zhang, L., Snavely, N., Curless, B., Seitz, S.M. Spacetime faces: High resolution capture for modeling and animation. ACM Trans. Graph. (SIGGRAPH) (2004), 548--558. Google ScholarDigital Library
Index Terms
- Moving portraits
Recommendations
Bringing portraits to life
We present a technique to automatically animate a still portrait, making it possible for the subject in the photo to come to life and express various emotions. We use a driving video (of a different subject) and develop means to transfer the ...
Deep Shapely Portraits
MM '20: Proceedings of the 28th ACM International Conference on MultimediaWe present deep shapely portraits, a novel method based on deep learning, to automatically reshape an input portrait to be better proportioned and more shapely while keeping personal facial characteristics. Different from existing methods that may ...
Identity-Preserving Face Recovery from Stylized Portraits
Given an artistic portrait, recovering the latent photorealistic face that preserves the subject's identity is challenging because the facial details are often distorted or fully lost in artistic portraits. We develop an Identity-preserving Face ...
Comments