Moving portraits

Abstract

We present an approach for generating face animations from large image collections of the same person. Such collections, which we call photobios, are remarkable in that they summarize a person's life in photos; the photos sample the appearance of a person over changes in age, pose, facial expression, hairstyle, and other variations. Yet, browsing and exploring photobios is infeasible due to their large volume. By optimizing the quantity and order in which photos are displayed and cross dissolving between them, we can render smooth transitions between face pose (e.g., from frowning to smiling), and create moving portraits from collections of still photos. Used in this context, the cross dissolve produces a very strong motion effect; a key contribution of the paper is to explain this effect and analyze its operating range. We demonstrate results on a variety of datasets including time-lapse photography, personal photo collections, and images of celebrities downloaded from the Internet. Our approach is completely automatic and has been widely deployed as the "Face Movies" feature in Google's Picasa.

References

1. Ahonen, T., Hadid, A., Pietikäinen, M. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28, 12 (2006), 2037--2041.
2. Arikan, O., Forsyth, D.A. Interactive motion generation from examples. ACM Trans. Graph. 21, 3 (2002), 483--490.
3. Beier, T., Neely, S. Feature-based image metamorphosis. ACM Trans. Graph. (SIGGRAPH) (1992), 35--42.
4. Berg, T.L., Berg, A.C., Edwards, J., Maire, M., White, R., Teh, Y.W., Learned-Miller, E., Forsyth, D.A. Names and faces in the news. In CVPR (2004), 848--854.
5. Bourdev, L., Brandt, J. Robust object detection via soft cascade. In CVPR (2005).
6. Bregler, C., Covell, M., Slaney, M. Video rewrite: Driving visual speech with audio. ACM Trans. Graph. (SIGGRAPH) (1997), 75--84.
7. Chen, S.E., Williams, L. View interpolation for image synthesis. ACM Trans. Graph. (SIGGRAPH) (1993), 279--288.
8. Dalal, N., Triggs, B. Histograms of oriented gradients for human detection. In CVPR (2005), 886--893.
9. Everingham, M., Sivic, J., Zisserman, A. "Hello! My name is … Buffy"---Automatic naming of characters in TV video. In Proceedings of the British Machine Vision Conference (2006).
10. Goldman, D.B., Gonterman, C., Curless, B., Salesin, D., Seitz, S.M. Video object annotation, navigation, and composition. In UIST (2008), 3--12.
11. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49. University of Massachusetts, Amherst, 2007.
12. Joshi, N., Szeliski, R., Kriegman, D.J. PSF estimation using sharp edge prediction. In CVPR (2008).
13. Katz, S., Tal, A., Basri, R. Direct visibility of point sets. ACM Trans. Graph. (SIGGRAPH 2007) 26, 3 (2007).
14. Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., Seitz, S.M. Being John Malkovich. In ECCV (2010).
15. Kemelmacher-Shlizerman, I., Shechtman, E., Garg, R., Seitz, S.M. Exploring photobios. ACM Trans. Graph. 30, 4 (2011), 61:1--61:10.
16. Kovar, L., Gleicher, M., Pighin, F. Motion graphs. ACM Trans. Graph. (SIGGRAPH) (2002), 473--482.
17. Lasseter, J. Principles of traditional animation applied to 3D computer animation. ACM Trans. Graph. (SIGGRAPH) (1987), 35--44.
18. Levoy, M., Hanrahan, P. Light field rendering. ACM Trans. Graph. (SIGGRAPH) (1996), 31--42.
19. Marr, D., Hildreth, E. Theory of edge detection. Proc. R. Soc. Lond. B 207 (1980), 187--217.
20. Nalwa, V.S., Binford, T.O. On detecting edges. IEEE Trans. Pattern Anal. Mach. Intell. 8, (1986), 699--714.
21. Picasa, 2010. http://googlephotos.blogspot.com/2010/08/picasa-38-face-movies-picnik.html.
22. Seitz, S.M., Dyer, C.R. View morphing. ACM Trans. Graph. (SIGGRAPH) (1996), 21--30.
23. Shashua, A. Geometry and photometry in 3D visual recognition. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA (1992).
24. Szeliski, R., Shum, H.Y. Creating full view panoramic image mosaics and environment maps. ACM Trans. Graph. (SIGGRAPH) (1997), 251--258.
25. Zhang, L., Snavely, N., Curless, B., Seitz, S.M. Spacetime faces: High resolution capture for modeling and animation. ACM Trans. Graph. (SIGGRAPH) (2004), 548--558.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation