skip to main content
Inverse rendering for computer graphics
Publisher:
  • Cornell University
  • PO Box 250, 124 Roberts Place Ithaca, NY
  • United States
ISBN:978-0-591-93739-8
Order Number:AAI9839924
Pages:
196
Bibliometrics
Skip Abstract Section
Abstract

Creating realistic images has been a major focus in the study of computer graphics for much of its history. This effort has led to mathematical models and algorithms that can compute predictive, or physically realistic, images from known camera positions and scene descriptions that include the geometry of objects, the reflectance of surfaces, and the lighting used to illuminate the scene. These images accurately describe the physical quantities that would be measured from a real scene. Because these algorithms can predict real images, they can also be used in inverse problems to work backward from photographs to attributes of the scene.

Work on three such inverse rendering problems is described. The first, inverse lighting, assumes knowledge of geometry, reflectance, and the recorded photograph and solves for the lighting in the scene. A technique using a linear least-squares system is proposed and demonstrated. Also demonstrated is an application of inverse lighting, called re-lighting, which modifies lighting in photographs.

The second two inverse rendering problems solve for unknown reflectance, given images with known geometry, lighting, and camera positions. Photographic texture measurement concentrates on capturing the spatial variation in an object's reflectance. The resulting system begins with scanned 3D models of real objects and uses photographs to construct accurate, high-resolution textures suitable for physically realistic rendering. The system is demonstrated on two complex natural objects with detailed surface textures.

Image-based BRDF measurement takes the opposite approach to reflectance measurement, capturing the directional characteristics of a surface's reflectance by measuring the bidirectional reflectance distribution function, or BRDF. Using photographs of an object with spatially uniform reflectance, the BRDFs of paints and papers are measured with completeness and accuracy that rival that of measurements obtained using specialized devices. The image-based approach and novel light source positioning technique require only general-purpose equipment, so the cost of the apparatus is low compared to conventional approaches. In addition, very densely sampled data can be measured very quickly, when the wavelength spectrum of the BRDF does not need to be measured in detail.

Cited By

  1. ACM
    Ramamoorthi R and Hanrahan P A Signal-Processing Framework for Inverse Rendering Seminal Graphics Papers: Pushing the Boundaries, Volume 2, (327-338)
  2. ACM
    Mubarik M, Kanungo R, Zirr T and Kumar R Hardware Acceleration of Neural Graphics Proceedings of the 50th Annual International Symposium on Computer Architecture, (1-12)
  3. ACM
    Li Q, Guo J, Fei Y, Li F and Guo Y NeuLighting: Neural Lighting for Free Viewpoint Outdoor Scene Relighting with Unconstrained Photo Collections SIGGRAPH Asia 2022 Conference Papers, (1-9)
  4. Lyu L, Tewari A, Leimkühler T, Habermann M and Theobalt C Neural Radiance Transfer Fields for Relightable Novel-View Synthesis with Global Illumination Computer Vision – ECCV 2022, (153-169)
  5. ACM
    Zhao Y, Chen H, Sahli H, Lu K and Jiang D Uncertainty-Aware Semi-Supervised Learning of 3D Face Rigging from Single Image Proceedings of the 30th ACM International Conference on Multimedia, (170-179)
  6. ACM
    Zhang X, Srinivasan P, Deng B, Debevec P, Freeman W and Barron J (2021). NeRFactor, ACM Transactions on Graphics, 40:6, (1-18), Online publication date: 1-Dec-2021.
  7. ACM
    Pediredla A, Veeraraghavan A and Gkioulekas I (2019). Ellipsoidal path connections for time-gated rendering, ACM Transactions on Graphics, 38:4, (1-12), Online publication date: 31-Aug-2019.
  8. ACM
    Li D, Langlois T and Zheng C (2018). Scene-aware audio for 360° videos, ACM Transactions on Graphics, 37:4, (1-12), Online publication date: 31-Aug-2018.
  9. Tetzlaff M and Meyer G Using flash photography and image-based rendering to document cultural heritage artifacts Proceedings of the 14th Eurographics Workshop on Graphics and Cultural Heritage, (137-146)
  10. ACM
    Mallett I and Yuksel C Parameterization of Tabulated BRDFs Proceedings of the 33rd Computer Graphics International, (45-48)
  11. Hegedus R, Lucat A, Redon J and Pacanowski R Isotropic BRDF measurements with quantified uncertainties Proceedings of the Eurographics 2016 Workshop on Material Appearance Modeling, (9-14)
  12. ACM
    Nagano K, Collins T, Chen C and Nakano A GPU-based inverse rendering with multi-objective particle swarm optimization SIGGRAPH Asia 2015 Visualization in High Performance Computing, (1-4)
  13. ACM
    Aittala M, Weyrich T and Lehtinen J (2015). Two-shot SVBRDF capture for stationary materials, ACM Transactions on Graphics, 34:4, (1-13), Online publication date: 27-Jul-2015.
  14. ACM
    Dong Y, Chen G, Peers P, Zhang J and Tong X (2014). Appearance-from-motion, ACM Transactions on Graphics, 33:6, (1-12), Online publication date: 19-Nov-2014.
  15. Herbort S and Wöhler C (2011). An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods, 3D Research, 2:3, (1-17), Online publication date: 1-Sep-2011.
  16. Bai J, Chandraker M, Ng T and Ramamoorthi R A dual theory of inverse and forward light transport Proceedings of the 11th European conference on Computer vision: Part II, (294-307)
  17. ACM
    Holroyd M, Lawrence J and Zickler T A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance ACM SIGGRAPH 2010 papers, (1-12)
  18. ACM
    Holroyd M, Lawrence J and Zickler T (2010). A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance, ACM Transactions on Graphics, 29:4, (1-12), Online publication date: 26-Jul-2010.
  19. ACM
    Zinke A, Rump M, Lay T, Weber A, Andriyenko A and Klein R A practical approach for photometric acquisition of hair color ACM SIGGRAPH Asia 2009 papers, (1-9)
  20. ACM
    Gutierrez D, Jensen H, Jarosz W and Donner C Scattering ACM SIGGRAPH ASIA 2009 Courses, (1-620)
  21. ACM
    Zinke A, Rump M, Lay T, Weber A, Andriyenko A and Klein R (2009). A practical approach for photometric acquisition of hair color, ACM Transactions on Graphics, 28:5, (1-9), Online publication date: 1-Dec-2009.
  22. ACM
    Gutierrez D, Jarosz W, Donner C and Narasimhan S Scattering ACM SIGGRAPH 2009 Courses, (1-397)
  23. Ghosh A, Chen T, Peers P, Wilson C and Debevec P Estimating specular roughness and anisotropy from second order spherical gradient illumination Proceedings of the Twentieth Eurographics conference on Rendering, (1161-1170)
  24. ACM
    Dorsey J and Rushmeier H Light and materials in virtual cities ACM SIGGRAPH 2008 classes, (1-4)
  25. ACM
    Rushmeier H The perception of simulated materials ACM SIGGRAPH 2008 classes, (1-12)
  26. Theobalt C, Ahmed N, Lensch H, Magnor M and Seidel H (2007). Seeing People in Different Light-Joint Shape, Motion, and Reflectance Capture, IEEE Transactions on Visualization and Computer Graphics, 13:4, (663-674), Online publication date: 1-Jul-2007.
  27. Patow G, Pueyo X and Vinacua A (2007). Technical Section, Computers and Graphics, 31:3, (501-515), Online publication date: 1-Jun-2007.
  28. Zickler T, Ramamoorthi R, Enrique S and Belhumeur P (2006). Reflectance Sharing, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28:8, (1287-1302), Online publication date: 1-Aug-2006.
  29. ACM
    Narasimhan S, Gupta M, Donner C, Ramamoorthi R, Nayar S and Jensen H Acquiring scattering properties of participating media by dilution ACM SIGGRAPH 2006 Papers, (1003-1012)
  30. ACM
    Narasimhan S, Gupta M, Donner C, Ramamoorthi R, Nayar S and Jensen H (2006). Acquiring scattering properties of participating media by dilution, ACM Transactions on Graphics, 25:3, (1003-1012), Online publication date: 1-Jul-2006.
  31. Chandra K, Adabala N and Toyama K Aerial image relighting Proceedings of the 24th international conference on Advances in Computer Graphics, (594-605)
  32. Magallon J, Patow G, Seron F and Pueyo X Parallelization of inverse design of luminaire reflectors Proceedings of the 6th Eurographics conference on Parallel Graphics and Visualization, (99-107)
  33. Mahajan D, Ramamoorthi R and Curless B A theory of spherical harmonic identities for BRDF/Lighting transfer and image consistency Proceedings of the 9th European conference on Computer Vision - Volume Part IV, (41-55)
  34. Birkbeck N, Cobzas D, Sturm P and Jagersand M Variational shape and reflectance estimation under changing light and viewpoints Proceedings of the 9th European conference on Computer Vision - Volume Part I, (536-549)
  35. Malassiotis S and Strintzis M (2005). Robust face recognition using 2D and 3D data, Pattern Recognition, 38:12, (2537-2548), Online publication date: 1-Dec-2005.
  36. Magnor M, Kindlmann G, Hansen C and Duric N (2005). Reconstruction and Visualization of Planetary Nebulae, IEEE Transactions on Visualization and Computer Graphics, 11:5, (485-496), Online publication date: 1-Sep-2005.
  37. ACM
    Lensch H, Goesele M, Chuang Y, Hawkins T, Marschner S, Matusik W and Mueller G Realistic materials in computer graphics ACM SIGGRAPH 2005 Courses, (1-es)
  38. Zickler T, Enrique S, Ramamoorthi R and Belhumeur P Reflectance sharing: image-based rendering from a sparse set of images Proceedings of the Sixteenth Eurographics conference on Rendering Techniques, (253-264)
  39. Ngan A, Durand F and Matusik W Experimental analysis of BRDF models Proceedings of the Sixteenth Eurographics conference on Rendering Techniques, (117-126)
  40. Hawkins T, Einarsson P and Debevec P A dual light stage Proceedings of the Sixteenth Eurographics conference on Rendering Techniques, (91-98)
  41. Stark M, Arvo J and Smits B (2005). Barycentric Parameterizations for Isotropic BRDFs, IEEE Transactions on Visualization and Computer Graphics, 11:2, (126-138), Online publication date: 1-Mar-2005.
  42. Magnor M, Kindlmann G and Hansen C Constrained Inverse Volume Rendering for Planetary Nebulae Proceedings of the conference on Visualization '04, (83-90)
  43. ACM
    Tsumura N, Ojima N, Sato K, Shiraishi M, Shimizu H, Nabeshima H, Akazaki S, Hori K and Miyake Y Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin ACM SIGGRAPH 2003 Papers, (770-779)
  44. ACM
    Gardner A, Tchou C, Hawkins T and Debevec P Linear light source reflectometry ACM SIGGRAPH 2003 Papers, (749-758)
  45. ACM
    Tsumura N, Ojima N, Sato K, Shiraishi M, Shimizu H, Nabeshima H, Akazaki S, Hori K and Miyake Y (2003). Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin, ACM Transactions on Graphics, 22:3, (770-779), Online publication date: 1-Jul-2003.
  46. ACM
    Gardner A, Tchou C, Hawkins T and Debevec P (2003). Linear light source reflectometry, ACM Transactions on Graphics, 22:3, (749-758), Online publication date: 1-Jul-2003.
  47. Luong Q, Fua P and Leclerc Y (2002). The Radiometry of Multiple Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24:1, (19-33), Online publication date: 1-Jan-2002.
  48. ACM
    Hawkins T, Cohen J and Debevec P A photometric approach to digitizing cultural artifacts Proceedings of the 2001 conference on Virtual reality, archeology, and cultural heritage, (333-342)
  49. Bernardini F, Martin I and Rushmeier H (2001). High-Quality Texture Reconstruction from Multiple Scans, IEEE Transactions on Visualization and Computer Graphics, 7:4, (318-332), Online publication date: 1-Oct-2001.
  50. ACM
    Ramamoorthi R and Hanrahan P A signal-processing framework for inverse rendering Proceedings of the 28th annual conference on Computer graphics and interactive techniques, (117-128)
  51. ACM
    Debevec P (1999). Image-based modeling and lighting, ACM SIGGRAPH Computer Graphics, 33:4, (46-50), Online publication date: 4-Nov-1999.
  52. Yu Y, Debevec P, Malik J and Hawkins T Inverse global illumination Proceedings of the 26th annual conference on Computer graphics and interactive techniques, (215-224)
  53. Marschner S, Westin S, Lafortune E, Torrance K and Greenberg D Image-based BRDF measurement including human skin Proceedings of the 10th Eurographics conference on Rendering, (131-144)
  54. Rocchini C, Cignoni P, Montani C and Scopigno R Multiple textures stitching and blending on 3D objects Proceedings of the 10th Eurographics conference on Rendering, (119-130)
Contributors
  • Cornell University
  • Stanford University

Recommendations