Ravi Ramamoorthi
ABSTRACT
Realism in computer-generated images requires accurate input models for lighting, textures and BRDFs. One of the best ways of obtaining high-quality data is through measurements of scene attributes from real photographs by inverse rendering. However, inverse rendering methods have been largely limited to settings with highly controlled lighting. One of the reasons for this is the lack of a coherent mathematical framework for inverse rendering under general illumination conditions. Our main contribution is the introduction of a signal-processing framework which describes the reflected light field as a convolution of the lighting and BRDF, and expresses it mathematically as a product of spherical harmonic coefficients of the BRDF and the lighting. Inverse rendering can then be viewed as deconvolution. We apply this theory to a variety of problems in inverse rendering, explaining a number of previous empirical results. We will show why certain problems are ill-posed or numerically ill-conditioned, and why other problems are more amenable to solution. The theory developed here also leads to new practical representations and algorithms. For instance, we present a method to factor the lighting and BRDF from a small number of views, i.e. to estimate both simultaneously when neither is known.
- 1.R. Basri and D. Jacobs. Lambertian reflectance and linear subspaces. In International Conference on Computer Vision, 2001.Google Scholar
Cross Ref
- 2.R. Bastos, K. Hoff, W. Wynn, and A. Lastra. Increased photorealism for interactive architectural walkthroughs. In I3D 99, pages 183-190, 1999. Google ScholarDigital Library
- 3.B. Cabral, N. Max, and R. Springmeyer. Bidirectional reflection functions from surface bump maps. In SIGGRAPH 87, pages 273-281, 1987. Google ScholarDigital Library
- 4.B. Curless and M. Levoy. A volumetric method for building complex models from range images. In SIGGRAPH 96, pages 303-312, 1996. Google ScholarDigital Library
- 5.K. Dana, B. Ginneken, S. Nayar, and J. Koenderink. Reflectance and texture of real-world surfaces. ACM Transactions on Graphics, 18(1):1-34, January 1999. Google ScholarDigital Library
- 6.P. Debevec, T. Hawkins, C. Tchou, H.P. Duiker, W. Sarokin, and M. Sagar. Acquiring the reflectance field of a human face. In SIGGRAPH 00, pages 145-156. Google ScholarDigital Library
- 7.R. Dror, E. Adelson, and A. Willsky. Estimating surface reflectance properties from images under unknown illumination. In SPIE Photonics West: Human Vision and Electronic Imaging VI, 2001.Google ScholarCross Ref
- 8.M. D'Zmura. Computational Models of Visual Processing, chapter Shading Ambiguity: Reflectance and Illumination, pages 187-207. MIT Press, 1991.Google Scholar
- 9.Numerical Algorithms Group. NAG C Library Manual, Mark 5. 1999.Google Scholar
- 10.K. Ikeuchi and K. Sato. Determining reflectance properties of an object using range and brightness images. PAMI, 13(11):1139-1153, 1991. Google ScholarDigital Library
- 11.T. Inui, Y. Tanabe, and Y. Onodera. Group theory and its applications in physics. Springer Verlag, 1990.Google ScholarCross Ref
- 12.K. F. Karner, H. Mayer, and M. Gervautz. An image based measurement system for anisotropic reflection. Computer Graphics Forum, 15(3):119-128, 1996.Google ScholarCross Ref
- 13.G.J. Klinker, S.A. Shafer, and T. Kanade. The measurement of highlights in color images. IJCV, 2(1):7-32, 1988.Google ScholarCross Ref
- 14.J. J. Koenderink and A. J. van Doorn. Phenomenological description of bidirectional surface reflection. JOSA A, 15(11):2903-2912, 1998.Google ScholarCross Ref
- 15.E. Land and J. McCann. Lightness and retinex theory. Journal of the Optical Society of America, 61(1):1-11, 1971.Google ScholarCross Ref
- 16.M. Levoy, K. Pulli, B. Curless, S. Rusinkiewicz, D. Koller, L. Pereira, M. Ginzton, S. Anderson, J. Davis, J. Ginsberg, J. Shade, and D. Fulk. The digital michelangelo project: 3D scanning of large statues. In SIGGRAPH 00, pages 131-144. Google ScholarDigital Library
- 17.R.C. Love. Surface Reflection Model Estimation from Naturally Illuminated Image Sequences. PhD thesis, Leeds, 1997.Google Scholar
- 18.R. Lu, J. Koenderink, and A. Kappers. Optical properties (bidirectional reflection distribution functions) of velvet. Applied Optics, 37(25):5974-5984, 1998.Google ScholarCross Ref
- 19.T.M. MacRobert. Spherical harmonics; an elementary treatise on harmonic functions, with applications. Dover Publications, 1948.Google Scholar
- 20.S.R. Marschner and D.P. Greenberg. Inverse lighting for photography. In Fifth Color Imaging Conference, pages 262-265, 1997.Google ScholarCross Ref
- 21.S.R. Marschner, S.H. Westin, E.P.F. Lafortune, and K.E. Torrance. Image-Based BRDF measurement. Applied Optics, 39(16):2592-2600, 2000.Google ScholarCross Ref
- 22.N. McCormick. Inverse radiative transfer problems: a review. Nuclear Science and Engineering, 112:185-198, 1992.Google ScholarCross Ref
- 23.G. Miller and C. Hoffman. Illumination and reflection maps: Simulated objects in simulated and real environments. SIGGRAPH 84 Advanced Computer Graphics Animation seminar notes, 1984.Google Scholar
- 24.F. E. Nicodemus, J. C. Richmond, J. J. Hsia, I. W. Ginsberg, and T. Limperis. Geometric Considerations and Nomenclature for Reflectance. National Bureau of Standards (US), 1977.Google Scholar
- 25.J. Nimeroff, E. Simoncelli, and J. Dorsey. Efficient re-rendering of naturally illuminated environments. In EGWR 94, pages 359-373.Google Scholar
- 26.R.W. Preisendorfer. Hydrologic Optics. US Dept Commerce, 1976.Google Scholar
- 27.R. Ramamoorthi and P. Hanrahan. Analysis of planar light fields from homogeneous convex curved surfaces under distant illumination. In SPIE Photonics West: Human Vision and Electronic Imaging VI, 2001.Google ScholarCross Ref
- 28.R. Ramamoorthi and P. Hanrahan. An efficient representation for irradiance environment maps. In SIGGRAPH 01, 2001. Google ScholarDigital Library
- 29.R. Ramamoorthi and P. Hanrahan. On the relationship between radiance and irradiance: Determining the illumination from images of a convex lambertian object. To appear, Journal of the Optical Society of America A, 2001.Google ScholarCross Ref
- 30.I. Sato, Y. Sato, and K. Ikeuchi. Illumination distribution from brightness in shadows: adaptive estimation of illumination distribution with unknown reflectance properties in shadow regions. In ICCV 99, pages 875 - 882, 1999.Google Scholar
- 31.Y. Sato and K. Ikeuchi. Reflectance analysis under solar illumination. Technical Report CMU-CS-94-221, Carnegie Mellon University, 1994. Google Scholar
- 32.Y. Sato, M. D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In SIGGRAPH 97, pages 379-388, 1997. Google ScholarDigital Library
- 33.F. X. Sillion, J. Arvo, S. H. Westin, and D. Greenberg. A global illumination solution for general reflectance distributions. In SIGGRAPH 91, pages 187-196. Google ScholarDigital Library
- 34.S.R.Marschner. Inverse Rendering for Computer Graphics. PhD thesis, Cornell, 1998. Google ScholarDigital Library
- 35.S. Tominaga and N. Tanaka. Estimating reflection parameters from a single color image. IEEE Computer Graphics & Applications, 20(5):58-66, 2000. Google ScholarDigital Library
- 36.K. E. Torrance and E. M. Sparrow. Theory for off-specular reflection from roughened surfaces. JOSA, 57(9):1105-1114, 1967.Google ScholarCross Ref
- 37.G. J. Ward. Measuring and modeling anisotropic reflection. In SIGGRAPH 92, pages 265-272, 1992. Google ScholarDigital Library
- 38.Y. Yu, P. Debevec, J. Malik, and T. Hawkins. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In SIGGRAPH 99, pages 215-224, 1999. Google ScholarDigital Library
- 39.Y. Yu and J. Malik. Recovering photometric properties of architectural scenes from photographs. In SIGGRAPH 98, pages 207-218, 1998. Google ScholarDigital Library
- 40.Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000. Google ScholarDigital Library
Index Terms
- A signal-processing framework for inverse rendering
Recommendations
An efficient representation for irradiance environment maps
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniquesWe consider the rendering of diffuse objects under distant illumination, as specified by an environment map. Using an analytic expression for the irradiance in terms of spherical harmonic coefficients of the lighting, we show that one needs to compute ...
A Signal-Processing Framework for Inverse Rendering
Seminal Graphics Papers: Pushing the Boundaries, Volume 2Realism in computer-generated images requires accurate input models for lighting, textures and BRDFs. One of the best ways of obtaining high-quality data is through measurements of scene attributes from real photographs by inverse rendering. However, ...
A signal-processing framework for reflection
We present a signal-processing framework for analyzing the reflected light field from a homogeneous convex curved surface under distant illumination. This analysis is of theoretical interest in both graphics and vision and is also of practical ...
Reviews
Thomas W. Crockett
In computer graphics, rendering is the process of generating an image from an abstract description of a scene. In this paper, the authors address the inverse rendering problem, in which properties of a scene are deduced from a series of images. The focus in this case is on extracting illumination parameters and surface properties from known geometry under complex lighting conditions. To guide their efforts, the authors first develop a theoretical framework in which inverse rendering can be understood mathematically as a deconvolution operation. This theory is then used to determine what quantities can and cannot be recovered successfully by inverse rendering, and under what circumstances problems are well-posed or ill-conditioned. Although the analysis is based on a fairly restrictive set of assumptions (known geometry, convex isotropic surfaces, no interreflections), it leads to algorithms that can be applied to a wider range of problems. The successes and limitations of the method are demonstrated by comparing photographs of real scenes to renderings under known and unknown lighting conditions. The paper is well organized, well written, and amply illustrated. It is also heavily mathematical in content, and will therefore be of most interest to readers with a firm grounding in the mathematics of illumination and signal processing. The paper represents a fine example of the scientific method in computer science, using thorough analysis to guide the development of practical algorithms, which are verified through careful experimentation. The results obtained are compelling enough to suggest that this paper will become one of the classics in the field of inverse rendering. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments