Abstract
We enhance underexposed, low dynamic range videos by adaptively and independently varying the exposure at each photoreceptor in a post-process. This virtual exposure is a dynamic function of both the spatial neighborhood and temporal history at each pixel. Temporal integration enables us to expand the image's dynamic range while simultaneously reducing noise. Our non-linear exposure variation and denoising filters smoothly transition from temporal to spatial for moving scene elements. Our virtual exposure framework also supports temporally coherent per frame tone mapping. Our system outputs restored video sequences with significantly reduced noise, increased exposure time of dark pixels, intact motion, and improved details.
Supplemental Material
- Acosta-Serafini, P. M., Masaki, I., and Sodini, C. G. 2004, Predictive Multiple Sampling Algorithm with Overlapping Integration Intervals for Linear Wide Dynamic Range Integrating Image Sensors. IEEE Transactions on Intelligent Transportation Systems, 5, 1, 33--41. Google ScholarDigital Library
- Barash, D. 2002. A Fundamental Relationship Between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation. Transactions on Pattern Matching and Machine Learning, 24, 6, 844--847. Google ScholarDigital Library
- Bennett E. P. and McMillan, L. 2003. Proscenium: A Framework for Spatio-Temporal Video Editing. In Proceedings of ACM Multimedia 2003, 177--183. Google ScholarDigital Library
- Bidermann, W., El Gamal, A., Ewedemi, S., Reyneri, J., Tian, H., Wile, D., and Yang, D., 2003. A .18μm High Dynamic Range NTSC/PAL Imaging System-on-Chip with Embedded DRAM Frame Buffer, In Proceedings of the IEEE International Solid-State Circuits Conference, 212--213.Google Scholar
- Boomgaard R. V. D., and Weijer, J. V. D. 2002. On the Equivalence of Local-Mode Finding, Robust Estimation and Mean Shift Analysis As Used In Early Vision Tasks. In Proceedings of the International Conference on Pattern Recognition, 927--930. Google ScholarDigital Library
- Choudhury, P. and Tumblin, J. 2003. The Trilateral Filter for High Contrast Images and Meshes. In Proceedings of the Eurographics Symposium on Rendering 2003. 1--11. Google ScholarDigital Library
- Cohen, M., Colburn, A., and Drucker, S. 2003. Image Stacks. Microsoft Research Technical Report, MSR-TR-2003-40.Google Scholar
- Debevec, P. E. and Malik, J. 1997, Recovering High Dynamic Range Radiance Maps from Photographs. In Proceedings of ACM SIGGRAPH 1997. ACM SIGGRAPH / Addison Wesley, Computer Graphics Proceedings, Annual Conference Series, 369--378. Google ScholarDigital Library
- Drago, F., Myszkowski, K., Annen, T., and Chiba, N. 2003. Adaptive Logarithmic Mapping for Displaying High Contrast Scenes. In Proceedings of EUROGRAPHICS 2003, 22, 3, 419--426.Google Scholar
- Dubois, E. and Sabri, S., 1984. Noise Reduction in Image Sequences Using Motion-Compensated Temporal Filtering, IEEE Transactions on Communications, 32, 7, 826--831.Google ScholarCross Ref
- Durand, F. and Dorsey, J. 2002. Fast Bilateral Filtering for the Display of High-Dynamic Range Images. ACM Transactions on Graphics, 21, 3, 257--266. Google ScholarDigital Library
- Eisemann, E. and Durand, F. 2004. Flash Photography Enhancement via Intrinsic Relighting. ACM Transactions on Graphics, 23, 3, 670--675. Google ScholarDigital Library
- Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient Domain High Dynamic Range Compression. ACM Transactions on Graphics, 21, 3, 249--256. Google ScholarDigital Library
- Francis, J. J. and Jager, G. D. 2003. The Bilateral Median Filter. In Proceedings of the 14th Symposium of the Pattern Recognition Association of South Africa.Google Scholar
- Jobson, D. J., Rahman, Z.-U., and Woodell, G. A. 1997. A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observation of Scenes. IEEE Transactions on Image Processing, 6, 7, 965--976. Google ScholarDigital Library
- Jostschulte, K., Amer, A., Schu, M., and Scroder, H., 1998. Perception Adaptive Temporal TV-Noise Reduction Using Contour Preserving Prefilter Techniques, IEEE Transactions on Consumer Electronics, 44, 3 (August), 1091--1096. Google ScholarDigital Library
- Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High Dynamic Range Video, ACM Transactions on Graphics, 22, 3, 319--325. Google ScholarDigital Library
- Lee, S. H. and Kang, M. G. 1998. Spatio-Temporal Video Filtering Algorithm based on 3-D Anisotropic Diffusion Equation. In Proceedings of the International Conference on Image Processing, 98, 2, 447--450.Google Scholar
- Liu, X., and El Gamal, A., 2003. Synthesis of High Dynamic Range Motion Blur Free Image From Multiple Captures. IEEE Transactions on Circuits and Systems, Fundamental Theory and Applications, 50, 4, 530--539.Google ScholarCross Ref
- Nayar, S. and Branzoi, V. 2003. Adaptive Dynamic Range Imaging: Optical Control of Pixel Exposures over Space and Time. In Proceedings of the International Conference on Computer Vision, 1--8. Google ScholarDigital Library
- Nayar, S. and Branzoi, V. 2004. Programmable Imaging Using a Digital Micromirror Array. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 436--443.Google Scholar
- Pattanaik, S. N., Tumblin, J., Yee, H. and Greenberg, D. 2000. Time Dependent Visual Adaptation for Fast Realistic Image Display. In Proceedings of ACM SIGGRAPH 2000, ACM SIGGRAPH / Addison Wesley, Computer Graphics Proceedings, 47--54. Google ScholarDigital Library
- Perona, P. and Malik, J. 1990. Scale-Space and Edge Detection Using Anisotropic Diffusion. IEEE Transactions of Pattern Matching and Machine Intelligence, 12, 7, 629--639. Google ScholarDigital Library
- Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M. F., and Toyama, K. 2004. Digital Photograph with Flash and No-Flash Pairs. ACM Transactions on Graphics, 23, 3, 661--669. Google ScholarDigital Library
- Raskar, R., Ilie, A., and Yu, J. 2004. Image Fusion for Context Enhancement and Video Surrealism. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering, 85--94. Google ScholarDigital Library
- Reibel, Y., Jung, M., Bouhifd. M., Cunin, B., and Draman, C. 2003. CCD or CMOS Camera Noise Characteristics. In Proceedings of the European Physical Journal of Applied Physics, 75--80.Google Scholar
- Sand, P. and Teller, S. 2004. Video Matching. ACM Transactions on Graphics, 23, 3, 592--599. Google ScholarDigital Library
- Stockham, T. G. 1972. Image Processing in the Context of a Visual Model, In Proceedings of the IEEE, 60, 828--842.Google ScholarCross Ref
- Tomasi, C. and Manduchi, R. 1998. Bilateral Filtering for Gray and Color Images. In Proceedings of the International Conference on Computer Vision, 836--846. Google ScholarDigital Library
- Tumblin, J. and Rushmeier, H. E. 1993. Tone Reproduction for Realistic Images. IEEE Computer Graphics and Applications, 13,6,42--48. Google ScholarDigital Library
- Tumblin, J. and Turk, G. 1999. LCIS: A boundary hierarchy for detail preserving contrast reductions. In Proceedings of SIGGRAPH 1999,83--90. Google ScholarDigital Library
- Ward, G. 1991. Real Pixels. Graphics Gems II. Academic Press. 80--83.Google Scholar
- Yee, H., Pattanaik, S, and Greenberg, D. P. 2001. Spatio-Temporal Sensitivity and Visual Attention for Efficient Rendering of Dynamic Environments. ACM Transactions on Graphics, 20, 1, 39--65. Google ScholarDigital Library
Index Terms
- Video enhancement using per-pixel virtual exposures
Recommendations
Video enhancement using per-pixel virtual exposures
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersWe enhance underexposed, low dynamic range videos by adaptively and independently varying the exposure at each photoreceptor in a post-process. This virtual exposure is a dynamic function of both the spatial neighborhood and temporal history at each ...
A design framework for hybrid approaches of image noise estimation and its application to noise reduction
Noise estimation is an important process in digital imaging systems. Many noise reduction algorithms require their parameters to be adjusted based on the noise level. Filter-based approaches of image noise estimation usually were more efficient but had ...
Multispectral Bilateral Video Fusion
We present a technique for enhancing underexposed visible-spectrum video by fusing it with simultaneously captured video from sensors in nonvisible spectra, such as Short Wave IR or Near IR. Although IR sensors can accurately capture video in low-light ...
Comments