Surface flows for image-based shading design

Abstract

We present a novel method for producing convincing pictures of shaded objects based entirely on 2D image operations. This approach, which we call image-based shading design, offers direct artistic control in the picture plane by deforming image primitives so that they appear to conform to specific 3D shapes. Using a differential analysis of reflected radiance, we identify the two types of surface flows involved in the depiction of shaded objects, which are consistent with recent perceptual studies. We then introduce two novel deformation operators that closely mimic surface flows while providing direct artistic controls in real-time.

References

1. Anderson, B. L., and Kim, J. 2009. Image statistics do not explain the perception of gloss and lightness. J. Vis. 9, 11, 1--17.
2. Anjyo, K., Wemler, S., and Baxter, W. 2006. Tweakable light and shade for cartoon animation. In Proc. symposium on Non-photorealistic animation and rendering, ACM, 133--139.
3. Bigün, J., and Granlund, G. 1987. Optimal orientation detection of linear symmetry. In Proc. IEEE First International Conference on Computer Vision, 433--438.
4. Breton, P., and Zucker, S. W. 1996. Shadows and shading flow fields. In IEEE Conf. Computer Vision and Pattern Recognition, 782--789.
5. Colbert, M., Pattanaik, S., and Krivanek, J. 2006. BRDF-Shop: Creating Physically Correct Bidirectional Reflectance Distribution Functions. IEEE Comput. Graph. Appl. 26, 30--36.
6. DelPozo, A., and Savarese, S. 2007. Detecting specular surfaces on natural images. In IEEE Conf. Computer Vision and Pattern Recognition, 18--23.
7. Fang, H., and Hart, J. C. 2004. Textureshop: texture synthesis as a photograph editing tool. ACM Trans. Graph. 23, 354--359.
8. Fleming, R. W., Torralba, A., and Adelson, E. H. 2004. Specular reflections and the perception of shape. J. Vis. 4, 9, 798--820.
9. Fleming, R. W., Torralba, A., and Adelson, E. H. 2009. Shape from sheen. Tech. rep., MIT-CSAIL-TR-2009-051.
10. Fleming, R. W., Holtmann-Rice, D., and Bülthoff, H. H. 2011. Estimation of 3D shape from image orientations. Proc. National Academy of Sciences 108, 51, 20438--20443.
11. Fleming, R., Jäkel, F., and Maloney, L. 2011. Visual perception of thick transparent materials. Psychological Science 22, 6, 812--820.
12. Gibson, J. J. 1950. The Perception of the Visual World. Houghton Mifflin.
13. Huggins, P. S., and Zucker, S. W. 2001. Folds and cuts: How shading flows into edges. In Proc. IEEE Int. Conf. Computer Vision, 153--158.
14. Jeschke, S., Cline, D., and Wonka, P. 2011. Estimating color and texture parameters for vector graphics. Computer Graphics Forum 30, 2, 523--532.
15. Johnston, S. F. 2002. Lumo: illumination for cel animation. In Proc. symposium on Non-photorealistic animation and rendering, ACM, 45--ff.
16. Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Trans. Graph. 25, 654--663.
17. Kim, J., and Anderson, B. L. 2010. Image statistics and the perception of surface gloss and lightness. J. Vis. 10, 9, 3.
18. Kim, J., Marlow, P., and Anderson, B. 2011. The perception of gloss depends on highlight congruence with surface shading. J Vis 11, 9.
19. Koenderink, J. J., and Van Doorn, A. J. 1980. Photometric invariants related to solid shape. Optica Acta 27, 7, 981--996.
20. Li, A., and Zaidi, Q. 2000. Perception of three-dimensional shape from texture is based on patterns of oriented energy. Vision Research 40, 2, 217--42.
21. McCann, J., and Pollard, N. S. 2008. Real-time gradient-domain painting. ACM Trans. Graph. 27, 3, 93:1--93:7.
22. Motoyoshi, I., Nishida, S., and Adelson, E. H. 2005. Luminance re-mapping for the control of apparent material. In Proc. Symposium on Applied Perception in Graphics and Visualization, ACM, 165--165.
23. Motoyoshi, I. 2010. Highlight-shading relationship as a cue for the perception of translucent and transparent materials. J Vis 10, 9, 6.
24. Nicodemus, F. E., Richmond, J. C., Hsia, J. J., Ginsberg, I. W., and Limperis, T. 1977. Geometric Considerations and Nomenclature for Reflectance. National Bureau of Standards.
25. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In ACM Trans. Graph., 433--442.
26. Okabe, M., Matsushita, Y., Shen, L., and Igarashi, T. 2007. Illumination Brush: Interactive Design of All-Frequency Lighting. In Pacific Conference on Computer Graphics and Applications, 171--180.
27. Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., and Salesin, D. 2008. Diffusion curves: a vector representation for smooth-shaded images. ACM Trans. Graph. 27, 3, 92:1--92:8.
28. Pacanowski, R., Granier, X., Schlick, C., and Poulin, P. 2008. Sketch and Paint-based Interface for Highlight Modeling. In Eurographics workshop on Sketch-Based Interfaces and Modeling, 17--23.
29. Pellacini, F., Battaglia, F., Morley, R. K., and Finkelstein, A. 2007. Lighting with paint. ACM Trans. Graph. 26, 2.
30. Pellacini, F. 2010. envylight: an interface for editing natural illumination. ACM Trans. Graph. 29, 34:1--34:8.
31. Pharr, M., and Humphreys, G. 2010. Physically Based Rendering, Second Edition: From Theory To Implementation, 2nd ed. Morgan Kaufmann Publishers Inc.
32. Quixel, 2011. ndo2. http://quixel.se/.
33. Ramamoorthi, R., Mahajan, D., and Belhumeur, P. 2007. A first-order analysis of lighting, shading, and shadows. ACM Trans. Graph. 26, 1.
34. Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: towards a new standard for image fidelity. ACM Trans. Graph. 26, 3.
35. Ritschel, T., Okabe, M., Thormählen, T., and Seidel, H.-P. 2009. Interactive reflection editing. ACM Trans. Graph. 28, 5, 129:1--129:7.
36. Ritschel, T., Thormählen, T., Dachsbacher, C., Kautz, J., and Seidel, H.-P. 2010. Interactive on-surface signal deformation. ACM Trans. Graph. 29, 4.
37. Schlick, C. 1994. Graphics Gems V. Morgan Kauffman, ch. A Fast Alternative to Phong's Specular Model, 385--384.
38. Shao, C., Bousseau, A., Sheffer, A., and Singh, K. 2012. Crossshade: Shading concept sketching using cross-section curves. ACM Transactions on Graphics (Proc. SIGGRAPH) 31, 4 (Aug.).
39. Shepard, D. 1968. A two-dimensional interpolation function for irregularly-spaced data. In Proc. 1968 23rd ACM national conference, 517--524.
40. Sloan, P.-P. J., Martin, W., Gooch, A., and Gooch, B. 2001. The lit sphere: a model for capturing npr shading from art. In Graphics interface, 143--150.
41. Sun, J., Liang, L., Wen, F., and Shum, H.-Y. 2007. Image vectorization using optimized gradient meshes. ACM Trans. Graph. 26, 3, 11.
42. Tappen, M. F. 2011. Recovering shape from a single image of a mirrored surface from curvature constraints. In IEEE Conf. Computer Vision and Pattern Recognition, 2545--2552.
43. Todd, J. T., and Akerstrom, R. A. 1987. Perception of three-dimensional form from patterns of optical texture. J. experimental psychology. Human perception and performance 13, 2, 242--255.
44. Todd, J. T., and Mingolla, E. 1983. Perception of Surface Curvature and Direction of Illuminant from Patterns of Shading. J. Experimental Psychology: Human Perception and Performance 9, 583--595.
45. Toler-Franklin, C., Finkelstein, A., and Rusinkiewicz, S. 2007. Illustration of complex real-world objects using images with normals. In Proc. symposium on Non-photorealistic animation and rendering, ACM, 111--119.
46. Vasilyev, Y., Zickler, T., Gortler, S. J., and BenShahar, O. 2011. Shape from specular flow: Is one flow enough? In IEEE Conf. Computer Vision and Pattern Recognition, 2561--2568.
47. Vergne, R., Pacanowski, R., Barla, P., Granier, X., and Schlick, C. 2011. Improving Shape Depiction under Arbitrary Rendering. IEEE Trans. Visualization and Computer Graphics 17, 8, 1071--1081.
48. Winnemöller, H., Orzan, A., Boissieux, L., and Thollot, J. 2009. Texture Design and Draping in 2D Images. Computer Graphics Forum 28, 4, 1091--1099.
49. Wu, T.-P., Tang, C.-K., Brown, M. S., and Shum, H.-Y. 2007. ShapePalettes: interactive normal transfer via sketching. ACM Trans. Graph. 26, 3, 44.
50. Yeung, S. K., Tang, C.-K., Brown, M. S., and Kang, S. B. 2011. Matting and compositing of transparent and refractive objects. ACM Trans. Graph. 30, 1, 2:1--2:13.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Rendering