Karen Panetta
Abstract
Fine Art Photography is one of the most popular art forms, which creates lasting impressions that elicit various human emotional reactions. Photo aesthetic enhancement aims at improving the aesthetic level of the photo to please humans by updating color appearance or modifying the geometry structure of objects within that photo. Even though several aesthetic enhancement methods have been proposed, to our knowledge, there is no research to explore, highlight, and accentuate photos’ intrinsic aesthetic value to elicit a stronger response from the human observer about the photos’ theme. To meet this challenge, a new multimedia technology called automatic color theme--based aesthetic enhancement (CT-AEA) is proposed by leveraging big online data to perform timely collection and learning of humans’ current aesthetic perception-behavior over photos and color themes in art, fashion, and design. Unlike existing aesthetic enhancement that examines the composition, such as the geometric structure of the image contents and color/luminance-related (color tone and luminance distribution) characteristics, this CT-AEA takes into consideration the importance of a suitable color theme, namely a set of dominant colors for the design when assessing the aesthetic appearance of a photo. This algorithm is composed of (1) utilizing the knowledge gained from the human evaluator's perception of beauty from existing online datasets, rather than simply applying prior existing knowledge of color harmony theory; (2) developing a new color theme difference equation that exhibits order-invariance and percentage-sensitive properties; (3) designing an optimal color theme recommendation to maximize the aesthetic performance, while minimizing the color modification cost to solve the problems of color inconsistencies and distortion. Experimental results, quantitative measure, and comparison tests demonstrate the algorithm's effectiveness, advantages, and potential for use in many color-related art and design applications.
- Y. Zhang, L. Zhang, and R. Zimmermann. 2015. Aesthetics-guided summarization from multiple user generated videos. ACM Trans. Multimedia Comput. Commun. Appl. 11 (2015), 1--23. Google Scholar
Digital Library
- C.-H. Yeh, B. A. Barsky, and M. Ouhyoung. 2014. Personalized photograph ranking and selection system considering positive and negative user feedback. ACM Trans. Multimedia Comput. Commun. Appl. 10 (2014), 1--20. Google ScholarDigital Library
- R. Schifanella, M. Redi, and L. Aiello. 2015. An image is worth more than a thousand favorites: Surfacing the hidden beauty of Flickr pictures. In Proceedings of the International AAAI Conference on Web and Social Media.Google Scholar
- J. Chen, G. Bai, S. Liang, and Z. Li. 2016. Automatic image cropping: A computational complexity study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 507--515.Google Scholar
- R. Datta, D. Joshi, J. Li, and J. Z. Wang. 2006. Studying Aesthetics in Photographic Images Using a Computational Approach. Springer Berlin, 288--301. Google ScholarDigital Library
- R. Hong, L. Zhang, and D. Tao. 2016. Unified photo enhancement by discovering aesthetic communities from Flickr. IEEE Trans. Image Proc. 25 (2016), 1124--1135.Google ScholarDigital Library
- Y. Deng, C. C. Loy, and X. Tang. 2017. Aesthetic-Driven image enhancement by adversarial learning. Retrieved from: arXiv preprint arXiv:1707.05251.Google Scholar
- S. Bhattacharya, R. Sukthankar, and M. Shah. 2011. A holistic approach to aesthetic enhancement of photographs. ACM Trans. Multimedia Comput. Commun. Appl. 7S (2011), 1--21. Google ScholarDigital Library
- M. Nishiyama, T. Okabe, I. Sato, and Y. Sato. 2011. Aesthetic quality classification of photographs based on color harmony. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’11). 33--40. Google ScholarDigital Library
- L. Wei, W. Xiaogang, and X. Tang. 2011. Content-based photo quality assessment. In Proceedings of the International Conference on Computer Vision. 2206--2213. Google ScholarDigital Library
- J. Yan, S. Lin, S. B. Kang, and X. Tang. 2013. Learning the change for automatic image cropping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 971--978. Google ScholarDigital Library
- W. Wang, J. Shen, and L. Shao. 2015. Consistent video saliency using local gradient flow optimization and global refinement. IEEE Trans. Image Proc. 24 (2015), 4185--4196.Google ScholarDigital Library
- W. Wang, J. Shen, L. Shao, and F. Porikli. 2016. Correspondence driven saliency transfer. IEEE Trans. Image Proc. 25 (2016), 5025--5034. Google ScholarDigital Library
- Y. Gao, J. Guo, T. Ren, and J. Bei. 2016. Personal photo enhancement via saliency driven color transfer. In Proceedings of the International Conference on Internet Multimedia Computing and Service. Google ScholarDigital Library
- M. B. Islam, W. Lai-Kuan, and W. Chee-Onn. 2017. A survey of aesthetics-driven image recomposition. Multimedia Tools Appl. 76 (2017), 9517--9542. Google ScholarDigital Library
- M. B. Islam, W. Lai-Kuan, W. Chee-Onn, and K. L. Low. 2015. Stereoscopic image warping for enhancing composition aesthetics. In Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition (ACPR’15). 645--649.Google Scholar
- K. Jeong and H.-J. Cho. 2016. Photo quality enhancement by relocating subjects. Cluster Comput. 19 (2016), 939--948. Google ScholarDigital Library
- L. Liu, R. Chen, L. Wolf, and D. Cohen‐Or. 2010. Optimizing photo composition. Comput. Graph. Forum. 29, 2 (2010), 469--478.Google ScholarCross Ref
- Y. W. Guo, M. Liu, T. T. Gu, and W. P. Wang. 2012. Improving photo composition elegantly: Considering image similarity during composition optimization. Comput. Graph. Forum 31, 7 (2012) 2193--2202. Google ScholarDigital Library
- L. Liu, Y. Jin, and Q. Wu. 2010. Realtime aesthetic image retargeting. In Proceedings of the 6th International Conference on Computational Aesthetics in Graphics, Visualization and Imaging. Google ScholarDigital Library
- F. L. Zhang, M. Wang, and S. M. Hu. 2013. Aesthetic image enhancement by dependence-aware object recomposition. IEEE Trans. Multimedia 15 (2013), 1480--1490. Google ScholarDigital Library
- S. H. Lim and D. A. Silverstein. 2016. Image tone adjustment using local tone curve computation, S. H. Lim and D. A. Silverstein (Eds.). U.S. Patent 9369684. Retrieved from https://patents.google.com/patent/US9369684B2/en?q=Image8q=tone8q=adjustment8q=local8q=tone8q=curve8q=computation8oq=Image+tone+adjustment+using+local+tone+curve+computation.Google Scholar
- J. Choi, S. Koh, J. Kwack, Y. Kwon, and H. Shim. 2016. Data-driven approach to aesthetic enhancement. Electron. Imag. 14 (2016), 1--5.Google Scholar
- B. Wang, Y. Yu, and Y.-Q. Xu. 2011. Example-based image color and tone style enhancement. ACM Trans. Graph. 30 (2011), 1--12. Google ScholarDigital Library
- L. Bao, K. Panetta, and S. Agaian. 2016. A new color transfer quality measure. In Proceedings of SPIE 9869, Mobile Multimedia/Image Processing, Security, and Applications 2016, 986904.Google Scholar
- K. Panetta, L. Bao, and S. Agaian. 2016. Novel multi-color transfer algorithms and quality measure. IEEE Trans. Consum. Electron. 62 (2016), 292--300. Google ScholarDigital Library
- L. Bao, K. Panetta, and S. Agaian. 2017. Fast color transfer for camouflage applications. In Proceedings of the IEEE International Symposium on Technologies for Homeland Security (HST’17). 1--5.Google Scholar
- B. Wang, Y. Yu, T.-T. Wong, C. Chen, and Y.-Q. Xu. 2010. Data-driven image color theme enhancement. ACM Trans. Graph. 29 (2010), 1--10. Google ScholarDigital Library
- H. Q. Phan, H. Fu, and A. B. Chan. 2018. Color orchestra: Ordering color palettes for interpolation and prediction. IEEE Trans. Vis. Comput. Graph. 24 (2018), 1942--1955.Google ScholarCross Ref
- C. C. Gramazio, D. H. Laidlaw, and K. B. Schloss. 2017. Colorgorical: Creating discriminable and preferable color palettes for information visualization. IEEE Trans. Vis. Comput. Graph. 23 (2017), 521--530. Google ScholarDigital Library
- K. Naz and H. Epps. 2004. Relationship between color and emotion: A study of college students. Coll. Stud. J. 38 (2004), 396.Google Scholar
- P. T.-F. Chong, C. Minchew, and K. Ewing. 2015. Color selection system based on desired color emotion and color harmony, P. T.-F. Chong, C. Minchew, and K. Ewing. U.S. Patent No. 9134179. Retrieved from https://patents.google.com/patent/US9134179B2/en.Google Scholar
- S. Liu and H. Luo. 2016. Hierarchical emotional color theme extraction. Color Res. Appl. 41 (2016), 513--522.Google ScholarCross Ref
- P. O'Donovan, A. Agarwala, and A. Hertzmann. 2014. Collaborative filtering of color aesthetics. In Proceedings of the Workshop on Computational Aesthetics. Google ScholarDigital Library
- P. O'Donovan, A. Agarwala, and A. Hertzmann. 2011. Color compatibility from large datasets. In Proceedings of the ACM Transactions on Graphics 30, 4 (2011), 63. Google ScholarDigital Library
- J. Nayak, B. Naik, and H. Behera. 2015. Fuzzy C-means (FCM) clustering algorithm: A decade review from 2000 to 2014. In Computational Intelligence in Data Mining, Vol. 2. Springer, 133--149.Google Scholar
- C. Liu, W. Huang, F. Sun, M. Luo, and C. Tan. 2019. LDS-FCM: A linear dynamical system based fuzzy c-means method for tactile recognition. IEEE Trans. Fuzzy Syst. 27 (2019), 72--83. Google ScholarDigital Library
- J. F. Mota, L. Weizman, N. Deligiannis, Y. C. Eldar, and M. R. Rodrigues. 2017. Reweighted l1-norm minimization with guarantees: An incremental measurement approach to sparse reconstruction. In Proceedings of the 6th Signal Processing with Adaptive Sparse Structured Representations Workshop (SPARS’17).Google Scholar
- A. Olmos and F. A. A. Kingdom. 2004. A biologically inspired algorithm for the recovery of shading and reflectance images. Perception 33 (2004), 1463--1473.Google ScholarCross Ref
- L. Bossard, M. Guillaumin, and L. Van Gool. 2014. Food-101—Mining discriminative components with random forests. In Proceedings of the European Conference on Computer Vision. 446--461.Google Scholar
- K. Lant. 2017. Colors in marketing and advertising. 99designs. Retrieved from https://99designs.com/blog/tips/colors-marketing-advertising/.Google Scholar
- D. Hasler and S. Süsstrunk. 2003. Measuring colourfulness in natural images. In Proceedings of the Conference on Human Vision and Electronic Imaging VIII.Google Scholar
- L. Bao, K. Panetta, and S. Agaian. 2015. A no reference image quality measure using a distance doubling variance. In Proceedings of the IEEE International Conference on Technologies for Practical Robot Applications (TePRA’15). 1--6.Google Scholar
- K. Panetta, L. Bao, and S. Agaian. 2016. A human visual no-reference image quality measure. IEEE Instrument. Measur. Magazine 19 (2016), 34--38.Google ScholarCross Ref
Index Terms
- Color Theme--based Aesthetic Enhancement Algorithm to Emulate the Human Perception of Beauty in Photos
Recommendations
Data-driven image color theme enhancement
It is often important for designers and photographers to convey or enhance desired color themes in their work. A color theme is typically defined as a template of colors and an associated verbal description. This paper presents a data-driven method for ...
Data-driven image color theme enhancement
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papersIt is often important for designers and photographers to convey or enhance desired color themes in their work. A color theme is typically defined as a template of colors and an associated verbal description. This paper presents a data-driven method for ...
Skin color enhancement based on favorite skin color in HSV color space
Skin color enhancement based on favorite skin color is proposed to make skin color displayed on large screen flat panel TVs agree with human favorite skin color. A robust skin detection method in different intensity is obtained after analyzing the ...
Comments