Michael Riegler
Martha Larson
Mathias Lux
ABSTRACT
In this paper, we introduce the concept of intentional framing, defined as the sum of the choices that a photographer makes on how to portray the subject matter of an image. We carry out analysis experiments that demonstrate the existence of a correspondence between image similarity that is calculated automatically on the basis of global feature representations, and image similarity that is perceived by humans at the level of intentional frames. Intentional framing has profound implications: The existence of a fundamental image-interpretation principle that explains the importance of global representations in capturing human-perceived image semantics reaches beyond currently dominant assumptions in multimedia research. The ability of fast global-feature approaches to compete with more `sophisticated' approaches, which are computationally more complex, is demonstrated using a simple search method (SimSea) to classify a large (2M) collection of social images by tag class. In short, intentional framing provides a principled connection between human interpretations of images and lightweight, fast image processing methods. Moving forward, it is critical that the community explicitly exploits such approaches, as the social image collections that we tackle, continue to grow larger.
- ImageCLEF Task. http://imageclef.org/2012. {lv., 08, 13}.Google Scholar
- R. Barthes. Image Music Text. Hill and Wang, 1977.Google Scholar
- A. Bosch, A. Zisserman, and X. Munoz. Representing Shape with a Spatial Pyramid Kernel. In Proceedings of the CIVR '07, pages 401--408, New York, NY, USA, 2007. Google ScholarDigital Library
- S.-F. Chang, T. Sikora, and A. Puri. Overview of the MPEG-7 Standard. IEEE Transactions on Circuits and Systems for Video Technology, 11(6):688--695, June 2001. Google ScholarDigital Library
- S. A. Chatzichristofis, Y. S. Boutalis, and M. Lux. Selection of the Proper Compact Composite Descriptor for Improving Content Based Image Retrieval. In Proceedings of the SPPRA '09, 2009.Google Scholar
- R. M. Entman. Framing: Toward Clarification of a Fractured Paradigm. Journal of communication, 43(4):51--58, 1993.Google ScholarCross Ref
- A. Friedman. Framing Pictures: The Role of Knowledge in Automatized Encoding and Memory for Gist. Journal of experimental psychology: General, 108(3):316, 1979.Google Scholar
- J. Hays and A. A. Efros. IM2GPS: Estimating Geographic Information from a Single Image. In Proceedings of the CVPR '08, pages 1--8.Google Scholar
- J. Hays and A. A. Efros. Scene Completion Using Millions of Photographs. In Proceedings of the ACM TOG '07, volume 26, page 4, 2007. Google ScholarDigital Library
- M. J. Huiskes, B. Thomee, and M. S. Lew. New Trends and Ideas in Visual Concept Detection: The MIR Flickr Retrieval Evaluation Initiative. In Proceedings of the ACM ICMR '10, pages 527--536, 2010. Google ScholarDigital Library
- G. Humphrey. The Psychology of the Gestalt. Journal of Educational Psychology, 15(7):401, 1924.Google ScholarCross Ref
- J. Kim. Events as Property Exemplifications. In Action theory, pages 159--177. Springer, 1976.Google ScholarCross Ref
- M. Larson, M. Melenhorst, M. Menéndez, and P. Xu. Using Crowdsourcing to Capture Complexity in Human Interpretations of Multimedia Content. In Fusion in Computer Vision, pages 229--269. Springer, 2014.Google ScholarCross Ref
- X. Li, M. Larson, and A. Hanjalic. Geo-visual Ranking for Location Prediction of Social Images. In Proceedings of ACM ICMR '13, pages 81--88. ACM, 2013. Google ScholarDigital Library
- X. Li, C. G. Snoek, and M. Worring. Learning Social Tag Relevance by Neighbor Voting. Multimedia, IEEE Transactions on, 11(7):1310--1322, 2009. Google ScholarDigital Library
- D. Liu, S. Yan, X.-S. Hua, and H.-J. Zhang. Image Retagging Using Collaborative Tag Propagation. Multimedia, IEEE Transactions on, 13(4):702--712, 2011. Google ScholarDigital Library
- B. Loni, L. Y. Cheung, M. Riegler, A. Bozzon, L. Gottlieb, and M. Larson. Fashion 10000: An Enriched Social Image Dataset for Fashion and Clothing. In Proceedings of ACM MMSys '14, pages 41--46, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- M. Lux, M. Kogler, and M. del Fabro. Why Did You Take This Photo: A Study on User Intentions in Digital Photo Productions. In Proceedings of SAPMIA '10, SAPMIA '10, pages 41--44, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- M. Lux and O. Marques. Visual Information Retrieval Using Java and LIRE. Synthesis Lectures on Information Concepts, Retrieval, and Services, 5(1):1--112, 2013.Google Scholar
- M. Lux, M. Taschwer, and O. Marques. A Closer Look at Photographers' Intentions: A Test Dataset. In Proceedings of the ACM CrowdMM '12, pages 17--18. ACM, 2012. Google ScholarDigital Library
- E. Mantziou, S. Papadopoulos, and Y. Kompatsiaris. Scalable Training with Approximate Incremental Laplacian Eigenmaps and PCA. In Proceedings of the ACM MM '13, pages 381--384, 2013. Google ScholarDigital Library
- T. B. Moeslund, O. Javed, Y.-G. Jiang, and R. Manmatha. Special Issue on Multimedia Event Detection. Mach. Vision Appl., 25(1):1--4, Jan. 2014. Google ScholarDigital Library
- M. Naphade, J. R. Smith, J. Tesic, S. Chang, W. Hsu, L. Kennedy, A. Hauptmann, and J. Curtis. Large-scale Concept Ontology for Multimedia. MM IEEE, pages 86--91, 2006. Google ScholarDigital Library
- A. Oliva and A. Torralba. Building the Gist of a Scene: The Role of Global Image Features in Recognition. Progress in brain research, 155:23--36, 2006.Google Scholar
- D. Pelleg, A. W. Moore, et al. X-means: Extending K-means with Efficient Estimation of the Number of Clusters. In Proceedings of ICML '00, pages 727--734, 2000. Google ScholarDigital Library
- G. Petkos, S. Papadopoulos, and Y. Kompatsiaris. Social Event Detection Using Multimodal Clustering and Integrating Supervisory Signals. In Proceedings of ACM ICMR '12, page 23, 2012. Google ScholarDigital Library
- R. A. Rensink. Scene Perception. 7:151--155, 2000.Google Scholar
- T. Reuter and P. Cimiano. Event-based Classification of Social Media Streams. In Proceedings of ACM ICMR '12, page 22, 2012. Google ScholarDigital Library
- D. A. Scheufele. Framing as a Theory of Media Effects. Journal of communication, 49(1):103--122, 1999.Google ScholarCross Ref
- Y.-C. Su, T.-H. Chiu, G.-L. Wu, C.-Y. Yeh, F. Wu, and W. Hsu. Flickr-tag Prediction Using Multi-modal Fusion and Meta Information. In Proceedings of ACM MM '13, pages 353--356, 2013. Google ScholarDigital Library
- F. Travel. Framing of Images from Photographers View. http://www.fodors.com/travel-photography/. {lv., 11, 13}.Google Scholar
- K. E. van de Sande, T. Gevers, and C. G. Snoek. Evaluating Color Descriptors for Object and Scene Recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(9):1582--1596, 2010. Google ScholarDigital Library
- S. Vinner. Concept Definition, Concept Image and the Notion of Function. International Journal of Mathematical Education in Science and Technology, 14(3):293--305, 1983.Google Scholar
- L. Wang, L. Yang, and X. Tian. Query Aware Visual Similarity Propagation for Image Search Reranking. In Proceedings of ACM MM '09, pages 725--728. ACM, 2009. Google ScholarDigital Library
- L. Yang and A. Hanjalic. Supervised Reranking for Web Image Search. In Proceedings of ACM MM '10, pages 183--192. ACM, 2010. Google ScholarDigital Library
Index Terms
- How 'How' Reflects What's What: Content-based Exploitation of How Users Frame Social Images
Recommendations
Does Haze Removal Help CNN-Based Image Classification?
Computer Vision – ECCV 2018AbstractHazy images are common in real scenarios and many dehazing methods have been developed to automatically remove the haze from images. Typically, the goal of image dehazing is to produce clearer images from which human vision can better identify the ...
Blurred Image Recognition: A Joint Motion Deblurring and Classification Loss-Aware Approach
Artificial Neural Networks and Machine Learning – ICANN 2021AbstractImage motion blur can severely affect the performance of the image recognition model. Traditional methods to tackle this problem usually involve image motion deblurring to improve the image quality before its recognition. However, traditional ...
No-Reference Image Blur Assessment in the DWT Domain and Blurred Image Classification
ITNG '15: Proceedings of the 2015 12th International Conference on Information Technology - New GenerationsWe propose several new no-reference/blind image quality indices for blur assessment based on the discrete wavelet transform (DWT) and demonstrate that a given image can be classified based on blur by using these indices. Our approach relies on the ...
Comments