Superimage: Packing Semantic-Relevant Images for Indexing and Retrieval

ABSTRACT

As an important procedure in image retrieval, off-line indexing focuses on organizing relevant images together and making them easy to access. However, most of existing indexing strategies view database images individually and only consider partial relevance, i.e., either visual or semantic relevance among them. To overcome these issues and design better indexing strategy, we propose to package semantically relevant images into superimages, and then index superimages instead of single images. Superimage effectively packages multiple images into one new unit, hence significantly decreases the number of images to be indexed. This naturally saves the memory cost and retrieval time. To make the final index file discriminative to both visual and semantic relevances, we extract local descriptors from superimages and index them with inverted file. During online retrieval, we only need to extract local descriptors from queries, but could get semantic-aware retrieval results. This is because during our off-line indexing stage, both the semantically and visually relevant images are organized together. Therefore, our approach is superior to many online retrieval fusion algorithms. Experimental results on UKbench, Holidays, and one large-scale dataset all manifest the promising performance of our approach, i.e., competitive precision, better efficiency, and only about 1/2 memory consumption compared with state-of-the-arts.

References

1. Large scale visual recognition challenge. http://www.image-net.org/challenges/LSVRC/2010, 2010. 2, 3.1
2. J. Deng, A. C. Berg, and L. Fei-Fei. Hierarchical semantic indexing for large scale image retrieval. In CVPR, 2011. 1, 2
3. M. Douze, A. Ramisa, and C. Schmid. Combining attributes and fisher vectors for effcient image retrieval. In CVPR, 2011. 1, 1, 2, 2
4. C. Fellbaum. Wordnet: an electronic lexical database. Bradford Books, 1998. 2
5. B. J. Frey and D. Dueck. Clustering by passing messages between data points. Science, 315(5814):972--976, 2007. 3.2
6. A. Gionis, P. Indyky, and R. Motwaniz. Similarity search in high dimensions via hashing. In International Conference on Very Large Data Bases (VLDB), pages 518--529, 1999. 1
7. M. J. Huiskes and M. S. Lew. The mir flickr retrieval evaluation. In MIR '08: Proceedings of the 2008 ACM ICMIR, New York, NY, USA, 2008. ACM. 5.1
8. H. Jégou, M. Douze, and C. Schmid. Hamming embedding and weak geometric consistency for large scale image search. In ECCV, 2008. 2, 5.1, 2
9. H. Jégou, M. Douze, and C. Schmid. Improving bag-of-feature for large scale image search. IJCV, 87(3):316--336, 2010. 2
10. H. Jégou, M. Douze, and C. Schmid. Product quantization for nearest neighbor search. TPAMI, 33(1):117--128, 2011. 3.2
11. R. M. Karp. Reducibility among combinatorial problems. Springer, 1972. 3.2
12. A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012. 2
13. Z. Liu, H. Li, W. Zhou, and Q. Tian. Embedding spatial context into inverted file for large-scale image search. In ACM Multimedia, 2012. 1, 2, 2
14. D. G. Lowe. Distinctive image features from scale invariant keypoints. IJCV, 60(2):91--110, 2004. 1, 2
15. K. Makino and T. Uno. New algorithms for enumerating all maximal cliques. In Algorithm Theory-SWAT 2004, pages 260--272. Springer, 2004. 1, 3.2
16. T. Mei, Y. Rui, S. Li, and Q. Tian. Multimedia search reranking: A literature survey. ACM Computing Survey, 2013. 1
17. A. Y. Ng, M. I. Jordan, Y. Weiss, et al. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 2:849--856, 2002. 3.2
18. D. Nistér and H. Stewénius. Scalable recognition with a vocabulary tree. In CVPR, 2006. 1, 1, 1, 2, 4.1, 5.1, 5.2, 3
19. F. Perronnin, J. Sánchez, and T. Mensink. Improving the fisher kernel for large-scale image classification. In ECCV, volume 4, pages 143--156, 2010. 1, 2
20. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007. 2, 2
21. J. Sivic and A. Zisserman. Video google: A text retrieval approach to object matching in videos. In ICCV, 2003. 1, 2
22. E. Tomita, A. Tanaka, and H. Takahashi. The worst-case time complexity for generating all maximal cliques and computational experiments. Theoretical Computer Science, 363(1):28--42, 2006. 1, 3.2
23. A. Torralba, R. Fergus, and Y. Weiss. Small codes and large image databases for recognition. In CVPR, 2008. 1
24. L. Torresani, M. Szummer, and A. Fitzgibbon. Efficient object category recognition using classemes. In ECCV, pages 776--789, 2010. 1, 2, 3.1, 5.3
25. Z. Wu, Q. Ke, M. Isard, and J. Sun. Bundling feature for large scale partial-duplicated web image search. In CVPR, 2009. 1, 2
26. G. Ye, D. Liu, I.-H. Jhuo, and S.-F. Chang. Robust late fusion with rank minimization. In CVPR, 2012. 1, 1, 2
27. S. Zhang, Q. Huang, G. Hua, S. Jiang, W. Gao, and Q. Tian. Building contextual visual vocabulary for large-scale image applications. In ACM Multimedia, 2010. 1, 1
28. S. Zhang, Q. Tian, G. Hua, Q. Huang, and W. Gao. Descriptive visual words and visual phrases for image applications. In ACM Multimedia, 2009. 1, 2
29. S. Zhang, Q. Tian, K. Lu, Q. Huang, and W. Gao. Edge-sift: Discriminative binary descriptor for scalable partial-duplicate mobile search. TIP, 2013. 1
30. S. Zhang, M. Yang, T. Cour, K. Yu, and D. N. Metaxas. Query specific fusion for image retrieval. In ECCV, volume 2, pages 660--673, 2012. 1, 1, 2
31. S. Zhang, M. Yang, X. Wang, Y. Lin, and Q. Tian. Sematnic-aware co-indexing for image retrieval. In ICCV, 2013. 2, 3.1
32. Y. Zhang, Z. Jia, and T. Chen. Image retrieval with geometry-preserving visual phrases. In CVPR, 2011. 1, 2, 2

Index Terms

• Information systems

  • Information retrieval

    • Document representation