Personalized Visual Vocabulary Adaption for Social Image Retrieval

ABSTRACT

With the popularity of mobile devices and social networks, users can easily build their personalized image sets. Thus, personalized image analysis, indexing, and retrieval have become important topics in social media analysis. Because of users' diverse preferences, their personalized image sets are usually related to specific topics and show large feature distribution bias from general Internet images. Therefore, the visual vocabulary trained on general Internet images may could not fit across users' personalized image sets very well. To improve the image retrieval performance on personalized image sets, we propose the personalized visual vocabulary adaption which removes non-discriminative visual words and replaces them with more exact and discriminative ones, i.e., adapt a general vocabulary toward a specific user's image set. The proposed algorithm updates the visual vocabulary during off-line feature quantization, and operates on a limited number of visual words, hence shows satisfying efficiency. Extensive experiments of image search on public datasets demonstrate the efficiency and superior performance of our approach.

References

1. R. Arandjelovi and A. Zisserman. All about vlad. In CVPR, 2013.
2. J. Cai, Q. Liu, C. Francine, D. Joshi, and Q. Tian. Scalable image search with multiple index tables. In ICMR, 2014.
3. P. Cui, S. Liu, W. Zhu, H. Luan, T.-S. Chua, and S. Yang. Social-sensed image search. In ACM Transactions on Information System, 2014.
4. H. Jégou, M. Douze, and C. Schmid. Hamming embedding and weak geometric consistency for large scale image search. In ECCV, 2008.
5. D. G. Lowe. Distinctive image features from scale invariant keypoints. IJCV, 60(2):91--110, 2004.
6. D. Nistér and H. Stewénius. Scalable recognition with a vocabulary tree. In CVPR, 2006.
7. Z. Niu, G. Hua, X. Gao, and Q. Tian. Context aware topic model for scene recognition. In CVPR, 2012.
8. Z. Niu, G. Hua, X. Gao, and Q. Tian. Semi-supervised relational topic model for weakly annotated image recognition in social media. In CVPR, 2014.
9. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007.
10. X. Shen, Z. Lin, J. Brandt, S. Avidan, and Y. Wu. Object retrieval and localization with spatially-constrained similarity measure and k-NN reranking. In CVPR, 2012.
11. J. Sivic and A. Zisserman. Video google: A text retrieval approach to object matching in videos. In ICCV, 2003.
12. X. Wang, M. Yang, T. Cour, S. Zhu, K. Yu, and T. X. Han. Contextual weighting for vocabulary tree based image retrieval. In ICCV, 2011.
13. S. Zhang, Q. Huang, G. Hua, S. Jiang, W. Gao, and Q. Tian. Building contextual visual vocabulary for large-scale image applications. In ACM Multimedia, 2010.
14. S. Zhang, Q. Tian, G. Hua, Q. Huang, and W. Gao. Descriptive visual words and visual phrases for image applications. In ACM Multimedia, 2009.
15. S. Zhang, M. Yang, T. Cour, K. Yu, and D. N. Metaxas. Query specific fusion for image retrieval. In ECCV, volume 2, pages 660--673, 2012.
16. S. Zhang, M. Yang, X. Wang, Y. Lin, and Q. Tian. Semantic-aware co-indexing for image retrieval. In ICCV, 2013.
17. L. Zheng and S. Wang. Visual phraselet: Refining spatial constraints for large scale image search. In Signal Processing Letters, 2013.

Index Terms

• Information systems

  • Information retrieval

    • Document representation

      • Dictionaries