Exploring Consistent Preferences: Discrete Hashing with Pair-Exemplar for Scalable Landmark Search

ABSTRACT

Content-based visual landmark search (CBVLS) enjoys great importance in many practical applications. In this paper, we propose a novel discrete hashing with pair-exemplar (DHPE) to support scalable and efficient large-scale CBVLS. Our approach mainly solves two essential problems in scalable landmark hashing: 1) Intra-landmark visual diversity, and 2) Discrete optimization of hashing codes. Motivated by the characteristic of landmark, we explore the consistent preferences of tourists on landmark as pair-exemplars for scalable discrete hashing learning. In this paper, a pair-exemplar is comprised of a canonical view and the corresponding representative tags. Canonical view captures the key visual component of landmarks, and representative tags potentially involve landmark-specific semantics that can cope with the visual variations of intra-landmark. Based on pair-exemplars, a unified hashing learning framework is formulated to combine visual preserving with exemplar graph and the semantic guidance from representative tags. Further, to guarantee direct semantic transfer for hashing codes and remove information redundancy, we design a novel optimization method based on augmented Lagrange multiplier to explicitly deal with the discrete constraint, the bit-uncorrelated constraint and balance constraint. The whole learning process has linear computation complexity and enjoys desirable scalability. Experiments demonstrate the superior performance of DHPE compared with state-of-the-art methods.

References

1. Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Found. Trends Mach. Learn. Vol. 3, 1 (2011), 1--122.
2. Rick Chartrand. 2012. Nonconvex Splitting for Regularized Low-Rank Sparse Decomposition. IEEE Trans. Signal Process. Vol. 60, 11 (2012), 5810--5819.
3. Tao Chen, Kim-Hui Yap, and Dajiang Zhang. 2014. Discriminative Soft Bag-of-Visual Phrase for Mobile Landmark Recognition. IEEE Trans. Multimedia Vol. 16, 3 (2014), 612--622.
4. Zhiyong Cheng and Jialie Shen. 2016. On very large scale test collection for landmark image search benchmarking. Signal Process. Vol. 124 (2016), 13 -- 26.
5. David Crandall, Yunpeng Li, Stefan Lee, and Daniel Huttenlocher. 2016. Recognizing landmarks in large-scale social image collections Visual Analysis and Geolocalization of Large Scale Imagery, Asaad Hakeem, Richard Szeliski, Mubarak Shah, Luc Van Gool, and Amir Zamir (Eds.). Springer.
6. G. Ding, Y. Guo, J. Zhou, and Y. Gao. 2016. Large-Scale Cross-Modality Search via Collective Matrix Factorization Hashing. IEEE Trans. Image Process. Vol. 25, 11 (2016), 5427--5440.
7. Ling-Yu Duan, Jie Chen, Rongrong Ji, Tiejun Huang, and Wen Gao. 2013. Learning Compact Visual Descriptors for Low Bit Rate Mobile Landmark Search. AI Magazine, Vol. 34, 2 (2013), 67--85.
8. Y. Gong, S. Kumar, H. A. Rowley, and S. Lazebnik. 2013 a. Learning Binary Codes for High-Dimensional Data Using Bilinear Projections CVPR. 484--491.
9. Yunchao Gong, S. Lazebnik, A. Gordo, and F. Perronnin. 2013 b. Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-Scale Image Retrieval. IEEE Trans. Pattern Anal. Mach. Intell. Vol. 35, 12 (2013), 2916--2929.
10. Rongrong Ji, Ling-Yu Duan, Jie Chen, Hongxun Yao, Junsong Yuan, Yong Rui, and Wen Gao. 2012. Location Discriminative Vocabulary Coding for Mobile Landmark Search. Int. J. Comput. Vision Vol. 96, 3 (2012), 290--314.
11. Qing-Yuan Jiang and Wu-Jun Li. 2015. Scalable Graph Hashing with Feature Transformation. IJCAI. 2248--2254.
12. Z. Jin, C. Li, Y. Lin, and D. Cai. 2014. Density Sensitive Hashing. IEEE Trans. Cybern. Vol. 44, 8 (2014), 1362--1371.
13. Wang-Cheng Kang, Wu-Jun Li, and Zhi-Hua Zhou. 2016. Column Sampling Based Discrete Supervised Hashing. AAAI. 1230--1236.
14. Shaishav Kumar and Raghavendra Udupa. 2011. Learning Hash Functions for Cross-View Similarity Search. IJCAI. 1360--1365.
15. Jingjing Li, Yue Wu, Jidong Zhao, and Ke Lu. 2016 a. Low-rank discriminant embedding for multiview learning. IEEE Trans. Cybern. (2016).
16. Jingjing Li, Jidong Zhao, and Ke Lu. 2016 b. Joint Feature Selection and Structure Preservation for Domain Adaptation. IJCAI. 1697--1703.
17. Wei Liu, Junfeng He, and Shih-Fu Chang. 2010. Large Graph Construction for Scalable Semi-Supervised Learning ICML. 679--686.
18. Wei Liu, Wang Jun, Sanjiv Kumar, and Shih-Fu Chang. 2011. Hashing with Graphs. ICML. 1--8.
19. Wei Liu, Cun Mu, Sanjiv Kumar, and Shih-Fu Chang. 2014. Discrete Graph Hashing. In NIPS. 3419--3427.
20. Yadan Luo, Yang Yang, Fumin Shen, Zi Huang, Pan Zhou, and Heng Tao Shen. 2017. Robust discrete code modeling for supervised hashing. Pattern Recognit. (2017).
21. Yadong Mu, Wei Liu, Cheng Deng, Zongting Lv, and Xinbo Gao. 2016. Coordinate Discrete Optimization for Efficient Cross-View Image Retrieval IJCAI. 1860--1866.
22. Maxim Raginsky and Svetlana Lazebnik. 2009. Locality-sensitive binary codes from shift-invariant kernels NIPS. 1509--1517.
23. Fumin Shen, Wei Liu, Shaoting Zhang, Yang Yang, and Heng Tao Shen. 2015 a. Learning Binary Codes for Maximum Inner Product Search ICCV. 4148--4156.
24. Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. 2015 b. Supervised Discrete Hashing. In CVPR. 37--45.
25. F. Shen, X. Zhou, Y. Yang, J. Song, H. T. Shen, and D. Tao. 2016. A Fast Optimization Method for General Binary Code Learning. IEEE Trans. Image Process. Vol. 25, 12 (2016), 5610--5621.
26. Xiaoshuang Shi, Fuyong Xing, Jinzheng Cai, Zizhao Zhang, Yuanpu Xie, and Lin Yang. 2016. Kernel-Based Supervised Discrete Hashing for Image Retrieval ECCV. 419--433.
27. Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR Vol. abs/1409.1556 (2014).
28. Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Heng Tao Shen. 2013. Inter-media Hashing for Large-scale Retrieval from Heterogeneous Data Sources SIGMOD. 785--796.
29. J. Wang, T. Zhang, j. song, N. Sebe, and H. T. Shen. 2017. A Survey on Learning to Hash. IEEE Trans. Pattern Anal. Mach. Intell. (2017).
30. Yair Weiss, Antonio Torralba, and Robert Fergus. 2008. Spectral Hashing NIPS. 1753--1760.
31. Tobias Weyand and Bastian Leibe. 2013. Discovering Details and Scene Structure with Hierarchical Iconoid Shift ICCV. 3479--3486.
32. Tobias Weyand and Bastian Leibe. 2015. Visual landmark recognition from Internet photo collections: A large-scale evaluation. Comput. Vis. Image Underst. Vol. 135 (2015), 1--15.
33. Yan Xia, K. He, P. Kohli, and J. Sun. 2015. Sparse projections for high-dimensional binary codes CVPR. 3332--3339.
34. Liang Xie, Jialie Shen, and Lei Zhu. 2016 a. Online Cross-Modal Hashing for Web Image Retrieval AAAI. 294--300.
35. Liang Xie, Lei Zhu, and Guoqi Chen. 2016 b. Unsupervised multi-graph cross-modal hashing for large-scale multimedia retrieval. Multimedia Tools Appl. Vol. 75, 15 (2016), 9185--9204.
36. Liang Xie, Lei Zhu, Peng Pan, and Yansheng Lu. 2016 c. Cross-Modal Self-Taught Hashing for large-scale image retrieval. Signal Process. Vol. 124 (2016), 81--92.
37. Yang Yang, Yadan Luo, Weilun Chen, Fumin Shen, Jie Shao, and Heng Tao Shen. 2016 a. Zero-Shot Hashing via Transferring Supervised Knowledge MM. 1286--1295.
38. Yang Yang, Fumin Shen, Zi Huang, and Heng Tao Shen. 2016 b. A Unified Framework for Discrete Spectral Clustering IJCAI. 2273--2279.
39. Felix X. Yu, Sanjiv Kumar, Yunchao Gong, and Shih-Fu Chang. 2014. Circulant Binary Embedding. In ICML. 946--954.
40. Jile Zhou, Guiguang Ding, and Yuchen Guo. 2014 a. Latent Semantic Sparse Hashing for Cross-modal Similarity Search SIGIR. 415--424.
41. Wengang Zhou, Ming Yang, Houqiang Li, Xiaoyu Wang, Yuanqing Lin, and Qi Tian. 2014 b. Towards Codebook-Free: Scalable Cascaded Hashing for Mobile Image Search. IEEE Trans. Multimedia Vol. 16, 3 (2014), 601--611.
42. Lei Zhu, Jialie She, Xiaobai Liu, Liang Xie, and Liqiang Nie. 2016. Learning Compact Visual Representation with Canonical Views for Robust Mobile Landmark Search. In IJCAI. 3959--3965.
43. Lei Zhu, Jialie Shen, Hai Jin, Liang Xie, and Ran Zheng. 2015 a. Landmark Classification With Hierarchical Multi-Modal Exemplar Feature. IEEE Trans. Multimedia Vol. 17, 7 (2015), 981--993.
44. Lei Zhu, Jialie Shen, Hai Jin, Ran Zheng, and Liang Xie. 2015 b. Content-Based Visual Landmark Search via Multimodal Hypergraph Learning. IEEE Trans. Cybern. Vol. 45, 12 (2015), 2756--2769.
45. L. Zhu, J. Shen, L. Xie, and Z. Cheng. 2016. Unsupervised Topic Hypergraph Hashing for Efficient Mobile Image Retrieval. IEEE Trans. Cybern. (2016).
46. Lei Zhu, Jialie Shen, Liang Xie, and Zhiyong Cheng 2017. Unsupervised Visual Hashing with Semantic Assistant for Content-Based Image Retrieval. IEEE Trans. on Knowl. and Data Eng. Vol. 29, 2 (2017), 472--486.
47. Xiaofeng Zhu, Zi Huang, Heng Tao Shen, and Xin Zhao. 2013. Linear Cross-modal Hashing for Efficient Multimedia Search MM. 143--152.
48. X. Zhu, L. Zhang, and Z. Huang. 2014. A Sparse Embedding and Least Variance Encoding Approach to Hashing. IEEE Trans. Image Process. Vol. 23, 9 (2014), 3737--3750.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

        • Visual content-based indexing and retrieval

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Image search