Long-tail Hashtag Recommendation for Micro-videos with Graph Convolutional Network

ABSTRACT

Hashtags, a user provides to a micro-video, are the ones which can well describe the semantics of the micro-video's content in his/her mind. At the same time, hashtags have been widely used to facilitate various micro-video retrieval scenarios (e.g., search, browse, and categorization). Despite their importance, numerous micro-videos lack hashtags or contain inaccurate or incomplete hashtags. In light of this, hashtag recommendation, which suggests a list of hashtags to a user when he/she wants to annotate a post, becomes a crucial research problem. However, little attention has been paid to micro-video hashtag recommendation, mainly due to the following three reasons: 1) lack of benchmark dataset; 2) the temporal and multi-modality characteristics of micro-videos; and 3) hashtag sparsity and long-tail distributions. In this paper, we recommend hashtags for micro-videos by presenting a novel multi-view representation interactive embedding model with graph-based information propagation. It is capable of boosting the performance of micro-videos hashtag recommendation by jointly considering the sequential feature learning, the video-user-hashtag interaction, and the hashtag correlations. Extensive experiments on a constructed dataset demonstrate our proposed method outperforms state-of-the-art baselines. As a side research contribution, we have released our dataset and codes to facilitate the research in this community.

References

1. Jingyuan Chen, Xuemeng Song, Liqiang Nie, Xiang Wang, Hanwang Zhang, and Tat-Seng Chua. 2016. Micro Tells Macro: Predicting the Popularity of Micro-Videos via a Transductive Model. In Proceedings of ACM Conference on Multimedia Conference. 898--907.
2. Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. 2019. Multi-Label Image Recognition with Graph Convolutional Networks. arXiv preprint arXiv:1904.03582 (2019).
3. Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose Catherine Kanjirathinkal, and Mohan S. Kankanhalli. [n. d.]. MMALFM: Explainable Recommendation by Leveraging Reviews and Images. ACM Transactions on Information Systems ([n. d.]).
4. Zhiyong Cheng, Ying Ding, Xiangnan He, Lei Zhu, Xuemeng Song, and Mohan S. Kankanhalli. 2018a. A^ 3NCF: An Adaptive Aspect Attention Model for Rating Prediction. In Proceedings of the International Joint Conference on Artificial Intelligence . 3748--3754.
5. Zhiyong Cheng, Ying Ding, Lei Zhu, and Mohan S. Kankanhalli. 2018b. Aspect-Aware Latent Factor Model: Rating Prediction with Ratings and Reviews. In The World Wide Web Conference. 639--648.
6. Emily Denton, Jason Weston, Manohar Paluri, Lubomir D. Bourdev, and Rob Fergus. 2015. User Conditional Hashtag Prediction for Images. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . 1731--1740.
7. Zhuoye Ding, Xipeng Qiu, Qi Zhang, and Xuanjing Huang. 2013. Learning Topical Translation Model for Microblog Hashtag Suggestion. In Proceedings of International Joint Conference on Artificial Intelligence . 2078--2084.
8. Miles Efron. 2010. Hashtag retrieval in a microblogging environment. In Proceeding of ACM SIGIR Conference on Research and Development in Information Retrieval . 787--788.
9. Tian Gan, Junnan Li, Yongkang Wong, and Mohan S. Kankanhalli. 2019. A Multi-sensor Framework for Personal Presentation Analytics. Transaction on Multimedia Computing, Communications, and Applications , Vol. 15, 2 (2019), 30:1--30:21.
10. Tian Gan, Yongkang Wong, Bappaditya Mandal, Vijay Chandrasekhar, and Mohan S. Kankanhalli. 2015. Multi-sensor Self-Quantification of Presentations. In ACMMM. 601--610.
11. Tian Gan, Yongkang Wong, Daqing Zhang, and Mohan S. Kankanhalli. 2013. Temporal encoded F-formation system for social interaction detection. In Proceedings of ACM International Conference on Multimedia. 937--946.
12. Yuqi Gao, Jitao Sang, Tongwei Ren, and Changsheng Xu. 2017. Hashtag-centric Immersive Search on Social Media. In Proceedings of ACM on Multimedia Conference . 1924--1932.
13. Yuyun Gong and Qi Zhang. 2016. Hashtag Recommendation Using Attention-Based Convolutional Neural Network. In Proceedings of International Joint Conference on Artificial Intelligence . 2782--2788.
14. Yeyun Gong, Qi Zhang, and Xuanjing Huang. 2015. Hashtag Recommendation Using Dirichlet Process Mixture Models Incorporating Types of Hashtags. In Proceedings of Conference on Empirical Methods in Natural Language Processing. 401--410.
15. Elaheh Malekzadeh Hamedani and Marjan Kaedi. 2019. Recommending the long tail items through personalized diversification. Knowledge Based Systems (2019), 348--357.
16. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770--778.
17. Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In Proceedings of International Conference on World Wide Web. 173--182.
18. Yu-Chieh Ho, Yi-Ting Chiang, and Jane Yung-jen Hsu. 2014. Who likes it more?: mining worth-recommending items from long tails by modeling relative preference. In ACM International Conference on Web Search and Data Mining. 253--262.
19. Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations .
20. Svetlana Kordumova, Jan C. van Gemert, and Cees G. M. Snoek. 2016. Exploring the Long Tail of Social Media Tags. In MultiMedia Modeling. 51--62.
21. Su Mon Kywe, Tuan-Anh Hoang, Ee-Peng Lim, and Feida Zhu. 2012. On Recommending Hashtags in Twitter Networks. In Social Informatics. 337--350.
22. Chung-Wei Lee, Wei Fang, Chih-Kuan Yeh, and Yu-Chiang Frank Wang. 2018. Multi-Label Zero-Shot Learning With Structured Knowledge Graphs. In IEEE Conference on Computer Vision and Pattern Recognition. 1576--1585.
23. Chenliang Li, Cong Quan, Li Peng, Yunwei Qi, Yuming Deng, and Libing Wu. 2019. A Capsule Network for Recommendation and Explaining What You Like and Dislike. Computing Research Repository (2019).
24. Qiang Li, Maoying Qiao, Wei Bian, and Dacheng Tao. 2016b. Conditional Graphical Lasso for Multi-label Image Classification. In IEEE Conference on Computer Vision and Pattern Recognition. 2977--2986.
25. Xin Li, Feipeng Zhao, and Yuhong Guo. 2014. Multi-label Image Classification with A Probabilistic Label Enhancement Model. In Proceedings of Conference on Uncertainty in Artificial Intelligence. 430--439.
26. Yang Li, Ting Liu, Jing Jiang, and Liang Zhang. 2016a. Hashtag Recommendation with Topical Attention-Based LSTM. In International Conference on Computational Linguistics. 3019--3029.
27. Jie Liu, Zhicheng He, and Yalou Huang. 2018. Hashtag2Vec: Learning Hashtag Representation with Relational Hierarchical Embedding Model. In Proceedings of International Joint Conference on Artificial Intelligence. 3456--3462.
28. Meng Liu, Liqiang Nie, Meng Wang, and Baoquan Chen. 2017. Towards Micro-video Understanding by Joint Sequential-Sparse Modeling. In Proceedings of the ACM International Conference on Multimedia. 970--978.
29. Dhruv Mahajan, Vishwajit Kolathur, Chetan Bansal, Suresh Parthasarathy, Sundararajan Sellamanickam, S. Sathiya Keerthi, and Johannes Gehrke. 2016. Hashtag Recommendation for Enterprise Applications. In Proceedings of ACM Conference on Information and Knowledge Management. 893--902.
30. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Annual Conference on Neural Information Processing Systems. 3111--3119.
31. Minseok Park, Hanxiang Li, and Junmo Kim. 2016. HARRISON: A Benchmark on HAshtag Recommendation for Real-world Images in Social Networks. Computing Research Repository , Vol. abs/1605.05054 (2016).
32. Yoon-Joo Park. 2013. The Adaptive Clustering Method for the Long Tail Problem of Recommender Systems. IEEE Transactions on Knowledge and Data Engineering , Vol. 25, 8 (2013), 1904--1915.
33. Yogesh Singh Rawat and Mohan S. Kankanhalli. 2016. ConTagNet: Exploiting User Context for Image Tag Recommendation. In Proceedings of ACM Conference on Multimedia Conference. 1102--1106.
34. Ruslan Salakhutdinov, Antonio Torralba, and Joshua B. Tenenbaum. 2011. Learning to share visual appearance for multiclass object detection. In IEEE Conference on Computer Vision and Pattern Recognition. 1481--1488.
35. Bidisha Samanta, Abir De, Abhijnan Chakraborty, and Niloy Ganguly. 2017. LMPP: A Large Margin Point Process Combining Reinforcement and Competition for Modeling Hashtag Popularity. In Proceedings of International Joint Conference on Artificial Intelligence. 2679--2685.
36. Lei Shi. 2013. Trading-off among accuracy, similarity, diversity, and long-tail: a graph-based recommendation approach. In ACM Conference on Recommender Systems . 57--64.
37. Andreas Veit, Maximilian Nickel, Serge J. Belongie, and Laurens van der Maaten. 2018. Separating Self-Expression and Visual Content in Hashtag Supervision. In IEEE Conference on Computer Vision and Pattern Recognition. 5919--5927.
38. Xiaolong Wang, Furu Wei, Xiaohua Liu, Ming Zhou, and Ming Zhang. 2011. Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach. In Proceedings of ACM Conference on Information and Knowledge Management. 1031--1040.
39. Yilin Wang, Suhang Wang, Jiliang Tang, Guo-Jun Qi, Huan Liu, and Baoxin Li. 2017. CLARE: A Joint Approach to Label Classification and Tag Recommendation. In Proceedings of AAAI Conference on Artificial Intelligence. 210--216.
40. Yinwei Wei, Xiang Wang, Weili Guan, Liqiang Nie, Zhouchen Lin, and Baoquan Chen. 2019. Neural Multimodal Cooperative Learning Towards Micro-video Understanding. IEEE Transactions on Image Processing (2019).
41. Lei Wu, Linjun Yang, Nenghai Yu, and Xian-Sheng Hua. 2009. Learning to tag. In Proceedings of International Conference on World Wide Web. 361--370.
42. Zhibiao Wu and Martha Palmer. 1994. Verbs Semantics and Lexical Selection. In Proceedings of Annual Meeting on Association for Computational Linguistics. 133--138.
43. Chen Xing, Yuan Wang, Jie Liu, Yalou Huang, and Wei-Ying Ma. 2016. Hashtag-Based Sub-Event Discovery Using Mutually Generative LDA in Twitter. In Proceedings of AAAI Conference on Artificial Intelligence. 2666--2672.
44. Toshihiko Yamasaki, Jiani Hu, Shumpei Sano, and Kiyoharu Aizawa. 2017. FolkPopularityRank: Tag Recommendation for Enhancing Social Popularity using Text Tags in Content Sharing Services. In Proceedings of International Joint Conference on Artificial Intelligence. 3231--3237.
45. Jianglong Zhang, Liqiang Nie, Xiang Wang, Xiangnan He, Xianglin Huang, and Tat-Seng Chua. 2016. Shorter-is-Better: Venue Category Estimation from Micro-Video. In Proceedings of ACM Conference on Multimedia Conference. 1415--1424.
46. Qi Zhang, Jiawen Wang, Haoran Huang, Xuanjing Huang, and Yeyun Gong. 2017. Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network. In Proceedings of the International Joint Conference on Artificial Intelligence . 3420--3426.
47. Xiangxin Zhu, Dragomir Anguelov, and Deva Ramanan. 2014. Capturing Long-Tail Distributions of Object Subcategories. In IEEE Conference on Computer Vision and Pattern Recognition. 915--922.
48. Xiaoyu Zhu, Tian Gan, Xuemeng Song, and Zhumin Chen. 2017. Sentiment Analysis for Social Sensor. In Advances in Multimedia Information Processing, Pacific-Rim Conference on Multimedia. 893--902.

Index Terms

• Information systems

  • World Wide Web

    • Web searching and information discovery

      • Personalization

      • Social recommendation

      • Social tagging