Personalized Hashtag Recommendation for Micro-videos

ABSTRACT

Personalized hashtag recommendation methods aim to suggest users hashtags to annotate, categorize, and describe their posts. The hashtags, that a user provides to a post (e.g., a micro-video), are the ones which in her mind can well describe the post content where she is interested in. It means that we should consider both users' preferences on the post contents and their personal understanding on the hashtags. Most existing methods rely on modeling either the interactions between hashtags and posts or the interactions between users and hashtags for hashtag recommendation. These methods have not well explored the complicated interactions among users, hashtags, and micro-videos. In this paper, towards the personalized micro-video hashtag recommendation, we propose a Graph Convolution Network based Personalized Hashtag Recommendation (GCN-PHR) model, which leverages recently advanced GCN techniques to model the complicate interactions among <users, hashtags, micro-videos> and learn their representations. In our model, the users, hashtags, and micro-videos are three types of nodes in a graph and they are linked based on their direct associations. In particular, the message-passing strategy is used to learn the representation of a node (e.g., user) by aggregating the message passed from the directly linked other types of nodes (e.g., hashtag and micro-video). Because a user is often only interested in certain parts of a micro-video and a hashtag is typically used to describe the part (of a micro-video) that the user is interested in, we leverage the attention mechanism to filter the message passed from micro-videos to users and hashtags, which can significantly improve the representation capability. Extensive experiments have been conducted on two real-world micro-video datasets and demonstrate that our model outperforms the state-of-the-art approaches by a large margin.

References

1. Hamidreza Alvari. 2017. Twitter hashtag recommendation using matrix factorization. arXiv preprint arXiv:1705.10453 (2017).
2. Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2016. A simple but tough-to-beat baseline for sentence embeddings. In Proceedings of International Conference of Learning Representation .
3. Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017. Graph convolutional matrix completion. Proceedings of ACM SIGKDD International Conference on Knowledge Discovery & Data Mining .
4. Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences. In The World Wide Web Conference. ACM, 151--161.
5. Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose C Kanjirathinkal, and Mohan Kankanhalli. 2019. MMALFM: Explainable recommendation by leveraging reviews and images. ACM Transactions on Information Systems , Vol. 37, 2 (2019), 16.
6. Zhiyong Cheng, Ying Ding, Lei Zhu, and Mohan Kankanhalli. 2018. Aspect-aware latent factor model: Rating prediction with ratings and reviews. In Proceedings of World Wide Web Conference. 639--648.
7. Zhiyong Cheng, Shen Jialie, and Steven CH Hoi. 2016. On effective personalized music retrieval by exploring online user behaviors. In Proceedings of International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 125--134.
8. Cesc Chunseong Park, Byeongchang Kim, and Gunhee Kim. 2017. Attend to you: Personalized image captioning with context sequence memory networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition . 895--903.
9. Emily Denton, Jason Weston, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. 2015. User conditional hashtag prediction for images. In Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1731--1740.
10. Kuntal Dey, Ritvik Shrivastava, Saroj Kaushik, and L Venkata Subramaniam. 2017. Emtagger: a word embedding based novel method for hashtag recommendation on twitter. In Proceedings of IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 1025--1032.
11. Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Ben-Hur. 2017. Protein interface prediction using graph convolutional networks. In Advances in Neural Information Processing Systems. 6530--6539.
12. Victor Garcia and Joan Bruna. 2018. Few-Shot Learning with Graph Neural Networks. In Proceedings of International Conference on Learning Representations. 1--12.
13. Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. (2017).
14. Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. 1024--1034.
15. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
16. Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial personalized ranking for recommendation. In Proceedings of International ACM SIGIR Conference on Research & Development in Information Retrieval . ACM, 355--364.
17. Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et almbox. 2017. CNN architectures for large-scale audio classification. In Proceedings of IEEE international conference on acoustics, speech and signal processing. IEEE, 131--135.
18. Ashesh Jain, Amir R. Zamir, Silvio Savarese, and Ashutosh Saxena. 2015. Structural-RNN: Deep Learning on Spatio-Temporal Graphs. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 1--10.
19. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. In Proceedings of International Conference on Learning Representations . 1--14.
20. Sofia Ira Ktena, Sarah Parisot, Enzo Ferrante, Martin Rajchl, Matthew Lee, Ben Glocker, and Daniel Rueckert. 2017. Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks. In International Conference on Medical Image Computing & Computer-assisted Intervention. 469--477.
21. Yikang Li, Wanli Ouyang, Bolei Zhou, Jianping Shi, Chao Zhang, and Xiaogang Wang. 2018a. Factorizable net: an efficient subgraph-based framework for scene graph generation. In Proceedings of the European Conference on Computer Vision. 335--351.
22. Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. 2018b. Learning deep generative models of graphs. In Proceedings of the International Conference on Machine Learning. 1--22.
23. Meng Liu, Liqiang Nie, Meng Wang, and Baoquan Chen. 2017. Towards Micro-video Understanding by Joint Sequential-Sparse Modeling. In Proceedings of ACM Multimedia Conference on Multimedia Conference. 970--978.
24. Kenneth Marino, Ruslan Salakhutdinov, and Harikrishna Mulam. 2017. The More You Know: Using Knowledge Graphs for Image Classification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 20--28.
25. Liqiang Nie Meng Liu, Xiang Wang, Qi Tian, and Baoquan Chen. 2019. Online Data Organizer: Micro-Video Categorization by Structure-Guided Multimodal Dictionary Learning. IEEE Transactions on Image Processing , Vol. 28, 3 (2019), 1235--1247.
26. Federico Monti, Michael Bronstein, and Xavier Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks. In Advances in Neural Information Processing Systems. 3697--3707.
27. Liqiang Nie, Xuemeng Song, and Tat-Seng Chua. 2016. Learning from multiple social networks. Synthesis Lectures on Information Concepts, Retrieval, and Services , Vol. 8, 2 (2016), 1--118.
28. Liqiang Nie, Xiang Wang, Jianglong Zhang, Xiangnan He, Hanwang Zhang, Richang Hong, and Qi Tian. 2017. Enhancing Micro-video Understanding by Harnessing External Sounds. In Proceedings of ACM Multimedia Conference on Multimedia Conference. 1192--1200.
29. Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. 2016. Learning convolutional neural networks for graphs. In Proceedings of the International conference on machine learning. 2014--2023.
30. Sarah Parisot, Sofia Ira Ktena, Enzo Ferrante, Matthew Lee, Ricardo Guerrerro Moreno, Ben Glocker, and Daniel Rueckert. 2017. Spectral graph convolutions for population-based disease prediction. In International Conference on Medical Image Computing and Computer-Assisted Intervention . 177--185.
31. Yogesh Singh Rawat and Mohan S Kankanhalli. 2016. ConTagNet: Exploiting user context for image tag recommendation. In Proceedings of the ACM international conference on Multimedia. ACM, 1102--1106.
32. Van Cuong Tran, Dosam Hwang, and Ngoc Thanh Nguyen. 2018. Hashtag Recommendation Approach Based on Content and User Characteristics. Cybernetics and Systems , Vol. 49, 5--6 (2018), 368--383.
33. Andreas Veit, Maximilian Nickel, Serge Belongie, and Laurens van der Maaten. 2018. Separating self-expression and visual content in hashtag supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 5919--5927.
34. Meng Wang, Richang Hong, Guangda Li, Zheng-Jun Zha, Shuicheng Yan, and Tat-Seng Chua. 2012. Event driven web video summarization by tag localization and key-shot identification. IEEE Transactions on Multimedia , Vol. 14, 4 (2012), 975--985.
35. Meng Wang, Changzhi Luo, Bingbing Ni, Jun Yuan, Jianfeng Wang, and Shuicheng Yan. 2017a. First-person daily activity recognition with manipulated object proposals and non-linear feature fusion. IEEE Transactions on Circuits and Systems for Video Technology , Vol. 28, 10 (2017), 2946--2955.
36. Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019 a. KGAT: Knowledge Graph Attention Network for Recommendation. In Proceedings of ACM SIGKDD international conference on knowledge discovery and data mining .
37. Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019 b. Neural Graph Collaborative Filtering. In International ACM SIGIR Conference on Research and Development in Information Retrieval. 165--174.
38. Xiaolong Wang, Furu Wei, Xiaohua Liu, Ming Zhou, and Ming Zhang. 2011. Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach. In Proceedings of the ACM international conference on Information and knowledge management. ACM, 1031--1040.
39. Yilin Wang, Suhang Wang, Jiliang Tang, Guojun Qi, Huan Liu, and Baoxin Li. 2017b. CLARE: A joint approach to label classification and tag recommendation. In Proceedings of AAAI Conference on Artificial Intelligence. 210--216.
40. Zhenghua Xu, Thomas Lukasiewicz, Cheng Chen, Yishu Miao, and Xiangwu Meng. 2017. Tag-Aware Personalized Recommendation Using a Hybrid Deep Model. In Proceedings of International Joint Conference on Artificial Intelligence. 3196--3202.
41. Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 974--983.
42. Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. 2018. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems. 6412--6422.
43. Huaiwen Zhang, Quan Fang, Shengsheng Qian, and Changsheng Xu. 2018. Learning Multimodal Taxonomy via Variational Deep Graph Embedding and Clustering. In Proceedings of the ACM international conference on Multimedia. 681--689.
44. Qi Zhang, Yeyun Gong, Xuyang Sun, and Xuanjing Huang. 2014. Time-aware personalized hashtag recommendation on social media. In Proceedings of International Conference on Computational Linguistics: Technical Papers . 203--212.
45. Marinka Zitnik, Monica Agrawal, and Jure Leskovec. 2018. Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics , Vol. 34, 13 (2018), i457--i466.

Index Terms

• Computing methodologies

  • Machine learning

    • Machine learning approaches

      • Neural networks

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

      • Recommender systems

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

  • World Wide Web

    • Web searching and information discovery

      • Personalization