Recurrent knowledge graph embedding for effective recommendation

ABSTRACT

Knowledge graphs (KGs) have proven to be effective to improve recommendation. Existing methods mainly rely on hand-engineered features from KGs (e.g., meta paths), which requires domain knowledge. This paper presents RKGE, a KG embedding approach that automatically learns semantic representations of both entities and paths between entities for characterizing user preferences towards items. Specifically, RKGE employs a novel recurrent network architecture that contains a batch of recurrent networks to model the semantics of paths linking a same entity pair, which are seamlessly fused into recommendation. It further employs a pooling operator to discriminate the saliency of different paths in characterizing user preferences towards items. Extensive validation on real-world datasets shows the superiority of RKGE against state-of-the-art methods. Furthermore, we show that RKGE provides meaningful explanations for recommendation results.

References

1. Phil Blunsom, Edward Grefenstette, and Nal Kalchbrenner. 2014. A Convolutional Neural Network for Modelling Sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL). Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.
2. Rose Catherine and William Cohen. 2016. Personalized Recommendations using Knowledge Graphs: A Probabilistic Logic Programming Approach. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys). ACM, 325--332.
3. Sneha Chaudhari, Amos Azaria, and Tom Mitchell. 2017. An Entity Graph based Recommender System. AI Communications 30, 2 (2017), 141--149.
4. Tianqi Chen, Weinan Zhang, Qiuxia Lu, Kailong Chen, Zhao Zheng, and Yong Yu. 2012. SVDFeature: A Toolkit for Feature-based Collaborative Filtering. Journal of Machine Learning Research (JMLR) 13, Dec (2012), 3619--3622.
5. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural Language Processing (almost) from Scratch. Journal of Machine Learning Research (JMLR) 12, Aug (2011), 2493--2537.
6. Francois Fouss, Alain Pirotte, Jean-Michel Renders, and Marco Saerens. 2007. Random-walk Computation of Similarities Between Nodes of a Graph with Application to Collaborative Recommendation. IEEE Transactions on Knowledge and Data Engineering (TKDE) 19, 3 (2007), 355--369.
7. László Grad-Gyenge, Peter Filzmoser, and Hannes Werthner. 2015. Recommendations on a Knowledge Graph. In 1st International Workshop on Machine Learning Methods for Recommender Systems, MLRec. 13--20.
8. Aditya Grover and Jure Leskovec. 2016. Node2vec: Scalable Feature Learning for Networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, 855--864.
9. Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In Proceedings of the 26th International Conference on World Wide Web (WWW). International World Wide Web Conferences Steering Committee, 173--182.
10. Alexandros Karatzoglou, Xavier Amatriain, Linas Baltrunas, and Nuria Oliver. 2010. Multiverse Recommendation: N-dimensional Tensor Factorization for Context-aware Collaborative Filtering. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys). ACM, 79--86.
11. Yann LeCun, Yoshua Bengio, et al. 1995. Convolutional Networks for Images, Speech, and Time Series. The handbook of Brain Theory and Neural Networks 3361, 10 (1995), 1995.
12. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proceedings of 29th AAAI Conference on Artificial Intelligence (AAAI), Vol. 15. 2181--2187.
13. Zemin Liu, Vincent W Zheng, Zhou Zhao, Fanwei Zhu, Kevin Chen-Chuan Chang, Minghui Wu, and Jing Ying. 2017. Semantic Proximity Search on Heterogeneous Graph by Proximity Embedding. In Proceedings of 31st AAAI Conference on Artificial Intelligence (AAAI).
14. Zemin Liu, Vincent W Zheng, Zhou Zhao, Fanwei Zhu, Kevin Chen-Chuan Chang, Minghui Wu, and Jing Ying. 2018. Distance-aware dag embedding for proximity search on heterogeneous graphs. In Proceedings of 32nd AAAI Conference on Artificial Intelligence (AAAI). AAAI.
15. Chen Luo, Wei Pang, Zhe Wang, and Chenghua Lin. 2014. Hete-cf: Social-based Collaborative Filtering Recommendation using Heterogeneous Relations. In Data Mining (ICDM), 2014 IEEE International Conference on. IEEE, 917--922.
16. Andriy Mnih and Ruslan R Salakhutdinov. 2008. Probabilistic Matrix Factorization. In Advances in Neural Information Processing Systems. 1257--1264.
17. Wenjie Pei, Tadas Baltrušaitis, David MJ Tax, and Louis-Philippe Morency. 2017. Temporal Attention-gated Model for Robust Sequence Classification. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on. IEEE, 820--829.
18. Wenjie Pei, Jie Yang, Zhu Sun, Jie Zhang, Alessandro Bozzon, and David MJ Tax. 2017. Interacting Attention-gated Recurrent Networks for Recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM). ACM, 1459--1468.
19. Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online Learning of Social Representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, 701--710.
20. Tuan-Anh Nguyen Pham, Xutao Li, Gao Cong, and Zhenjie Zhang. 2015. A General Graph-based Model for Recommendation in Event-based Social Networks. In Data Engineering (ICDE), 2015 IEEE 31st International Conference on. IEEE, 567--578.
21. Tuan-Anh Nguyen Pham, Xutao Li, Gao Cong, and Zhenjie Zhang. 2016. A General Recommendation Model for Heterogeneous Networks. IEEE Transactions on Knowledge and Data Engineering (TKDE) 28, 12 (2016), 3140--3153.
22. M Ross Quillan. 1968. Semantic Memory. In Semantic Information Processing.
23. Steffen Rendle. 2010. Factorization Machines. In Data Mining (ICDM), 2010 IEEE 10th International Conference on. IEEE, 995--1000.
24. Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI). AUAI Press, 452--461.
25. Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based Collaborative Filtering Recommendation Algorithms. In Proceedings of the 10th International Conference on World Wide Web (WWW). ACM, 285--295.
26. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional Recurrent Neural Networks. IEEE Transactions on Signal Processing 45, 11 (1997), 2673--2681.
27. Chuan Shi, Jian Liu, Fuzhen Zhuang, S Yu Philip, and Bin Wu. 2016. Integrating Heterogeneous Information via Flexible Regularization Framework for Recommendation. Knowledge and Information Systems 49, 3 (2016), 835--859.
28. Chuan Shi, Zhiqiang Zhang, Ping Luo, Philip S Yu, Yading Yue, and Bin Wu. 2015. Semantic Path based Personalized Recommendation on Weighted Heterogeneous Information Networks. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM). ACM, 453--462.
29. Yue Shi, Martha Larson, and Alan Hanjalic. 2014. Collaborative Filtering Beyond the User-item Matrix: A Survey of the State of The Art and Future Challenges. ACM Computing Surveys (CSUR) 47, 1 (2014), 3.
30. Ajit P Singh and Geoffrey J Gordon. 2008. Relational Learning via Collective Matrix Factorization. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, 650--658.
31. Alexander J Smola and Risi Kondor. 2003. Kernels and Regularization on Graphs. In Learning Theory and Kernel Machines (COLT). Springer, 144--158.
32. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overrating. The Journal of Machine Learning Research (JMLR) 15, 1 (2014), 1929--1958.
33. Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S Yu, and Tianyi Wu. 2011. Path-sim: Meta Path-based Top-k Similarity Search in Heterogeneous Information Networks. Proceedings of the VLDB Endowment 4, 11 (2011), 992--1003.
34. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems(NIPS). 3104--3112.
35. Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale Information Network Embedding. In Proceedings of the 24th International Conference on World Wide Web (WWW). International World Wide Web Conferences Steering Committee, 1067--1077.
36. Ellen M Voorhees et al. 1999. The TREC-8 Question Answering Track Report. In Trec, Vol. 99. 77--82.
37. Yueyang Wang, Yuanfang Xia, Siliang Tang, Fei Wu, and Yueting Zhuang. 2017. Flickr Group Recommendation with Auxiliary Information in Heterogeneous Information Networks. Multimedia Systems 23, 6 (2017), 703--712.
38. Paul J Werbos. 1988. Generalization of Backpropagation with Application to a Recurrent Gas Market Model. Neural networks 1, 4 (1988), 339--356.
39. Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017. Recurrent Recommender Networks. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM). ACM, 495--503.
40. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. In International Conference on Machine Learning (ICML). 2048--2057.
41. Xiao Yu, Xiang Ren, Quanquan Gu, Yizhou Sun, and Jiawei Han. 2013. Collaborative Filtering with Entity Similarity Regularization in Heterogeneous Information Networks. Proceedings of 5th IJCAI Workshop on Heterogeneous Information Network Analysis (IJCAI-HINA) 27 (2013).
42. Xiao Yu, Xiang Ren, Yizhou Sun, Quanquan Gu, Bradley Sturt, Urvashi Khandelwal, Brandon Norick, and Jiawei Han. 2014. Personalized Entity Recommendation: A Heterogeneous Information Network Approach. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM). ACM, 283--292.
43. Xiao Yu, Xiang Ren, Yizhou Sun, Bradley Sturt, Urvashi Khandelwal, Quanquan Gu, Brandon Norick, and Jiawei Han. 2013. Recommendation in Heterogeneous Information Networks with Implicit User Feedback. In Proceedings of the 7th ACM conference on Recommender Systems (RecSys). ACM, 347--350.
44. Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative Knowledge base Embedding for Recommender Systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, 353--362.
45. Jing Zheng, Jian Liu, Chuan Shi, Fuzhen Zhuang, Jingzhi Li, and Bin Wu. 2017. Recommendation in Heterogeneous Information Network via Dual Similarity Regularization. International Journal of Data Science and Analytics 3, 1 (2017), 35--48.

Index Terms

• Information systems

  • Information systems applications

    • Data mining

      • Collaborative filtering