ABSTRACT
Neural network based models for collaborative filtering have started to gain attention recently. One branch of research is based on using deep generative models to model user preferences where variational autoencoders were shown to produce state-of-the-art results. However, there are some potentially problematic characteristics of the current variational autoencoder for CF. The first is the too simplistic prior that VAEs incorporate for learning the latent representations of user preference. The other is the model's inability to learn deeper representations with more than one hidden layer for each network.
Our goal is to incorporate appropriate techniques to mitigate the aforementioned problems of variational autoencoder CF and further improve the recommendation performance. Our work is the first to apply flexible priors to collaborative filtering and show that simple priors (in original VAEs) may be too restrictive to fully model user preferences and setting a more flexible prior gives significant gains. We experiment with the VampPrior, originally proposed for image generation, to examine the effect of flexible priors in CF. We also show that VampPriors coupled with gating mechanisms outperform SOTA results including the Variational Autoencoder for Collaborative Filtering by meaningful margins on 2 popular benchmark datasets (MovieLens & Netflix).
- Alemi, A. A., Fischer, I., Dillon, J. V., & Murphy, K. (2016). Deep variational information bottleneck. arXiv preprint arXiv:1612.00410.Google Scholar
- Alemi, A., Poole, B., Fischer, I., Dillon, J., Saurous, R. A., & Murphy, K. (2018, July). Fixing a Broken ELBO. In International Conference on Machine Learning (pp. 159--168).Google Scholar
- Burgess, C. P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., & Lerchner, A. (2018). Understanding disentangling in $\beta $-VAE. arXiv preprint arXiv:1804.03599.Google Scholar
- Dauphin, Y. N., Fan, A., Auli, M., & Grangier, D. (2017, August). Language modeling with gated convolutional networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 (pp. 933--941). JMLR. org. Google ScholarDigital Library
- Dieng, A. B., Kim, Y., Rush, A. M., & Blei, D. M. (2018). Avoiding latent variable collapse with generative skip models. arXiv preprint arXiv:1807.04863.Google Scholar
- Dilokthanakul, N., Mediano, P. A., Garnelo, M., Lee, M. C., Salimbeni, H., Arulkumaran, K., & Shanahan, M. (2016). Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648.Google Scholar
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press. Google ScholarDigital Library
- Goyal, P., Hu, Z., Liang, X., Wang, C., & Xing, E. P. (2017). Nonparametric variational auto-encoders for hierarchical representation learning. In Proceedings of the IEEE International Conference on Computer Vision (pp. 5094--5102).Google ScholarCross Ref
- He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. S. (2017, April). Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web (pp. 173--182). International World Wide Web Conferences Steering Committee. Google ScholarDigital Library
- Hidasi, B., Karatzoglou, A., Baltrunas, L., & Tikk, D. (2015). Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939.Google Scholar
- Hoffman, M. D., & Johnson, M. J. (2016). Elbo surgery: yet another way to carve up the variational evidence lower bound. In Workshop in Advances in Approximate Bayesian Inference, NIPS.Google Scholar
- Hsu, W. N., Zhang, Y., & Glass, J. (2017). Unsupervised learning of disentangled and interpretable representations from sequential data. In Advances in neural information processing systems (pp. 1878--1889). Google ScholarDigital Library
- Hu, Y., Koren, Y., & Volinsky, C. (2008, December). Collaborative Filtering for Implicit Feedback Datasets. In ICDM (Vol. 8, pp. 263--272). Google ScholarDigital Library
- Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.Google Scholar
- Liang, D., Krishnan, R. G., Hoffman, M. D., & Jebara, T. (2018, April). Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference on World Wide Web (pp. 689--698). International World Wide Web Conferences Steering Committee. Google ScholarDigital Library
- Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., & Frey, B. (2015). Adversarial autoencoders. arXiv preprint arXiv:1511.05644.Google Scholar
- Marlin, B. (2004). Collaborative filtering: A machine learning perspective (pp. 2239--2239). Toronto: University of Toronto.Google Scholar
- Nalisnick, E., & Smyth, P. (2017, April). Stick-breaking variational autoencoders. In International Conference on Learning Representations (ICLR).Google Scholar
- Ning, X., & Karypis, G. (2011, December). Slim: Sparse linear methods for top-n recommender systems. In 2011 IEEE 11th International Conference on Data Mining (pp. 497--506). IEEE. Google ScholarDigital Library
- Quadrana, M., Karatzoglou, A., Hidasi, B., & Cremonesi, P. (2017, August). Personalizing session-based recommendations with hierarchical recurrent neural networks. In Proceedings of the Eleventh ACM Conference on Recommender Systems (pp. 130--137). ACM. Google ScholarDigital Library
- Rakesh, V., Wang, S., Shu, K., & Liu, H. (2019, January). Linked variational autoencoders for inferring substitutable and supplementary items. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (pp. 438--446). ACM. Google ScholarDigital Library
- Sachdeva, N., Manco, G., Ritacco, E., & Pudi, V. (2019, January). Sequential Variational Autoencoders for Collaborative Filtering. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (pp. 600--608). ACM. Google ScholarDigital Library
- Sedhain, S., Menon, A. K., Sanner, S., & Xie, L. (2015, May). Autorec: Autoencoders meet collaborative filtering. In Proceedings of the 24th International Conference on World Wide Web (pp. 111--112). ACM. Google ScholarDigital Library
- Sønderby, C. K., Raiko, T., Maaløe, L., Sønderby, S. K., & Winther, O. (2016). Ladder variational autoencoders. In Advances in neural information processing systems (pp. 3738--3746).Google Scholar
- Tomczak, J., & Welling, M. (2018, March). VAE with a VampPrior. In International Conference on Artificial Intelligence and Statistics (pp. 1214--1223).Google Scholar
- Van den Oord, A., Dieleman, S., & Schrauwen, B. (2013). Deep content-based music recommendation. In Advances in neural information processing systems (pp. 2643--2651). Google ScholarDigital Library
- Wang, H., Wang, N., & Yeung, D. Y. (2015, August). Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1235--1244). ACM. Google ScholarDigital Library
- Wang, J., Yu, L., Zhang, W., Gong, Y., Xu, Y., Wang, B., ... & Zhang, D. (2017, August). Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval (pp. 515--524). ACM. Google ScholarDigital Library
- Wu, C. Y., Ahmed, A., Beutel, A., Smola, A. J., & Jing, H. (2017, February). Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining (pp. 495--503). ACM. Google ScholarDigital Library
- Wu, Y., DuBois, C., Zheng, A. X., & Ester, M. (2016, February). Collaborative denoising auto-encoders for top-n recommender systems. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (pp. 153--162). ACM. Google ScholarDigital Library
- Zhang, S., Yao, L., Sun, A., & Tay, Y. (2019). Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), 52(1), 5. Google ScholarDigital Library
Index Terms
- Enhancing VAEs for collaborative filtering: flexible priors & gating mechanisms
Recommendations
Neural Variational Collaborative Filtering for Top-K Recommendation
Trends and Applications in Knowledge Discovery and Data MiningAbstractCollaborative Filtering (CF) is one of the most widely applied models for recommender systems. However, CF-based methods suffer from data sparsity and cold-start, more attention has been drawn to hybrid methods by using both the rating and content ...
Augmented Variational Autoencoders for Collaborative Filtering with Auxiliary Information
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge ManagementRecommender systems offer critical services in the age of mass information. A good recommender system selects a certain item for a specific user by recognizing why the user might like the item. This awareness implies that the system should model the ...
Trust-based collaborative filtering: tackling the cold start problem using regular equivalence
RecSys '18: Proceedings of the 12th ACM Conference on Recommender SystemsUser-based Collaborative Filtering (CF) is one of the most popular approaches to create recommender systems. This approach is based on finding the most relevant k users from whose rating history we can extract items to recommend. CF, however, suffers ...
Comments