Carlos A. Gomez-Uribe
Abstract
This article discusses the various algorithms that make up the Netflix recommender system, and describes its business purpose. We also describe the role of search and related algorithms, which for us turns into a recommendations problem as well. We explain the motivations behind and review the approach that we use to improve the recommendation algorithms, combining A/B testing focused on improving member retention and medium term engagement, as well as offline experimentation using historical member engagement data. We discuss some of the issues in designing and interpreting A/B tests. Finally, we describe some current areas of focused innovation, which include making our recommender system global and language aware.
- Chris Alvino and Justin Basilico. 2015. Learning a Personalized Homepage. Retrieved December 6, 2015 from http://techblog.netflix.com/2015/04/learning-personalized-homepage.html.Google Scholar
- Xavier Amatriain and Justin Basilico. 2012. Netflix Recommendations: Beyond the 5 stars (Part 2). Retrieved December 6, 2015 from http://techblog.netflix.com/2012/06/netflix-recommendations-beyond-5-stars.htmlGoogle Scholar
- David M Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. The Journal of Machine Learning Research 3, 993--1022. Google ScholarDigital Library
- Olivier Chapelle, Thorsten Joachims, Filip Radlinski, and Yisong Yue. 2012. Large-scale validation and analysis of interleaved search evaluation. ACM Transactions on Information Systems 30, 1. DOI:http://dx.doi.org/10.1145/2094072.2094078 Google ScholarDigital Library
- Alex Deng, Ya Xu, Ron Kohavi, and Toby Walker. 2013. Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. In WSDM. Google ScholarDigital Library
- Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2011. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd. ed.). Springer.Google Scholar
- Yehuda Koren. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, NY). Google ScholarDigital Library
- Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 8, 30--37. Google ScholarDigital Library
- Andriy Mnih and Ruslan Salakhutdinov. 2007. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems. 1257--1264.Google Scholar
- Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge MA. Google ScholarDigital Library
- Prasanna Padmanabhan, Kedar Sadekar, and Gopal Krishnan. 2015. What’s trending on Netflix. Retrieved December 6, 2015 from http://techblog.netflix.com/2015/02/whats-trending-on-netflix.html.Google Scholar
- Arkadiusz Paterek. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proceedings of KDD Cup and Workshop. 5--8.Google Scholar
- Leo Pekelis, David Walsh, and Ramesh Johari. 2015. The New Stats Engine. Internet. Retrieved December 6, 2015 from http://pages.optimizely.com/rs/optimizely/images/stats_engine_technical_paper.pdf.Google Scholar
- Netflix Prize. 2009. The Netflix Prize. Retrieved December 6, 2015 from http://www.netflixprize.com/.Google Scholar
- Steffen Rendle. 2010. Factorization machines. In 2010 IEEE 10th International Conference on Data Mining (ICDM). IEEE, 995--1000. Google ScholarDigital Library
- Joseph L. Schafer. 1997. Analysis of Incomplete Multivariate Data. CRC Press, Boca Raton, FL.Google Scholar
- Barry Schwartz. 2015. The Paradox of Choice: Why More Is Less. Harper Perennial, New York, NY.Google Scholar
- Bryan Gumm. 2013. Appendix 2: Metrics and the Statistics Behind A/B Testing. In A/B Testing: The Most Powerful Way to Turn Clicks into Customers, Dan Siroker and Pete Koomen (Eds.). Wiley, Hoboken, NJ.Google Scholar
- Yee Whye Teh, Michael I. Jordan, Matthew J. Beal, and David M. Blei. 2006. Hierarchical Dirichlet processes. Journal of the American Statistical Association 101, 476.Google ScholarCross Ref
Index Terms
- The Netflix Recommender System: Algorithms, Business Value, and Innovation
Recommendations
A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data MiningRecommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
Acquiring User Information Needs for Recommender Systems
WI-IAT '13: Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 03Most recommender systems attempt to use collaborative filtering, content-based filtering or hybrid approach to recommend items to new users. Collaborative filtering recommends items to new users based on their similar neighbours, and content-based ...
Investigating serendipity in recommender systems based on real user feedback
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied ComputingOver the past several years, research in recommender systems has emphasized the importance of serendipity, but there is still no consensus on the definition of this concept and whether serendipitous items should be recommended is still not a well-...
Comments