GAPfm: optimal top-n recommendations for graded relevance domains

ABSTRACT

Recommender systems are frequently used in domains in which users express their preferences in the form of graded judgments, such as ratings. Current ranking techniques are based on one of two sub-optimal approaches: either they optimize for a binary metric such as Average Precision, which discards information on relevance levels, or they optimize for Normalized Discounted Cumulative Gain (NDCG), which ignores the dependence of an item's contribution on the relevance of more highly ranked items. We address the shortcomings of existing approaches by proposing GAPfm, the Graded Average Precision factor model, which is a latent factor model for top-N recommendation in domains with graded relevance data. The model optimizes the Graded Average Precision metric that has been proposed recently for assessing the quality of ranked results lists for graded relevance. GAPfm's advantages are twofold: it maintains full information about graded relevance and also addresses the limitations of models that optimize NDCG. Experimental results show that GAPfm achieves substantial improvements on the top-N recommendation task, compared to several state-of-the-art approaches.

References

1. D. Agarwal and B.-C. Chen. Regression-based latent factor models. KDD '09, pages 19--28. ACM, 2009.
2. D. Agarwal, B.-C. Chen, P. Elango, and X. Wang. Personalized click shaping through lagrangian duality for online recommendation. SIGIR '12, pages 485--494. ACM, 2012.
3. S. Balakrishnan and S. Chopra. Collaborative ranking. WSDM '12, pages 143--152. ACM, 2012.
4. C. J. C. Burges, R. Ragno, and Q. V. Le. Learning to rank with nonsmooth cost functions. NIPS '06, pages 193--200, 2006.
5. O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In Proceedings of the 18th ACM conference on Information and knowledge management, CIKM '09, pages 621--630, New York, NY, USA, 2009. ACM.
6. O. Chapelle and M. Wu. Gradient descent optimization of smoothed information retrieval metrics. Inf. Retr., 13:216--235, June 2010.
7. P. Cremonesi, Y. Koren, and R. Turrin. Performance of recommender algorithms on top-n recommendation tasks. RecSys '10, pages 39--46. ACM, 2010.
8. A. Gunawardana and G. Shani. A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res., 10:2935--2962, December 2009.
9. T. Hofmann. Latent semantic models for collaborative filtering. ACM Trans. Inf. Syst., 22:89--115, January 2004.
10. L. Hong, R. Bekkerman, J. Adler, and B. D. Davison. Learning to rank social update streams. SIGIR '12, pages 651--660. ACM, 2012.
11. K. Järvelin and J. Kekalainen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst., 20(4):422--446, Oct. 2002.
12. Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. KDD '08, pages 426--434. ACM, 2008.
13. Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. Computer, 42:30--37, August 2009.
14. N. N. Liu and Q. Yang. Eigenrank: a ranking-oriented approach to collaborative filtering. SIGIR '08, pages 83--90. ACM, 2008.
15. N. N. Liu, M. Zhao, and Q. Yang. Probabilistic latent preference analysis for collaborative filtering. CIKM '09, pages 759--766. ACM, 2009.
16. T.-Y. Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3):225--331, 2009.
17. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. Distributed graphlab: A framework for machine learning in the cloud. PVLDB, 5(8):716--727, 2012.
18. C. D. Manning, P. Raghavan, and H. Schütze. Introduction to information retrieval. Cambridge Univ. Press, Cambridge {u.a.}, 1. publ. edition, 2008.
19. S. Rendle, C. Freudenthaler, Z. Gantner, and S.-T. Lars. Bpr: Bayesian personalized ranking from implicit feedback. UAI '09, pages 452--461. AUAI Press, 2009.
20. S. E. Robertson, E. Kanoulas, and E. Yilmaz. Extending average precision to graded relevance judgments. SIGIR '10, pages 603--610. ACM, 2010.
21. Y. Shi, A. Karatzoglou, L. Baltrunas, M. Larson, A. Hanjalic, and N. Oliver. TFMAP: optimizing map for top-n context-aware recommendation. SIGIR '12, pages 155--164. ACM, 2012.
22. Y. Shi, A. Karatzoglou, L. Baltrunas, M. Larson, N. Oliver, and A. Hanjalic. CLiMF: learning to maximize reciprocal rank with collaborative less-is-more filtering. RecSys '12, pages 139--146. ACM, 2012.
23. M. Taylor, J. Guiver, S. Robertson, and T. Minka. Softrank: optimizing non-smooth rank metrics. WSDM '08, pages 77--86. ACM, 2008.
24. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun. Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res., 6:1453--1484, 2005.
25. M. N. Volkovs and R. S. Zemel. Collaborative ranking with 17 parameters. NIPS '12, 2012.
26. E. M. Voorhees. The trec-8 question answering track report. In TREC-8, 1999.
27. J. Wang and J. Zhu. On statistical analysis and optimization of information retrieval effectiveness metrics. SIGIR '10, pages 226--233. ACM, 2010.
28. M. Weimer, A. Karatzoglou, Q. Le, and A. Smola. Cofirank - maximum margin matrix factorization for collaborative ranking. NIPS'07, pages 1593--1600, 2007.
29. J. Xu and H. Li. Adarank: a boosting algorithm for information retrieval. SIGIR '07, pages 391--398. ACM, 2007.
30. J. Xu, T.-Y. Liu, M. Lu, H. Li, and W.-Y. Ma. Directly optimizing evaluation measures in learning to rank. SIGIR '08, pages 107--114. ACM, 2008.
31. S.-H. Yang, B. Long, A. J. Smola, H. Zha, and Z. Zheng. Collaborative competitive filtering: learning recommender using context of user choice. SIGIR '11, pages 295--304. ACM, 2011.
32. Y. Yue, T. Finley, F. Radlinski, and T. Joachims. A support vector method for optimizing average precision. SIGIR '07, pages 271--278. ACM, 2007.

Index Terms

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

      • Document filtering

      • Information extraction