Collaborative Metric Learning

ABSTRACT

Metric learning algorithms produce distance metrics that capture the important relationships among data. In this work, we study the connection between metric learning and collaborative filtering. We propose Collaborative Metric Learning (CML) which learns a joint metric space to encode not only users' preferences but also the user-user and item-item similarity. The proposed algorithm outperforms state-of-the-art collaborative filtering algorithms on a wide range of recommendation tasks and uncovers the underlying spectrum of users' fine-grained preferences. CML also achieves significant speedup for Top-K recommendation tasks using off-the-shelf, approximate nearest-neighbor search, with negligible accuracy reduction.

References

1. D. Agarwal and B.-C. Chen. flda: matrix factorization through latent dirichlet allocation. In WSDM'10, pages 91--100. ACM, 2010.
2. Y. Bachrach, Y. Finkelstein, R. Gilad-Bachrach, L. Katzir, N. Koenigstein, N. Nice, and U. Paquet. Speeding up the xbox recommender system using a euclidean transformation for inner-product spaces. In RecSys'14, pages 257--264. ACM, 2014.
3. L. Baltrunas and X. Amatriain. Towards time-dependant recommendation based on implicit feedback. In Workshop on context-aware recommender systems (CARS'09), 2009.
4. M. Bawa, T. Condie, and P. Ganesan. Lsh forest: self-tuning indexes for similarity search. In WWW'05, pages 651--660. ACM, 2005.
5. Y. Bengio, I. J. Goodfellow, and A. Courville. Deep learning. An MIT Press book in preparation., 2015.
6. E. Bernhardsson. Annoy. https://github.com/spotify/annoy, 2016.
7. O. Canévet and F. Fleuret. Efficient sample mining for object detection. In ACML, 2014.
8. S. Chopra, R. Hadsell, and Y. LeCun. Learning a similarity metric discriminatively, with application to face verification. In CVPR'05, volume 1, pages 539--546. IEEE, 2005.
9. M. Cogswell, F. Ahmed, R. Girshick, L. Zitnick, and D. Batra. Reducing overfitting in deep networks by decorrelating representations. ICLR'15, 2015.
10. J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul):2121--2159, 2011.
11. J. Friedman, T. Hastie, and R. Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics Springer, Berlin, 2001.
12. Z. Gantner, S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme. Mymedialite: a free recommender system library. In RecSys'11, pages 305--308. ACM, 2011.
13. F. M. Harper and J. A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst., 5(4), Dec. 2015.
14. S. S. Haykin, S. S. Haykin, S. S. Haykin, and S. S. Haykin. Neural networks and learning machines, volume 3. Pearson Upper Saddle River, NJ, USA:, 2009.
15. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. arXiv:1512.03385, 2015.
16. R. He and J. McAuley. Vbpr: visual bayesian personalized ranking from implicit feedback. arXiv:1510.01784, 2015.
17. C.-K. Hsieh. Collaborative metric learning. https://github.com/changun/CollMetric, 2016.
18. C.-K. Hsieh, L. Yang, H. Wei, M. Naaman, and D. Estrin. Immersive recommendation: News and event recommendations using personal digital traces. In WWW'16, pages 51--62. International World Wide Web Conferences Steering Committee, 2016.
19. Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In ICDM'08, pages 263--272. Ieee, 2008.
20. T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR'05, pages 154--161. Acm, 2005.
21. D. Kedem, S. Tyree, F. Sha, G. R. Lanckriet, and K. Q. Weinberger. Non-linear metric learning. In NIPS'12, pages 2573--2581, 2012.
22. D. Kelly and J. Teevan. Implicit feedback for inferring user preference: a bibliography. In ACM SIGIR Forum, volume 37, pages 18--28. ACM, 2003.
23. M. Khoshneshin and W. N. Street. Collaborative filtering via euclidean embedding. In RecSys'10, pages 87--94. ACM, 2010.
24. N. Koenigstein and Y. Koren. Towards scalable and accurate item-oriented recommendations. In RecSys'13, pages 419--422. ACM, 2013.
25. Y. Koren, R. Bell, C. Volinsky, et al. Matrix factorization techniques for recommender systems. Computer, 42(8):30--37, 2009.
26. M. Köstinger, M. Hirzer, P. Wohlhart, P. M. Roth, and H. Bischof. Large scale metric learning from equivalence constraints. In CVPR'12, pages 2288--2295. IEEE, 2012.
27. M. Kula. Metadata embeddings for user and item cold-start recommendations. arXiv:1507.08439, 2015.
28. M. Kula. Lightfm. https://github.com/lyst/lightfm, 2016.
29. B. Kulis. Metric learning: A survey. Foundations and Trends in Machine Learning, 5(4):287--364, 2012.
30. B. Kumar, G. Carneiro, and I. Reid. Learning local image descriptors with deep siamese and triplet convolutional networks by minimising global loss functions. arXiv:1512.09272, 2015.
31. G. Linden, B. Smith, and J. York. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, 7(1):76--80, 2003.
32. W. Liu and I. W. Tsang. Large margin metric learning for multi-label prediction. In AAAI'15, pages 2800--2806, 2015.
33. L. v. d. Maaten and G. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579--2605, 2008.
34. B. McFee, T. Bertin-Mahieux, D. P. Ellis, and G. R. Lanckriet. The million song dataset challenge. In WWW'12, pages 909--916. ACM, 2012.
35. R. Pan, Y. Zhou, B. Cao, N. N. Liu, R. Lukose, M. Scholz, and Q. Yang. One-class collaborative filtering. In ICDM'08, pages 502--511. IEEE, 2008.
36. P. Ram and A. G. Gray. Maximum inner-product search using cone trees. In KDD'12, pages 931--939. ACM, 2012.
37. S. Rendle. Factorization machines. In ICDM'10, pages 995--1000. IEEE, 2010.
38. S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In UAI'09, pages 452--461. AUAI Press, 2009.
39. S. Rendle and L. Schmidt-Thieme. Pairwise interaction tensor factorization for personalized tag recommendation. In WSDM'10, pages 81--90. ACM, 2010.
40. P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl. Grouplens: an open architecture for collaborative filtering of netnews. In CSCW'94, pages 175--186. ACM, 1994.
41. G. Shani and A. Gunawardana. Evaluating recommendation systems. In Recommender systems handbook, pages 257--297. Springer, 2011.
42. A. Shrivastava and P. Li. Asymmetric lsh (alsh) for sublinear time maximum inner product search (mips). In NIPS'14, pages 2321--2329, 2014.
43. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. Deepface: Closing the gap to human-level performance in face verification. In CVPR'14, pages 1701--1708, 2014.
44. Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. arXiv:1605.02688, 2016.
45. J. Wan, D. Wang, S. C. H. Hoi, P. Wu, J. Zhu, Y. Zhang, and J. Li. Deep learning for content-based image retrieval: A comprehensive study. In Proceedings of the 22nd ACM international conference on Multimedia, pages 157--166. ACM, 2014.
46. C. Wang and D. M. Blei. Collaborative topic modeling for recommending scientific articles. In KDD'11, pages 448--456. ACM, 2011.
47. H. Wang, N. Wang, and D.-Y. Yeung. Collaborative deep learning for recommender systems. In KDD'15, pages 1235--1244. ACM, 2015.
48. J. Wang, H. T. Do, A. Woznica, and A. Kalousis. Metric learning with multiple kernels. In NIPS'11, pages 1170--1178, 2011.
49. K. Q. Weinberger and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research, 10(Feb):207--244, 2009.
50. J. Weston, S. Bengio, and N. Usunier. Large scale image annotation: learning to rank with joint word-image embeddings. Machine learning, 81(1):21--35, 2010.
51. E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell. Distance metric learning with application to clustering with side-information. NIPS'03, 15:505--512, 2003.
52. Z. E. Xu, M. Chen, K. Q. Weinberger, and F. Sha. From sbow to dcot marginalized encoders for text representation. In CIKM'12, pages 1879--1884. ACM, 2012.
53. T. Zhao, J. McAuley, and I. King. Improving latent factor models via personalized feature projection for one class recommendation. In CIKM'15, pages 821--830. ACM, 2015.
54. C.-N. Ziegler, S. M. McNee, J. A. Konstan, and G. Lausen. Improving recommendation lists through topic diversification. In WWW'05, pages 22--32. ACM, 2005.

Index Terms

• Computing methodologies

  • Machine learning

    • Machine learning approaches

      • Learning latent representations

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

      • Recommender systems