Robust Factorization Machines for User Response Prediction

ABSTRACT

Factorization machines (FMs) are a state-of-the-art model class for user response prediction in the computational advertising domain. Rapid growth of internet and mobile device usage has given rise to multiple customer touchpoints. This coupled with factors like high cookie churn rate results in a fragmented view of user activity at the advertiser»s end. Current literature assumes procured user signals as the absolute truth, which is contested by the absence of deterministic identity linkage across a user's multiple avatars. In this work, we characterize the data uncertainty using Robust Optimization (RO) paradigm to design approaches that are immune against perturbations. We propose two novel algorithms: robust factorization machine (RFM) and its field-aware variant (RFFM), under interval uncertainty. These formulations are generic and can find applicability in any classification setting under noise. We provide a distributed and scalable Spark implementation using parallel stochastic gradient descent. In the experiments conducted on three real-world datasets, the robust counterparts outperform the baselines significantly under perturbed settings. Our experimental findings reveal interesting connections between choice of uncertainty set and the noise-proofness of resulting models.

References

1. Adroll. 2016. Factorization Machines. http://tech.adroll.com/blog/data-science/2015/08/25/factorization-machines.html
2. Dimitris Bertsimas, David B. Brown, and Constantine Caramanis. 2011. Theory and Applications of Robust Optimization. SIAM Rev., Vol. 53, 3 (Aug.. 2011), 464--501. https://books.google.co.in/books?id=x6hoBG_MAYIC
3. Jun Wang, Weinan Zhang, and Shuai Yuan. 2016. Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting. CoRR Vol. abs/1610.03013 (2016). showeprint{arxiv}1610.03013 http://arxiv.org/abs/1610.03013
4. Matei Zaharia, Reynold S. Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J. Franklin, Ali Ghodsi, Joseph Gonzalez, Scott Shenker, and Ion Stoica. 2016. Apache Spark: A Unified Engine for Big Data Processing. Commun. ACM, Vol. 59, 11 (Oct.. 2016), 56--65. showISSN0001-0782
5. Qian Zhao, Yue Shi, and Liangjie Hong. 2017. GB-CENT: Gradient Boosted Categorical Embedding and Numerical Trees Proceedings of the 26th International Conference on World Wide Web (WWW '17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 1311--1319.
6. Martin Zinkevich, Markus Weimer, Lihong Li, and Alex J. Smola. 2010. Parallelized Stochastic Gradient Descent. Advances in Neural Information Processing Systems 23, bibfieldeditorJ. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta (Eds.). Curran Associates, Inc., 2595--2603. http://papers.nips.cc/paper/4006-parallelized-stochastic-gradient-descent.pdf

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Supervised learning

        • Supervised learning by classification

    • Machine learning algorithms

      • Regularization

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User models

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

      • Clustering and classification

  • Information systems applications

    • Computational advertising

  • World Wide Web

    • Online advertising

      • Display advertising

• Theory of computation

  • Design and analysis of algorithms

    • Mathematical optimization

      • Continuous optimization