Zhabiz Gharibshah
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Online advertising, as a vast market, has gained significant attention in various platforms ranging from search engines, third-party websites, social media, and mobile apps. The prosperity of online campaigns is a challenge in online marketing and is usually evaluated by user response through different metrics, such as clicks on advertisement (ad) creatives, subscriptions to products, purchases of items, or explicit user feedback through online surveys. Recent years have witnessed a significant increase in the number of studies using computational approaches, including machine learning methods, for user response prediction. However, existing literature mainly focuses on algorithmic-driven designs to solve specific challenges, and no comprehensive review exists to answer many important questions. What are the parties involved in the online digital advertising eco-systems? What type of data are available for user response prediction? How do we predict user response in a reliable and/or transparent way? In this survey, we provide a comprehensive review of user response prediction in online advertising and related recommender applications. Our essential goal is to provide a thorough understanding of online advertising platforms, stakeholders, data availability, and typical ways of user response prediction. We propose a taxonomy to categorize state-of-the-art user response prediction methods, primarily focusing on the current progress of machine learning methods used in different online platforms. In addition, we also review applications of user response prediction, benchmark datasets, and open source codes in the field.
Supplemental Material
Available for Download
zip
gharibshah.zip (126.8 KB)
Supplemental movie, appendix, image and software files for, User Response Prediction in Online Advertising
- Deepak Agarwal, Rahul Agrawal, Rajiv Khanna, and Nagaraj Kota. 2010. Estimating rates of rare events with multiple hierarchies through scalable log-linear models. In KDD. 213--222.Google Scholar
- Deepak Agarwal, Andrei Zary Broder, Deepayan Chakrabarti, Dejan Diklic, Vanja Josifovski, and Mayssam Sayyadian. 2007. Estimating rates of rare events at multiple resolutions. In KDD. 16--25.Google Scholar
- Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proceedings of the KDD. ACM, 945--955.Google ScholarDigital Library
- Ethem Alpaydin. 2010. Introduction to Machine Learning (2nd ed.). The MIT Press.Google Scholar
- Kamelia Aryafar, Devin Guillory, and Liangjie Hong. 2017. An ensemble-based approach to click-through rate prediction for promoted listings at Etsy. In Proceedings of the ADKDD. ACM, 6 pages. DOI:10.1145/3124749.3124758Google ScholarDigital Library
- Afroze Ibrahim Baqapuri and Ilya Trofimov. 2014. Using neural networks for click prediction of sponsored search. CoRR abs/1412.6601 (2014).Google Scholar
- D. Barbará, Y. Li, and J. Couto. 2002. COOLCAT: An entropy-based algorithm for categorical clustering. In CIKM. 582--589.Google ScholarDigital Library
- Eduardo Barbaro, Eoin Martino Grua, Ivano Malavolta, Mirjana Stercevic, Esther Weusthof, and Jeroen van den Hoven. 2020. Modelling and predicting User Engagement in mobile applications. J. Data Sci. 3, 2 (2020), 61--77. DOI:10.3233/DS-190027Google ScholarCross Ref
- Nicola Barbieri, Fabrizio Silvestri, and Mounia Lalmas. 2016. Improving post-click user engagement on native ads via survival analysis. In WWW. 761--770.Google Scholar
- Shawn D Baron, Caryn Brouwer, and Amaya Garbayo. 2014. A model for delivering branding value through high-impact digital advertising. J. Advert. Res. 54, 3 (2014), 286--291. DOI:10.2501/jar-54-3-286-291Google ScholarCross Ref
- Sonja Bidmon and Johanna Röttl.2018. Advertising Effects of In-Game-Advertising vs. In-App-Advertising. Springer, 73--86.Google Scholar
- L. Bigon, G. Cassani, C. Greco, L. Lacasa, M. Pavoni, A. Polonioli, and J. Tagliabue. 2019. Prediction is very hard, especially about conversion. Predicting user purchases from clickstream data in fashion e-commerce. CoRR abs/1907.00400 (2019).Google Scholar
- M. Blondel, A. Fujino, N. Ueda, and M. Ishihata. 2016. Higher-order factorization machines. In NIPS. 3359--3367.Google Scholar
- Patrick P. K. Chan, Xian Hu, Lili Zhao, Daniel S. Yeung, Dapeng Liu, and Lei Xiao. 2018. Convolutional neural networks based click-through rate prediction with multiple feature sequences. In IJCAI. 2007--2013.Google Scholar
- Xuchao Zhang Chuxu Zhang Jiashu Zhao Dawei Yin Chao Huang, Xian Wu and Nitesh Chawla. 2019. Online purchase prediction via multi-scale modeling of behavior dynamics. In KDD, 2613--2622.Google Scholar
- Olivier Chapelle. 2014. Modeling delayed feedback in display advertising. In KDD. 1097--1105.Google Scholar
- Olivier Chapelle, Eren Manavoglu, and Rómer Rosales. 2014. Simple and scalable response prediction for display advertising. ACM Trans. Intell. Syst. Technol. 5, 4, Article 61 (2014), 61:1--61:34. DOI:10.1145/2532128Google ScholarDigital Library
- Gong Chen, Jacob H. Cox, A. Selcuk Uluagac, and John A. Copeland. 2016. In-depth survey of digital advertising technologies. IEEE Commu. Surv. Tutor. 18 (2016), 2124--2148.Google ScholarDigital Library
- Junxuan Chen, Baigui Sun, Hao Li, Hongtao Lu, and Xian-Sheng Hua. 2016. Deep CTR prediction in display advertising. In Proceedings of the MM. ACM, 811--820. DOI:10.1145/2964284.2964325Google ScholarDigital Library
- Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention. In SIGIR. 335--344.Google Scholar
- Wenqiang Chen, Lizhang Zhan, Yuanlong Ci, Minghua Yang, Chen Lin, and Dugang Liu. 2019. FLEN: Leveraging field for scalable CTR prediction. CoRR abs/1911.04690 (2019).Google Scholar
- H. Cheng, L. Koc, J. Harmsen, and et al.2016. Wide & deep learning for recommender systems. In DLRS. 7--10.Google Scholar
- Hana Choi, Carl F. Mela, Santiago R. Balseiro, and Adam Leary. 2019. Online display advertising markets: A literature review and future directions. J. Inf. Syst. Res. 31 (2019), 556--575.Google ScholarCross Ref
- Shu-Chuan Chu. 2011. Viral advertising in social media: Participation in Facebook groups and responses among college-aged users. J. Interact. Advert. 12 (2011), 30--43.Google ScholarCross Ref
- P. Covington, J. Adams, and E. Sargin. 2016. Deep neural networks for YouTube recommendations. In RecSys. 191--198.Google Scholar
- Brian Dalessandro, Daizhuo Chen, Troy Raeder, Claudia Perlich, Melinda Han Williams, and Foster Provost. 2014. Scalable hands-free transfer learning for online advertising. In KDD. 1573--1582.Google Scholar
- Brian Dalessandro, Rod Hook, Claudia Perlich, and Foster Provost. 2015. Evaluating and optimizing online advertising: Forget the click, but there are good proxies. J. Big Data 3 (2015), 90--102.Google ScholarCross Ref
- Y. Deng, Y. Shen, and H. Jin. 2017. Disguise adversarial networks for click-through rate prediction. In IJCAI. 1589--1595.Google Scholar
- Ernesto Diaz-Aviles, Lucas Drumond, Lars Schmidt-Thieme, and Wolfgang Nejdl. 2012. Real-time top-n recommendation in social streams. In RecSys. 59--66.Google Scholar
- Daniel M. Dunlavy, Tamara G. Kolda, and Evrim Acar. 2011. Temporal link prediction using matrix and tensor factorizations. ACM Trans. Knowl. Discov. Data 5 (2011), 10:1--10:27.Google ScholarDigital Library
- N. Ebadi, B. Lwowski, M. Jaloli, and P. Rad. 2019. Implicit life event discovery from call transcripts using temporal input transformation network. IEEE Access 7 (2019), 172178--172189.Google ScholarCross Ref
- Bora Edizel, Amin Mantrach, and Xiao Bai. 2017. Deep character-level click-through rate prediction for sponsored search. In Proceedings of the SIGIR. ACM, 305--314. DOI:10.1145/3077136.3080811Google ScholarDigital Library
- Muhammad Junaid Effendi and Syed Abbas Ali. 2017. Click through rate prediction for contextual advertisment using linear regression. CoRR abs/1701.08744 (2017).Google Scholar
- F. Maurizio, C. Paolo, and J. Dietmar.2020. Methodological issues in recommender systems research. In IJCAI. 4706--4710.Google Scholar
- Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019. Metapath-guided heterogeneous graph neural network for intent recommendation. ACM Trans. Knowl. Discov. Data (2019), 2478--2486.Google Scholar
- Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, and Keping Yang. 2019. Deep session interest network for click-through rate prediction. In IJCAI. 2301--2307.Google Scholar
- Hongchang Gao, Deguang Kong, Miao Lu, Xiao Bai, and Jian Yang. 2018. Attention convolutional neural network for advertiser-level click-through rate forecasting. In WWW. 1855--1864.Google Scholar
- T. Ge, L. Zhao, G. Zhou, and et al.2018. Image matters: Visually modeling user behaviors using advanced model server. In CIKM, 2087--2095.Google Scholar
- Zhabiz Gharibshah, Xingquan Zhu, Arthur Hainline, and M. Conway. 2020. Deep learning for user interest and response prediction in online display advertising. Data Sci. Eng. 5 (2020), 12--26.Google Scholar
- Djordje Gligorijevic, Jelena Gligorijevic, and A. Flores. 2019. Time-aware prospective modeling of users for online display advertising. CoRR abs/1911.05100 (2019).Google Scholar
- Jelena Gligorijevic, Djordje Gligorijevic, Ivan Stojkovic, Xiao Bai, Amit Goyal, and Zoran Obradovic. 2018. Deeply supervised semantic model for click-through rate prediction in sponsored search. CoRR abs/1803.10739 (2018).Google Scholar
- Thore Graepel, Joaquin Quiñonero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale bayesian click-through rate prediction for sponsored search advertising in Microsoft’s bing search engine. In ICML. 13--20.Google Scholar
- Cheng Guo and Felix Berkhahn. 2016. Entity embeddings of categorical variables. CoRR abs/1604.06737 (2016).Google Scholar
- Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A factorization-machine based neural network for CTR prediction. In IJCAI. 1725--1731.Google Scholar
- H. Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, X. He, and Zhenhua Dong. 2018. DeepFM: An end-to-end wide & deep learning framework for CTR prediction. CoRR abs/1804.04950 (2018).Google Scholar
- Rajan T. Gupta and Saibal K. Pal. 2019. Click-through rate estimation using CHAID classification tree model. In Advances in Analytics and Applications. 45--58.Google Scholar
- H. Dustin, S. Stefan, M. Eren, R. Hema, and L. Chirs.2010. Improving ad relevance in sponsored search. In WSDM. 361--370.Google Scholar
- Xinran He, Stuart Bowers, Joaquin Quiñonero Candela, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, and Ralf Herbrich. 2014. Practical lessons from predicting clicks on ads at Facebook. In ADKDD. 1--9.Google Scholar
- Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In SIGIR. 355--364.Google Scholar
- X. He, K. Deng, X. Wang, Y. Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In SIGIR, 639--648.Google Scholar
- X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua. 2017. Neural collaborative filtering. In WWW. 173--182.Google Scholar
- Y. Hu, Y. Koren, and C. Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In ICDM, 263--272.Google Scholar
- Tongwen Huang, Zhiqi Zhang, and Junlin Zhang. 2019. FiBiNET: Combining feature importance and bilinear feature interaction for click-through rate prediction. In RecSys. 169--177.Google Scholar
- Dietmar Jannach, Gabriel de Souza P. Moreira, and Even Oldridge. 2020. Why are deep learning models not consistently winning recommender systems competitions yet? A position paper. In RecSys. 44--49.Google Scholar
- Zilong Jiang, Shu Gao, and Wei Dai. 2016. Research on CTR prediction for contextual advertising based on deep architecture model. Contr. Eng. Appl. Inf. 18 (Mar. 2016), 11--19.Google Scholar
- Zilong Jiang, S. X. Gao, and Mingjiang Li. 2018. An improved advertising CTR prediction approach based on the fuzzy deep neural network. PLoS ONE.Google Scholar
- Yuchin Juan, Damien Lefortier, and Olivier Chapelle. 2017. Field-aware factorization machines in a real-world online advertising system. In Proceedings of the WWW. International World Wide Web Conferences Steering Committee, 680--688.Google ScholarDigital Library
- Shubhra Karmaker, Parikshit Sondhi, and ChengXiang Zhai. 2017. On application of learning to rank for E-commerce search. In SIGIR.Google Scholar
- K-M Kim, D. Kwak, H. Kwak, Y-J Park, S. Sim, J-H Cho, M. Kim, J. Kwon, Nako Sung, and J-W Ha. 2019. Tripartite heterogeneous graph propagation for large-scale social recommendation. In RecSys. 56--60.Google Scholar
- Michael A. King, Alan S. Abrahams, and Cliff T. Ragsdale. 2015. Ensemble learning methods for pay-per-click campaign management. Expert Syst. Appl. 42 (2015), 4818--4829.Google ScholarDigital Library
- Y. Koren. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In KDD. 426--434.Google Scholar
- N. Kota and D. Agarwal. 2011. Temporal multi-hierarchy smoothing for estimating rates of rare events. In KDD. 1361--1369.Google Scholar
- S. Krishnan and R. Sitaraman. 2013. Understanding the effectiveness of video ads: A measurement study. In IMC. 149--162.Google Scholar
- S. Ktena, A. Tejani, L. Theis, P. Myana, D. Dilipkumar, F. Huszar, S. Yoo, and W. Shi. 2019. Addressing delayed feedback for continuous training with neural networks in CTR prediction. In Proceedings of the RecSys. ACM, 187--195.Google Scholar
- Ashish Kumar and Jari Salo. 2016. Effects of link placements in email newsletters on their click-through rate. J. Market. Commun. 24, 5 (Mar. 2016), 535--548.Google Scholar
- Rohan Kumar, Mohit Kumar, Neil Shah, and Christos Faloutsos. 2018. Did we get it right? Predicting query performance in e-commerce search. CoRR abs/1808.00239 (2018).Google Scholar
- Mounia Lalmas, Janette Lehmann, Guy Shaked, Fabrizio Silvestri, and Gabriele Tolomei. 2015. Promoting positive post-click experience for in-stream Yahoo Gemini users. In KDD. 1929--1938.Google Scholar
- Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past erformance data. In KDD. 768--776.Google Scholar
- S. Leong, M. Mahdian, and S. Vassilvitskii. 2014. Advertising in a stream. In WWW.Google Scholar
- Cheng Li, Yue Lu, Qiaozhu Mei, Dong Wang, and Sandeep Pandey. 2015. Click-through prediction for advertising in Twitter timeline. In KDD. 1959--1968.Google Scholar
- Feng Li, Zhenrui Chen, Pengjie Wang, Yi Ren, Di Zhang, and Xiaoyu Zhu. 2019. Graph intention network for click-through rate prediction in sponsored search. In SIGIR, 961--964.Google Scholar
- Xiang Li, Chao Wang, Jiwei Tan, Xiaoyi Zeng, Dan Ou, and Bo Zheng. 2020. Adversarial multimodal representation learning for click-through rate prediction. In Proceedings of the Web Conference. ACM, 827--836. DOI:10.1145/3366423.3380163Google ScholarDigital Library
- Zeyu Li, Wei Cheng, Yang Chen, Haifeng Chen, and Wei Wang. 2020. Interpretable click-through rate prediction through hierarchical attention. In WSDM, 313--321.Google Scholar
- Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Fi-GNN: Modeling feature interactions via graph neural networks for CTR prediction. In CIKM. 539--548.Google Scholar
- J. Lian, X. Zhou, F. Zhang, Z. Chen, X. Xie, and G. Sun. 2018. xDeepFM: Combining explicit and implicit feature interactions for recommender systems. In KDD (2018).Google Scholar
- X. Lin, H. Chen, C. Pei, F. Sun, X. Xiao, H. Sun, Y. Zhang, W. Ou, and P. Jiang. 2019. A pareto-efficient algorithm for multiple objective optimization in e-commerce recommendation. In RecSys. 20--28.Google Scholar
- X. Ling, W. Deng, C. Gu, and et al.2017. Model ensemble for click prediction in bing search ads. In WWW. 689--698.Google Scholar
- Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, and Yuzhou Zhang. 2019. Feature generation by convolutional neural network for click-through rate prediction. In WWW. 1119--1129.Google Scholar
- Bin Liu, Niannan Xue, Huifeng Guo, Ruiming Tang, Stefanos Zafeiriou, Xiuqiang He, and Zhenguo Li. 2020. AutoGroup: Automatic feature grouping for modelling explicit high-order feature interactions in CTR prediction.Google Scholar
- Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, X. He, Z. Li, and Y. Yu. 2020. AutoFIS: Automatic feature interaction selection in factorization models for click-through rate prediction. In KDD. 2636--2645.Google Scholar
- Hui Liu, Xingquan Zhu, Kristopher Kalish, and Jeremy Kayne. 2017. ULTR-CTR: Fast page grouping using URL truncation for real-time click through rate estimation. In IEEE IRI.Google Scholar
- Qiang Liu, Shu Wu, and Liang Wang. 2015. Collaborative prediction for multi-entity interaction with hierarchical representation. In CIKM. 613--622.Google Scholar
- Qiang Liu, Feng Yu, Shu Wu, and Liang Wang. 2015. A convolutional click prediction model. In CIKM. 1743--1746.Google Scholar
- Xun Liu, Wei Xue, Lei Xiao, and Bo Zhang. 2017. PBODL: Parallel Bayesian online deep learning for click-through rate prediction in tencent advertising system. CoRR abs/1707.00802 (2017).Google Scholar
- Yozen Liu, Xiaolin Shi, Lucas Pierce, and Xiang Ren. 2019. Characterizing and forecasting user engagement with in-app action graph: A case study of Snapchat. In KDD, 2023--2031.Google ScholarDigital Library
- Zhe Liu, Xianzhi Wang, Lina Yao, Jake An, Lei Bai, and Ee-Peng Lim. 2020. Face to purchase: Predicting consumer choices with structured facial and behavioral traits embedding. CoRR abs/2007.06842 (2020).Google Scholar
- Amit Livne, Roy Dor, Eyal Mazuz, Tamar Didi, Bracha Shapira, and Lior Rokach. 2020. Iterative boosting deep neural networks for predicting click-through rate. CoRR abs/2007.13087 (2020).Google Scholar
- Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In SIGIR. 1137--1140.Google Scholar
- Miriam Marciel, Rubén Cuevas, Albert Banchs, Roberto González, Stefano Traverso, Mohamed Ahmed, and Arturo Azcorra. 2016. Understanding the detection of view fraud in video content portals. In WWW. 357--368.Google Scholar
- P. Mattson, C. Cheng, C. Coleman, G. Diamos, and et al.2019. MLPerf training benchmark. CoRR abs/1910.01500.Google Scholar
- Stephen McCreery and Dean M. Krugman. 2017. Tablets and TV advertising: Understanding the viewing experience. J. Curr. Issues Res. Advert. 38, 2 (Mar. 2017), 197--211.Google ScholarCross Ref
- B. McMahan, G. Holt, D. Sculley, and et al.2013. Ad click prediction: A view from the trenches. In KDD. 1222--1230.Google Scholar
- Tao Mei, Xian-Sheng Hua, Linjun Yang, and Shipeng Li. 2007. VideoSense: Towards effective online video advertising. In MM. 1075--1084.Google Scholar
- Wei Meng, Xinyu Xing, Anmol Sheth, Udi Weinsberg, and Wenke Lee. 2014. Your online interests: Pwned! A pollution attack against targeted advertising. In CCS. 129--140.Google Scholar
- Aditya Krishna Menon, Krishna-Prasad Chitrapura, Sachin Garg, Deepak Agarwal, and Nagaraj Kota. 2011. Response prediction using collaborative filtering with hierarchies and side-information. In KDD. 141--149.Google Scholar
- M. Naumov, D. Mudigere, H. Michael Shi, and et. al.2019. Deep learning recommendation model for personalization and recommendation systems. CoRR abs/1906.00091 (2019).Google Scholar
- Chenglei Niu, Guojing Zhong, Y. Liu, Yandong Zhang, Y. Sun, Ailong He, and Zhaoji Chen. 2018. Unstructured semantic model supported deep neural network for click-through rate prediction. CoRR abs/1812.01353 (2018).Google Scholar
- R. Oentaryo, E. Lim, M. Finegold, and et al.2014. Detecting click fraud in online advertising: A data mining approach. J. Mach. Learn. Res. 15 (2014), 99--140.Google ScholarDigital Library
- R. Oentaryo, E. Lim, J. Low, D. Lo, and M. Finegold. 2014. Predicting response in mobile advertising with hierarchical importance-aware factorization machine. In WSDM. 123--132.Google Scholar
- Wentao Ouyang, Xiuwu Zhang, Li Li, Heng Zou, Xin Xing, Zhaojie Liu, and Yanlong Du. 2019. Deep spatio-temporal neural networks for click-through rate prediction. In KDD (2019), 2078--2086.Google ScholarDigital Library
- Wentao Ouyang, Xiuwu Zhang, Shukui Ren, Li Li, Zhaojie Liu, and Yanlong Du. 2019. Click-through rate prediction with the user memory network. In Proceedings of the DLP-KDD. ACM, 4 pages. DOI:10.1145/3326937.3341258Google ScholarDigital Library
- Wentao Ouyang, Xiuwu Zhang, Shukui Ren, Chao Qi, Zhaojie Liu, and Yanlong Du. 2019. Representation learning-assisted click-through rate prediction. In IJCAI.Google Scholar
- Feiyang Pan, Shuokai Li, Xiang Ao, Pingzhong Tang, and Qing He. 2019. Warm up cold-start advertisements: Improving CTR predictions via learning to learn ID embeddings. In SIGIR, 695--704.Google Scholar
- Junwei Pan, Yizhi Mao, Alfonso Lobos Ruiz, Yu Sun, and Aaron Flores. 2019. Predicting different types of conversions with multi-task learning in online advertising. In KDD’19 (2019), 2689--2697.Google Scholar
- Jing Pan, Weian Sheng, and Santanu Dey. 2019. Order matters at fanatics recommending sequentially ordered products by LSTM embedded with Word2Vec. CoRR abs/1911.09818 (2019).Google Scholar
- Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018. Field-weighted factorization machines for click-through rate prediction in display advertising. In WWW. 1349--1357.Google Scholar
- Zhen Pan, Enhong Chen, Qi Liu, Tong Xu, Haiping Ma, and Hongjie Lin. 2016. Sparse factorization machines for click-through rate prediction. In Proceedings of the IEEE 16th Intl. Conf. on Data Mining (ICDM’16). 400--409.Google ScholarCross Ref
- Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou, and Yongfeng Zhang. 2019. Value-aware recommendation based on reinforcement profit maximization. In WWW. 3123--3129.Google Scholar
- Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In Proceedings of the KDD. ACM, 701--710. DOI:10.1145/2623330.2623732Google ScholarDigital Library
- Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In ACM SIGKDD.Google Scholar
- M. Safaei Pour, A. Mangino, K. Friday, Matthias Rathbun, E. Bou-Harb, F. Iqbal, Kh. Shaban, and A. Erradi. 2019. Data-driven curation, learning and analysis for inferring evolving IoT botnets in the wild. In ARES. Article 6.Google Scholar
- S. Punjabi and P. Bhatt. 2018. Robust factorization machines for user response prediction. In WWW. 669--678.Google Scholar
- P. Qi, X. Zhu, G. Zhou, Y. Zhang, Z. Wang, L. Ren, Y. Fan, and K. Gai. 2020. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the CIKM. ACM, 2685--2692.Google Scholar
- Jiarui Qin, Weinan Zhang, Xin Wu, Jiarui Jin, Yuchen Fang, and Yong Yu. 2020. User behavior retrieval for click-through rate prediction. In Proceedings of the SIGIR. ACM, 2347--2356.Google ScholarDigital Library
- Ruihong Qiu, Zi Huang, Jingjing Li, and Hongzhi Yin. 2020. Exploiting cross-session information for session-based recommendation with graph neural networks. ACM Transactions on Information Systems 38 (2020).Google Scholar
- Yanru Qu, Ting Bai, Weinan Zhang, Jianyun Nie, and Jian Tang. 2019. An end-to-end neighborhood-based interaction model for knowledge-enhanced recommendation. In Proceedings of the DLP-KDD. ACM, 9 pages. DOI:10.1145/3326937.3341257Google ScholarDigital Library
- Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-based neural networks for user response prediction. In ICDM, 1149--1154.Google Scholar
- Yanru Qu, Bohui Fang, Wei-Nan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, yong Yu, and Xiuqiang He. 2018. Product-based neural networks for user response prediction over multi-field categorical data. ACM Transactions on Information Systems (2018).Google Scholar
- Regelson, Moira, Fain, and Daniel C. 2006. Predicting click-through rate using keyword clusters.Google Scholar
- K. Ren, J. Qin, Y. Fang, W. Zhang, L. Zheng, W. Bian, G. Zhou, J. Xu, Y. Yu, X. Zhu, and K. Gai. 2019. Lifelong sequential modeling with personalized memorization for user response prediction. In SIGIR’19.Google Scholar
- Kan Ren, Weinan Zhang, Yifei Rong, Haifeng Zhang, Yong Yu, and Jun Wang. 2016. User response learning for directly optimizing campaign performance in display advertising. In CIKM. 679--688.Google Scholar
- Steffen Rendle. 2010. Factorization machines. In ICDM, 995--1000.Google Scholar
- S. Rendle, L. Zhang, and Y. Koren. 2019. On the difficulty of evaluating baselines: A study on recommender systems. CoRR abs/1905.01395 (2019).Google Scholar
- Jenna Reps, Uwe Aickelin, Jonathan Garibaldi, and Chris Damski. 2014. Personalising mobile advertising based on users’ installed apps. In ICDM Workshop, 338--345.Google ScholarCross Ref
- Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: Estimating the click-through rate for new ads. In WWW. 521--530.Google Scholar
- Qiu Ruihong, Yin Hongzhi, Huang Zi, and Tong Chen. 2020. GAG: Global attributed graph neural network for streaming session-based recommendation. In Proceedings of the SIGIR. ACM, 669--678. DOI:10.1145/3397271.3401109Google ScholarDigital Library
- Oliver Rutz, Ashwin Aravindakshan, and Olivier Rubel. 2019. Measuring and forecasting mobile game app engagement. Int. J. Res. Market. 36, 2 (Jun. 2019), 185--199.Google ScholarCross Ref
- Oliver J. Rutz and Randolph E. Bucklin. 2011. From generic to branded: A model of spillover in paid search advertising. J. Market. Res. (2011), 87--102.Google Scholar
- Rubén Saborido, Foutse Khomh, Giuliano Antoniol, and Yann-Gaël Guéhéneuc. 2017. Comprehension of ads-supported and paid android applications: Are they different? In ICPC, 143--153.Google Scholar
- Ying Shan, T. Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and J. C. Mao. 2016. Deep crossing: Web-scale modeling without manually crafted combinatorial features. In KDD. 255--262.Google Scholar
- Neha Sharma and Nirmal Gaud. 2015. K-modes clustering algorithm for categorical data. Int. J. Comput. Appl. 127 (2015), 1--6.Google Scholar
- Weichen Shen. 2018. Easy-to-use, Modular and Extendible Package of Deep-learning Based CTR Models. Retrieved from https://github.com/shenweichen/DeepCTR.Google Scholar
- Weichen Shen. 2019. (PyTorch) Easy-to-use, Modular and Extendible Package of Deep-learning Based CTR Models. Retrieved from https://github.com/shenweichen/DeepCTR-Torch.Google Scholar
- Shu-Ting Shi, Wenhao Zheng, Jun Tang, Qing-Guo Chen, Yao Hu, Jianke Zhu, and Ming Li. 2020. Deep time-stream framework for click-through rate prediction by tracking interest evolution. AAAI 34, 4 (2020), 5726--5733. DOI:10.1609/aaai.v34i04.6028Google ScholarCross Ref
- Enno Shioji and Masayuki Arai. 2017. Neural feature embedding for user response prediction in real-time bidding (RTB). CoRR abs/1702.00855 (2017).Google Scholar
- Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, and Xia Hu. 2020. Towards automated neural interaction discovery for click-through rate prediction. In Proceedings of the KDD. ACM, 945--955.Google ScholarDigital Library
- Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. AutoInt: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of the CIKM. ACM, 1161--1170. DOI:10.1145/3357384.3357925Google ScholarDigital Library
- Yuhan Su, Zhongming Jin, Ying Chen, Xinghai Sun, Yaming Yang, Fangzheng Qiao, Fen Xia, and Wei Xu. 2017. Improving click-through rate prediction accuracy in online advertising by transfer learning. In WI. 1018--1025.Google Scholar
- Anh-Phuong Ta. 2015. Factorization machines with follow-the-regularized-leader for CTR prediction in display advertising. In IEEE Big Data (2015), 2889--2891.Google Scholar
- G. S. Thejas, Kianoosh G. Boroojeni, Kshitij Chandna, Isha Bhatia, S. S. Iyengar, and N. R. Sunitha. 2019. Deep Learning-based Model to Fight Against Ad Click Fraud. In ACM SE. 176--181.Google Scholar
- T. Tian, J. Zhu, F. Xia, X. Zhuang, and T. Zhang. 2015. Crowd fraud detection in internet advertising. In WWW. 1100--1110.Google Scholar
- Gabriele Tolomei, Mounia Lalmas, Ayman Farahat, and Andrew Haines. 2018. You must have clicked on this ad by mistake! Data-driven identification of accidental clicks on mobile ads with applications to advertiser cost discounting and click-through rate prediction. Data Sci. Analyt. 7 (2018), 53--66.Google Scholar
- Sergio Duarte Torres, Ingmar Weber, and Djoerd Hiemstra. 2014. Analysis of search and browsing behavior of young users on the web. ACM Trans. Web 8 (2014), 7:1--7:54.Google ScholarDigital Library
- Vinh Truong, Mathews Nkhoma, and Wanni Pansuwong. 2019. An integrated effectiveness framework of mobile in-app advertising. Australas. J. Inf. Syst. 23 (2019).Google Scholar
- Flavian Vasile, Damien Lefortier, and Olivier Chapelle. 2017. Cost-sensitive learning for utility optimization in online advertising auctions. In Proceedings of the ADKDD. 1--6. DOI:10.1145/3124749.3124751Google ScholarDigital Library
- Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. RippleNet: Propagating user preferences on the knowledge graph for recommender systems. In CIKM’18, 417--426.Google Scholar
- Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. DKN: Deep knowledge-aware network for news recommendation. In Proceedings of the WWW. International World Wide Web Conferences Steering Committee, 1835--1844.Google ScholarDigital Library
- H. Wang, F. Zhang, M. Zhang, J. Leskovec, M. Zhao, W. Li, and Z. Wang. 2019. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems. In KDD (2019), 968--977.Google Scholar
- Hongwei Wang, Fuzheng Zhang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2019. Multi-task feature learning for knowledge graph enhanced recommendation. In WWW. 2000--2010.Google Scholar
- Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale commodity embedding for E-commerce recommendation in Alibaba. KDD, 839--848.Google Scholar
- Jun Wang, Weinan Zhang, and Shuai Yuan. 2016. Display advertising with real-time bidding (RTB) and behavioural targeting. Found. Trends Inf. Retr. 11 (2016), 297--435.Google ScholarCross Ref
- Qianqian Wang, Fang’ai Liu, Shuning Xing, and Xiaohui Zhao. 2018. A new approach for advertising CTR prediction based on deep neural network via attention mechanism. Comp. Math. Methods Med. 2018 (2018), 1--11. DOI:10.1155/2018/8056541Google ScholarCross Ref
- Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29 (2017), 2724--2743.Google ScholarCross Ref
- Ruoxi Wang, Bin Fu, Gang Fu, et al. 2017. Deep & cross network for ad click predictions. In Proceedings of the ADKDD. ACM, 7 pages.Google ScholarDigital Library
- Ruoxi Wang, Rakesh Shivanna, D. Cheng, S. Jain, D. Lin, L. Hong, and Ed Huai hsin Chi. 2020. DCN V2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. arXiv arXiv:2008.13535.Google Scholar
- X. Wang, X. He, M. Wang, F. Feng, and T. Chua. 2019. Neural graph collaborative filtering. In Proceedings of the SIGIR. ACM, 165--174. DOI:10.1145/3331184.3331267Google ScholarDigital Library
- Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Peng Cui, P. Yu, and Yanfang Ye. 2019. Heterogeneous graph attention network. In Proceedings of the World Wide Web Conference. ACM, 2022--2032. DOI:10.1145/3308558.3313562Google ScholarDigital Library
- X. Wang, W. Li, Y. Cui, R. Zhang, and J. Mao. 2011. Click-through rate estimation for rare events in online advertising.Google Scholar
- Hong Wen, Jing Zhang, Quan Lin, Keping Yang, and Pipei Huang. 2019. Multi-level deep cascade trees for conversion rate prediction. AAAI 33 (2019), 338--345. DOI:0.1609/aaai.v33i01.3301338Google ScholarDigital Library
- Hong Wen, Jing Zhang, Yuan Wang, Wentian Bao, Quan Lin, and Keping Yang. 2019. Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction. In Proceedings of the SIGIR. ACM, 2377--2386. DOI:10.1145/3397271.3401443Google ScholarDigital Library
- Xian Wu, Baoxu Shi, Yuxiao Dong, Chao Huang, and Nitesh V. Chawla. 2019. Neural tensor factorization for temporal interaction learning. In WSDM. 537--545.Google Scholar
- Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In Proceedings of the IJCAI. AAAI Press, 3119--3125.Google ScholarCross Ref
- Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang, Changhua Pei, Fei Sun, Jian Wu, Hanxiao Sun, and Wenwu Ou. 2020. Privileged features distillation at Taobao recommendations. In Proceedings of the KDD. ACM, 2590--2598.Google ScholarDigital Library
- C. Xu and M. Wu. 2020. Learning feature interactions with lorentzian factorization machine. In AAAI. 6470--6477.Google Scholar
- Hongxia Yang, Quan Lu, Angus Xianen Qiu, and Chun Han. 2016. Large scale CVR prediction through dynamic transfer learning of global and local features. In Proceedings of Machine Learning Research, Vol. 53. 103--119.Google Scholar
- Xiao Yang, Tao Deng, Weihan Tan, Xutian Tao, Junwei Zhang, Shouke Qin, and Zongyao Ding. 2019. Learning compositional, visual and relational representations for CTR prediction in sponsored search. In CIKM. 2851--2859.Google Scholar
- Y. Yang, X. Guan, and J. You. 2002. CLOPE: A fast and effective clustering algorithm for transactional data. In KDD.Google ScholarDigital Library
- Yi Yang, Baile Xu, Furao Shen, and Jian Zhao. 2019. Operation-aware neural networks for user response prediction. Neural Netw. 121 (2019), 161--168.Google ScholarDigital Library
- Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. CoRR abs/1806.01973 (2018).Google Scholar
- Xu Yong, Chen Jiahui, Huang Chao, Zhang Bo, Xing Hao, Dai Peng, and Bo Liefeng. 2020. Joint modeling of local and global behavior dynamics for session-based recommendation*. In ECAI, 545--552.Google Scholar
- Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang. 2020. Parameter-efficient transfer from sequential behaviors for user modeling and recommendation. In SIGIR. 1469--1478.Google Scholar
- Yuan Yuan, Xiaojing Dong, Chen Dong, Yiwen Sun, Zhenyu Yan, and Abhishek Pani. 2018. Dynamic hierarchical empirical Bayes: A predictive model applied to online advertising. CoRR abs/1809.02213 (2018).Google Scholar
- Y. Yuan, F. Wang, J. Li, and R. Qin. 2014. A survey on real time bidding advertising. In IEEE SOLI. 418--423.Google Scholar
- C. Zhang, D. Song, C. Huang, A. Swami, and N. Chawla. 2019. Heterogeneous graph neural network. In KDD. 793--803.Google Scholar
- Chuxu Zhang, A. Swami, and Nitesh V. Chawla. 2019. SHNE: Representation learning for semantic-associated heterogeneous networks. In WSDM (2019).Google ScholarDigital Library
- J. Zhang, T. Huang, and Z. Zhang. 2019. FAT-DeepFFM: Field attentive deep field-aware factorization machine. In ICDM.Google Scholar
- Li Zhang, Weichen Shen, Shijian Li, and Gang Pan. 2019. Field-aware neural factorization machine for click-through rate prediction. IEEE Access 7 (2019), 75032--75040.Google ScholarCross Ref
- Weinan Zhang, Lingxi Chen, and Jun Wang. 2016. Implicit look-alike modelling in display ads - Transfer collaborative filtering to CTR estimation. CoRR abs/1601.02377 (2016).Google Scholar
- Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data - - A case study on user response prediction. In ECIR.Google Scholar
- Weinan Zhang, Tianxiong Zhou, Jun Wang, and Jian Xu. 2016. Bid-aware gradient descent for unbiased learning with censored data in display advertising. In KDD. 665--674.Google Scholar
- Yuyu Zhang, Hanjun Dai, Chang Xu, Jun Feng, Taifeng Wang, Jiang Bian, Bin Wang, and Tie-Yan Liu. 2014. Sequential click prediction for sponsored search with recurrent neural networks. In AAAI. 1369--1375.Google Scholar
- Yang Zhang, Fuli Feng, Chenxu Wang, Xiangnan He, Meng Wang, Yan Li, and Yongdong Zhang. 2020. How to retrain recommender system? A sequential meta-learning method. In SIGIR. 1479--1488.Google Scholar
- Y. Zhang, P. Zhao, Y. Guan, L. Chen, K. Bian, L. Song, B. Cui, and X. Li. 2020. Preference-aware mask for session-based recommendation with bidirectional transformer. In ICASSP. 3412--3416.Google Scholar
- Weijie Zhao, Deping Xie, Ronglai Jia, Yulei Qian, Ruiquan Ding, Mingming Sun, and Ping Li. 2020. Distributed hierarchical GPU parameter server for massive scale deep learning ads systems. CoRR abs/2003.05622 (2020).Google Scholar
- Xiangyu Zhao, Xudong Zheng, Xiwang Yang, Xiaobing Liu, and Jiliang Tang. 2020. Jointly learning to recommend and advertise. In Proceedings of the KDD. ACM, 3319--3327.Google ScholarDigital Library
- Yifei Zhao, Yu-Hang Zhou, Mingdong Ou, Huan Xu, and Nan Li. 2020. Maximizing cumulative user engagement in sequential recommendation: An online optimization perspective. In KDD. 2784--2792.Google Scholar
- Hua Zheng, Dong Wang, Qi Zhang, Hang Li, and Tinghao Yang. 2010. Do clicks measure recommendation relevancy? An empirical user study. In RecSys. 249--252.Google Scholar
- Guorui Zhou, Ying Fan, Runpeng Cui, Weijie Bian, Xiaoqiang Zhu, and Kun Gai. 2018. Rocket launching: A universal and efficient framework for training well-performing light net. AAAI 32, 1 (2018).Google Scholar
- Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. AAAI 33 (2019), 5941--5948.Google ScholarDigital Library
- Guorui Zhou, Chengru Song, Xiaoqiang Zhu, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In KDD (2018), 1059--1068.Google ScholarDigital Library
- Guorui Zhou, Kailun Wu, Weijie Bian, Zhao Yang, Xiaoqiang Zhu, and Kun Gai. 2019. Res-embedding for deep learning based click-through rate prediction modeling. In Proceedings of the DLP-KDD. ACM, 9 pages.Google ScholarDigital Library
- K. Zhou, M. Redi, A. Haines, and et al.2016. Predicting pre-click quality for native advertisements. In WWW. 299--310.Google Scholar
- Wen-Yuan Zhu, Chun-Hao Wang, Wen-Yueh Shih, W. Peng, and J. Huang. 2017. SEM: A softmax-based ensemble model for CTR estimation in real-time bidding advertising. In IEEE BigComp.5--12.Google Scholar
- Xingquan Zhu and Ian Davidson. 2007. Knowledge Discovery and Data Mining: Challenges and Realities. IGI Global.Google Scholar
- X. Zhu, H. Tao, Z. Wu, J. Cao, K. Kalish, and J. Kayne. 2017. Fraud Prevention in Online Digital Advertising. Springer.Google Scholar
- B. Zoph and Q. Le. 2016. Neural architecture search with reinforcement learning. CoRR abs/1611.01578 (2016).Google Scholar
- Zhabiz Gharibshah and Xingquan Zhu. 2020. TriNE: Network representation learning for tripartite heterogeneous networks. In IEEE International Conference on Knowledge Graph (ICKG’20). IEEE, 497--504. DOI:0.1109/ICBK50248.2020.00076Google ScholarCross Ref
Index Terms
- User Response Prediction in Online Advertising
Recommendations
The pathologies of online display advertising marketplaces
Display advertising marketplaces place "banner" ads on all manner of popular sites. While these services are widely used, they suffer significant challenges, including weak user response and low accountability for both advertisers and web site ...
Calibrating User Response Predictions in Online Advertising
Machine Learning and Knowledge Discovery in Databases: Applied Data Science TrackAbstractPredicting user response probability such as click-through rate (CTR) and conversion rate (CVR) accurately is essential to online advertising systems. To obtain accurate probability, calibration is usually used to transform predicted probabilities ...
Online Advertising: Experimental Facts on Ethics, Involvement, and Product Type
The purpose of this chapter is to provide some insights into advertisements on the Iranian websites. Firstly, in publisher side, is the ethic a matter of fact in accepting Internet advertisements to publish? Second, to provide a preliminary insight into ...
Comments