GP-BPR: Personalized Compatibility Modeling for Clothing Matching

ABSTRACT

Owing to the recent advances in the multimedia processing domain and the publicly available large-scale real-world data provided by online fashion communities, like the IQON and Chictopia, researchers are enabled to investigate the automatic clothing matching solutions. In a sense, existing methods mainly focus on modeling the general item-item compatibility from the aesthetic perspective, but fail to incorporate the user factor. In fact, aesthetics can be highly subjective, as different people may hold different clothing preferences. In light of this, in this work, we attempt to tackle the problem of personalized compatibility modeling from not only the general aesthetics but also the personal preference perspectives. In particular, we present a personalized compatibility modeling scheme GP-BPR, comprising of two essential components: general compatibility modeling and personal preference modeling, which characterize the item-item and user-item interactions, respectively. In particular, due to the concern that both the modalities (e.g., the image and context description) of fashion items can deliver important cues regarding user personal preference, we present a comprehensive personal preference modeling method. Moreover, for evaluation, we create a large-scale dataset, IQON3000, from the online fashion community IQON. Extensive experiment results on IQON3000 verify the effectiveness of the proposed scheme. As a byproduct, we have released the dataset, codes, and involved parameters to benefit other researchers.

References

1. Jesús Bobadilla, Rodolfo Bojorque, Antonio Hernando Esteban, and Remigio Hurtado. 2018. Recommender systems clustering using Bayesian non negative matrix factorization. IEEE Access , Vol. 6, 3549--3564.
2. Da Cao, Liqiang Nie, Xiangnan He, Xiaochi Wei, Shunzhi Zhu, and Tat-Seng Chua. 2017. Embedding factorization models for jointly recommending items and user generated lists. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 585--594.
3. Chih-Ming Chen, Ming-Feng Tsai, Jen-Yu Liu, and Yi-Hsuan Yang. 2013. Using emotional context from article for contextual music recommendation. In Proceedings of the ACM International Conference on Multimedia. ACM, 649--652.
4. Jingyuan Chen, Xuemeng Song, Liqiang Nie, Xiang Wang, Hanwang Zhang, and Tat-Seng Chua. 2016. Micro tells macro: predicting the popularity of micro-videos via a transductive model. In Proceedings of the ACM International Conference on Multimedia. ACM, 898--907.
5. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2018. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 40, 4, 834--848.
6. Zunlei Feng, Zhenyun Yu, Yezhou Yang, Yongcheng Jing, Junxiao Jiang, and Mingli Song. 2018. Interpretable partitioned embedding for customized multi-item fashion outfit Composition. In Proceedings of the International Conference on Multimedia Retrieval. ACM, 143--151.
7. Xianjing Han, Xuemeng Song, Jianhua Yin, Yinglong Wang, and Liqiang Nie. 2019. Prototype-guided attribute-wise interpretable scheme for clothing matching. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 785--794.
8. Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry S. Davis. 2017. Learning fashion compatibility with bidirectional LSTMs. In Proceedings of the ACM International Conference on Multimedia. 1078--1086.
9. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016b. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
10. Ruining He and Julian McAuley. 2016. VBPR: Visual Bayesian personalized ranking from implicit feedback. In Proceedings of the International Joint Conference on Artificial Intelligence . AAAI, 144--150.
11. Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the ACM International WWW Conference. ACM, 173--182.
12. Xiangnan He, Hanwang Zhang, Min Yen Kan, and Tat Seng Chua. 2016a. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the International ACM SIGIR Conference. 549--558.
13. Yang Hu, Xi Yi, and Larry S Davis. 2015. Collaborative fashion recommendation: a functional tensor factorization approach. In Proceedings of the ACM International Conference on Multimedia. ACM, 129--138.
14. Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard H. Hovy, and Eric P. Xing. 2016. Harnessing deep neural networks with logic rules. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, 2410--2420.
15. Vignesh Jagadeesh, Robinson Piramuthu, Anurag Bhardwaj, Wei Di, and Neel Sundaresan. 2014. Large scale visual recommendations from street fashion images. In Proceedings of the International ACM SIGKDD Conference. ACM, 1925--1934.
16. Lu Jiang, Shoou-I Yu, Deyu Meng, Yi Yang, Teruko Mitamura, and Alexander G Hauptmann. 2015. Fast and accurate content-based semantic search in 100m internet videos. In Proceedings of the ACM International Conference on Multimedia. ACM, 49--58.
17. Aditya Khosla, Atish Das Sarma, and Raffay Hamid. 2014. What makes an image popular?. In Proceedings of the ACM International WWW Conference. ACM, 867--876.
18. Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, and Hwanjo Yu. 2016. Convolutional matrix factorization for document context-aware recommendation. In Proceedings of the ACM Conference on Recommender Systems. ACM, 233--240.
19. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing . 1746--1751.
20. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 .
21. Yehuda Koren and Robert Bell. 2015. Advances in collaborative filtering. Recommender Systems Handbook , 145--186.
22. Yuncheng Li, Liangliang Cao, Jiang Zhu, and Jiebo Luo. 2017. Mining fashion outfit composition using an end-to-end deep learning approach on set data. IEEE Transactions on Multimedia , Vol. 19, 8, 1946--1955.
23. Jian Han Lim, Nurul Japar, Chun Chet Ng, and Chee Seng Chan. 2018. Unprecedented usage of pre-trained CNNs on beauty product. In Proceedings of the ACM International Conference on Multimedia. ACM, 2068--2072.
24. Meng Liu, Liqiang Nie, Meng Wang, and Baoquan Chen. 2017. Towards micro-video understanding by joint sequential-sparse modeling. In Proceedings of the ACM International Conference on Multimedia. 970--978.
25. Meng Liu, Xiang Wang, Liqiang Nie, Qi Tian, Baoquan Chen, and Tat-Seng Chua. 2018a. Cross-modal moment localization in videos. In Proceedings of the ACM International Conference on Multimedia. ACM, 843--851.
26. Si Liu, Jiashi Feng, Zheng Song, Tianzhu Zhang, Hanqing Lu, Changsheng Xu, and Shuicheng Yan. 2012. Hi, magic closet, tell me what to wear!. In Proceedings of the ACM International Conference on Multimedia. ACM, 619--628.
27. Siyuan Liu, Qiong Wu, and Chunyan Miao. 2018b. Personalized recommendation considering secondary implicit feedback. In Proceedings of the IEEE International Conference on Agents. IEEE, 87--92.
28. Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. 2016. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1096--1104.
29. Babak Loni, Roberto Pagano, Martha Larson, and Alan Hanjalic. 2016. Bayesian personalized ranking with multi-channel user feedback. In Proceedings of the ACM Conference on Recommender Systems. ACM, 361--364.
30. Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval . ACM, 43--52.
31. Charles Packer, Julian McAuley, and Arnau Ramisa. 2018. Visually-aware personalized recommendation using interpretable image representations. arXiv preprint arXiv:1806.09820 .
32. Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the International Conference on Uncertainty in Artificial Intelligence. AUAI Press, 452--461.
33. Aliaksei Severyn and Alessandro Moschitti. 2015. Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 959--962.
34. Hiroyuki Shinnou, Masayuki Asahara, K Komiya, and M Sasaki. 2017. Nwjc2vec: Word embedding data constructed from NINJAL Web Japanese Gorpus. Journal of Natural Language Processing , Vol. 24, 4, 705--720.
35. Xuemeng Song, Fuli Feng, Xianjing Han, Xin Yang, Wei Liu, and Liqiang Nie. 2018. Neural compatibility modeling with attentive knowledge distillation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 5--14.
36. Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, and Jun Ma. 2017. NeuroStylist: Neural compatibility modeling for clothing matching. In Proceedings of the ACM International Conference on Multimedia. 753--761.
37. Xuemeng Song, Liqiang Nie, Luming Zhang, Mohammad Akbari, and Tat-Seng Chua. 2015a. Multiple social network learning and its application in volunteerism tendency prediction. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 213--222.
38. Xuemeng Song, Liqiang Nie, Luming Zhang, Maofu Liu, and Tat-Seng Chua. 2015b. Interest inference via structure-constrained multi-source multi-task learning. In Proceedings of the International Joint Conference on Artificial Intelligence . AAAI Press, 2371--2377.
39. Guang Lu Sun, Zhi Qi Cheng, Xiao Wu, and Qiang Peng. 2017. Personalized clothing recommendation combining user social circle and fashion style consistency. Multimedia Tools and Applications , Vol. 77, 6, 1--24.
40. Thanh Tran, Kyumin Lee, Yiming Liao, and Dongwon Lee. 2018. Regularizing matrix factorization with user and item embeddings for recommendation. In Proceedings of the ACM International Conference on Information and Knowledge Management. ACM, 687--696.
41. Zheng Wang, Xiang Bai, Mang Ye, and Shin'ichi Satoh. 2018. Incremental deep hidden attribute learning. In Proceedings of the ACM International Conference on Multimedia. ACM, 72--80.
42. Xun Yang, Yunshan Ma, Lizi Liao, Meng Wang, and Tat-Seng Chua. 2018. TransNFCM: Translation-based neural fashion compatibility modeling. arXiv preprint arXiv:1812.10021 .
43. Jiangchao Yao, Yanfeng Wang, Ya Zhang, Jun Sun, and Jun Zhou. 2018. Joint latent dirichlet allocation for social tags. IEEE Transactions on Multimedia , Vol. 20, 1, 224--237.
44. Hongzhi Yin, Hongxu Chen, Xiaoshuai Sun, Hao Wang, Yang Wang, and Quoc Viet Hung Nguyen. 2017. SPTF: A scalable probabilistic tensor factorization model for semantic-aware behavior prediction. In Proceedings of the IEEE International Conference on Data Mining. 585--594.
45. Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Yue Gao, and Tat-Seng Chua. 2013. Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval. In Proceedings of the ACM International Conference on Multimedia. ACM, 33--42.

Index Terms

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

  • World Wide Web

    • Web searching and information discovery

      • Personalization