Wenjie Wang
Fumin Shen
ABSTRACT
The past decade has witnessed the boom of human-machine interactions, particularly via dialog systems. In this paper, we study the task of response generation in open-domain multi-turn dialog systems. Many research efforts have been dedicated to building intelligent dialog systems, yet few shed light on deepening or widening the chatting topics in a conversational session, which would attract users to talk more. To this end, this paper presents a novel deep scheme consisting of three channels, namely global, wide, and deep ones. The global channel encodes the complete historical information within the given context, the wide one employs an attention-based recurrent neural network model to predict the keywords that may not appear in the historical context, and the deep one trains a Multi-layer Perceptron model to select some keywords for an in-depth discussion. Thereafter, our scheme integrates the outputs of these three channels to generate desired responses. To justify our model, we conducted extensive experiments to compare our model with several state-of-the-art baselines on two datasets: one is constructed by ourselves and the other is a public benchmark dataset. Experimental results demonstrate that our model yields promising performance by widening or deepening the topics of interest.
- James F. Allen, Bradford W. Miller, Eric K. Ringger, and Teresa Sikorski . 1996. A Robust System for Natural Spoken Dialogue. In Proceedings of Annual Meeting of the Association for Computational Linguistics. ACL, 62--70. Google Scholar
Digital Library
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio . 2014. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:1409.0473 (2014).Google Scholar
- Jimmy Lei Ba. Diederik P. Kingma . 2015. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980 (2015).Google Scholar
- Warren R. Greiff . 1998. A Theory of Term Weighting Based on Exploratory Data Analysis Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 11--19. Google ScholarDigital Library
- Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan . 2016 a. A Diversity-Promoting Objective Function for Neural Conversation Models Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technologies. ACL, 110--119.Google Scholar
- Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao . 2016 b. Deep Reinforcement Learning for Dialogue Generation Proceedings of the Conference on Empirical Methods in Natural Language Processing. ACL, 1192--1202.Google Scholar
- Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu . 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset Proceedings of the International Joint Conference on Natural Language Processing. ACL, 986--995.Google Scholar
- Chia-Wei Liu, Ryan Lowe, Iulian Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau . 2016. How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation Proceedings of the Conference on Empirical Methods in Natural Language Processing. ACL, 2122--2132.Google Scholar
- Meng Liu, Liqiang Nie, Meng Wang, and Baoquan Chen . 2017. Towards Micro-video Understanding by Joint Sequential-Sparse Modeling Proceedings of the 2017 ACM on Multimedia Conference. ACM, 970--978. Google ScholarDigital Library
- Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau . 2015. The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems. In Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue. SIGDIAL, 285--294.Google ScholarCross Ref
- Hongyuan Mei, Mohit Bansal, and Matthew R. Walter . 2017. Coherent Dialogue with Attention-Based Language Models Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 3252--3258.Google ScholarDigital Library
- L. Nie, M. Wang, Y. Gao, Z. J. Zha, and T. S. Chua . 2013. Beyond Text QA: Multimedia Answer Generation by Harvesting Web Information. IEEE Transactions on Multimedia Vol. 15, 2 (2013), 426--441. Google ScholarDigital Library
- L. Nie, M. Wang, L. Zhang, S. Yan, B. Zhang, and T. S. Chua . 2015. Disease Inference from Health-Related Questions via Sparse Deep Learning. IEEE Transactions on Knowledge and Data Engineering, Vol. 27, 8 (2015), 2107--2119.Google ScholarDigital Library
- Liqiang Nie, Yi-Liang Zhao, Xiangyu Wang, Jialie Shen, and Tat-Seng Chua . 2014. Learning to Recommend Descriptive Tags for Questions in Social Forums. ACM Trans. Inf. Syst. Vol. 32, 1 (2014), 5:1--5:23. Google ScholarDigital Library
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu . 2002. BLEU: a Method for Automatic Evaluation of Machine Translation Proceedings of Annual Meeting of the Association for Computational Linguistics. ACL, 311--318. Google ScholarDigital Library
- Volha Petukhova, Martin Gropp, Dietrich Klakow, Anna Schmidt, Gregor Eigner, Mario Topf, Stefan Srb, Petr Motlicek, Blaise Potard, and John Dines . 2014. The DBOX Corpus Collection of Spoken Human-Human and Human-Machine Dialogues Proceedings of International Conference on Language Resources and Evaluation. ELRA, 252--258.Google Scholar
- Alan Ritter, Colin Cherry, and William B. Dolan . 2011. Data-driven Response Generation in Social Media. Proceedings of the Conference on Empirical Methods in Natural Language Processing. ACL, 583--593. Google ScholarDigital Library
- Thomas Roelleke . 2003. A Frequency-based and a Poisson-based Definition of the Probability of Being Informative Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval. ACM, 227--234. Google ScholarDigital Library
- Lina Maria Rojas-Barahona, Milica Gasic, Nikola Mrksic, Pei-Hao Su, Stefan Ultes, Tsung-Hsien Wen, Steve J. Young, and David Vandyke . 2017. A Network-based End-to-End Trainable Task-oriented Dialogue System Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics. ACL, 438--449.Google Scholar
- Iulian Serban, Tim Klinger, Gerald Tesauro, Kartik Talamadupula, Bowen Zhou, Yoshua Bengio, and Aaron Courville . 2017 a. Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 3288--3294.Google Scholar
- Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C. Courville, and Joelle Pineau . 2016. Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI Press, 3776--3784. Google ScholarDigital Library
- Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio . 2017 b. A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 3295--3301.Google Scholar
- Lifeng Shang, Zhengdong Lu, and Hang Li . 2015. Neural Responding Machine for Short-Text Conversation Proceedings of the Annual Meeting of the Association for Computational Linguistics on Natural Language Processing. ACL, 1577--1586.Google Scholar
- Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan . 2015. A Neural Network Approach to Context-Sensitive Generation of Conversational Responses Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technologies. ACL, 196--205.Google Scholar
- Ilya Sutskever, Oriol Vinyals, and Quoc V. Le . 2014. Sequence to Sequence Learning with Neural Networks Proceedings of the Neural Information Processing Systems Conference on Neural Information Processing Systems. MIT Press, 3104--3112. Google ScholarDigital Library
- Hao Wang, Zhengdong Lu, Hang Li, and Enhong Chen . 2013. A Dataset for Research on Short-Text Conversations Proceedings of the Conference on Empirical Methods in Natural Language Processing. ACL, 935--945.Google Scholar
- Mingxuan Wang, Zhengdong Lu, Hang Li, and Qun Liu . 2015. Syntax-Based Deep Matching of Short Texts. In Proceedings of the International Joint Conference on Artificial Intelligence. AAAI Press, 1354--1361. Google ScholarDigital Library
- Jason D. Williams, Antoine Raux, Deepak Ramachandran, and Alan W. Blac . 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL Conference on Discourse and Dialogue. SIGDIAL, 404--413.Google Scholar
- Jason D. Williams and Geoffrey Zweig . 2016. End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning. arXiv preprint arXiv:1606.01269 (2016).Google Scholar
- Ho Chung Wu, Robert Wing Pong Luk, Kam Fai Wong, and Kui Lam Kwok . 2008. Interpreting TF-IDF Term Weights As Making Relevance Decisions. ACM Transactions on Information System Vol. 26, 3 (2008), 13:1--13:37. Google ScholarDigital Library
- Yu Wu, Wei Wu, Chen Xing, Ming Zhou, and Zhoujun Li . 2017. Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. ACL, 496--505.Google ScholarCross Ref
- Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma . 2017. Topic Aware Neural Response Generation. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press, 3351--3357.Google Scholar
- Rui Yan, Yiping Song, and Hua Wu . 2016. Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System. In Proceedings of the International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 55--64. Google ScholarDigital Library
- Rui Yan, Dongyan Zhao, and Weinan E. . 2017. Joint Learning of Response Ranking and Next Utterance Suggestion in Human-Computer Conversation System. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 685--694. Google ScholarDigital Library
- Kaisheng Yao, Geoffrey Zweig, and Baolin Peng . 2015. Attention with Intention for a Neural Network Conversation Model. arXiv preprint arXiv:1510.08565 (2015).Google Scholar
Index Terms
- Chat More: Deepening and Widening the Chatting Topic via A Deep Model
Recommendations
Response generation in multi-modal dialogues with split pre-generation and cross-modal contrasting
AbstractDue to the natural multi-modal occurrence format (text, audio, vision) of the dialogues, textual response generation in dialogues should rely on the multi-modal contexts beyond text only. However, most existing studies normally ignore the rich ...
Highlights- Exploration of the multi-modal scenario with aligned text and audio temporal sequences for textual response generation.
- Split pre-generation strategy is proposed to generate diverse responses.
- Cross-modal contrastive learning ...
Extending the Transformer with Context and Multi-dimensional Mechanism for Dialogue Response Generation
Natural Language Processing and Chinese ComputingAbstractThe existing work of using generative model in multi-turn dialogue system is often based on RNN (Recurrent neural network) even though the Transformer structure has achieved great success in other fields of NLP. In the multi-turn conversation task,...
A multimodal dialogue system for improving user satisfaction via knowledge-enriched response and image recommendation
AbstractTask-oriented multimodal dialogue systems have important application value and development prospects. Existing methods have made significant progress, but the following challenges still exist: (1) Most existing methods focus on improving the ...
Comments