CQAVis: Visual Text Analytics for Community Question Answering

ABSTRACT

Community question answering (CQA) forums can provide effective means for sharing information and addressing a user's information needs about particular topics. However, many such online forums are not moderated, resulting in many low quality and redundant comments, which makes it very challenging for users to find the appropriate answers to their questions. In this paper, we apply a user-centered design approach to develop a system, CQAVis, which supports users in identifying high quality comments and get their questions answered. Informed by the user's requirements, the system combines both text analytics and interactive visualization techniques together in a synergistic way. Given a new question posed by the user, the text analytic module automatically finds relevant answers by exploring existing related questions and the comments within their threads. Then the visualization module presents the search results to the user and supports the exploration of related comments. We have evaluated the system in the wild by deploying it within a CQA forum among thousands of real users. Through the online study, we gained deeper insights about the potential utility of the system, as well as learned generalizable lessons for designing visual text analytics systems for the domain of CQA forums.

References

1. 2016. Colorbrewer. (2016). http://colorbrewer2.org/.
2. Alberto Barrón-Cedeño, Simone Filice, Giovanni Da San Martino, Shafiq Joty, Lluís Màrquez, Preslav Nakov, and Alessandro Moschitti. 2015. Thread-Level Information for Comment Classification in Community Question Answering. In Proc. ACL-IJCNLP.
3. Sheelagh Carpendale. 2008. Evaluating information visualizations. In Information Visualization: Human-Centered Issues and Perspectives. Springer, 19--45.
4. Andy Cockburn, Amy Karlson, and Benjamin B Bederson. 2008. A review of overview+ detail, zooming, and focus+ context interfaces. ACM Computing Surveys (CSUR) 41, 1 (2008), 2.
5. Michael Collins and Nigel Duffy. 2002. Convolution Kernels for Natural Language. In Advances in NIPS, T. G. Dietterich, S. Becker, and Z. Ghahramani (Eds.). MIT Press, 625--632.
6. Kushal Dave, Martin Wattenberg, and Michael Muller. 2004. Flash forums and forumReader: navigating a new kind of large-scale online discussion. In Proc. ACM Conf. on CSCW. 232--241.
7. Park Deokgun, Sachar Simranjit, Diakopoulos Nicholas, and Elmqvist Niklas. 2016. Supporting Comment Moderators in Identifying High Quality Online News Comments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 769--778.
8. Cicero dos Santos, Luciano Barbosa, Dasha Bogdanova, and Bianca Zadrozny. 2015. Learning Hybrid Representations to Retrieve Semantically Equivalent Questions. In Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 694--699.
9. Nicole B Ellison, Rebecca Gray, Jessica Vitak, Cliff Lampe, and Andrew T Fiore. 2013. Calling All Facebook Friends: Exploring Requests for Help on Facebook.. In Proceedings of the ICWSM.
10. Minwei Feng, Bing Xiang, Michael R Glass, Lidan Wang, and Bowen Zhou. 2015. Applying deep learning to answer selection: A study and an open task. arXiv preprint arXiv:1508.01585 (2015).
11. F. Maxwell Harper, Daphne Raban, Sheizaf Rafaeli, and Joseph A. Konstan. 2008. Predictors of Answer Quality in Online Q&A Sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). 865--874.
12. Marti A. Hearst. 2011. Natural Search User Interfaces. Commun. ACM 54, 11 (Nov. 2011), 60--67.
13. Enamul Hoque and Giuseppe Carenini. 2014. ConVis: A Visual Text Analytic System for Exploring Blog Conversations. Computer Graphics Forum (Proc. EuroVis) 33, 3 (2014), 221--230.
14. Enamul Hoque and Giuseppe Carenini. 2015. ConVisIT: Interactive Topic Modeling for Exploring Asynchronous Online Conversations. In Proc. ACM conf. on Intelligent User Interfaces.
15. Enamul Hoque and Giuseppe Carenini. 2016a. Interactive Topic Modeling for Exploring Asynchronous Online Conversations: Design and Evaluation of ConVisIT. ACM Transactions on Interactive Intelligent Systems 6, 1 (Feb. 2016), 7:1--7:24.
16. Enamul Hoque and Giuseppe Carenini. 2016b. MultiConVis: A Visual Text Analytics System for Exploring a Collection of Online Conversations. In Proc. ACM IUI. 96--107.
17. Damon Horowitz and Sepandar D Kamvar. 2010. The anatomy of a large-scale social search engine. In Proceedings of the 19th international conference on World wide web. ACM, 431--440.
18. Gary Hsieh and Scott Counts. 2009. mimir: A market-based real-time question and answer service. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 769--778.
19. Jiwoon Jeon, W. Bruce Croft, and Joon Ho Lee. 2005. Finding Similar Questions in Large Question and Answer Archives. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management (CIKM '05). 84--90.
20. Bernard Kerr. 2003. Thread arcs: An email thread visualization. In IEEE Symposium on Information Visualization. 211--218.
21. H. Lam. 2008. A Framework of Interaction Costs in Information Visualization. IEEE Trans. Visualization & Comp. Graphics 14, 6 (2008), 1149--1156.
22. H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale. 2012. Empirical Studies in Information Visualization: Seven Scenarios. IEEE Trans. Visualization & Comp. Graphics 18, 9 (2012), 1520--1536.
23. Qiaoling Liu, Tomasz Jurczyk, Jinho Choi, and Eugene Agichtein. 2015. Real-Time Community Question Answering: Exploring Content Recommendation and User Notification Strategies. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI '15). 50--61.
24. Meredith Ringel Morris, Jaime Teevan, and Katrina Panovich. 2010. What Do People Ask Their Social Networks, and Why?: A Survey Study of Status Message Q&A Behavior. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). 1739--1748.
25. Alessandro Moschitti. 2006. Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees. In Machine Learning: ECML 2006, Johannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou (Eds.). Lecture Notes in Computer Science, Vol. 4212. 318--329.
26. Tamara Munzner. 2014. Visualization Analysis and Design. CRC Press.
27. Preslav Nakov, Lluís Màrquez, Walid Magdy, Alessandro Moschitti, Jim Glass, and Bilal Randeree. 2015. SemEval-2015 Task 3: Answer Selection in Community Question Answering. In Proc. Workshop SemEval. 269--281.
28. Preslav Nakov, Lluís Màrquez, Alessandro Moschitti, Walid Magdy, Hamdy Mubarak, Abed Alhakim Freihat, Jim Glass, and Bilal Randeree. 2016. SemEval-2016 Task 3: Community Question Answering. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval '16). Association for Computational Linguistics, San Diego, California.
29. Massimo Nicosia, Simone Filice, Alberto Barrón-Cedeño, Iman Saleh, Hamdy Mubarak, Wei Gao, Preslav Nakov, Giovanni Da San Martino, Alessandro Moschitti, Kareem Darwish, Lluís Màrquez, Shafiq Joty, and Walid Magdy. 2015. QCRI: Answer Selection for Community Question Answering - Experiments for Arabic and English. In Proc. Workshop SemEval.
30. Vıctor Pascual-Cid and Andreas Kaltenbrunner. 2009. Exploring asynchronous online discussions through hierarchical visualisation. In IEEE Conf. on Information Visualization. 191--196.
31. Xipeng Qiu and Xuanjing Huang. 2015. Convolutional Neural Tensor Network Architecture for Community-based Question Answering. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI 2015).
32. D Raban and F Harper. 2008. Motivations for answering questions online. New media and innovative technologies 73 (2008).
33. Warren Sack. 2000. Conversation map: an interface for very-large-scale conversations. Journal of Management Information Systems 17, 3 (2000), 73--92.
34. Michael Sedlmair, Miriah Meyer, and Tamara Munzner. 2012. Design study methodology: reflections from the trenches and the stacks. IEEE Trans. Visualization & Comp. Graphics 18, 12 (2012), 2431--2440.
35. Aliaksei Severyn and Alessandro Moschitti. 2012. Structural Relationships for Large-scale Learning of Answer Re-ranking. In Proc. SIGIR. 741--750.
36. Aliaksei Severyn and Alessandro Moschitti. 2015. Learning to rank short text pairs with convolutional deep neural networks. In Proc. SIGIR. 373--382.
37. Yikang Shen, Wenge Rong, Zhiwei Sun, Yuanxin Ouyang, and Zhang Xiong. 2015. Question/Answer Matching for CQA System via Combining Lexical and Sequential Information.. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2015). 275--281.
38. Ming Tan, Bing Xiang, and Bowen Zhou. 2015. LSTM-based Deep Learning Models for non-factoid answer selection. arXiv preprint arXiv:1511.04108 (2015).
39. Gina Danielle Venolia and Carman Neustaedter. 2003. Understanding sequence and reply relationships within email conversations: a mixed-model visualization. In Proc. Conf. CHI. 361--368.
40. Fernanda B Viégas, Scott Golder, and Judith Donath. 2006. Visualizing email content: portraying relationships from conversational histories. In Proc. Conf. CHI. 979--988.
41. Di Wang and Eric Nyberg. 2015. A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering. In ACL. 707--712.
42. Martin Wattenberg and David Millen. 2003. Conversation thumbnails for large-scale discussions. In extended abstracts on CHI. 742--743.
43. Furu Wei, Shixia Liu, Yangqiu Song, Shimei Pan, Michelle X Zhou, Weihong Qian, Lei Shi, Li Tan, and Qiang Zhang. 2010. Tiara: a visual exploratory text analytic system. In Proc. ACM Conf. on Knowledge Discovery and Data Mining. 153--162.
44. Jian Zhao, C. Collins, F. Chevalier, and R. Balakrishnan. 2013. Interactive Exploration of Implicit and Explicit Relations in Faceted Datasets. IEEE Trans. Visualization & Comp. Graphics 19, 12 (2013), 2080--2089.
45. Guangyou Zhou, Tingting He, Jun Zhao, and Po Hu. 2015a. Learning Continuous Word Embedding with Metadata for Question Retrieval in Community Question Answering. In Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 250--259.
46. Xiaoqiang Zhou, Baotian Hu, Qingcai Chen, Buzhou Tang, and Xiaolong Wang. 2015b. Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 713--718.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Natural language processing

• Human-centered computing

  • Human computer interaction (HCI)