Analysis of Open Answers to Survey Questions through Interactive Clustering and Theme Extraction

ABSTRACT

This paper describes design principles for and the implementation of Gavagai Explorer---a new application which builds on interactive text clustering to extract themes from topically coherent text sets such as open text answers to surveys or questionnaires. An automated system is quick, consistent, and has full coverage over the study material. A system allows an analyst to analyze more answers in a given time period; provides the same initial results regardless of who does the analysis, reducing the risks of inter-rater discrepancy; and does not risk miss responses due to fatige or boredom. These factors reduce the cost and increase the reliability of the service. The most important feature, however, is relieving the human analyst from the frustrating aspects of the coding task, freeing the effort to the central challenge of understanding themes. Gavagai Explorer is available on-line.

References

1. Douglass R Cutting, David R Karger, Jan O Pedersen, and John W Tukey. 1992. Scatter/gather: A cluster-based approach to browsing large document collections. In Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval. ACM.
2. Gavagai. 2016. What do you most wish for the coming year? Stockholm. http: //gavagai.se/wp-content/uploads/2016/03/AMFPension-CustomerCase.pdf
3. Gavagai. 2017. What makes airline passengers happy? Stockholm. http://gavagai. se/blog/2017/04/24/what-makes-airline-passengers-happy/
4. Amaru Cuba Gyllensten and Magnus Sahlgren. 2015. Navigating the Semantic Horizon using Relative Neighborhood Graphs. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP.
5. Svenska institutet. 2016. Feministisk utrikespolitik: rott skynke eller vit flagg' ¨ Stockholm. https://si.se/wp-content/uploads/2016/12/Sverigebilden-Rapportom synen pa -jamstalldhet.pdf (In Swedish; A slide deck with a summary in English is at http://gavagai.se/Gender Equality Study.pdf).
6. Nick Jardine and Cornelis Joost van Rijsbergen. 1971. The use of hierarchic clustering in information retrieval. Information storage and retrieval 7, 5 (1971).
7. Sofus A. Macskassy, Arunava Banerjee, Brian D. Davison, and Haym Hirsh. 1998. Human Performance on Clustering Web Pages: A Preliminary Study. In Proceedings of the Conference on Knowledge Discovery and Data Mining (KDD).
8. Andrew McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. 1999. A machine learning approach to building domain-specific search engines. In Proceedings of the International Joint Conference on Artificial Intelligence. IJCAI.
9. Alicia O'Cathain and Kate J Thomas. 2004. " Any other comments?" Open questions on questionnaires--a bane or a bonus to research? BMC medical research methodology 4, 1 (2004).
10. Peter Pirolli, Patricia Schank, Marti Hearst, and Christine Diehl. 1996. Scatter/gather browsing communicates the topic structure of a very large text collection. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM.
11. Dmitri G Roussinov and Hsinchun Chen. 1999. Document clustering for electronic meetings: an experimental comparison of two techniques. Decision Support Systems 27, 1 (1999).
12. Magnus Sahlgren, Amaru Cuba Gyllensten, Fredrik Espinoza, Ola Hamfors, Jussi Karlgren, Fredrik Olsson, Per Persson, Akshay Viswanathan, and Anders Holst. 2016. The Gavagai Living Lexicon. In Language Resources and Evaluation Conference. ELRA.
13. Mark Sanderson and Bruce Croft. 1999. Deriving concept hierarchies from text. In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval. ACM.
14. Greg Schohn and David Cohn. 2000. Less is More: Active Learning with Support Vector Machines. In Proceedings of the International Conference on Machine Learning. ACM.

Index Terms

• Human-centered computing

  • Interaction design

    • Interaction design process and methods

      • User interface design

• Information systems

  • Data management systems

    • Database management system engines

      • Online analytical processing engines

  • Information systems applications

    • Data mining

      • Clustering