Athanasia Koumpouri
ABSTRACT
Until recently, the lack of user activity on search results was perceived as a sign of user dissatisfaction from retrieval performance. However, recent studies have reported that some queries might not be followed by clicks to the content of the retrieved results, because the search task can be satisfied in the list of retrieved results the user views without the need to click through them. In this paper, we propose a method for evaluating user satisfaction from the results of searches that are not followed by clickthrough activity to the retrieved results. We found that there is a strong association between some implicit measures of user activity and user's explicit satisfaction judgments. Moreover, we developed a predictive model of user satisfaction based on implicit measures, achieving accuracy up to 86%.
- Claypool, M., Lee, P., Wased, M., and Brown, D. 2001. Implicit interest indicators. In Proceedings on the International Conference on Intelligent User Interfaces. Santa Fe, NM. Google Scholar
Digital Library
- Fox, S., Karnawat, K., Mydland, M., Dumais, S. and White, T. 2005. Evaluating implicit measures to improve Web search. ACM Transactions on Information System. Google ScholarDigital Library
- Li, J., Huffman, S. B., and Tokuda, A. 2009. Good abandonment in mobile and PC internet search. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Boston, MA, USA. Google ScholarDigital Library
- Stamou, S. and Efthimiadis, E. 2010. Interpreting user inactivity on search results In Proceedings of the Advances in Information Retrieval, 32nd European Conference on IR Research, volume 5993 of Lecture Notes in Computer Science, 100--113. Google ScholarDigital Library
Index Terms
- Queries without clicks: evaluating retrieval effectiveness based on user feedback
Recommendations
New measures for the evaluation of interactive information retrieval systems: normalized task completion time and normalized user effectiveness
ASIS&T '10: Proceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem - Volume 47User satisfaction, though difficult to measure, is the main goal of Information Retrieval (IR) systems. In recent years, as Interactive Information Retrieval (IIR) systems have become increasingly popular, user effectiveness also has become critical in ...
Using clicks as implicit judgments: expectations versus observations
ECIR'08: Proceedings of the IR research, 30th European conference on Advances in information retrievalClickthrough data has been the subject of increasing popularity as an implicit indicator of user feedback. Previous analysis has suggested that user click behaviour is subject to a quality bias--that is, users click at different rank positions when ...
Predicting clicks: estimating the click-through rate for new ads
WWW '07: Proceedings of the 16th international conference on World Wide WebSearch engine advertising has become a significant element of the Web browsing experience. Choosing the right ads for the query and the order in which they are displayed greatly affects the probability that a user will see and click on each ad. This ...
Comments