A comparison of query and term suggestion features for interactive searching

ABSTRACT

Query formulation is one of the most difficult and important aspects of information seeking and retrieval. Two techniques, term relevance feedback and query suggestion, provide methods to help users formulate queries, but each is limited in different ways. In this research we combine these two techniques by automatically creating query suggestions using term relevance feedback techniques. To evaluate our approach, we conducted an interactive information retrieval study with 55 subjects and 20 topics. Each subject completed four topics, half with a term suggestion system and half with a query suggestion system. We also investigated the source of the suggestions: approximately half of all subjects were provided with system-generated suggestions, while half were provided with user-generated suggestions. Results show that subjects used more query suggestions than term suggestions and saved more documents with these suggestions, even though there were no significant differences in performance. Subjects preferred the query suggestion system and rated it higher along a number of dimensions including its ability to help them think of new approaches to searching. Qualitative data provided insight into subjects' usage and ratings, and indicated that subjects often used the suggestions even when they did not click on them.

References

1. Beaulieu, M.&Jones, S. (1998). Interactive searching and interface issues in the Okapi best match probabilistic retrieval system. Interacting with Computers, 10, 237--248.
2. Beerferman, D.,&Berger, A. (2000). Agglomerative clustering of a search engine query log. Proceedings ACM SIGKDD, Boston, MA, 407--416.
3. Belkin, N. J., Cool, C., Kelly, D., Lin, S. J., Park, S. Y., Perez-Carballo, J.,&Sikora, C. (2001). Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Information Processing&Management, 37(3), 404--434.
4. Billerbeck, B., Scholer, F., Williams, H. E.,&Zobel, J. (2003). Query expansion using associated queries. Proceedings CIKM '03, New Orleans, LA, 2--9.
5. Fitzpatrick, L.,&Dent, M. (1997). Automatic feedback using past queries: Social searching? Proceedings of SIGIR '97, Philadelphia, PA, 306--313.
6. Freyne, J., Farzan, R., Brusilovsky, P., Smyth, B.,&Coyle, M. (2007). Collecting community wisdom: Integrating social search&social navigation. Proceedings of IUI '07, Honolulu, HI, 52--61.
7. Hsieh-Yee, I. (1993).Effects of search experience and subject knowledge on online search behavior: Measuring the search tactics of novice and experienced searchers. Journal of the American Society for Information Science 44, 161--174.
8. Järvelin, K.&Kekääläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20, 422--446.
9. Järvelin, K., Price, S. L, Delcambre, L. M. L.,&Nielsen, M. L. (2008). Discounted cumulated gain based evaluation of multiple-query IR sessions. Proceedings of ECIR '08, Glasgow, Scotland.
10. Joho, H., Coverson, C., Sanderson, M.,&Beaulieu, M. (2002). Hierarchical presentation of expansion terms. In Proceedings of the 17th Annual ACM Symposium on Applied Computing (SAC '02), Madrid, Spain, 645--649.
11. Jones, R., Rey, B., Madani, O.,&Greiner, W. (2006). Generating query substitutions. Proceedings of the WWW Conference, Edinburgh, Scotland, 387--396.
12. Kelly, D.,&Fu, X. (2006). Elicitation of term relevance feedback: An investigation of term source and context. Proceedings of SIGIR '06, 453--460.
13. Kelly, D., Gyllstrom, K.,&Bailey, E. W. (2009). Remote vs. face-to-face study mode: Differences in user behaviors, performance and feedback. SILS Tech. Report, 2009-02 (http://sils.unc.edu/research/techreports.html).
14. Mei, Q., Zhou, D.,&Church, K. (2008). Query suggestion using hitting time. Proceedings CIKM, Napa Valley, CA, 469--477.
15. Raghavan, V. V.,&Sever, H. (1995). On the reuse of past optimal queries. Proceedings of SIGIR '95, Seattle, WA, 344-350.
16. Ruthven, I. (2003). Re-examining the potential effectiveness of interactive query expansion. Proceedings of SIGIR '03, Toronto, CA, 213--220.
17. Ruthven, I.,&Lalmas, M. (2003). A survey on the use of relevance feedback for information access systems. Knowledge Engineering Review, 18(2), 95--145.
18. Smyth, B., Balfe, E., Freyne, J., Briggs, P., Coyle, M.,&Boydell, O. (2004). Exploiting query repetition and regularity in an adaptive community-based Web search Engine. User Modeling and User-Adapted Interaction, 14(5), 382--423.
19. White, R. W., Bilenko, M.,&Cucerzan, S. (2007). Studying the use of popular destinations to enhance web search interaction. Proceedings of SIGIR '07, Amsterdam, The Netherlands, 159--166.
20. White, R. W.&Marchionini, G. (2007). Examining the effectiveness of real-time query expansion. Information Processing&Management, 43, 685--704.
21. Vakkari, P. (2004). Changes in search tactics and relevance judgments when preparing a research proposal: A summary of the findings of a longitudinal study. Information Retrieval, 4(3--4), 295--310.
22. Voorhees, E. M. (2006). Overview of the TREC 2005 Robust Retrieval Track. Proceedings of TREC-14.

Index Terms

• Information systems

  • Information retrieval

    • Information retrieval query processing