Personalized query expansion for the web

ABSTRACT

The inherent ambiguity of short keyword queries demands for enhanced methods for Web retrieval. In this paper we propose to improve such Web queries by expanding them with terms collected from each user's Personal Information Repository, thus implicitly personalizing the search output. We introduce five broad techniques for generating the additional query keywords by analyzing user data at increasing granularity levels, ranging from term and compound level analysis up to global co-occurrence statistics, as well as to using external thesauri. Our extensive empirical analysis under four different scenarios shows some of these approaches to perform very well, especially on ambiguous queries, producing a very strong increase in the quality of the output rankings. Subsequently, we move this personalized search framework one step further and propose to make the expansion process adaptive to various features of each query. A separate set of experiments indicates the adaptive algorithms to bring an additional statistically significant improvement over the best static expansion approach.

References

1. J. Allan and H. Raghavan. Using part-of-speech patterns to reduce query ambiguity. In Proc. of the 25th Intl. ACM SIGIR Conf. on Research and development in information retrieval, 2002.
2. P. G. Anick and S. Tipirneni. The paraphrase search assistant: Terminological feedback for iterative information seeking. In Proc. of the 22nd Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 1999.
3. D. Carmel, E. Farchi, Y. Petruschka, and A. Soffer. Automatic query refinement using lexical affinities with maximal information gain. In Proc. of the 25th Intl. ACM SIGIR Conf. on Research and development in information retrieval, pages 283--290, 2002.
4. C. Carpineto, R. de Mori, G. Romano, and B. Bigi. An information-theoretic approach to automatic query expansion. ACM TOIS, 19(1):1--27, 2001.
5. C.-H. Chang and C.-C. Hsu. Integrating query expansion and conceptual relevance feedback for personalized web information retrieval. In Proc. of the 7th Intl. Conf. on World Wide Web, 1998.
6. P. A. Chirita, C. Firan, and W. Nejdl. Summarizing local context to personalize global web search. In Proc. of the 15th Intl. CIKM Conf. on Information and Knowledge Management, 2006.
7. S. Cronen-Townsend, Y. Zhou, and W. B. Croft. Predicting query performance. In Proc. of the 25th Intl. ACM SIGIR Conf. on Research and development in information retrieval, 2002.
8. H. Cui, J.-R. Wen, J.-Y. Nie, and W.-Y. Ma. Probabilistic query expansion using query logs. In Proc. of the 11th Intl. Conf. on World Wide Web, 2002.
9. T. Dunning. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19:61--74, 1993.
10. H. P. Edmundson. New methods in automatic extracting. Journal of the ACM, 16(2):264--285, 1969.
11. E. N. Efthimiadis. User choices: A new yardstick for the evaluation of ranking algorithms for interactive query expansion. Information Processing and Management, 31(4):605--620, 1995.
12. D. Fogaras and B. Racz. Scaling link based similarity search. In Proc. of the 14th Intl. World Wide Web Conf., 2005.
13. T. Haveliwala. Topic-sensitive pagerank. In Proc. of the 11th Intl. World Wide Web Conf., Honolulu, Hawaii, May 2002.
14. B. He and I. Ounis. Inferring query performance using pre-retrieval predictors. In Proc. of the 11th Intl. SPIRE Conf. on String Processing and Information Retrieval, 2004.
15. K. Järvelin and J. Keklinen. Ir evaluation methods for retrieving highly relevant documents. In Proc. of the 23th Intl. ACM SIGIR Conf. on Research and development in information retrieval, 2000.
16. G. Jeh and J. Widom. Scaling personalized web search. In Proc. of the 12th Intl. World Wide Web Conference, 2003.
17. M.-C. Kim and K.-S. Choi. A comparison of collocation-based similarity measures in query expansion. Inf. Proc. and Mgmt., 35(1):19--30, 1999.
18. S.-B. Kim, H.-C. Seo, and H.-C. Rim. Information retrieval using word senses: root sense tagging approach. In Proc. of the 27th Intl. ACM SIGIR Conf. on Research and development in information retrieval, 2004.
19. R. Kraft and J. Zien. Mining anchor text for query refinement. In Proc. of the 13th Intl. Conf. on World Wide Web, 2004.
20. R. Krovetz and W. B. Croft. Lexical ambiguity and information retrieval. ACM Trans. Inf. Syst., 10(2), 1992.
21. A. M. Lam-Adesina and G. J. F. Jones. Applying summarization techniques for term selection in relevance feedback. In Proc. of the 24th Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2001.
22. S. Liu, F. Liu, C. Yu, and W. Meng. An effective approach to document retrieval via utilizing wordnet and recognizing phrases. In Proc. of the 27th Intl. ACM SIGIR Conf. on Research and development in information retrieval, 2004.
23. G. Miller. Wordnet: An electronic lexical database. Communications of the ACM, 38(11):39--41, 1995.
24. L. Nie, B. Davison, and X. Qi. Topical link analysis for web search. In Proc. of the 29th Intl. ACM SIGIR Conf. on Res. and Development in Inf. Retr., 2006.
25. L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank citation ranking: Bringing order to the web. Technical report, Stanford Univ., 1998.
26. F. Qiu and J. Cho. Automatic indentification of user interest for personalized search. In Proc. of the 15th Intl. WWW Conf., 2006.
27. Y. Qiu and H. P. Frei. Concept based query expansion. In Proc. of the 16th Intl. ACM SIGIR Conf. on Research and Development in Inf. Retr., 1993.
28. J. Rocchio. Relevance feedback in information retrieval. The Smart Retrieval System: Experiments in Automatic Document Processing, pages 313--323, 1971.
29. I. Ruthven. Re-examining the potential effectiveness of interactive query expansion. In Proc. of the 26th Intl. ACM SIGIR Conf., 2003.
30. T. Sarlos, A. A. Benczur, K. Csalogany, D. Fogaras, and B. Racz. To randomize or not to randomize: Space optimal summaries for hyperlink analysis. In Proc. of the 15th Intl. WWW Conf., 2006.
31. C. Shah and W. B. Croft. Evaluating high accuracy retrieval techniques. In Proc. of the 27th Intl. ACM SIGIR Conf. on Research and development in information retrieval, pages 2--9, 2004.
32. K. Sugiyama, K. Hatano, and M. Yoshikawa. Adaptive web search based on user profile constructed without any effort from users. In Proc. of the 13th Intl. World Wide Web Conf., 2004.
33. D. Sullivan. The older you are, the more you want personalized search, 2004. http://searchenginewatch.com/searchday/article.php/3385131.
34. J. Teevan, S. Dumais, and E. Horvitz. Personalizing search via automated analysis of interests and activities. In Proc. of the 28th Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 2005.
35. E. Volokh. Personalization and privacy. Commun. ACM, 43(8), 2000.
36. E. M. Voorhees. Query expansion using lexical-semantic relations. In Proc. of the 17th Intl. ACM SIGIR Conf. on Res. and development in Inf. Retr., 1994.
37. J. Xu and W. B. Croft. Query expansion using local and global document analysis. In Proc. of the 19th Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, 1996.
38. S. Yu, D. Cai, J. -R. Wen, and W. -Y. Ma. Improving pseudo-relevance feedback in web information retrieval using web page segmentation. In Proc. of the 12th Intl. Conf. on World Wide Web, 2003.

Index Terms

• Information systems

  • Information retrieval

  • World Wide Web

    • Web applications

    • Web services