Reconsideration of the simulated work task situation: a context instrument for evaluation of information retrieval interaction

ABSTRACT

The present paper reports on the initial study and the preliminary findings of how the concept of simulated work task situation is reported used in the research literature. The overall objective of the study is in a systematic manner to learn how and for what types of evaluations the concept is applied. In particular we are interested to learn whether the recommendations for how to apply simulated work task situations are followed.

The preliminary findings indicate a need for clarifications of the recommendations of how to use simulated work task situations. Particularly with respect to 'realism' of the simulated work task situations, which is emphasised through the need for tailoring of the simulated work task situations towards the group of study participant to ensure the depicted situations are realistic and interesting from the participants' point of view. Likewise it seems that the recommendation to involve the study participants' own information needs (to function as baseline of search interaction) is generally neglected in the reported studies.

References

1. Airio, E. (2008). Who benefits from CLIR in web retrieval? Journal of Documentation, 64(5), 760--778.
2. Arapakis, I., Jose, J.m. & Gray, P.D. (2008). Affective Feedback: An investigation into the role of emotion in the information seeking process. In: Myaeng, S.H., Oard, D.W. & Sebastiani, F. (eds). Proceedings of the 31st ACM Sigir Conference on Research and Development of Information Retrieval (SIGIR 2008). Singapore, 2008. ACM Press, 395--402.
3. Bell, J.D. & Ruthven, I. (2004). Searcher's assessment of task complexity for web searching. In: McDonald, S. & Tait, J. (eds). Proceeding of the 26th European Conference on Information Retrieval (ECIR 2004). 5th-7th April 2004, University of Sunderland, U.K. Berlin/Heidelberg : Springer Verlag, 57-71.
4. Bierig, R. & A. Goker. (2006). Time, location and interest: An empirical and user-centred study. In: Ruthven, I., Borlund, P., Ingwersen, P., Belkin, N., Tombros, A. & Vakkari, P. (eds.). Proceedings of the 1st International Symposium on Information Interaction in Context (IIiX 2006). October 18-20 2006, Copenhagen, Denmark. ACM Press, 79--87.
5. Bierig, R. (2008). Event and map content personalisation in a mobile and context-aware environment. Robert Gordon University. 2008. {Doctoral Thesis}.
6. Blandford, A. et al. (2008). The PRET A Rapporter framework: Evaluating digital libraries from the perspective of information work. Information Processing & Management, 44(1), 4--21.
7. Blomgren, L., Vallo, H. & Bystrom, K. (2004). Evaluation of an information system in an information seeking process. In: Heery, R. & Lyon, L. (eds.). Proceedings of the 8th International European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2004). September 12-17 2004, Bath, UK. Berlin/Heidelberg : Springer Verlag, 57--68.
8. Borlund, P. & Ingwersen, P. (1997). The development of a method for the evaluation of interactive information retrieval systems. Journal of Documentation, 53(3), 225--250.
9. Borlund, P. & Ingwersen, P. (1998). Measures of relative relevance and ranked half-life: performance indicators for interactive IR. In: Croft, B.W, Moffat, A., van Rijsbergen, C.J, Wilkinson, R. & Zobel, J., (eds.). Proceedings of the 21st ACM Sigir Conference on Research and Development of Information Retrieval (SIGIR 1998). Melbourne, Australia, 1998. ACM Press/York Press, 324--331.
10. Borlund, P. & Ingwersen, P. (1999). The application of work tasks in connection with the evaluation of interactive information retrieval systems: empirical results. In: Draper, S.W., Dunlop, M.D., Ruthven, I. & van Rijsbergen, C.J., (eds.). Mira '99: Evaluating interactive information retrieval, Glasgow, UK, 14-16 April 1999. British Computer Society, 1--18.
11. Borlund, P. (2000). Evaluation of Interactive Information Retrieval Systems. Åbo Akademi University : Åbo Akademi University Press. 2000. {Doctoral thesis, Åbo/Turku}.
12. Borlund, P. (2000). Experimental components for the evaluation of interactive information retrieval systems. Journal of Documentation, 56(1), 71--90.
13. Borlund, P. (2003). The concept of relevance in IR. Journal of the American Society for Information Science and Technology, 54(10), 913--925.
14. Borlund, P. (2003). The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3). Available at: http://informationr.net/ir/8-3/paper152.html
15. Borlund, P. (2009). User--centred evaluation of information retrieval systems. In: Goker, A. & Davies, J. (eds.). Information Retrieval: Searching in the 21st Century. John Wiley & Sons, Ldt. ISBN-13: 978-0470027622. 21--37.
16. Brajnik, G., Mizzaro, S. & Tasso, C. (2002). Strategic help in user interfaces for information retrieval. Journal of the American Society for Information Science and Technology, 53(5), 343--358.
17. Byström, K. & Hansen, P. (2005). Conceptual framework for tasks in information studies. Journal of the American Society for Information Science and Technology, 56(10), 1050--1061.
18. Chen, K.H. (2002). Evaluating Chinese text retrieval with multilingual queries. Knowledge Organization, 29(3), 156--170.
19. Cole, C. (2001). The concept of situation in Information Science. In: Williams, M.E. (ed.). Annual Review of Information Science and Technology (ARIST). Medford, NJ : Learned Information, INC., 41, 5--42.
20. Craven J. & Nietzio, A. (2007). A task-based approach to assessing the accessibility of web sites. Performance Measurement and Metrics, 8(2), 98--109.
21. Craven, J. (2003). Access to electronic resources by visually impaired people. Information Research, 8(4). Available at: http://informationr.net/ir/8-4/paper156.html
22. Fachry, K.N., Kamps, J. & Zhang, J. (2008). Access to Archival Material in Context. In: Borlund, P., Schneider, J.W., Tombros, A., Lalmas, M., Feather, J., Kelly, D., de Vries, A. & Azzopardi, L. (eds.). Proceeding of the second IIiX Symposium on Information Interaction in Context (IIiX 2008). London, UK, October 2008. ACM Press, 102--109.
23. Freund, L. & Toms, E.G. (2006). Enterprise search behavior of software engineers. In: Efthimiadis, E.N., Dumais, S.T., Hawking, D. & Järvelin, K. (eds.). Proceedings of the 29th ACM Sigir Conference on Research and Development of Information Retrieval (SIGIR 2006). Seattle, Washington, USA, August 6-11, 2006. ACM Press, 645--646.
24. Freund, L.S. (2008). Exploiting task-document relations in support of information retrieval in the workplace. University of Toronto. 2008. {Doctoral Thesis}.
25. Göker, A. & Myrhaug, H. (2008). Evaluation of a mobile information system in context. Information Processing & Management, 44(1), 39--65.
26. Haggerty, A., White, R.W. & Jose, J.M. (2003). NewsFlash: Adaptive TV news delivery on the web. In: Nurnberger, A. & Detyniecki, M. (eds.). Proceedings of the 1st International Workshop on Adaptive Multimedia Retrieval, Hamburg (AMR 2003). September 15--16 2003, Germany. Berlin/Heidelberg : Springer Verlag, 72--86.
27. Hammer--Aebi, B., Christensen, K.W., Lund, H. & Larsen, B. (2006). Users, structured documents and overlap: Interactive searching of elements and the influence of context on search behavior. In: Ruthven, I., Borlund, P., Ingwersen, P., Belkin, N., Tombros, A. & Vakkari, P. (eds.). Proceedings of the 1st International Symposium on Information Interaction in Context (IIiX 2006). October 18-20 2006, Copenhagen, Denmark. ACM Press, 46--55.
28. Harper, D.J., Koychev, I. & Sun, Y.X. (2003). Query-based document skimming: A user-centred evaluation of relevance profiling. In: Sebastiani, F. (ed). Proceeding of the 25th European Conference on Information Retrieval (ECIR 2003). Pisa, Italy. Berlin/Heidelberg : Springer Verlag, 377--392.
29. He, D.Q., Brusiloviksy, P., Grady, J., Li, Q. & Ahn, J.W. (2007). How up-to-date should it be? In: Proceedings of ACM International Conference on Web Intelligence (WI 2007), 2-5 November 2007, Silicon Valley, CA, USA. IEEE 2007, 699--705.
30. He, D.Q., Brusilovsky, P., Ahn, J., Grady, J., Farzan, R., Peng, Y., Yang, Y. & Rogati, M. (2008). An evaluation of adaptive filtering in the context of realistic task-based information exploration. Information Processing & Management, 44(2), 511--533.
31. Hoeber, O. (2007). Exploring Web search results by visually specifying utility functions. In: Proceedings of ACM International Conference on Web Intelligence (WI 2007). 2-5 November 2007, Silicon Valley, CA, USA. IEEE, 650--654.
32. Ingwersen, P. (1992). Information retrieval interaction. London: Taylor Graham. X, 246 pp. Available at: http://vip.db.dk/pi/iri/index.htm
33. Ingwersen, P. & Järvelin, k. (2005). The turn: Integration of information seeking and retrieval in context. Dordrecht, The Netherlands : Springer Verlag. ISBN:140203850X.
34. Jansen, B.J. & McNeese, M.D. (2005). Evaluating the effectiveness of and patterns of interactions with automated searching assistance. Journal of the American Society for Information Science and Technology, 56(14), 1480--1503.
35. Järvelin, K. (2007). An analysis of two approaches in information retrieval: From frameworks to study designs. Journal of the American Society for Information Science and Technology, 58(7), 971--986.
36. Joho, H. & Jose, J.M. (2005). A comparative study of the effectiveness of search result presentation on the Web. In: Lalmas, M., MacFarlane, A., Rüger, S.M., Tombros, A., Tsikrika, T. & Yavlinsky A. (eds.). Proceedings of the 28th European Conference on IR Research (ECIR 2006). London, UK, April 10-12, 2006. Berlin/Heidelberg : Springer Verlag, 302--313.
37. Joho, H. & Jose, J. M. (2006). Slicing and dicing the information space using local context. In: Ruthven, I., Borlund, P., Ingwersen, P., Belkin, N., Tombros, A. & Vakkari, P. (eds.). Proceedings of the 1st International Symposium on Information Interaction in Context (IIiX 2006). October 18--20 2006, Copenhagen, Denmark. ACM Press, 66--74.
38. Joho, H. & Jose, J.M. (2008). Effectiveness of additional representations for the search result presentation on the web. Information Processing & Management, 44(1), 226--241.
39. Joho, H., Hannah, D. & Jose, J. M. (2008). Comparing collaborative and independent search in a re-call-oriented task. In: Borlund, P., Schneider, J.W., Tombros, A., Lalmas, M., Feather, J., Kelly, D., de Vries, A. & Azzopardi, L. (eds.). Proceeding of the second IIiX Symposium on Information Interaction in Context (IIiX 2008). London, UK, October 2008. ACM Press, 89--96.
40. Jose, J.M., Furner, J. & Harper, D.J. (1998). Spatial querying for image retrieval: a user-oriented evaluation. In: Croft, B.W, Moffat, A., van Rijsbergen, C.J, Wilkinson, R. & Zobel, J. (eds.). Proceedings of the 21st ACM Sigir Conference on Research and Development of Information Retrieval (SIGIR 1998). Melbourne, Australia, 1998. ACM Press/York Press, 323--240.
41. Karlgren, J. & Hansen, P. (2002). SICS at WLEF 2002: Cross-language relevance assessment and task context. In: Peters, C., Braschler,M., GonzaloJ. & Kluck, M. (eds.). Proceedings of the 3rd Workshop of the Cross-Language Evaluation Forum (CLEF 2002). Rome, Italy, September 19--20, 2002. Berlin/Heidelberg : Springer Verlag, 383--391.
42. Kekäläinen, J. (2005). Binary and graded relevance in IR evaluations: Comparison of the effects on ranking of IR systems. Information Processing & Management, 41, 1019--1033.
43. Kelly, D. (2009). Methods for evaluating interactive information retrieval systems with users. Foundations and Trends in Information Retrieval, 3(1-2), 1--224.
44. Kelly, D. & Belkin N.J. (2004). Display times as implicit feedback: Understanding task effects. In: Sanderson, M., Järvelin, K., Allan, J. & Bruza, P. (eds.). Proceedings of the 27th Conference on Research and Development in Information Retrieval (SIGIR 2004). Sheffield, UK, July 25-29, 2004. ACM, 377--384.
45. Kelly, D., Wacholder, N., Rittman, R., Sun, Y., Kantor, P., Small, S. & Strzalkowski, T. (2007). Using interview data to identify evaluation criteria for interactive, analytical question-answering systems. Journal of the American Society for Information Science and Technology, 58(7), 1032--1043.
46. Kim, J. (2007). Task as a context of information seeking: An investigation of daily life tasks on the web. Libri, 58, 172--181.
47. Kules, B. & Shneiderman, B. (2008). Users can change their web search tactics: Design guidelines for categorized overviews. Information Processing & Management, 44(2), 463--484.
48. Langer, L. & Frøkjær, E. (2008). Improving Web search transparency by using a venn diagram interface. In: Tollmar, K. & Jönsson, B. (eds). Proceedings of the 5th Nordic conference on human-computer interaction: Building bridges. Lund, Sweden, October 2008. ACM, 249--256.
49. Larsen, B. (2002). Exploiting citation overlaps for information retrieval: Generating a boomerang effect from the network of scientific papers. Scientometrics, 54(2), 155--178.
50. Larsen, B., Malik, S. & Tombros, A. (2005). The interactive track at INEX 2005. In: Fuhr, N., Lalmas, M., Malik, S. & Kazai, G. (eds). Proceedings of the 4th International Workshop of the Initiative for the Evaluation of XML Retrieval (INEX 2005). Dagstuhl Castle, Germany, November 28-30, 2005. Berlin/Heidelberg : Springer Verlag, 398--410.
51. Malik, S., Klas, C.P., Fuhr, N., Larsen, B. & Tombros, A. (2006). Designing a user interface for interactive retrieval of structured documents: Lessons learned from the INEX interactive track. In: Gonzalo, J. et al (eds). Proceedings of 10th European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2006), Alicante, Spain, September 17-22 2006. Berlin/Heidelberg : Springer Verlag, 291--302.
52. Malik, S., Larsen, b. & Tombros, A. (2007). Report on the INEX interactive track. ACM SIGIR Forum, June 2007, 41(1), 67--74
53. Mizzaro, S. (1998). How many relevances in information retrieval? Interacting with Computers, 10(3), 303--320.
54. Nielsen, M.L. (2004). Task-based evaluation of associative thesaurus in real-life environment. In: Schamber, L. & Barry, C.L. (eds.). The Proceedings of the 67th ASIS&T Annual Meeting. Providence, RI., November 12-17 2004. Medford, NJ: Information Today, Inc., 437--447.
55. Nielsen, M.L. (2002). The word association method: A gateway to work-task based retrieval. Åbo Akademi University : Åbo Akademi University Press. 2002. {Doctoral thesis, Åbo/Turku}.
56. Otterbacher, J., Radev, D. & Kareem, O. (2008). Hierarchical summarization for delivering information to mobile devices. Information Processing & Management, 44(2), 931--947.
57. Pansanato, L.T.E. & Fortes, R.P.M. (2007). System description: An orienteering strategy to browse semantically-enhanced educational wiki pages. In: Franconi, E., Kifer, M., May, W. et al. (eds.). Proceedings of the 4th European Semantic Web Conference, Innsbruck, Austria, June 3-7 2007. Berlin/Heidelberg : Springer Verlag, 809--818.
58. Papaeconomou, C., Zijlema, A. & Ingwersen, P. (2008). Searchers' relevance judgments and criteria in evaluating web pages in a learning style perspective. In: Borlund, P., Schneider, J.W., Tombros, A., Lalmas, M., Feather, J., Kelly, D., de Vries, A. & Azzopardi, L. (eds.). Proceeding of the second IIiX Symposium on Information Interaction in Context (IIiX 2008). London, UK, October 2008. ACM Press, 123--132.
59. Petrelli, D. (2008). On the role of user-centred evaluation in the advancement of interactive information retrieval. Information Processing & Management, 44(1), 22--38.
60. Petrelli, D. & Auld, D. (2008). An examination of automatic video retrieval technology on access to the contents of an historical video archive. Program Electronic Library and Information Systems, 42(2), 115--136.
61. Petrelli, D., Beaulieu, M., Sanderson, M., Demetriou, G., Herring, P. & Hansen, P. (2004). Observing users, designing clarity: A case study on the user-centered design of a cross-language information retrieval system. Journal of the American Society for Information Science and Technology, 55(10), 923--934.
62. Petrelli, D., Demetriou, G., Herring, P., Beaulieu, M. & Sanderson, M. (2002). Exploring the effect of query translation when searching cross-language. In: Peters, C., Braschler,M., GonzaloJ. & Kluck, M. (eds.). Proceedings of the 3rd Workshop of the Cross-Language Evaluation Forum (CLEF 2002). Rome, Italy, September 19-20 2002. Berlin/Heidelberg : Springer Verlag, 430--445.
63. Petrelli, D., Levin, S., Beaulieu, M. & Sanderson, M. (2006). Which user interaction for cross-language information retrieval? Design issues and reflections. Journal of the American Society for Information Science and Technology, 57(5), 709--722.
64. Pharo, N. (2008). The effect of granularity and order in XML element retrieval. Information Processing & Management, 44(5), 1732--1740.
65. Pharo, N. & Nordlie, R. (2005, Jun 04-08). Context matters: An analysis of assessments of XML documents. In: Crestani, F. & Ruthven, I. (eds.). Proceedings of the 5th International Conference on Conceptions of Library and Information Sciences (CoLIS 2005). Glasgow, UK, June 4-8 2005. Berlin/Heidelberg : Springer Verlag, 238--248.
66. Price, S.L., Nielsen, M.L., Delcambre, L.M.L. & Vedsted, P. (2007). Semantic components enhance retrieval of domain-specific documents. In: Silva, M.J., Laender, A.H.F., Baeza-Yates, R.A., McGuinness, D.L., Olstad,B., Olsen, Æ.H. & Falcão, A.O. (eds.). Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management (CIKM 2007). Lisbon, Portugal, November 6-10 2007. ACM, 429--438.
67. Qu, Y. & Furnas, G.W. (2008). Model-driven formative evaluation of exploratory search: A study under a sensemaking framework. Information Processing & Management, 44(2), 534--555.
68. Rodden, K., Basalaj, W., Sinclair, D. & Wood, K. (2001). Does organisation by similarity assist image browsing? In: Proceedings of the SIGCHI on human factors in computing systems. March 31-April 5 2001, Seattle, WA, USA. ACM, 3(1), 190--197.
69. Ruthven, I. (2001). Abduction, explanation and relevance feedback. Glasgow University. 2001. {Doctoral thesis}.
70. Ruthven, I., Lalmas, M. & van Rijsbergen, K. (2002, Jul 21-25). Ranking expansion terms with partial and ostensive evidence. In: Bruce, H., Fidel, R., Ingwersen, P. & Vakkari, P. (eds.). The proceedings of the 4th International Conference on Conceptions of Library and Information Science (CoLIS 2002). Seattle, Wa, July 21-25 2002. Colerado: Libraries Unlimited, 151--165.
71. Ruthven, I., Lalmas, M. & van Rijsbergen, K. (2003). Incorporating user search behavior into relevance feedback. Journal of the American Society for Information Science and Technology, 54(6), 529--549.
72. Sagara, Y. (1999). The implications of the non-keyword information on the user's judgment of relevance in information retrieval. {Japanese Journal of} Library and Information Science, 42, 23--46.
73. Sihvonen, A. & Vakkari, P. (2004). Subject knowledge improves interactive query expansion assisted by a thesaurus. Journal of Documentation, 60(6), 673--690.
74. Skov, M. & Ingwersen, P. (2008). Exploring information seeking behavior in a digital museum context. In: Borlund, P., Schneider, J.W., Tombros, A., Lalmas, M., Feather, J., Kelly, D., de Vries, A. & Azzopardi, L. (eds.). Proceeding of the second IIiX Symposium on Information Interaction in Context (IIiX 2008). London, UK, October 2008. ACM Press, 110--115.
75. Skov, M. (2009). The reinvented museum: Exploring information seeking behaviour in a digital museum context. Royal School of Library and Information Science, Copenhagen. 2009. {Doctoral Thesis}.
76. Skov, M., Larsen, B. & Ingwersen, P. (2008). Inter and intra-document contexts applied in polyrepresentation for best match IR. Information Processing & Management, 44(5), 1673--1683.
77. Spink, A. & Dee, C. (2007). Cognitive shifts related to interactive information retrieval. Online Information Review, 31(6), 845--860.
78. Suomela, S. & Kekäläinen, J. (2005). Ontology as a search-tool: A study of real users' query formulation with and without conceptual support. In: Losada, D.E. & Fernández--Luna, J.M. (eds.). Proceedings of the 27th European Conference on IR Research (ECIR 2005). Santiago de Compostela, Spain, March 21-23 2005. Berlin/Heidelberg : Springer Verlag, 315--329.
79. Suomela, S. & Kekalainen, J. (2005). User evaluation of ontology as query construction tool. Information Retrieval, 9, 455--475.
80. Sutcliffe, A.G., Ennis, M. & Watkinson, S.J. (2000). Empirical studies of end-user information searching. Journal of the American Society for Information Science, 51(13), 1211--1231.
81. Szlavik, Z., Tombros, A. & Lalmas, M. (2006) Investigating the use of summarization for interactive XML retrieval. In: Proceedings of the 21st Annual ACM Symposium on Applied Computing (SAC 2006). Dijon, France, April 23-27 2006. ACM, 1068--1072.
82. Szlavik, Z., Tombros, A. & Lalmas, M. (2006). The use of summaries in XML retrieval. In: Gonzalo, J. et al (eds). Proceedings of 10th European Conference on Research and Advanced Technology for Digital Libraries (ECDL 2006). Alicante, Spain, September 17-22 2006. Berlin/Heidelberg : Springer Verlag, 75--86.
83. Szlavik, Z., Tombros, A., & Lalmas, M. (2007). Feature- and query-based table of contents generation for XML documents. In: Amati, G., Carpineto, C. & Romano, G. (eds.). Proceedings the 29th European Conference on IR Research (ECIR 2007). Rome, Italy, April 2-5 2007. Berlin/Heidelberg : Springer Verlag, 456--467.
84. Tombros, A., Larsen, B. & Malik, S. (2004). The interactive track at INEX 2004. In: Fuhr, N., Lalmas, M., Malik, S. & Szlávik, Z. (eds.). Proceedings of the 3rd International Workshop of the Initiative for the Evaluation of XML Retrieval (INEX 2004). Dagstuhl Castle, Germany, December 6-8 2004. Berlin/Heidelberg : Springer Verlag, 410--423.
85. Tombros, A., Malik, S. & Larsen, B. (2004). Report on the INEX 2004 interactive track. ACM SIGIR Forum, 39(1), 43--49.
86. Tombros, A., Ruthven, I. & Jose, J.M. (2003). Searchers' criteria for assessing web pages. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003). July 28-August 1 2003, Toronto, Canada. ACM, 385--386.
87. Tombros, A., Ruthven, I. & Jose, J.M. (2005). How users assess web pages for information seeking. Journal of the American Society for Information Science and Technology, 56(4), 327--344.
88. Toms, E.G., O'Brien, H., Mackenzie, T., Jordan, C., Freund, L., Toze, S., Dawe, E. & MacNutt, A. (2007). Task effects on interactive search: The query factor. In: Fuhr, N., Kamps, J., Lalmas, M. & Trotman, A. (eds.). Proceedings of the 6th International Workshop of the Initiative for the Evaluation of XML Retrieval (INEX), Dagstuhl, Germany. Berlin/Heidelberg : Springer Verlag, 359--372.
89. Villa, R., Gildea, N. & Jose, J.M. (2008). Facetbrowser: A user interface for complex search tasks. In: El-Saddik, A., Vuong, S., Griwodz, C., Del Bimbo, A., Candan, K.S. & Jaimes, A. (eds.). Proceedings of the 16th International Conference on Multimedia 2008 (MM '08). Vancouver, British Columbia, Canada, October 26-31 2008. ACM, 489--498.
90. Wacholder, N., Kelly, D., Kantor, P., Rittman, R., Sun, Y., Bai, B., Small, S., Yamrom, B. & Strzalkowski, T. (2007). A model for quantitative evaluation of an end-to-end question-answering system. Journal of the American Society for Information Science and Technology, 58(8), 1082--1099.
91. Wang, Y.D. & Forgionne, G. (2008). Testing a decision-theoretic approach to the evaluation of information retrieval systems. Journal of Information Science, 34(6), 861--876.
92. Wen, L., Ruthven, I. & Borlund, P. (2006). The effects on topic familiarity on online search behavior and use of relevance criteria. In: Lalmas, M., MacFarlane, A., Rüger, S., Tombros, A., Tsikrika, T. & Yavlinsky, A. (eds.). Proceedings of the 28th European Conference on IR Research (ECIR 2006). London, UK, April 2006. Berlin/Heidelberg: Springer Verlag, 456--459.
93. White, R.W. & Marchionini, G. (2006). A study of real-time query expansion effectiveness. In: Efthimiadis, E.N., Dumais, S.T., Hawking, D. & Järvelin, K. (eds.). Proceedings of the 29th ACM Sigir Conference on Research and Development of Information Retrieval (SIGIR 2006). Seattle, Washington, USA, August 6-11, 2006. ACM Press, 715--716.
94. White, R.W & Ruthven, I. (2006). A study of interface support mechanisms for interactive information retrieval. Journal of the American Society for Information Science and Technology, 57(7), 933--948.
95. White, R.W., Jose, J.M. & Ruthven, I. (2001). Query-based web page summarization: A task-oriented evaluation. In: Croft, W.B., Harper, D.J., Kraft, D.H. & Zobel, J. (eds.). Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001). September 9-13 2001, New Orleans, Louisiana, USA. ACM, 412--413.
96. White, R.W., Jose, J.M. & Ruthven, I. (2003). A task-oriented study on the influencing effects of query-biased summarisation in web searching. Information Processing & Management, 39(5), 707--733.
97. White, R.W., Jose, J.M. & Ruthven, I. (2005). Using top-ranking sentences to facilitate effective information access. Journal of the American Society for Information Science and Technology, 56(10), 1113--1125.
98. White, R.W., Jose, J.M. & Ruthven, I. (2006). An implicit feedback approach for interactive information retrieval. Information Processing & Management, 42(1), 166--190.
99. White, R., Bilenko, M. & Cucerzan, S. (2007). Studying the use of popular destinations to enhance web search interaction. In: Kraaij, W., de Vries, A.P., Clarke, C.L.A., Fuhr, N. & Kando, N. (eds.). Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007). Amsterdam, the Netherlands, July 23-27 2007. ACM, 159--166.
100. White, R., Bilenko, M. & Cucerzan, S. (2008). Leveraging popular destinations to enhance web search interaction. ACM Transactions on the Web, 2(3), 30 Pp.
101. White, R., Ruthven, I. & Jose, J.M. (2001). Web document summarisation: a task-oriented evaluation. In: Tjoa, A.T. & Wagner, R. (eds.). Proceedings of the 12th International Conference on Database and Expert Systems Applications (DEXA 2001). Munich, Germany, September 3-7 2001. Berlin/Heidelberg: Springer Verlag, 551--555.
102. White, R.W., Ruthven, I. & Jose, J.M. (2002). Finding relevant documents using top-ranking sentences: An evaluation of two alternative schemes. In: Beaulieu, M., Baeza-Yates, R., Myaeng, S.H. & Järvelin, K. (eds.). Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002). August 11-15 2002, Tampere, Finland. ACM, 57--64.
103. White, R.W., Ruthven, I. & Jose, J.M. (2005). A study of factors affecting the utility of implicit relevance feedback. In: Baeza-Yates, R.A., Ziviani, N., Marchionini, G., Moffat, A. & Tait, J. (eds.). Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2005). Salvador, Brazil, August 15--19 2005. ACM, 35--42.
104. Wu, M., Muresan, G. McLean, A., Tang, M., Wilkinson, R., Li. Y., Lee, H. & Belkin, N.J. (2004). Human versus machine in the topic distillation task. In: Sanderson, M., Järvelin, K., Allan, J. & Bruza, P. (eds.). Proceedings of the 27th Conference on Research and Development in Information Retrieval (SIGIR 2004). Sheffield, UK, July 25-29, 2004. ACM, 385--392.
105. Xie, I. (2008). Interactive information retrieval in digital environments. Hershey, PA : IGI Pub. ISBN: 978-1-59904-240-4.
106. Yuan, X. & Belkin, N.J. (2007). Supporting Multiple Information--seeking strategies in a single system framework. In: Kraaij, W., de Vries, A.P., Clarke, C.L.A., Fuhr, N. & Kando, N. (eds.). Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007). July 23-27 2007, Amsterdam, the Netherlands. ACM, 247--254.
107. Yuan, X. (2006). Supporting Multiple Information-seeking strategies in a single system framework. In: Moore, R.C., Bilmes, J.A., Chu-Carroll, J. & Sanderson, M. (eds.). Proceedings of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics. June 4-9 2006, New York, USA. The Association for Computational Linguistics, 147--249.

Index Terms

• Information systems

  • Information retrieval