Playing Your Cards Right: The Effect of Entity Cards on Search Behaviour and Workload

ABSTRACT

In addition to merging results of different types (e.g.~images, videos, news items) into a ranked list of Web documents, modern search engines have also started displaying entity cards (ECs) on the results page. Entity cards are intended to enhance search experience in several ways: (i) they help searchers navigate diversified results, (ii) provide a summary of relevant content directly on the results page and (iii) support exploratory search by highlighting relevant entities associated with a given user query. We conducted a large-scale crowd-sourced user study, with more than $700$ unique searchers, to investigate the effects of entity cards on search behaviour and perceived workload. We find that the presence of ECs has a strong effect on both the way users interact with search results and their perceived task workload. Furthermore, by manipulating EC properties content, coherence and vertical diversity), we uncover different effects and interactions between card properties on measures of search behaviour and workload. Our study contributes an in-depth analysis of the effects of entity cards on user interaction with modern Web search interfaces.

References

1. J. Arguello and R. Capra. The effect of aggregated search coherence on search behavior. ACM CIKM '12, pages 1293--1302.
2. J. Arguello and R. Capra. The effects of vertical rank and border on aggregated search coherence and search behavior. ACM CIKM '14, pages 539--548, 2014.
3. J. Arguello, R. Capra, and W.-C. Wu. Factors affecting aggregated search coherence and search behavior. ACM CIKM '13, pages 1989--1998.
4. J. Arguello, W.-C. Wu, D. Kelly, and A. Edwards. Task complexity, vertical display and user interaction in aggregated search. ACM SIGIR '12, pages 435--444.
5. A. Ashkan, C. Clarke, E. Agichtein, and Q. Guo. Classifying and characterizing query intent. In M. Boughanem, C. Berrut, J. Mothe, and C. Soule-Dupuy, editors, Advances in Information Retrieval, volume 5478 of Lecture Notes in Computer Science, pages 578--586. Springer Berlin Heidelberg, 2009.
6. R. Blanco, B. B. Cambazoglu, P. Mika, and N. Torzec. Entity recommendations in web search. In The Semantic Web--ISWC 2013, pages 33--48. Springer, 2013.
7. D. J. Brenes, D. Gayo-Avello, and K. Pérez-González. Survey and evaluation of query intent detection methods. In Proceedings of the 2009 Workshop on Web Search Click Data, WSCD '09, pages 1--7, New York, NY, USA, 2009. ACM.
8. K. Brennan, D. Kelly, and J. Arguello. The effect of cognitive abilities on information search for tasks of varying levels of complexity. ACM IIiX '14, pages 165--174.
9. R. Capra, J. Arguello, and F. Scholer. Augmenting web search surrogates with images. ACM CIKM '13, pages 399--408, 2013.
10. F. Diaz, R. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. ACM CIKM '13, pages 1451--1460.
11. S. G. Hart and L. E. Staveland. Development of nasa-tlx (task load index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati, editors, Human Mental Workload, volume 52 of Advances in Psychology, pages 139--183. North-Holland, 1988.
12. S. Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2):pp. 65--70, 1979.
13. C. J. Huberty and J. D. Morris. Multivariate analysis versus multiple univariate analyses. Psychological Bulletin, 105(2):302--308, 1989.
14. D. Lagun, C.-H. Hsieh, D. Webster, and V. Navalpakkam. Towards better measurement of attention and satisfaction in mobile search. In ACM SIGIR '14, pages 113--122.
15. Z. Liu, Y. Liu, K. Zhou, M. Zhang, and S. Ma. Influence of vertical result in web search examination. SIGIR '15, pages 193--202, New York, NY, USA, 2015. ACM.
16. T. Megaw. The definition and measurement of mental workload. In J. R. and E. N. Corlett, editors, Evaluation of Human Work. CRC Press, 2005.
17. M. Melucci and R. Baeza-Yates. Advanced topics in information retrieval. pages 109--123. Springer Publishing Company, 2011.
18. J. Meyers-Levy and B. Sternthal. A two-factor explanation of assimilation and contrast effects. Journal of Marketing Research, 30(3):pp. 359--368, 1993.
19. A. C. Morales and G. J. Fitzsimons. Product contagion: Changing consumer evaluations through physical contact with "disgusting" products. Journal of Marketing Research, 44(2):272--283, 2007.
20. V. Navalpakkam, L. Jentzsch, R. Sayres, S. Ravi, A. Ahmed, and A. Smola. Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts. In ACM WWW '13, pages 953--964.
21. M. Sanderson. Ambiguous queries: Test collections need more sense. ACM SIGIR '08, pages 499--506.
22. C. Speier and M. G. Morris. The influence of query interface design on decision-making performance. MIS Q., 27(3):397--423, Sept. 2003.
23. L. L. D. Stasi, A. Antoli, M. Gea, and J. J. Canas. A neuroergonomic approach to evaluating mental workload in hypermedia interactions. International Journal of Industrial Ergonomics, 41(3):298 -- 304, 2011.
24. S. Sushmita, H. Joho, M. Lalmas, and R. Villa. Factors affecting click-through behavior in aggregated search interfaces. In ACM CIKM '10, pages 519--528. ACM, 2010.
25. C. Wang, Y. Liu, M. Zhang, S. Ma, M. Zheng, J. Qian, and K. Zhang. Incorporating vertical results into search click models. ACM SIGIR '13, pages 503--512.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Empirical studies in HCI

    • Interaction paradigms

      • Web-based interaction

• Information systems

  • Information retrieval

    • Users and interactive retrieval

      • Search interfaces