Abstract
A recent series of experiments demonstrated that introducing ranking bias to election-related search engine results can have a strong and undetectable influence on the preferences of undecided voters. This phenomenon, called the Search Engine Manipulation Effect (SEME), exerts influence largely through order effects that are enhanced in a digital context. We present data from three new experiments involving 3,600 subjects in 39 countries in which we replicate SEME and test design interventions for suppressing the effect. In the replication, voting preferences shifted by 39.0%, a number almost identical to the shift found in a previously published experiment (37.1%). Alerting users to the ranking bias reduced the shift to 22.1%, and more detailed alerts reduced it to 13.8%. Users' browsing behaviors were also significantly altered by the alerts, with more clicks and time going to lower-ranked search results. Although bias alerts were effective in suppressing SEME, we found that SEME could be completely eliminated only by alternating search results -- in effect, with an equal-time rule. We propose a browser extension capable of deploying bias alerts in real-time and speculate that SEME might be impacting a wide range of decision-making, not just voting, in which case search engines might need to be strictly regulated.
- Amr Ahmed, Mohamed Aly, Abhimanyu Das, Alexander J Smola, and Tasos Anastasakos. 2012. Web-scale multi-task feature selection for behavioral targeting Proceedings of the 21st ACM International Conference on Information and Knowledge Management. ACM, 1737--1741. Google ScholarDigital Library
- Ahmed Allam, Peter Johannes Schulz, and Kent Nakamoto. 2014. The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes: two experiments manipulating Google output. Journal of Medical Internet Research Vol. 16, 4 (2014), e100.Google ScholarCross Ref
- Sinan Aral and Dylan Walker. 2012. Identifying influential and susceptible members of social networks. Science, Vol. 337, 6092 (2012), 337--341.Google ScholarCross Ref
- Eytan Bakshy, Solomon Messing, and Lada A Adamic. 2015. Exposure to ideologically diverse news and opinion on Facebook. Science, Vol. 348, 6239 (2015), 1130--1132.Google Scholar
- John A Bargh, Peter M Gollwitzer, Annette Lee-Chai, Kimberly Barndollar, and Roman Trötschel. 2001. The automated will: nonconscious activation and pursuit of behavioral goals. Journal of Personality and Social Psychology, Vol. 81, 6 (2001), 1014.Google ScholarCross Ref
- James R Beniger. 1987. Personalization of mass media and the growth of pseudo-community. Communication research Vol. 14, 3 (1987), 352--371.Google Scholar
- George R Bergus, Irwin P Levin, and Arthur S Elstein. 2002. Presenting risks and benefits to patients. Journal of General Internal Medicine Vol. 17, 8 (2002), 612--617.Google ScholarCross Ref
- Adam J Berinsky, Gregory A Huber, and Gabriel S Lenz. 2012. Evaluating online labor markets for experimental research: Amazon. com's Mechanical Turk. Political Analysis, Vol. 20, 3 (2012), 351--368.Google ScholarCross Ref
- Shlomo Berkovsky, Jill Freyne, and Harri Oinas-Kukkonen. 2012. Influencing individually: fusing personalization and persuasion. ACM Transactions on Interactive Intelligent Systems (TiiS), Vol. 2, 2 (2012), 9. Google ScholarDigital Library
- Edelman Berland. 2017. 2017 Edelman Trust Barometer. http://www.edelman.com/trust2017/. (2017). Accessed: 2017-03-07.Google Scholar
- Robert M Bond, Christopher J Fariss, Jason J Jones, Adam DI Kramer, Cameron Marlow, Jaime E Settle, and James H Fowler. 2012. A 61-million-person experiment in social influence and political mobilization. Nature, Vol. 489, 7415 (2012), 295--298.Google Scholar
- Jack W Brehm. 1966. A theory of psychological reactance.Academic Press.Google Scholar
- Michael Buhrmester, Tracy Kwang, and Samuel D Gosling. 2011. Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science Vol. 6, 1 (2011), 3--5.Google ScholarCross Ref
- Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, Vol. 356, 6334 (2017), 183--186.Google Scholar
- David Chavalarias. 2016. The unlikely encounter between von Foerster and Snowden: When second-order cybernetics sheds light on societal impacts of Big Data. Big Data & Society, Vol. 3, 1 (2016), 1--11.Google ScholarCross Ref
- Eric Chen, Gábor Simonovits, Jon A Krosnick, and Josh Pasek. 2014. The impact of candidate name order on election outcomes in North Dakota. Electoral Studies Vol. 35 (2014), 115--122.Google ScholarCross Ref
- Le Chen, Alan Mislove, and Christo Wilson. 2016. An empirical analysis of algorithmic pricing on Amazon marketplace Proceedings of the 25th International World Wide Web Conference. Google ScholarDigital Library
- Ye Chen, Dmitry Pavlov, and John F Canny. 2009. Large-scale behavioral targeting. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 209--218. Google ScholarDigital Library
- Sidharth Chhabra and Paul Resnick. 2012. Cubethat: news article recommender. In Proceedings of the Sixth ACM Conference on Recommender Systems. ACM, 295--296. Google ScholarDigital Library
- Sidharth Chhabra and Paul Resnick. 2013. Does clustered presentation lead readers to diverse selections? CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM, 1689--1694. Google ScholarDigital Library
- Chun-Fang Chiang and Brian Knight. 2011. Media Bias and Influence: Evidence from Newspaper Endorsements. The Review of Economic Studies Vol. 78, 3 (2011), 795--820.Google ScholarCross Ref
- Inc comScore. 2017. comScore Explicit Core Search Query Report (Desktop Only). https://www.comscore.com/Insights/Rankings. (2017). Accessed: 2017-02--12.Google Scholar
- Stefano DellaVigna and Ethan Kaplan. 2007. The Fox News effect: Media bias and voting. The Quarterly Journal of Economics Vol. 122, 3 (2007), 1187--1234.Google ScholarCross Ref
- Nicholas Diakopoulos. 2014. Algorithmic accountability reporting: On the investigation of black boxes. Tow Center for Digital Journalism, Columbia University (2014).Google Scholar
- Nicholas Diakopoulos. 2016. Accountability in algorithmic decision making. Commun. ACM Vol. 59, 2 (2016), 56--62. Google ScholarDigital Library
- Dimitar Dimitrov, Philipp Singer, Florian Lemmerich, and Markus Strohmaier. 2016. Visual positions of links and clicks on wikipedia. Proceedings of the 25th International Conference Companion on World Wide Web. International World Wide Web Conferences Steering Committee, 27--28. Google ScholarDigital Library
- Luke Dormehl. 2014. The formula: How algorithms solve all our problems--and create more. Penguin, New York, NY.Google Scholar
- James N Druckman and Michael Parkin. 2005. The impact of media bias: How editorial slant affects voters. Journal of Politics, Vol. 67, 4 (2005), 1030--1049.Google ScholarCross Ref
- Hermann Ebbinghaus. 1913. Memory: A contribution to experimental psychology. Number 3. University Microfilms.Google ScholarCross Ref
- Robert Epstein and Ronald E Robertson. 2015. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences, Vol. 112, 33 (2015), E4512--E4521.Google ScholarCross Ref
- Robert Epstein and Ronald E Robertson. 2017. A method for detecting bias in search rankings, with evidence of systematic bias related to the 2016 presidential election. Technical Report White Paper no. WP-17-02. American Institute for Behavioral Research and Technology, Vista, CA.Google Scholar
- Eyal Ert and Aliza Fleischer. 2016. Mere Position Effect in Booking Hotels Online. Journal of Travel Research Vol. 55, 3 (2016), 311--321.Google ScholarCross Ref
- Motahhare Eslami, Amirhossein Aleyasen, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015 a. Feedvis: A path for exploring news feed curation algorithms Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing. Google ScholarDigital Library
- Motahhare Eslami and Karrie Karahalios. 2017. Understanding and Designing around Users' Interaction with Hidden Algorithms in Sociotechnical Systems. In CSCW Companion. 57--60. Google ScholarDigital Library
- Motahhare Eslami, Karrie Karahalios, Christian Sandvig, Kristen Vaccaro, Aimee Rickman, Kevin Hamilton, and Alex Kirlik. 2016. First I like it, then I hide it: Folk theories of social feeds Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems. ACM, 2371--2382. Google ScholarDigital Library
- Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015 b. I always assumed that I wasn't really that close to {her}: Reasoning about Invisible Algorithms in News Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 153--162. Google ScholarDigital Library
- Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, and Kevin Hamilton. 2017. "Be careful; things can be worse than they appear": Understanding biased algorithms and users' behavior around them in rating platforms. ICWSM. 62--71.Google Scholar
- Facebook. 2017. News Feed FYI: Addressing Hoaxes and Fake News. https://newsroom.fb.com/news/2016/12/news-feed-fyi-addressing-hoaxes-and-fake-news/. (2017). Accessed: 2017-07--13.Google Scholar
- Baruch Fischoff. 1982. Debiasing. Cambridge University Press, Cambridge, MA.Google Scholar
- B.J. Fogg. 2002. Persuasive technology: Using computers to change what we think and do. Morgan Kaufmann, San Francisco, CA. Google ScholarDigital Library
- Marieke L Fransen, Bob M Fennis, Ad Th H Pruyn, and Enny Das. 2008. Rest in peace? Brand-induced mortality salience and consumer behavior. Journal of Business Research Vol. 61, 10 (2008), 1053--1061.Google ScholarCross Ref
- Jonathan L Freedman and David O Sears. 1965 a. Selective exposure. Advances in Experimental Social Psychology Vol. 2 (1965), 57--97.Google Scholar
- Jonathan L Freedman and David O Sears. 1965 b. Warning, distraction, and resistance to influence. Journal of Personality and Social Psychology, Vol. 1, 3 (1965), 262--266.Google ScholarCross Ref
- Alan S Gerber, Dean Karlan, and Daniel Bergan. 2009. Does the media matter? A field experiment measuring the effect of newspapers on voting behavior and political opinions. American Economic Journal: Applied Economics, Vol. 1, 2 (2009), 35--52.Google ScholarCross Ref
- Tarleton Gillespie. 2014. The relevance of algorithms. Vol. Vol. 167. MIT Press Cambridge, Cambridge, MA.Google Scholar
- David F Gleich, Paul G Constantine, Abraham D Flaxman, and Asela Gunawardana. 2010. Tracking the random surfer: empirically measured teleportation parameters in PageRank Proceedings of the 19th International Conference on World Wide Web. ACM, 381--390. Google ScholarDigital Library
- Google. 2017. Fact Check now available in Google Search and News around the world. https://www.blog.google/products/search/fact-check-now-available-google-search-and-news-around-world/. (2017). Accessed: 2017-07--13.Google Scholar
- Laura A Granka, Thorsten Joachims, and Geri Gay. 2004. Eye-tracking analysis of user behavior in WWW search Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 478--479. Google ScholarDigital Library
- Edith Greene, Marlene S Flynn, and Elizabeth F Loftus. 1982. Inducing resistance to misleading information. Journal of Verbal Learning and Verbal Behavior, Vol. 21, 2 (1982), 207--219.Google ScholarCross Ref
- Zhiwei Guan and Edward Cutrell. 2007. An eye tracking study of the effect of target rank on web search Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 417--420. Google ScholarDigital Library
- Xunhua Guo, Mingyue Zhang, Chenyue Yang, et almbox. 2016. Order Effects in Online Product Recommendation: A Scenario-based Analysis Proceedings of the 22nd Americas Conference on Information Systems.Google Scholar
- Kevin Hamilton, Karrie Karahalios, Christian Sandvig, and Motahhare Eslami. 2014. A path to understanding the effects of algorithm awareness CHI'14 Extended Abstracts on Human Factors in Computing Systems. ACM, 631--642. Google ScholarDigital Library
- Aniko Hannak, Piotr Sapie.zy'nski, Arash Molavi Kakhki, Balachander Krishnamurthy, David Lazer, Alan Mislove, and Christo Wilson. 2013. Measuring Personalization of Web Search. In Proceedings of the 22nd International World Wide Web Conference. Google ScholarDigital Library
- Aniko Hannak, Gary Soeller, David Lazer, Alan Mislove, and Christo Wilson. 2014. Measuring price discrimination and steering on e-commerce Web sites Proceedings of the 2014 ACM Conference on Internet Measurement. Google ScholarDigital Library
- Aniko Hannak, Claudia Wagner, David Garcia, Alan Mislove, Markus Strohmaier, and Christo Wilson. 2017. Bias in online freelance marketplaces: Evidence from TaskRabbit and Fiverr 20th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2017). Portland, OR. Google ScholarDigital Library
- Daniel E Ho and Kosuke Imai. 2008. Estimating causal effects of ballot order from a randomized natural experiment the California alphabet lottery, 1978--2002. Public Opinion Quarterly Vol. 72, 2 (2008), 216--240.Google ScholarCross Ref
- Statistics Brain Research Institute. 2017. Facebook Company Statistics. http://www.statisticbrain.com/facebook-statistics/. (2017). Accessed: 2017-02--12.Google Scholar
- Tiffany A Ito, Jeff T Larsen, N Kyle Smith, and John T Cacioppo. 1998. Negative information weighs more heavily on the brain: the negativity bias in evaluative categorizations. Journal of Personality and Social Psychology, Vol. 75, 4 (1998), 887.Google ScholarCross Ref
- Mohit Iyyer, Peter Enns, Jordan Boyd-Graber, and Philip Resnik. 2014. Political Ideology Detection Using Recursive Neural Networks Association for Computational Linguistics.Google Scholar
- Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Transactions on Information Systems (TOIS), Vol. 25, 2 (2007), 7. Google ScholarDigital Library
- Yvonne Kammerer and Peter Gerjets. 2014. The role of search result position and source trustworthiness in the selection of web search results when using a list or a grid interface. International Journal of Human-Computer Interaction, Vol. 30, 3 (2014), 177--191.Google ScholarCross Ref
- José H Kerstholt and Janet L Jackson. 1998. Judicial decision making: Order of evidence presentation and availability of background information. Applied Cognitive Psychology Vol. 12, 5 (1998), 445--454.Google ScholarCross Ref
- Nuri Kim, Jon Krosnick, and Daniel Casasanto. 2015. Moderators of candidate name-order effects in elections: An experiment. Political Psychology, Vol. 36, 5 (2015), 525--542.Google ScholarCross Ref
- Chloe Kliman-Silver, Aniko Hannak, David Lazer, Christo Wilson, and Alan Mislove. 2015. Location, location, location: The impact of geolocation on Web search personalization Proceedings of the 2015 ACM Conference on Internet Measurement. Google ScholarDigital Library
- Eric S Knowles and Jay A Linn. 2004. Resistance and persuasion. Psychology Press.Google Scholar
- Ron Kohavi, Alex Deng, Brian Frasca, Toby Walker, Ya Xu, and Nils Pohlmann. 2013. Online controlled experiments at large scale. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1168--1176. Google ScholarDigital Library
- Alona Kolomiiets, Nathalie Dens, and Patrick De Pelsmacker. 2016. The wrap effect in online review sets revisited: How perceived usefulness mediates the effect on intention formation. Journal of Electronic Commerce Research Vol. 17, 4 (2016), 280--288.Google Scholar
- Jonathan GS Koppell and Jennifer A Steen. 2004. The effects of ballot position on election outcomes. Journal of Politics, Vol. 66, 1 (2004), 267--281.Google ScholarCross Ref
- Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, Vol. 111, 24 (2014), 8788--8790.Google ScholarCross Ref
- Jon A Krosnick, Joanne M Miller, and Michael P Tichy. 2004. An unrecognized need for ballot reform: Effects of candidate name order. Rethinking the vote: The politics and prospects of american election reform, bibfieldeditorA.N Crigler, M.R. Just, and E.J. McCaffery (Eds.). Oxford University Press, New York, NY, 51--74.Google Scholar
- Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P Gummadi, and Karrie Karahalios. 2017. Quantifying search bias: Investigating sources of bias for political searches in social media 20th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2017). Google ScholarDigital Library
- Howard Lavine, Joseph W Huff, Stephen H Wagner, and Donna Sweeney. 1998. The moderating influence of attitude strength on the susceptibility to context effects in attitude surveys. Journal of Personality and Social Psychology, Vol. 75, 2 (1998), 359--373.Google ScholarCross Ref
- Nicole R LaVoie, Brian L Quick, Julius M Riles, and Natalie J Lambert. 2015. Are graphic cigarette warning labels an effective message strategy? A test of psychological reactance theory and source appraisal. Communication Research (2015), 416--436.Google Scholar
- David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-László Barabási, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Tony Jebara, Gary King, Michael Macy, Deb Roy, and Marshall Van Alstyne. 2009. Computational Social Science. Science, Vol. 323, 5915 (2009), 721--723.Google Scholar
- Bing Liu, Minqing Hu, and Junsheng Cheng. 2005. Opinion observer: analyzing and comparing opinions on the web Proceedings of the 14th international conference on World Wide Web. ACM, 342--351. Google ScholarDigital Library
- Elizabeth F Loftus and Guido Zanni. 1975. Eyewitness testimony: The influence of the wording of a question. Bulletin of the Psychonomic Society Vol. 5, 1 (1975), 86--88.Google ScholarCross Ref
- Lori Lorigo, Maya Haridasan, Hrönn Brynjarsdóttir, Ling Xia, Thorsten Joachims, Geri Gay, Laura Granka, Fabio Pellacini, and Bing Pan. 2008. Eye tracking and online search: Lessons learned and challenges ahead. Journal of the Association for Information Science and Technology, Vol. 59, 7 (2008), 1041--1052. Google ScholarDigital Library
- Ramona Ludolph, Ahmed Allam, and Peter J Schulz. 2016. Manipulating Google's Knowledge Graph box to counter biased information processing during an online search on vaccination: application of a technological debiasing strategy. Journal of Medical Internet research Vol. 18, 6 (2016).Google ScholarCross Ref
- William J McGuire. 1964. Some contemporary approaches. Advances in Experimental Social Psychology Vol. 1 (1964), 191--229.Google ScholarCross Ref
- Jakub Mikians, László Gyarmati, Vijay Erramilli, and Nikolaos Laoutaris. 2012. Detecting price and search discrimination on the Internet Proceedings of the 11th ACM Workshop on Hot Topics in Networks. Google ScholarDigital Library
- Jakub Mikians, László Gyarmati, Vijay Erramilli, and Nikolaos Laoutaris. 2013. Crowd-assisted search for price discrimination in e-commerce: First results Proceedings of the Ninth ACM Conference on Emerging Networking Experiments and Technologies. Google ScholarDigital Library
- Katherine L Milkman, Dolly Chugh, and Max H Bazerman. 2009. How can decision making be improved? Perspectives on psychological science Vol. 4, 4 (2009), 379--383.Google Scholar
- Claude H Miller. 2015. Persuasion and psychological reactance: The effects of explicit, high controlling language. Springer.Google Scholar
- Ronald B Mincy. 1993. The Urban Institute audit studies: their research and policy context. Clear and Convincing Evidence: Measurement of Discrimination in America (1993), 165--86.Google Scholar
- Amy Mitchell, Jeffrey Gottfried, and Katerina Eva Matsa. 2015. Millennials and political news. Technical Report. Pew Research Center.Google Scholar
- Sean A Munson, Stephanie Y Lee, and Paul Resnick. 2013. Encouraging Reading of Diverse Political Viewpoints with a Browser Widget ICWSM.Google Scholar
- Sean A Munson and Paul Resnick. 2010. Presenting diverse political opinions: how and how much Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 1457--1466. Google ScholarDigital Library
- Bennet B Murdock. 1962. The serial position effect of free recall. Journal of Experimental Psychology Vol. 64, 5 (1962), 482--488.Google ScholarCross Ref
- Jamie Murphy, Charles Hofacker, and Richard Mizerski. 2006. Primacy and recency effects on clicking behavior. Journal of Computer-Mediated Communication Vol. 11, 2 (2006), 522--535.Google ScholarCross Ref
- Helen Nissenbaum. 1996. Accountability in a computerized society. Science and Engineering Ethics Vol. 2, 1 (1996), 25--42.Google ScholarCross Ref
- Helen Nissenbaum. 2009. Privacy in context: Technology, policy, and the integrity of social life. Stanford University Press, Stanford, CA. Google ScholarDigital Library
- Alamir Novin and Eric Meyers. 2017. Making sense of conflicting science information: Exploring bias in the search engine result page Proceedings of the 2017 Conference on Human Information Interaction and Retrieval. ACM, 175--184. Google ScholarDigital Library
- Alice H Oh, Hyun-Jong Lee, and Young-Min Kim. 2009. User Evaluation of a System for Classifying and Displaying Political Viewpoints of Weblogs. ICWSM.Google Scholar
- Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank citation ranking: Bringing order to the Web. Technical Report. Stanford InfoLab.Google Scholar
- Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura Granka. 2007. In google we trust: Users' decisions on rank, position, and relevance. Journal of Computer-Mediated Communication Vol. 12, 3 (2007), 801--823.Google ScholarCross Ref
- Eli Pariser. 2011. The filter bubble: What the Internet is hiding from you. Penguin UK. Google ScholarDigital Library
- Souneil Park, Seungwoo Kang, Sangyoung Chung, and Junehwa Song. 2009. NewsCube: delivering multiple aspects of news to mitigate media bias Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 443--452. Google ScholarDigital Library
- Souneil Park, Seungwoo Kang, Sangyoung Chung, and Junehwa Song. 2012. A computational framework for media bias mitigation. ACM Transactions on Interactive Intelligent Systems (TiiS), Vol. 2, 2 (2012), 8. Google ScholarDigital Library
- Souneil Park, Minsam Ko, Jungwoo Kim, Ying Liu, and Junehwa Song. 2011. The politics of comments: predicting political orientation of news stories with commenters' sentiment patterns. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work. ACM, 113--122. Google ScholarDigital Library
- Josh Pasek, Daniel Schneider, Jon A Krosnick, Alexander Tahk, Eyal Ophir, and Claire Milligan. 2014. Prevalence and moderators of the candidate name-order effect evidence from statewide general elections in California. Public Opinion Quarterly Vol. 78, 2 (2014), 416--439.Google ScholarCross Ref
- Frank Pasquale. 2015. The black box society: The secret algorithms that control money and information. Harvard University Press, Cambridge, MA. Google Scholar
- Eyal Peer, Joachim Vosgerau, and Alessandro Acquisti. 2014. Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior research methods Vol. 46, 4 (2014), 1023--1031.Google Scholar
- Richard E Petty and Pablo Brinol. 2010. Attitude change. Oxford University Press Oxford, England, 217--259.Google Scholar
- Politico. 2017. Facebook undermines its own effort to fight fake news. http://www.politico.com/story/2017/09/07/facebook-fake-news-social-media-242407. (2017). Accessed: 2017-09-07.Google Scholar
- Kristen Purcell, Joanna Brenner, and Lee Rainie. 2012. Search engine use 2012. Technical Report. Pew Research Center's Internet and American Life Project.Google Scholar
- Emilee Rader and Rebecca Gray. 2015. Understanding user beliefs about algorithmic curation in the Facebook news feed Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 173--182. Google ScholarDigital Library
- Stephen A Rains. 2013. The nature of psychological reactance revisited: A meta-analytic review. Human Communication Research Vol. 39, 1 (2013), 47--73.Google ScholarCross Ref
- Advanced Web Ranking. 2017. CTR study. https://www.advancedwebranking.com/cloud/ctrstudy. (2017). Accessed: 2017-04-01.Google Scholar
- Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Linguistic Models for Analyzing and Detecting Biased Language 51st Annual Meeting of the Association for Computational Linguistics. ACL, 1650--1659.Google Scholar
- Derek D Rucker, Zakary L Tormala, and Richard E Petty. 2004. Individual differences in resistance to persuasion: The role of beliefs and meta-beliefs. Resistance and persuasion (2004), 83.Google Scholar
- Matthew J Salganik, Peter Sheridan Dodds, and Duncan J Watts. 2006. Experimental study of inequality and unpredictability in an artificial cultural market. Science, Vol. 311, 5762 (2006), 854--856.Google Scholar
- Matthew J Salganik and Duncan J Watts. 2008. Leading the herd astray: An experimental study of self-fulfilling prophecies in an artificial cultural market. Social psychology quarterly Vol. 71, 4 (2008), 33Google Scholar
Index Terms
- Suppressing the Search Engine Manipulation Effect (SEME)
Recommendations
Toward creating a fairer ranking in search engine results
Highlights- Top web search results from search engines are topically biased.
- Diversity ...
AbstractWith the increasing popularity and social influence of search engines in IR, various studies have raised concerns on the presence of bias in search engines and the social responsibilities of IR systems. As an essential component of ...
Comments