skip to main content
Skip header Section
The Black Box Society: The Secret Algorithms That Control Money and InformationJanuary 2015
Publisher:
  • Harvard University Press
  • 79 Garden St. Cambridge, MA
  • United States
ISBN:978-0-674-36827-9
Published:05 January 2015
Pages:
320
Skip Bibliometrics Section
Bibliometrics
Skip Abstract Section
Abstract

Every day, corporations are connecting the dots about our personal behaviorsilently scrutinizing clues left behind by our work habits and Internet use. The data compiled and portraits created are incredibly detailed, to the point of being invasive. But who connects the dots about what firms are doing with this information? The Black Box Society argues that we all need to be able to do soand to set limits on how big data affects our lives. Hidden algorithms can make (or ruin) reputations, decide the destiny of entrepreneurs, or even devastate an entire economy. Shrouded in secrecy and complexity, decisions at major Silicon Valley and Wall Street firms were long assumed to be neutral and technical. But leaks, whistleblowers, and legal disputes have shed new light on automated judgment. Self-serving and reckless behavior is surprisingly common, and easy to hide in code protected by legal and real secrecy. Even after billions of dollars of fines have been levied, underfunded regulators may have only scratched the surface of this troubling behavior. Frank Pasquale exposes how powerful interests abuse secrecy for profit and explains ways to rein them in. Demanding transparency is only the first step. An intelligible society would assure that key decisions of its most important firms are fair, nondiscriminatory, and open to criticism. Silicon Valley and Wall Street need to accept as much accountability as they impose on others.

Cited By

  1. ACM
    Andalibi N, Pyle C, Barta K, Xian L, Jacobs A and Ackerman M Conceptualizing Algorithmic Stigmatization Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, (1-18)
  2. Davidson E, Wessel L, Winter J and Winter S (2023). Future directions for scholarship on data governance, digital innovation, and grand challenges, Information and Organization, 33:1, Online publication date: 1-Mar-2023.
  3. ACM
    Kingsley S, Sinha P, Wang C, Eslami M and Hong J (2022). "Give Everybody [..] a Little Bit More Equity": Content Creator Perspectives and Responses to the Algorithmic Demonetization of Content Associated with Disadvantaged Groups, Proceedings of the ACM on Human-Computer Interaction, 6:CSCW2, (1-37), Online publication date: 7-Nov-2022.
  4. ACM
    Vigna F Co-regulation Approach for Governing Big Data: Thoughts on Data Protection Law Proceedings of the 15th International Conference on Theory and Practice of Electronic Governance, (59-63)
  5. ACM
    Gansky B and McDonald S CounterFAccTual: How FAccT Undermines Its Organizing Principles 2022 ACM Conference on Fairness, Accountability, and Transparency, (1982-1992)
  6. ACM
    Susser D Decision Time: Normative Dimensions of Algorithmic Speed 2022 ACM Conference on Fairness, Accountability, and Transparency, (1410-1420)
  7. ACM
    Fischer M, Hirsbrunner S, Jentner W, Miller M, Keim D and Helm P Promoting Ethical Awareness in Communication Analysis: Investigating Potentials and Limits of Visual Analytics for Intelligence Applications 2022 ACM Conference on Fairness, Accountability, and Transparency, (877-889)
  8. ACM
    Norval C, Cornelius K, Cobbe J and Singh J Disclosure by Design: Designing information disclosures to support meaningful transparency and accountability 2022 ACM Conference on Fairness, Accountability, and Transparency, (679-690)
  9. Suryanarayana S, Sarne D and Kraus S Justifying Social-Choice Mechanism Outcome for Improving Participant Satisfaction Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, (1246-1255)
  10. ACM
    Grill G and Andalibi N (2022). Attitudes and Folk Theories of Data Subjects on Transparency and Accuracy in Emotion Recognition, Proceedings of the ACM on Human-Computer Interaction, 6:CSCW1, (1-35), Online publication date: 30-Mar-2022.
  11. Kennedy J and Holcombe-James I (2022). “It’s almost impossible to buy a dumb TV” Experiences of automated decision-making and smart technologies in low-income homes, Telematics and Informatics, 68:C, Online publication date: 1-Mar-2022.
  12. ACM
    Seidelin C, Moreau T, Shklovski I and Holten Møller N (2022). Auditing Risk Prediction of Long-Term Unemployment, Proceedings of the ACM on Human-Computer Interaction, 6:GROUP, (1-12), Online publication date: 14-Jan-2022.
  13. ACM
    Ryoo J, Morris A and Margolis J (2021). “What Happens to the Raspado man in a Cash-free Society?”: Teaching and Learning Socially Responsible Computing, ACM Transactions on Computing Education, 21:4, (1-28), Online publication date: 31-Dec-2022.
  14. ACM
    Cloete R, Norval C and Singh J (2021). Auditable Augmented/Mixed/Virtual Reality, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5:4, (1-24), Online publication date: 27-Dec-2022.
  15. ACM
    DeVito M (2021). Adaptive Folk Theorization as a Path to Algorithmic Literacy on Changing Platforms, Proceedings of the ACM on Human-Computer Interaction, 5:CSCW2, (1-38), Online publication date: 13-Oct-2021.
  16. ACM
    Posada J, Weller N and Wong W We Haven't Gone Paperless Yet: Why the Printing Press Can Help Us Understand Data and AI Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, (864-872)
  17. ACM
    Ochigame R and Ye K Search Atlas: Visualizing Divergent Search Results Across Geopolitical Borders Designing Interactive Systems Conference 2021, (1970-1983)
  18. ACM
    Kaspersen M, Bilstrup K, Van Mechelen M, Hjorth A, Bouvin N and Petersen M VotestratesML: A High School Learning Tool for Exploring Machine Learning and its Societal Implications FabLearn Europe / MakeEd 2021 - An International Conference on Computing, Design and Making in Education, (1-10)
  19. Carter D, Acker A and Sholler D (2021). Investigative approaches to researching information technology companies, Journal of the Association for Information Science and Technology, 72:6, (655-666), Online publication date: 14-May-2021.
  20. ACM
    Ariadne Muller D The Computer for the 21st Century – Second Edition for Europe Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, (1-11)
  21. ACM
    Whitney C, Naval T, Quepons E, Singh S, Rick S and Irani L HCI Tactics for Politics from Below: Meeting the Challenges of Smart Cities Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, (1-15)
  22. ACM
    Cobbe J, Lee M and Singh J Reviewable Automated Decision-Making Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, (598-609)
  23. ACM
    Albert K and Delano M This Whole Thing Smacks of Gender Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, (342-352)
  24. ACM
    Kaspersen M, Bilstrup K and Petersen M The Machine Learning Machine: A Tangible User Interface for Teaching Machine Learning Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, (1-12)
  25. ACM
    Shneiderman B (2020). Bridging the Gap Between Ethics and Practice, ACM Transactions on Interactive Intelligent Systems, 10:4, (1-31), Online publication date: 3-Dec-2020.
  26. Hatwell J, Gaber M and Azad R (2020). CHIRPS: Explaining random forest classification, Artificial Intelligence Review, 53:8, (5747-5788), Online publication date: 1-Dec-2020.
  27. de Fine Licht K and de Fine Licht J (2020). Artificial intelligence, transparency, and public decision-making, AI & Society, 35:4, (917-926), Online publication date: 1-Dec-2020.
  28. ACM
    Cunha J and Aguiar Y Reflections on the role of nudges in human-computer interaction for behavior change Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, (1-6)
  29. ACM
    Udoh E Is the data fair? Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance, (1-7)
  30. ACM
    Rajtmajer S and Susser D Automated influence and the challenge of cognitive security Proceedings of the 7th Symposium on Hot Topics in the Science of Security, (1-9)
  31. Jones K, Rubel A and LeClere E (2019). A matter of trust, Journal of the Association for Information Science and Technology, 71:10, (1227-1241), Online publication date: 11-Sep-2020.
  32. Stark L, Stanhaus A and Anthony D (2020). “I Don't Want Someone to Watch Me While I'm Working”, Journal of the Association for Information Science and Technology, 71:9, (1074-1088), Online publication date: 6-Aug-2020.
  33. ACM
    Bilstrup K, Kaspersen M and Petersen M Staging Reflections on Ethical Dilemmas in Machine Learning Proceedings of the 2020 ACM Designing Interactive Systems Conference, (1211-1222)
  34. Robbins S (2019). AI and the path to envelopment: knowledge as a first step towards the responsible regulation and use of AI-powered machines, AI & Society, 35:2, (391-400), Online publication date: 1-Jun-2020.
  35. Carabantes M (2019). Black-box artificial intelligence: an epistemological and critical analysis, AI & Society, 35:2, (309-317), Online publication date: 1-Jun-2020.
  36. Hagendorff T and Wezel K (2019). 15 challenges for AI: or what AI (currently) can’t do, AI & Society, 35:2, (355-365), Online publication date: 1-Jun-2020.
  37. Poulain R and Tarissan F (2020). Investigating the lack of diversity in user behavior, Information Processing and Management: an International Journal, 57:2, Online publication date: 1-Mar-2020.
  38. Hagendorff T (2020). The Ethics of AI Ethics: An Evaluation of Guidelines, Minds and Machines, 30:1, (99-120), Online publication date: 1-Mar-2020.
  39. ACM
    Awad E, Dsouza S, Bonnefon J, Shariff A and Rahwan I (2020). Crowdsourcing moral machines, Communications of the ACM, 63:3, (48-55), Online publication date: 24-Feb-2020.
  40. Robbins S (2019). A Misdirected Principle with a Catch: Explicability for AI, Minds and Machines, 29:4, (495-514), Online publication date: 1-Dec-2019.
  41. ACM
    Elsden C, Symons K, Bunduchi R, Speed C and Vines J (2019). Sorting Out Valuation in the Charity Shop, Proceedings of the ACM on Human-Computer Interaction, 3:CSCW, (1-25), Online publication date: 7-Nov-2019.
  42. ACM
    Pinter A, Jiang J, Gach K, Sidwell M, Dykes J and Brubaker J (2019). "Am I Never Going to Be Free of All This Crap?", Proceedings of the ACM on Human-Computer Interaction, 3:CSCW, (1-23), Online publication date: 7-Nov-2019.
  43. ACM
    Frey S, Krafft P and Keegan B (2019). "This Place Does What It Was Built For", Proceedings of the ACM on Human-Computer Interaction, 3:CSCW, (1-31), Online publication date: 7-Nov-2019.
  44. ACM
    Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F and Pedreschi D (2018). A Survey of Methods for Explaining Black Box Models, ACM Computing Surveys, 51:5, (1-42), Online publication date: 30-Sep-2019.
  45. Watson D (2019). The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence, Minds and Machines, 29:3, (417-440), Online publication date: 1-Sep-2019.
  46. ACM
    Seo H, Xiong A and Lee D Trust It or Not Proceedings of the 10th ACM Conference on Web Science, (265-274)
  47. ACM
    J. Domanski R The A.I. Pandorica Proceedings of the 20th Annual International Conference on Digital Government Research, (409-416)
  48. Pham A and Castro C (2019). The moral limits of the market, Ethics and Information Technology, 21:2, (117-126), Online publication date: 1-Jun-2019.
  49. Marciano A (2019). Reframing biometric surveillance, Ethics and Information Technology, 21:2, (127-136), Online publication date: 1-Jun-2019.
  50. ACM
    Eslami M, Vaccaro K, Lee M, Elazari Bar On A, Gilbert E and Karahalios K User Attitudes towards Algorithmic Opacity and Transparency in Online Reviewing Platforms Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, (1-14)
  51. ACM
    Geoffroy-Terryn L Regulating 'Social Scores' in Data-Driven Societies Proceedings of the 12th International Conference on Theory and Practice of Electronic Governance, (516-518)
  52. ACM
    Benthall S Situated information flow theory Proceedings of the 6th Annual Symposium on Hot Topics in the Science of Security, (1-10)
  53. ACM
    Mittelstadt B, Russell C and Wachter S Explaining Explanations in AI Proceedings of the Conference on Fairness, Accountability, and Transparency, (279-288)
  54. ACM
    Passi S and Barocas S Problem Formulation and Fairness Proceedings of the Conference on Fairness, Accountability, and Transparency, (39-48)
  55. ACM
    Susser D Invisible Influence Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, (403-408)
  56. ACM
    Castillo C (2019). Fairness and Transparency in Ranking, ACM SIGIR Forum, 52:2, (64-71), Online publication date: 17-Jan-2019.
  57. ACM
    Passi S and Jackson S (2018). Trust in Data Science, Proceedings of the ACM on Human-Computer Interaction, 2:CSCW, (1-28), Online publication date: 1-Nov-2018.
  58. ACM
    Khalil M, Prinsloo P and Slade S The unbearable lightness of consent Proceedings of the Fifth Annual ACM Conference on Learning at Scale, (1-11)
  59. ACM
    Pierce J, Fox S, Merrill N, Wong R and DiSalvo C An Interface without A User Proceedings of the 2018 Designing Interactive Systems Conference, (1345-1358)
  60. ACM
    Ramadan Q, Ahmadian A, Strüber D, Jürjens J and Staab S Model-based discrimination analysis Proceedings of the International Workshop on Software Fairness, (22-28)
  61. ACM
    Skirpan M, Cameron J and Yeh T More Than a Show Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-13)
  62. ACM
    Elsden C, Manohar A, Briggs J, Harding M, Speed C and Vines J Making Sense of Blockchain Applications Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-14)
  63. ACM
    Eslami M, Krishna Kumaran S, Sandvig C and Karahalios K Communicating Algorithmic Process in Online Behavioral Advertising Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-13)
  64. ACM
    Diaz M, Johnson I, Lazar A, Piper A and Gergle D Addressing Age-Related Bias in Sentiment Analysis Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-14)
  65. ACM
    DeVito M, Birnholtz J, Hancock J, French M and Liu S How People Form Folk Theories of Social Media Feeds and What it Means for How We Study Self-Presentation Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-12)
  66. ACM
    Rader E, Cotter K and Cho J Explanations as Mechanisms for Supporting Algorithmic Transparency Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-13)
  67. ACM
    Shackell C and Sitbon L Cognitive Externalities and HCI Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, (1-10)
  68. ACM
    DeVito M, Hancock J, French M, Birnholtz J, Antin J, Karahalios K, Tong S and Shklovski I The Algorithm and the User Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, (1-6)
  69. Robertson R, Lazer D and Wilson C Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages Proceedings of the 2018 World Wide Web Conference, (955-965)
  70. ACM
    Epstein R, Robertson R, Lazer D and Wilson C (2017). Suppressing the Search Engine Manipulation Effect (SEME), Proceedings of the ACM on Human-Computer Interaction, 1:CSCW, (1-22), Online publication date: 6-Dec-2017.
  71. ACM
    Obar J and Oeldorf-Hirsch A Clickwrap Impact Proceedings of the 8th International Conference on Social Media & Society, (1-5)
  72. ACM
    Karanasiou A and Pinotsis D Towards a legal definition of machine intelligence Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law, (119-128)
  73. ACM
    Brkan M AI-supported decision-making under the general data protection regulation Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law, (3-8)
  74. ACM
    Lee M and Baykal S Algorithmic Mediation in Group Decisions Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, (1035-1048)
  75. ACM
    Paraschakis D Recommender Systems from an Industrial and Ethical Perspective Proceedings of the 10th ACM Conference on Recommender Systems, (463-466)
  76. Doneda D and Almeida V (2016). What Is Algorithm Governance?, IEEE Internet Computing, 20:4, (60-63), Online publication date: 1-Jul-2016.
  77. Frizzo-Barker J, Chow-White P, Mozafari M and Ha D (2016). An empirical study of the rise of big data in business scholarship, International Journal of Information Management: The Journal for Information Professionals, 36:3, (403-413), Online publication date: 1-Jun-2016.
  78. Arnold T and Scheutz M (2016). Against the moral Turing test, Ethics and Information Technology, 18:2, (103-115), Online publication date: 1-Jun-2016.
  79. ACM
    Lustig C, Pine K, Nardi B, Irani L, Lee M, Nafus D and Sandvig C Algorithmic Authority Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, (1057-1062)
  80. ACM
    Anthes G (2016). Search engine agendas, Communications of the ACM, 59:4, (19-21), Online publication date: 23-Mar-2016.
Contributors

Recommendations