Thomas W. Malone
Jeffrey V. Nickerson
Robert J. Laubacher
ABSTRACT
A key issue, whenever people work together to solve a complex problem, is how to divide the problem into parts done by different people and combine the parts into a solution for the whole problem. This paper presents a novel way of doing this with groups of contests called contest webs. Based on the analogy of supply chains for physical products, the method provides incentives for people to (a) reuse work done by themselves and others, (b) simultaneously explore multiple ways of combining interchangeable parts, and (c) work on parts of the problem where they can contribute the most.
The paper also describes a field test of this method in an online community of over 50,000 people who are developing proposals for what to do about global climate change. The early results suggest that the method can, indeed, work at scale as intended.
- Lada A. Adamic. 2002. Zipf, Power-laws, and Pareto - a ranking tutorial. Retrieved August 5, 2016 from http://www.hpl.hp.com/research/idl/papers/ranking/ranking.htmlGoogle Scholar
- Robert J. Allio. 2004. CEO Interview: The InnoCentive Model of Open Innovation. Strategy & Leadership 32, 4: 4--9.Google ScholarCross Ref
- Jeff Alstott, Ed Bullmore, and Dietmar Plenz. 2014. powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions. PLOS ONE 9, 1: e85777.Google ScholarCross Ref
- Kenneth J. Arrow. 1962. Economic Welfare and the Allocation of Resources for Invention. In The Rate and Direction of Inventive Activity: Economic and Social Factors, Universities-National Bureau Committee for and Committee on Economic Growth of the Social Science Research Council (eds.). Princeton University Press, Rochester, NY, 609--626. Retrieved from www.nber.org/chapters/c2144.pdfGoogle Scholar
- Kenneth J. Arrow and Gerard Debreu. 1954. Existence of an equilibrium for a competitive economy. Econometrica: Journal of the Econometric Society: 265--290.Google ScholarCross Ref
- Carliss Young Baldwin and Kim B. Clark. 2000. Design Rules: The Power of Modularity. MIT Press, Cambridge, MA. Google ScholarDigital Library
- Joel Chan, Steven P. Dow, and Christian D. Schunn. 2015. Do the best design ideas (really) come from conceptually distant sources of inspiration? Design Studies 36: 31--58.Google ScholarCross Ref
- Lydia B. Chilton, Juho Kim, Paul André, Felicia Cordeiro, James A. Landay, Daniel S. Weld, Steven P. Dow, Robert C. Miller, and Haoqi Zhang. 2014. Frenzy: Collaborative Data Organization for Creating Conference Sessions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1255--1264. Google ScholarDigital Library
- Lydia B. Chilton, Greg Little, Darren Edge, Daniel S. Weld, and James A. Landay. 2013. Cascade: Crowdsourcing Taxonomy Creation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 1999--2008. Google ScholarDigital Library
- Ellen Christiaanse and Kuldeep Kumar. 2000. ICT-enabled coordination of dynamic supply webs. International Journal of Physical Distribution & Logistics Management 30, 3/4: 268--285.Google ScholarCross Ref
- Gavin Clarkson and Marshall Van Alstyne. 2007. The Social Efficiency of Fairness: An Innovation Economics Approach to Innovation. In 40th Annual Hawaii International Conference on System Sciences (HICSS '07), 155b--155b. Google ScholarDigital Library
- Ronald H. Coase. 1937. The Nature Of The Firm. Economica 4, 16: 386--405.Google ScholarCross Ref
- Kenneth Davis. 2015. InnoCentive.com Collaboration Case Study. Journal of Management 3, 1: 20--22.Google Scholar
- Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM 51, 1: 107--113. Google ScholarDigital Library
- Erik P. Duhaime, Gary M. Olson, and Thomas W. Malone. 2015. Broad Participation in Collective Problem Solving Can Influence Participants and Lead to Better Solutions: Evidence from the MIT Climate CoLab. MIT Center for Collective Intelligence, Cambridge, MA. Retrieved from http://cci.mit.edu/working_papers_2012_2013/duhaime%20colab%20wp%206--2015%20final.pdfGoogle Scholar
- David Easley and Jon Kleinberg. 2010. Power Laws and Rich-Get-Richer Phenomena. In Networks, Crowds, and Markets: Reasoning About a Highly Connected World. Cambridge University Press, 543--560. Retrieved from https://www.cs.cornell.edu/home/kleinber/networks-book/networks-book-ch18.pdfGoogle Scholar
- Jay R. Galbraith. 1995. Designing organizations: An executive briefing on strategy, structure, and process. Jossey-Bass.Google Scholar
- David Hounshell. 1985. From the American System to Mass Production, 1800--1932: The Development of Manufacturing Technology in the United States. JHU Press.Google Scholar
- James Howison and James D. Herbsleb. 2013. Incentives and Integration in Scientific Software Production. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW '13), 459--470. Google ScholarDigital Library
- Joshua Introne, Robert Laubacher, and Thomas Malone. 201 Enabling Open Development Methodologies in Climate Change Assessment Modeling. IEEE Software 28, 6: 56--61. Google ScholarDigital Library
- Joshua Introne, Robert Laubacher, Gary Olson, and Thomas Malone. 2011. The Climate CoLab: Large Scale Model-based Collaborative Planning. In International Conference on Collaboration Technologies and Systems (CTS), 40--47.Google Scholar
- Juho Kim, Haoqi Zhang, Paul André, Lydia B. Chilton, Wendy Mackay, Michel Beaudouin-Lafon, Robert C. Miller, and Steven P. Dow. 2013. Cobi: A Community-informed Conference Scheduling Tool. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13), 173--182. Google ScholarDigital Library
- Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. The Future of Crowd Work. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW '13), 1301--1318. Google ScholarDigital Library
- Aniket Kittur, Boris Smus, Susheel Khamkar, and Robert E. Kraut. 2011. CrowdForge: Crowdsourcing Complex Work. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11), 43--52. Google ScholarDigital Library
- Anand Kulkarni, Matthew Can, and Björn Hartmann. 2012. Collaboratively Crowdsourcing Workflows with Turkomatic. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW '12), 1003--1012. Google ScholarDigital Library
- Karim Lakhani, David A. Garvin, and Eric Lonstein. 2010. TopCoder (A): Developing Software Through Crowdsourcing. Harvard Business School General Management Unit Case, 610--32.Google Scholar
- Karim R. Lakhani, Anne-Laure Fayard, Natalia Levina, and Stephanie Healy Pokrywa. 2012. OpenIDEO. Harvard Business School Technology & Operations Management Unit Case, 612--66.Google Scholar
- Thomas D. LaToza, W. Ben Towne, Christian M. Adriano, and André van der Hoek. 2014. Microtask Programming: Building Software with a Crowd. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14), 43--54. Google ScholarDigital Library
- Paul R. Lawrence and Jay W. Lorsch. 1967. Differentiation and Integration in Complex Organizations. Administrative Science Quarterly 12, 1: 1--47.Google ScholarCross Ref
- Charlotte P. Lee and Drew Paine. 2015. From The Matrix to a Model of Coordinated Action (MoCA): A Conceptual Framework of and for CSCW. In Proceedings of the ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15), 179--194. Google ScholarDigital Library
- Thomas W. Malone and Kevin Crowston. 1994. The Interdisciplinary Study of Coordination. ACM Computing Surveys (CSUR) 26, 1: 87--119. Google ScholarDigital Library
- Thomas W. Malone and Mark Klein. 2007. Harnessing Collective Intelligence to Address Global Climate Change. Innovations 2, 3: 15--26.Google ScholarCross Ref
- Thomas W. Malone, Robert Laubacher, and Laur Fisher. 2014. How Millions of People Can Help Solve Climate Change. PBS NOVA Next. Retrieved January 18, 2014 from http://www.pbs.org/wgbh/nova/next/earth/crowdsourcing-climate-change-solutions/Google Scholar
- David A. Nadler and Michael L. Tushman. 1997. Competing by Design: The Power of Organizational Architecture. Oxford University Press, New York.Google Scholar
- Jeffrey V. Nickerson and Andres Monroy-Hernandez. 2011. Appropriation and Creativity: User-Initiated Contests in Scratch. In 44th Hawaii International Conference on System Sciences (HICSS '11), 1--10. Google ScholarDigital Library
- Pinar Ozturk, Yue Han, W. Ben Towne, and Jeffrey V. Nickerson. 2016. Topic Prevalence and Reuse in an Open Innovation Community. In Collective Intelligence. Retrieved from https://sites.google.com/a/stern.nyu.edu/collective-intelligence-conference/Google Scholar
- Jerome H. Reichman. 2009. Intellectual Property in the Twenty-First Century: Will the Developing Countries Lead or Follow? Houston Law Review 46, 4: 1115--1185.Google Scholar
- Daniela Retelny, Sébastien Robaszkiewicz, Alexandra To, Walter S. Lasecki, Jay Patel, Negar Rahmati, Tulsee Doshi, Melissa Valentine, and Michael S. Bernstein. 2014. Expert Crowdsourcing with Flash Teams. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14), 75--85. Google ScholarDigital Library
- Lloyd S. Shapley. 1953. Stochastic Games. Proceedings of the National Academy of Sciences 39, 10: 1095--1100.Google ScholarCross Ref
- Herbert Alexander Simon. 1996. The Sciences of the Artificial. MIT Press. Google ScholarDigital Library
- Adam Smith. 1776. An Inquiry Into the Nature and Causes of the Wealth of Nations. Strahan & Cadell, The Strand, London. Retrieved from https://books.google.com/books?id=C5dNAAAAcAAJGoogle Scholar
- Thomas F. Stocker, Dahe Qin, Gian-Kasper Plattner, Melinda M.B. Tignor, Simon K. Allen, Judith Boschung, Alexander Nauels, Yu Xia, Vincent Bex, and Pauline M. Midgley (eds.). 2013. Climate Change 2013- The Physical Science Basis: Working Group I Contribution to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, UK. Retrieved from https://www.ipcc.ch/pdf/assessment-report/ar5/wg1/WG1AR5_SummaryVolume_FINAL.pdfGoogle Scholar
- James D. Thompson. 1967. Organizations in Action: Social Science Bases of Administrative Theory. Transaction Publishers.Google Scholar
- W. Ben Towne, Carolyn P. Rosé, and James D. Herbsleb. 2016. The Key Role of Navigation in Online Challenge Platforms. Retrieved from https://sites.google.com/a/stern.nyu.edu/collective-intelligence-conference/Google Scholar
- U.S. Energy Information Administration. 2013. International Energy Outlook 2013. Energy Information Administration, Washington, DC. Retrieved July 25, 2013 from https://www.eia.gov/forecasts/archive/ieo13/Google Scholar
- David Weinberger. 2003. Small Pieces Loosely Joined: A Unified Theory Of The Web. Basic Books. Google ScholarDigital Library
- Oliver E. Williamson. 1975. Markets and Hierarchies, Analysis and Antitrust Implications: A Study in the Economics of Internal Organization. Free Press. Retrieved from https://books.google.com/books?id=IFm3AAAAIAAJGoogle Scholar
- Lixiu Yu, Aniket Kittur, and Robert E. Kraut. 2014. Distributed Analogical Idea Generation: Inventing with Crowds. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1245--1254. Google ScholarDigital Library
- Lixiu Yu, Aniket Kittur, and Robert E. Kraut. 2014. Searching for Analogical Ideas with Crowds. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1225--1234. Google ScholarDigital Library
- Haoqi Zhang, Edith Law, Rob Miller, Krzysztof Gajos, David Parkes, and Eric Horvitz. 2012. Human Computation Tasks with Global Constraints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 217--226. Google ScholarDigital Library
- Jonathan Zittrain. 2008. The Future of the Internet--And How to Stop It. Yale University Press. Google ScholarDigital Library
Index Terms
- Putting the Pieces Back Together Again: Contest Webs for Large-Scale Problem Solving
Recommendations
Putting the Human Back in the Protocol
Security ProtocolsHello, everyone, and welcome to the 14th International Security Protocols Workshop. I'm going to start with a quotation from someone who, at least in principle, is in charge of a very different security community than ours:
Our enemies are innovative and ...
Putting the Human Back in Voting Protocols
Security ProtocolsI'd like to talk about the role of the human in voting protocols. Basically I want to argue that voting protocols seem to be particularly interesting from the point of view of the theme of this workshop, in the sense that the users of the system ...
Bridging the service gap with student employees: curse or blessing?
SIGUCCS '01: Proceedings of the 29th annual ACM SIGUCCS conference on User servicesMany of us must rely on student employees to staff our facilities. It takes a great deal of time, energy and imagination to train and manage student employees. We are often forced to accept whatever skill level they may have already attained (or not) ...
Comments