Financial incentives and the "performance of crowds"

Abstract

The relationship between financial incentives and performance, long of interest to social scientists, has gained new relevance with the advent of web-based "crowd-sourcing" models of production. Here we investigate the effect of compensation on performance in the context of two experiments, conducted on Amazon's Mechanical Turk (AMT). We find that increased financial incentives increase the quantity, but not the quality, of work performed by participants, where the difference appears to be due to an "anchoring" effect: workers who were paid more also perceived the value of their work to be greater, and thus were no more motivated than workers paid less. In contrast with compensation levels, we find the details of the compensation scheme do matter--specifically, a "quota" system results in better work for less pay than an equivalent "piece rate" system. Although counterintuitive, these findings are consistent with previous laboratory studies, and may have real-world analogs as well.

References

1. Benkler, Y. The Wealth of Networks: How Social Production Transforms Markets and Freedom. Yale University Press, New Haven, CT, 2007.
2. Malone, T. W., Laubacher, R. and Dellarocas, C. Harnessing Crowds: Mapping the Genome of Collective Intelligence. MIT, City, 2009.
3. von Ahn, L. Games with a purpose. Computer, 39, 6 2006), 92--94.
4. Howe, J. Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business. Crown Business, New York, 2008.
5. Kleeman, F., Voss, G. G. and Rieder, K. Un(der)paid Innovators: The Commercial Utilization of Consumer Work through Crowdsourcing. Science, Technology & Innovation Studies, 4, 1 2008), 5--26.
6. Nov, O., Naaman, M. and Ye, C. What drives content tagging: the case of photos on Flickr. In Proceedings of the Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (Florence, Italy, 2008). ACM, {insert City of Publication},{insert 2008 of Publication}.
7. Prendergast, C. The provision of incentives in firms. J. Econ. Lit., 37, 1 1999), 7--63.
8. Lazear, E. P. Performance pay and productivity. American Economic Review, 90, 5 (Dec 2000), 1346--1361.
9. Murphy, K. J. Executive Compensation. SSRN, 1998.
10. Hall, B. and Liebman, J. B. Are CEOs Really Paid Like Bureaucrats? SSRN, 1997.
11. Gneezy, U. and Rustichini, A. Pay enough or don't pay at all. Q. J. Econ., 115, 3 2000), 791--810.
12. Heyman, J. and Ariely, D. Effort for Payment: A Tale of Two Markets. Psychological Science, 15, 11 2004), 787--793.
13. Camerer, C., Babcock, L., Loewenstein, G. and Thaler, R. Labor supply of New York City cabdrivers: One day at a time. Q. J. Econ., 112, 2 1997), 407--441.
14. Ariely, D., Gneezy, U., Loewenstein, G. and Mazar, N. Large Stakes and Big Mistakes. Review of Economic Studies Forthcoming, 2008).
15. Holmstrom, B. and Milgrom, P. Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design. Journal of Law, Economics, & Organization, 71991), 24--52.
16. Bandiera, O., Barankay, I. and Rasul, I. Social Preferences and the Response to Incentives: Evidence from Personnel Data*. Q. J. Econ., 120, 3 2005), 917--962.
17. Kohn, A. Why Incentive Plans Cannot Work. Harvard Business ReviewSeptember-October, 1993), 54--63.
18. Herzberg, F. One More Time: How do You Motivate Employees? Harvard Business ReviewSeptember-October, 1987), 5--16.
19. Tversky, A. and Kahneman, D. Judgment under Uncertainty: Heuristics and Biases. Science, 185, 4157 1974), 1124--1131.
20. Chapman, G. B. and Johnson, E. J. The limits of anchoring. Journal of Behavioral Decision Making, 7, 4 1994), 223--242.
21. Ariely, D., Loewenstein, G. and Prelec, D. Coherent Arbitrariness: Stable Demand Curves Without Stable Preferences. Q. J. Econ., 118, 1 2003), 73--105.
22. Kittur, A., Chi, E. H. and Suh, B. Crowdsourcing user studies with Mechanical Turk. In Proceedings of the Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (Florence, Italy, 2008). ACM, {insert City of Publication},{insert 2008 of Publication}.
23. Gelman, A. and Hill, J. Data Analysis Using Regression and Multilevel/hierarchical Models. Cambridge University Press, Cambridge, UK, 2006.
24. Jenkins, G. D., Mitra, A., Gupta, N. and Shaw, J. D. Are financial incentives related to performance? A meta-analytic review of empirical research. Journal of Applied Psychology, 83, 5 1998), 777--787.
25. Bonner, S. E., Hastie, R., Sprinkle, G. B. and Young, S. M. A review of the effects of financial incentives on performance in laboratory tasks: Implications for management accounting. Journal of Management Accounting Research, 122000), 19--64.
26. Conyon, M. J. and Murphy, K. J. The Prince and the Pauper? CEO Pay in the United States and United Kingdom. The Economic Journal, 110, 467 2000), F640-F671.
27. Blau, F. D. and Kahn, L. M. Gender Differences in Pay. The Journal of Economic Perspectives, 14, 4 2000), 75--99.
28. Choi, S. J., Gulati, G. M. and Posner, E. A. Are Judges Overpaid? A Skeptical Response to the Judicial Salary Debate. The Journal of Legal Analysis, 1, 1 2009).
29. Schwarz, N. and Strack, F. Context Effects in Attitude Surveys: Applying Cognitive Theory to Social Research. European Review of Social Psychology(Jan 1 1991).
30. Herr, P. M., Sherman, S. J. and Fazio, R. H. On the consequences of priming: assimilation and contrast effects. Journal of experimental social psychology(Print)(Jan 1 1983).
31. Bavelas, A. Communication Patterns in Task?Oriented Groups. The Journal of the Acoustical Society of America(Jan 1 1950).
32. Nosofsky, R. M. Attention, similarity, and the identification categorization relationship. Journal of Experimental Psychology: General(Jan 1 1986).
33. Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J. and Couper, M. Psychological Research Online: Report of Board of Scientific Affairs' Advisory Group on the Conduct of Research on the Internet. Am. Psychol., 59, 2 2004), 105--117.

Index Terms

• Applied computing

  • Law, social and behavioral sciences