Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr

ABSTRACT

Online freelancing marketplaces have grown quickly in recent years. In theory, these sites offer workers the ability to earn money without the obligations and potential social biases associated with traditional employment frameworks. In this paper, we study whether two prominent online freelance marketplaces - TaskRabbit and Fiverr - are impacted by racial and gender bias. From these two platforms, we collect 13,500 worker profiles and gather information about workers' gender, race, customer reviews, ratings, and positions in search rankings. In both marketplaces, we find evidence of bias: we find that gender and race are significantly correlated with worker evaluations, which could harm the employment opportunities afforded to the workers. We hope that our study fuels more research on the presence and implications of discrimination in online environments.

References

1. Aleo, M., and Svirsky, P. Foreclosure fallout: The banking industry's attack on disparate impact race discrimination claims under the fair housing act and the equal credit opportunity act. BU Pub. Int. LJ 18 (2008).
2. Altonji, J. G., and Blank, R. M. Chapter 48 race and gender in the labor market. In Handbook of Labor Economics, vol. 3, Part C. Elsevier, 1999, pp. 3143--3259.
3. Arias, E., Heron, M., Tejada-Vera, B., et al. National vital statistics reports. National Vital Statistics Reports 61, 9 (2013).
4. Ayres, I., Banaji, M., and Jolls, C. Race effects on eBay. The RAND Journal of Economics 46, 4 (2015), 891--917.
5. Bashir, M. A., Arshad, S., Robertson, W., and Wilson, C. Tracing information flows between ad exchanges using retargeted ads. In Proceedings of Usenix Security (2016).
6. Berger, J., Fisek, M. H., Norman, R. Z., and Zelditch Jr, M. Status characteristics and interaction: An expectation states approach. NY: Elsevier, 1977.
7. Berger, J., Rosenholtz, S. J., and Zelditch, M. Status organizing processes. Annual review of sociology (1980), 479--508.
8. Bertrand, M., and Mullainathan, S. Are emily and greg more employable than lakisha and jamal? A field experiment on labor market discrimination. American Economic Review 94, 4 (2004), 991--1013.
9. Bird, S., Klein, E., and Loper, E. Natural Language Processing with Python, 1st ed. O'Reilly Media, Inc., 2009.
10. Bolton, G., Greiner, B., and Ockenfels, A. Engineering trust: Reciprocity in the production of reputation information. Management Science 59, 2 (2013), 265--285.
11. Brashears, M. E. Sex, society, and association: A cross-national examination of status construction theory. Social Psychology Quarterly 71, 1 (2008), 7--85.
12. Carlsson, M., and Rooth, D.-O. Evidence of ethnic discrimination in the swedish labor market using experimental data. Labour Economics 14, 4 (2007), 716--729. European Association of Labour Economists 18th annual conference CERGE-EI, Prague, Czech Republic 21-23 September 2006.
13. Che, Y. K., and Horner, J. Optimal Design for Social Learning. Levine's Bibliography 786969000000001075, UCLA Department of Economics, Sept. 2015.
14. Chen, L., Mislove, A., and Wilson, C. Peeking beneath the hood of uber. In Proceedings of the 2015 ACM Conference on Internet Measurement (2015).
15. Chen, L., Mislove, A., and Wilson, C. An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace. In Proceedings of the 25th International World Wide Web Conference (May 2016).
16. Cohen, E. G. Expectation states and interracial interaction in school settings. Annual Review of Sociology 8, 1 (1982), 209--235.
17. Correll, S. J. Gender and the career choice process: The role of biased self-assessments1. American journal of Sociology 106, 6 (2001), 1691--1730.
18. Craswell, N., Zoeter, O., Taylor, M., and Ramsey, B. An experimental comparison of click position-bias models. In Proceedings of the 2008 International Conference on Web Search and Data Mining (2008).
19. Dai, W., Jin, G. Z., Lee, J., and Luca, M. Optimal aggregation of consumer ratings: An application to yelp.com. Working Paper 18567, National Bureau of Economic Research, November 2012.
20. Datta, A., Tschantz, M. C., and Datta, A. Automated experiments on ad privacy settings: A tale of opacity, choice, and discrimination. In In Proceedings of the 15th Privacy Enhancing Technologies Symposium (2015).
21. David G. Wagner, J. B. Gender and interpersonal task behaviors: Status expectation accounts. Sociological Perspectives 40, 1 (1997), 1--32.
22. Dovidio, J. F., and Gaertner, S. L. Aversive racism and selection decisions: 1989 and 1999. Psychological Science 11, 4 (2000), 315--319.
23. Edelman, B. G., Luca, M., and Svirsky, D. Racial discrimination in the sharing economy: Evidence from a field experiment, 2015. http://ssrn.com/abstract=2701902.
24. Einav, L., Farronato, C., Levin, J. D., and Sundaresan, N. Sales mechanisms in online markets: What happened to internet auctions? Working Paper 19021, National Bureau of Economic Research, May 2013.
25. Eslami, M., Aleyasen, A., Karahalios, K., Hamilton, K., and Sandvig, C. Feedvis: A path for exploring news feed curation algorithms. In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing (2015).
26. Fradkin, A., Grewal, E., Holtz, D., and Pearson, M. Bias and reciprocity in online reviews: Evidence from field experiments on airbnb. In Proceedings of the Sixteenth ACM Conference on Economics and Computation (2015).
27. Gorham, B. W. News media's relationship with stereotyping: The linguistic intergroup bias in response to crime news. Journal of communication 56, 2 (2006), 289--308.
28. Guha, S., Cheng, B., and Francis, P. Challenges in measuring online advertising systems. In Proceedings of the 2010 ACM Conference on Internet Measurement (2010).
29. Hannak, A., Sapiezyński, P., Kakhki, A. M., Krishnamurthy, B., Lazer, D., Mislove, A., and Wilson, C. Measuring Personalization of Web Search. In Proceedings of the 22nd International World Wide Web Conference (May 2013).
30. Hannak, A., Soeller, G., Lazer, D., Mislove, A., and Wilson, C. Measuring price discrimination and steering on e-commerce web sites. In Proceedings of the 2014 ACM Conference on Internet Measurement (2014).
31. Hopcroft, R. L. Is gender still a status characteristic? Current Research in Social Psychology 7, 20 (2002), 339--346.
32. Hoque, F. How the rising gig economy is reshaping businesses. Fast Company, Sept. 2015. https://www.fastcompany.com/3051315/the-future-ofwork/the-gig-economy-is-going-global-heres-why-andwhat-it-means.
33. Intuit. Intuit 2020 report. http://http-download.intuit.com/http.intuit/CMO/intuit/futureofsmallbusiness/intuit_2020_report.pdf.
34. Kliman-Silver, C., Hannak, A., Lazer, D., Wilson, C., and Mislove, A. Location, location, location: The impact of geolocation on web search personalization. In Proceedings of the 2015 ACM Conference on Internet Measurement (2015).
35. Kricheli-Katz, T., and Regev, T. How many cents on the dollar? women and men in product markets. Science Advances 2, 2 (2016).
36. Laumeister, G. The Next Big Thing In E-Commerce: Online Labor Marketplaces. Forbes, 2014. http://www.forbes.com/sites/groupthink/2014/10/21/the-nextbig-thing-in-e-commerce-online-labor-marketplaces/.
37. Lécuyer, M., Ducoffe, G., Lan, F., Papancea, A., Petsios, T., Spahn, R., Chaintreau, A., and Geambasu, R. Xray: Enhancing the web's transparency with differential correlation. In Proceedings of the 23rd USENIX Conference on Security Symposium (2014).
38. Lee, M. K., Kusbit, D., Metsky, E., and Dabbish, L. Working with machines: The impact of algorithmic and data-driven management on human workers. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (2015).
39. Lemkau, J. P. Personality and background characteristics of women in male-dominated occupations: A review. Psychology of Women Quarterly 4, 2 (1979), 221--240.
40. Maass, A., Salvi, D., Arcuri, L., and Semin, G. R. Language use in intergroup contexts: the linguistic intergroup bias. Journal of personality and social psychology 57, 6 (1989), 981.
41. Martha Foschi, Larissa Lai, K. S. Gender and double standards in the assessment of job applicants. Social Psychology Quarterly 57, 4 (1994), 326--339.
42. Mikians, J., Gyarmati, L., Erramilli, V., and Laoutaris, N. Detecting price and search discrimination on the internet. In Proceedings of the 11th ACM Workshop on Hot Topics in Networks (2012).
43. Mikians, J., Gyarmati, L., Erramilli, V., and Laoutaris, N. Crowd-assisted search for price discrimination in e-commerce: First results. In Proceedings of the Ninth ACM Conference on Emerging Networking Experiments and Technologies (2013).
44. Morton, F., Zettelmeyer, F., and Silva-Risso, J. Consumer information and discrimination: Does the internet affect the pricing of new cars to women and minorities? Quantitative Marketing and Economics 1, 1 (2003), 65--92.
45. Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., and Handelsman, J. Science faculty's subtle gender biases favor male students. Proceedings of the National Academy of Sciences 109, 41 (2012), 16474--16479.
46. Musthag, M., and Ganesan, D. Labor dynamics in a mobile micro-task market. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2013).
47. Otterbacher, J. Linguistic bias in collaboratively produced biographies: Crowdsourcing social stereotypes? In Proceedings of the 9th International AAAI Conference on Web and Social Media (2015).
48. Richardson, M. Predicting clicks: Estimating the click-through rate for new ads. In Proceedings of the 16th International World Wide Web Conference (2007).
49. Ridgeway, C. L. Framed by gender - how gender inequality persists in the modern world. Oxford University Press (2011).
50. Ridgeway, C. L. Why status matters for inequality. American Sociological Review 79, 1 (2014), 1--16.
51. Sandvig, C., Hamilton, K., Karahalios, K., and Langbort, C. Auditing algorithms: Research methods for detecting discrimination on internet platforms. In Proceedings of "Data and Discrimination: Converting Critical Concerns into Productive Inquiry", a preconference at the 64th Annual Meeting of the International Communication Association (2014).
52. Schmader, T. Gender identification moderates stereotype threat effects on women's math performance. Journal of Experimental Social Psychology 38, 2 (2002), 194--201.
53. Soeller, G., Karahalios, K., Sandvig, C., and Wilson, C. MapWatch: Detecting and Monitoring International Border Personalization on Online Maps. In Proceedings of the 25th International World Wide Web Conference (May 2016).
54. Steele, Claude M.; Aronson, J. Stereotype threat and the intellectual test performance of african americans. Journal of Personality and Social Psychology 69, 5 (1995), 797--811.
55. Sundararajan, A. The 'gig economy' is coming. what will it mean for work? The Guardian, July 2015. https://www.theguardian.com/commentisfree/2015/jul/26/will-we-get-by-gig-economy.
56. Support, T. Are your taskers screened and background checked? TaskRabbit, May 2016. https://support.taskrabbit.com/hc/en-us/articles/204411630Are-your-Taskers-screened-and-background-checked-.
57. Sweeney, L. Discrimination in online ad delivery, 2013. http://ssrn.com/abstract=2208240.
58. Teodoro, R., Ozturk, P., Naaman, M., Mason, W., and Lindqvist, J. The motivations and experiences of the on-demand mobile workforce. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work and; Social Computing (2014).
59. Thebault-Spieker, J., Terveen, L. G., and Hecht, B. Avoiding the south side and the suburbs: The geography of mobile crowdsourcing markets. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work and Social Computing (2015).
60. Todisco, M. Share and share alike? considering racial discrimination in the nascent room-sharing economy. Stanford Law Review Online 67 (2015), 121--129.
61. Wagner, C., Garcia, D., Jadidi, M., and Strohmaier, M. It's a man's wikipedia? assessing gender inequality in an online encyclopedia. CoRR abs/1501.06307 (2015).
62. Wagner, C., Garcia, D., Jadidi, M., and Strohmaier, M. It's a man's wikipedia? assessing gender inequality in an online encyclopedia. In In Proceedings of the 9th Annual International AAAI Conference on Web and Social Media (2015).
63. Wang, G., Mohanlal, M., Wilson, C., Wang, X., Metzger, M., Zheng, H., and Zhao, B. Y. Social turing tests: Crowdsourcing sybil detection. In In Proceedings of the 20th Annual Network & Distributed System Security Symposium (2013).
64. Wilson, T., Wiebe, J., and Hoffmann, P. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on human language technology and empirical methods in natural language processing (2005).
65. Zyskowski, K., Morris, M. R., Bigham, J. P., Gray, M. L., and Kane, S. K. Accessible crowdwork?: Understanding the value in and challenge of microtask employment for people with disabilities. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work and Social Computing (2015).

Index Terms

• Applied computing

  • Law, social and behavioral sciences

• Human-centered computing

  • Collaborative and social computing

    • Collaborative and social computing systems and tools

• Information systems

  • World Wide Web

• Social and professional topics

  • Professional topics

    • Computing and business

      • Employment issues