Dynamic Recommendations for Sequential Hiring Decisions in Online Labor Markets

ABSTRACT

Online labor markets facilitate transactions between employers and a diverse set of independent contractors around the globe. When making hiring decisions in these markets, employers have to assess a large and heterogeneous population of contractors. Because many of the contractors' characteristics are latent, employers often make risky decisions that end up in negative outcomes. In this work, we address this issue by proposing a framework for recommending contractors who are likely to get hired and successfully complete the task at hand. We start our analysis by acknowledging that employers' hiring behavior dynamically evolves with time; Employers learn to choose contractors according to the outcomes of their previously completed tasks. To capture this dynamic evolution, we propose a structured Hidden Markov Model that explicitly models task outcomes through the employers' evolution. We build and evaluate the proposed framework on a dataset of real online hiring decisions. We then compare our approach with a set of previously proposed static algorithms and we show that our proposed framework provides up to 24% improved recommendations. We conclude by discussing the positive impact that such better recommendations of candidates can have on employers, contractors, and the market itself.

References

1. Ajay Agrawal, John Horton, Nicola Lacetera, and Elizabeth Lyons. 2015. Digitization and the contract labor market. Economic Analysis of the Digital Economy (2015), 219.
2. Ajay K. Agrawal, Nicola Lacetera, and Elizabeth Lyons. 2013. Does information help or hinder job applicants from less developed countries in online markets? Technical Report. National Bureau of Economic Research.
3. George A Akerlof. 1970. The market for "lemons": Quality uncertainty and the market mechanism. The Quarterly Journal of Economics (1970), 488--500.
4. Ashish Arora and Chris Forman. 2007. Proximity and information technology outsourcing: How local are IT services markets? Journal of Management Information Systems Vol. 24, 2 (2007), 73--102.
5. J. Yannis Bakos. 1997. Reducing buyer search costs: Implications for electronic marketplaces. Management Science Vol. 43, 12 (1997), 1676--1692.
6. Rajiv D. Banker and Iny Hwang. 2008. Importance of measures of past performance: Empirical evidence on quality of e-service providers. Contemporary Accounting Research Vol. 25, 2 (2008), 307--337.
7. C. M. Bishop et almbox. 2006. Pattern recognition and machine learning. Vol. 4. springer New York.
8. Richard H. Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. 1995. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing Vol. 16, 5 (1995), 1190--1208.
9. Jason Chan and Jing Wang. 2017. Hiring preferences in online labor markets: Evidence of a female hiring bias. Management Science (forthcoming) (2017).
10. Kenneth Ward Church and Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. Computational Linguistics Vol. 16, 1 (1990), 22--29.
11. David Gefen and Erran Carmel. 2008. Is the world really flat? A look at offshoring at an online programming marketplace. MIS Quarterly (2008), 367--384.
12. David Gefen, Gavriel Gefen, and Erran Carmel. 2015. How project description length and expected duration affect bidding and project success in crowdsourcing software development. Journal of Systems and Software (2015).
13. Tomer Geva and Maytal Saar-Tsechansky. 2016. Who's a good decision maker? Data-driven expert worker ranking under unobservable quality. International Conference on Information Systems (ICIS) (2016).
14. Ejaz Ghani, William R. Kerr, and Christopher Stanton. 2014. Diasporas and outsourcing: evidence from oDesk and India. Management Science Vol. 60, 7 (2014), 1677--1697.
15. Anindya Ghose, Param Vir Singh, and Vilma Todri. 2017. Got Annoyed? Examining the Advertising Effectiveness and Annoyance Dynamics. In International Conference on Information Systems (ICIS).
16. Paulo Goes and Mingfeng Lin. 2012. Does information really unravel? Understanding factors that motivate sellers to seek third-party certifications in an online labor market. NET Institute Working Paper 12-02 (2012).
17. Yili Hong and Paul A. Pavlou. 2017. On buyer selection of service providers in online outsourcing platforms for IT services. Information Systems Research (2017).
18. John Horton and Joseph Golden. 2015. Reputation inflation: evidence from an online labor market. Working Paper, NYU (2015).
19. John Joseph Horton and Ramesh Johari. 2015. At what quality and what price? Eliciting buyer preferences as a market design problem. Proceedings of the Sixteenth ACM Conference on Economics and Computation (2015), 507--507.
20. N. Hu, J. Zhang, and P. A. Pavlou. 2009. Overcoming the J-shaped distribution of product reviews. Commun. ACM Vol. 52, 10 (2009), 144--147.
21. Kinshuk Jerath, Peter S. Fader, and Bruce G. S. Hardie. 2011. New perspectives on customer "death" using a generalization of the Pareto/NBD model. Marketing Science Vol. 30, 5 (2011), 866--880.
22. Marios Kokkodis. 2012. Learning from positive and unlabeled Amazon reviews: Towards identifying trustworthy reviewers. In Proceedings of the 21st International Conference on World Wide Web (WWW). ACM, 545--546.
23. Marios Kokkodis and Panos Ipeirotis. 2014. The utility of skills in online labor markets. Proceedings of the AIS International Conference on Information Systems (ICIS) (2014).
24. Marios Kokkodis and Panagiotis G Ipeirotis. 2013. Have you done anything like that? predicting performance using inter-category reputation. In Proceedings of the sixth ACM International Conference on Web Search and Data Mining (WSDM). ACM, 435--444.
25. Marios Kokkodis and Panagiotis G. Ipeirotis. 2015. Reputation transferability in online labor markets. Management Science Vol. 62, 6 (2015), 1687--1706.
26. Marios Kokkodis and Theodoros Lappas. 2016. Realizing the Activation Potential of Online Communities International Conference on Information Systems (ICIS).
27. Marios Kokkodis, Panagiotis Papadimitriou, and Panagiotis G. Ipeirotis. 2015. Hiring behavior models for online labor markets. In Proceedings of the eighth ACM International Conference on Web Search and Data Mining (WSDM). ACM.
28. Daphne Koller and Nir Friedman. 2009. Probabilistic graphical models: principles and techniques. MIT press.
29. Gueorgi Kossinets and Duncan J. Watts. 2009. Origins of homophily in an evolving social network. Amer. J. Sociology Vol. 115, 2 (2009), 405--450.
30. Mingfeng Lin, Yong Liu, and Siva Viswanathan. 2018. Effectiveness of Reputation in Contracting for Customized Production: Evidence from Online Labor Markets. Management Science Vol. 64, 1 (2018), 345--359.
31. Mingfeng Lin and Siva Viswanathan. 2015. Home bias in online investments: An empirical study of an online crowdfunding market. Management Science Vol. 62, 5 (2015), 1393--1414.
32. Roy Mill. 2011. Hiring and learning in online global labor markets. Available at SSRN 1957962 (2011).
33. Antonio Moreno and Christian Terwiesch. 2014. Doing business with strangers: Reputation in online service marketplaces. Information Systems Research Vol. 25, 4 (2014), 865--886.
34. Kevin P. Murphy. 2012. Machine learning: a probabilistic perspective. The MIT Press.
35. Amanda Pallais. 2014. Inefficient hiring in entry-level labor markets. The American Economic Review Vol. 104, 11 (2014), 3565--3599.
36. F. Provost and T. Fawcett. 2001. Robust classification for imprecise environments. Machine Learning Vol. 42, 3 (2001), 203--231.
37. Eli M. Snir and Lorin M. Hitt. 2003. Costly bidding in online markets for IT services. Management Science Vol. 49, 11 (2003), 1504--1520.
38. Christopher Stanton and Catherine Thomas. 2015. Landing the first job: The value of intermediaries in online hiring. The Review of Economic Studies (2015), rdv042.
39. Hema Yoganarasimhan. 2013. The value of reputation in an online freelance marketplace. Marketing Science Vol. 32, 6 (2013), 860--891.

Index Terms

• Information systems

  • Information systems applications

    • Decision support systems

      • Data analytics