Examining the Impact and Detection of the "Urban Legend" of Common Method Bias

Abstract

Common Method Bias (CMB) represents one of the most frequently cited concerns among Information System (IS) and social science researchers. Despite the broad number of commentaries lamenting the importance of CMB, most empirical studies have relied upon Monte Carlo simulations, assuming that all of the sources of bias are homogenous in their impact. Comparatively analyzing field-based data, we address the following questions: (1) What is the impact of different sources of CMB on measurement and structural models? (2) Do the most commonly utilized approaches for detecting CMB produce similar estimates? Our results provide empirical evidence that the sources of CMB have differential impacts on measurement and structural models, and that many of the detection techniques commonly utilized within the IS field demonstrate inconsistent accuracy in discerning these differences.

References

1. Agarwal, R., and Prasad, I. 1998. "A Conceptual and Optional Definition of Personal lnnovativeness in the Domain of lnformation Technology," Information Systems Research (9:2), pp 204--215.
2. Baumgartner, H., and Steenkamp, J. B. 200 "Response styles in marketing research: A cross-national investigation," Journal of Marketing Research), pp 143--156.
3. Baumgartner, H., and Weijters, B. 2012. "Commentary on ?Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies?," Journal of Retailing (88:4), pp 563--566.
4. Burton-Jones, A. 2009. "Minimizing method bias through programmatic research," MIS Quarterly (33:3), pp 445--471.
5. Campbell, D., T., and Fiske, D. 1959. "Convergent and discriminant validation by the multitrait-multi-method matrix," Psychological Bulletin (52:2), pp 81--105.
6. Chin, W. W. 1998. "The partial least squares approach for structural equation modeling," in Modern methods for business research, G. A. Marcoulides (ed.), Erlbaum: Mahwah, NJ, pp. 295--336.
7. Chin, W. W., Thatcher, J. B., and Wright, R. T. 2012. "Assessing Common Method Bias: Problems With the ULMC Technique," MIS Quarterly (36:3), pp 1003--1019.
8. Clements, K., and Turpin, G. 1996. "The Life Events Scale For Students: Validation For Use With British Samples," Personality and Individual Differences (20:6), pp 747--751.
9. Crampton, S. M., and Wagner, J. A. 1998. "Percept-percept inflation in microorganizational research: An investigation of prevalence and effect," Journal of Applied Psychology (79:3), pp 421--135.
10. Dwyer, D., Ringstaff, C., and Sandholtz, J. 1991. "Changes in teachers? beliefs and practices in technology-rich classrooms," Educational Leadership (48:8), pp 45--52.
11. Feldman, J. M., and Lynch, J. G. 1998. "Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior," Journal of Applied Psychology (73:3), pp 421--135.
12. Gefen, D., Rigdon, D., and Straub, D. W. 2011. "An Update and Extension to SEM Guidelines for Administrative and Social Science Research," MIS Quarterly (35:2), pp iii-xiv.
13. Hufnagel, E. M., and Conca, C. 1994. "User response data: The potential for errors and biases," Information Systems Research (5:1), pp 48--73.
14. Jackson, D. N. 1967. "Acquiescence response styles: Problems of identification and control," in Response set in personality assessment, I. A. Barg (ed.), Aldine: Chicago, IL.
15. Kemery, E. R., and Dunlap, W. P. 1982. "Partialling factor scores does not control method variance: A reply to Podsakoff and Todor," Journal of Management (12:4), pp 525--544.
16. Lewis, W., Agarwal, R., and Sambamurthy, V. 2003. "Sources of Influence on Beliefs about Information Technology Use: An Empirical Study of Knowledge Workers," MIS Quarterly (27:4), pp 657--678.
17. Lindell, M. K., and Whitney, D. J. 2001. "Accounting for common method variance in cross-sectional research designs," Journal of Applied Psychology (86:1), pp 114--121.
18. MacKenzie, S. B., and Podsakoff, P. M. 2012. "Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies," Journal of Retailing).
19. Malhotra, N. K., Kim, S. S., and Patil, A. 2006. "Common Method Variance in IS Research: A Comparison of Alternative Approaches and a Reanalysis of Past Research," Management Science (52:12), pp 1865--1883.
20. Meade, A. W., Watson, A. M., and Kroustalis, C. M. Year. "Assessing common methods bias in organizational research," 22nd annual meeting of the Society for Industrial and Organizational Psychology, New York2007, pp. 1--6.
21. Ostroff, C., Kinicki, A. J., and Clark, M. A. 2002. "Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships," Journal of Applied Psychology (87:2), pp 355--368.
22. Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., and Podsakoff., N. P. 2003. "Common Method Biases in behavioral research: A critical review of the literature and recommended remedies," Journal of Applied Psychology (88:5), pp 879--903.
23. Richardson, H. A., Simmering, M. J., and Sturman, M. C. 2009. "A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method variance," Organizational Research Methods (12:4), pp 762--800.
24. Rizzuto, T.E. & Park, S. (under development). Predicting training behaviors through the analysis of Willingness to Learn: A construct validation study. Human Resource Development Quarterly.
25. Rorer, L. G. 1965. "The great response-style myth," Psychological Bulletin (63), pp 129--156.
26. Sharma, R., Yetton, P., and Crawford, J. 2009. "Estimating the effect of common method variance: the method--method pair technique with an illustration from TAM research," Applied Psychology (86:1), pp 114--121.
27. Spector, P. E. 2006. "Method Variance in Organizational Research: Truth or Urban Legend?," Research Methods (9), pp 221--232.
28. Spector, P. E., and Brannick, M. T. 2010. "Common method issues: An introduction to the feature topic in organizational research methods," Organizational Research Methods (13:3), pp 403--406.
29. Straub, D., Boudreau, M. C., and Gefen, D. 2004. "Validation guidelines for IS positivist research," Communications of the Association of Information Systems (13), pp 380--427.
30. Viswanathan, M., and Kayande, U. 2012. "Commentary on ?Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies?," Journal of Retailing (88:4), pp 556--562.
31. Watson, D., Clark, L. A., and Tellegen, A. 1988. "Development and validation of brief measures of positive and negative affect: the PANAS scales," Journal of Personality and Social Psychology (54:6), pp 1063--1070.
32. Westland, C. J. 2010. "Lower bounds on sample size in structural equation modeling," Electronic Commerce Research and Applications (9:6), pp 476--487.
33. Williams, L. J., and Anderson, S. E. 1994. "An alternative approach to method effects by using latent-variable models: Applications in organizational behavior research," Journal of Applied Psychology (79:3), pp. 323--331.
34. Wold, H. 1985. "Systems analysis by partial least squares," in Measuring the Unmeasurable, P. Nijkamp, Leitner, H., & Wrigley, N. (ed.), Martinus Nijhoff: Dordrecht, The Netherlands, pp. 221--251.

Index Terms

• General and reference

  • Document types

    • Reference works

• Mathematics of computing

  • Probability and statistics