Faith, hope, and love: an essay on software science's neglect of human factors

Abstract

Research in the area of programming languages has different facets -- from formal reasoning about new programming language constructs (such as type soundness proofs for new type systems) over inventions of new abstractions, up to performance measurements of virtual machines. A closer look into the underlying research methods reveals a distressing characteristic of programming language research: developers, which are the main audience for new language constructs, are hardly considered in the research process. As a consequence, it is simply not possible to state whether a new construct that requires some kind of interaction with the developer has any positive impact on the construction of software. This paper argues for appropriate research methods in programming language research that rely on studies of developers -- and argues that the introduction of corresponding empirical methods not only requires a new understanding of research but also a different view on how to teach software science to students.

References

1. }}Bartsch, M., and Harrison, R. An exploratory study of the effect of aspect-oriented programming on maintainability. Software Quality Control 16, 1 (2008), 23--44.
2. }}Basili, V. R. The role of experimentation in software engineering: past, current, and future. In ICSE ’96: Proceedings of the 18th international conference on Software engineering (Washington, DC, USA, 1996), IEEE Computer Society, pp. 442--449.
3. }}Basili, V. R., Selby, R. W., and Hutchens, D. H. Experimentation in software engineering. IEEE Trans. Software Eng. 12, 7 (1986), 733--743.
4. }}Blackburn, S. M., Garner, R., Hoffmann, C., Khang, A. M., McKinley, K. S., Bentzur, R., Diwan, A., Feinberg, D., Frampton, D., Guyer, S. Z., Hirzel, M., Hosking, A., Jump, M., Lee, H., Moss, J. E. B., Moss, B., Phansalkar, A., Stefanovi, D., VanDrunen, T., von Dincklage, D., and Wiedermann, B. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA ’06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications (New York, NY, USA, 2006), ACM, pp. 169--190.
5. }}Boehm, B., Boehm, B., Clark, B., Horowitz, E., Westl, C., Madachy, R., and Selby, R. Cost models for future software life cycle processes:. In Annals of Software Engineering (1995), pp. 57--94.
6. }}Bortz, J., and Döring, N. Forschungsmethoden und Evaluation: für Human- und Sozialwissenschaftler, 4. ed. Springer, Heidelberg, 2006.
7. }}Denning, P. J. Is computer science science? Commun. ACM 48, 4 (2005), 27--31.
8. }}Dijkstra, E. W. The humble programmer. Commun. ACM 15, 10 (1972), 859--866.
9. }}Dijkstra, E. W. A Discipline of Programming. Prentice Hall, Inc., October 1976.
10. }}Easterbrook, S. M., Singer, J., Storey, M., and Damian, D. Selecting empirical methods for software engineering research. In Guide to Advanced Empirical Software Engineering, F. Shull, J. Singer, and D. Sjøberg, Eds. Springer, 2007.
11. }}Endres, A., and Rombach, D. A Handbook of Software and Systems Engineering. Pearson Addison-Wesley, 2003.
12. }}Fenton, N. E., and Pfleeger, S. L. Software Metrics: A Rigorous and Practical Approach. PWS Publishing Co., Boston, MA, USA, 1998.
13. }}Filman, R. E., Elrad, T., Clarke, S., and Ak\c sit, M., Eds. Aspect-Oriented Software Development. Addison-Wesley, Boston, 2005.
14. }}Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47, 6 (June 1954), 262--269.
15. }}Génova, G. Is computer science truly scientific? Commun. ACM 53, 7 (2010), 37--39.
16. }}Halstead, M. H. Elements of Software Science (Operating and programming systems series). Elsevier Science Inc., New York, NY, USA, 1977.\\
17. }}Hick, W. E. On the rate of gain of information. The Quarterly Journal of Experimental Psychology 4, 1 (1952), 11--26.
18. }}Johnson, R. E. Documenting frameworks using patterns. In OOPSLA ’92: Conference proceedings on Object-oriented programming systems, languages, and applications (New York, NY, USA, 1992), ACM, pp. 63--76.
19. }}Juristo, N., and Moreno, A. M. Basics of Software Engineering Experimentation. Springer, 2001.
20. }}Kitchenham, B., Al-Khilidar, H., Babar, M. A., Berry, M., Cox, K., Keung, J., Kurniawati, F., Staples, M., Zhang, H., and Zhu, L. Evaluating guidelines for empirical software engineering studies. In ISESE ’06: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering (New York, NY, USA, 2006), ACM, pp. 38--47.
21. }}Madsen, O. L., and Møller-Pedersen, B. What object-oriented programming may be - and what it does not have to be. In Proceedings of European Conference on Object-Oriented Programming (ECOOP’88) (1988), Springer, pp. 1--20.
22. }}National Research Council. Computer Science: Reflections on the Field, Reflections from the Field. National Academy Press, 2004.
23. }}Pfleeger, S. L. Soup or art? the role of evidential force in empirical software engineering. IEEE Software 22 (2005), 66--73.
24. }}Popper, K. In Logik der Forschung (2007), H. Keuth, Ed., Akademie Verlag GmbH.
25. }}Prechelt, L. Kontrollierte Experimente in der Softwaretechnik. Springer, Berlin, March 2001.
26. }}Prechelt, L., Unger, B., and Tichy, W. Two controlled experiments assessing the usefulness of design pattern documentation in program maintenance. IEEE Transactions on Software Engineering 28 (2002), 595--606.
27. }}Sheil, B. A. The psychological study of programming. ACM Comput. Surv. 13, 1 (1981), 101--120.
28. }}Shneiderman, B. Software Psychology: Human Factors in Computer and Information Systems. Winthrop Publishers, August 1980.
29. }}Shneiderman, B., and Plaisant, C. Designing the User Interface: Strategies for Effective Human-Computer Interaction, 5. ed. Pearson Addison-Wesley, Upper Saddle River, NJ, 2009.
30. }}Sjøberg, D. I. K., Dyba, T., and Jørgensen, M. The future of empirical methods in software engineering research. In FOSE ’07: 2007 Future of Software Engineering (Washington, DC, USA, 2007), IEEE Computer Society, pp. 358--378.
31. }}Sjøberg, D. I. K., Hannay, J. E., Hansen, O., By Kampenes, V., Karahasanović, A., Liborg, N.-K., and C. Rekdal, A. A survey of controlled experiments in software engineering. IEEE Trans. Softw. Eng. 31, 9 (2005), 733--753.
32. }}Snelting, G. Paul Feyerabend und die Softwaretechnologie. Informatik-Spektrum 21, 5 (October 1998), 273--276.
33. }}Snelting, G. Feyerabend - zwei Jahre sp\"ater. Softwaretechnik-Trends 21, 1 (February 2001), 40--43.
34. }}Sommerville, I. Software Engineering, 9. ed. Addison-Wesley, Harlow, England, 2010.
35. }}Tichy, W. F. Die Bedeutung der Empirie für die Softwaretechnik. Keynote at German Conference on Software Engineering, Essen, March, 8--11, 2005.
36. }}Tichy, W. F. Should computer scientists experiment more? IEEE Computer 31 (1998), 32--40.
37. }}Tichy, W. F., Lukowicz, P., Prechelt, L., and Heinz, E. A. Experimental evaluation in computer science: A quantitative study. Journal of Systems and Software 28, 1 (1995), 9--18.
38. }}Ungar, D., and Smith, R. B. Self: The power of simplicity. In OOPSLA ’87: Conference proceedings on Object-oriented Programming Systems, Languages, and Applications (December 1987), ACM, pp. 227--242.
39. }}Weinberg, G. M. The Psychology of Computer Programming. John Wiley & Sons, Inc., New York, NY, USA, 1985.
40. }}Wilkinson, L., and the Task Force on Statistical Inference. Statistical methods in psychology journals: Guidelines and explanations. American Psychologist 54 (1999), 594--604.
41. }}Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., and Wesslén, A. Experimentation in software engineering: an introduction. Kluwer Academic Publishers, Norwell, MA, USA, 2000.
42. }}Zelkowitz, M. V., and Wallace, D. R. Experimental models for validating technology. Computer 31 (1998), 23--31.

Index Terms

• Software and its engineering

  • Software notations and tools

    • General programming languages