A view of 20th and 21st century software engineering

ABSTRACT

George Santayana's statement, "Those who cannot remember the past are condemned to repeat it," is only half true. The past also includes successful histories. If you haven't been made aware of them, you're often condemned not to repeat their successes.In a rapidly expanding field such as software engineering, this happens a lot. Extensive studies of many software projects such as the Standish Reports offer convincing evidence that many projects fail to repeat past successes.This paper tries to identify at least some of the major past software experiences that were well worth repeating, and some that were not. It also tries to identify underlying phenomena influencing the evolution of software engineering practices that have at least helped the author appreciate how our field has gotten to where it has been and where it is.A counterpart Santayana-like statement about the past and future might say, "In an era of rapid change, those who repeat the past are condemned to a bleak future." (Think about the dinosaurs, and think carefully about software engineering maturity models that emphasize repeatability.)This paper also tries to identify some of the major sources of change that will affect software engineering practices in the next couple of decades, and identifies some strategies for assessing and adapting to these sources of change. It also makes some first steps towards distinguishing relatively timeless software engineering principles that are risky not to repeat, and conditions of change under which aging practices will become increasingly risky to repeat.

References

1. ISO. Standard for Information Technology - Software Life Cycle Processes. ISO/IEC 12207, 1995.]]
2. ISO. Systems Engineering - System Life Cycle Processes. ISO/IEC 15288, 2002.]]
3. FCIO (Federal CIO Council), A Practical Guide to Federal Enterprise Architecture, Version 1.0, FCIO, Washington, DC, February 2001. zaqi4.]]
4. Alford, M.W., Software Requirements Engineering Methodology (SREM) at the Age of Four, COMPSAC 80 Proceedings, (October 1980) pp 366--374.]]
5. Anthes, G., The Future of IT. Computerworld, (March 7, 2005) 27--36.]]
6. Aron, J. Software Engineering, NATO Science Committee Report, January 1969.]]
7. Arthur, W. B., Increasing Returns and the New World of Business. Harvard Business Review (July/August, 1996) 100--109.]]
8. Baker, F. T. Structured programming in a production programming environment. In Proceedings of the international Conference on Reliable Software. Los Angeles, California, April 21-23, 1975.]]
9. Balzer, R.M. A Global View Of Automatic Programming, Proc. 3rd Joint Conference On Artificial Intelligence, August, 1973, pp. 494--499.]]
10. Balzer, R. M., T. E. Cheatham, and C. Green, Software Technology in the 1990's: Using a New Paradigm, Computer, Nov. 1983, pp. 3945.]]
11. Basili V. R. and M. V. Zelkowitz, Analyzing medium scale software development, Third International Conf. On Software Engineering, Atlanta, Ga. (May, 1978) 116--123.]]
12. Bass, L., Clements, P., and Kazman, R. Software Architecture in Practice, Addison-Wesley, 1998.]]
13. Bass, L. and Coutaz, J., Developing Software for the User Interface, Addison Wesley, 1991.]]
14. Beck, K. Extreme Programming Explained, Addison-Wesley, 2000.]]
15. Belady, L. A. and Lehman, M. M., Characteristics of large systems, in Research Directions in Software Technology, P.Wegner (editor), MIT-Press, Cambridge, Massachusetts, 1979.]]
16. Benington, H. D., Production of Large Computer Programs, Proceedings of the ONR Symposium on Advanced Program Methods for Digital Computers, June 1956, pp. 15--27. Also available in Annals of the History of Computing, October 1983, pp. 350--361.]]
17. Berners-Lee, T., World Wide Web Seminar. http://www.w3.org/Talks/General.html (1991).]]
18. Berners-Lee, T., Cailliau, R., Luotonen, A., Frystyk, H., Nielsen, F., and Secret, A.. The World-Wide Web, in Comm. ACM (CACM), 37(8), 76--82, 1994.]]
19. Best Current Practices: Software Architecture Validation, AT&T Bell Labs, 1990.]]
20. Biffl, S., Aurum, A., Boehm, B., Erdogmus, H., Gruenbacher, P. (eds.). Value-Based Software Engineering. Springer Verlag (2005).]]
21. Bjorner, D., On the use of formal methods in software development. In Proceedings of the 9th International Conference on Software Engineering (Monterey, California, United States), 1987.]]
22. Boehm, B., Software and its impact: a quantitative assessment. Datamation, pages 48--59, May 1973.]]
23. Boehm, B. W., J. R. Brown, H. Kaspar, M. Lipow, G. J. MacLeod, M. J. Merritt, Characteristics of Software Quality, TRW Software Series TRW-SS-73-09, December 1973.]]
24. Boehm, B., Software engineering. IEEE Trans. Computers, 100(25):1226--1241, 1976.]]
25. Boehm, B., Software Engineering: As it is. ICSE 1979: 11--21.]]
26. Boehm, B., Software Engineering Economics, Prentice-Hall 1981.]]
27. Boehm, B., Standish, T. A. Software Technology in the 1990's: Using an Evolutionary Paradigm. IEEE Computer 16(11): 30--37 (1983).]]
28. Boehm, B., A Spiral Model of Software Development and Enhancement, Computer, May 1988, pp. 61--72.]]
29. Boehm B., An Early Application generator and Other Recollections, in Glass, R. L., In The Beginning: Recollections of Software Pioneers, IEEE CS Press, 1988.]]
30. Boehm, B., Software Risk Management, CS Press, Los Alamitos, Calif., 1989.]]
31. Boehm, B. and Ross, R., Theory-W Software Project Management: Principles and Examples, IEEE Trans. SW Engineering., July 1989, pp. 902--916.]]
32. Boehm, B., A Spiral Model for Software Development and Enhancement, Computer, vol. 21, May 1988, pp. 61--72.]]
33. Boehm, B., Anchoring the Software Process, IEEE Software, Vol. 13, No. 14, July 1996.]]
34. Boehm, B., Managing Software Productivity and Reuse. Computer 32, 9 (Sep. 1999), 111--113.]]
35. Boehm, B., Brown, A.W, Basili, V. and Turner R., Spiral Acquisition of Software-Intensive Systems. CrossTalk, May 2004, pp. 4--9.]]
36. Boehm, B., Turner, R., Balancing Agility and Discipline, Addison Wesley (2004).]]
37. Boehm, B., Jain, A. An Initial Theory of Value-Based Software Engineering. In: Aurum, A., Biffl, S., Boehm, B., Er-dogmus, H., Gruenbacher, P. (eds.): Value-Based Software Engineering, Springer Verlag (2005).]]
38. Boehm, B., Some Future Trends and Implications for Systems and Software Engineering Processes, Systems Engineering, vol. 9, No. 1, 2006, pp 1--19.]]
39. Boehm, B. and Lane, J., 21st Century Processes for Acquiring 21st Century Software Intensive Systems of Systems, Cross Talk, May 2006 (to appear).]]
40. Böhm, C. and Jacopini, G. Flow diagrams, turing machines and languages with only two formation rules. Comm. ACM 9, 5 (May. 1966), 366--371.]]
41. Booch, G., Rumbaugh, J. and Jacobson, L. The Unified Modeling Language User Guide. Addison-Wesley Longman Inc., 1999.]]
42. Brooks, F. P., The Mythical Man-Month, Addison Wesley, 1975.]]
43. Brooks, F. P., No silver bullet: Essence and accidents of software engineering. IEEE Computer, 20(4):10--19, April 1987.]]
44. Buxton J. N. and Randell B. (Eds.) Software Engineering Techniques: Report on a Conference Sponsored by the NATO Science Committee, Rome, Italy, 27--31 Oct. 1969. Scientific Affairs Division, NATO, Brussels (May 1970).]]
45. PDL/74 Program Design Language Reference Guide (Processor Version 3), Caine Farber Gordon Inc., 1977.]]
46. Charette, R. N., Software Engineering Risk Analysis and Management, McGraw-Hill, 1989.]]
47. Clemm, G., The Workshop System - A Practical Knowledge-Based Software Environment, Proceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, ACM SIGPLAN Notices, 13, 5, pp. 55--64 (Nov 1988).]]
48. Constantine, L. L. The Programming Profession, Programming Theory, and Programming Education. Computers and Automation 17,2 (Feb. 1968) pp. 14--19.]]
49. Crawford, D. Editorial Pointers. Comm. ACM (October, 2001) 5.]]
50. Cooper, J. D., Characteristics of the Average Coder, personal communication, May 1975.]]
51. Conrow, E. H., Risk Management for Systems of Systems, CrossTalk, v. 18, n. 2 (February 2005), pages 8--12.]]
52. Cusumano, M. A. The Software Factory: A Historical Interpretation. IEEE Softw. 6, 2 (Mar. 1989), 23--30.]]
53. Cusumano, M., Selby, R.: Microsoft's Secrets. HarperCollins (1996).]]
54. Dahl, O., and Nygaard, K. Simula: a language for programming and description of discrete event systems. Tech. rep., Norwegian Computing Center, 1967.]]
55. DeMarco, T., Structured analysis and system specification. Prentice Hall, 1978.]]
56. Dijkstra, E. Cooperating Sequential Processes. Academic Press, 1968.]]
57. Drexler, E.K.: Engines of Creation. Anchor Press (1986)]]
58. Drexler, K.E., Peterson, C., Pergamit, G.: Unbounding the Future: The Nanotechnology Revolution. William Morrow & Co. (1991).]]
59. Dreyfus, H., Dreyfus, S.: Mind over Machine. Macmillan (1986).]]
60. Druffel, L.E.. and Buxton, J.N. Requirements for an Ada programming support environment: Rationale for STONEMAN. In Proceedings of COMPSAC 80. IEEE Computer Society, (Oct. 1980), 66--72.]]
61. Dyson, G. B.: Darwin Among the Machines: The Evolution of Global Intelligence, Helix Books/Addison Wesley (1997).]]
62. Ehn, P. (ed.): Work-Oriented Design of Computer Artifacts, Lawrence Earlbaum Assoc. (1990).]]
63. Elssamadisy, A. and Schalliol, G., Recognizing and Responding to 'Bad Smells' in Extreme Programming, Proceedings, ICSE 2002, pp. 617--622.]]
64. Estublier, J., Leblang, D., Hoek, A., Conradi, R., Clemm, G., Tichy, W., and Wiborg-Weber, D. Impact of software engineering research on the practice of software configuration management. ACM Trans. Softw. Eng. Methodol. 14, 4 (Oct. 2005), 383--430.]]
65. Fagan, M.E., Design and Code inspections to reduce errors in program development, 1976, IBM Systems Journal, Vol. 15, No 3, Page 258--287.]]
66. Friedman, T. L., The World Is Flat: A Brief History of the Twenty-First Century. Farrar, Straus & Giroux. New York. 2005.]]
67. Gamma, E., Helm, R., Johnson, R., and Vlissides, J., Design Patterns: Elements of Reusable Object Oriented Software. Addison-Wesley, Reading, MA. (1994).]]
68. Grant, E., and H. Sackman., An Exploratory Investigation of Programmer Performance Under On-Lineand Off-Line Conditions. Report SP-2581, System Development Corp., September 1966.]]
69. Gosling, J., Joy, B., Steele, G., The Java Language Specification, Sun Microsystems, Inc. (1989).]]
70. Hoare, C. A. R., An axiomatic basis for computer programming. Comm. ACM, 12:576--583, 1969.]]
71. Hoare, C. A. R., Assertions: A Personal Perspective, IEEE Annals of the History of Computing, v.25 n.2, p.14--25, April 2003.]]
72. Floyd, C. Records and References in Algol-W, Computer Science Department, Stanford University, Stanford, California, 1969.]]
73. Freiman, F. R. and Park, R. E., The PRICE Software Cost Model, RCA Price Systems, Cherry Hill, NJ, Feb. 1979.]]
74. Highsmith, J. Adaptive Software Development. Dorset House (2000)]]
75. Guttag, J. V., The Specification and Application to Programming of Abstract Data Types, Ph.D. dissertation, Computer Science Department, University of Toronto, Canada, 1975.]]
76. Halstead, M., Elements of Software Science, North Holland, 1977.]]
77. Harned, D., Lundquist, J. "What Transformation Means for the Defense Industry". The McKinsey Quarterly, November 3, 2003: 57--63.]]
78. Holtzmann, G, The SPIN Model Checker, Addison Wesley, 2004.]]
79. Hofstede, G., Culture and Organizations. McGraw Hill (1997).]]
80. Hosier, W. A., Pitfalls and Safeguards in Real-Time Digital Systems with Emphasis on Programming, IRE Trans. Engineering Management, EM-8, June, 1961.]]
81. Humphrey, W. S., Managing the Software Process. Reading, MA: Addison-Wesley, 1989.]]
82. Jackson M. A., Principle of Program Design, Acad. Press, 1975.]]
83. Jacobson, I., Griss, M., Jonsson, P., Software Reuse: Architecture, Process and Organization for Business Success, Addison Wesley, 1997.]]
84. Jacobson, I., Booch, G. and Rumbaugh, J., Unified Software Development Process, Addison-Wesley, 1999.]]
85. Joy, B.: Why the Future Doesn't Need Us: Wired (April, 2000).]]
86. Kurzweil, R., The Age of Spiritual Machines. Penguin Books (1999).]]
87. Kruchten, P. B., The Rational Unified Process (An Introduction). Addison Wesley 1999.]]
88. Lane, J., Valerdi, R. "Synthesizing System-of-Systems Concepts for Use in Cost Estimation," IEEE SMC, 2005.]]
89. Levy, S., Hackers: Heroes of the Computer Revolution, Anchor Press/Doubleday, 1984.]]
90. Lientz, B.P. and E.B. Swanson, Software Maintenance Management, Addison-Wesley, Reading, Mass., 1980.]]
91. Lim, W.C., Managing Software Reuse, Prentice Hall, 1998.]]
92. Liskov, B. and Zilles, S. N., Programming with abstract data types. In Proceedings of Symposium on Very High Level Programming Languages, 1974.]]
93. Littlewood, B. and Verrall, J. L., A Bayesian Reliability Growth Model for Computer Software, Appl. Statist., Vol. 22, pp. 332--346 (1973).]]
94. Maier, M., Architecting Principles for Systems of Systems, Proceedings of the Sixth Annual International Symposium, International Council on Systems Engineering, Boston, 1996, pages 567--574.]]
95. Marca, D. and Bock, G. eds., Groupware, IEEE CS Press, 1992.]]
96. Maranzano, J. F., Sandra A. Rozsypal, Gus H. Zimmerman, Guy W. Warnken, Patricia E. Wirth, David M. Weiss, Architecture Reviews: Practice and Experience, IEEE Software, vol. 22, no. 2, pp. 34--43, Mar/Apr, 2005.]]
97. Martin, J., Rapid Applications Development, Macmillan, 1991.]]
98. Maslow, A., Motivation and Personality, Harper and Row (1954).]]
99. Matsumoto, Y., Evaluation of the digital prediction filter applied to control a class of sevomotor by microcomputers, IEEE Transactions on IECI, Vol. IECI-23, No.4, pp.359--363(Nov. 1976).]]
100. McCabe, T., A Software Complexity Measure, IEEE Trans. Software Engineering Vol 2, No 12, 1976.]]
101. McCracken, D.D. A maverick approach to systems analysis and design. In System Analysis and Design: A Foundation for the 1980's, W.W. Cotterman, J.D. Gouger, N.L. Enger, and F. Harold, Eds. North-Holland, Amsterdam, 1981, pp. 446--451.]]
102. McConnell, S. Rapid Development. Microsoft Press, 1996.]]
103. Meller, S. J., Clark, A. N., and Futagami, T., Model-Driven Development, IEEE Software, (Special Issue on Model-Driven Development) Vol. 20, No. 5, 2003, 14--18.]]
104. Meyer, B. Object-Oriented Software Construction, Second Edition. Prentice Hall, 1997.]]
105. Myers, G., Reliable Systems through Composite Design. New York: Petrocelli/Charter, 1975.]]
106. National Institute of Standards and Technology, Gaithersberg. Reference Model for Frameworks of Software Engineering Environments, draft version 1.5 edition, 1991.]]
107. Naur, P. and Randell, B. (Eds.). Software Engineering: Report of a conference sponsored by the NATO Science Committee (7-11 Oct. 1968), Garmisch, Germany. Brussels, Scientific Affairs Division, NATO, 1969.]]
108. Neighbors, J.M., The Draco Approach to constructing Software from reusable components, IEEE Trans. Software Engineering, vol.SE-10, No.5, pp.564--574, September, 1984.]]
109. Nelson, E., Management Handbook for the Estimation of Computer Programming Costs, Systems Development Corporation, Oct. 1966.]]
110. Nunamaker, J. et. al., "Lessons from a Dozen years of Group Support Systems Research: A Discussion of lab and Field Findings," J.MIS, vol. B, no.3 (1996-1997), pp. 163--207.]]
111. Osterweil, L., Software Processes are Software Too. Proceedings, In Ninth International Conference on Software Engineering, (Monterey, CA, 1987), ACM, IEEE, LosAlamitos, CA, 2--13.]]
112. Osterweil, L., Ghezzi, C., Kramer, J., and Wolf, A. 2005. Editorial. ACM Trans. Softw. Eng. Methodol. 14, 4 (Oct. 2005), 381--382.]]
113. Paulk, M., Weber, C., Curtis, B., Chrissis, M., The Capability Maturity Model. Addison Wesley (1994).]]
114. Paulk, M. C., Weber, C. V., Curtis B, Chrissis M. B. et al., The Capability Maturity Model: Guidelines for Improving the Software Process. Addison Wesley, 1995.]]
115. Parnas, D.L. On the Criteria To Be Used in Decomposing Systems Into Modules, Comm. ACM, Vol. 5, No. 12, December 1972, pp. 1053--1058.]]
116. Parnas, D. L. On the design and development of program families, IEEE Trans. Software Engineering 2(1), 1--9, 1976.]]
117. Parnas, D. L. Designing software for ease of extension and contraction, IEEE Trans. Software Engineering SE-5(2), 128--138, 1979.]]
118. Patterson, D.: 20th Century vs. 21st Century C&C: The SPUR Manifesto. ACM Comm. (March, 2005) 15--16.]]
119. Penedo, M. H. and Stuckle, E. D., PMDB--A Project Master Database for Software Engineering Environments. In Proceedings of the 8th International Conference on Software Engineering, pages 150--157. IEEE Computer Society, August 1985.]]
120. Perry, D. E. and Wolf, A. L., Foundations for the study of software architecture. Software Engineering Notes, vol 17, no 4, October 1992]]
121. Phongpaibul, M., Boehm, B.: Improving Quality Through Software Process Improvement in Thailand: Initial Analysis. Proceedings, ICSE 2005 Workshop on Software Quality (May, 2005).]]
122. PITAC (President's Information Technology Advisory Committee), Report to the President: Information Technology Research: Investing in Our Future (1999).]]
123. Poore, J. H. and Carmen J. Trammell, Bringing Respect to Testing Through Statistical Science, American Programmer, Vol. 10, No. 8, August, 1997.]]
124. Poulin, J.S., Measuring Software Reuse, Addison Wesley, 1997.]]
125. Putnam, L. H., A General Empirical Solution to the Macro Software Sizing and Estimating Problem, IEEE Trans. Software Engineering, Vol 4, No 4, 1978.]]
126. Putman, J. Architecting with RM-ODP. Prentice Hall, 2001.]]
127. Raymond, E.S, The Cathedral and the Bazaar, O'Reilly, 1999.]]
128. Reifer, D.J., Practical Software Reuse, Wiley, 1997.]]
129. Redwine, S. and Riddle, W., Software Technology maturation, Proceedings of the 8th ICSE, 1985.]]
130. Ross, D. Structured Analysis (SA): A Language for Communicating Ideas. IEEE Trans. Software Engineering, SE-3, 1 (Jan. 1977). 16--34.]]
131. Royce, W. W., Managing the Development of Large Software Systems: Concepts and Techniques, Proceedings of WESCON, August 1970.]]
132. Royce, W., Software Project Management - A Unified Framework, Addison Wesley 1998.]]
133. Ryder, B. G., Soffa, M. L., and Burnett, M. 2005. The impact of software engineering research on modern progamming languages. ACM Trans. Softw. Eng. Methodol. 14, 4 (Oct. 2005).]]
134. Sage, A., Cuppan, C., On the Systems Engineering and Management of Systems of Systems and Federations of Systems, Information, Knowledge, and Systems Management, v. 2 (2001), pages 325--345.]]
135. Schmidt, D. C., Model-Driven Engineering, IEEE Computer, 39(2), February 2006.]]
136. Shaw, M. and Garlan, D., Software Architecture. Perspectives on an emerging discipline, Prentice Hall, Upper Saddle River, NJ 07458, (1996).]]
137. Shneiderman, B., Software psychology: human factors in computer and information systems. 1980: Cambridge, Mass, Winthrop Publishers. 320.]]
138. Shooman, M.L., Probabilistic Models for Software Reliability Prediction. Statistical Computer Performance Evaluation, Academic Press, New York, pp. 485--502 (1972).]]
139. Stallman, R. M., Free Software Free Society: Selected Essays of Richard M. Stallman, GNU Press, 2002.]]
140. Standish Group, Extreme Chaos, http://www.standishgroup.com (2001).]]
141. Stroustrup, B., The C++ Programming Language. Addison-Wesley, 1986.]]
142. Teichroew, D. and Sayani, H. Automation of system building. Datamation 17,16 (August 15, 1971), 25--30.]]
143. The Open Road: A History of Free Software, http://www.sdmagazine.com/opensourcetimeline/sources.htm]]
144. Toulmin, S., Cosmopolis, University of Chicago Press (1992).]]
145. Tracz, W., Test and analysis of Software Architectures, In Proceedings of the international Symposium on software Testing and Analysis (ISSTA '96), ACM press, New York, NY, pp 1--3.]]
146. Webster's Collegiate Dictionary, Merriam-Webster, 2002.]]
147. Weinberg, G.M., The Psychology of Computer Programming, New York: Van Nostrand Reinhold (1971).]]
148. Wirth, N. The programming language Pascal. Acta Infformatica, 1:35--63, 1971.]]
149. Wirth, N. Programming in Modula-2. Springer, 1974.]]
150. Wulf, W. A., London, R., and Shaw, M., An Introduction to the Construction and Verification of Alphard Program. IEEE Trans. Software Engineering, SE-2, 1976, pp. 253--264.]]
151. Yang, Y., Bhuta, J., Port, D., and Boehm, B.: Value-Based Processes for COTS-Based Applications. IEEE Software (2005).]]
152. Yang, Y., Boehm, B., and Port, D. A Contextualized Study of COTS-Based E-Service Projects. Proceedings, Software Process Workshop 2005 (May, 2005).]]
153. Yourdon E. and Constantine, L. L., Structured Design. Yourdon Press, 1975.]]
154. Zachman, J. "A Framework for Information Systems Architecture." IBM Systems Journal, 1987.]]

Index Terms

• Social and professional topics

  • Professional topics

    • Management of computing and information systems

      • Implementation management

        • Pricing and resource allocation

      • Project and people management

• Software and its engineering

  • Software creation and management

  • Software notations and tools

    • Software configuration management and version control systems