An empirical investigation of changes in some software properties over time

ABSTRACT

Software metrics are easy to define, but not so easy to justify. It is hard to prove that a metric is valid, i.e., that measured numerical values imply anything on the vaguely defined, yet crucial software properties such as complexity and maintainability.

This paper employs statistical analysis and tests to check some plausible assumptions on the behavior of software and metrics measured for this software in retrospective on its versions evolution history. Among those are the reliability assumption implicit in the application of any code metric, and the assumption that the magnitude of change, i.e., increase or decrease of its size, in a software artifact is correlated with changes to its version number.

Putting a suite of 36 metrics to the trial, we confirm most of the assumptions on a large repository of software artifacts. Surprisingly, we show that a substantial portion of the reliability of some metrics can be observed even in random changes to architecture. Another surprising result is that Boolean-valued metrics tend to flip their values more often in minor software version increments than in major increments.

References

1. E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design patterns: elements of reusable object-oriented software. Addison-Wesley Professional, 1995.
2. S. R. Chidamber and C. F. Kemerer, "A metrics suite for object oriented design," IEEE Transactions on Software Engineering., vol. 20, no. 6, pp. 476--493, 1994.
3. M. J. Ordoñez and H. M. Haddad, "The state of metrics in software industry," in Proc. of the Fifth International Conference on Information Technology: New Generations, (ITNG 2008). Las Vegas, Nevada, USA: IEEE Computer Society, Apr. 7--8 2008, pp. 453--458.
4. G. Lajios, "Software metrics suites for project landscapes," in Proc. of the 2009 European Conference on Software Maintenance and Reengineering., vol. 0. Los Alamitos, CA, USA: IEEE Computer Society, 2009, pp. 317--318.
5. M. Lorenz and J. Kidd, Object-oriented software metrics: a practical guide. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 1994.
6. B. Henderson-Sellers, Object-oriented metrics: measures of complexity. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 1996.
7. N. E. Fenton, Software Metrics: A Rigorous and Practical Approach, 2nd ed. Boston, MA, USA: International Thomson Computer Press, 1996.
8. B. Neate, W. Irwin, and N. Churcher, "Coderank: A new family of software metrics." in Proc. of the 17th Australian Software Engineering Conference, (ASWEC 2006). Sydney, Australia: IEEE Computer Society, 2006, pp. 369--378.
9. R. E. Tarjan, "Depth-first search and linear graph algorithms," SIAM J. Comput., vol. 1, no. 2, pp. 146--160, 1972.
10. A. V. Aho, J. E. Hopcroft, and J. D. Ullman, The Design and Analysis of Computer Algorithms. Reading, Mass.: Addison-Wesley, 1974.
11. R. Sedgewick, Algorithms. Addison-Wesley, 1983.
12. M. G. Kendall and J. D. Gibbons, Rank Correlation Methods, 5th ed. Oxford University Press, New York, 1990.
13. P. S. Almeida, "Balloon types: Controlling sharing of state in data types," in Proc. of the Eleventh European Conference on Object-Oriented Programming, (ECOOP'97). Jyväskylä, Finland: Springer-Verlag, Jun. 9--13 1997, pp. 32--59.
14. J. Y. Gil and I. Maman, "Micro patterns in java code," SIGPLAN Not., vol. 40, no. 10, pp. 97--116, 2005.
15. M. Page-Jones and L. L. Constantine, Fundamentals of object-oriented design in UML. Boston, MA, USA: Addison-Wesley, 2000.
16. U. Brandes, "A faster algorithm for betweenness centrality," Journal of Mathematical Sociology, vol. 25, pp. 163--177, 2001.