Work experience versus refactoring to design patterns: a controlled experiment

ABSTRACT

Program refactoring using design patterns is an attractive approach for facilitating anticipated changes. Its benefit depends on at least two factors, namely the effort involved in the refactoring and how effective it is. For example, the benefit would be small if too much effort is required to translate a program correctly into a refactorized form, and whether such a form could effectively guide maintainers to complete anticipated changes is unknown. A metric of effectiveness is the maintainers' performance, which can be affected by their work experience, in realizing the changes. Hence, an interesting question arises. Is program refactoring to introduce additional patterns beneficial regardless of the work experience of the maintainers? In this paper, we report a controlled experiment on maintaining JHotDraw, an open source system deployed with multiple patterns. We compared maintainers with and without work experience. Our empirical results show that, to complete a maintenance task of perfective nature, the time spent even by the inexperienced maintainers on a refactorized version is much shorter than that of the experienced subjects on the original version. Moreover, the quality of their delivered programs, in terms of correctness, is found to be comparable.

References

1. E.L.A. Baniassad, G.C. Murphy, and C. Schwanninger, "Design Pattern Rational Graphs: Linking Design to Source", in Proceedings of the 25th International Conference on Software Engineering (ICSE 2003), IEEE Computer Society Press, Portland, Oregon, USA, Mar. 2003, pp. 352--362.
2. L. Bass, P. Clements, and R. Kazman, Software Architecture in Practice. Addison Wesley, 1997.
3. K. Beck, R. Crocker, G. Meszaros, J. Vlissides, J.O. Coplien, L. Dominick, and F. Paulisch, "Industrial Experience with Design Patterns", in Proceedings of the 18th International Conference on Software Engineering (ICSE 1996), IEEE Computer Society Press, Berlin, Germany, Mar. 1996, pp. 103--114.
4. S. Belmonte, J.G. Consuegra, J.L. Gavilan, and F.J. Honrubia, "Development and Maintenance of a GIS Family Products: A Case Study", Next Generation Geospatial Information - 2003 (NG2I 2003), Cambridge, Massachusetts, USA, Oct. 2003.
5. K.H. Bennett, "Software Evolution: Past, Present and Future", Information and Software Technology, 39(11):673--680, 1996.
6. J.M. Bieman, D. Jain and H.J. Yang. OO Design Patterns, "Design Structure, and Program Changes: An Industrial Case Study", in Proceedings of International Conference on Software Maintenance (ICSM 2001), IEEE Computer Society Press, Florence, Italy, Nov. 2001, pp. 580--589.
7. J.M. Bieman, G. Straw, H. Wang, P.W. Munger and R.T. Alexander, "Design Patterns and Change Proneness: An Examination of Five Evolving Systems", in Proceedings of the 9th International Software Metrics Symposium (METRIC 2003), IEEE Computer Society Press, Sydney, Australia, Sep. 2003, pp. 40--49.
8. F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad and M. Stal, Pattern-Oriented Software Architecture: A System of Patterns. Wiley, 1996.
9. M.P. Cline, "The Pros and Cons of Adopting and Applying Design Patterns in the Real World", Communications of the ACM, 39(10):47--49, 1996.
10. J. Cohen, Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, 1988.
11. M. Fowler, Refactoring: Improving the Design of Existing Code. Addison Wesley, 1999.
12. E. Gamma, R. Helm, R. Johnson and J. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software. Addison Wesley, 1995.
13. J2EE EJB, http://java.sun.com/products/ejb/. (Last accessed: 3 Apr 2006)
14. JHotDraw, http://sourceforge.net/projects/jhotdraw/. (Last accessed: 3 Apr 2006)
15. JUnit, http://junit.sourceforge.net/. (Last accessed: 3 Apr 2006)
16. J. Kerievsky, Refactoring to Patterns. Addison Wesley, 2005.
17. B.A. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K.E. Emam, and J. Rosenberg, "Preliminary Guidelines for Empirical Research in Software Engineering", IEEE Transactions on Software Engineering, 28(8):721--734, 2002.
18. B.P. Lientz, E.B. Swanson, G.E. Tompkins, "Characteristics of Application Software Maintenance", Communications of the ACM, 21(6):466--471, 1978.
19. Mircosoft COM, http://www.microsoft.com/com/. (Last accessed: 3 Apr 2006)
20. L. Prechelt, B. Unger, M. Philippsen, and W.F. Tichy, "Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance", IEEE Transactions on Software Engineering, 28(6):595--606, 2002.
21. L. Prechelt, B. Unger, W.F. Tichy, P. Brössler, and L.G. Votta. A Controlled Experiment in MainTenance Comparing Design Patterns to Simpler Solutions. IEEE Transactions on Software Engineering, 27(12):1134--1144, 2001.
22. J. Rajesh, and D. Janakiram, "JIAD: A Tool to Inter Design Patterns in Refactoring", in Proceedings of the 6th ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming (PPDP 2004), ACM Press, Verona, Italy, 2004, pp. 227--237.
23. M.P. Robillard, "Automatic Generation of Suggestion for Program Investigation", in Proceedings of the 10th European Software Engineering Conference jointly with the 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-13), ACM Press, Lisbon, Portugal, Sep. 2005, pp. 11--20.
24. M.P. Robillard, W. Coelho, and G.C. Murphy, "How Effective Developers Investigate Source Code: An Exploratory Study", IEEE Transactions on Software Engineering, 30(12):889--903, 2004.
25. Source Forge, http://sourceforge.net/. (Last accessed: 3 Apr 2006)
26. T.H. Ng, and S.C. Cheung, "Enhancing Class Commutability in the Deployment of Design Patterns", Information and Software Technology, 47(12):797--804, 2005.
27. T.H. Ng, and S.C. Cheung, "Proactive Views on Concrete Aspects: A Pattern Documentation Approach for Software Evolution", in Proceedings of the 27th International Conference on Computer Software and Applications (COMPSAC 2003), IEEE Computer Society Press, Dallas, Texas, USA, Nov. 2003, pp. 242--247.
28. M. Vokáč, "Defect Frequency and Design Patterns: An Empirical Study of Industrial Code", IEEE Transactions on Software Engineering, 30(12):904--917, 2004.
29. M. Vokáč, W. Tichy, D.I.K. Sjøberg, E. Arisholm, and M. Aldrin, "A Controlled Experiment Comparing the Maintainability of Programs Designed With And Without Design Patterns: A Replication In A Real Programming Environment", Empirical Software Engineering 9(3):149--195, 2004.

Index Terms

• Software and its engineering

  • Software creation and management

    • Software development techniques

      • Object oriented development

  • Software notations and tools

    • Development frameworks and environments

      • Object oriented frameworks

    • General programming languages

      • Language features

        • Patterns