Walid Maalej
Rebecca Tiarks
Tobias Roehm
Skip Abstract Section
Abstract
Research in program comprehension has evolved considerably over the past decades. However, only little is known about how developers practice program comprehension in their daily work. This article reports on qualitative and quantitative research to comprehend the strategies, tools, and knowledge used for program comprehension. We observed 28 professional developers, focusing on their comprehension behavior, strategies followed, and tools used. In an online survey with 1,477 respondents, we analyzed the importance of certain types of knowledge for comprehension and where developers typically access and share this knowledge.
We found that developers follow pragmatic comprehension strategies depending on context. They try to avoid comprehension whenever possible and often put themselves in the role of users by inspecting graphical interfaces. Participants confirmed that standards, experience, and personal communication facilitate comprehension. The team size, its distribution, and open-source experience influence their knowledge sharing and access behavior. While face-to-face communication is preferred for accessing knowledge, knowledge is frequently shared in informal comments.
Our results reveal a gap between research and practice, as we did not observe any use of comprehension tools and developers seem to be unaware of them. Overall, our findings call for reconsidering the research agendas towards context-aware tool support.
- Mark Ackerman, Volkmar Pipek, and Volker Wulf. 2003. Sharing Expertise Beyond Knowledge Management. The MIT Press. Google Scholar
Digital Library
- B. Adelson and E. Soloway. 1985. The role of domain expenence in software design. IEEE Trans. Softw. Eng. 11, 11 (1985), 1351--1360. DOI: http://dx.doi.org/10.1109/TSE.1985.231883 Google ScholarDigital Library
- T. Anderson and H. Kanuka. 2003. e-RESEARCH Methods, Strategies, and Issues. Pearson Education.Google Scholar
- F. O. Bjørnson and T. Dingsøyr. 2008. Knowledge management in software engineering: A systematic review of studied concepts, findings and research methods used. Inf. Softw. Technol. 50, 11 (2008), 1055--1068. DOI: http://dx.doi.org/10.1016/j.infsof.2008.03.006 Google ScholarDigital Library
- B. W. Boehm. 1976. Software engineering. IEEE Trans. Comput. C-25, 12 (1976), 1226--1241. Google ScholarDigital Library
- J. Brandt, M. Dontcheva, M. Weskamp, and S. R. Klemmer. 2010. Example-centric programming: Integrating Web search into the development environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'10). ACM, New York, NY, 513--522. DOI: http://dx.doi.org/10.1145/1753326.1753402 Google ScholarDigital Library
- J. Brandt, Ph. J. Guo, J. Lewenstein, M. Dontcheva, and S. R. Klemmer. 2009. Two studies of opportunistic programming: Interleaving Web foraging, learning, and writing code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'09). ACM, New York, NY, 1589--1598. DOI: http://dx.doi.org/10.1145/1518701.1518944 Google ScholarDigital Library
- S. Breu, R. Premraj, J. Sillito, and Th. Zimmermann. 2010. Information needs in bug reports: Improving cooperation between developers and users. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'10). ACM, New York, NY, 301--310. DOI: http://dx.doi.org/10.1145/1718918.1718973 Google ScholarDigital Library
- R. E. Brooks. 1983. Towards a theory of the comprehension of computer programs. Int. J. Man-Machine Stud. 18, 6 (1983), 543--554.Google ScholarCross Ref
- A. Cabrera and E. F. Cabrera. 2002. Knowledge-sharing dilemmas. Org. Stud. 23 (2002), 687--710.Google ScholarCross Ref
- J. Carroll. 1998. Minimalism beyond the Nurnberg Funnel. MIT Press.Google Scholar
- M. Cataldo and J. D. Herbsleb. 2008. Communication networks in geographically distributed software development. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'08). ACM, New York, NY, 579--588. DOI: http://dx.doi.org/10.1145/1460563.1460654 Google ScholarDigital Library
- J. W. Creswell. 2009. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications, Inc.Google Scholar
- B. Dagenais and M. P. Robillard. 2010. Creating and evolving developer documentation: Understanding the decisions of open source contributors. In Proceedings of the 18th ACM SIGSOFT International Symposium on the Foundations of Software Engineering. 127--136. Google ScholarDigital Library
- R. DeLine, A. Khella, M. Czerwinski, and G. Robertson. 2005. Towards understanding programs through wear-based filtering. In Proceedings of the ACM Symposium on Software Visualization (SoftVis'05). ACM, 183--192. Google ScholarDigital Library
- Kevin C. Desouza. 2003. Barriers to effective use of knowledge management systems in software engineering. Commun. ACM 46, 1 (2003), 99--101. DOI: http://dx.doi.org/10.1145/602421.602458 Google ScholarDigital Library
- Francoise Detienne. 2002. Software Design -- Cognitive Aspect. Springer. Google ScholarDigital Library
- K. A. Ericsson and H. A. Simon. 1993. Protocol Analysis: Verbal Reports as Data. Massachusetts Institute of Technology.Google ScholarCross Ref
- R. K. Fjeldstad and W. T. Hamlen. 1979. Application program maintenance study: Report to our respondents. In Proceedings of GUIDE 48. PA.Google Scholar
- A. Forward and T. C. Lethbridge. 2002. The relevance of software documentation, tools and technologies: A survey. In Proceedings of the ACM Symposium on Document Engineering. ACM, 26--33. Google ScholarDigital Library
- Th. Fritz and G. C. Murphy. 2010. Using information fragments to answer the questions developers ask. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering (ICSE'10). ACM, New York, NY, 175--184. DOI: http://dx.doi.org/10.1145/1806799.1806828 Google ScholarDigital Library
- G. Gilder. 2013. Knowledge and Power: The Information Theory of Capitalism and How it is Revolutionizing our World. Regnery Publishing.Google Scholar
- A. M. Grubb and A. Begel. 2012. On the perceived interdependence and information sharing inhibitions of enterprise software engineers. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'12). ACM, New York, NY, 1337--1346. DOI: http://dx.doi.org/10.1145/2145204.2145403 Google ScholarDigital Library
- J. D. Herbsleb and A. Mockus. 2003. An empirical study of speed and communication in globally-distributed software development. IEEE Trans. Softw. Eng. 29, 6 (2003), 481--494. Google ScholarDigital Library
- R. Hoffmann, J. Fogarty, and D. S. Weld. 2007. Assieme: Finding and leveraging implicit references in a Web search interface for programmers. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST'07). ACM, New York, NY, 13--22. DOI: http://dx.doi.org/10.1145/1294211.1294216 Google ScholarDigital Library
- R. Holmes and R. J. Walker. 2013. Systematizing pragmatic software reuse. ACM Trans. Softw. Eng. Methodol. 21, 4, Article 20 (2013). DOI: http://dx.doi.org/10.1145/2377656.2377657 Google ScholarDigital Library
- Miryung Kim, L. Bergman, T. Lau, and D. Notkin. 2004. An ethnographic study of copy and paste programming practices in OOPL. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE'04). 83--92. DOI: http://dx.doi.org/10.1109/ISESE.2004.1334896 Google ScholarDigital Library
- A. J. Ko, R. DeLine, and G. Venolia. 2007. Information needs in collocated software development teams. In Proceedings of the 29th International Conference on Software Engineering (ICSE'07). IEEE Computer Society, 344--353. DOI: http://dx.doi.org/10.1109/ICSE.2007.45 Google ScholarDigital Library
- A. J. Ko and B. A. Myers. 2004. Designing the whyline: A debugging interface for asking questions about program behavior. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'04). ACM, 151--158. Google ScholarDigital Library
- A. J. Ko and B. A. Myers. 2008. Debugging reinvented: Asking and answering why and why not questions about program behavior. In Proceedings of the 30th International Conference on Software Engineering (ICSE'08). ACM, New York, NY, 301--310. DOI: http://dx.doi.org/10.1145/1368088.1368130 Google ScholarDigital Library
- A. J. Ko, B. A. Myers, M. J. Coblenz, and H. H. Aung. 2006. An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Trans. Softw. Eng. 32, 12 (2006), 971--987. DOI: http://dx.doi.org/10.1109/TSE.2006.116 Google ScholarDigital Library
- Th. D. LaToza, D. Garlan, J. D. Herbsleb, and B. A. Myers. 2007. Program comprehension as fact finding. In Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC-FSE'07). ACM, New York, NY, 361--370. DOI: http://dx.doi.org/10.1145/1287624.1287675 Google ScholarDigital Library
- Th. D. LaToza, G. Venolia, and R. DeLine. 2006. Maintaining mental models: A study of developer work habits. In Proceeding of the 28th International Conference on Software Engineering (ICSE'06). ACM, New York, NY, 492--501. DOI: http://dx.doi.org/10.1145/1134285.1134355 Google ScholarDigital Library
- T. C. Lethbridge, J. Singer, and A. Forward. 2003. How software engineers use documentation: The state of the practice. IEEE Softw. 20, 6 (2003), 35--39. Google ScholarDigital Library
- S. Letovsky and E. Soloway. 1986. Delocalized plans and program comprehension. IEEE Softw. 3, 3 (1986), 41--49. Google ScholarDigital Library
- D. C. Littman, J. Pinto, S. Letovsky, and E. Soloway. 1987. Mental models and software maintenance. J. Syst. Softw. 7, 4 (1987), 341--355. Google ScholarDigital Library
- W. Maalej. 2009. Task-first or context-first? Tool integration revisited. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE'09). IEEE Computer Society, 344--355. DOI: http://dx.doi.org/10.1109/ASE.2009.36 Google ScholarDigital Library
- Walid Maalej, Thomas Fritz, and Romain Robbes. 2014. Collecting and processing interaction data for recommendation systems. In Recommendation Systems in Software Engineering, Martin P. Robillard, Walid Maalej, Robert J. Walker, and Thomas Zimmermann (Eds.), Springer, Berlin, 173--197. DOI: http://dx.doi.org/10.1007/978-3-642-45135-5_7Google Scholar
- W. Maalej and H.-J. Happel. 2010. Can development work describe itself?. In Proceedings of the 7th IEEE International Working Conference on Mining Software Repositories (MSR'10). IEEE, 191--200.Google Scholar
- Walid Maalej and Martin P. Robillard. 2013. Patterns of knowledge in API reference documentation. IEEE Trans. Softw. Eng. 39, 9 (2013), 1264--1282. DOI: http://dx.doi.org/10.1109/TSE.2013.12 Google ScholarDigital Library
- A. E. Milewski, M. Tremaine, R. Egan, S. Zhang, F. Kobler, and P. O'Sullivan. 2008. Guidelines for effective bridging in global software engineering. In Proceedings of the IEEE International Conference on Global Software Engineering. IEEE Computer Society, 23--32. DOI: http://dx.doi.org/10.1109/ICGSE.2008.16 Google ScholarDigital Library
- A. Mockus and J. D. Herbsleb. 2002. Expertise browser: A quantitative approach to identifying expertise. In Proceedings of the 24th International Conference on Software Engineering (ICSE'02). ACM, New York, NY, 503--512. DOI: http://dx.doi.org/10.1145/581339.581401 Google ScholarDigital Library
- G. C. Murphy, M. Kersten, and L. Findlater. 2006. How are Java software developers using the Eclipse IDE? IEEE Softw. 23, 4 (2006), 76--83. DOI: http://dx.doi.org/10.1109/MS.2006.105 Google ScholarDigital Library
- E. Murphy-Hill. 2014. The future of social learning in software engineering. Computer 47, 1 (2014), 48--54. DOI: http://dx.doi.org/10.1109/MC.2013.406 Google ScholarDigital Library
- E. Murphy-Hill, R. Jiresal, and G. C. Murphy. 2012a. Improving software developers' fluency by recommending development environment commands. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (FSE'12). ACM, New York, NY, Article 42, 11 pages. DOI: http://dx.doi.org/10.1145/2393596.2393645 Google ScholarDigital Library
- E. Murphy-Hill and G. C. Murphy. 2011. Peer Interaction effectively yet infrequently, enables programmers to discover new tools. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'11). ACM, New York, NY, 405--414. DOI: http://dx.doi.org/10.1145/1958824.1958888 Google ScholarDigital Library
- E. Murphy-Hill, C. Parnin, and A. P. Black. 2012b. How we refactor, and how we know it. IEEE Trans. Softw. Eng. 38, 1 (2012), 5--18. DOI: http://dx.doi.org/10.1109/TSE.2011.41 Google ScholarDigital Library
- Ch. Parnin and S. Rugaber. 2009. Resumption strategies for interrupted programming tasks. In Proceedings of the IEEE 17th International Conference on Program Comprehension (ICPC). IEEE.Google Scholar
- F. Rahman, Ch. Bird, and P. Devanbu. 2012. Clones: What is that smell? Empirical Softw. Eng. 17, 4--5 (2012), 503--530. DOI: http://dx.doi.org/10.1007/s10664-011-9195-3 Google ScholarDigital Library
- V. Rajlich and N. Wilde. 2002. The role of concepts in program comprehension. In Proceedings of the 10th International Workshop on Program Comprehension. 271--278. Google ScholarDigital Library
- T. Reichling, M. Veith, and V. Wulf. 2007. Expert recommender: Designing for a Network Organization. Comput. Supported Coop. Work 16, 4--5 (2007), 431--465. DOI: http://dx.doi.org/10.1007/s10606-007-9055-2 Google ScholarDigital Library
- M. Robillard, W. Coelho, and G. Murphy. 2004. How effective developers investigate source code: An exploratory Study. IEEE Trans. Softw. Eng. 30, 12 (2004), 889--903. Google ScholarDigital Library
- M. Robillard and R. DeLine. 2010. A field study of API learning obstacles. Empirical Softw. Eng. (2010), 1--30. DOI: http://dx.doi.org/10.1007/s10664-010-9150-8 Google ScholarDigital Library
- Martin P. Robillard. 2009. What makes APIs hard to learn? Answers from developers. IEEE Softw. 26, 6 (2009), 27--34. DOI: http://dx.doi.org/10.1109/MS.2009.193 Google ScholarDigital Library
- T. Roehm and W. Maalej. 2012. Automatically detecting developer activities and problems in software development work: NIER track. In Proceedings of the 34th International Conference on Software Engineering, (ICSE'12). Google ScholarDigital Library
- T. Roehm, R. Tiarks, R. Koschke, and W. Maalej. 2012. How do professional developers comprehend software? In Proceedings of the International Conference on Software Engineering (ICSE'12). IEEE Press, 255--265. http://dl.acm.org/citation.cfm?id=2337223.2337254. Google ScholarDigital Library
- R. L. Rosnow and R. Rosenthal. 2007. Beginning Behavioral Research: A Conceptual Primer (6th. Ed.). Pearson.Google Scholar
- J. Sillito, K. de Voider, B. Fisher, and G. Murphy. 2005. Managing software change tasks: An exploratory study. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE'05). DOI: http://dx.doi.org/10.1109/ISESE.2005.1541811Google Scholar
- J. Sillito, G. C. Murphy, and K. de Volder. 2008. Asking and answering questions during a programming change task. Trans. Softw. Eng. 34 (2008), 434--451. Issue 4. DOI: http://dx.doi.org/10.1109/TSE.2008.26. Google ScholarDigital Library
- J. Singer, T. Lethbridge, N. Vinson, and N. Anquetil. 1997. An examination of software engineering work practices. In Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research (CASCO). IBM Press, 21. Google ScholarDigital Library
- L.-G. Singer. 2013. Improving the Adoption of Software Engineering Practices through Persuasive Interventions. lulu.com.Google Scholar
- D. K. Sjøberg, B. Anda, E. Arisholm, T. Dybå, M. Jørgensen, A. Karahasanović, and M. Vokáč. 2003. Challenges and recommendations when increasing the realism of controlled software engineering experiments. In Empirical Methods and Studies in Software Engineering, Reidar Conradi and AlfInge Wang (Eds.), Lecture Notes in Computer Science, vol. 2765, Springer, Berlin, 24--38. DOI: http://dx.doi.org/10.1007/978-3-540-45143-3_3.Google Scholar
- E. Soloway and K. Ehrlich. 1984. Empirical studies of programming knowledge. IEEE Trans. Softw. Eng. 10, 5 (1984), 595--609. DOI: http://dx.doi.org/10.1109/TSE.1984.5010283 Google ScholarDigital Library
- Tracy L. Tuten, David J. Urban, and Michael Bosnjak. 2002. Internet Surveys and Data Quality: A Review. Hogrefe and Huber Publishers.Google Scholar
- M. Vakilian, N. Chen, S. Negara, B. A. Rajkumar, B. P. Bailey, and R. E. Johnson. 2012. Use, disuse, and misuse of automated refactorings. In Proceedings of the International Conference on Software Engineering (ICSE'12). IEEE Press, 233--243. http://dl.acm.org/citation.cfm?id=2337223.2337251. Google ScholarDigital Library
- A. Von Mayrhauser and A. M. Vans. 1996. Identification of dynamic comprehension processes during large scale maintenance. IEEE Trans. Softw. Eng. 22, 6 (1996), 424--437. Google ScholarDigital Library
- A. Von Mayrhauser, A. M. Vans, and S. Lang. 1998. Program comprehension and enhancement of software. In Proceedings of the IFIP World Computing Congress - Information Technology and Knowledge Engineering.Google Scholar
- J. C. F. D. Winter and D. Dodou. 2010. Five-point Likert items: t test versus Mann-Whitney-Wilcoxon. Practical Assess. Res. Eval. 15, 11 (2010).Google Scholar
Index Terms
- On the Comprehension of Program Comprehension
Recommendations
Analysis of code reading to gain more insight in program comprehension
Koli Calling '11: Proceedings of the 11th Koli Calling International Conference on Computing Education ResearchCode reading, although an integral part of program comprehension, is rarely reflected. In this paper, we want to argue for a research approach and direction exploiting the potential that lies in the analysis of reading processes. Based on the vast ...
A comparison of program comprehension strategies by blind and sighted programmers
ICSE '18: Proceedings of the 40th International Conference on Software EngineeringProgrammers who are blind use a screen reader to speak source code one word at a time, as though the code were text. For example, "float f = 5.23;" can be read as "float f equals five point two three semicolon". This process of reading is in stark ...
Measuring program comprehension: a large-scale field study with professionals
ICSE '18: Proceedings of the 40th International Conference on Software EngineeringDuring software development and maintenance, developers spend a considerable amount of time on program comprehension. Previous studies show that program comprehension takes up as much as half of a developer's time. However, most of these studies are ...
Comments