Peter J. Clarke
Tariq M. King
Skip Abstract Section
Abstract
As software becomes more ubiquitous and complex, the cost of software bugs continues to grow at a staggering rate. To remedy this situation, there needs to be major improvement in the knowledge and application of software validation techniques. Although there are several software validation techniques, software testing continues to be one of the most widely used in industry. The high demand for software engineers in the next decade has resulted in more software engineering (SE) courses being offered in academic institutions. However, due to the number of topics to be covered in SE courses, little or no attention is given to software testing, resulting in students entering industry with little or no testing experience.
We propose a minimally disruptive approach of integrating software testing into SE courses by providing students access to a collaborative learning environment containing learning materials on testing techniques and testing tools. In this article, we describe the learning environment and the studies conducted to measure the benefits accrued by students using the learning environment in the SE courses.
- ACM/IEEE-CS Interim Review Task Force. 2008. Computer Science Curriculum 2008: An Interim Revision of CS 2001. Retrieved August 7, 2014, from http://www.acm.org/education/curricula/ComputerScience 2008.pdf.Google Scholar
- Paul Ammann and Jeff Offutt. 2008. Introduction to Software Testing. Cambridge University Press, New York, NY. Google ScholarDigital Library
- Tara Astigarraga, Eli M. Dow, Christina Lara, Richard Prewitt, and Maria R. Ward. 2010. The emerging role of software testing in curricula. In Transforming Engineering Education: Creating Interdisciplinary Skills for Complex Global Environments, 2010 IEEE. IEEE, Los Alamitos, CA, 1--26. DOI: http://dx.doi.org/10.1109/TEE.2010.5508833Google ScholarCross Ref
- Ellen Francine Barbosa, José Carlos Maldonado, Richard LeBlanc, and Mark Guzdial. 2003. Introducing testing practices into objects and design course. In Proceedings of the 16th Conference on Software Engineering Education and Training. IEEE, Los Alamitos, CA, 279--286. Google ScholarDigital Library
- Victor R. Basili, Richard W. Selby, and David H. Hutchens. 1986. Experimentation in software engineering. IEEE Transactions on Software Engineering SE-12, 7, 733--743. DOI: http://dx.doi.org/10.1109/TSE.1986.6312975 Google ScholarDigital Library
- Sebastian Bergmann. 2013. PHPUnit. Retrieved August 7, 2014, from https://github.com/sebastianbergmann/phpunit/.Google Scholar
- Robert V. Binder. 1999. Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison Wesley Longman, Boston, MA. Google ScholarDigital Library
- Pierre Bourque and Robert Dupuis. 2004. Guide to the Software Engineering Body of Knowledge 2004 Version. IEEE, Los Alamitos, CA.Google Scholar
- Bernd Bruegge and Allen H. Dutoit. 2009. Object-Oriented Software Engineering Using UML, Patterns, and Java (3rd ed.). Prentice Hall, Upper Saddle River, NJ. Google ScholarDigital Library
- Lilian Cassel, Lois Delcambre, Edward Fox, and Richard Furuta. 2012. Ensemble: Computing Portal Connecting Computing Educators. Retrieved August 7, 2014, from http://www.computingportal.org/.Google Scholar
- Yaofei Chen, Rose Dios, Ali Mili, Lan Wu, and Kefei Wang. 2005. An empirical study of programming language trends. IEEE Software 22, 3, 72--78. Google ScholarDigital Library
- CITIDEL Team. 2010. CITIDEL: Computing and Information Technology Interactive Digital Educational Library. Retrieved August 7, 2014, from http://www.citidel.org/.Google Scholar
- Mike Clark. 2012. JDepend. Retrieved August 7, 2014, from http://clarkware.com/software/JDepend.html.Google Scholar
- Peter J. Clarke, Andrew A. Allen, Tariq M. King, Edward L. Jones, and Prathiba Natesan. 2010. Using a Web-based repository to integrate testing tools into programming courses. In Proceedings of the ACM OOPSLA 2010 Companion (SPLASH’10). ACM, New York, NY, 193--200. Google ScholarDigital Library
- Peter J. Clarke, Jairo Pava, Debra Davis, and Tariq M. King. 2012. Using WReSTT in SE courses: An empirical study. In Proceedings of the 43rd SIGCSE Conference. ACM, New York, NY, 307--312. Google ScholarDigital Library
- Peter J. Clarke, Jairo Pava, Yali Wu, and Tariq M. King. 2011. Collaborative Web-based learning of testing tools in SE courses. In Proceedings of the 42nd SIGCSE Conference. ACM, New York, NY, 147--152. Google ScholarDigital Library
- CNN. 2012. Is Knight’s $440 million glitch the costliest computer bug ever? (2012). CNN Money, August 9. Retrieved August 7, 2014, from http://money.cnn.com/2012/08/09/technology/knight-expensive- computer-bug/index.html.Google Scholar
- CNSS. 2005. Software 2015: A National Software Strategy to Ensure U.S. Security and Competitiveness. Technical Report. Center for National Software Studies, Upper Marlboro, MD.Google Scholar
- Cobertura Team. 2012. Cobertura. Retrieved August 7, 2014, from http://cobertura.sourceforge.net/.Google Scholar
- CoverStory Team. 2013. CoverStory. Retrieved August 7, 2014, from http://code.google.com/p/coverstory/.Google Scholar
- Tony Cowling. 2012. Stages in teaching software testing. In Proceedings of the 2012 International Conference on Software Engineering (ICSE’12). IEEE, Los Alamitos, CA, 1185--1194. Available at http://dl.acm.org/citation.cfm?id=2337223.2337379. Google ScholarDigital Library
- David C. Crowther and Peter J. Clarke. 2005. Examining software testing tools. Dr. Dobbs Journal 373, 1, 26--33.Google Scholar
- Chetan Desai, David S. Janzen, and John Clements. 2009. Implications of integrating test-driven development into CS1/CS2 curricula. ACM SIGCSE Bulletin 41, 1, 148--152. Google ScholarDigital Library
- Adrián Domínguez, Joseba Saenz de Navarrete, Luis de Marcos, Luis Fernández-Sanz, Carmen Pagés, and José-Javier Martínez-Herráiz. 2013. Gamifying learning experiences: Practical implications and outcomes. Computers and Education 63, 380--392. Google ScholarDigital Library
- Drupal Community. 2012. Drupal. Retrieved August 7, 2014, from http://drupal.org/.Google Scholar
- Thomas Dvornik, David S. Janzen, John Clements, and Olga Dekhtyar. 2011. Supporting introductory test-driven labs with WebIDE. In Proceedings of the 24th IEEE-CS Software Engineering Education and Training Conference (CSEET’11). IEEE, Los Alamitos, CA, 51--60. Google ScholarDigital Library
- Stephen H. Edwards. 2003. Rethinking computer science education from a test-first perspective. In Companion of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA’03). ACM, New York, NY, 148--155. Google ScholarDigital Library
- Facebook Team. 2012. Facebook. Retrieved August 7, 2014, from http://www.facebook.com/.Google Scholar
- Michael Feathers. 2012. CppUnit. Retrieved August 7, 2014, from http://sourceforge.net/projects/cppunit/.Google Scholar
- Stephen Frezza. 2002. Integrating testing and design methods for undergraduates: Teaching software testing in the context of software design. In Proceedings of the 32nd Annual Frontiers in Education, 2002 (FIE’02), Vol 3. IEEE, Los Alamitos, CA, SIG-1--SIG-4.Google ScholarCross Ref
- Michael P. Gallaher and Brian M. Kropp. 2011. The Economic Impacts of Inadequate Infrastructure for Software Testing. Retrieved August 7, 2014, from http://www.rti.org/pubs/software_testing.pdf.Google Scholar
- Erich Gamma and Kent Beck. 2012. JUnit. Retrieved August 7, 2014, from http://www.junit.org/.Google Scholar
- Vahid Garousi. 2010. An open modern software testing laboratory courseware—an experience report. In Proceedings of the 23rd IEEE-CS Software Engineering Education and Training Conference (CSEET’10). IEEE, Los Alamitos, CA, 177--184. Google ScholarDigital Library
- Michael H. Goldwasser. 2002. A gimmick to integrate software testing throughout the curriculum. In Proceedings of the 33rd SIGCSE Conference. ACM, New York, NY, 271--275. Google ScholarDigital Library
- Starr R. Hiltz. 1998. Collaborative learning in asynchronous learning networks: Building learning communities. (1998). In Proceedings of WEB98.Google Scholar
- Marc R. Hoffmann. 2012. EclEmma. Retrieved August 8, 2014, from http://www.eclemma.org/.Google Scholar
- IBM. 2012. Rational Functional Tester. Retrieved August 7, 2014, from http://www-01.ibm.com/software/awdtools/tester/functional/.Google Scholar
- David S. Janzen and Hossein Saiedian. 2006. Test-driven learning: Intrinsic integration of testing into the CS/SE curriculum. ACM SIGCSE Bulletin 38, 1, 254--258. DOI: http://dx.doi.org/10.1145/1124706.1121419 Google ScholarDigital Library
- Edward L. Jones. 2000. Software testing in the computer science curriculum—a holistic approach. In Proceedings of the Australasian Conference on Computing Education (ACSE’00). ACM, New York, NY, 153--157. Google ScholarDigital Library
- Cem Kaner, James Bach, and Bret Pettichord. 2001. Lessons Learned in Software Testing. John Wiley & Sons, New York, NY. Google ScholarDigital Library
- Timothy C. Lethbridge, Jorge Diaz-Herrera, Richard J. LeBlanc Jr., and J. Barrie Thompson. 2007. Improving software practice through education: Challenges and future trends. In Proceedings of the 2007 Future of Software Engineering (FOSE’07). IEEE, Los Alamitos, CA, 12--28. Google ScholarDigital Library
- Cen Li, Zhijiang Dong, Roland H. Untch, and Michael Chasteen. 2013. Engaging computer science students through gamification in an online social network based collaborative learning environment. International Journal of Information and Education Technology 3, 1, 72--77.Google ScholarCross Ref
- Qing Li, Rynson W. H. Lau, Timothy K. Shih, and Frederick W. B. Li. 2008. Technology supports for distributed and collaborative learning over the Internet. Transactions on Internet Technology 8, 2, Article No. 5. DOI: http://dx.doi.org/10.1145/1323651.1323656 Google ScholarDigital Library
- Michael J. Lutz, W. Michael McCracken, Susan Mengel, Mark Sebern, Greg W. Hislop, and Thomas B. Hilburn. 2010. SWENET—SEEK Category: Software Verification and Validation (VAV). Retrieved August 7, 2014, from http://www.swenet.org/browseModules.aspx?categoryID=11.Google Scholar
- Thomas W. Malone. 1980. What makes things fun to learn? Heuristics for designing instructional computer games. In Proceedings of the 3rd ACM SIGSMALL Symposium and the 1st SIGPC Symposium on Small Systems (SIGSMALL’80). ACM, New York, NY, 162--169. DOI: http://dx.doi.org/10.1145/800088.802839. Google ScholarDigital Library
- Aditya P. Mathur. 2008. Foundations of Software Testing. Pearson Education, Delhi, India. Google ScholarDigital Library
- Microsoft Corporation. 2013. Using Testing Tools in Visual Studio Professional Edition. Retrieved August 7, 2014, from http://msdn.microsoft.com/en-us/library/bb385902(v=VS.90).aspx.Google Scholar
- Glenford J. Myers. 2004. Art of Software Testing (2nd ed.). John Wiley & Sons, New York, NY. Google ScholarDigital Library
- John Neter, William Wasserman, and Michael H. Kutner. 1990. Applied Linear Statistical Models. Irwin Press, Boston, MA.Google Scholar
- William Perry. 2006. Effective Methods for Software Testing (3rd ed.). John Wiley & Sons, New York, NY. Google ScholarDigital Library
- Shari Lawrence Pfleeger and Joanne M. Atlee. 2009. Software Engineering: Theory and Practice (4th ed.). Pearson Education, Cranbury Township, NJ.Google Scholar
- Alex Ramírez and Geoffrey C. Fox. 2011. A Report of the National Science Foundation Advisory Committee for Cyberinfrastructure Task Force on Cyberlearning and Workforce Development. Retrieved August 7, 2014, from http://www.nsf.gov/cise/aci/taskforces/TaskForceReport_Learning.pdf.Google Scholar
- Sen:te. 2013. Moving OCUnit tests between Logic and Application tests. Retrieved August 7, 2014, from http://www.sente.ch/?p=276&lang=en.Google Scholar
- Terry Shepard, Margaret Lamb, and Diane Kelly. 2001. More testing should be taught. Communications of the ACM 44, 6, 103--108. DOI: http://dx.doi.org/10.1145/376134.376180 Google ScholarDigital Library
- Charlie Y. Shim, Mina Choi, and Jung Yeop Kim. 2009. Promoting collaborative learning in software engineering by adapting the PBL strategy. In Proceedings of the WASET International Conference on Computer and Information Technology (ICCIT ’09). IEEELos Alamitos, CA, 1167--1170.Google Scholar
- Sidney Siegel and N. John Castellan. 1988. Nonparametric Statistics for the Behavioral Sciences (2nd ed.). McGraw-Hill, New York, NY.Google Scholar
- Barbara Leigh Smith and Jean T. MacGregor. 1992. What is collaborative learning? In Collaborative Learning: A Sourcebook for Higher Education, A. S. Goodsell, M. R. Maher, and V. Tinto (Eds.). National Center on Postsecondary Teaching, Learning, and Assessment, University Park, PA.Google Scholar
- Software Quality Engineering Research Group (SoftQual). 2012. SoftQual Repository of Software Testing Laboratory Courseware. Retrieved August 7, 2014, from http://www.softqual.ucalgary.ca/projects/testing_labs/.Google Scholar
- Ian Sommerville. 2004. Software Engineering (7th ed.). Pearson Addison Wesley, Boston, MA. Google ScholarDigital Library
- Bénédicte Talon, Dominique Leclet, Arnaud Lewandowski, and Grégory Bourguin. 2009. Learning software testing using a collaborative activities oriented platform. In Proceedings of the 2009 9th IEEE International Conference on Advanced Learning Technologies (ICALT’09). IEEE, Los Alamitos, CA, 443--445. DOI: http://dx.doi.org/10.1109/ICALT.2009.61 Google ScholarDigital Library
- William Trochim. 2001. The Research Methods Knowledge Base (2nd ed.). Atomic Dog Publishing, Cincinnati, OH.Google Scholar
- Tom Tullis and Bill Albert. 2008. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Elsevier: Morgan Kaufmann, San Francisco, CA. Google ScholarDigital Library
- Ultimate Software. 2012. Simple Web Automation Toolkit. Retrieved August 7, 2014, from http://sourceforge.net/projects/ulti-swat/.Google Scholar
- Wikipedia. 2012. List of Software Bugs. Retrieved August 7, 2014, from http://en.wikipedia.org/wiki/List_of_software_bugs.Google Scholar
- Laurie Williams. 2010. Software Engineering: Testing. OpenSeminar. Retrieved August 7, 2014, from http://openseminar.org/se/modules/7/index/screen.do.Google Scholar
- Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Bjöorn Regnell, and Anders Wesslén. 2000. Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Norwell, MA. Google ScholarDigital Library
- WReSTT Team. 2012. WReSTT: Web-based Repository of Software Testing Tools. Retrieved August 7, 2014, from http://wrestt.cis.fiu.edu/.Google Scholar
Index Terms
- Integrating Testing into Software Engineering Courses Supported by a Collaborative Learning Environment
Recommendations
Impact of Using Tools in an Undergraduate Software Testing Course Supported by WReSTT
Software continues to affect a major part of our daily lives, including the way we use our phones, home appliances, medical devices, and cars. The pervasiveness of software has led to a growing demand for software developers over the next decade. To ...
Collaborative web-based learning of testing tools in SE courses
SIGCSE '11: Proceedings of the 42nd ACM technical symposium on Computer science educationOne of the main concerns in the software industry continues to be the development of high quality software. This concern will be exacerbated as software systems become more complex. The training of software developers continues to grow in academia since ...
Teaching Testing with Modern Technology Stacks in Undergraduate Software Engineering Courses
ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1Students' experience with software testing in undergraduate computing courses is often relatively shallow, as compared to the importance of the topic. This experience report describes introducing industrial-strength testing into CMPSC 156, an upper ...
Comments