Environment modeling in model-based testing: concepts, prospects and research challenges: a systematic literature review

ABSTRACT

In this paper, we describe a systematic literature review (SLR) on the use of environment models in model-based testing (MBT). By applying selection criteria, we narrowed down the identified studies from two hundred ninety seven papers to sixty one papers which are used in this analysis. The results show that environment models are especially useful in testing systems with high complexity and nondeterministic behaviors in terms of facilitating automatic test generation. However, building environment models is not a trivial task due to the lack of a systematic methodology and of supporting tools for automation.

References

1. The UML Profile for MARTE: Modeling and Analysis of Real-Time and Embedded Systems. http://www.omgmarte.org/, 2013. {Online; accessed 22-December-2014}.
2. N. Adjir, P. De Saqui-Sannes, and K. Rahmouni. Testing Real-Time Systems Using TINA. In M. Núñez, P. Baker, and M. Merayo, editors, Testing of Software and Communication Systems, volume 5826 of LNCS, pages 1--15. Springer Berlin Heidelberg, 2009.
3. R. Alur and D. L. Dill. A theory of timed automata. Theoretical Computer Science, 126: 183--235, 1994.
4. M. Auguston, J. B. Michael, and M.-T. Shing. Environment behavior models for automation of testing and assessment of system safety. Information and Software Technology, 48(10): 971--980, 2006. Advances in Model-based Testing.
5. B. Berthomieu and F. Vernadat. Time petri nets analysis with TINA. In Quantitative Evaluation of Systems, 2006. QEST 2006. Third International Conference on, pages 123--124. IEEE, 2006.
6. T. Elrad, O. Aldawud, and A. Bader. Aspect-oriented modeling: Bridging the gap between implementation and design. In Generative Programming and Component Engineering, pages 189--201. Springer, 2002.
7. G. Fraser and F. Wotawa. Test-Case Generation and Coverage Analysis for Nondeterministic Systems Using Model-Checkers. In International Conference on Software Engineering Advances, pages 45--45, Aug 2007.
8. S. Friedenthal, A. Moore, and R. Steiner. A practical guide to SysML: the systems modeling language. Elsevier, 2011.
9. M. Iqbal, A. Arcuri, and L. Briand. Environment Modeling with UML/MARTE to Support Black-Box System Testing for Real-Time Embedded Systems: Methodology and Industrial Case Studies. In D. Petriu, N. Rouquette, and Ø. Haugen, editors, Model Driven Engineering Languages and Systems, LNCS, pages 286--300. Springer Berlin Heidelberg, 2010.
10. S. Jalali and C. Wohlin. Systematic Literature Studies: Database Searches vs. Backward Snowballing. In Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM, pages 29--38, New York, NY, USA, 2012. ACM.
11. G. Karsai, S. Neema, and D. Sharp. Model-driven architecture for embedded software: A synopsis and an example. Science of Computer Programming, 73(1): 26--38, 2008. Special Issue on Foundations and Applications of Model Driven Architecture (MDA).
12. T. Kishi and N. Noda. Aspect-oriented context modeling for embedded systems. Early Aspects: Aspect-Oriented Requirements Engineering and Architecture Design, page 69, 2004.
13. B. Kitchenham, O. Pearl Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman. Systematic Literature Reviews in Software Engineering - A Systematic Literature Review. Inf. Softw. Technol., 51(1): 7--15, Jan. 2009.
14. K. G. Larsen, M. Mikucionis, B. Nielsen, and A. Skou. Testing real-time embedded software using uppaal-tron: An industrial case study. In Proceedings of the 5th ACM International Conference on Embedded Software, EMSOFT '05, pages 299--306, New York, NY, USA, 2005. ACM.
15. G. Li, S. Yuen, and M. Adachi. Environmental simulation of real-time systems with nested interrupts. In Theoretical Aspects of Software Engineering, 2009. TASE 2009. Third IEEE International Symposium on, pages 21--28, July 2009.
16. L. Lucio, L. Pedro, and D. Buchs. A methodology and a framework for model-based testing. In Rapid Integration of Software Engineering Techniques, pages 57--70. Springer, 2005.
17. M. Mews, J. Svacina, and S. Weissleder. From AUTOSAR Models to Co-simulation for MiL-Testing in the Automotive Domain. In International Conference on Software Testing, Verification and Validation, pages 519--528, April 2012.
18. R. Pascal, R. Yvan, and J. Erwan. Lutin: A language for specifying and executing reactive scenarios. EURASIP Journal on Embedded Systems, 2008, 2008.
19. J. Rumbaugh, I. Jacobson, and G. Booch. Unified Modeling Language Reference Manual, The. Pearson Higher Education, 2004.
20. F. Siavashi and D. Truscan. A systematic literature review on environment modeling techniques in model-based testing. Technical Report 1129, 2015, http://tucs.fi/publications/view/?pub_id=tSiTr15a.
21. O. Tkachuk, M. Dwyer, and C. Pasareanu. Automated environment generation for software model checking. In International Conference on Automated Software Engineering, 2003. Proceedings., pages 116--127, Oct 2003.
22. M. Utting and B. Legeard. Practical model-based testing: a tools approach. Morgan Kaufmann, 2010.
23. M. Utting, A. Pretschner, and B. Legeard. A Taxonomy of Model-based Testing Approaches. Softw. Test. Verif. Reliab., 22(5): 297--312, Aug. 2012.
24. S. Yang, B. Liu, Shihai, and M. Lu. Model-based robustness testing for avionics-embedded software. Chinese Journal of Aeronautics, 26(3): 730--740, 2013.

Index Terms

• Computing methodologies

  • Modeling and simulation

    • Simulation support systems

      • Simulation environments

• Software and its engineering

  • Software organization and properties

    • Software functional properties

      • Formal methods

        • Model checking

• Theory of computation

  • Logic

    • Verification by model checking