Active continuous quality control

ABSTRACT

We present Active Continuous Quality Control (ACQC), a novel approach that employs incremental active automata learning technology periodically in order to infer evolving behavioral automata of complex applications accompanying the development process. This way we are able to closely monitor and steer the evolution of applications throughout their whole life-cycle with minimum manual effort. Key to this approach is to establish a stable level for comparison via an incrementally growing behavioral abstraction in terms of a user-centric communication alphabet: The letters of this alphabet, which may correspond to whole use cases, are intended to directly express the functionality from the user perspective. At the same time their choice allows one to focus on specific aspects, which establishes tailored abstraction levels on demand, which may be refined by adding new letters in the course of the systems evolution. This way ACQC does not only allow us to reveal serious bugs simply by inspecting difference views of the (tailored) models, but also to visually follow and control the effects of (intended) changes, which complements our model-checking-based quality control. All this will be illustrated along real-life scenarios that arose during the component-based development of a commercial editorial system.

References

1. F. Aarts, B. Jonsson, and J. Uijen. Generating Models of Infinite-State Communication Protocols using Regular Inference with Abstraction. In ICTSS'10, volume 6435 of LNCS, pages 188--204. Springer, 2010.
2. F. Aarts, J. Schmaltz, and F. Vaandrager. Inference and abstraction of the biometric passport. In ISoLA 2010, volume 6415 of LNCS, pages 673--686. Springer-Verlag, 2010.
3. D. Angluin. Learning Regular Sets from Queries and Counterexamples. Inf. Comput., 75(2):87--106, 1987.
4. O. Bauer, J. Neubauer, B. Steffen, and F. Howar. Reusing system states by active learning algorithms. In Eternal Systems, volume 255 of CCSE, pages 61--78. Springer-Verlag, 2012.
5. T. Berg, O. Grinchtein, B. Jonsson, M. Leucker, H. Raffelt, and B. Steffen. On the Correspondence Between Conformance Testing and Regular Inference. In FASE'05, volume 3442 of LNCS, pages 175--189. Springer, 2005.
6. S. Beydeda and V. Gruhn. Integrating white- and black-box techniques for class-level regression testing. In COMPSAC'01, pages 357--362, Washington, DC, USA, 2001. IEEE Computer Society.
7. G. Bossert, G. Hiet, and T. Henin. Modelling to Simulate Botnet Command and Control Protocols for the Evaluation of Network Intrusion Detection Systems. In SAR-SSI'11, pages 1--8. IEEE Computer Society, 2011.
8. Y. Chen, R. L. Probert, and H. Ural. Model-based regression test suite generation using dependence analysis. In A-MOST'07, pages 54--62, New York, NY, USA, 2007. ACM.
9. T. S. Chow. Testing Software Design Modeled by Finite-State Machines. IEEE Transactions on Software Engineering, 4(3):178--187, May 1978.
10. E. M. Clarke, O. Grumberg, and D. A. Peled. Model Checking. The MIT Press, 1999.
11. S. Gnesi and T. Margaria. Formal Methods for Industrial Critical Systems: A Survey of Applications. Wiley, 2012.
12. A. Hagerer, H. Hungar, O. Niese, and B. Steffen. Model generation by moderated regular extrapolation. In FASE'02, pages 80--95, London, UK, UK, 2002. Springer-Verlag.
13. G. T. Heineman and W. T. Councill, editors. Component-based software engineering: putting the pieces together. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2001.
14. J. E. Hopcroft. An n log n algorithm for minimizing states in a finite automaton. Technical report, Stanford, CA, USA, 1971.
15. F. Howar, O. Bauer, M. Merten, B. Steffen, and T. Margaria. The teachers' crowd: The impact of distributed oracles on active automata learning. In ISoLA 2012, Communications in Computer and Information Science, pages 232--247. Springer-Verlag, 2012.
16. F. Howar, B. Steffen, and M. Merten. From zulu to rers. In ISoLA 2010, volume 6415 of LNCS, pages 687--704. Springer-Verlag, 2010.
17. H. Hungar, O. Niese, and B. Steffen. Domain-specific optimization in automata learning. In Computer Aided Verification, volume 2725 of LNCS, pages 315--327. Springer-Verlag, 2003.
18. B. Jonsson, T. Margaria, G. Naeser, J. Nyström, and B. Steffen. Incremental requirement specification for evolving systems. Nord. J. Comput., 8(1):65--87, 2001.
19. S. Jörges, A.-L. Lamprecht, T. Margaria, I. Schaefer, and B. Steffen. A constraint-based variability modeling framework. International Journal on Software Tools for Technology Transfer, 14:511--530, 2012.
20. M. Karusseit and T. Margaria. Feature-based modelling of a complex, online-reconfigurable decision support service. In WWV '05, March 2005. ENTCS 1132.
21. B. Korel and A. M. Al-Yami. Automated regression test generation. SIGSOFT Softw. Eng. Notes, 23(2):143--152, Mar. 1998.
22. T. Margaria, O. Niese, H. Raffelt, and B. Steffen. Efficient test-based model generation for legacy reactive systems. In HLDVT '04, pages 95--100, Washington, DC, USA, 2004. IEEE Computer Society.
23. T. Margaria and B. Steffen. Agile IT: Thinking in User-Centric Models. In ISoLA 2009, volume 17 of Communications in Computer and Information Science, pages 490--502. Springer-Verlag, 2009.
24. M. Merten, B. Steffen, F. Howar, and T. Margaria. Next generation learnlib. In TACAS 2011, volume 6605 of LNCS, pages 220--223. Springer-Verlag, 2011.
25. M. Müller-Olm, D. Schmidt, and B. Steffen. Model-Checking - A Tutorial Introduction. In Static Analysis, volume 1694 of LNCS, pages 330--354. Springer-Verlag, 1999.
26. H. Raffelt, M. Merten, B. Steffen, and T. Margaria. Dynamic testing via automata learning. STTT, 11(4):307--324, 2009.
27. H. Raffelt, B. Steffen, and T. Berg. Learnlib: a library for automata learning and experimentation. In FMICS'05, pages 62--71. ACM, 2005.
28. H. Raffelt, B. Steffen, T. Berg, and T. Margaria. LearnLib: a framework for extrapolating behavioral models. International Journal on Software Tools for Technology Transfer, 11(5):393--407, 2009.
29. R. L. Rivest and R. E. Schapire. Inference of finite automata using homing sequences. Information and Computation, 103(2):299--347, 1993.
30. G. Rothermel, M. J. Harrold, and J. Dedhia. Regression test selection for C++software. Software Testing, Verification and Reliability, 10:2000, 1999.
31. M. Shahbaz, K. C. Shashidhar, and R. Eschbach. Iterative refinement of specification for component based embedded systems. In ISSTA'11, pages 276--286. ACM, 2011.
32. B. Steffen. Characteristic Formulae. In ICALP, volume 372 of LNCS, pages 723--732, 1989.
33. B. Steffen, A. Claßen, M. Klein, J. Knoop, and T. Margaria. The Fixpoint-Analysis Machine. In CONCUR'95, volume 962 of LNCS. Springer, 1995.
34. B. Steffen, F. Howar, and M. Merten. Introduction to active automata learning from a practical perspective. In Formal Methods for Eternal Networked Software Systems, volume 6659 of LNCS, pages 256--296. Springer-Verlag, 2011.
35. B. Steffen and H. Hungar. Behavior-based model construction. In Verification, Model Checking, and Abstract Interpretation, volume 2575 of LNCS, pages 5--19. Springer-Verlag, 2003.
36. B. Steffen, T. Margaria, V. Braun, and N. Kalt. Hierarchical Service Definition. Annual Review of Communications of the ACM, 51:847--856, 1997.
37. B. Steffen, T. Margaria, A. Claßen, and V. Braun. Incremental formalization: A key to industrial success. Software - Concepts and Tools, 17(2), 1996.
38. J. Tretmans. Model based testing with labelled transition systems. Formal methods and testing, pages 1--38, 2008.
39. J. Tretmans. Model-based testing and some steps towards test-based modelling. In M. Bernardo and V. Issarny, editors, Formal Methods for Eternal Networked Software Systems, volume 6659 of LNCS, pages 297--326. Springer-Verlag, 2011.
40. M. Utting, A. Pretschner, and B. Legeard. A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab., 22(5):297--312, Aug. 2012.
41. M. Vieira and D. Richardson. The role of dependencies in component-based systems evolution. In Proceedings of the International Workshop on Principles of Software Evolution, IWPSE '02, pages 62--65, New York, NY, USA, 2002. ACM.
42. M. Vigder. Component-based software engineering. chapter The evolution, maintenance, and management of component-based systems, pages 527--539. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2001.
43. Q. Wang, J. Shen, X. Wang, and H. Mei. A component-based approach to online software evolution: Research articles. J. Softw. Maint. Evol., 18(3):181--205, May 2006.
44. L. Xu, M. Dias, and D. Richardson. Generating regression tests via model checking. In COMPSAC'04, pages 336--341, Washington, DC, USA, 2004. IEEE Computer Society.

Index Terms

• General and reference

  • Cross-computing tools and techniques

    • Validation

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Empirical software validation

      • Formal software verification

      • Process validation

      • Software defect analysis

        • Software testing and debugging

  • Software organization and properties

    • Software functional properties

      • Formal methods

        • Model checking

• Theory of computation

  • Logic

    • Verification by model checking