Manoranjan Satpathy
ABSTRACT
The Simulink/Stateflow (SL/SF) environment from Math-works is becoming the de facto standard in industry for model based development of embedded control systems. Many commercial tools are available in the market for test case generation from SL/SF designs; however, we have observed that these tools do not achieve satisfactory coverage in cases when designs involve nonlinear blocks and Stateflow blocks occur deep inside the Simulink blocks.
The recent past has seen the emergence of several novel techniques for testing large C, C++ and Java programs; prominent among them are directed automated random testing (DART), hybrid concolic testing and feedback-directed random testing. We believe that some of these techniques could be lifted to testing of SL/SF based designs; REDIRECT (RandomizEd DIRECted Testing), the proposed testing method of this paper, is an attempt towards this direction. Specifically, REDIRECT uses a careful combination of the above techniques, and in addition, the method uses a set of pattern-guided heuristics for tackling nonlinear blocks. A prototype tool has been developed and the tool has been applied to many industrial strength case studies. Our experiments indicate that a careful choice of heuristics and certain combinations of random and directed testing achieve better coverages as compared to the existing commercial tools. 1
- Applied Dynamics International. BEACON for Simulink/Stateflow, http://www.adi.comGoogle Scholar
- R. Alur, A.Kanade, S.Ramesh, and K.C. Shashidhar. Symbolic Analysis for Improving Coverage of Simulink/Stateflow Models, In International Conference on Embedded Software, 2008, To appear. Google ScholarDigital Library
- R. Cleaveland, S.A. Smolka, and S.T. Sims. An Instrumentation-Based Approach to Controller Model Validation, Available at http://aswsd.ucsd.edu/2006/pdfs/Smolka-vm-slides.pdfGoogle Scholar
- ETSI. ETS 300 406: Methods for Testing and Specification (MTS); Protocol and profile conformance testing specifications; Standardization Methodology, European Telecommunication Standard, 1995.Google Scholar
- A. Gadkari, A. Yeolekar, J. Suresh, S. Ramesh, S. Mohalik, and K.C. Shashidhar. AutoMOTGen: Automatic Model Oriented Test Generator for Embedded Control Systems, In Proc. of the CAV'08, LNCS Volume 5123, pages 204--208, 2008. Google ScholarDigital Library
- P. Godefroid, N. Klarlund, and K. DART: Directed Automated Random Testing, In Proc. of the PLDI'05, Chicago, pp. 213--223, 2005. Google ScholarDigital Library
- G. Hamon, L. de Moura, and J. Rushby. Automated Test Generation with SAL, CSL Technical Note, January 2005.Google Scholar
- ISO. OSI Conformance Testing Methodology and Framework- ISO 9646, 1998.Google Scholar
- A. Agung Julius, G. Fainekos, M. Anand, I. Lee, and G.J. Pappas. Robust test generation and coverage for hybrid systems, In LNCS Volume 4416, Springer, pages 329--342, 2007. Google ScholarDigital Library
- R. Majumdar, and K. Sen. Hybrid Concolic Testing, In Proc. of the ICSE, Minneaplois, pages 416--426, 2007. Google ScholarDigital Library
- The Mathworks, http://www.mathworks.comGoogle Scholar
- The Mathworks, Simulink Design verifier, http://www.mathworks.comGoogle Scholar
- S.P Miller, E.A. Anderson, L.G. Wagner, M.W. Whalen, and M.P.E. Heimdahl. Formal Verification of Flight Control Software, In Proc. of the AIAA Guidance, Navigation and Control Conference and Exhibit, San Francisco, pages 1--16, August 2005.Google ScholarCross Ref
- J. Offutt, S. Liu, A. Abdurazik, and P. Ammann. Generating Test Data from State Based Specifications, Software Testing, Verification and Reliability, John Wiley, 13(1): 25--53, March 2003.Google ScholarCross Ref
- C. Pacheco, S.K. Lahiri, M. Ernst, T. Ball. Feedback-directed Random Test Generation, In Proc. of the ICSE, Minneapolis, pages 75--84, May 2007. Google ScholarDigital Library
- Reactis: http://www.reactive-systems.comGoogle Scholar
- Reactive Systems. Model Based Testing and Validation with Reactis, Reactive Systems Inc., http://www.reactive-systems.comGoogle Scholar
- D.J. Richardson, A. Leif Aha, and T.O. O'Malley. Specification-based Test Oracles for Reactive Systems, In Proc. of ICSE, Melbourne, pp. 105--118, 1992. Google ScholarDigital Library
- SRI International. SAL home page http://sal.csl.sri.comGoogle Scholar
- S. Sims, and D.C DuVarney. Experience Report: The Reactis Validation Tool, In Proc. of the ACM International Conference on Functional Programming, Freiburg, pages 137--139, October 2007. Google ScholarDigital Library
- STB. Safety Test Builder, Automatic Test Generation for Simulink/Stateflow, TIN Software.Google Scholar
- T-Vec. T-Vec Tester for Simulink, http://www.t-vec.comGoogle Scholar
- H. Zhu, P.A.V. Hall, and J.H.R. May. Software Unit Test Coverage and Adequacy, ACM Computing Surveys, 29(4):366--427, 1997. Google ScholarDigital Library
Index Terms
- Randomized directed testing (REDIRECT) for Simulink/Stateflow models
Recommendations
SimCoTest: a test suite generation tool for simulink/stateflow controllers
ICSE '16: Proceedings of the 38th International Conference on Software Engineering CompanionWe present SimCoTest, a tool to generate small test suites with high fault revealing ability for Simulink/Stateflow controllers. SimCoTest uses meta-heuristic search to (1) maximize the likelihood of presence of specific failure patterns in output ...
Assessing model-based testing: an empirical study conducted in industry
ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software EngineeringWe compare manual testing without any automation performed by a tester at a software company with model-based testing (MBT) performed by a tester at a research center.
The system under test (SUT), of which two different versions were tested by each ...
Smart Unit Testing Framework
ISSREW '12: Proceedings of the 2012 IEEE 23rd International Symposium on Software Reliability Engineering WorkshopsUnit testing(UT) is an important step in ensuring the quality of software. Considerable effort is spent in unit testing. There are several frameworks to help with UT. Some common frameworks are Cunit, Junit, Nunit etc. All of these have very similar ...
Comments