Search-based testing of service level agreements

ABSTRACT

The diffusion of service oriented architectures introduces the need for novel testing approaches. On the one side, testing must be able to identify failures in the functionality provided by service. On the other side, it needs to identify cases in which the Service Level Agreement (SLA) negotiated between the service provider and the service consumer is not met. This would allow the developer to improve service performances, where needed, and the provider to avoid promising Quality of Service (QoS) levels that cannot be guaranteed. This paper proposes the use of Genetic Algorithms to generate inputs and configurations for service-oriented systems that cause SLA violations. The approach has been implemented in a tool and applied to an audio processing workflow and to a service for chart generation. In both cases, the approach was able to produce test data able to violate some QoS constraints.

References

1. A. Andrieux, K. Czajkowski, A. Dan, K. Keahey, H. Ludwig, J. Pruyne, J. Rofrano, S. Tuecke, and M. Xu. Web Services Agreement Specification (WS-Agreement). http://www.gridforum.org/Meetings/GGF11/Documents/draft-ggf-graap-agreement. pdf last accessed: March 9th 2007.
2. X. Bai, W. Dong, W.-T. Tsai, and Y. Chen. In WSDL-Based Automatic Test Case Generation for Web Services Testing pages 215--220, Los Alamitos, CA, USA, 2005. IEEE Computer Society.
3. L. Baresi, C. Ghezzi, and S. Guinea. Smart monitors for composed services. In Proc. 2nd International Conference on Service Oriented Computing (ICSOC'04) New York, USA, Nov 2004. ACM.
4. A. Bertolino and A. Polini. The audition framework for testing Web services interoperability. In 31st EUROMICRO Conference on Software Engineering and Advanced Applications (EUROMICRO-SEAA 2005), 30 August - 3 September 2005, Porto, Portugal pages 134--142, 2005.
5. L. C. Briand, Y. Labiche, and M. Shousha. Stress testing real-time systems with genetic algorithms. In Genetic and Evolutionary Computation Conference, GECCO 2005, Proceedings, Washington DC, USA, June 25-29, 2005 pages 1021--1028, 2005.
6. M. Bruno, G. Canfora, M. DiPenta, G. Esposito, and V. Mazza. Using test cases as contract to ensure service compliance across releases. In Service-Oriented Computing - ICSOC 2005, Third International Conference, Amsterdam, The Netherlands, December 12-15, 2005, Proceedings pages 87--100, 2005.
7. G. Canfora and M. DiPenta. Testingservicesand service-centric systems:Challenges and opportunities. IT Professional 8(2):10--17, 2006.
8. G. Canfora, M. DiPenta, R. Esposito, F. Perfetto, and M. L. Villani. Service composition (re) binding driven by application-specific qos. In Service-Oriented Computing - ICSOC 2006, 4th International Conference, Chicago, IL, USA, December 4-7, 2006, Proceedings pages 141--152, 2006.
9. G. Canfora, M. DiPenta, R. Esposito, and M. L. Villani. An Approach for QoS-aware Service Composition based on Genetic Algorithms. In Proc. of the Genetic and Computation Conference (GECCO'05) pages 1069. 1075, Washington, USA, June 2005. ACM.
10. J. Cardoso. Quality of Service and Semantic Composition of Workflows PhDthesis, Univ. of Georgia, 2002.
11. C. Fu, B. G. Ryder, A. Milanova, andD. Wonnacott. Testing of java web services for robustness. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2004, Boston, Massachusetts, USA, July 11-14, 2004 pages 23--34, 2004.
12. V. Garousi, L. C. Briand, and Y. Labiche. Traffic-aware stress testing of distributed systems based on UML models. In 28th International Conference on Software Engineering (ICSE 2006), Shanghai, China, May 20-28, 2006 pages 391--400, 2006.
13. M. Harman, L. Hu, R. M. Hierons, J. Wegener, H. Sthamer, A. Baresel, and M. Roper. Testability transformation. IEEE Trans. Software Eng. 30(1):3--16, 2004.
14. E. Martin, S. Basu, and T. Xie. Automated robustness testing of web services. In Proceedings of the 4th International Workshop on SOA And Web Services Best Practices (SOAWS 2006) October 2006.
15. P. McMinn. Search-based software test data generation: a survey. Softw. Test., Verif. Reliab. 14(2):105--156, 2004.
16. J. O. utt and W. Xu. Generating test cases for web services using data perturbation. SIGSOFT Softw. Eng. Notes - SECTION: Workshop on testing, analysis and verification of web services (TAV-WEB) 29(5):1--10, 2004.
17. M. Paolucci, T. Kawamura, T. R. Payne, and K. Sycara. Semantic matching of web services capabilities. In Proceedings of the first International Semantic Web Conference (ISWC 2002) volume 2348 of Lecture Notes on Computer Science pages 333--347. Springer-Verlag, June 2002.
18. W.-T. Tsai, R. J. Paul, Y. Wang, C. Fan, and D. Wang. Extending WSDL to facilitate Web services testing. In 7th IEEE International Symposium on High-Assurance Systems Engineering (HASE 2002), 23-25 October 2002, Tokyo, Japan pages 171--172, 2002.
19. M. Turner, D. Budgen, and P. Brereton. Turning software into aservice. IEEE Computer 36(10):38--44, 2003.
20. J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information & Software Technology 43(14):841--854, 2001.
21. J. Wegener and M. Grochtmann. Testing temporal correctness of real-time systems by means of genetic algorithms. In Quality Week 1997.
22. L. Zeng, B. Benatallah, A. H. H. Ngu, M. Dumas, J. Kalagnanam, and H. Chang. QoS-aware middleware for web services composition. IEEE Transactions on Software Engineering 30(5), May 2004.

Index Terms

• Software and its engineering