Search-based test and improvement of machine-learning-based anomaly detection systems

ABSTRACT

Machine-learning-based anomaly detection systems can be vulnerable to new kinds of deceptions, known as training attacks, which exploit the live learning mechanism of these systems by progressively injecting small portions of abnormal data. The injected data seamlessly swift the learned states to a point where harmful data can pass unnoticed. We focus on the systematic testing of these attacks in the context of intrusion detection systems (IDS). We propose a search-based approach to test IDS by making training attacks. Going a step further, we also propose searching for countermeasures, learning from the successful attacks and thereby increasing the resilience of the tested IDS. We evaluate our approach on a denial-of-service attack detection scenario and a dataset recording the network traffic of a real-world system. Our experiments show that our search-based attack scheme generates successful attacks bypassing the current state-of-the-art defences. We also show that our approach is capable of generating attack patterns for all configuration states of the studied IDS and that it is capable of providing appropriate countermeasures. By co-evolving our attack and defence mechanisms we succeeded at improving the defence of the IDS under test by making it resilient to 49 out of 50 independently generated attacks.

References

1. 4SIC. {n.d.}. 4SICS geek lounge SCADA network capture. Retrieved January 25, 2019 from http://www.netresec.com/?page=PCAP4SICS
2. F. A. A. Alseiari and Z. Aung. 2015. Real-time anomaly-based distributed intrusion detection systems for advanced Metering Infrastructure utilizing stream data mining. In 2015 International Conference on Smart Grid and Clean Energy Technologies (ICSGCE). 148–153.
3. Dennis Appelt, Cu D. Nguyen, Annibale Panichella, and Lionel C. Briand. 2018. A Machine-Learning-Driven Evolutionary Approach for Testing Web Application Firewalls. IEEE Trans. Reliability 67, 3 (2018), 733–757.
4. Marco Barreno, Blaine Nelson, Russell Sears, Anthony D. Joseph, and J. D. Tygar. 2006. Can Machine Learning Be Secure?. In Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security (ASIACCS ’06). ACM, 16–25.
5. R. Berthier, D. I. Urbina, A. A. Cárdenas, M. Guerrero, U. Herberg, J. G. Jetcheva, D. Mashima, J. H. Huh, and R. B. Bobba. 2014. On the practicality of detecting anomalies with encrypted traffic in AMI. In 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm). 890–895.
6. A. L. Buczak and E. Guven. 2016. A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection. IEEE Communications Surveys Tutorials 18, 2 (2016), 1153–1176.
7. Yixin Chen and Li Tu. 2007. Density-based Clustering for Real-time Stream Data. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’07). ACM, 133–142. 1281192.1281210
8. Gilbert Hendry and Shanchieh Yang. 2008.
9. Intrusion signature creation via clustering anomalies. Proceedings of SPIE - The International Society for Optical Engineering 6973 (03 2008).
10. Incapsula. 2014.
11. What DDoS Attacks Really Cost Business? Retrieved 25 January 2019 from https://lp.incapsula.com/rs/incapsulainc/images/eBook-DDoSImpactSurvey.pdf
12. Aleksandar Milenkoski, Marco Vieira, Samuel Kounev, Alberto Avritzer, and Bryan Payne. 2015. Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices. Comput. Surveys 48 (09 2015), 12:1–. 1145/2808691
13. Steve Muller, Jean Lancrenon, Carlo Harpes, Yves Le Traon, Sylvain Gombault, and Jean-Marie Bonnin. 2018. A training-resistant anomaly detection system. Computers & Security 76 (03 2018).
14. Mohammad Sazzadul Hoque. 2012. An Implementation of Intrusion Detection System Using Genetic Algorithm. International Journal of Network Security & Its Applications 4, 2 (Mar 2012), 109–120.
15. K. G. Srinivasa, S. Chandra, S. Kajaria, and S. Mukherjee. 2011. IGIDS: Intelligent intrusion detection system using genetic algorithms. In 2011 World Congress on Information and Communication Technologies. 852–857.
16. Dusan Stevanovic and Natalija Vlajic. 2014. Next Generation Application-Layer DDoS Defences: Applying the Concepts of Outlier Detection in Data Streams with Concept Drift. In Proceedings of the 2014 13th International Conference on Machine Learning and Applications (ICMLA ’14). IEEE Computer Society, Washington, DC, USA, 456–462.
17. L. Tomlin, M. R. Farnam, and S. Pan. 2016. A clustering approach to industrial network intrusion detection. In INSuRECon ’16.
18. Omer Tripp, Omri Weisman, and Lotem Guy. 2013.
19. Finding Your Way in the Testing Jungle: A Learning Approach to Web Security Testing. In ISSTA ’13 (ISSTA 2013). ACM, New York, NY, USA, 347–357.
20. 2483776
21. Komkrit Udommanetanakit, Thanawin Rakthanmanon, and Kitsana Waiyamai. 2007. E-Stream: Evolution-Based Technique for Stream Clustering. In Proceedings of the 3rd International Conference on Advanced Data Mining and Applications (ADMA ’07). Springer-Verlag, 605–615. 3- 540- 73871- 8_58
22. David A. Wagner and Paolo Soto. 2002. Mimicry attacks on host-based intrusion detection systems. In Proceedings of the 9th ACM Conference on Computer and Communications Security, CCS 2002, Washington, DC, USA, November 18-22, 2002.
23. 255–264.
24. Ting-Fang Yen and Michael K. Reiter. 2008.
25. Traffic Aggregation for Malware Detection. In Detection of Intrusions and Malware, and Vulnerability Assessment, Diego Zamboni (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 207–227.
26. Shi Zhong, Taghi M. Khoshgoftaar, and Shyarn V. Nath. 2005.

Index Terms

• Security and privacy

  • Intrusion/anomaly detection and malware mitigation

    • Intrusion detection systems

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging