The increasing abundance of large high-quality datasets, combined with significant technical advances over the last several decades have made machine learning into a major tool employed across a broad array of tasks including vision, language, finance, and security. However, success has been accompanied with important new challenges: many applications of machine learning are adversarial in nature. Some are adversarial because they are safety critical, such as autonomous driving. An adversary in these applications can be a malicious party aimed at causing congestion or accidents, or may even model unusual situations that expose vulnerabilities in the prediction engine. Other applications are adversarial because their task and/or the data they use are. For example, an important class of problems in security involves detection, such as malware, spam, and intrusion detection. The use of machine learning for detecting malicious entities creates an incentive among adversaries to evade detection by changing their behavior or the content of malicius objects they develop. The field of adversarial machine learning has emerged to study vulnerabilities of machine learning approaches in adversarial settings and to develop techniques to make learning robust to adversarial manipulation. This book provides a technical overview of this field. After reviewing machine learning concepts and approaches, as well as common use cases of these in adversarial settings, we present a general categorization of attacks on machine learning. We then address two major categories of attacks and associated defenses: decision-time attacks, in which an adversary changes the nature of instances seen by a learned model at the time of prediction in order to cause errors, and poisoning or training time attacks, in which the actual training dataset is maliciously modified. In our final chapter devoted to technical content, we discuss recent techniques for attacks on deep learning, as well as approaches for improving robustness of deep neural networks. We conclude with a discussion of several important issues in the area of adversarial learning that in our view warrant further research. Given the increasing interest in the area of adversarial machine learning, we hope this book provides readers with the tools necessary to successfully engage in research and practice of machine learning in adversarial settings.
Cited By
- Machado G, Silva E and Goldschmidt R (2021). Adversarial Machine Learning in Image Classification: A Survey Toward the Defender’s Perspective, ACM Computing Surveys, 55:1, (1-38), Online publication date: 31-Jan-2023.
- Estornell A, Das S, Liu Y and Vorobeychik Y Group-Fair Classification with Strategic Agents Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, (389-399)
- Deldjoo Y, Noia T and Merra F (2021). A Survey on Adversarial Recommender Systems, ACM Computing Surveys, 54:2, (1-38), Online publication date: 31-Mar-2022.
- Lee D and Verma R Adversarial Machine Learning for Text Proceedings of the Sixth International Workshop on Security and Privacy Analytics, (33-34)
- Luo Z, Zhao S, Lu Z, Sagduyu Y and Xu J Adversarial machine learning based partial-model attack in IoT Proceedings of the 2nd ACM Workshop on Wireless Security and Machine Learning, (13-18)
- Tong L, Li B, Hajaj C, Xiao C, Zhang N and Vorobeychik Y Improving robustness of ML classifiers against realizable evasion attacks using conserved features Proceedings of the 28th USENIX Conference on Security Symposium, (285-302)
- Shi Y, Davaslioglu K and Sagduyu Y Generative Adversarial Network for Wireless Signal Spoofing Proceedings of the ACM Workshop on Wireless Security and Machine Learning, (55-60)
- Hajaj C, Yu S, Joveski Z, Guo Y and Vorobeychik Y Adversarial Coordination on Social Networks Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, (1515-1523)
- Davaslioglu K and Sagduyu Y Trojan Attacks on Wireless Signal Classification with Adversarial Machine Learning 2019 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN), (1-6)
- Jun K, Li L, Ma Y and Zhu X Adversarial attacks on stochastic bandits Proceedings of the 32nd International Conference on Neural Information Processing Systems, (3644-3653)
Index Terms
- Adversarial Machine Learning
Recommendations
Adversarial machine learning
AISec '11: Proceedings of the 4th ACM workshop on Security and artificial intelligenceIn this paper (expanded from an invited talk at AISEC 2010), we discuss an emerging field of study: adversarial machine learning---the study of effective machine learning techniques against an adversarial opponent. In this paper, we: give a taxonomy for ...
Defending against adversarial machine learning attacks using hierarchical learning: A case study on network traffic attack classification
AbstractMachine learning is key for automated detection of malicious network activity to ensure that computer networks and organizations are protected against cyber security attacks. Recently, there has been growing interest in the domain of ...