Michael Hind
Dennis Wei
ABSTRACT
Artificial intelligence systems are being increasingly deployed due to their potential to increase the efficiency, scale, consistency, fairness, and accuracy of decisions. However, as many of these systems are opaque in their operation, there is a growing demand for such systems to provide explanations for their decisions. Conventional approaches to this problem attempt to expose or discover the inner workings of a machine learning model with the hope that the resulting explanations will be meaningful to the consumer. In contrast, this paper suggests a new approach to this problem. It introduces a simple, practical framework, called Teaching Explanations for Decisions (TED), that provides meaningful explanations that match the mental model of the consumer. We illustrate the generality and effectiveness of this approach with two different examples, resulting in highly accurate explanations with no loss of prediction accuracy for these two examples.
- Yessenalina Ainur, Yejin Choi, and Claire Cardie. 2010. Automatically Generating Annotator Rationales to Improve Sentiment Classification. In Proceedings of the ACL 2010 Conference Short Papers. 336--341. Google Scholar
Digital Library
- Osbert Bastani, Carolyn Kim, and Hamsa Bastani. 2018. Interpreting Blackbox Models via Model Extraction. arXiv preprint arXiv:1705.08504 (2018).Google Scholar
- Or Biran and Courtenay Cotton. 2017. Explanation and Justification in Machine Learning: A Survey. In IJCAI-17 Workshop on Explainable AI (XAI).Google Scholar
- Alex Campolo, Madelyn Sanfilippo Meredith Whittaker, and Kate Crawford. 2017. 2017 Annual Report. Technical Report. AI NOW.Google Scholar
- Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission. In Proc. ACM SIGKDD Int. Conf. Knowl. Disc. Data Min. Sydney, Australia, 1721--1730. Google ScholarDigital Library
- Sanjeeb Dash, Oktay Gunluk, and Dennis Wei. 2018. Boolean Decision Rules via Column Generation. In Advances in Neural Information Processing Systems. Google ScholarDigital Library
- Amit Dhurandhar, Vijay Iyengar, Ronny Luss, and Karthikeyan Shanmugam. 2017. A Formal Framework to Characterize Interpretability of Procedures. In Proc. ICML Workshop Human Interp. Mach. Learn. Sydney, Australia, 1--7.Google Scholar
- Jeff Donahue and Kristen Grauman. 2011. Annotator Rationales for Visual Recognition. In ICCV. Google ScholarDigital Library
- Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. In https://arxiv.org/abs/1702.08608v2.Google Scholar
- Finale Doshi-Velez, Mason Kortz, Ryan Budish, Chris Bavitz, Sam Gershman, David O'Brien, Stuart Schieber, James Waldo, David Weinberger, and Alexandra Wood. 2017. Accountability of AI Under the Law: The Role of Explanation. CoRR, Vol. abs/1711.01134 (2017). arxiv: 1711.01134 http://arxiv.org/abs/1711.01134Google Scholar
- Kun Duan, Devi Parikh, David Crandall, and Kristen Grauman. 2012. Discovering Localized Attributes for Fine-grained Recognition. In CVPR.Google Scholar
- FICO. 2018. Explainable Machine Learning Challenge. https://community.fico.com/s/explainable-machine-learning-challenge?tabset-3158a=2Google Scholar
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning .MIT Press. http://www.deeplearningbook.org. Google ScholarDigital Library
- Bryce Goodman and Seth Flaxman. 2016. EU Regulations on Algorithmic Decision-Making and a `Right to Explanation'. In Proc. ICML Workshop Human Interp. Mach. Learn. New York, NY, 26--30.Google Scholar
- Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schiele, and Trevor Darrell. 2016. Generating Visual Explanations. In European Conference on Computer Vision.Google Scholar
- Been Kim. 2017. Tutorial on Interpretable machine learning. "http://people.csail.mit.edu/beenkim/papers/BeenK_FinaleDV_ICML2017_tutorial.pdf"Google Scholar
- Been Kim, Dmitry M. Malioutov, Kush R. Varshney, and Adrian Weller (Eds.). 2017. 2017 ICML Workshop on Human Interpretability in Machine Learning. https://arxiv.org/abs/1708.02666v1Google Scholar
- Been Kim, Kush R. Varshney, and Adrian Weller (Eds.). 2018. 2018 Workshop on Human Interpretability in Machine Learning. https://sites.google.com/view/whi2018/homeGoogle Scholar
- Todd Kulesza, Simone Stumpf, Margaret Burnett, Sherry Yang, Irwin Kwan, and Weng-Keen Wong. 2013. Too Much, Too Little, or Just Right? Ways Explanations Impact End Users' Mental Models. In Proc. IEEE Symp. Vis. Lang. Human-Centric Comput. San Jose, CA, 3--10.Google ScholarCross Ref
- Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In EMNLP.Google Scholar
- Zachary C Lipton. 2016. The mythos of model interpretability. In ICML Workshop on Human Interpretability of Machine Learning.Google Scholar
- Tania Lombrozo. 2007. Simplicity and probability in causal explanation. Cognitive Psychol., Vol. 55, 3 (Nov. 2007), 232--257.Google ScholarCross Ref
- Scott Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances of Neural Inf. Proc. Systems. Google ScholarDigital Library
- T. McDonnell, M. Lease, M. Kutlu, and T. Elsayed. 2016. Why Is That Relevant? Collecting Annotator Rationales for Relevance Judgments. In Proc. AAAI Conf. Human Comput. Crowdsourc.Google Scholar
- Tim Miller. 2017. Explanation in Artificial Intelligence: Insights from the Social Sciences. arXiv preprint arXiv:1706.07269 (June 2017).Google Scholar
- Tim Miller, Piers Howe, and Liz Sonenberg. 2017. Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences. In Proc. IJCAI Workshop Explainable Artif. Intell. Melbourne, Australia.Google Scholar
- Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. 2017. Methods for interpreting and understanding deep neural networks. Digital Signal Processing (2017).Google Scholar
- P. Peng, Y. Tian, T. Xiang, Y. Wang, and T. Huang. 2016. Joint Learning of Semantic and Latent Attributes. In ECCV 2016, Lecture Notes in Computer Science, Vol. 9908.Google Scholar
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proc. ACM SIGKDD Int. Conf. Knowl. Disc. Data Min. San Francisco, CA, 1135--1144. Google ScholarDigital Library
- A. D. Selbst and J. Powles. 2017. Meaningful Information and the Right to Explanation. Int. Data Privacy Law, Vol. 7, 4 (Nov. 2017), 233--242.Google ScholarCross Ref
- Qiang Sun and Gerald DeJong. 2005. Explanation-Augmented SVM: an Approach to Incorporating Domain Knowledge into SVM Learning. In 22nd International Conference on Machine Learning. Google ScholarDigital Library
- James Vacca. 2018. A Local Law in relation to automated decision systems used by agencies. Technical Report. The New York City Council.Google Scholar
- Kush R. Varshney. 2016. Engineering Safety in Machine Learning. In Information Theory and Applications Workshop.Google Scholar
- S. Wachter, B. Mittelstadt, and L. Floridi. 2017. Transparent, explainable, and accountable AI for robotics. Science Robotics, Vol. 2 (May 2017). Issue 6.Google Scholar
- Sandra Wachter, Brent Mittelstadt, and Luciano Floridi. 2017. Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation. Int. Data Privacy Law, Vol. 7, 2 (May 2017), 76--99.Google Scholar
- Omar F. Zaidan and Jason Eisner. 2007. Using 'annotator rationales' to improve machine learning for text categorization. In In NAACL-HLT. 260--267.Google Scholar
- Omar F. Zaidan and Jason Eisner. 2008. Modeling Annotators: A Generative Approach to Learning from Annotator Rationales. In Proceedings of EMNLP 2008. 31--40. Google ScholarDigital Library
- Ye Zhang, Iain James Marshall, and Byron C. Wallace. 2016. Rationale-Augmented Convolutional Neural Networks for Text Classification. In Conference on Empirical Methods in Natural Language Processing (EMNLP).Google Scholar
Index Terms
- TED: Teaching AI to Explain its Decisions
Recommendations
Counterfactual Explainable Recommendation
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge ManagementBy providing explanations for users and system designers to facilitate better understanding and decision making, explainable recommendation has been an important research problem. In this paper, we propose Counterfactual Explainable Recommendation (...
Scientific Exploration and Explainable Artificial Intelligence
AbstractModels developed using machine learning are increasingly prevalent in scientific research. At the same time, these models are notoriously opaque. Explainable AI aims to mitigate the impact of opacity by rendering opaque models transparent. More ...
The Privacy Issue of Counterfactual Explanations: Explanation Linkage Attacks
Black-box machine learning models are used in an increasing number of high-stakes domains, and this creates a growing need for Explainable AI (XAI). However, the use of XAI in machine learning introduces privacy risks, which currently remain largely ...
Comments