This is the first entry-level book on algorithmic (also known as automatic) differentiation (AD), providing fundamental rules for the generation of first- and higher-order tangent-linear and adjoint code. The author covers the mathematical underpinnings as well as how to apply these observations to real-world numerical simulation programs. Readers will find many examples and exercises, including hints to solutions. Also included are the prototype AD tools dco and dcc for use with the examples and exercises. The derivative code compiler dcc provides first- and higher-order tangent-linear and adjoint modes for a limited subset of C/C++. In addition, readers will have access to a supplementary website containing sources of all software discussed in the book, additional exercises and comments on their solutions (growing over the coming years), links to other sites on AD, and errata. Audience: This book is intended for undergraduate and graduate students in computational science, engineering, and finance as well as applied mathematics and computer science. It will provide researchers and developers at all levels with an intuitive introduction to AD.
Cited By
- Deussen J and Naumann U (2023). Subdomain separability in global optimization, Journal of Global Optimization, 86:3, (573-588), Online publication date: 1-Jul-2023.
- Cherubin S and Agosta G (2020). Tools for Reduced Precision Computation, ACM Computing Surveys, 53:2, (1-35), Online publication date: 31-Mar-2021.
- Kofman E, Fernández J and Marzorati D (2021). Compact sparse symbolic Jacobian computation in large systems of ODEs, Applied Mathematics and Computation, 403:C, Online publication date: 15-Aug-2021.
- Mitusch S, Funke S and Kuchta M (2021). Hybrid FEM-NN models, Journal of Computational Physics, 446:C, Online publication date: 1-Dec-2021.
- Georgiou S, Rizou S and Spinellis D (2019). Software Development Lifecycle for Energy Efficiency, ACM Computing Surveys, 52:4, (1-33), Online publication date: 31-Jul-2020.
- Jonasson K, Sigurdsson S, Yngvason H, Ragnarsson P and Melsted P (2020). Algorithm 1005, ACM Transactions on Mathematical Software, 46:1, (1-20), Online publication date: 31-Mar-2020.
- Gray J, Hwang J, Martins J, Moore K and Naylor B (2019). OpenMDAO, Structural and Multidisciplinary Optimization, 59:4, (1075-1104), Online publication date: 1-Apr-2019.
- Hückelheim J, Hovland P, Strout M and Müller J (2019). Reverse-mode algorithmic differentiation of an OpenMP-parallel compressible flow solver, International Journal of High Performance Computing Applications, 33:1, (140-154), Online publication date: 1-Jan-2019.
- Naumann U (2019). Adjoint Code Design Patterns, ACM Transactions on Mathematical Software, 45:3, (1-32), Online publication date: 30-Sep-2019.
- Schanen M, Maldonado D and Anitescu M A Framework for Distributed Approximation of Moments with Higher-Order Derivatives Through Automatic Differentiation Computational Science – ICCS 2019, (251-260)
- Menon H, Lam M, Osei-Kuffuor D, Schordan M, Lloyd S, Mohror K and Hittinger J ADAPT Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, (1-13)
- Menon H, Lam M, Osei-Kuffuor D, Schordan M, Lloyd S, Mohror K and Hittinger J ADAPT Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, (1-13)
- Merriënboer B, Moldovan D and Wiltschko A Tangent Proceedings of the 32nd International Conference on Neural Information Processing Systems, (6259-6268)
- Günther S, Gauger N and Schroder J (2018). A non-intrusive parallel-in-time adjoint solver with the XBraid library, Computing and Visualization in Science, 19:3-4, (85-95), Online publication date: 1-Jul-2018.
- Meyer X, Chopard B and Salamin N Scheduling Finite Difference Approximations for DAG-Modeled Large Scale Applications Proceedings of the Platform for Advanced Scientific Computing Conference, (1-12)
- Khan K, Watson H and Barton P (2017). Differentiable McCormick relaxations, Journal of Global Optimization, 67:4, (687-729), Online publication date: 1-Apr-2017.
- Vassiliadis V, Riehme J, Deussen J, Parasyris K, Antonopoulos C, Bellas N, Lalis S and Naumann U Towards automatic significance analysis for approximate computing Proceedings of the 2016 International Symposium on Code Generation and Optimization, (182-193)
- Sluşanschi E and Dumitrel V (2016). ADiJaC -- Automatic Differentiation of Java Classfiles, ACM Transactions on Mathematical Software, 43:2, (1-33), Online publication date: 2-Sep-2016.
- Lotz J, Schwalbach M and Naumann U (2016). A Case Study in Adjoint Sensitivity Analysis of Parameter Calibration, Procedia Computer Science, 80:C, (201-211), Online publication date: 1-Jun-2016.
- Safiran N, Lotz J and Naumann U (2016). Algorithmic Differentiation of Numerical Methods, Procedia Computer Science, 80:C, (2231-2235), Online publication date: 1-Jun-2016.
- Naumann U, Lotz J, Leppkes K and Towara M (2015). Algorithmic Differentiation of Numerical Methods, ACM Transactions on Mathematical Software, 41:4, (1-21), Online publication date: 26-Oct-2015.
- Safiran N, Lotz J and Naumann U (2015). Second-order Tangent Solvers for Systems of Parameterized Nonlinear Equations, Procedia Computer Science, 51:C, (231-238), Online publication date: 1-Sep-2015.
- Lotz J, Naumann U, Hannemann-Tamas R, Ploch T and Mitsos A (2015). Higher-order Discrete Adjoint ODE Solver in C++ for Dynamic Optimization, Procedia Computer Science, 51:C, (256-265), Online publication date: 1-Sep-2015.
- Elsheikh A (2015). An equation-based algorithmic differentiation technique for differential algebraic equations, Journal of Computational and Applied Mathematics, 281:C, (135-151), Online publication date: 1-Jun-2015.
- Towara M, Schanen M and Naumann U (2015). MPI-Parallel Discrete Adjoint OpenFOAM, Procedia Computer Science, 51:C, (19-28), Online publication date: 1-Sep-2015.
- Eriksson-Bique S, Polishchuk V and Sysikaski M Optimal Geometric Flows via Dual Programs Proceedings of the thirtieth annual symposium on Computational geometry, (100-109)
- Hascoet L and Pascual V (2013). The Tapenade automatic differentiation tool, ACM Transactions on Mathematical Software, 39:3, (1-43), Online publication date: 1-Apr-2013.
- Gebremedhin A, Nguyen D, Patwary M and Pothen A (2013). ColPack, ACM Transactions on Mathematical Software, 40:1, (1-31), Online publication date: 1-Sep-2013.
- Lotz J, Naumann U, Sagebaum M and Schanen M Discrete adjoints of PETSc through dco/c++ and adjoint MPI Proceedings of the 19th international conference on Parallel Processing, (497-507)
- Förster M and Naumann U Solving a least-squares problem with algorithmic differentiation and OpenMP Proceedings of the 19th international conference on Parallel Processing, (763-774)
- Minh B, Förster M and Naumann U Towards tangent-linear GPU programs using OpenACC Proceedings of the 4th Symposium on Information and Communication Technology, (27-34)
- Schanen M and Naumann U A wish list for efficient adjoints of one-sided MPI communication Proceedings of the 19th European conference on Recent Advances in the Message Passing Interface, (248-257)
Index Terms
- The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation
Recommendations
Learning mathematics with recursive computer programs
Proceedings of the SIGCSE-SIGCUE joint symposium on Computer science educationRecursion is a powerful idea*—with correspondingly powerful implications for learning and teaching mathematics. Computer scientists have previously pointed out that the use of recursion often permits more lucid and concise descriptions of algorithms [1];...