Browse Books

Reviewer: Renato Leone

Finally, a book devoted to dynamic programming and written using the language of operations research (OR)! This beautiful book fills a gap in the libraries of OR specialists and practitioners. The book is subdivided into three parts. Part 1 is an introduction to dynamic programming using flat-state representation, and contains four chapters. The meaning of the subtitle is explained in the first chapter by means of a simple example. Of extreme interest is chapter 2, where the characteristics of dynamic programming are introduced through various examples: discrete and continuous budgeting problems, a stochastic shortest-path problem, a gambling problem, an asset pricing and asset acquisition problem, and a dynamic assignment problem. Chapter 3 is devoted to Markov decision theory. Lastly, approximate dynamic programming is discussed in chapter 4. Chapters 5 through 9 make up Part 2, which focuses on approximate dynamic programming. Approximate dynamic programming offers an important set of strategies and methods for solving problems that are difficult due to size, the lack of a formal model of the information process, or in view of the fact that the transition function is unknown. Chapter 5 is devoted to time and resource modeling techniques. The next two chapters are devoted to stochastic approximation methods and approximating value functions, respectively. Chapter 8 is the bulk of the book: starting from the introduction presented in chapter 4, various algorithms are described in detail, with an emphasis on the finite-horizon case. Finally, Part 3, chapters 10 to 13, is devoted to special topics and applications. In particular, the last chapter describes a number of issues arising in real-world applications of approximate dynamic programming: convergence issues, modeling (online versus offline), and debugging. As a last point, it is worth mentioning that additional material is available online (additional exercises and solutions to selected problems); at the moment, only some datasets are downloadable. The intended audience of the book is OR practitioners and undergraduate or masters students. The more advanced material, suitable for graduate students, is clearly marked. The sections titled "Why Does It Work__?__" are remarkable?these are where the proofs of the various results are collected. In my opinion, the audience for this book is much larger than the author had in mind. In fact, this book should be on the shelf of every OR researcher and practitioner as a standard reference book. My overall impression of this book is extremely positive. The author succeeds in focusing on the core of each aspect of dynamic programming with a simple and clear exposition of the material, emphasizing computation and applications, while maintaining and elevating the standard of the theory. Online Computing Reviews Service