Название: From Shortest Paths to Reinforcement Learning: A MATLAB-Based Tutorial on Dynamic Programming Автор: Paolo Brandimarte
Издательство: Springer
Год: 2021
Страниц: 216
Язык: английский
Формат: pdf (true), epub
Размер: 18.7 MB
Dynamic programming (DP) has a relevant history as a powerful and flexible optimization principle, but has a bad reputation as a computationally impractical tool. This book fills a gap between the statement of DP principles and their actual software implementation. Using MATLAB throughout, this tutorial gently gets the reader acquainted with DP and its potential applications, offering the possibility of actual experimentation and hands-on experience. The book assumes basic familiarity with probability and optimization, and is suitable to both practitioners and graduate students in engineering, applied mathematics, management, finance and economics.
There are multiple viewpoints that an author may take when writing a book on dynamic programming (DP), depending on the research community that (s)he belongs to and the emphasis on specific applications. DP is used in operations research and management science, as well as in economics, control theory, and machine learning. DP can be applied to a wide array of problems ranging from the valuation of financial derivatives, the dynamic pricing of goods and services, and to the control of energy systems and the development of computer programs to play games like backgammon or chess, just to name a few.
The viewpoint I am taking here is mostly geared towards operations research and management science. However, I will try to be as general as possible, both in terms of modeling style and solution methods. This is why I include examples that arise in supply chain and revenue management, but also finance and economics. It should also be emphasized that my viewpoint is that reinforcement learning is one possible approach to apply dynamic programming. Unlike books on reinforcement learning, I also cover standard numerical methods.
Another deliberate choice that I made in planning the book is to steer away from convergence proofs and the formal analysis of algorithmic efficiency. There are excellent books dealing with this side of the coin, and there is no reason to write another one. By the same token, in order to keep the booklet to a manageable size, I will not provide the reader with a comprehensive description of all the variations on the theme that have been proposed in the literature. Nevertheless, I will try to give a general overview of the different flavors in terms of both applications and modeling: I cover finite- as well as infinite-horizon problems; for the latter case, I cover discounted as well as average contribution per stage. On the other hand, one of the main difficulties that newcomers have to face is the implementation of principles in a working program. Since implementation and experimentation, even though on toy problems, is the only way to really get a firm understanding of dynamic programming, the book includes a fair amount of working MATLAB code.
Скачать From Shortest Paths to Reinforcement Learning: A MATLAB-Based Tutorial on Dynamic Programming