Abstract Dynamic Programming and Reinforcement Learning

Reinforcement learning (RL), a prominent subfield of Artificial Intelligence, is fundamentally based on Dynamic Programming (DP), a solid and broadly applicable optimization methodology for sequential decision making.In this talk, we will discuss an abstract view of DP, which focuses on its fundamental structure, and develops its mathematical theory, uncluttered by extraneous assumptions. Surprisingly, this abstract view provides insight into the inner workings of RL and its connections to Model Predictive Control (MPC), a core methodology for systems control and optimization. The key idea is that MPC/RL can be viewed as a step of Newton's method for solving the fundamental Bellman fixed point equation of DP. This implies a superlinear relation between approximation error and performance error, which can guide the RL design process and is consistent with empirical observations.

Note: The talk is based on my course at ASU on DP/RL (2019-2025); see the website https://web.mit.edu/dimitrib/www/RLbook.html which also contains .pdf of the course textbook "A Course in Reinforcement Learning," 2nd edition, 2025, and two related research monographs:
Abstract Dynamic Programming, 3nd Edition, 2022.
Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control, 2022.

Bio
https://search.asu.edu/profile/3410924
https://www.mit.edu/~dimitrib/home.html