Related Video Lectures · Lecture Notes
MIT OpenCourseWare
Browse materials
Dynamic Programming and Stochastic Control
Art: not stated, Fletcher's Combined Block Telegraph Apparatus and Bell, 1889 · Public domain
- Instructor
- Prof. Dimitri Bertsekas
- Term
- Fall 2015
- Level
- Graduate
- What's inside
- 15 videos · 12 solutions · 2 lecture notes · 42 other files · 6 web pages
- Materials open
- Videos here · files on MIT OCW
About this course
Source: MIT OpenCourseWare 6.231 Dynamic Programming and Stochastic Control, Fall 2015, Prof. Dimitri Bertsekas — original course page on ocw.mit.edu ↗. License: CC BY-NC-SA 4.0. Study order: MIT's own listing order, section by section. Taken from MIT's published course files, downloaded from ocw.mit.edu on September 23, 2026. How RedLectern is built.
Materials are grouped the way MIT organized them. A verified, course-wide order has not been established here.
Has MIT solutions
Before you start
Prerequisites are not listed here. Check MIT's own syllabus for this course.
How this course was organized
What's here, and what isn't
- MIT's own materials here did not support a verified, course-wide order, so they are grouped by section instead.
Not included here
7 items from the source could not be safely included:
- MIT's page format for this item wasn't recognized clearly enough to place it safely. (6 items)
- Referenced in a table, but without MIT's own verified order, so it isn't used to claim a sequence.
Materials by section
Grouped by section, as MIT organized them. Not sequenced or ordered.
Syllabus
Lecture Slides
24 files
- 6.231 Fall 2015 Complete Lecture Notes Download
- 6.231 Fall 2015 Lecture 1: Introduction to Dynamic Programming, Examples of Dynamic Programming, Significance of Feedback Download
- 6.231 Fall 2015 Lecture 2: The Basic Problem, Principle of Optimality, The General Dynamic Programming Algorithm, State Augmentation Download
- 6.231 Fall 2015 Lecture 3: Deterministic Finite-State Problem, Backward Shortest Path Algorithm, Forward Shortest Path Algorithm, Alternative Shortest Path Algorithms Download
- 6.231 Fall 2015 Lecture 4: Examples of Stochastic Dynamic Programming Problems, Linear-Quadratic Problems, Inventory Control Download
- 6.231 Fall 2015 Lecture 5: Stopping Problems, Scheduling Problems, Minimax Control Download
- 6.231 Fall 2015 Lecture 6: Problems with Imperfect State Info, Reduction to the Perfect State Info Cast, Linear Quadratic Problems, Separation of Estimation and Control Download
- 6.231 Fall 2015 Lecture 7: Imperfect State Information, Sufficient Statistics, Conditional State Distribution as a Sufficient Statistic, Finite-State Analysis Download
- 6.231 Fall 2015 Lecture 8: Suboptimal Control, Cost Approximation Methods: Classification, Certainty Equivalent Control, Limited Lookahead Policies, Performance Bounds, Problem Approximation Approach, Parametric Cost-To-Go Approximation Download
- 6.231 Fall 2015 Lecture 9: Rollout Algorithms, Cost Improvement Property, Discrete Deterministic Problems, Approximations to Rollout Algorithms, Model Predictive Control (MPS), Discretization of Continuous Time, Discretization of Continuous Space, Other Suboptimal Approaches Download
- 6.231 Fall 2015 Lecture 10: Infinite Horizon Problems, Stochastic Shortest Path (SSP) Problems, Bellman’s Equation, Dynamic Programming – Value Iteration, Discounted Problems as a Special Case of SSP Download
- 6.231 Fall 2015 Lecture 11: Review of Stochastic Shortest Path Problems, Computation Methods for SSP, Computational Methods for Discounted Problems Download
- 6.231 Fall 2015 Lecture 12: Average Cost Per Stage Problems, Connection With Stochastic Shortest Path Problems, Bellman’s Equation, Value Iteration, Policy Iteration Download
- 6.231 Fall 2015 Lecture 13: Control of Continuous-Time Markov Chains: Semi-Markov Problems, Problem Formulation: Equivalence to Discrete-Time Problems, Problem Formulation: Equivalence to Discrete-Time Problems, Average Cost Problems Download
- 6.231 Fall 2015 Lecture 14: Introduction to Advanced Infinite Horizon Dynamic Programming and Approximation Methods Download
- 6.231 Fall 2015 Lecture 15: Review of Basic Theory of Discounted Problems, Monotonicity of Contraction Properties, Contraction Mappings in Dynamic Programming, Discounted Problems: Countable State Space with Unbounded Costs, Generalized Discounted Dynamic Programming, An Introduction to Abstract Dynamic Programming Download
- 6.231 Fall 2015 Lecture 16: Review of Computational Theory of Discounted Problems, Value Iteration (VI), Optimistic PI, Computational Methods for Generalized Discounted Dynamic Programming, Asynchronous Algorithms Download
- 6.231 Fall 2015 Lecture 17: Undiscounted Problems, Stochastic Shortest Path Problems, Proper and Improper Policies, Analysis and Computational Methods for SSP, Pathologies of SSP, SSP Under Weak Conditions Download
- 6.231 Fall 2015 Lecture 18: Undiscounted Total Cost Problems, Positive and Negative Cost Problems, Proper and Improper Policies, Adaptive (Linear Quadratic) Dynamic Programming, Affine Monotomic and Risk Sensitive Problems Download
- 6.231 Fall 2015 Lecture 19: Introduction to affirmative Dynamic Programming, Approximation in Policy Space, Approximation in Value Space, Rollout/Simulation-based Single Policy Iteration, Approximation in Value Space Using Problem Approximation Download
- 6.231 Fall 2015 Lecture 20: Discounted Problems, Approximate (fitted) VI, Approximate PI, The Projected Equation, Contraction Properties: Error Bounds, Matrix Form of the Projected Equation, Simulation-based Implementation, LSTD and LSPE Methods Download
- 6.231 Fall 2015 Lecture 21: Review of Approximate Policy Iteration, Projected Equation methods for Policy Evaluation, Simulation-Based Implementation Issues, Multistep Projected Equation Methods, Bias-Variance Tradeoff, Exploration-Enhanced Implementations, Oscillations Download
- 6.231 Fall 2015 Lecture 22: Aggregation as an Approximation Methodology, Aggregate Problem, Simulation-based Aggregation, Q-Learning Download
- 6.231 Fall 2015 Lecture 23: Additional Topics in Advanced Dynamic Programming, Stochastic Shortest Path Problems, Average Cost Problems, Generalizations, Basis Function Adaption, Gradient-based Approximation in Policy Space, An Overview Download
Assignments
9 files
- 6.231 Homework 8, Fall 2015 Download
- 6.231 Homework Solution 1, Fall 2015 Download
- 6.231 Homework Solution 2, Fall 2015 Download
- 6.231 Homework Solution 3, Fall 2015 Download
- 6.231 Homework Solution 4, Fall 2015 Download
- 6.231 Homework Solution 5, Fall 2015 Download
- 6.231 Homework Solution 6, Fall 2015 Download
- 6.231 Homework Solution 7, Fall 2015 Download
- 6.231 Homework Solution 8, Fall 2015 Download
Exams
- 6.231 Dynamic Programming and Stochastic Control, Fall 2008 Midterm and Solutions Download
- 6.231 Dynamic Programming and Stochastic Control, Fall 2009 Midterm Download
- 6.231 Dynamic Programming and Stochastic Control, Fall 2009 Solutions Download
- 6.231 Dynamic Programming and Stochastic Control, Fall 2011 Midterm and Solutions Download
- 6.231 Dynamic Programming and Stochastic Control, Fall 2015 Midterm Download
- 6.231 Dynamic Programming and Stochastic Control, Fall 2015 Solutions Download
Projects
Related Video Lectures
15 files
- Approximate Dynamic Programming, Lecture 7 Download
- Summer 2012 Approximate Dynamic Programming, Lecture 1 Download
- Summer 2012 Approximate Dynamic Programming, Lecture 2 Download
- Summer 2012 Approximate Dynamic Programming, Lecture 3 Download
- Summer 2012 Approximate Dynamic Programming, Lecture 4 Download
- Summer 2012 Approximate Dynamic Programming, Lecture 5 Download
- Summer 2012 Approximate Dynamic Programming, Lecture 6 Download
- Summer 2012 Approximate Dynamic Programming, Lecture Notes Download
- 6.231 Dynamic Programming and Stochastic Control, Complete Supplemental Notes Download
- 6.231 Dynamic Programming and Stochastic Control, Supplemental Lecture 1 Download
- 6.231 Dynamic Programming and Stochastic Control, Supplemental Lecture 2 Download
- 6.231 Dynamic Programming and Stochastic Control, Supplemental Lecture 3 Download
- 6.231 Dynamic Programming and Stochastic Control, Supplemental Lecture 4 Download
- 6.231 Dynamic Programming and Stochastic Control, Supplemental Lecture 5 Download
- 6.231 Dynamic Programming and Stochastic Control, Supplemental Lecture 6 Download
15 videos
- Approximate Dynamic Programming, Lecture 1, Part 1 Open ↗
- Approximate Dynamic Programming, Lecture 1, Part 2 Open ↗
- Approximate Dynamic Programming, Lecture 1, Part 3 Open ↗
- Approximate Dynamic Programming, Lecture 2, Part 1 Open ↗
- Approximate Dynamic Programming, Lecture 2, Part 2 Open ↗
- Approximate Dynamic Programming, Lecture 2, Part 3 Open ↗
- Approximate Dynamic Programming, Lecture 3, Part 1 Open ↗
- Approximate Dynamic Programming, Lecture 3, Part 2 Open ↗
- Approximate Dynamic Programming, Lecture 4, Part 1 Open ↗
- Approximate Dynamic Programming, Lecture 4, Part 2 Open ↗
- Approximate Dynamic Programming, Lecture 5, Part 1 Open ↗
- Approximate Dynamic Programming, Lecture 5, Part 2 Open ↗
- Approximate Dynamic Programming, Lecture 5, Part 3 Open ↗
- Approximate Dynamic Programming, Lecture 6, Part 1 Open ↗
- Approximate Dynamic Programming, Lecture 6, Part 2 Open ↗
Course outline
Related Video Lectures · Lecture Notes
Lecture 2: Exact DP: Large-scale Computational Methods (PDF)
Related Video Lectures · Lecture Notes
Lecture 3: General Issues of Approximation and Simulation (PDF)
Related Video Lectures · Lecture Notes
Lecture 4: Temporal Differences (TD), Projected Equations, Galerkin Approximation (PDF)
Related Video Lectures · Lecture Notes
Lecture 5: Aggregation Methods (PDF)
Related Video Lectures · Lecture Notes
Lecture 6: Stochastic Approximation, Q-learning, and Other Methods (PDF)
Related Video Lectures · Lecture Notes
Lecture 7: Monte Carlo Methods (PDF)
MIT OpenCourseWare, Prof. Dimitri Bertsekas, 6.231 Dynamic Programming and Stochastic Control, Fall 2015.
License: CC BY-NC-SA 4.0 · Edition: Fall 2015 · Open the original course on OCW ↗
Organized by RedLectern, independent of MIT and not endorsed by MIT.
Per-resource rights are recorded on each resource; All-Rights-Reserved items are link-only, never mirrored.