MIT OpenCourseWare

Browse materialsMIT 2.997 · Spring 2004

Decision Making in Large Scale Systems

Subjects: Game Theory · Mechanical Engineering

Art: Frederick Collins, Collins steam engine drawing, 1918 · Public domain

Instructor: Prof. Daniela Pucci De Farias
Term: Spring 2004
Level: Graduate
What's inside: 1 lecture note · 1 reading · 45 other files · 4 web pages
Materials open: MIT OCW

Start here

Start course: Lecture 1: Markov Decision Processes Finite-Horizon Problems…Find a topic

Progress is saved in this browser.

Export or import progress

About this course

Source: MIT OpenCourseWare 2.997 Decision Making in Large Scale Systems, Spring 2004, Prof. Daniela Pucci De Farias — original course page on ocw.mit.edu ↗. License: CC BY-NC-SA 4.0. Study order: MIT's own listing order, section by section. Taken from MIT's published course files, downloaded from ocw.mit.edu on September 23, 2026. How RedLectern is built.

Materials are grouped the way MIT organized them. A verified, course-wide order has not been established here.

Has MIT solutions 1 file in this course is named by MIT as solutions.

Before you start

Prerequisites are not listed here. Check MIT's own syllabus for this course.

This course’s cited books, in our reading lists: Dynamic Programming and Optimal Control · Neuro-Dynamic Programming

How this course was organized

What's here, and what isn't

A lecture sequence of 24 items is confirmed. It stays as organized materials rather than a full course order.

Not included here

18 items from the source could not be safely included:

MIT's page format for this item wasn't recognized clearly enough to place it safely. (18 items)

Materials by section

Grouped by section, as MIT organized them. Not sequenced or ordered.

Lecture Notes

21 files

Assignments

Projects

Other materials from this course

14 files

MIT OpenCourseWare, Prof. Daniela Pucci De Farias, 2.997 Decision Making in Large Scale Systems, Spring 2004.

License: CC BY-NC-SA 4.0 · Edition: Spring 2004 · Open the original course on OCW ↗

Organized by RedLectern, independent of MIT and not endorsed by MIT.

Per-resource rights are recorded on each resource; All-Rights-Reserved items are link-only, never mirrored.

Decision Making in Large Scale Systems

Before you start

What's here, and what isn't

Not included here

Materials by section

Lecture Notes

Assignments

Projects

Other materials from this course

Lecture 1: Markov Decision Processes Finite-Horizon Problems: Backwards Induction Discounted-Cost Problems: Cost-to-Go Function, Bellman’s Equation

Lecture 2: Value Iteration Existence and Uniqueness of Bellman’s Equation Solution Gauss-Seidel Value Iteration

Lecture 3: Optimality of Policies derived from the Cost-to-go Function Policy Iteration Asynchronous Policy Iteration

Lecture 4: Average-Cost Problems Relationship with Discounted-Cost Problems Bellman’s Equation Blackwell Optimality

Lecture 5: Average-Cost Problems Computational Methods

Lecture 6: Application of Value Iteration to Optimization of Multiclass Queueing Networks Introduction to Simulation-based Methods Real-Time Value Iteration

Lecture 7: Q-Learning Stochastic Approximations

Lecture 8: Stochastic Approximations: Lyapunov Function Analysis The ODE Method Convergence of Q-Learning

Lecture 9: Exploration versus Exploitation: The Complexity of Reinforcement Learning

Lecture 10: Introduction to Value Function Approximation Curse of Dimensionality Approximation Architectures

Lecture 11: Model Selection and Complexity

Lecture 12: Introduction to Value Function Approximation Algorithms Performance Bounds

Lecture 13: Temporal-Difference Learning with Value Function Approximation

Lecture 14: Temporal-Difference Learning with Value Function Approximation (cont.)

Lecture 15: Temporal-Difference Learning with Value Function Approximation (cont.) Optimal Stopping Problems General Control Problems

Lecture 16: Approximate Linear Programming

Lecture 17: Approximate Linear Programming (cont.)

Lecture 18: Efficient Solutions for Approximate Linear Programming

Lecture 19: Efficient Solutions for Approximate Linear Programming: Factored MDPs

Lecture 20: Policy Search Methods

Lecture 21: Policy Search Methods (cont.)

Lecture 22: Policy Search Methods for POMDPs Application: Call Admission Control Actor-Critic Methods

Lecture 23: Guest Lecture: Prof. Nick Roy Approximate POMDP Compression

Lecture 24: Policy Search Methods: PEGASUS Application: Helicopter Control