MIT OpenCourseWare

Browse materialsMIT 6.441 · Spring 2010

Information Theory

Art: Palmer, Edward, 1803-1872, Figures 113-123. Plate Electrical Machines, 1840 · Public domain

Instructor: Prof. Muriel Médard
Term: Spring 2010
Level: Graduate
What's inside: 1 lecture note · 1 reading · 33 other files · 3 web pages
Materials open: MIT OCW

Other versions of this course

6.441 · Spring 2016 · Study here
6.441 · Spring 2010 · Study here (this version)

Start here

Start course: Lecture 1: Introduction, entropy Find a topic

Progress is saved in this browser.

Export or import progress

About this course

Source: MIT OpenCourseWare 6.441 Information Theory, Spring 2010, Prof. Muriel Médard — original course page on ocw.mit.edu ↗. License: CC BY-NC-SA 4.0. Study order: MIT's own listing order, section by section. Taken from MIT's published course files, downloaded from ocw.mit.edu on September 23, 2026. How RedLectern is built.

Materials are grouped the way MIT organized them. A verified, course-wide order has not been established here.

Before you start

Prerequisites are not listed here. Check MIT's own syllabus for this course.

This course’s cited books, in our reading lists: Elements of Information Theory

How this course was organized

What's here, and what isn't

A lecture sequence of 23 items is confirmed. Assignment and exam materials exist too, but MIT's own order for them could not be verified, so they're shown as materials below rather than a sequence.

Not included here

1 item from the source could not be safely included:

Referenced in a table, but without MIT's own verified order, so it isn't used to claim a sequence.

Materials by section

Grouped by section, as MIT organized them. Not sequenced or ordered.

Lecture Notes

24 files

Assignments

9 problem sets

MIT OpenCourseWare, Prof. Muriel Médard, 6.441 Information Theory, Spring 2010.

License: CC BY-NC-SA 4.0 · Edition: Spring 2010 · Open the original course on OCW ↗

Organized by RedLectern, independent of MIT and not endorsed by MIT.

Per-resource rights are recorded on each resource; All-Rights-Reserved items are link-only, never mirrored.

Information Theory

Before you start

What's here, and what isn't

Not included here

Materials by section

Lecture Notes

Assignments

Lecture 1: Introduction, entropy

Lecture 2: Jensen’s inequality, data processing theorem, Fanos’s inequality

Lecture 3: Different types of convergence, asymptotic equipartition property (AEP), typical set, joint typicality

Lecture 4: Entropies of stochastic processes

Lecture 5: Data compression, Kraft inequality, optimal codes

Lecture 6: Huffman codes

Lecture 7: Shannon-Fano-Elias codes, Slepian-Wolf

Lecture 8: Channel capacity, binary symmetric and erasure channels

Lecture 9: Maximizing capacity, Blahut-Arimoto

Lecture 10: The channel coding theorem

Lecture 11: Strong coding theorem, types of errors

Lecture 12: Strong coding theorem, error exponents

Lecture 13: Fano’s inequality and the converse to the coding theorem

Lecture 14: Feedback capacity

Lecture 15: Joint source channel coding

Lecture 16: Differential entropy, maximizing entropy

Lecture 17: Additive Gaussian noise channel

Lecture 18: Gaussian channels: parallel, colored noise, inter-symbol interference

Lecture 19: Gaussian channels with feedback

Lecture 20: Multiple access channels

Lecture 21: Broadcast channels

Lecture 22: Finite state Markov channels

Lecture 23: Channel side information, wide-band channels