Help support MIT OpenCourseWare by shopping at Amazon.com! MIT OpenCourseWare offers direct links to Amazon.com to purchase the books cited in this course. Click on the book titles and purchase the book from Amazon.com, and MIT OpenCourseWare will receive up to 10% of all purchases you make. Your support will enable MIT to continue offering open access to MIT courses.

Schedule

• Lecture 6
• Recitation 7

Lecture Handouts

• Lecture Notes- Chapter 7: Processes (PDF)
• A summary in concise mathematics (mostly algebra) of many topics in information theory, coding, communications, by Dr. M. Z. Wang, Hong Kong. There is a section on Discrete Memoryless Channels.
• Gallager, Robert G. "Claude E. Shannon: A Retrospective on His Life, Work, and Impact." IEEE Transactions on Information Theory 47, no. 7 (November 2001): 2681-2695.

Assignment

• Problem Set 6 (PDF)
• Problem Set 6 Solutions (PDF)

Resources

Technical

• Shannon, Claude E. "A Mathematical Theory of Communication." Bell System Technical Journal 27 (July and October 1948): 379-423 (Part I), 623-656 (Part II). These seminal papers are available in several forms (see  bibliographic notes)
  • Original papers with corrections but without Shannon's 1949 modifications. (PDF)
  • Shannon, Claude E., and Warren Weaver. The Mathematical Theory of Communication. Urbana, IL: University of Illinois Press, 1949. ISBN: 0252725484. with later editions 1963 and 1998 (incorporating a number of modifications and corrections by Shannon).
  • Shannon, Claude E. A Mathematical Theory of Communication. 50th Anniversary Edition, printed for the 1998 IEEE International Symposium on Information Theory, MIT, Cambridge, MA; August 16-21, 1998. ISBN: 0252725484. (based on 1949 book, with corrections).
  • Reprinted in Key Papers in the Development of Information Theory. Edited by D. Slepian. New York, NY: IEEE Press, 1974. ISBN: 0879420278.
  • Reprinted in Sloane, N. J. A., and A. D. Wyner, eds. Claude Elwood Shannon: Collected Papers. New York, NY: IEEE Press, 1993. ISBN: 0780304349.

Because he treated a general case, Shannon was able to distinguish loss and noise, in the way done in 6.050J/2.110J. Many others treat only channels with similar inputs and outputs in which case often L = N so it may not be obvious why they should be different concepts.

Historical

• Claude E. Shannon (1916 - 2001)
  obituary, Tech Talk, MIT
• Wyner, Aaron D. The Significance of Shannon's Work.
• Gallager, Robert G. "Claude E. Shannon: A Retrospective on His Life, Work, and Impact." IEEE Transactions on Information Theory 47, no. 7 (November 2001): 2681-2695.

General Technical Books

There are many excellent texts on communications, most of which assume a familiarity with mathematics beyond introductory calculus. Some have treatments of the discrete memoryless channel, but not with the same emphasis given here. Here are a few:

• Gallager, Robert G. Information Theory and Reliable Communications. New York, NY: John Wiley & Sons, Inc., 1968. ISBN: 0471290483.
  One of the early textbooks, designed for first-year graduate students.
• R. Hambley, Allan R. An Introduction to Communication Systems. Computer Science Press, 1990.
• Cover, Thomas M., and Joy A. Thomas. Elements of Information Theory. New York, NY: John Wiley & Sons, Inc., 1991. ISBN: 0471062596.
  Aimed at university seniors and first-year graduate students. One of several excellent books of that era.
• Haykin, Simon. Communication Systems. 4th ed. New York, NY: John Wiley and Sons, Inc., 2001. ISBN: 047140182X.