<p>This is a list of recommended books, videos and web sites copied from the Further Readings section of my book on information theory (given at the end of this post).</p>

<p>Applebaum D (2008). Probability and Information: An Integrated Approach. A thorough introduction to information theory, which strikes a good balance between intuitive and technical explanations.</p>

<p>Avery J (2012). Information Theory and Evolution. An engaging account of how information theory is relevant to a wide range of natural and man-made systems, including evolution, physics, culture and genetics. Includes interesting background stories on the development of ideas within these different disciplines.</p>

<p>Baeyer HV (2005). Information: The New Language of Science Erudite, wide-ranging, and insightful account of information theory. Contains no equations, which makes it very readable.</p>

<p>Cover T and Thomas J (1991). Elements of Information Theory. Comprehensive, and highly technical, with historical notes and an equation summary at the end of each chapter.</p>

<p>Ghahramani Z (2002). Information Theory. Encyclopedia of Cognitive Science. An excellent, brief overview of information.</p>

<p>Gleick J (2012). The Information. An informal introduction to the history of ideas and people associated with information theory.</p>

<p>Guizzo EM (2003). The Essential Message: Claude Shannon and the Making of Information Theory. Master’s Thesis, Massachusetts Institute of Technology. One of the few accounts of Shannon’s role in the development of information theory. 
See <a href="http://dspace.mit.edu/" rel="noreferrer">http://dspace.mit.edu/</a> bitstream/handle/1721.1/39429/54526133.pdf.</p>

<p>Laughlin, SB (2006). The Hungry Eye: Energy, Information and Retinal Function, Excellent lecture on the energy cost of Shannon information in eyes. See <a href="http://www.crsltd.com/guest-talks/" rel="noreferrer">http://www.crsltd.com/guest-talks/</a> crs-guest-lecturers/simon-laughlin.</p>

<p>MacKay DJC (2003). Information Theory, Inference, and Learning Algorithms. The modern classic on information theory. A very readable text that roams far and wide over many topics. The book’s web site (below) also has a link to an excellent series of video lectures by MacKay. Available free online at <a href="http://www.inference.phy" rel="noreferrer">http://www.inference.phy</a>. cam.ac.uk/mackay/itila/.</p>

<p>Pierce JR (1980). An Introduction to Information Theory: Symbols, Signals and Noise. Second Edition. Pierce writes with an informal, tutorial style of writing, but does not flinch from presenting the fundamental theorems of information theory. This book provides a good balance between words and equations.</p>

<p>Reza FM (1961). An Introduction to Information Theory. A more comprehensive and mathematically rigorous book than Pierce’s book, it should be read only after first reading Pierce’s more informal text.</p>

<p>Seife C (2007). Decoding the Universe: How the New Science of Information Is Explaining Everything in the Cosmos, From Our Brains to Black Holes. A lucid and engaging account of the relationship between information, thermodynamic entropy and quantum computing. Highly recommended.</p>

<p>Shannon CE and Weaver W (1949). The Mathematical Theory of Communication. University of Illinois Press. A surprisingly accessible book, written in an era when information theory was known only to a privileged few. This book can be downloaded from <a href="http://cm" rel="noreferrer">http://cm</a>. bell-labs.com/cm/ms/what/shannonday/paper.html</p>

<p>For the complete novice, the videos at the online Kahn Academy provide an excellent introduction. </p>

<p>Additionally, the online Scholarpedia web page by Latham and Rudi provides a lucid technical account of mutual information: <a href="http://www.scholarpedia.org/article/Mutual_information" rel="noreferrer">http://www.scholarpedia.org/article/Mutual_information</a>.</p>

<p>Finally, some historical perspective is provided in a long interview with Shannon conducted in 1982: <a href="http://www.ieeeghn.org/wiki/" rel="noreferrer">http://www.ieeeghn.org/wiki/</a> index.php/Oral-History:Claude_E._Shannon.</p>

<p>Finally, my book is called Information Theory: A Tutorial Introduction (2015). Chapter 1 can be downloaded from here:
<a href="http://jim-stone.staff.shef.ac.uk/BookInfoTheory/InfoTheoryBookMain.html" rel="noreferrer">http://jim-stone.staff.shef.ac.uk/BookInfoTheory/InfoTheoryBookMain.html</a></p>

<p><a href="https://i.sstatic.net/DIOjk.jpg" rel="noreferrer"><img src="https://i.sstatic.net/DIOjk.jpg" alt="enter image description here"></a></p>

CC BY-SA 4.0
2020-03-17T05:35:13.357Z
The books of Applebaum and Stone (you) were sufficient for me.
Carlos Pinzón
CC BY-SA 3.0
2018-01-25T11:09:36.490Z
2018-01-25T11:50:23.847Z
2018-01-25T11:50:23.847Z
James V Stone
<p>I have started Information Theory classes just recently and was wondering what would be a standard book to purchase. I know I can go for basic introductory books but I also like to purchase standard books that I can use throughout my career for future reference purposes. I get some threads in forums about introductory books or basic books on information theory, but I can't find any about standard books on information theory.
P.S.:By standard book I mean like Cormen is for Algorithms</p>

CC BY-SA 3.0
2016-12-09T12:25:08.613Z
see D. MacKay's book: http://www.inference.phy.cam.ac.uk/itila/book.html
Radu GRIGore
CC BY-SA 3.0
2016-12-09T13:47:06.267Z
Cover and Thomas's Elements of Information Theory is the standard reference.
Sasho Nikolov
CC BY-SA 3.0
2016-12-09T17:14:22.993Z
You have a long list here https://en.wikipedia.org/wiki/Information_theory#References
drzbir
<it.information-theory>
What are some standard books/papers on Information Theory?
Anshuman Sabath
# What are some standard books/papers on Information Theory?

(Asked by: username_0 on 2016-12-09)

I have started Information Theory classes just recently and was wondering what would be a standard book to purchase. I know I can go for basic introductory books but I also like to purchase standard books that I can use throughout my career for future reference purposes. I get some threads in forums about introductory books or basic books on information theory, but I can't find any about standard books on information theory.
P.S.:By standard book I mean like Cormen is for Algorithms

**Tags:** it.information-theory
> **Comments:**
> **username_1**: see D. MacKay's book: http://www.inference.phy.cam.ac.uk/itila/book.html  
> **username_2**: Cover and Thomas's Elements of Information Theory is the standard reference.  
> **username_3**: You have a long list here https://en.wikipedia.org/wiki/Information_theory#References  

---

## 1 answers

### Answer by username_4 on 2018-01-25. Score: 16 | Accepted answer

This is a list of recommended books, videos and web sites copied from the Further Readings section of my book on information theory (given at the end of this post).

Applebaum D (2008). Probability and Information: An Integrated Approach. A thorough introduction to information theory, which strikes a good balance between intuitive and technical explanations.

Avery J (2012). Information Theory and Evolution. An engaging account of how information theory is relevant to a wide range of natural and man-made systems, including evolution, physics, culture and genetics. Includes interesting background stories on the development of ideas within these different disciplines.

Baeyer HV (2005). Information: The New Language of Science Erudite, wide-ranging, and insightful account of information theory. Contains no equations, which makes it very readable.

Cover T and Thomas J (1991). Elements of Information Theory. Comprehensive, and highly technical, with historical notes and an equation summary at the end of each chapter.

Ghahramani Z (2002). Information Theory. Encyclopedia of Cognitive Science. An excellent, brief overview of information.

Gleick J (2012). The Information. An informal introduction to the history of ideas and people associated with information theory.

Guizzo EM (2003). The Essential Message: Claude Shannon and the Making of Information Theory. Master’s Thesis, Massachusetts Institute of Technology. One of the few accounts of Shannon’s role in the development of information theory.
See <http://dspace.mit.edu/> bitstream/handle/1721.1/39429/54526133.pdf.

Laughlin, SB (2006). The Hungry Eye: Energy, Information and Retinal Function, Excellent lecture on the energy cost of Shannon information in eyes. See <http://www.crsltd.com/guest-talks/> crs-guest-lecturers/simon-laughlin.

MacKay DJC (2003). Information Theory, Inference, and Learning Algorithms. The modern classic on information theory. A very readable text that roams far and wide over many topics. The book’s web site (below) also has a link to an excellent series of video lectures by MacKay. Available free online at <http://www.inference.phy>. cam.ac.uk/mackay/itila/.

Pierce JR (1980). An Introduction to Information Theory: Symbols, Signals and Noise. Second Edition. Pierce writes with an informal, tutorial style of writing, but does not flinch from presenting the fundamental theorems of information theory. This book provides a good balance between words and equations.

Reza FM (1961). An Introduction to Information Theory. A more comprehensive and mathematically rigorous book than Pierce’s book, it should be read only after first reading Pierce’s more informal text.

Seife C (2007). Decoding the Universe: How the New Science of Information Is Explaining Everything in the Cosmos, From Our Brains to Black Holes. A lucid and engaging account of the relationship between information, thermodynamic entropy and quantum computing. Highly recommended.

Shannon CE and Weaver W (1949). The Mathematical Theory of Communication. University of Illinois Press. A surprisingly accessible book, written in an era when information theory was known only to a privileged few. This book can be downloaded from <http://cm>. bell-labs.com/cm/ms/what/shannonday/paper.html

For the complete novice, the videos at the online Kahn Academy provide an excellent introduction.

Additionally, the online Scholarpedia web page by Latham and Rudi provides a lucid technical account of mutual information: <http://www.scholarpedia.org/article/Mutual_information>.

Finally, some historical perspective is provided in a long interview with Shannon conducted in 1982: <http://www.ieeeghn.org/wiki/> index.php/Oral-History:Claude\_E.\_Shannon.

Finally, my book is called Information Theory: A Tutorial Introduction (2015). Chapter 1 can be downloaded from here:
<http://jim-stone.staff.shef.ac.uk/BookInfoTheory/InfoTheoryBookMain.html>

[![enter image description here](https://i.sstatic.net/DIOjk.jpg)](https://i.sstatic.net/DIOjk.jpg)

> **Comments:**
> **username_5**: The books of Applebaum and Stone (you) were sufficient for me.  

---

