Tag Archives: entropy

Intuitions on problems from Elements of Information Theory Chapter 2

Posted on March 16, 2015 by

Table of Contents 1. meaning of conditional mutual information 2. comments on some selected problems from Chapter 2 2.1. Problem 2.6 2.2. Problem 2.14 2.3. Problem 2.23 2.4. Problem 2.27 2.5. Problem 2.41 This post is not about sharing solutions … Continue reading →

Posted in Mathematics | Tagged , , , | Leave a comment

Shannon’s entropy of random variables and partitions

Posted on March 23, 2014 by

Table of Contents 1. entropy for dummies 1.1. definition of entropy of r.v. and joint entropy 1.2. the town 1.3. intuition for entropy and its properties 1.4. intuition for the notion of surprisal 2. intuition for conditional entropy and its … Continue reading →

Posted in Mathematics | Tagged , , , , | 1 Comment