
Recent Posts
Archives
 November 2016
 April 2015
 March 2015
 February 2015
 November 2014
 September 2014
 August 2014
 July 2014
 June 2014
 May 2014
 April 2014
 March 2014
 February 2014
 January 2014
 December 2013
 November 2013
 October 2013
 September 2013
 August 2013
 July 2013
 June 2013
 May 2013
 March 2013
 July 2012
 March 2012
 December 2011
Categories
 Adobe Reader AUCTeX autopair blogging bug C cllib closure Common Lisp comparison dash debug documentary dynamic scoping elisp ELPA English entropy ergodic theory export exposition fish fix font gotcha html htmlize image immutable information theory Internet intuition Japanese javascript JPEG js2mode KAIST keyboard kimchi Korea LaTeX lexicalbinding lexical scoping LuaTeX managed measure theory monospace movie mutable namespace NEF orgmode package PDF Photoshop PNG pointer portable prefix process Python rainbowdelimiters region rice SLIME specialvariable stew tablet Ubuntu unicode Windows Windows 8 wireless Wordpress XeTeX
Meta
Tag Archives: information theory
Intuitions on problems from Elements of Information Theory Chapter 2
Table of Contents 1. meaning of conditional mutual information 2. comments on some selected problems from Chapter 2 2.1. Problem 2.6 2.2. Problem 2.14 2.3. Problem 2.23 2.4. Problem 2.27 2.5. Problem 2.41 This post is not about sharing solutions … Continue reading
Posted in Mathematics
Tagged entropy, information theory, intuition, mutual information
Leave a comment
Shannon’s entropy of random variables and partitions
Table of Contents 1. entropy for dummies 1.1. definition of entropy of r.v. and joint entropy 1.2. the town 1.3. intuition for entropy and its properties 1.4. intuition for the notion of surprisal 2. intuition for conditional entropy and its … Continue reading
Posted in Mathematics
Tagged entropy, ergodic theory, exposition, information theory, intuition
1 Comment