
Recent Posts
Archives
 November 2016
 April 2015
 March 2015
 February 2015
 November 2014
 September 2014
 August 2014
 July 2014
 June 2014
 May 2014
 April 2014
 March 2014
 February 2014
 January 2014
 December 2013
 November 2013
 October 2013
 September 2013
 August 2013
 July 2013
 June 2013
 May 2013
 March 2013
 July 2012
 March 2012
 December 2011
Categories
 Adobe Reader
 AUCTeX
 autopair
 blogging
 bug
 C
 cllib
 closure
 Common Lisp
 comparison
 dash
 debug
 documentary
 dynamic scoping
 elisp
 ELPA
 English
 entropy
 ergodic theory
 export
 exposition
 fish
 fix
 font
 gotcha
 html
 htmlize
 image
 immutable
 information theory
 Internet
 intuition
 Japanese
 javascript
 JPEG
 js2mode
 KAIST
 keyboard
 kimchi
 Korea
 LaTeX
 lexicalbinding
 lexical scoping
 LuaTeX
 managed
 measure theory
 monospace
 movie
 mutable
 namespace
 NEF
 orgmode
 package
 Photoshop
 PNG
 pointer
 portable
 prefix
 process
 Python
 rainbowdelimiters
 region
 rice
 SLIME
 specialvariable
 stew
 tablet
 Ubuntu
 unicode
 Windows
 Windows 8
 wireless
 Wordpress
 XeTeX
Meta
Tag Archives: entropy
Intuitions on problems from Elements of Information Theory Chapter 2
Table of Contents 1. meaning of conditional mutual information 2. comments on some selected problems from Chapter 2 2.1. Problem 2.6 2.2. Problem 2.14 2.3. Problem 2.23 2.4. Problem 2.27 2.5. Problem 2.41 This post is not about sharing solutions … Continue reading
Posted in Mathematics
Tagged entropy, information theory, intuition, mutual information
Leave a comment
Shannon’s entropy of random variables and partitions
Table of Contents 1. entropy for dummies 1.1. definition of entropy of r.v. and joint entropy 1.2. the town 1.3. intuition for entropy and its properties 1.4. intuition for the notion of surprisal 2. intuition for conditional entropy and its … Continue reading
Posted in Mathematics
Tagged entropy, ergodic theory, exposition, information theory, intuition
1 Comment