-
Recent Posts
Archives
- November 2016
- April 2015
- March 2015
- February 2015
- November 2014
- September 2014
- August 2014
- July 2014
- June 2014
- May 2014
- April 2014
- March 2014
- February 2014
- January 2014
- December 2013
- November 2013
- October 2013
- September 2013
- August 2013
- July 2013
- June 2013
- May 2013
- March 2013
- July 2012
- March 2012
- December 2011
Categories
- Adobe Reader
- AUCTeX
- autopair
- blogging
- bug
- C
- cl-lib
- closure
- Common Lisp
- comparison
- dash
- debug
- documentary
- dynamic scoping
- elisp
- ELPA
- English
- entropy
- ergodic theory
- export
- exposition
- fish
- fix
- font
- gotcha
- html
- htmlize
- image
- immutable
- information theory
- Internet
- intuition
- Japanese
- javascript
- JPEG
- js2-mode
- KAIST
- keyboard
- kimchi
- Korea
- LaTeX
- lexical-binding
- lexical scoping
- LuaTeX
- managed
- measure theory
- monospace
- movie
- mutable
- namespace
- NEF
- org-mode
- package
- Photoshop
- PNG
- pointer
- portable
- prefix
- process
- Python
- rainbow-delimiters
- region
- rice
- SLIME
- special-variable
- stew
- tablet
- Ubuntu
- unicode
- Windows
- Windows 8
- wireless
- Wordpress
- XeTeX
Meta
Tag Archives: entropy
Intuitions on problems from Elements of Information Theory Chapter 2
Table of Contents 1. meaning of conditional mutual information 2. comments on some selected problems from Chapter 2 2.1. Problem 2.6 2.2. Problem 2.14 2.3. Problem 2.23 2.4. Problem 2.27 2.5. Problem 2.41 This post is not about sharing solutions … Continue reading
Posted in Mathematics
Tagged entropy, information theory, intuition, mutual information
Leave a comment
Shannon’s entropy of random variables and partitions
Table of Contents 1. entropy for dummies 1.1. definition of entropy of r.v. and joint entropy 1.2. the town 1.3. intuition for entropy and its properties 1.4. intuition for the notion of surprisal 2. intuition for conditional entropy and its … Continue reading
Posted in Mathematics
Tagged entropy, ergodic theory, exposition, information theory, intuition
1 Comment