Mrs. Gerber's Lemma: X is a length-n binary word with arbitrary distribution and Y is a corruption of X through a binary symmetric channel with crossover probability p. If H(X) \geq nv then H(Y) \geq nh(v')

## Articles in the Blog category

## Modifying a Plustek 8100 to Scan Medium Format

The Plustek 8100 is a relatively cheap but good quality film scanner. It produces much nicer scans than most flatbed-type scanners, but it can only capture small format film. In this page I show how to modify a Plustek 8100 to scan medium format (120 film) by stitching together multiple …

## A Class of Multivariate Functions that Behave Like Single-Letter Random Variables

Vector-encoders of a certain pretty restricted class happen to carry precisely the same information content as univariate encoders.

### Claim

Take X _ 1,\dots, X _ n iid over finite alphabet \mathcal{X} and f:\mathcal{X}^n \to \mathbb{N …

## Extracting Relevant Information over Many Trials

The information bottleneck scenario goes like this:

We are interested in a quantity Y but only observe X which we must process down to a rate R \in (0, H(X)). To this end, what random variables (U,Q) minimize H …

## Hypersphere Cap Volumes in Matlab

The volume of a height-h cap of a ball of radius r in dimension n can be derived:

\begin{equation} \operatorname{vol}(\operatorname{cap}) = \frac{1}{2} \cdot \frac{\pi^{n/2}}{\Gamma\left(1+n/2\right)} \cdot r^n \cdot I_{(2rh-h^2)/r …