aka data processing inequality

No clever manipulation of the data can improve the inferences that can be made from the data (unless the manipulation includes extra information about what you want to infer...

Data processing theorem#

Given a Markov chain $X \rightarrow Y \rightarrow Z$, then $I(X;Y) \geq I(X;Z)$

An important special case:

For any function, $f$, and Random variables $x$, $y$,

where $I$ is the Mutual information

A function $f$ for which the above inequality becomes an equality is a Sufficient statistic