In Information theory, the mutual information between Random variables $X$, and $Y$ is defined as:

$I(X;Y) = \sum_{x,y} p(x,y) \log{\frac{p(x,y)}{p(x)p(y)}} = E \log{\frac{p(X,Y)}{p(X)p(Y)}}$

where $E$ denotes expectation. The mutual information measures the amount of information we obtain about $X$ by knowing $Y$ (see result below).

video

The mutual information between a random variable and itself is equal to its entropy

Some results (video):

$I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X)$

$H(X|Y)$ is the Conditional entropy and thus gives you the information about $X$ that $Y$ doesn't give you.

estimating mutual info with neural nets: https://arxiv.org/abs/1801.04062