In Data compression, particularly Universal source coding, we define the redundancy of a code with Codeword lengths $l(x)$, and implied probability $q(x)=2^{-l(x)}$ (see Source coding theorem), as the difference between the expected length of the code (under the true Information source $X$ distribution $p$) and the lower limit for the expected length:

$R(p,q) = E_p[l(X)] - E_p\left [ \log{\frac{1}{p(X)}}\right]$

= $\sum_x p(x) \left ( l(x) - \log{\frac{1}{p(x)}}\right)$

$=\sum_x p(x) \left (\log{\frac{1}{q(x)}} - \log{\frac{1}{p(x)}}\right)$

$= \sum_x p(x) \log{\frac{p(x)}{q(x)}}$

$=D(p||q)$

where $q(x) = 2^{-l(x)}$ is the distribution that corresponds to the codeword lengths $l(X)$, and $D(\cdot || \cdot)$ is the Relative entropy