Data processing inequality
The Data processing inequality is an information theoretic concept which states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'.Definition
Let three random variables form the Markov chain, implying that the conditional distribution of depends only on and is conditionally independent of. Specifically, we have such a Markov chain if the joint probability mass function can be written as
In this setting, no processing of Y, deterministic or random, can increase the information that Y contains about X. Using the mutual information, this can be written as :
With the equality if and only if, i.e. and contain the same information about, and also forms a Markov chain.
