Minimum Fisher information


In information theory, the principle of minimum Fisher information is a variational principle which, when applied with the proper constraints needed to reproduce empirically known expectation values, determines the best probability distribution that characterizes the system.

Measures of information

are the most important tools of information theory. They measure either the amount of positive information or of "missing" information an observer possesses with regards to any system of interest. The most famous IM is the so-called Shannon-entropy , which determines how much additional information the observer still requires in order to have all the available knowledge regarding a given system S, when all he/she has is a probability density function defined on appropriate elements of such system. This is then a "missing" information measure. The IM is a function of the PD only. If the observer does not have such
a PD, but only a finite set of empirically determined mean values of the system, then a fundamental scientific principle called the Maximum Entropy one asserts that the "best" PD is the one that, reproducing the known expectation values, maximizes otherwise Shannon's IM.

Fisher's information measure

, named after Ronald Fisher, is another kind of measure, in two respects, namely,
1) it reflects the amount of information of the observer,

2) it depends not only on the PD but also on its first derivatives, a property that makes it a local quantity.
The corresponding counterpart of MaxEnt is now the FIM-minimization, since Fisher's measure grows when Shannon's diminishes, and vice versa. The minimization here referred to is an important theoretical tool in a manifold of disciplines, beginning with physics. In a sense it is clearly superior to MaxEnt because the later procedure yields always as the solution an exponential PD, while the MFI solution is a differential equation for the PD, which allows for greater flexibility and versatility.

Applications of the MFI

Thermodynamics

Much effort has been devoted to Fisher's information measure, shedding much light upon the manifold physical applications. As a small sample, it can be shown that the whole field of thermodynamics can be derived from the MFI approach. Here FIM is specialized to the particular but important case of translation families, i.e., distribution functions whose form does not change under translational transformations. In this case, Fisher measure becomes shift-invariant. Such minimizing of Fisher's measure leads to a Schrödinger-like equation for the probability amplitude, where the ground state describes equilibrium physics and the excited states account for non-equilibrium situations.

Scale-invariant phenomena

More recently, Zipf's law has been shown to arise as the variational solution of the MFI when scale invariance is introduced in the measure, leading for the first time an explanation of this regularity from first principles. It has been also shown that MFI can be used to formulate a thermodynamics based on scale invariance instead of translational invariance, allowing the definition of the Scale-Free Ideal Gas, the scale invariant equivalent of the Ideal Gas.