Observed Fisher vs Expected Fisher

The message length expression () from the previous subsection is also called the Farr-Wallace [#!Farr:1999!#,#!Wallace:2000!#] approximation. It can be used successfully, although it is not invariant [#!Farr:1999!#]. As described in subsection , the property of invariance is considered important and worth striving for, even if it means making further approximations. As such, the approximation $F(x,\theta) \approx F(\theta)$ is used, which makes the estimator invariant.

This approximation results in a message length expression that is invariant under isomorphic re-parameterisation. However, it is an approximation nonetheless and causes the failure of MML when this condition is not met [#!Grunwald.Kontkanen.Myllymaki.Silander.Tirri:1998!#,#!Grunwald:1998!#]. However, in such cases, Wallace [#!Wallace:2000!#] has suggested using the unapproximated expression () instead, at the expense of invariance. An alternative, as suggested by Dowe [#!Dowe:private!#], is to require that the message length calculations be performed with a uniform prior. As a result, if the approximation is taken, the message length can be expressed as -

$$\textrm{MesgLen~} = -\log_e \frac{h(\theta)f(x\vert\theta)}{\sqrt{F(\theta)}} + \frac{1}{2} (1 + \log_e \frac{1}{12})$$ (15)