Gaussian Distribution

During the year, I have perform some mathematical derivation for the single-variate Gaussian distribution that has not reached a sufficient state to be included in the main thesis. This is mainly due to a lack of time.

The Gaussian distribution studied within this Appendix is the single-variate version with an uniform prior over $\mu$ and a constant $\sigma$. The Gaussian distribution has a prior and likelihood function as follows -

$$\begin{split} h(\mu) &= \frac{1}{\mu_{max}-\mu_{min}} \\ f(x... ...igma\sqrt{2\pi}}e^{-\frac{1}{2\sigma^2}(x_i-\mu)^2} \end{split}$$ (71)

The Observed Fisher $F(x, \mu)$ can be calculated to be

$$\begin{split} F(x, \mu) &= \frac{\partial^2}{\partial \mu^2}... ...^N \frac{1}{\sigma^2} \\ &= \frac{N}{\sigma^2} \\ \end{split}$$ (72)

The variable $G(x, \mu)$ is therefore

$$G(x, \mu) = \frac{\partial^2}{\partial \mu^2} [\frac{N}{\sigma^2}] = 0$$ (73)

Since neither $F(x, \mu)$ or $G(x, \mu)$ depends on x, the expected versions are exactly the same. Therefore, there is no difference between the Wallace and Freeman [#!Wallace.Freeman:1987!#] MML estimator and the Farr-Wallace estimator [#!Farr:1999!#] (ie the Wallace and Freeman 1987 MML approximation with Observed Fisher). Furthermore, since $G(x, \mu)=0$, the fourth(and higher) order extension of the message length is exactly the same as the second-order case.

$$\begin{split} \textrm{MesgLen} &= -\log_e \Big[ \frac{h(\mu)... ...{2\sigma^2}(x_i-\mu)^2 \Bigg] + (1 - \log_e 12) \\ \end{split}$$ (74)

If we differentiate with respect to $\mu$, we get

$$\begin{split} 0 &= \sum_{i=1}^N \frac{-1}{\sigma^2}(x_i-\mu) ... ...{\mu} &= \frac{1}{N} \sum_{i=1}^N x_i = \bar{x} \\ \end{split}$$ (75)

Therefore, the second-order MML estimator is $\bar{x}$. This is exactly the same as the Maximum Likelihood, second-order with Observed Fisher, fourth-order and MMLD [#!Dowe:private!#] estimator.