Point estimation of the parameter

Once an optimal coding region is found, it would be useful to convert it to a point estimate of the parameter. The parameter estimation is derived from the optimal coding region using the posterior probability function $g(\theta\vert x)$and the probability $p(y\vert\theta)$ that future data y is generated given the model $\theta$ [#!Dowe:private!#]. The method is convolving [#!Dowe.Baxter.Oliver.Wallace:1998!#] the posterior $g(\theta\vert x)$ over the probability of future data $p(y\vert\theta)$ to produce a population of future expected data, then finding the most likely model using a Maximum Likelihood fit.

The method for generating point estimate from an optimal coding region is similar to the method described by Dowe et.al. [#!Dowe.Baxter.Oliver.Wallace:1998!#] for deriving the minEKL. The difference between MMLD and minEKL, is that integration occurs over the optimal coding region for MMLD and not the whole parameter-space as for minEKL.

$$\begin{split} \hat{\theta}(y) &= \Big[ \int_R h(\theta)f(x\v... ..._R h(\theta)f(x\vert\theta) p(y\vert\theta) d\theta \end{split}$$ (22)

where $\Big[ \int_R h(\theta)f(x\vert\theta) d\theta \Big]^{-1}$ is the normalisation constant. $\hat{\theta}$ is then the maximum likelihood fit to $\hat{\theta}(y)$. In multiple dimensions, this equation converts to

$$\begin{split} \hat{\theta}_i(y) &= \Big[ \int_R h(\vec{\thet... ...\vec{\theta}) p(y_i\vert\vec{\theta}) d\vec{\theta} \end{split}$$ (23)

where $\theta_i$ is the parameter affecting the data y_i and $\Big[ \int_R h(\vec{\theta}) f(\vec{x}\vert\vec{\theta}) d\vec{\theta} \Big]^{-1}$is the normalisation constant. Again $\hat{\theta}_i$ is then the maximum likelihood fit to $\hat{\theta}_i(y)$.