Further Work

Neither the Kullback-Leibler distance or the Root-Mean-Square distance are ideal objective metrics when ranking MML estimators. This is one of the limitations of this thesis which can be remedied by choosing an alternative objective function. If more time was available, the absolute error between the SMML estimator and the MML estimators would have been ideal. Although, this is feasible only for the binomial distribution. Other potential objective metrics includes the Euclidean distance or the total squared error - which will penalise larger errors more. Nonetheless, the objective metrics chosen still provide useful results regarding comparisons between the five MML estimators tested.

When the Kullback-Leibler distance or Root-Mean-Square distance is used in isolation, the results both indicate that ``4th'' and ``MMLD'' performs better than ``WF87'', but does not necessarily conclude that they are the best MML estimators. However, when the results are combined, it becomes clearer that ``MMLD'' and ``4th'' estimators produced better results overall than the other estimators, especially ``WF87''. Therefore, this thesis has succeeded in improving the ``WF87'' MML estimate.

Regardless of the relative ranking of the estimators, the absolute difference between the estimators (figure and ) also has to be considered. ``4th'' and ``MMLD'' might produce more accurate results than ``WF87'', but it also requires greater amounts of computational time.

Due to a lack of time, other desirable areas of work remain incomplete. The MMLD and fourth-order MML approximation techniques should be applied to more distributions, both univariate and multivariate. The beginnings of some of these attempts are found in the Appendices. The asymmetric coding region should also be investigated to see if a parameter estimate can be derived. The implementation of the binomial experiment is not as flexible or efficient as it could be. Correctness was more highly valued when coding the numerical experiments. The increased use of dynamic programming techniques should substantially decrease running times.

There appears to be no clear framework for applying MML to problems with discrete model-spaces. It would be useful and interesting to determine an appropriate definition of a neighbourhood or coding region about a discrete model. Alternatively, under what criterion can a discrete model $\theta$ be considered to have a neighbourhood at all. If no neighbourhood exists, then the model can be effectively described as being independent of other models within the model-space. A possible avenue of investigation would be to treat the discrete distributions as continuous distributions. Although the ``continuous'' distribution will consist of distinctly non-continuous $\delta$-functions. Obviously, this would mean that the second (or higher) order approximations cannot be taken. However, it should still be possible for MMLD to be applied.

There have also been additional questions raised during the experimentation in this thesis. Additional assumptions must be made about the coding region or likelihood function within the region for an effective algorithm to apply MML in higher dimensions. For the fourth-order extension, the current parameter estimator is not invariant unless the requirement of an uniform prior is imposed [#!Dowe:private!#]. Therefore, it would be an interesting whether the fourth-order approximation can be altered to be invariant via other means.