School of Computer Science and Software Engineering
Monash University

Bachelor of Science (Computer Science) with Honours (0051)
Honours Thesis 2000


Improved Approximations in MML


by


Edmund Lam 12274739
1212epl@cs.monash.edu.au

Supervisor: Dr David L. Dowe
1212dld@cs.monash.edu.au

Second Examiner: Dr Kevin B. Korb
1212korb@csse.monash.edu.au

Abstract:

M

inimum Message Length (MML) is an invariant statistical inference principle which takes an information-theoretic Bayesian approach. MML provides a framework for inferring a model from a set of data. It has been developed as an approximation to Strict MML (SMML), since SMML is generally intractable. The approximations inherent within the Wallace-Freeman (1987) MML estimate are listed, analysed and improvements suggested. Three MML approximation techniques (MMLD, Asymmetric coding region, fourth-order extension) are introduced and two (MMLD, fourth-order extension) are implemented to extract experimental results. Using the Kullback-Leibler and Root-Mean-Square distances, the new techniques, when applied to the binomial distribution, are compared with some earlier MML estimates including Wallace-Boulton (1968), Wallace-Freeman (1987) and Farr-Wallace (private communications). The results shows that MMLD and fourth-order performs better, using either objective functions, than the Wallace-Freeman's (1987) MML estimate.