SCHOOL OF COMPUTER SCIENCE AND SOFTWARE ENGINEERING
MONASH UNIVERSITY
--------------------------------------------------------------------------------
Leigh
J. Fitzgibbon, PhD
Supervisors: Dr Lloyd Allison and Associate Professor David Dowe
ABSTRACT
The Minimum Message Length (MML) principle recasts inductive inference as a coding process. It requires the construction of a codebook (or part thereof) that would hypothetically allow for the noiseless transmission of the observed data in a two-part message as briefly as possible. The MML codebook determines an epitome of the posterior distribution that can be used for the purposes of Bayesian Posterior Comprehension (BPC): point estimation, human comprehension, and fast approximation of posterior expectations.
In this dessertation, new methods of constructing MML codebooks, that are closer to the theoretically optimal codebook by design than existing methods, are devised and investgated. These more efficient codebooks produce a succinct epitome of the posterior distribution than can be used for BPC purposes.
A large portion of the algorithms and approximations presented are based on Markov chain Monte Carlo (MCMC) methods. The general strategy employed is to partition a sample from the posterior distribution of the parameters into regions that approximate entries in the MML codebook. Message lengths and point estimates are approximated for these regions using Monte Carlo integration with importance sampling.
This methodology is applied to multiple change-point and regression problems using Gibbs and Reversible Jump Markov chain Monte Carlo samplers. However, in theory, it can be applied to any problem provided that a suitable posterior sampler and means of calculating, or approximating, the Kullback-Leibler distance are available. Therefore, the methodology is widely applicable and can be used as a post-processing method for producing an epitome of the posterior distribution in Bayesian Markov chain Monte Carlo analyses.