next up previous contents
Next: Invariance of Minimum Message Up: Introduction Previous: Introduction

   
Minimum Message Length principle

The general principle of MML is to minimise the total message length of a two-part message. The first part contains the optimally coded description of the model, encoded with the aid of the prior knowledge. The second part contains the data encoded with the aid of the model. It can be a very simple model (eg none), but then every data element would have to be encoded individually without taking into account the global properties of the dataset. Alternatively, an elaborate model, which encodes the data directly within the model, could be used. This would mean that there is little or no data to encode. However, it is the belief and the basis of the MML principle that the message lengths of either of the above scenarios would be greater than the minimal message length possible. Indeed, if the optimal encoding did not require any model, it should be concluded that the data is random.

MML interprets the encoded model within the minimal message, as the model that best describes the data. That is, the model is justified on the basis of the data and is not too simplified or overly complex. As such, it can be considered as a quantitative form of ``Occam's Razor'' [#!Dowe.Oliver.Allison.Wallace.Dix:1993!#,#!Wallace:book!#,#!Needham.Dowe:2000!#] Which states, ``Entities must not be multiplied beyond what is necessary''.


next up previous contents
Next: Invariance of Minimum Message Up: Introduction Previous: Introduction
Edmund Lam
2000-12-04