Multi-variate distributions

The message length expression can also be extended to multi-variate distributions. Assuming a symmetric coding region as in subsection , the coding region can be extended from an interval about $\theta$ to a hyper-cube in n-dimensions. An interval being a hyper-cube in 1-dimension.

There are some slight notational changes. The model $\theta$ is now a vector $\vec{\theta}$ in n-dimensions. The data x is now represented by $\vec{x}$. The spacing parameter $s(\vec{\theta})$ represents the volume of a hypercube. Finally, an additional approximation has been made. Each of the individual $\theta_i$ are assumed to be independent of each other. In general, this would not be true, although the original problem can always be mapped to a domain whereby the variables are independent.

The resulting message length expression is -

$$\textrm{MesgLen~} = -\log_e \frac{h(\vec{\theta})f(\vec{x}\v... ...sqrt{F(\vec{\theta})}} + \frac{n}{2} (1 + \log_e \frac{1}{12})$$ (16)

$F(\vec{\theta})$ is the expected Fisher information in higher dimensions. This corresponds with the expression

$$F(\vec{\theta}) = E_{\vec{x}} \Bigg[ \det \Big[ \frac{\partia... ...rtial \theta_j} -\log f(\vec{x}\vert\vec{\theta}) \Big] \Bigg]$$ (17)

Since each of the $\theta_i$ are independent, the matrix is a diagonal matrix, with non-diagonal entries being zero.