TECHNICAL REPORT 1999/28
J R Neil, C S Wallace and K B Korb
ABSTRACT
One factor contributing to the complexity of Bayesian network inference and discovery algorithms is the exponential number of parameters required by their parameterization. We investigate a log-linear conditional probability model for Bayesian networks in which the causes of a variable are restricted so that they do not intereact. This limits the expressive power of the model to non-monotonic interactions but has the benefit over the commonly used full conditional model of significantly reducing the number of parameters. Further, it allows the expression of a prior belief that the direction of the effect of a change in one causal factor tends to be the same regardless of other influences. We compare our first-order (FOM) and the full conditional models with a dual model that uses Wallace's Minimum Message Length (MML) to select the model most supported by the sample data. Using stochastically generated data we find that on message length, negative expected log likelihood (-ELL), and errors in the inferred parent set, the FOM outperforms the full model on average, as the strength of monotonic efforts are increased, while the converse is true with strong non-monotonic effects. The dual model compares very favorably with either of the other models individually, demonstrating the ability of MML to select successfully between the first order and full conditional models and the value of representing non-interacting causes explicitly in the hypothesis space. The full conditional model is also more likely to underfit the data, while the FOM has a tendency to overfit the data, with the dual model finding a balance between the two. In addition, the FOM's reduced parameter set make it beneficial as the number of conditioning parents is increased. Finally, we find that the FOM tends to be more accurate on parent set errors and on -ELL with small samples, N < 1000, while the full conditional model is superior with large samples, N > 5000.