Geoff Webb's Research Programs

Research Programs: Geoff Webb

	Learning Complex Conditional Probabilities from Data. Naive Bayes is a popular machine learning technique due to its efficiency, direct theoretical foundation and strong classification performance. Our techniques seek to strengthen its accuracy by overcoming the deficiencies of its attribute independence assumption. Averaged One Dependence Estimators (AODE) provides particularly high prediction accuracy with relatively modest computational overheads. Lazy Bayesian Rules (LBR) provides very high prediction accuracy for large training sets, and is computationally efficient when few objects are to be classified for each training set. [AODE, LBR and related papers]
	K-Optimal Pattern Discovery. I have pioneered exploratory pattern discovery techniques that seek the k optimal patterns (also known as k-most interesting patterns or top-k patterns), rather then applying the minimum-support constraint used in association rule discovery. [ K-optimal pattern discovery papers ]
	Many machine-learning researchers have utilized Occam's razor [also frequently spelt as Ockham's razor], preferring less complex classifiers in the belief that doing so is likely to reduce prediction error. I believe that this is misguided and provide philosophical and experimental support for this opinion. [ Occam's razor papers ] [ Occam's razor software ]
	OPUS is an efficient search algorithm for exploring the space of conjunctive patterns. It supports extremely fast rule discovery. [ OPUS papers ] [ OPUS pattern discovery s oftware ]
	Statistically sound exploratory pattern discovery. Exploratory pattern discovery encompasses association rule discovery, k-optimal rule discovery, emerging pattern discovery and contrast discovery. These methods explore large pattern spaces to identify all patterns that satisfy some user-specified criteria with respect to given data. Due to the large numbers of patterns that are considered, they typically suffer large risk of type-1 error, that is, of finding patterns that appear interesting only due to chance artefacts of the process by which the sample data were generated. Most attempts to control this risk have done so at the cost of high risk of type-2 error, that is, of falsely rejecting non-spurious patterns. I have developed strategies for strictly controlling type-1 error during exploratory pattern discovery without the level of risk of type-2 error suffered by previous approaches. [ Statistically sound exploratory pattern discovery papers ]
	MultiBoosting [also known as Boost Bagging] combines boosting and bagging, obtaining most of boosting's superior bias reduction together with most of bagging's superior variance reduction. MultiBoosting is an example of Multi-Strategy Ensemble Learning. We have shown that combining ensemble learning techniques can substantially reduce error. [ MultiBoosting and Multi-Strategy Ensemble Learning papers ]
	Decision tree grafting is a postprocess that adds tests and nodes to existing decision trees, resulting in bagging-like variance reduction while providing a single directly interpretable decision tree. [ Decision tree grafting papers ][ Decision tree grafting s oftware ]
	The Knowledge Factory is an interactive machine learning environment that provides tight integration between machine learning and knowledge acquisition from experts. [ Interactive machine learning papers ] [ Interactive machine learning s oftware ]
	Generality is predictive of prediction accuracy. I argue that manipulation of generality, through appropriate generalization and specialization, can modify the performance of a classifier in predictable and useful ways. [ Generality papers ]
	Prepending is a generic approach to decision list learning that has lower computation and develops shorter decision lists than the classical approach, without any general increase in prediction error. [ Prepending papers ] [ Prepending s oftware ]
	Discretization for Naive Bayes. Due to its attribute independence assumption, naive Bayes has distinct requirements of a discretization strategy. This work provides theoretical analysis of those requirements and new techniques that improve classification accuracy. [ Discretization papers ]
	Impact Rules [also known as quantitative association rules] provide analysis similar to association rules except that the target is a distribution on a numeric value. Impact Rules support segmentation for optimisation of a numeric value. [ Impact rules papers ]
	Feature Based Modeling is a generic approach to the use of attribute-value machine learning for agent modeling. Applications in student modeling have demonstrated high prediction accuracy. This program produced some of the earliest examples of inspectable or glass-box user models and student models and used associations for modeling before association rules were popularised. [ Feature Based Modeling papers ] [ Feature Based Modeling talk ]
	Bias and variance provide a powerful conceptual tool for analyzing classification performance. Previous approaches to conducting bias-variance experiments have provided little control over the types of data distribution from which bias and variance are estimated. I have developed new techniques for bias-variance analysis that provide greater control over the data distribution. Experiments show that the type of distribution used for bias-variance experiments can greatly affect the results obtained. [ Bias-variance papers ]
	Learning from large datasets. Early machine learning used very small data sets. Data mining often involves very large data sets. Work on 'scaling-up' to large data sets has concentrated on reducing the computational complexity of existing algorithms. We contend that this may not be appropriate, that learning from large data sets is fundamentally different to learning from small data sets and that different types of algorithm may be most effective. [ Learning from large data papers ]
	Bioinformatics. Along with colleagues in the Victorian Bioinformatics Consortium and the Monash School of Physics & Materials Engineering I am investigating applications of data mining in bioinformatics. [ Bioinformatics papers ]
	Engineering Applications. Along with colleagues in Engineering at Deakin University I have investigated applications of data mining in process control. [ Datamining for engineering papers ]