CSE5312 Neural Networks Syllabus

Neural Network Fundamentals

Dr Andrew P. Paplinski, /www.csse.monash.edu.au/~app

Syllabus

This units examines mathematical and computational fundamentals of artificial neural networks and their applications in signal and image processing, pattern recognition and modelling. The syllabus includes:

Basic concepts of neurocomputing:: Artificial Neural Networks (ANN) and their biological roots and motivations. ANNs as numerical data/signal/image processing devices. Encoding (training phase) and decoding (active phase).Taxonomy of neural networks: feedforward and recurrent networks with supervised and unsupervised learning laws. Static and dynamic processing systems. Basic data structures: mapping of vector spaces, clusters, principal components.
Basic terminology related to an artificial neuron:: a summing dendrite, synapses and their weights, pre- and post-synaptic signals, activation potential and activation function. Excitatory and inhibitory synapses. The biasing input. Types of activating functions.
The Perceptron: The Perceptron and its learning law. Classification of linearly separable patterns.
Linear Networks.: Adaline --- the adaptive linear element. Linear regression. The Wiener-Hopf equation. The Least-Mean-Square (Widrow-Hoff) learning algorithm. Method of steepest descent. Adaline as a linear adaptive filter. A sequential regression algorithm.
Multi-Layer Feedforward Neural Networks:: aka Multi-Layer Perceptrons. Supervised Learning. Approximation and interpolation of functions. Back-Propagation Learning law. Fast training algorithms. Applications of multilayer perceptrons: Image coding, Paint-quality inspection, Nettalk.
Self-Organising systems.: Unsupervised Learning. Local learning laws. Generalised Hebbian Algorithm. The Oja's and Sanger's rules. Principal component analysis --- Karhunen-Loeve transform.
Competitive Learning:: MinNet and MaxNet networks. Clustering. Learning Vector Quantisation. Codebooks. Application in data compression.
Self-Organising Feature Maps: Kohonen networks.
Radial-Basis function networks: Radial-Basis function (RBF) networks and their application in function interpolation, approximation and modelling probability distributions.
Recurrent networks: Hopfield networks.

Prerequisite knowledge

Basic knowledge of vectors and matrices is assumed.

Recommended references:

Simon Haykin, Neural Networks -- a Comprehensive Foundation, Prentice Hall, 2nd ed., 1999, ISBN 0-13-273350-1
Andrew P. Paplinski -- Lecture notes: ~app/CSE5312
H. Demuth, M. Beale, Neural Network Toolbox User's Guide. For use with MATLAB, The MathWorks Inc, (file:/sw/matlab/help/fulldocset.html from Unix workstations)
Martin T. Hagan,. H. Demuth,M. Beale, Neural Network Design, PWS Publishing, 1996, ISBN 0-534-94332-2

Computing Requirements

Practical work related to the subject is based on the MATLAB package. MATLAB is available on many Unix/Linux platforms. To run it try /usr/local/bin/matlab on fangorn or nexus.

Subject structure and organisation

The subject format is based on two hours a week of lectures and practical work conducted in an unsupervised mode. It is expected that a student spends approximately 4 hours a week on practical work.

week	Topic
1	Artificial Neural Networks and their Biological Motivation
2	Basic structures and properties of Artificial Neural Networks
3	Perceptron, its learning law and applications
4	Adaline -- The Adaptive Linear Element, its Strucure and Learning laws.
5	Feedforward Multilayer Neural Networks. Backpropagation algorithm.
6	Applications of Multilayer Neural networks.
7	Advanced Learning Algorithms for Multilayer Perceptrons
8	Generalised Hebbian Learning. Principal Component Analysis.
9	Competitive Neural Networks. Vector Quantization
10	Self-Organizing Feature Maps.
11	Hopfield Networks
12	RBF Networks
13	Revision

Plagiarism

Students should consult University materials on cheating, in particular:

Student Resource Guide - section on Student Rights and Responsibilities at http://www.monash.edu.au/pubs/handbooks/srg/srg0059.htm
Student Resource Guide at http://www.monash.edu.au/pubs/handbooks/srg/, particularly the section on Cheating at http://www.monash.edu.au/pubs/handbooks/srg/srg0071.htm
Faculty policy at http://www.csse.monash.edu.au/~ajh/adt/policies/cheating.html
Statute 4.1 on Discipline at http://www.monash.edu.au/pubs/calendar/statutes/statute4.1.html

It is the student's responsibility to make themselves familiar with the contents of these documents.

Andrew P. Paplinski
20 June 2004