Bibliography of the paper "Simplifying Neural networks by Soft Weight-Sharing"

[   About   |  Browse   |   Statistics   ]

Number of references:	1526	Last update:	December 3, 1993
Number of online publications:	0	Supported:	no
Most recent reference:	1992

Search the bibliography

Information on the Bibliography

Authors:

Steven J. Nowlan
Computational Neurobiology Laboratory
The Salk Institute
P.O. Box 85800
San Diego, CA 92186-5800, USA

Geoffrey E. Hinton
Department of Computer Science
University of Toronto
Toronto, Canada M5S 1A4

Abstract:

Abstract of "Simplifying Neural networks by Soft Weight-Sharing":

One way of simplifying neural networks so they generalize better is to add an extra term to the error function that will penalize complexity. Simple versions of this approach include penalizing the sum of the squares of the weights or penalizing the number of non-zero weights. We propose a more complicated penalty term in which the distribution of weight values is modelled as a mixture of multiple gaussians. A set of weights is simple if the weights have high probability densities under the mixture model. This can be achieved by clustering the weights into subsets with the weights in each cluster having very similar values. Since we do not know the appropriate means or variances of the clusters in advance, we allow the parameters of the mixture model to adapt at the same time as the network learns. Simulations on two different problems demonstrate that this complexity term is more effective than previous complexity terms.

Browsing the bibliography

• Original source:

  Used to be available at
  ftp://helmholtz.sdsc.edu/pub/nowlan/soft-share/
  but has disappeared.

• Local copy in BibTeX format (134 KB, compressed with gzip)

Bibliographic Statistics

Types:

article(637), inproceedings(229), techreport(228), incollection(179), misc(93), book(92), unpublished(35), phdthesis(25), mastersthesis(5), inbook(3)

Fields:

title(1526), year(1524), author(1523), key(984), pages(767), journal(640), volume(631), booktitle(413), publisher(380), address(357), number(234), institution(232), annote(217), note(170), type(147), editor(142), month(122), organization(79), bibdate(70), editors(66), keywords(58), school(30), chapter(10), place(4), edition(3), page(3), howpublished(2), notes(2), addresss(1), comment(1), ken(1), keyword(1)

Distribution of publication dates:

Please direct comments regarding the bibliography collection to <liinwwwa@ira.uka.de>. This page is part of the Computer Science Bibliography Collection. Copyright © 1994-2004, Alf-Christian Achilles. All Rights Reserved.