Simplifying Neural Network Soft Weight-sharing Measures by Soft Weight-measure Soft Weight Sharing
نویسندگان
چکیده
منابع مشابه
Soft Weight-Sharing for Neural Network Compression
The success of deep learning in numerous application domains created the desire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to a growing interest in compression. Recent work by Han et al. (2015a) propose a pipeline that involves retraining, pruning and quantization of neural network weights, obtaining sta...
متن کاملWeight Quantization for Multi-layer Perceptrons Using Soft Weight Sharing
We propose a novel approach for quantizing the weights of a multi-layer perceptron (MLP) for efficient VLSI implementation. Our approach uses soft weight sharing, previously proposed for improved generalization and considers the weights not as constant numbers but as random variables drawn from a Gaussian mixture distribution; which includes as its special cases k-means clustering and uniform q...
متن کاملAdaptive Soft Weight Tying using Gaussian Mixtures
Geoffrey E. Hinton Department of Computer Science . U ni versi ty of Toran to Toronto, Canada M5S lA4 One way of simplifying neural networks so they generalize better is to add an extra t.erm 10 the error fUll c tion that will penalize complexit.y. \Ve propose a new penalt.y t.erm in which the dist rihution of weight values is modelled as a mixture of multiple gaussians . C nder this model, a s...
متن کاملCan Soft Drink Taxes Reduce Population Weight?
Soft drink consumption has been hypothesized as one of the major factors in the growing rates of obesity in the US. Nearly two-thirds of all states currently tax soft drinks using excise taxes, sales taxes, or special exemptions to food exemptions from sales taxes to reduce consumption of this product, raise revenue, and improve public health. In this paper, we evaluate the impact of changes in...
متن کاملSingle image depth estimation by dilated deep residual convolutional neural network and soft-weight-sum inference
This paper proposes a new residual convolutional neural network (CNN) architecture for single image depth estimation. Compared with existing deep CNN based methods, our method achieves much better results with fewer training examples and model parameters. The advantages of our method come from the usage of dilated convolution, skip connection architecture and soft-weight-sum inference. Experime...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Connection Science
سال: 1994
ISSN: 0954-0091,1360-0494
DOI: 10.1080/09540099408915712