Dropout Variational Inference, or Dropout Sampling, has been recently proposed as an approximation technique for Bayesian Deep Learning and evaluated for image classification and regression tasks. This paper investigates the utility of Dropout Sampling for object detection for the first time. We demonstrate how label uncertainty can be extracted from a state-of-the-art object detection system v...

Dropout is a relatively new algorithm for training neural networks which relies on stochastically “dropping out” neurons during training in order to avoid the co-adaptation of feature detectors. We introduce a general formalism for studying dropout on either units or connections, with arbitrary probability values, and use it to analyze the averaging and regularizing properties of dropout in bot...

• Gal and Gharamani (2015) reinterpreted dropout regularisation as approximate inference in BNNs •Dropout probabilities pl are variational parameters of the approximate posterior qθ(ω) = ∏ k qMk,pk(Wk), where Wk = Mk · diag (zk) and zkl iid ∼Bernoulli(1− pk) • Concrete distribution (Maddison et al., Jang et al.) relaxes Categorical distribution to obtain gradients wrt the probability vector – E...

Recurrent neural networks (RNNs) form an important class of architectures among neural networks useful for language modeling and sequential prediction. However, optimizing RNNs is known to be harder compared to feed-forward neural networks. A number of techniques have been proposed in literature to address this problem. In this paper we propose a simple technique called fraternal dropout that t...

Recently, dropout has seen increasing use in deep learning. For deep convolutional neural networks, dropout is known to work well in fully-connected layers. However, its effect in pooling layers is still not clear. This paper demonstrates that max-pooling dropout is equivalent to randomly picking activation based on a multinomial distribution at training time. In light of this insight, we advoc...

This paper presents the design of an output-capacitorless low-dropout voltage regulator (OCL-LDO) capable driving a wide range load capacitance and supplying current while maintaining excellent line regulations, thanks to combined indirect-direct nested Miller compensation which ensures stability maintains high loop gain over whole condition. Fabricated in 0.18-μm CMOS process consuming ...

In this paper we analyze the major determinants of university enrollment and dropout in Brazil. The econometric model consists of two simultaneous equations. The first equation determines whether the student was accepted at the university or not; the second determines students decision to dropout or not from higher education, given they were accepted at the university. Gender, age and marriage ...

Long Short-Term Memory networks (LSTMs) are a component of many state-of-the-art DNN-based speech recognition systems. Dropout is a popular method to improve generalization in DNN training. In this paper we describe extensive experiments in which we investigated the best way to combine dropout with LSTMs– specifically, projected LSTMs (LSTMP). We investigated various locations in the LSTM to pl...

We consider the problem of designing models to leverage a recently introduced approximate model averaging technique called dropout. We define a simple new model called maxout (so named because its output is the max of a set of inputs, and because it is a natural companion to dropout) designed to both facilitate optimization by dropout and improve the accuracy of dropout’s fast approximate model...