Bayesian dropout

Dropout has recently emerged as a powerful and simple method for training neural networks preventing co-adaptation by stochastically omitting neurons. is currently not grounded in explicit modelling assumptions which so far precluded its adoption Bayesian modelling. Using entropic reasoning we show that dropout can be interpreted optimal inference under constraints. We demonstrate this on an analytically tractable regression model providing interpretation of mechanism regularizing well connection to other techniques. also discuss two general approximate techniques applying models, one based analytical approximation the stochastic variational These are then applied Baysian logistic problem shown improve performance become more misspecified. Our framework roots theoretically justified practical tool statistical allowing Bayesians tap into benefits training.