Recurrent neural networks (RNNs) stand at the forefront of many recent developments in deep learning. Yet a major difficulty with these models is their tendency to overfit, with dropout shown to fail when applied to recurrent layers. Recent results at the intersection of Bayesian modelling and deep learning offer a Bayesian interpretation of common deep learning techniques such as dropout. This...

A common problem encountered in statistical analysis is that of missing data, which occurs when some variables have missing values in some units. The present paper deals with the analysis of longitudinal continuous measurements with incomplete data due to non-ignorable dropout. In repeated measurements data, as one solution to a such problem, the selection model assumes a mechanism of outcome-d...

We propose a novel framework to adaptively adjust the dropout rates for the deep neural network based on a Rademacher complexity bound. The state-of-the-art deep learning algorithms impose dropout strategy to prevent feature co-adaptation. However, choosing the dropout rates remains an art of heuristics or relies on empirical grid-search over some hyperparameter space. In this work, we show the...

Missing data and especially dropouts frequently arise in longitudinal data. Maximum likelihood estimates are consistent when data are missing at random (MAR) but, as this assumption is not checkable, pattern mixture models (PMM) have been developed to deal with informative dropout. More recently, latent class models (LCM) have been proposed as a way to relax PMM assumptions. The aim of this pap...

This study examines dropout incidence, moment of dropout, and switching behavior in organized exercise programs for seniors in the Netherlands, as determined in a prospective cohort study (with baseline measurements at the start of the exercise program and follow-up after 6 months; N = 1,725, response rate 73%). Participants were community-living individuals 50+ who participated in different fo...

The big breakthrough on the ImageNet challenge in 2012 was partially due to the ‘dropout’ technique used to avoid overfitting. Here, we introduce a new approach called ‘Spectral Dropout’ to improve the generalization ability of deep neural networks. We cast the proposed approach in the form of regular Convolutional Neural Network (CNN) weight layers using a decorrelation transform with fixed ba...

Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks. This paper proposes a theoretical explanation for this phenomenon: we show that, under a generative Poisson topic model with long documents, dropout training improves the exponent in the generalization bound for empirical risk minimization. Dropout achieve...

Prior research on school dropout has often focused on stable person- and institution-level variables. In this research, we investigate longitudinally perceived stress and optimism as predictors of dropout intentions over a period of four years, and distinguish between stable and temporary predictors of dropout intentions. Findings based on a nationally representative sample of 16-20 year-olds i...