We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within the pooling region. The approach is hyper-parameter free and can be combined wi...

Monotonic Regression (MR) is a standard method for extracting a monotone function from non-monotonic data, and it is used in many applications. However, a known drawback of this method is that its fitted response is a piecewise constant function, while practical response functions are often required to be continuous. The method proposed in this paper achieves monotonicity and smoothness of the ...

Recent studies have shown that the generalization ability of deep neural networks (DNNs) is closely related to Fisher information matrix (FIM) calculated during early training phase. Several methods been proposed regularize FIM for increased DNNs. However, they cannot be used directly Bayesian (BNNs) because variable parameters BNNs make it difficult calculate FIM. To address this problem, we a...

Artificial neural networks have been used as predictive systems for variety of medical domains, but none of the systems encountered by Baxt (1995) and Dybowski & Gant (1995) in their review of the literature provided any measure of confidence in the predictions made by those systems. In a medical setting, measures of confidence are of paramount importance (Holst, Ohlsson, Peterson & Edenbrandt ...

This paper studies the performance of neural networks estimated with Bayesian regularization to model and forecast time series where the data generating process is in fact linear. A simulation experiment is carried out to compare the forecasts made by linear autoregressive models and neural networks.

This work considers the applicability of applying the derivatives of stepwise linear regression modelling (specifically the p-values which indicate the importance of a variable to the modelling process) as a feature extraction technique. We utilise it in conjunction with several data sets of varying levels of complexity, and compare our results to other dimensionality reduction techniques such ...