This note provides a family of classification problems, indexed by a positive integer k, where all shallow networks with fewer than exponentially (in k) many nodes exhibit error at least 1/6, whereas a deep network with 2 nodes in each of 2k layers achieves zero error, as does a recurrent network with 3 distinct nodes iterated k times. The proof is elementary, and the networks are standard feed...

This contribution presents the application of feed-forward neural networks to the problem of time series forecasting. This forecast technique is applied to the water flow and wind speed time series. The results obtained from the forecasting of these two renewable resources can be used to determine the power generation capacity of micro or mini-hydraulic plants, and wind parks, respectively. The...

The central theme of this paper is to overcome the inability of feedforward neural networks with hard limiting units to provide confidence evaluation. We consider a Madaline architecture for a 2-group classification problem and concentrate on the probability density function for the neural activation of the first-layer units. As the following layers perform a Boolean table, the expectation valu...

Many methods to codify Artificial Neural Networks have been developed to avoid the defects of direct encoding schema, improving the search into the solution's space. A method to estimate how the search space is covered and how are the movements along search process applying genetic operators is needed in order to evaluate the different encoding strategies for Feedforward Neural Networks. A firs...

Existing metrics for the learning performance of feed-forward neural networks do not provide a satisfactory basis for comparison because the choice of the training epoch limit can determine the results of the comparison. I propose new metrics which have the desirable property of being independent of the training epoch limit. The efficiency measures the yield of correct networks in proportion to...

We present a generalization bound for feedforward neural networks in terms of the product of the spectral norms of the layers and the Frobenius norm of the weights. The generalization bound is derived using a PAC-Bayes analysis.