Although feedforward neural networks are well suited to function approximation, in some applications networks experience problems when learning a desired function. One problem is interference which occurs when learning in one area of the input space causes unlearning in another area. Networks that are less susceptible to interference are referred to as spatially local networks. To understand th...

We experimentally demonstrate a sigmoidal variation of the composition profile across semiconductor heterointerfaces. The wide range of material systems (III-arsenides, III-antimonides, III-V quaternary compounds, III-nitrides) exhibiting such a profile suggests a universal behavior. We show that sigmoidal profiles emerge from a simple model of cooperative growth mediated by two-dimensional isl...

This paper proves that the task of computing near optimal weights for sigmoidal nodes under the L regression norm is NP Hard For the special case where the sigmoid is piecewise linear we prove a slightly stronger result namely that computing the optimal weights is NP Hard These results parallel that for the one node pattern recognition problem namely that determining the optimal weights for a t...

A general branching process model is proposed to describe the shortening of telomeres in eukaryotic chromosomes. The model is flexible and incorporates many special cases to be found in the literature. In particular, we show how telomere shortening can give rise to sigmoidal growth curves, an idea first expressed by Portugal et al. [A computational model for telomere-dependent cell-replicative ...

1 1+x 2. This shows that arbitrary (not exp-ra denable) analytic functions may result in architectures with in-nite VC dimension. (Moreover, the architecture used is the simplest one that appears in neural nets practice.) Note that if we wish the x i 's to be bounded, for instance to be restricted to the interval [01; 1], one may replace the above x i 's and w j 's by xi c and cw j , where c = ...

Deep neural network acoustic models produce substantial gains in large vocabulary continuous speech recognition systems. Emerging work with rectified linear (ReL) hidden units demonstrates additional gains in final system performance relative to more commonly used sigmoidal nonlinearities. In this work, we explore the use of deep rectifier networks as acoustic models for the 300 hour Switchboar...

A neural tree is a feedforward neural network with at most one edge outgoing from each node. We investigate the number of examples that a learning algorithm needs when using neural trees as hypothesis class. We give bounds for this sample complexity in terms of the VC dimension. We consider trees consisting of threshold, sigmoidal and linear gates. In particular, we show that the class of thres...

Interference in neural networks occurs when learning in one area of the input space causes unlearning in another area. Networks that are less susceptible to interference are referred to as spatially local networks. To obtain a better understanding of these properties, a theoretical framework, consisting of a measure of interference and a measure of network localization, is developed. These meas...