The autoencoder is an artificial neural network that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar to the eigenvalues in PCA that are paired with eigenvectors. We propose a novel su...

This paper investigates an information-theoretic approach to the problem of collaborative representation learning: how to extract salient features of statistical relationships in order to build cooperatively meaningful representations of some relevant content. Modeling the structure of data and its hidden representations by independently identically distributed samples, our goal is to study fun...

The artificial neural network with one hidden unit and the input nodes connected to the output node is considered. It is proven that the error surface of this network for the patterns of the XOR problem has minimum values with zero error and that all other stationary points of the error surface are saddle points. Also, the volume of the regions in weight space with saddle points is zero, hence ...