Neural networks with a continuous squashing function in the output are universal approximators

In 1989 Hornik as well as Funahashi established that multilayer feedforward networks without the squashing function in the output layer are universal approximators. This result has been often used improperly because it has been applied to multilayer feedforward networks with the squashing function in the output layer. In this paper, we will prove that also this kind of neural networks are universal approximators, i.e. they are capable of approximating any Borel measurable function from one finite dimensional space into (0,1)" to any desired degree of accuracy, provided sufficiently many hidden units are available.