Constructing Hidden Units Using Examples and Queries
نویسندگان
چکیده
While the network loading problem for 2-layer threshold nets is NP-hard when learning from examples alone (as with backpropagation), (Baum, 91) has now proved that a learner can employ queries to evade the hidden unit credit assignment problem and PAC-load nets with up to four hidden units in polynomial time. Empirical tests show that the method can also learn far more complicated functions such as randomly generated networks with 200 hidden units. The algorithm easily approximates Wieland's 2-spirals function using a single layer of 50 hidden units, and requires only 30 minutes of CPU time to learn 200-bit parity to 99.7% accuracy.
منابع مشابه
A new strategy for adaptively constructing multilayer feedforward neural networks
In this paper a new strategy for adaptively and autonomously constructing a multi-hidden-layer feedforward neural network (FNN) is introduced. The proposed scheme belongs to a class of structure level adaptation algorithms that adds both new hidden units and new hidden layers one at a time when it is determined to be needed. Using this strategy, a FNN may be constructed having as many hidden la...
متن کاملA Hidden Manifesto
This paper describes a programme of research that aims to combine the advantages of the object and logic paradigms using`hidden algebra'. We give examples to show that this provides a foundation for specifying, constructing and verifying systems of concurrent, interacting objects.
متن کاملImproving Generalization Performance of Neural Networks by Constructing Orthonormal Weight Vectors
A new method to avoid overfitting in two-layered feed forward networks is presented. By adding a penalty term to the usual sum square error function orthonormal weight vectors from the input to the hidden units are built. The idea is to restrict the weight space by constructing distinct features in the hidden units. By applying analytical methods we develop this penalty term without introducing...
متن کاملUnsupervised Hidden Markov Modeling of Spoken Queries for Spoken Term Detection without Speech Recognition
We propose an unsupervised technique to model the spoken query using hidden Markov model (HMM) for spoken term detection without speech recognition. By unsupervised segmentation, clustering and training, a set of HMMs, referred to as acoustic segment HMMs (ASHMMs), is generated from the spoken archive to model the signal variations and frame trajectories. An unsupervised technique is also desig...
متن کاملQuery learning for maximum information gain in amulti - layer neural
In supervised learning, the redundancy contained in random examples can be avoided by learning from queries, where training examples are chosen to be maximally informative. Using the tools of statistical mechanics, we analyse query learning in a simple multi-layer network, namely, a large tree-committee machine. The generalization error is found to decrease exponentially with the number of trai...
متن کامل