On the Evaluation of Dividing Samples for Training an Extended Depth LSA Machine

A classic problem in neural networks is the depth investigation of the network. Is there any potential benefit when training depth-one threshold circuits by adding extra layers and further training them? This question is investigated in a powerful recently introduced artificial intelligence system, called the Logarithmic Simulated Annealing (LSA) machine, that combines the Simulated Annealing Algorithm with a Logarithmic cooling schedule and the classical perceptron algorithm. The first and second layers are trained with the LSA machine learning algorithm. For the learning procedure 50% of the available data are used for training the first layer. The first layer consists of v voting functions of P threshold circuits each one. The next 25% are displayed to the first layer and the outputs of the first layer are producing new samples of length v that are used for training the second layer. The remaining 25% are used for testing the entire network. The main idea is to smooth in the second layer the inaccuracies of the first layer, by training the second layer to evaluate the significance of each output gate of the first layer. Results of the depth investigation reveal that the second layer can produce slightly better results; however the cost of using fewer examples for training the first layer is also considerable. Key-Words: Simulated Annealing, Optimisation, Perceptron Algorithm, Threshold Circuits, Classification, Machine Learning