A Neural Network Implementation of the Moment-Preserving Technique and Its Application to Thresholding

shows the switching configuration corresponding to Fig. 3(b). Tables I and I1 summarize the simulation results, where the average number of iteration steps required for the convergence and the convergence frequency, and the average number of demands in the local minimum solutions are compared in seven models. Note that for each model, 100 simulation runs were performed from different initial values of Ut,. We conclude the simulation results as follows: 1) The comparisons of cases #1-#4 show that the decay term disturbs the convergence of the neural network to solutions. Although cases #2 and #3 are superior in the average number of iteration steps for the convergence to case #4, they are inferior in the frequency of the local minimum convergence and the solution quality to case #4. The decay term seems to make the local minimum deeper, so some initial states of Ut, can be quickly converged to the global minima. 2) The comparisons of cases #4 and #5 show that the McCulloch-Pitts neuron model and the sigmoid neuron model have similar performance in terms of the average number of iteration steps for the convergence and the convergence frequency. However, because of the exponential calculation in the sigmoid neuron model, it requires much longer computation time than the McCulloch-Pitts neuron model on a digital computer. The simple McCulloch-Pitts neuron model is superior to the sigmoid neuron model for practical uses. 3) The comparisons of cases #4-6 show that the hysteresis McCulloch-Pitts neuron model is superior to the McCulloch-Pitts neuron model and the sigmoid neuron model in terms of the frequency of the global minimum convergence. 4) The comparisons of cases #6-7 show that the two heuristics increase the frequency of the global minimum convergence, reduce the number of iteration steps for the convergence, and improve the solution quality. The hysteresis McCulloch-Pitts neuron model without the decay term and with the two heuristics provides the best performance among the seven models. We have observed similar behavior in other instances.