This paper investigates the use of three back-propagation training algorithms, Levenberg-Marquardt, conjugate gradient and resilient back-propagation, for the two case studies, stream-flow forecasting and determination of lateral stress in cohesionless soils. Several neural network (NN) algorithms have been reported in the literature. They include various representations and architectures and t...

Many algorithms have been recently reported for the training of analog multi-layer perceptron. Most of these algorithms were evaluated either from a computational or simulation view point. This paper applies several of these algorithms to the training of an analog multi-layer perceptron chip. The advantages and shortcomings of these algorithms in terms of training and generalisation performance...

We introduce coactive learning as a distributed learning approach to data mining in networked and distributed databases. The coactive learning algorithms act on independent data sets and cooperate by communicating training information, which is used to guide the algorithms’ hypothesis construction. The exchanged training information is limited to examples and responses to examples. It is shown ...

For realtime pattern classification applications (e.g. realtime image segmentation), the number of usable pattern classification algorithms is limited by the feasibility of high-speed hardware implementation. This paper describes a pattern classifier and associated hardware architecture and training algorithms. The classifier has both a feasible hardware implementation and other desirable prope...

This paper reviews diierent approaches to improving the real time recurrent learning (RTRL) algorithm and attempts to group them into common frameworks. The characteristics of sub-grouping strategy, mode exchange RTRL, and cellular genetic algorithms are discussed. The relationships between these algorithms are highlighted and their time complexities and convergence capability are compared. The...

Learning to Optimize (Li & Malik, 2016) is a recently proposed framework for learning optimization algorithms using reinforcement learning. In this paper, we explore learning an optimization algorithm for training shallow neural nets. Such high-dimensional stochastic optimization problems present interesting challenges for existing reinforcement learning algorithms. We develop an extension that...

Support vector machines (SVMs) are a very popular method for binary classification. Traditional training algorithms for SVMs, such as chunking and SMO, scale superlinearly with the number of examples, which quickly becomes infeasible for large training sets. Since it has been commonly observed that dataset sizes have been growing steadily larger over the past few years, this necessitates the de...

Machine learning algorithms have been known to perform better with more free parameters to tune and more training data. But learning algorithms are often too slow for large scale applications and thus the size of training models (free parameters) and data is limited in practice. So usage of GPUs to improve the speeds of these algorithms has attracted a lot of attention recently. I focused on a ...

Training deep neural networks is a highly nontrivial task, involving carefully selecting appropriate training algorithms, scheduling step sizes and tuning other hyperparameters. Trying different combinations can be quite labor-intensive and time consuming. Recently, researchers have tried to use deep learning algorithms to exploit the landscape of the loss function of the training problem of in...

In classifier (or regression) fusion the aim is to combine the outputs of several algorithms to boost overall performance. Standard supervised fusion algorithms often require accurate and precise training labels. However, accurate labels may be difficult to obtain in many remote sensing applications. This paper proposes novel classification and regression fusion models that can be trained given...