We extend the theory of boosting for regression problems to the online learning setting. Generalizing from the batch setting for boosting, the notion of a weak learning algorithm is modeled as an online learning algorithm with linear loss functions that competes with a base class of regression functions, while a strong learning algorithm is an online learning algorithm with smooth convex loss f...

We introduce an efficient online learning mechanism for nonnegative sparse coding in autoencoder neural networks. In this paper we compare the novel method to the batch algorithm non-negative matrix factorization with and without sparseness constraint. We show that the efficient autoencoder yields to better sparseness and lower reconstruction errors than the batch algorithms on the MNIST benchm...

Multi-phase batch process is common in industry, such as injection molding process, fermentation and sequencing batch reactor; however, it is still an open problem to control and analyze this kind of processes. Motivated by injection molding processes, the multi-phase batch process in each cycle is formulated as a switched system with internally forced switching instant. Controlling multi-phase...

We systematically explored a spectrum of normalization algorithms related to Batch Normalization (BN) and propose a generalized formulation that simultaneously solves two major limitations of BN: (1) online learning and (2) recurrent learning. Our proposal is simpler and more biologically-plausible. Unlike previous approaches, our technique can be applied out of the box to all learning scenario...

We consider a novel “online semi-supervised learning” setting where (mostly unlabeled) data arrives sequentially in large volume, and it is impractical to store it all before learning. We propose an online manifold regularization algorithm. It differs from standard online learning in that it learns even when the input point is unlabeled. Our algorithm is based on convex programming in kernel sp...

The primitive Extreme Learning Machine (ELM) [1, 2, 3] with additive neurons and RBF kernels was implemented in batch mode. In this paper, its sequential modification based on recursive least-squares (RLS) algorithm, which referred as Online Sequential Extreme Learning Machine (OS-ELM), is introduced. Based on OS-ELM, Online Sequential Fuzzy Extreme Learning Machine (Fuzzy-ELM) is also introduc...