Neuro-dynamic programming (NDP for short) is a relatively new class of dynamic programming methods for control and sequential decision making under uncertainty. These methods have the potential of dealing with problems that for a long time were thought to be intractable due to either a large state space or the lack of an accurate model. They combine ideas from the fields of neural networks, art...

In this paper we present stability and convergence results for Dynamic Programming-based reinforcement learning applied to Linear Quadratic Regulation (LQR). The specific algorithm we analyze is based on Q-learning and it is proven to converge to the optimal controller provided that the underlying system is controllable and a particular signal vector is persistently excited. This is the first c...

A novel two-dimensional warping algorithm is presented which searches for the optimal pixel mapping subject to continuity and monotonicity constraints. These constraints enable us to preserve topological structure in images. The search algorithm is based on dynamic programming (DP). As implementation techniques, acceleration by beam search and excessive warp suppression by penalty and/or range ...

Millions of three-wheelers in large cities Asia and Africa contribute to the already increasing urban air pollutants. An emerging method reduce adverse effects growing three-wheeler fleet is hybrid-electric technology. The overall efficiency a hybrid electric vehicle heavily depends on power management strategy used controlling main powertrain components vehicle. Recent studies highlight need f...

We propose and solve a stochastic dynamic programming (DP) formulation of the optimal provision of regulation service reserves (RSR) by controlling dynamic demand preferences in smart buildings. A major contribution over past dynamic pricing work is that we pioneer the relaxation of static, uniformly distributed utility of demand. In this paper we model explicitly the dynamics of energy user pr...

Second order elliptic problems with discontinuous coefficients are considered. The problem is discretized by the finite element method on geometrically conforming non-matching triangulations across the interface using the mortar technique. The resulting discrete problem is solved by a FETI-DP method. We prove that the method is convergent and its rate of convergence is almost optimal and indepe...

The minimum tour duration problem (MTDP) is the variant of the traveling salesman problem with time windows, which consists of finding a time window-feasible Hamiltonian path minimizing the tour duration. We present a new effective dynamic programming (DP)-based approach for the MTDP. When solving the traveling salesman problem with time windows with DP, two independent resources are propagated...

In the theory of dynamic programming (DP) the elimination of non-optimal actions is an important topic. For many DP problems the calculation is slow and action elimination helps to speed up the calculation. A great part of this thesis is dedicated to the development of action elimination procedures for various classes of DP problems. Common to all these action elimination procedures is that the...

This paper proposes Mandarin phrase recognition using dynamic programming (DP) prediction errors of singleton-type recurrent neural fuzzy networks (SRNFNs). This method is called DP-SRNFN. The recurrent property of SRNFN makes it suitable for processing temporal speech patterns. A Mandarin phrase comprises monosyllabic words. SRNFN training is based on the word unit. There are Nw SRNFNs for mod...