In this short note is studied the observer introduced in [1] and [5]. The relationship between the Lipschitz constant and the measurement stepsize is exhibited. For second order system, we evaluate the authorized measurement stepsize.

Empirical risk minimization (ERM) is recognized as a special form in standard convex optimization. When using a first order method, the Lipschitz constant of the empirical risk plays a crucial role in the convergence analysis and stepsize strategies for these problems. We derive the probabilistic bounds for such Lipschitz constants using random matrix theory. We show that, on average, the Lipsc...

Tuning stepsize between convergence rate and steady state error level or stability is a problem in some subspace tracking schemes. Methods in DPM and OJA class may show sparks in their steady state error sometimes, even with a rather small stepsize. By a study on the schemes’ updating formula, it is found that the update only happens in a specific plane but not all the subspace basis. Through a...

We consider the emphatic temporal-difference (TD) algorithm, ETD(λ), for learning the value functions of stationary policies in a discounted, finite state and action Markov decision process. The ETD(λ) algorithm was recently proposed by Sutton, Mahmood, and White [47] to solve a long-standing divergence problem of the standard TD algorithm when it is applied to off-policy training, where data f...

Conditions on Runge-Kutta algorithms can be obtained which ensure smooth stepsize selection when stability of the algorithm is restricting the stepsize. Some recently derived results are shown to hold for a more general test problem.

Hidden Markov models (HMMs) provide an excellent analysis of recordings with very poor signal/noise ratio made from systems such as ion channels which switch among a few states. This method has also recently been used for modeling the kinetic rate constants of molecular motors, where the observable variable-the position-steadily accumulates as a result of the motor's reaction cycle. We present ...

We consider the bilinear optimal control of an advection-reaction-diffusion system, where arises as velocity field in advection term. Such a problem is generally challenging from both theoretical analysis and algorithmic design perspectives, mainly because state variable depends nonlinearly on and, additional divergence-free constraint coupled together with equation. Mathematically, proof exist...