We consider partially observable Markov decision processes (POMDPs) with a set of target states and every transition is associated with an integer cost. The optimization objective we study asks to minimize the expected total cost till the target set is reached, while ensuring that the target set is reached almost-surely (with probability 1). We show that for integer costs approximating the opti...

We show that the problem of reaching a state set with probability 1 in probabilisticnondeterministic systems operating in parallel is EXPTIME-complete. We then show that this probabilistic reachability problem is EXPTIME-complete also for probabilistic timed automata.

We develop a direct Lyapunov method for the almost sure open-loop stabilizability and asymptotic stabilizability of controlled degenerate diffusion processes. The infinitesimal decrease condition for a Lyapunov function is a new form of Hamilton-Jacobi-Bellman partial differential inequality of 2nd order. We give local and global versions of the First and Second Lyapunov Theorems assuming the e...

This work is concerned with consensus problem of multi-agent systems with additive and multiplicative measurement noises. By developing general stochastic stability lemmas for nonautonomous stochastic differential equations, stochastic weak and strong consensus conditions are investigated under fixed and time-varying topologies, respectively. For the case with fixed topologies and additive nois...

In this paper, we study a new type of stochastic functional differential equations which are called hybrid pantograph (HPSFDEs). We investigate moment properties and sample the solutions such as exponential stability, almost sure stability polynomial stability. Moreover, several theoretical examples one practical example given to illustrate our theory.

Almost sure convergence rate Laws of iterated logarithms a b s t r a c t This paper presents almost sure convergence rates for system identification under binary, quantized, and regular sensors. To accommodate practical model complexity constraints, the system under consideration is represented by a modeled part together with an unknown-but-bounded unmodeled dynamics. Under uncorrelated noise s...

In this paper we study the stability (in the L as well as for the almost sure convergence sense) of the optimal investment-consumption strategy with respect to the choice of the utility function.

The exponential stability, in both mean square and almost sure senses, for energy solutions to a nonlinear and non-autonomous stochastic PDEs with finite memory is investigated. Various criteria for stability are obtained. An example is presented to demonstrate the main results.

This paper mainly tends to utilize [Formula: see text]-type function to investigate p-th moment and almost sure stability for a class of stochastic switched nonlinear systems. Based on the multiple Lyapunov functions approach, some sufficient conditions are derived to check the stability criteria of stochastic switched nonlinear systems. One numerical example is provided to demonstrate the effe...

This article considers the robust exponential stability of uncertain switched stochastic systems with time-delay. Both almost sure (sample) stability and stability in mean square are investigated. Based on Lyapunov functional methods and linear matrix inequality techniques, new criteria for exponential robust stability of switched stochastic delay systems with non-linear uncertainties are deriv...