The focus of this paper is on stochastic variational inequalities (VI) under Markovian noise. A prominent application our algorithmic developments the policy evaluation problem in reinforcement learning. Prior investigations literature focused temporal difference (TD) learning by employing nonsmooth finite time analysis motivated subgradient descent leading to certain limitations. These limitat...

This work, Part II, together with its companion-Part I develops a new framework for stochastic functional Kolmogorov equations, which are nonlinear differential equations depending on the current as well past states. Because of complexity problems, it is natural to divide our contributions into two parts answer long-standing question in biology and ecology. What minimal conditions long-term per...

Over the last decade formal methods have been extended towards performance and reliability evaluation. This paper tries to provide a rather intuitive explanation of the basic concepts and features in this area. The intention is to give an illustrative introduction to the basics of stochastic models, to stochastic modelling using process algebra, and to model checking as a technique to analyse s...

This paper refines the framework of ‘Formal Methods in Conformance Testing’ by introducing probabilities for concepts which have a stochastic nature. Test execution is refined into test runs, where each test run is considered as a stochastic process that returns a possible observation with a certain probability. This implies that not every possible observation that could be made, will actually ...

Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a reliable estimator of the marginal likelihood and iv) is a generalisation of de...

The development of surrogate models to study uncertainties in hydrologic systems requires significant effort the sampling strategies and forward model simulations. Furthermore, applications where prediction time is critical, such as hurricane storm surge, predictions system response can be required within short frames. Here, we develop an efficient stochastic shallow water address these issues....