Policy Gradient methods are model-free reinforcement learning algorithms which in recent years have been successfully applied to many real-world problems. Typically, Likelihood Ratio (LR) methods are used to estimate the gradient, but they suffer from high variance due to random exploration at every time step of each training episode. Our solution to this problem is to introduce a state-depende...

We are interested in stationary “fluid” random evolutions with independent increments. Under some mild assumptions, we show they are solutions of a stochastic differential equation (SDE). There are situations where these evolutions are not described by flows of diffeomorphisms, but by coalescing flows or by flows of probability kernels. In an intermediate phase, for which there exists a coalesc...

This paper explores a framework for recognition of image sequences using partially observable stochastic differential equation (SDE) models. Monte-Carlo importance sampling techniques are used for efficient estimation of sequence likelihoods and sequence likelihood gradients. Once the network dynamics are learned, we apply the SDE models to sequence recognition tasks in a manner similar to the ...

There is a need for better methods to quantify regional myocardial function. In the present study, we investigated the feasibility of quantifying regional function in terms of a segmental myocardial work index as derived from strain Doppler echocardiography (SDE) and invasive pressure. In 10 anesthetized dogs, we measured left ventricular (LV) pressure by micromanometer and myocardial longitudi...

OBJECTIVE To compare motivational interviewing-based education (MI) and structured diabetes education (SDE) for improving A1C and psychosocial measures in adolescents with type 1 diabetes. RESEARCH DESIGN AND METHODS This study was a 9-month randomized controlled trial comparing MI (n = 21) to SDE (n = 23). Interventions were at baseline (T0) and 3 months (T1), with A1C and psychosocial measure...

The present study investigated the effect of ceiling height on the symbolic distance effect (SDE). We hypothesized that a high/low ceiling height would produce a small/large SDE through divergent/convergent thinking. We prepared three SDE tasks: numerical, spatial, and temporal. Participants conducted these tasks in both high and low ceiling conditions. Results showed that in the spatial and te...

Stochastic differential equations (SDE) are used to model many situations including population dynamics, protein kinetics, turbulence, finance, and engineering [5, 6, 1]. Knowing the solution of the SDE in question leads to interesting analysis of the trajectories. Most SDE are unsolvable analytically and other methods must be used to analyze properties of the stochastic process. From the SDE, ...

We present a method for simultaneous dimension reduction, model fitting and metastability analysis of high dimensional time series. The approach is based on the combination of hidden Markov models (HMMs) with localized principal component analysis (PCA) and fitting of multidimensional stochastic differential equations (SDE). We derive explicit estimators for PCA-SDE model parameters and employ ...

In this paper, we present a scheme of stochastic hybrid system which introduces randomness to the deterministic framework of the traditional hybrid systems by allowing the flow inside each invariant set of the discrete state variables to be governed by stochastic differential equation (SDE) rather than the deterministic ones. The notion of embedded Markov chains is proposed for such systems and...

In this paper we consider multidimensional stochastic differential equations (SDEs) with discontinuous drift and possibly degenerate diffusion coefficient. We prove an existence and uniqueness result for this class of SDEs and we present a numerical method that converges with strong order 1/2. Our result is the first one that shows strong convergence for such a general class of SDEs. The proof ...