Importance sampling of rare transition events in Markov processes.

نویسندگان

  • Wei Cai
  • Malvin H Kalos
  • Maurice de Koning
  • Vasily V Bulatov
چکیده

We present an importance sampling technique for enhancing the efficiency of sampling rare transition events in Markov processes. Our approach is based on the design of an importance function by which the absolute probability of sampling a successful transition event is significantly enhanced, while preserving the relative probabilities among different successful transition paths. The method features an iterative stochastic algorithm for determining the optimal importance function. Given that the probability of sampling a successful transition event is enhanced by a known amount, transition rates can be readily computed. The method is illustrated in one- and two-dimensional systems.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Estimating Rewards & Rare Events in Nondeterministic Systems

Exhaustive verification can quantify critical behaviour arising from concurrency in nondeterministic models. Rare events typically entail no additional challenge, but complex systems are generally intractable. Recent work on Markov decision processes allows the extremal probabilities of a property to be estimated using Monte Carlo techniques, offering the potential to handle much larger models....

متن کامل

Improving Adaptive Importance Sampling Simulation of Markovian Queueing Models using Non-parametric Smoothing

Previous work on state-dependent adaptive importance sampling techniques for the simulation of rare events in Markovian queueing models used either no smoothing or a parametric smoothing technique, which was known to be non-optimal. In this paper, we introduce the use of kernel smoothing in this context. We derive expressions for the smoothed transition probabilities, compare several variations...

متن کامل

Adaptive Importance Sampling Technique for Markov Chains Using Stochastic Approximation

For a discrete-time finite-state Markov chain, we develop an adaptive importance sampling scheme to estimate the expected total cost before hitting a set of terminal states. This scheme updates the change of measure at every transition using constant or decreasing step-size stochastic approximation. The updates are shown to concentrate asymptotically in a neighborhood of the desired zero varian...

متن کامل

Rare-event Simulation Techniques: An Introduction and Recent Advances

In this chapter we review some of the recent developments for efficient estimation of rareevents, most of which involve application of importance sampling techniques to achieve variance reduction. The zero-variance importance sampling measure is well known and in many cases has a simple representation. Though not implementable, it proves useful in selecting good and implementable importance sam...

متن کامل

Importance Sampling for a Markov Modulated Queuing Network with Customer Impatience until the End of Service

For more than two decades, there has been a growing of interest in fast simulation techniques for estimating probabilities of rare events in queuing networks. Importance sampling is a variance reduction method for simulating rare events. The present paper carries out strict deadlines to the paper by Dupuis et al for a two node tandem network with feedback whose arrival and service rates are mod...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • Physical review. E, Statistical, nonlinear, and soft matter physics

دوره 66 4 Pt 2  شماره 

صفحات  -

تاریخ انتشار 2002