We consider the problem of off-policy policy selection in reinforcement learning: using historical data generated from running one policy to compare two or more policies. We show that approaches based on importance sampling can be unfair—they can select the worse of two policies more often than not. We give two examples where the unfairness of importance sampling could be practically concerning...

One of the main problems of importance sampling in Bayesian networks is representation of the importance function, which should ideally be as close as possible to the posterior joint distribution. Typically, we represent an importance function as a factorization, i.e., product of conditional probability tables (CPTs). Given diagnostic evidence, we do not have explicit forms for the CPTs in the ...

Importance sampling-based algorithms are a popular alternative when Bayesian network models are too large or too complex for exact algorithms. However, importance sampling is sensitive to the quality of the importance function. A bad importance function often leads to much oscillation in the sample weights, and, hence, poor estimation of the posterior probability distribution. To address this p...

Using a multivariable Faa di Bruno formula we give conditions on transformations τ : [0, 1] → X where X is a closed and bounded subset of R such that f ◦ τ is of bounded variation in the sense of Hardy and Krause for all f ∈ C(X ). We give similar conditions for f◦τ to be smooth enough for scrambled net sampling to attain O(n−3/2+ ) accuracy. Some popular symmetric transformations to the simple...

Recently, there has been growing interest in systematic search-based and importance sampling-based lifted inference algorithms for statistical relational models (SRMs). These lifted algorithms achieve significant complexity reductions over their propositional counterparts by using lifting rules that leverage symmetries in the relational representation. One drawback of these algorithms is that t...

This brief paper is an exploratory investigation of how we can apply sensitivity analysis over importance sampling weights in order to obtain sampling estimates of lower previsions described by a parametric family of distributions. We demonstrate our results on the imprecise Dirichlet model, where we can compare with the analytically exact solution. We discuss the computational limitations of t...

Most aircraft fleets nowadays are operating under the concept of damage tolerance, which requires an aircraft to have sufficient residual strength in the presence of damage in one of its principal structural elements (PSE) during the interval of service inspections. The residual strength however is significantly reduced due to multi site damage (MSD). In the present paper, a probabilistic frame...