It is difficult to learn about systems that contain state variables when those variables are not directly observable. This paper formalizes this learning problem and presents a method called the @rarlve exrension merhod for solving it. In the iterative extension method, the learner gradually constructs a partial theory of the state-containing system. At each stage, the learner applies this part...

The notation is presented in Fig. 1, where the experimental setup is schematically shown. In ineq. (2) the notation is somewhat shortened, so that A = A(φA, λ) and B = B(φB , λ) pertains to the first detector while B′ = B′(φB , λ) and C ′ = C′(φC , λ) pertains to the second, and the expressions E(AB′), E(AC ′), and E(BC ′) are correlations of the results at detector 1 and detector 2 for differe...

We argue that many models for multivariate longitudinal and cross-sectional data analysis have a common ancestry. They all are based on the qualitative idea that if we knew the actual state of the world, the relations between the observed quantities would be truly simple. This is shown to lead directly to factor analysis, IRT, state space models, mixture densities, latent Markov chains, MIMIC, ...

The FF system obtains a heuristic estimate for each state during a forward search by solving a relaxed version of the planning task, where the relaxation is to assume that all delete lists are empty. We show how this relaxation, and FF’s heuristic function, can naturally be extended to planning tasks with constraints and effects on numerical state variables. First results show that the implemen...

When they are available, safe state abstractions improve the efficiency of reinforcement learning algorithms by allowing an agent to ignore irrelevant distinctions between states while still learning an optimal policy. Prior work investigated how to incorporate state abstractions into existing algorithms, but most approaches required the user to provide the abstraction. How to discover this kin...

Unlike delta-hedging or similar methods based on Greeks, global hedging is an approach that optimizes some terminal criterion that depends on the difference between the value of a derivative security and that of its hedging portfolio at maturity or exercise. Global hedging methods in discrete time can be implemented using dynamic programming. They provide optimal strategies at all rebalancing d...

Securing the critical infrastructure power grid is one of the biggest challenges in securing cyberspace. In this environment, control devices are spread across large geographic distances and utilize several mediums for communication. Given the required network topology of the power grid several entry points may exist that can be utilized for compromising a control network. This article explores...

Hierarchical methods have attracted much recent attention as a means for scaling reinforcement learning algorithms to increasingly complex, real-world tasks. These methods provide two important kinds of abstraction that facilitate learning. First, hierarchies organize actions into temporally abstract high-level tasks. Second, they facilitate task dependent state abstractions that allow each hig...