Hilbert Space Embeddings of POMDPs
نویسندگان
چکیده
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes’ rule to these distribution embeddings. Policies and value functions are defined on the feature space over states, which leads to a feature space expression for the Bellman equation. Value iteration may then be used to estimate the optimal value function and associated policy. Experimental results confirm that the correct policy is learned using the feature space representation.
منابع مشابه
Hilbert Space Embeddings of PSRs
Many problems in machine learning and artificial intelligence involve discrete-time partially observable nonlinear dynamical systems. If the observations are discrete, then Hidden Markov Models (HMMs) (Rabiner, 1989) or, in the control setting, Partially Observable Markov Decision Processes (POMDPs) (Sondik, 1971) can be used to represent belief as a discrete distribution over latent states. Pr...
متن کاملHilbert Space Embeddings in Dynamical Systems
In this paper we study Hilbert space embeddings of dynamical systems and embeddings generated via dynamical systems. This is achieved by following the behavioural framework invented by Willems, namely by comparing trajectories of states. As important special cases we recover the diffusion kernels of Kondor and Lafferty, generalised versions of directed graph kernels of Gärtner, novel kernels on...
متن کاملAlmost Bi-lipschitz Embeddings and Almost Homogeneous Sets
This paper is concerned with embeddings of homogeneous spaces into Euclidean spaces. We show that any homogeneous metric space can be embedded into a Hilbert space using an almost bi-Lipschitz mapping (biLipschitz to within logarithmic corrections). The image of this set is no longer homogeneous, but ‘almost homogeneous’. We therefore study the problem of embedding an almost homogeneous subset ...
متن کاملHilbert Space Embeddings of Predictive State Representations
Predictive State Representations (PSRs) are an expressive class of models for controlled stochastic processes. PSRs represent state as a set of predictions of future observable events. Because PSRs are defined entirely in terms of observable data, statistically consistent estimates of PSR parameters can be learned efficiently by manipulating moments of observed training data. Most learning algo...
متن کاملNonparametric Tree Graphical Models via Kernel Embeddings
We introduce a nonparametric representation for graphical model on trees which expresses marginals as Hilbert space embeddings and conditionals as embedding operators. This formulation allows us to define a graphical model solely on the basis of the feature space representation of its variables. Thus, this nonparametric model can be applied to general domains where kernels are defined, handling...
متن کامل