We present an information theoretic approach to stochastic optimal control problems that can be used to derive general sampling based optimization schemes. This new mathematical method is used to develop a sampling based model predictive control algorithm. We apply this information theoretic model predictive control (IT-MPC) scheme to the task of aggressive autonomous driving around a dirt test...

Nonparametric density estimation aims to determine the sparsest model that explains a given set of empirical data and which uses as few assumptions as possible. Many of the currently existing methods do not provide a sparse solution to the problem and rely on asymptotic approximations. In this paper we describe a framework for density estimation which uses information-theoretic measures of mode...

The cross-entropy method is an efficient and general optimization algorithm. However, its applicability in reinforcement learning (RL) seems to be limited because it often converges to suboptimal policies. We apply noise for preventing early convergence of the cross-entropy method, using Tetris, a computer game, for demonstration. The resulting policy outperforms previous RL algorithms by almos...

Some real-world problems are partially decomposable, in that they can be decomposed into a set of coupled subproblems, that are each relatively easy to solve. However, when these sub-problem share some common variables, it is not sufficient to simply solve each sub-problem in isolation. We develop a technology for such problems, and use it to address the challenge of finding the concentrations ...

We apply the Cross-entropy method to the N persons Iterated Prisoners Dilemma and show that cooperation is more readily achieved than with existing methods such as genetic algorithms or reinforcement learning.

This paper studies the performance of cache-enabled dense small cell networks consisting of multi-antenna sub-6 GHz and millimeter-wave base stations. Different from the existing works which only consider a single antenna at each base station, the optimal content placement is unknown when the base stations have multiple antennas. We first derive the successful content delivery probability by ac...

This paper describes a new idea of finding the importance sampling density in rare events simulations: the MinxEnt method (shorthand for minimum cross-entropy). Some preliminary results show that the method might be very promising. 1 The minxent program Assume • X = (X1, . . . ,Xn) is a random vector (with values denoted by x); • h is the joint density function of X; • Sj(·) (j = 1, . . . , k) ...

In recent years, the cross-entropy method has been successfully applied to a wide range of discrete optimization tasks. In this paper we consider the cross-entropy method in the context of continuous optimization. We demonstrate the effectiveness of the cross-entropy method for solving difficult continuous multi-extremal optimization problems, including those with nonlinear constraints.

In this paper we propose a method that learns to play Pac-Man. We define a set of high-level observation and action modules. Actions are temporally extended, and multiple action modules may be in effect concurrently. A decision of the agent is represented as a rule-based policy. For learning, we apply the cross-entropy method, a recent global optimization algorithm. The learned policies reached...

We describe a system that attempts to predict the continuation of a symbolically encoded tabla composition at each time step using a variable-length n-gram model. Using cross-entropy as a measure of model fit, the best model attained an entropy rate of 0.780 in a cross-validation experiment, showing that symbolic tabla compositions can be effectively encoded using such a model. The choice of sm...