TsinghuAeolus is the champion team for the latest two RoboCup simulation league competitions. While our binary and nearly full source code for RoboCup 2001 had been publicly available for the entire year, we won the champion again in Fukuka, with more obvious advantage. This paper describes the key innovations that bring this improvement. They include an advice-taking mechanism which aims to im...

We consider the problem of learning hierarchical policies for Reinforcement Learning able to discover options, an option corresponding to a sub-policy over a set of primitive actions. Different models have been proposed during the last decade that usually rely on a predefined set of options. We specifically address the problem of automatically discovering options in decision processes. We descr...

In this article, we investigate the profitability of remanufacturing option when the manufactured and remanufactured products are segmented to different markets and the production capacity is finite. It is assumed that remanufactured products can be substituted by the manufactured ones. A single period profit model under substitution is constructed to investigate the system conditions under whi...

This paper describes a practical simulation-based algorithm, which we call the Stochastic Grid Bundling Method (SGBM) for pricing multidimensional Bermudan (i.e. discretely exercisable) options. The method generates a direct estimator of the option price, an optimal early-exercise policy as well as a lower bound value for the option price. An advantage of SGBM is that the method can be used for...

I introduce a new way of building initial knowledge into a reinforcement learning agent. The agent will acquire a set of initial options by generalizing from training examples provided by a human supervisor. I show how this can significantly increase convergence speed compared to learning with primitive actions only. Moreover, using this method we can create a single option for dealing with sim...

This paper introduces new variance reduction techniques and computational improvements to Monte Carlo methods for pricing American-style options. For simulation algorithms that compute lower bounds of American option values, we apply martingale control variates and introduce the local policy enhancement, which adopts a local simulation to improve the exercise policy. For duality-based upper bou...