XII.—On Feudal and Obligatory Knighthood

Feudal Reinforcement Learning

One way to speed up reinforcement learning is to enable learning to happen simultaneously at multiple resolutions in space and time. This paper shows how to create a Q-learning managerial hierarchy in which high level managers learn how to set tasks to their sub-managers who, in turn, learn how to satisfy them. Sub-managers need not initially understand their managers’ commands. They simply lea...

Feudal Q-learning

One popular way of exorcising the ddmon of dimensionality in dynamic programming is to consider spatial and temporal hierarchies for representing the value functions and policies. This paper develops a hierarchical method for Q-learning which is based on the familiar notion of a recursive feudal serfdom, with managers setting tasks and giving rewards and punishments to their juniors and in thei...

Feudal system in metabolic flux

FeUdal Networks for Hierarchical Reinforcement Learning

We introduce FeUdal Networks (FuNs): a novel architecture for hierarchical reinforcement learning. Our approach is inspired by the feudal reinforcement learning proposal of Dayan and Hinton, and gains power and efficacy by decoupling end-to-end learning across multiple levels – allowing it to utilise different resolutions of time. Our framework employs a Manager module and a Worker module. The ...

Optional and Obligatory Modal Subordination ∗

This paper raises the empirical point that modal subordination is not always obligatory, and that moreover, this is a point of lexical variation. Some modals, like will, which I call definite modals, undergo modal subordination obligatorily, and some, like gonna, which I call nondefinite modals, do so optionally. I propose a dynamic framework in which, following from Frank (1997), information s...