Linear Contextual Bandits with Global Constraints and Objective

We consider the linear contextual bandit problem with global convex constraints and a concaveobjective function. In each round, the outcome of pulling an arm is a vector, that depends linearly onthe context of that arm. The global constraints require the average of these vectors to lie in a certainconvex set. The objective is a concave function of this average vector. This problem turns out to bea common generalization of classic linear contextual bandits (linContextual) [8, 17, 1], bandits withconcave rewards and convex knapsacks (BwCR) [4], and the online stochastic convex programming(OSCP) problem [5]. We present algorithms with near-optimal regret bounds for this problem. Ourbounds compare favorably to results on the unstructured version of the problem [6, 12] where therelation between the contexts and the outcomes could be arbitrary, but the algorithm only competesagainst a fixed set of policies. We combine techniques from the work on linContextual, BwCR andOSCP in a nontrivial manner while also tackling new difficulties that are not present in any of thesespecial cases. Microsoft Research. [email protected] Research. [email protected].