We introduce the general and powerful scheme of predicting information re-use in optimization algorithms. This allows us to devise a computationally efficient algorithm for bandit convex optimization with new state-of-the-art guarantees for both Lipschitz loss functions and loss functions with Lipschitz gradients. This is the first algorithm admitting both a polynomial time complexity and a reg...

We consider the problem of undiscounted reinforcement learning in continuous state space. Regret bounds in this setting usually hold under various assumptions on the structure of the reward and transition function. Under the assumption that the rewards and transition probabilities are Lipschitz, for 1-dimensional state space a regret bound of Õ(T 3 4 ) after any T steps has been given by Ortner...

Regret over missed opportunities leads adults to take more risks. Given recent evidence that the ability to experience regret impacts decisions made by 6-year-olds, and pronounced interest in the antecedents to risk taking in adolescence, we investigated the age at which a relationship between missed opportunities and risky decision-making emerges, and whether that relationship changes at diffe...

The experience of regret rests on a counterfactual analysis of events. Previous research indicates that regret emerges at around 6 years of age, marginally later than the age at which children begin to answer counterfactual questions correctly. We hypothesized that the late emergence of regret relative to early counterfactual thinking is a result of the executive demands of simultaneously holdi...

Age differences in the associations among intensity of regret, control attributions, and intrusive thoughts were investigated (N = 122, age range = 20-87 years). Given that the opportunities to overcome regrettable behavior decline with age, older adults' attributions of low internal control were expected to serve self-protective functions and facilitate deactivation of regret. In younger adult...

Decision-related regret is a negative emotion associated with thinking about a past or future choice. The thinking component generally takes the form of a wish that things were otherwise and involves a comparison of what actually did or will take place with some better alternative--a "counterfactual thought." For predecisional (anticipated) regret, the thinking involves a mental simulation of t...

This article improves the existing proven rates of regret decay in optimal policy estimation. We give a margin-free result showing that the regret decay for estimating a within-class optimal policy is second-order for empirical risk minimizers over Donsker classes, with regret decaying at a faster rate than the standard error of an efficient estimator of the value of an optimal policy. We also ...

This research tests the general proposition that people are motivated to reduce future regret under escalation situations. This is supported by the findings that (a) escalation of commitment is stronger when the possibility of future regret about withdrawal is high than when this possibility is low (Studies 1a and 1b) and (b) escalation of commitment increases as the net anticipated regret abou...

Given a multi-armed bandit problem it may be desirable to achieve a smallerthan-usual worst-case regret for some special actions. I show that the price for such unbalanced worst-case regret guarantees is rather high. Specifically, if an algorithm enjoys a worst-case regret of B with respect to some action, then there must exist another action for which the worst-case regret is at least Ω(nK/B),...

We discuss a multiple-play multi-armed bandit (MAB) problem in which several arms are selected at each round. Recently, Thompson sampling (TS), a randomized algorithm with a Bayesian spirit, has attracted much attention for its empirically excellent performance, and it is revealed to have an optimal regret bound in the standard single-play MAB problem. In this paper, we propose the multiple-pla...