Problem 1: rewards from a small interval. Consider a version of the problem in which all the realized rewards are in the interval [12 , 1 2 + ] for some ∈ (0, 1 2). Define versions of UCB1 and Successive Elimination attain improved regret bounds (both logarithmic and root-T) that depend on the . Hint: Use a more efficient version of Hoeffding Inequality in the slides from the first lecture. It ...

The nonstochastic multi-armed bandit problem, first studied by Auer, Cesa-Bianchi, Freund, and Schapire in 1995, is a game of repeatedly choosing one decision from a set of decisions (“experts”), under partial observation: In each round t , only the cost of the decision played is observable. A regret minimization algorithm plays this game while achieving sublinear regret relative to each decisi...

Reinforcement learning (RL) is a challenging task, especially in highly competitive multiagent scenarios. We consider the route choice problem, in which self-interested drivers aim at choosing routes that minimise their travel times. Employing RL here is challenging because agents must adapt to each others’ decisions. In this paper, we investigate how agents can overcome such condition by minim...

Previous research found that children first experience regret at 5 years and relief at 7. In two experiments, we explored three possibilities for this lag: (1) relief genuinely develops later than regret; (2) tests of relief have previously been artefactually difficult; or (3) evidence for regret resulted from false positives. In Experiment 1 (N=162 4- to 7-year-olds) children chose one of two ...

This article deals with the rationality and functionality of the existence of regret and its in ̄ uence on decision making. First, regret is de® ned as a negative, cognitively based emotion that we experience when realizing or imagining that our present situation would have been better had we acted differently. Next, it is discussed whether this experience can be considered rational and it is ar...

In this section we show how the refined upper bound on the regret of the EXP algorithm proved using the potential function approach (KL divergence) also gives us a better bound for the expert game setup with bandit feedback. Last lecture we showed how in the case of expert prediction with bandit feedback using the Exp3 algorithm, the regret is upper bounded by T 2/3n1/3 using a rough upper boun...

We study the problem of online learning with a notion of regret defined with respect to a set of strategies. We develop tools for analyzing the minimax rates and for deriving regret-minimization algorithms in this scenario. While the standard methods for minimizing the usual notion of regret fail, through our analysis we demonstrate existence of regret-minimization methods that compete with suc...

Markdown as a pricing modality is ubiquitous in retail whereas everyday-low-price (EDLP) remains relatively rare (despite its several advantages, such as simplicity). Using a stylized model, we explore whether and why retailers can use either of these pricing modalities as an effective defense against a competitor entering the market with the alternative pricing modality to sell an identical pr...

We consider the framework of stochastic multi-armed bandit problems and study the possibilities and limitations of forecasters that perform an on-line exploration of the arms. These forecasters are assessed in terms of their simple regret, a regret notion that captures the fact that exploration is only constrained by the number of available rounds (not necessarily known in advance), in contrast...

IMPORTANCE To our knowledge, longitudinal assessment of depression, anxiety, and decision regret (a sense of disappointment or dissatisfaction in the decision) in patients undergoing prognostication for uveal melanoma does not exist. OBJECTIVE To report on depression, anxiety, and decision regret before and after testing to estimate uveal melanoma prognosis. DESIGN, SETTING, AND PARTICIPANT...