On the optimal response vigor and choice under variable motivational drives

Within a rational framework, a decision-maker selects actions based on the rewardmaximisation principle, i.e., acquiring the highest amount of reward with the lowest cost. Action selection can be divided into two dimensions: (i) selecting an action among several alternatives, and (ii) choosing the response vigor, i.e., how fast the selected action should be executed. Previous works have addressed the computational substrates of such a selection process under the assumption that outcome values are stationary and do not change during the course of a session. This assumption does not hold when the motivational drive of the decision-maker is variable, becuase it leads to changes in the values of the outcomes, e.g., satiety decreases the value of the outcome. Here, we utilize an optimal control framework and derive the optimal choice and response vigor under different experimental conditions. The results imply that, in contrast to previous suggestions, even under conditions that the values of the outcomes are changing during the session, the optimal response rate in an instrumental conditioning experiment is a constant response rate rather than decreasing. Furthermore, we prove that the uncertainty of the decision-maker about the duration of the session explains the commonly observed decrease in response rates within a session. We also show that when the environment consists of multiple outcomes, the model explains probability matching as well as maximisation choice strategies. These results, therefore, provide a quantitative analysis of optimal choice and response vigor under variable motivational drive, and provide predictions for future testing.