The nucleus accumbens (NAc) plays an important role in both appetitive and consummatory behavior. To examine how NAc neurons encode information during reward consumption, we recorded the firing activity of rat NAc neurons during the performance of a discriminative stimulus task. In this task, the animal must make an operant response to an intermittently presented cue in order to obtain a sucros...

Recent studies have suggested that the basal ganglia are related to motivational control of behavior. To study how motivational signals modulate motor signals in the basal ganglia, we examined activity of midbrain dopamine (DA) neurons and caudate (CD) projection neurons while monkeys were performing a one-direction-rewarded version (1DR) of memory-guided saccade task. The cue stimulus indicate...

Statistical regularities in the natural environment play a central role in adaptive behavior. Among other regularities, reward association is potentially the most prominent factor that influences our daily life. Recent studies have suggested that pre-established reward association yields strong influence on the spatial allocation of attention. Here we show that reward association can also impro...

Numerous applications in the area of computer system analysis can be effectively studied with Markov reward models. These models describe the behavior of the system with a continuous-time Markov chain, where a reward rate is associated with each state. In a reliability/availability model, upstates may have reward rate 1 and down states may have reward rate zero associated with them. In a queuei...

The ability to learn stimulus-reward associations on the basis of reward prediction errors critically depends on the mesolimbic dopaminergic system including the dopaminergic midbrain and the ventral striatum. It is known that healthy elderly and patients with Parkinson's disease are less proficient than healthy young adults in learning stimulus-reward contingencies, but it is unclear whether t...

Recent work has demonstrated that when artificial agents are limited in their ability to achieve their goals, the agent designer can benefit by making the agent’s goals different from the designer’s. This gives rise to the optimization problem of designing the artificial agent’s goals—in the RL framework, designing the agent’s reward function. Existing attempts at solving this optimal reward pr...

Predicting outcomes is a critical ability of humans and animals. The dopamine reward prediction error hypothesis, the driving force behind the recent progress in neural "value-based" decision making, states that dopamine activity encodes the signals for learning in order to predict a reward, that is, the difference between the actual and predicted reward, called the reward prediction error. How...

Eight pigeons learned either matching (to sample) or oddity (from sample) with or without reward for sample responding. The training stimuli were coarse-white, fine-black, or smooth-mauve gravels in pots with buried grain as the reinforcer. Oddity without sample reward was learned most rapidly, followed by matching with sample reward, oddity with sample reward, and matching without sample rewar...

In decision-making under uncertainty, economic studies emphasize the importance of risk in addition to expected reward. Studies in neuroscience focus on expected reward and learning rather than risk. We combined functional imaging with a simple gambling task to vary expected reward and risk simultaneously and in an uncorrelated manner. Drawing on financial decision theory, we modeled expected r...

Hippocampal replays are episodes of sequential place cell activity during sharp-wave ripple oscillations (SWRs). Conflicting hypotheses implicate awake replay in learning from reward and in memory retrieval for decision making. Further, awake replays can be forward, in the same order as experienced, or reverse, in the opposite order. However, while the presence or absence of reward has been rep...